Win32 Port of Julia

[ivanmantova]

Is there an on-going effort for porting Julia to win32?

Has anyone been working on it, or would be willing to help on such an effort?

[StefanKarpinski]

Unfortunately, there isn't such an effort ongoing. However, we would be thrilled for there to be one and would help any way we could. I suspect a mingw or cygwin port wouldn't be too hard, whereas a truly native Win32 port would be more of a challenge but doable. However, I haven't programed for Windows since 1995, so I'm hardly any sort of authority.

[Keno]

I think the biggest issue here are the dependencies. Having a quick look, LLVM has both MinGW and native Win32. However, looking over the rest of the dependencies, is not as promising. A port using MinGW should be possible however.

[StefanKarpinski]

That would be helped by modularization as well — one ought to be able to run a feature-restricted version of Julia as a native Win32 app, just without the dependencies that don't work natively on Win32. One of many reasons we need better modularity instead of loading all the julia libraries en masse like we do now.

[JeffBezanson]

Well, you can just comment out the libraries you don't want :)

[yocontra]

+1 to this. I think releasing a feature-restricted version is a bad choice though. Wouldn't a better way be working with the developers of the dependencies to make everything run on windows? You can always go the node route too and rewrite all of the libraries to be cross-compat.

[ViralBShah]

I am reasonably sure that most of the libraries will build. We should certainly get an effort going to see how far we can get. For the couple of libraries where we run into problems, we can work with the upstream to get it going. Most libraries are straight C/C++/Fortran.

If someone can get it going, I will also chime in with whatever I can.

[ViralBShah]

The library loading is all in sysimg.j and start_image.j. All non-essential libraries could be commented out at first to get the base julia working.

I agree with @contra that we should release a win32 version when all libraries are supported, rather than releasing a feature-restricted version.

[JeffBezanson]

The repl depends on the ability to select() on both sockets and stdin, and our shell framework uses fork(). We will have to include some kind of portability layer. 👎

[Keno]

@ViralBShah Yes, all the libraries build with MinGW32 (Just tried it). However, the makefiles don't work out of the box and I ended up using cmake where possible. Would you be opposed to adding a CMakeLists.txt for the dependencies (this would also make it easier to do native win32 builds if possible)? CMake also has some nice feature regarding BLAS/LaPACK detection, so maybe we could utilize that.

[ViralBShah]

I personally do not have opposition to cmake and CMakeLists.txt. How about we try it out for the Windows build and see how everybody likes it? In general, we have tried to just use tools that are available out of the box to minimize the build dependencies.

[StefanKarpinski]

I'm not sure that expecting a UNIX pipeline library to work on Windows is a reasonable thing. Under a UNIX emulation system, perhaps, but otherwise, that just seems kind of crazy. What's it supposed to do? Maybe we just need a different library altogether. I have no idea how one builds pipelines in Windows. And that thing was a beast to build in UNIX, which I know well.

[JeffBezanson]

True, it would be reasonable to make the process library a unix-only thing. But it's so damn useful that we use it a lot internally, especially in the multiprocessing code. SSH would be another issue. I guess we'd do only single-threaded first, and deal with that stuff later.

[jckarter]

Most of these issues can be overcome. Windows can start shell processes and has pipes just like Unix (though no analogue to fork of course), and the functionality of select can be had with completion ports and overlapping IO. Although console handles do not directly support completion ports on Windows, it's easy enough to slap an intermediary pipe over the console and do overlapped IO on the pipe.

[jckarter]

Could node's libuv be used for IO instead of hand-rolling nonblocking IO?

[JeffBezanson]

Wow libuv looks very promising.

[ViralBShah]

This does look very interesting.

-viral

On 20-Feb-2012, at 12:33 PM, JeffBezanson wrote:

Wow libuv looks very promising.

---

Reply to this email directly or view it on GitHub:
#399 (comment)

[StefanKarpinski]

Libuv does look awesome. Might as well have synergy with Node. Hip, young languages need to stick together.

[grandinj]

LibreOffice hacker here: LibreOffice builds well enough on windows using cygwin to run the various unix commands, including gmake.

As far as lower-level stuff goes, I think the Git guys have the right idea: They have a slightly-higher-than-posix abstraction layer for dealing with Windows vs. Linux differences.

For example, they have a start_command API for dealing for spawn/fork/exec stuff.

Windows also has a pretty decent non-blocking API for network and disk IO, which is not hard to program against (it was designed by the same guy who designed the grandaddy of todays asynchronous APIs, the VMS IO subsystem)

Also, the Visual Studio compiler is pretty decent (LibreOffice builds with it) and the Express Edition is free to use.

[yocontra]

I've had experience with libuv and can verify that it is pretty awesome. Using libuv to abstract out the IO should make this process a lot easier.

[StefanKarpinski]

Cool. We looked at APR and whatever Mozilla uses a long time ago, but decided they were a little too heavy. This might be the goldilocks amount of abstraction.

[Keno]

Ok, I have started doing some investigation into porting Julia to Windows and there's one problem that is immediately apparent: wchar_t is UTF32 on Linux, but UTF16 on windows. With your knowledge of the entire codebase, any ideas on how this is to be resolved?

[Keno]

I solved this by using uint32_t istead of wchar_t with wcwidth from http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c. The build is passing but unittest.lsp is failing with "eval: variable 2.0 has no value"

[StefanKarpinski]

I don't really understand how that's even possible — there aren't enough 16-bit values for Unicode code points.

[StefanKarpinski]

Also, I'm pretty sure this isn't a matter of UTF-32 versus UTF-16 since it's the type of a single character value, which is not encoded.

[Keno]

It's a UTF16 variable length storage, which really was the problem, because functions such as wcwidth would need at least two UTF16 wchar_t's, which would be a pain to implement and maintain. For now I just went with the above solution. Fortunately it isn't used that extensively within flisp/julia.

[Keno]

Well, on Unix it is, on Windows not so much.

-----Original Message-----
From: Stefan Karpinski [mailto:reply+i-3286100-
2585a0a03ab782be5bcc1ab95fcce4a04d16686e-1291671@reply.github.com]
Sent: Samstag, 25. Februar 2012 19:41
To: Keno Fischer
Subject: Re: [julia] Win32 Port of Julia (#399)

Also, I'm pretty sure this isn't a matter of UTF-32 versus UTF-16 since it's the
type of a single character value, which is not encoded.

---

Reply to this email directly or view it on GitHub:
#399 (comment)

[JeffBezanson]

The "variable 2.0" problem happens when strtod says "2.0" is not a valid number, which can only be due to locale settings. Does your copy of src/support/libsupportinit.c have the following line:

setlocale(LC_NUMERIC, "C");

If not, update and try again. If so, then windows is not respecting this locale setting, or we need to try a different one.

[Keno]

Ok that fixed that. However, now I'm getting "assert-failed (not (= 0.0 0.0))"

-----Original Message-----
From: JeffBezanson [mailto:reply+i-3286100-
2585a0a03ab782be5bcc1ab95fcce4a04d16686e-1291671@reply.github.com]
Sent: Samstag, 25. Februar 2012 19:56
To: Keno Fischer
Subject: Re: [julia] Win32 Port of Julia (#399)

The "variable 2.0" problem happens when strtod says "2.0" is not a valid
number, which can only be due to locale settings. Does your copy of
src/support/libsupportinit.c have the following line:

setlocale(LC_NUMERIC, "C");

If not, update and try again. If so, then windows is not respecting this locale
setting, or we need to try a different one.

---

Reply to this email directly or view it on GitHub:
#399 (comment)

[ViralBShah]

Closing as this is a dup of #521

[Keno]

Closed!

[ViralBShah]

Epic!