Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bundle Julia with a better terminal on Windows? #7267

Closed
StefanKarpinski opened this issue Jun 14, 2014 · 217 comments
Closed

bundle Julia with a better terminal on Windows? #7267

StefanKarpinski opened this issue Jun 14, 2014 · 217 comments
Labels
system:windows Affects only Windows

Comments

@StefanKarpinski
Copy link
Member

StefanKarpinski commented Jun 14, 2014

This seems like the nicest console emulator around on Windows:

http://bliker.github.io/cmder/
http://cmder.net
https://mintty.github.io

I think it would make sense, since Windows is not really a DIY platform, but more of a download-and-use-the-thing-that-you-can-click-on platform, that we include a really nice console emulator for people to use.

EDIT: fixed link
EDIT (@ViralBShah): mintty as the preferred choice

@lindahua
Copy link
Contributor

+1

@tkelman
Copy link
Contributor

tkelman commented Jun 15, 2014

That looks like a custom configuration wrapping up conemu, clink, and (optionally) msysgit. Clink is mostly readline, which we don't need any more. mintty might be an easier/smaller alternative to conemu.

@StefanKarpinski
Copy link
Member Author

Yes, I think that's the deal. Since I don't use Windows myself, I'm somewhat uninformed about these tools, but I'd be more interested in making sure the experience is as nice as possible than minimizing the dependency size. Mintty just doesn't seem to look or feel as nice as cmder.

@tkelman
Copy link
Contributor

tkelman commented Jun 15, 2014

I'm quite happy with mintty as used by Cygwin and MSYS2 (though there's still a bug making Julia not very usable inside mintty, I should check whether that's true in conemu), I don't see too much need for console tabs. There's some discussion of these and other alternatives in #6795

Any bugs that happen with whatever we choose would be harder to properly report or fix if we go with something that is a fork-of-fork-of-fork (or wrapper of wrapper of wrapper). Babun is another similar one of these projects that looks shiny, but is really just a custom configured version of Cygwin.

@quinnj
Copy link
Member

quinnj commented Jun 16, 2014

Here's a crazy mastermind plan. Let's build a repl and/or IJulia notebook frontend for LightTable and ship with that!
But seriously, the more I've been poking around with LightTable, the more I can envision a proper "Julia IDE" being all baked in to a nice Julia plugin. It has all the extensibility to allow something like RStudio or MatLab IDE, but even better because you can do inline plots, or something notebook-ish where the program execution is "graph-based" instead of repl line-by-line. I think Mike's GSoC is a great start in getting the fundamentals in place, but there's a ton of potential there.

Anyway, I digress. I think mintty wounds like a good option here.

@tkelman
Copy link
Contributor

tkelman commented Jun 25, 2014

I went through and tried a few of these options. cmder indeed looks decent.

With Console2 I was able to manually set it up to boot Julia (adding a Julia "tab" under the Settings - there's an xml config file so this could be automated) then running it as Console.exe -t Julia. Console2 didn't fix unicode display issues though. The default text select behavior is pretty awful, you hold left shift then click and drag. Holding a key to select can be omg-carpal-tunnel uncomfortable on a laptop.

I couldn't get cmder to start up Julia directly, but starting it manually worked okay, and displayed simple unicode properly. Cmder depends on the MSVC runtime dll's, was there a license concern in the particular terms under which you can redistribute those? The default selection behavior here was also left-shift (or right-clicking the icon, going to edit and switching to mark mode, same as in the default Windows console), but I found how to change it to not require simultaneous key+mouse.

ConEmu has some really annoying default behaviors, wanting to save its configuration in the registry by default, popping up a confirmation dialog every time I close it, etc. Was easy to start it directly in Julia, and simple unicode worked.

Mintty will take a little bit of work, but be the lightest and easiest option for users in the end. Some of the current issues with mintty are related to bigger libuv problems that will hopefully get fixed together. There's some code somewhere that tests specifically for Cygwin and MSYS2 named pipes, which I think would need to be checked and modified to work with the standalone download of mintty. We could maybe use MSYS2's version of mintty, but then we'd need msys-2.0.dll which is about 4x bigger than the msys-1.0.dll that we currently include as part of msysgit.

None of these options look to compile particularly easily with MinGW, and I wouldn't want to try cross-compiling. But just downloading the release binary should be good enough for these.

@StefanKarpinski
Copy link
Member Author

@ihnorton, over here you said:

+1 for mintty for now. Based on my own experiences, I would be strongly opposed to packaging conemu.

can you elaborate on why over here?

@ihnorton
Copy link
Member

ihnorton commented Jul 5, 2014

@StefanKarpinski basically because every time I have tried it, too many things are just off. I don't remember all of the issues, but very clearly remember the worst one: it does bad things with subprocesses and makes spawning vastly slower than it already is on Windows.

noop.bat: @echo off

 julia> @time for i in 1:1000
       open(`noop.bat`,"r")
       end

time in seconds, two trials each:

    default console: 1.30,1.31
    cmder: 53.0, 61.6 (actually restarted twice more and got similar times)

In fact, my mouse starts freezing when running this test code under cmder - but there is no such problem under default Julia console.

@ihnorton
Copy link
Member

ihnorton commented Jul 5, 2014

See here for part of the reason why: https://code.google.com/p/conemu-maximus5/wiki/ConEmuHk
and from the FAQ:

Q. mingw/bash/git works slow in ConEmu.
A. In rare cases on some configurations users may notice lags while executing commands in ConEmu tab. This may be when many child processes are created while command execution. I you are so unlucky - just uncheck "Inject ConEmuHk" on Features page. I'm still looking for the cause of the problem.

This "feature" can be disabled, but bad spawn performance has been a known issue for several years. It is concerning that such a broken feature is enabled by default (and as I said, this is only the worst issue).

@tkelman:

Cmder depends on the MSVC runtime dll's, was there a license concern in the particular terms under which you can redistribute those?

Right, they can't be redistributed with GPL software. Now, technically, we wouldn't be linking it from Julia. I don't know if that changes anything.

@jhasse
Copy link
Contributor

jhasse commented Jul 9, 2014

+1 for mintty. I'm using MSYS2 + Julia anyway and it works fine :)

@tkelman
Copy link
Contributor

tkelman commented Jul 15, 2014

Using mintty from MSYS2 essentially works now, as long as you also include stty.exe, msys-2.0.dll, msys-intl-8.dll, and msys-iconv-2.dll. And you need to explicitly set HOME=%HOMEDRIVE%%HOMEPATH% before starting, otherwise mintty tries to set HOME to wherever it's located.

I also tried with MSYS1's version of mintty since we already have msys-1.0.dll with msysgit, but that didn't quite work. It seems in MSYS1 mintty, uv_pipe_getsockname(pipe, NULL, &len) in jl_uv.c returns 0.

@ihnorton
Copy link
Member

mintty seems like the least-bad option. Just beware that support for shelling out to things using the Windows console API may be spotty. Pick your poison I guess, but this seems ok to me, in exchange for better "everything else".

(Have you tried something like git diff - does the pager work?)

@tkelman
Copy link
Contributor

tkelman commented Jul 15, 2014

No, does not look like the pager works, but that might just be because we're using a version of less from msysgit, which doesn't work with mintty properly. Inside a full MSYS2 or Cygwin installation, less works fine from Julia (calling that environment's version of less). Tried just copying over MSYS2 less, and the dll's it depends on (ncurses and, hah, pcre), but that seemed to stack dump for some reason...

@pao
Copy link
Member

pao commented Jul 16, 2014

@tkelman I believe I've had success with the version of less from gnuwin32, though it's not at all up to date.

@tkelman
Copy link
Contributor

tkelman commented Jul 16, 2014

Will give it a try. Might not be that hard to just write a pager in pure Julia, would it?

@vtjnash vtjnash closed this as completed Dec 21, 2014
@vtjnash
Copy link
Member

vtjnash commented Dec 21, 2014

i think lighttable/sublime/ijulia integration is going to be the way to go here. but that's a bigger question of how to bundle packages effectively (perhaps 8745 will help with that)

@tkelman
Copy link
Contributor

tkelman commented Dec 21, 2014

I'm still intending to work on mintty for the purposes of the basic REPL. Juno is immature and not to my personal liking yet, IJulia depends on Python in an unsatisfying way. Improving the usability of the basic REPL is still something I plan on doing. That work is blocked on libgit2 however, since we'll need to make fairly substantial changes to the windows packaging when we remove command-line git.

@tkelman
Copy link
Contributor

tkelman commented Dec 27, 2014

I just discovered the Powershell ISE. It's actually really nice, as long as you're using version 3.0 or newer. Julia doesn't start inside it at the moment, I get

julia.exe : ERROR: `convert` has no method matching convert(::Type{TTY}, ::Pipe)
At line:1 char:49

but that might be fixable. If so, maybe we should add a shortcut to start Julia inside it.

@tkelman
Copy link
Contributor

tkelman commented Dec 27, 2014

Looks like similar to mintty, the Powershell ISE uses pipes for stdout and stderr, except stdin is a normal tty. The pipes for stdout and stderr do not have names as far as I can tell, both the uv_pipe_getsockname calls in jl_ispty return 0. If I skip the first early return and make jl_ispty return 1 when len == 0 after the second call to uv_pipe_getsockname, then Julia can start but it's completely unresponsive. And it doesn't seem to recognize color codes either. Judging by http://blogs.msdn.com/b/powershell/archive/2009/02/04/console-application-non-support-in-the-ise.aspx it might not be possible, but oh well, was worth trying.

@ncnc
Copy link
Contributor

ncnc commented Jan 1, 2015

FWIW, I tried a fork of Console2 called ConsoleZ. Changing the defaults I get a reasonably sane copy-paste behavior (select with left mouse button), but similar to Console2 the unicode issues remain.

@StefanKarpinski
Copy link
Member Author

We still need to ship with a better terminal emulator on Windows. Watching people struggle with cmd.exe is awful – Julia needs to provide a better default experience out of the box.

@IainNZ
Copy link
Member

IainNZ commented Feb 6, 2015

I agree, if the details can be hammered out. Although IJulia is certainly smoothing over this, at least in Julia classes I've been involved in over the last month.

(A higher priority would be anything to speed up Pkg operations, which seem extra slow on Windows.)

@tkelman
Copy link
Contributor

tkelman commented Feb 6, 2015

As I said in December #7267 (comment), I will work on incorporating mintty. But not until we get rid of command-line git, since this will all require pretty significant rewriting of how the Windows binary builds get put together. Can we get moving on the libgit2 transition?

@StefanKarpinski
Copy link
Member Author

Right, thanks @tkelman – I mainly wanted to reopen the issue since it's still unresolved.

@davidanthoff
Copy link
Contributor

I like the plan with mintty, but I think it would be good if this was not the default if Julia is installed on a Win10 system. Win10 comes with a major overhaul over the console that probably solves all of the current issues, and I feel Julia should just adhere to the (then) good system default. Current Win10 RTM target seems to be June/summer, so certainly before Julia 0.5, and that kind of means that 0.4 will be the version that people will use on Win10.

@tkelman
Copy link
Contributor

tkelman commented Feb 12, 2015

@davidanthoff yeah I have been meaning to try out the finally-not-crappy console in Win10 (and still confused about them skipping 9). The approach I think I'll take here is get mintty built and included in the binaries, and install a separate shortcut for starting Julia inside it. I don't want to do anything too complicated with detecting Windows versions when 7 and 8 are going to be around and what most Windows users will stay on for some time. Judging by traffic on the Cygwin mailing list, they're keeping up with changes in the prereleases and making sure mintty and other bits and pieces will continue to work.

@davidanthoff
Copy link
Contributor

@tkelman That would certainly work. On the other hand, I like sensible defaults and am not a huge fan if new users have to make choices they don't understand when they first want to start julia. The vast majority of users will have no clue what mintty refers to, so when they see two shortcuts to start julia, and one is called something like JuliaMintty they will have no clue what that they should do. Maybe the installer could detect the version of windows, and then create an appropriate default shortcut? On the other hand, I agree, this shouldn't become a super complicated project, so if there is no easy way to achieve this, your suggestion might still be the best one.

@jebej
Copy link
Contributor

jebej commented Jan 18, 2020

For me ctrl-c does not kill the terminal, and ctrl-d works properly.
cd does not work and makes mintty/julia crash.

It seems that preferences also cannot be saved to disk, with mintty complaining about a non-existent .minttyrc file, even though I created it manually. Maybe this is a general filesystem issue.

FYI the mintty version is 3.0.2, which is not the latest one.

@jebej
Copy link
Contributor

jebej commented Jan 18, 2020

In addition, I do not get any issues when I use the version shipped with MSYS2, so these are probably compilation troubles.

@ViralBShah
Copy link
Member

I am guessing the ctrl-c is going to mintty instead of to Julia. Perhaps there is a way to change these defaults. I had to use a mintty that @vtjnash provided me. The latest binaries did not seem to work for me.

@oif2003
Copy link

oif2003 commented Jan 30, 2020

I noticed the mintty binary you provided leaves an extra blank "mintty.exe" window open when starting julia via the command line with "mintty %julia path%\julia.exe". If I send ctrl+c to that blank window, it interrupts code running in julia without closing out mintty.exe. This is not the behavior I get on mintty 2.9.5. At least as a crappy work around we can easily hide the extra window, intercept ctrl+c while julia REPL is the active window and redirect it to the hidden mintty.exe window.

@oif2003
Copy link

oif2003 commented Jan 30, 2020

One catch though, ctrl+c sent while julia is not executing code will result in it exiting.

EDIT:
It appears that the byte at memory location 0x1004AA3C8 stores julia's execution state. 1 = ready, 0 = busy. This is for 64bit julia.exe.

EDIT2: The address is for Mintty.exe that is hosting julia.exe.
I have made two scripts based on my previous post. One using Mintty 3.0.2, which crashes on cd(). The other using Mintty 2.9.5. The latter does not crash on cd(), and instead of sending keystrokes, it sends SIGINT directly to julia.exe via Windows API.

@aminya
Copy link

aminya commented Feb 9, 2020

I found this new cross-platform and customizable Terminal recently. It supports

It supports Linux, Windows, macOS, and even a web browser!
Watch a demo here:

U++ Terminal

@davidanthoff
Copy link
Contributor

Windows Terminal 1.0 shipped today.

@ViralBShah
Copy link
Member

Windows Terminal is the thing to use. However, the issue is that it seems to require a much more recent version of Windows than what we support. Is it possible to ship Windows Terminal (no idea how easy it is) and fallback to the cmd if it doesn't start?

Does winget solve all these problems for us somehow?

@StefanKarpinski
Copy link
Member Author

Can we just close this issue already? We’re clearly never going to do anything about it.

@aminya
Copy link

aminya commented May 19, 2020

Windows Terminal is not a terminal itself. It is a terminal manager and it uses other terminals. If we want to use a Microsoft Terminal, we should use PowerShell Core.
image

What I suggested here is a real terminal with mouse and image support.
#7267 (comment)

@musm
Copy link
Contributor

musm commented May 19, 2020

Can we just close this issue already? We’re clearly never going to do anything about it.

100% Please do so. Nothing productive has come out of this discussion.

@aminya
Copy link

aminya commented May 19, 2020

Can we just close this issue already? We’re clearly never going to do anything about it.

100% Please do so. Nothing productive has come out of this discussion.

This issue is not solved yet. There is an unsubscribe button for opting out of the notifications.

@PetrKryslUCSD
Copy link

Most (all?) of the issues with the inadequate access to the operating system facilities through the shell mode can be solved by running the editor from within Git bash. For instance be VS code, Atom, Sublime Text. All would start Julia in a capable terminal emulator, with the expected Linux commands working in the shell mode. In my opinion, adding support for the Windows Terminal will add nothing but complexity.

@StefanKarpinski
Copy link
Member Author

StefanKarpinski commented May 19, 2020

There is an unsubscribe button for opting out of the notifications.

Good point. This is officially the first Julia issue I’ve ever unsubscribed from. Someone send me a hologram when this issue gets resolved closed without doing anything in 2026 👋🏼

@ViralBShah
Copy link
Member

I did install Git-Bash and simply use that.

@musm
Copy link
Contributor

musm commented May 19, 2020

Windows Terminal is not a terminal itself. It is a terminal manager and it uses other terminals. If we want to use a Microsoft Terminal, we should use PowerShell Core.
image

What I suggested here is a real terminal with mouse and image support.
#7267 (comment)

This is not true. PowerShell Core, or now simply PowerShell (as of version 7), is a shell, just like cmd, pwsh, and bash are. On the other hand, Windows Terminal is a console or terminal, just like the myriad other terminals that are available, including MinTTY, ConEmu etc.

@aminya
Copy link

aminya commented May 20, 2020

I am referring to the GUI of the terminal/console/shell. I am not concerned with the name. Julia already has a REPL with its rules, and using it from CMD is similar to using it from PowerShell. The good reason to use something new is to add some GUI features.

Windows Terminal does not add much GUI features, and it needs a shell (like CMD or PowerShell) to be called from inside it.

@stevengj
Copy link
Member

stevengj commented May 20, 2020

Then can we stop worrying about Unicode rendering problems for people using the Windows Console?

For example, can we merge #33821 and tell Windows users who don't like seeing sparse matrices displayed as ?????? to download a better terminal?

Or do we continue to limit our display capabilities to the low bar set by the Windows console for the next 5-10 years, until the Windows Terminal is the default on all supported Windows systems? (We currently support the 11-year old Windows 7. How long will it be until we drop support for all Windows systems that don't ship with Windows Terminal?)

@musm
Copy link
Contributor

musm commented May 20, 2020

For example, can we merge #33821 and tell Windows users who don't like seeing sparse matrices displayed as ?????? to download a better terminal?

IMO, some may disagree, but I think this is a perfectly reasonable approach, since it allows us to innovate. We can add Windows 7, to "supported", but not all functionality in terms of rendering fonts may be available by default without installing a better terminal, to the platform list.

While Windows 10 remains fully supported and we add a link the new Windows Terminal. For example, that PR renders perfectly with the new Terminal, since it also includes a decent font with ligatures.

@ViralBShah
Copy link
Member

I think we should merge and move on. Given the ample number of options, we should not consider this our problem. I would also consider that a reasonable way to close this issue - in that we won't ship a terminal (we don't ship one on any OS).

@stevengj
Copy link
Member

stevengj commented May 20, 2020

Okay.

(There have always been better terminals available for Windows, so I expect that the release of the Windows Terminal won't really resolve anything for several years, until it becomes the default — it won't stop the endless stream of complaints about Unicode rendering in Windows. Most Windows users still won't know how run Julia in a better terminal. But if we officially give up on out-of-the-box experience and say "get a better terminal," I guess that is one way to answer those questions.)

@ViralBShah
Copy link
Member

ViralBShah commented May 20, 2020

Yes there have been better terminals available, but it somehow feels to me that it is much simpler today than even a year ago to install them.

To avoid the complaints - is there a way to test if a "good" terminal is being used so that a warning can be issued?

@ViralBShah
Copy link
Member

A lot of windows users also use Juno/vs-code nowadays which probably do ok with Unicode display.

@ViralBShah
Copy link
Member

Just like the other OSes, we will now expect users to "bring your own terminal". Once Windows Terminal is more widespread, hopefully we can detect it and start Julia using that, etc.

Documentation is updated about Terminals, and we have also updated the Downloads page to require Windows 10.

@davidanthoff
Copy link
Contributor

Just a FYI on the latest developments, see here. TLDR: MS is slowly starting to make the new terminal the default for everything on Windows 11 now. I assume that once that has finished rolling out, the Julia start menu shortcut will just automatically start things in the new terminal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
system:windows Affects only Windows
Projects
None yet
Development

No branches or pull requests