Right-shifted input (or double prompt) after accepting a line in bracketed-paste-mode with readline-8.2

[ok2]

When I use rlwrap with a command, which has a prompt, then I get very strange behaviour, when I type this:

$> rlwrap /bin/ed -p'* '
* q

typing q and then press "Enter key":

$> rlwrap /bin/ed -p'* '
* * q

It changes the line with the prompt to * * q. When I use a different prompt:

$> rlwrap /bin/ed -p'testprompt> '
testprompt> testprompt> q

I mean it works, but this prompt copy is disturbing. When I use with-readline instead rlwrap it does not happen. It also happens just on my Mac OS X 13.0, not on Linux. Apple silicon and Intel-based behave slightly differently but have a similar error. On Intel it looks like that after entering q and pressing enter (instead of prompt, it copies the command):

$> rlwrap /bin/ed -p '* '
* q q

❯ rlwrap /bin/ed -p 'testprompt> '
testprompt> q           q

I have rlwrap 0.46 (from brew or compiled manually makes no difference)

Does anybody have an idea what could be wrong?

[frou]

On macOS 12 (Monterey) with Terminal/iTerm, I still experience the #166 bug ("rlwrap rewrites the input line right-shifted") even though I'm running rlwrap 0.46. So things definitely seem to be dodgy on the macOS front at the moment.

[hanslub42]

What readline version do you use? And does putting

 set enable-bracketed-paste off

... into your .inputrc change anything?

[frou]

I think you're asking the OP, but if you'll excuse me I'll answer anyway:

What readline version do you use?

readline installed as a dependency of rlwrap by brew and it's version 8.2.1

And does putting set enable-bracketed-paste off into your .inputrc change anything?

Yes, that stops the thing I'm seeing from happening. I remember having to apply that workaround in the past for #108, but I removed it after rlwrap 0.44 was released with a fix.

[hanslub42]

Can (one of) you recompile with debugging enabled (autoreconf, ./configure --enable-debug, make)

then

 src/rlwrap --debug /bin/ed -p 'testprompt> '

...type q ENTER, and then attach the resulting /tmp/rlwrap.debug?

[ok2]

Now with debug. How I typed:

$> ./src/rlwrap --debug /bin/ed -p 'testprompt> '
testprompt> q

How it looked like after typing ENTER:

$> ./src/rlwrap --debug /bin/ed -p 'testprompt> '
testprompt>             q

(just a tiny discovery: with debug, it doesn't copy the prompt but puts spaces(?) there)

rlwrap.debug.txt

[hanslub42]

Comparing your debug file with mine, I don't see any (relevant) differences. Still, your terminal gets messed up and mine doesn't

Could you check out and compile branch bug-168-prompt-handling-on-OSX, or just comment out/remove two lines 313-314 in readline.c:

 if (bracketed_paste_enabled)
      my_putstr(term_enable_bracketed_paste);

It might be the case that outputting term_enable_bracketed_paste ("\033[?2004h") at that point confuses some terminals

If my hunch is right, the bug can also be avoided on OSX by using a terminal like rxvt of xterm (if you happen to have an X server lying around somewhere....)

[ok2]

The bug-168-prompt-handling-on-OSX does not change anything here, also commenting out the two lines do not change anything either:

$> ./src/rlwrap --debug /bin/ed -p 'testprompt> '
testprompt>             q

rlwrap.debug.txt
rlwrap.debug.txt

[ok2]

Maybe interesting enough, neither xterm (version 375 on xquartz 2.8.2) nor rxvt changes anything.

rlwrap.debug.txt

[hanslub42]

There is an undocumented debug option that slows down and shows cursor movement.

When I run

 ./src/rlwrap --debug=4096  /bin/ed -p 'testprompt> '

and then press q ENTER

I see:

1estprompt> q
2
testprompt> 3
testprompt> q

I suspect that the function move_cursor_to_start_of_prompt() doesn't do what its name suggests and you will see something like

testprompt> 2

(The relevant code is at readline.c:243)

Can you try this out for me?

[ok2]

Yes, sure, I have recorded a video how it looks like because it is not easy to explain 😄

Screen.Recording.2022-11-05.at.14.43.50.mov

[ok2]

By the way, writing set enable-bracketed-paste off to ~/.inputrc fixes this problem. I am not sure why bracketed paste mode influences the prompt.

[hanslub42]

See #108. It turned out that (only) in bracketed-paste mode, rl_redisplay() would not display the prompt, and the prompt would disappear after entering a line. That is why I added some code to print the prompt before calling rl_redisplay()

However, it now turns out that on some systems the prompt gets reprinted by rl_redisplay() after all, and we end with two prompts (or the prompt and spaces, when called with --debug, which might provide a clue.

I will have to delve into the readline source code to get to the bottom of this.

[hanslub42]

obfusk's comment in #108 maybe provides a clue:

• when bracketed-paste is enabled, rl_deprep_terminal() prints BRACK_PASTE_FINI ("\033[?2004l\r") to rl_outstream;
• this moves the cursor to the beginning of the line (instead of the position after the prompt), which causes the rlwrap prompt to be overwritten during rl_redisplay().

The current problem (especially the fact that rlwrap is outputting exaclty the same bytes in both buggy and non-buggy scenarios) would be explained if BRACK_PASTE_FINI would not move the cursor. on some terminals, or readline doesnt output it, even though it is difficult to imagine that this would happen on OSX xterm and not on linux xterm.

But, as Sherlock Holmes said: When you have eliminated the impossible, whatever remains, however improbable, must be the truth.

I'll try and extend the terminal testing code at the end ot term.c to make it possible to test this.

[hanslub42]

STOP PRESS: It turns out that I can reproduce this, after all, with readline-8.2

Even though I had compiled rlwrap with readline-8.2 it still picked up the system-wide shared library for readline-8.1, which put me on the wrong foot.

This is not OSX-specific after all, but rather dependent on the readline version, so I changed the bug title accordingly.

[hanslub42]

The current problem [...] would be explained if BRACK_PASTE_FINI would not move the cursor. on some terminals, or readline doesnt output it

BRACK_PASTE_FINI ends with "\r" so it will always move the cursor, and readline-8.2 still outputs it, so both explanations are wrong.

What happens is, rather, that readline now realises the cursor is at column 0. From readlines rltty.c:

if (terminal_prepped & TPX_BRACKPASTE)
{
  fprintf (rl_outstream, BRACK_PASTE_FINI);
  /* Since the last character in BRACK_PASTE_FINI is \r */
  _rl_last_c_pos = 0;

Those last two lines are not found in readline-8.1 but were added in 8.2

So now, when rl_redisplay() is called, readline moves the cursor to the right to where it thinks the prompt ends. Of course, as rlwrap has reprinted the prompt after rl_deprep_terminal() has output "\r", the cursor is no longer at position 0 and redisplay places its output strlen(prompt) positions too far right.

[hanslub42]

This should now be fixed in branch bug-168-prompt-handling-on-OSX. Can you confirm?

I essentially reverted the changes I made for #108 as readline-8.2 doesn't need those anymore.
To prevent #108 from resurfacing, rlwrap, when compiled with pre-8.2 readline will then no longer honour set enable-bracketed-paste on

People who want to use rlwrap with older readline and still have bracketed paste work should stay on rlwrap-0.45 or 0.46

[hanslub42]

merged bug-168-prompt-handling-on-OSX and released 0.46.1. Closing

[frou]

Thank you very much Hans. For the record I've submitted 0.46.1 to Homebrew in Homebrew/homebrew-core#114905

[ok2]

It works fine for me, thank you very much! 👍