add support for terminal multiplexers

[srlehn]

This adds preliminary support for terminal multiplexers (tmux, screen).
The wrapped escape code passes through the terminal multiplexer so that the terminal can respond.

Escape codes meant for the terminal have to be piped through muxwrap. Positional escape codes probably should be directed to the multiplexer.

You can test the function in a sixel capable terminal with and without tmux/screen like this (after defining the muxwrap function in the terminal by copy pasting):

printf '\033Pq#0;2;0;0;0#1;2;100;100;0#2;2;0;100;0#1~~@@vv@@~~@@~~$#2??}}GG}}??}}??-#1!14@\033\\' | muxwrap

The functionality of lsix is unchanged and you have to figure out where to wrap the escape codes.

I needed this for gizak/termui#233 - thought this could help here too.

The escape codes have to be wrapped multiple times for nested tmux/screen sessions.

[hackerb9]

I like that this is short and doesn't add much complexity to lsix.

Where does muxwrap get called? Just pipe the whole script through it?

Does the \o escape require GNU sed?

Why use 'sed b' instead of the more typical 'cat'?

I expect this won't slow things down much on a multiprocessor in a pipeline, but how much overhead does this add for folks on a single processor (e.g., Raspberry Pi Zero)?

Also, won't this overwrite the tmux status line and cause tmux to be confused about the screen state?

[hackerb9]

Also, what is "preliminary" about this? What is missing from your patch other than the call to muxwrap?

[srlehn]

It still needs figuring out when to make calls to muxwrap (for which escape codes) - This is why I called it preliminary. Piping the whole script through doesn't work.

There is no call to muxwrap yet - I provide here only the functionality for passing escape codes through to the terminal. I thought it would be best if you check for which escape codes it is needed.
\033[14t and such should not be wrapped - tmux will then tell the correct size of just the current pane not the whole terminal window. In my go code I don't have any problems with the status line in tmux - not sure what you mean. The image is correctly positioned and scaled to not go out of a specified character cell (not pixel) area. I think this should be possible in Shell too with the exception of the TIOCGWINSZ ioctl call. I didn't test screen thoroughly yet.

I used sed b instead of cat just to not introduce another dependency even if small. Is there another way to just pipe the data through without calling an external program?

I do consider support for tmux/screen on servers a nice feature for a script that is especially useful on servers without a graphical environment.

\o is probably gnu sed but it is possible to instead use bash's $'...\033...' to convert the octal code to the actual char - this would need a few backslashes at the right places.

[hackerb9]

I agree that tmux and screen support would be useful. I don't want to add preliminary code, though. Does your code work if the "montage" command in lsix is piped into muxwrap?

[srlehn]

I have enabled it now. The mlterm checks had to be changed because the TERM variable is set to screen or screen.* inside tmux / screen.

• mlterm+tmux/screen and xterm+tmux works xterm+screen doesn't.
• tmux sets the prompt over the image
• scrolling messes the status bar up. I now understand what you meant. For the termui I didn't notice such problems because of the non-scrolling nature of a TUI. Not sure how to solve this best. Perhaps pagination or a tmux command for temporarily disabling the status bar?

[srlehn]

I tried this for placing the prompt in tmux but it doesn't work:

    # set correct line for terminal multiplexers
    if [ "${TERM%%.*}" == screen ]; then
      IFS='[;' read -a REPLY -s -t "${timeout}" -d 'R' -p "$(muxwrap <<<$'\033[6n')" >&2
      printf '\e[%s;0H' "${REPLY[1]}"
    fi

It was placed in the main function after creating the images.

[hackerb9]

For mlterm detection, I probably should just talk to the developers and ask them to fix their interpretation of SIXEL scrolling. That worked for MinTTY.

For the situation with overwriting tmux, I think the problem is that we're looking at this the wrong way. Instead of lsix (and other sixel programs) using muxwrap to try to kludge VT graphics support into tmux, tmux ought to be extended so that it understands sixel. Using libsixel should make this less painful than it sounds. Tmux would need to be able to do the following:

1. Detect if the underlying terminal supports sixel.
2. Respond to DA1 requests stating that sixel is supported.
3. Virtually render requested sixel sequences. Images may be larger than buffer can show or placed in a location where they are chopped off.
   Note: Images always start at a specific character row & column — just like ordinary text — using ANSI escape sequences tmux already understands, so placement should be easy.
4. a. Use sixel to send only the parts of images which are visible.
   b. If the underlying terminal does not support sixel, fall back to libcaca to send ASCII art. (Just kidding.)

[ghost]

For the situation with overwriting tmux, I think the problem is that we're looking at this the wrong way. Instead of lsix (and other sixel programs) using muxwrap to try to kludge VT graphics support into tmux, tmux ought to be extended so that it understands sixel. Using libsixel should make this less painful than it sounds. Tmux would need to be able to do the following: {...}

My terminal library does these things, and lsix as-is works fine within multiple windows:

It was a bit of work though: parse images, break up into cells, put it all back together in end, and then for me re-encode to a new common sixel palette.

[hackerb9]

Wow! Great work. Did you handle different sized fonts?

[ghost]

The overall interface supports a single font size, but can pick between several different fonts at that size to render the glyphs: CJK, emoji, terminus (default), and then a catch-all fallback. Double-width/height are rendered using these fonts to images, and then those images are emitted as either sixel or straight bitmap; CJK and emoji can be either emitted as real UTF-8 or as rendered glyphs for the xterm backend. I don't have support for the application side to requests multiple fonts from the terminal. And for the Swing backend, the size can be changed on-the-fly via the tools menu -> Screen options dialog.