Handle carriage returns #73

dosisod · 2023-03-01T22:00:47Z

This is probably the best client-side library for rendering ANSI escape codes client-side, but one of the features that would be really nice to have is the ability to support carriage returns, since I am using it to stream terminal output.

When running the following Python code:

for i in range(10):
    print(f"{i}\r", end="")

print()

A single 9 is printed at the end, since the carriage return moves the cursor to the start of the line and each print overrides the last line.

The equivalent code in ansi_up doesn't do the same thing though:

<html>

<head>
<script src="./ansi_up.js"></script>
</head>

<body>
<pre id="console"></pre>

<script>
const ansi_up = new AnsiUp();

for (let i = 0; i < 10; i++) {
  const html = ansi_up.ansi_to_html(`${i}\r`);

  document.getElementById("console").innerHTML += html;
}
</script>
</body>
</html>

When running this you get:

I would imagine this could be implemented by popping the last HTML element, that or adding a new element with a relative vertical position (and making sure to set the background so the last line doesnt bleed through). I wouldn't mind implementing this myself, but I might need a few pointers on where to start.

Thanks!

The text was updated successfully, but these errors were encountered:

drudru · 2023-03-01T22:11:26Z

Hi - yes, I made that decision a while ago. It was easier to implement.
However, after many years, I agree, it would be much better to properly handle this.

I will think about what will be required to handle this.

Thanks for opening this issue.

drudru · 2023-03-16T21:23:56Z

Ok, I have reviewed this issue. This would be a big change.

When I started ansi_up, I tried to primarily address log output that had ansi sequences.
There was a sweet spot that avoided a lot of the complexity of building a terminal emulator. (Fun side note, I have built 2 hobby terminal emulators in my off time)
This is because, like logs, the output is basically append only.

In addition, it mostly side steps the whole buffering issue. One of the big problems that I kept facing during the history of this project was the fact that output streams are byte oriented. I tried to ignore it, but it kept coming back as corner cases.

The big challenge is this. If we have to start changing html that was already output, the complexity goes up a level. It still isn't as complex as building a terminal emulator, but it takes a big step in that direction.

Lets cover some of the possible strategies.

Start returning non-html data. The data would have to indicate that (XYZ) changed. This is complicated because I output data as quickly as it comes in.
If a program does 2 write calls for a single line, and those translate into two calls to ansi_up, then I output 2 chunks of HTML. This means that we have to start synchronizing identifiers for chunks. The API consumer would also have to replace both chunks.

Another strategy is to just have ansi_up take over the rendering. This is a huge shift in responsibility. I suspect many users have as many different UI frameworks. Playing nicely with all of those and providing all the config options would be a lot of work. This would be the 'effectively turn ansi_up into a terminal emulator' option.

Both of these increase the complexity and are a major API change.
Can you think of a way to handle carriage returns?

dosisod · 2023-03-17T20:15:17Z

I agree that having to keep track of XY positions of everything would be hard (and error-prone). What if we only kept track of the lines though? For instance, we could have an <ol> of rendered HTML elements, creating a new <li> element whenever we see a newline character. Then, whenever we see a \r character, we clear out the last <li> element in the list and add all the new data after the \r to the last <li>. If we get a stream of data that doesnt have a newline, we would append to the last <li> element in the list.

I don't think that we have to change the ansi_to_html function at all: instead we would create a wrapper around it that would manage all the newline and carriage return logic. Doing this would allow for backwards compatibility with the existing API, since it would be providing an abstraction on top of the lower-level HTML rendering as opposed to changing the ansi_to_html function itself.

As much as I think this would work, it would start to fall apart if you wanted to handle the backspace (\b) escape code, which is common for implementing spinners and such. Since \b operates on a "character" basis, we would have to keep track of the positions of each of the characters in the line. Alternatively we could just pop the last character of the last element we outputted (assuming it is being managed with something similar to the newline manager I was describing before), which might be good enough. It's a bit off topic, but something to keep in mind when discussing how to go about architecting this.

Ultimately it is up to you to decide which method best suits you and the direction you want to steer the project. I think we can both agree that keeping things simpler is better, but how we achieve that is up for debate.

drudru · 2023-03-21T04:29:18Z

I think this is an interesting direction.
Line handling isn't that complex.
I still feel like it would be an API change, but I don't mind releasing a new major version.
If people want the older version, I could support that as well.
There may be a way to support both.
I will think about this some more.

Thanks for this suggestion.

drudru · 2023-03-31T20:52:51Z

Ok - just a quick update. I pushed out a maintenance release to address other issues.
This is the next item to work on in my queue.

drudru · 2023-06-23T18:35:00Z

Hi - this is a big change right now, and I don't have the bandwidth.
However, I will keep it open as a reminder.

drudru · 2023-07-05T20:11:22Z

some notes on columns (one other thing to consider)

I was looking at some of my old terminal emulator code yesterday.
when handling data, you have to take into account the number of columns.
if the terminal is in line-wrap or auto-wrap, then output goes to the next line.
if the mode is changes, then that is turned off.

typical terminal aware output either avoids the last column (example: a progress bar),
or turns auto-wrap off.
in my experience, few programs rely on line-wrap.

If this feature is implemented, it will not auto wrap.
There won't be a set number of columns.

I just think I will have to mention it up front in the documentation.

drudru · 2023-07-05T20:12:38Z

At this point, the only thing to do is to figure out the protocol between ansi_up and its caller.

drudru · 2023-07-07T23:38:30Z

This will go into the next major release with the module change since both are big changes.
I just need to work on the design doc and then build a prototype.
I will then post the summary of those here.

drudru added this to the 6.0 milestone Jul 8, 2023

drudru modified the milestones: 6.0, 7.0 Jul 31, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle carriage returns #73

Handle carriage returns #73

dosisod commented Mar 1, 2023

drudru commented Mar 1, 2023

drudru commented Mar 16, 2023 •

edited

dosisod commented Mar 17, 2023

drudru commented Mar 21, 2023

drudru commented Mar 31, 2023

drudru commented Jun 23, 2023

drudru commented Jul 5, 2023 •

edited

drudru commented Jul 5, 2023

drudru commented Jul 7, 2023

Handle carriage returns #73

Handle carriage returns #73

Comments

dosisod commented Mar 1, 2023

drudru commented Mar 1, 2023

drudru commented Mar 16, 2023 • edited

dosisod commented Mar 17, 2023

drudru commented Mar 21, 2023

drudru commented Mar 31, 2023

drudru commented Jun 23, 2023

drudru commented Jul 5, 2023 • edited

drudru commented Jul 5, 2023

drudru commented Jul 7, 2023

drudru commented Mar 16, 2023 •

edited

drudru commented Jul 5, 2023 •

edited