Syntax for word-wrapping messages (format=flowed?)

[andersk-auto]

(See also BarnOwl:ticket:82.)

We should officially define some mechanism for automatically word-wrapping text in zephyrs. The basic observation is that while most text is prose can be reflowed, some text is preformatted (code, ASCII art, lists, etc.) and should not be automatically reflowed. It'd be nice to handle the former better without breaking the latter.

The existing standard here is from [http://www.ietf.org/rfc/rfc3676.txt RFC 3676], an extension to email to allow sane rewrapping of text -- primarily quoted text, but it helps unquoted text too. The gist is that the Content-Type is set to text/plain; format=flowed, and wrappable lines have a space at the end, i.e., space-LF is treated as a soft line break that can turn into a space, and signal that the current paragraph is rewrappable.

The format=flowed spec also implies that wrapped paragraphs can be unconditionally displayed in a proportional font, which I think most non-terminal clients (probably including the reference windowgram client) would appreciate being able to do wherever possible.

I suggest that we define some flag in messages for indicating that format=flowed is in effect, so that compliant receivers can follow the rules specified in that RFC for rendering it. I don't think we need to customize those rules, and we can demand that clients be fully conformant to the RFC, including space-stuffing and quote-handling and everything else. (We can probably hard-code DelSp to on; this effectively involves using two spaces at the end of lines.)

On the sending side, some clients, like barnowl, will allow reflowing long lines of text within the editor window (either automatically or with a command). These clients can therefore distinguish lines to be marked as "flowed" (with trailing spaces) and lines to be marked as "fixed". Other clients, like the reference zwrite client, don't have this mode of user interaction for rewrapping. Absent adding a richer UI to zwrite, perhaps the best option is to add a command-line flag, perhaps with a user-settable default, that says that the message is to be rewrapped according to the format=flowed rules.

It is at least somewhat possible that we can get away without a flag, and somehow sniff whether format=flowed interpretation is appropriate. Since the spec flowed paragraphs to consist of at least one flowed line followed by a fixed line, there's no question of what to do with a single long line (it is fixed, and should not be rewrapped). We'd need to define our own ad-hoc variant of format=flowed; I think it would suffice to skip the space-stuffing rules and require DelSp=yes and the spec would hold, and that there would be no issues with misparsing text that isn't intended to be flowed, but I'd have to think harder to be confident.

[andersk-auto]

Imported from trac issue 117. Created by geofft@1TS.ORG on 2013-08-10T22:11:10, last modified: 2013-08-30T06:44:07

[andersk-auto]

Trac comment by andersk@ATHENA.MIT.EDU on 2013-08-10 23:28:28:

Firstly, logging into this Trac is really hard (#98, #118, #119), so I’m not sure it’s the best place to have a productive discussion with all the relevant stakeholders (!BarnOwl, Roost, Zulip, !ZephyrPlus). Maybe we should reawaken [https://mailman.mit.edu/mailman/listinfo/zephyr zephyr@mit.edu]?

It is at least somewhat possible that we can get away without a flag, …

I guess this implies that senders need to add a soft linebreak to single-line paragraphs even if it would not otherwise be required?

[andersk-auto]

Trac comment by geofft@1TS.ORG on 2013-08-11 00:00:44:

Replying to [comment:1 andersk@…]:

I guess this implies that senders need to add a soft linebreak to single-line paragraphs even if it would not otherwise be required?

It is in fact otherwise required, at least by the RFC, as I mentioned (briefly) — paragraphs consist of one or more lines ending with a space, followed by a line not ending with a space. A single line not ending with a space, and not preceded by lines ending with a space, is "fixed" and not subject to rewrapping or displaying in a proportional font. See section 4.1 of the RFC.

(I'm not sure how MUAs, in practice, avoid displaying <78-character paragraphs in monospace. Maybe they just display all text/plain content, either flowed or fixed, monospace.)

Besides, sending parties SHOULD soft-wrap outgoing text at 78 characters.

We could define our own variant of the spec that wraps single-line paragraphs, or something, although now we're defining our own variant of the spec. The existing spec has the nice feature that e.g. copied-and-pasted code is at no risk of getting rewrapped.

We could also punt for Markdown (#120), where text is wrapped and proportional by default and there are mechanisms to insert hard breaks and monospace text, and say that we're just not going to bother rendering Scribe-formatted zephyrs as anything other than unwrapped monospace text.

[andersk-auto]

Trac comment by andersk@ATHENA.MIT.EDU on 2013-08-30 06:44:07:

geofft and I independently came up with a way to support this within the existing Scribe-based markup: using empty groups @() to mark soft line breaks. So maybe that’s not entirely crazy.