composebox_typeahead: Avoid generating broken links. #30071

kuv2707 · 2024-05-13T09:28:54Z

The #**stream>topic** syntax generates broken links for topics containing two backticks or starting/ending with *, because of architectural flaws in the backend markdown processor. So we avoid generating the syntax for such topics and instead generate the normal link syntax in markdown.

Fixes #19873

CZO discussion

Screenshots and Screen Captures

Self-review checklist

Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

Explains differences from previous plans (e.g., issue description).
Highlights technical choices and bugs encountered.
Calls out remaining decisions and concerns.
Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

Each commit is a coherent idea.
Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

Visual appearance of the changes.
Responsiveness and internationalization.
Strings and tooltips.
End-to-end functionality of buttons, interactions and flows.
Corner cases, error conditions, and easily imagined bugs.

kuv2707

I have made a preliminary implementation and highlighted the reasons behind some technical choices.
Once we decide the correct logic to detect faulty topic names, I'll refactor the functions and put them in a sensible place.

kuv2707 · 2024-05-13T09:35:53Z

web/src/composebox_typeahead.js

@@ -995,8 +997,29 @@ export function content_typeahead_selected(item, query, input_element, event) {
            // Stream + topic mention typeahead; close the stream+topic mention syntax
            // with the topic and the final **.  Note that token.length can be 0
            // if we are completing from `**streamname>`.
+            function will_produce_broken_link(topic) {


Currently, I have considered only two cases which produced buggy links:

When there are two or more backticks

When the topic name begins or ends with asterisks.

There can still be topic names which cause broken links. I had earlier tried to parse the syntax text using the frontend markdown parser and see if the link in the parsed html points to the correct stream and topic. But double-backtick topic names gave different urls through frontend and backend markdown processors (in fact, the frontend parser correctly parsed the double-backtick topic name, while the backend processor generated an incorrect url).

This means that checking if the frontend markdown processor parses the syntax correctly gives no guarantee about the correctness of the link in the final message. So I thought the best option would be to check for the individual reported cases which produce broken urls.

I infer from CZO discussions that we will eventually move to a different markdown processor, and this will no longer be an issue then.

web/src/composebox_typeahead.js

alya · 2024-05-14T22:09:44Z

@timabbott Do you want to take a look and provide some feedback here?

timabbott · 2024-05-17T17:54:53Z

web/src/composebox_typeahead.ts

+                    beginning.slice(0, syntax_start_index) +
+                    url_syntax(get_stream_name(beginning), item.topic) +
+                    " ";
+            }


Can you extract this logic to a function and/or add unit tests for it? It's complicated enough that it's very much worth having a test for.

I have extracted the logic and added some unit tests.

kuv2707 · 2024-05-20T08:34:57Z

@timabbott Addressed the suggestion. Please take a look.

timabbott · 2024-05-20T16:40:28Z

web/tests/composebox_typeahead.test.js

+    );
+    assert.equal(
+        ct.stream_topic_link_syntax("#**stream>t", "*asterisk"),
+        "[#stream>*asterisk](#narrow/stream/-1-unknown/topic/*asterisk) ",


Why do all of these have -1 as the stream ID in the URLs? The first couple don't, which makes me think you've got some sort of weird bug in either one test case or the other.

stream ids are only required when we generate the markdown link syntax (since we need to generate the url ourselves).
That is why the first two didn't need them.

web/tests/composebox_typeahead.test.js

kuv2707 · 2024-05-20T17:57:40Z

web/src/composebox_typeahead.ts

@@ -1180,6 +1185,31 @@ export function content_typeahead_selected(
    return beginning + rest;
 }

+function will_produce_broken_stream_topic_link(topic_name: string): boolean {
+    return /(\*+)|(.*`.*`.*)/.test(topic_name);


The links for topic names with double backticks are now generated correctly, but the double backticks cause the enclosed part of the topic name to render as an inline code block.
I don't know what to do in this case since backslash escaping is disabled.

Yeah, I think there's no great answer for that at present. This is #18202.

As long as the link works correctly, though, that's a much less severe issue than #19873 is.

A possible workaround is to use HTML character references, as suggested here: #18202 (comment)
I'd definitely leave that as a separate later commit, though.

gnprice

Thanks @kuv2707! This is an old issue people have repeatedly run into, and I'm happy to see progress being made on it.

gnprice · 2024-06-03T18:56:46Z

web/src/composebox_typeahead.ts

+function will_produce_broken_stream_topic_link(topic_name: string): boolean {
+    return /(\*+)|(.*`.*`.*)/.test(topic_name);


This should handle several other cases too, as discussed at #19873 (comment) :

containing $$ (rather than backticks or asterisks)

containing any of the above in the stream/channel name (rather than the topic)

And checking for two backticks is probably trickier than necessary — can just check if there's any backticks at all. That's conservative in that it'll cover all the same cases and more, and it simplifies thinking about whether there are relevant cases this misses. (In particular if there's one backtick in the stream/channel name and another in the topic, that's probably enough to mess things up.)

Also this regexp can be simplified:

we're not using the capture groups, so remove the ( and )

the leading and trailing .* are redundant

if it matches \*+, it'll match just \*, and vice versa

Thanks! I've simplified the regex and handled the $$ case.
(I noticed that a single $ doesn't actually cause problems in any case)

Additionally, stream names are also being checked now.

gnprice · 2024-06-03T19:07:25Z

web/src/composebox_typeahead.ts

@@ -1180,6 +1185,31 @@ export function content_typeahead_selected(
    return beginning + rest;
 }

+function will_produce_broken_stream_topic_link(topic_name: string): boolean {
+    return /(\*+)|(.*`.*`.*)/.test(topic_name);


Yeah, I think there's no great answer for that at present. This is #18202.

As long as the link works correctly, though, that's a much less severe issue than #19873 is.

A possible workaround is to use HTML character references, as suggested here: #18202 (comment)
I'd definitely leave that as a separate later commit, though.

gnprice · 2024-06-03T19:14:35Z

web/src/composebox_typeahead.ts

+    const stream = stream_data.get_sub(stream_name);
+    const stream_id = stream?.stream_id ?? -1;
+    return `[#${stream_name}>${topic_name}](${hash_util.by_stream_topic_url(stream_id, topic_name)}) `;


Where does -1 come from here?

I believe that will end up producing a link that's broken, and doesn't even have the stream name the user typed. In general, when we don't have a meaningful value to supply somewhere, it's best not to make up a bogus value like -1 — instead, either do something that explicitly means "there's no known value", or if that's not possible then throw an exception.

(And if throwing an exception feels like it'd be a real problem for users, then that's a sign we need to do some more work so as to actually handle the case where we don't have a real value: either get the value after all, or gracefully handle saying "there's no known value", or something else.)

Since this code path is only accessed through typeahead, it is improbable that stream_id would ever be -1. I think we can throw an exception.

Yes, an exception sounds correct to me.

The #**stream>topic** syntax generates broken links for topics containing two backticks or ending with *, because of architectural flaws in the backend markdown processor. So we avoid generating the syntax for such topics and instead generate the normal link syntax in markdown. Fixes zulip#19873

kuv2707 · 2024-06-04T17:44:59Z

@alya Does this PR have to go through the buddy and mentor review stages? (Since it's already reviewed by maintainers once)

alya · 2024-06-05T18:46:18Z

I think it's OK to skip those at this stage. Has all the feedback above been addressed?

zulipbot added size: S area: markdown (mentions) Mentions for users/groups/stream/topic/time. bug rust community request Issues of interest to the Rust community labels May 13, 2024

kuv2707 commented May 13, 2024

View reviewed changes

kuv2707 mentioned this pull request May 14, 2024

Compose Box: Implement formatting of channels and topics on paste #29302

Open

12 tasks

alya added the integration review Added by maintainers when a PR may be ready for integration. label May 14, 2024

kuv2707 force-pushed the 19873_topic_name_fix branch from 07cab3f to dbcdfa0 Compare May 17, 2024 10:16

timabbott reviewed May 17, 2024

View reviewed changes

kuv2707 force-pushed the 19873_topic_name_fix branch from dbcdfa0 to 0a026be Compare May 18, 2024 16:11

zulipbot added size: L and removed size: S labels May 18, 2024

kuv2707 force-pushed the 19873_topic_name_fix branch from 0a026be to ff0ba14 Compare May 20, 2024 08:07

kuv2707 requested a review from timabbott May 20, 2024 08:33

kuv2707 force-pushed the 19873_topic_name_fix branch from ff0ba14 to 7455824 Compare May 20, 2024 08:55

timabbott reviewed May 20, 2024

View reviewed changes

web/tests/composebox_typeahead.test.js Outdated Show resolved Hide resolved

kuv2707 commented May 20, 2024

View reviewed changes

kuv2707 force-pushed the 19873_topic_name_fix branch 2 times, most recently from a3c5613 to 8c243b1 Compare May 21, 2024 12:16

kuv2707 requested a review from timabbott May 21, 2024 15:38

gnprice reviewed Jun 3, 2024

View reviewed changes

kuv2707 force-pushed the 19873_topic_name_fix branch from 8c243b1 to 7704b04 Compare June 4, 2024 10:24

kuv2707 force-pushed the 19873_topic_name_fix branch from 7704b04 to d15f3fd Compare June 4, 2024 12:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

composebox_typeahead: Avoid generating broken links. #30071

composebox_typeahead: Avoid generating broken links. #30071

kuv2707 commented May 13, 2024 •

edited

kuv2707 left a comment

kuv2707 May 13, 2024 •

edited

alya commented May 14, 2024

timabbott May 17, 2024

kuv2707 May 18, 2024 •

edited

kuv2707 commented May 20, 2024

timabbott May 20, 2024

kuv2707 May 20, 2024

kuv2707 May 20, 2024

gnprice Jun 3, 2024

gnprice left a comment

gnprice Jun 3, 2024

gnprice Jun 3, 2024

kuv2707 Jun 4, 2024

gnprice Jun 3, 2024

gnprice Jun 3, 2024

kuv2707 Jun 4, 2024 •

edited

timabbott Jun 4, 2024

kuv2707 commented Jun 4, 2024

alya commented Jun 5, 2024

		function will_produce_broken_stream_topic_link(topic_name: string): boolean {
		return /(\+)\|(.`.`.)/.test(topic_name);

composebox_typeahead: Avoid generating broken links. #30071

Are you sure you want to change the base?

composebox_typeahead: Avoid generating broken links. #30071

Conversation

kuv2707 commented May 13, 2024 • edited

kuv2707 left a comment

Choose a reason for hiding this comment

kuv2707 May 13, 2024 • edited

Choose a reason for hiding this comment

alya commented May 14, 2024

Choose a reason for hiding this comment

kuv2707 May 18, 2024 • edited

Choose a reason for hiding this comment

kuv2707 commented May 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gnprice left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kuv2707 Jun 4, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kuv2707 commented Jun 4, 2024

alya commented Jun 5, 2024

kuv2707 commented May 13, 2024 •

edited

kuv2707 May 13, 2024 •

edited

kuv2707 May 18, 2024 •

edited

kuv2707 Jun 4, 2024 •

edited