IRCv3 server-time messages delivered out of order are appended to the buffer which breaks IRC as a p2p protocol

[narodnik]

Describe the bug

(Short) History

Originally IRC was meant to be a decentralized protocol where anybody can join. However due to lack of inbuilt spam protection, the network was vulnerable to abuse. Gradually servers began to quarantine themselves with a "Q-line" config which turned IRC into the centralized networks we know today.

Many projects consider IRC an outdated protocol instead migrating to proprietary alternatives like Discord, or their replacements like Matrix. But the author (and many others) believe that IRC is much better. We don't need emojis and bloated web GUIs.

Our Project for an Updated IRC

We are developing a fully anonymous and p2p IRC which has in-built spam protection, offline history replay and user-configurable moderation using modern cryptography. There are no servers and everybody is free to join the network.

For the spam-protection, we use a technique called rate-limit nullifiers. Basically it's a reverse Shamir's secret-sharing scheme, whereby every epoch a user must attach a ZK proof to their message to show construction of a secret share for their key. If a user floods and posts twice within a single epoch, then others can calculate their private key, and slash their stake which makes spamming a network costly.

For offline history replay, we have a data structure we call an event graph. Every time the p2p network receives an event, it links to all the unlinked tips. This creates a dependency graph where if a host sends you an event with no references to your current graph, you can ask for any missing dependency events to link it to your graph. This ensures strong synchronicity whereby nodes in the p2p network all reach eventual consistency.

Lastly user-configurable moderation means allowing users to select which set of moderators are chosen, but this is not too relevant to this discussion.

IRCv3 Timestamps

IRCv3 is a protocol extension which attempts to modernize IRC with missing features. The relevant extension we'd like to discuss is server-time.

The server-time extension allows clients to see the exact time that messages were sent and received. This allows bouncers to replay information with more accurate time tracking.

Issue

The Python script below is sending messages with timestamps that are misordered. In a p2p network, because consistency is eventual (as fits the original IRC design), we should expect messages to arrive out of order.

Steps to reproduce

I've written an example Python script below which demonstrates this extension:

import asyncio

async def send(w, msg):
    print("Server:", msg)
    w.write(f"{msg}\r\n".encode())
    await w.drain()

async def handle_echo(reader, writer):
    addr = writer.get_extra_info('peername')

    paused = False
    while True:
        data = await reader.readline()
        if not data:
            continue
        #print(data)
        assert data[-2:] == b"\r\n"
        message = data[:-2].decode()
        print("Client:", message)

        if message == "CAP LS 302":
            paused = True
            await send(writer, "CAP * LS :server-time")
        if message == "CAP REQ :server-time":
            await send(writer, "CAP * ACK server-time")
        if message == "CAP END":
            await send(writer, ":narodnik!x@x 001 narodnik :welcome")
            await send(writer, ":narodnik!x@x NICK narodnik")
            await send(writer, ":narodnik!x@x JOIN #hello")
            await send(writer, "@time=2020-10-19T16:40:51.620Z :Angel!angel@example.org PRIVMSG #hello :Hellonew")
            await asyncio.sleep(5)
            await send(writer, "@time=2011-10-19T16:40:51.620Z :Angel!angel@example.org PRIVMSG #hello :Hello")

    #print(f"Received {message!r} from {addr!r}")

    #print(f"Send: {message!r}")
    #writer.write(data)
    #await writer.drain()

    print("Close the connection")
    writer.close()
    await writer.wait_closed()

async def main():
    server = await asyncio.start_server(
        handle_echo, '127.0.0.1', 8888)

    addrs = ', '.join(str(sock.getsockname()) for sock in server.sockets)
    print(f'Serving on {addrs}')

    async with server:
        await server.serve_forever()

asyncio.run(main())

In a terminal run python irc-server-time-v3.py, and then in weechat connect with /connect -notls localhost/8888.

Current behavior

When weechat displays the messages in the buffer, it simply appends them to the end of the current buffer.

-- Mon, 19 Oct 2020 (Thu, 14 Mar 2024) --
18:40:51    Angel │ Hellonew
-- Wed, 19 Oct 2011 (Mon, 19 Oct 2020) --
18:40:51    Angel │ Hello
-- Thu, 14 Mar 2024 (Wed, 19 Oct 2011) --

Expected behavior

When weechat displays the messages in the buffer, it should instead insert them in the correct order.

The proposed fix is:

• For the old buffer and logs, display them as-is (no modification).
• When receiving messages with a timestamp, instead of simply appending to the end of the current buffer, insert them into the correct position ordered by timestamps.

Suggested solutions

No response

Additional information

No response

WeeChat version

4.2.1

What OS are you using?

Linux

On which terminal are you running WeeChat?

No response

Which terminal multiplexer are you using?

No response

[flashcode]

Messages are currently always added to the end of messages in buffer by design, even if the date is in past, so I transform this issue to a feature request (no worries, no impact on when it could be implemented).

Changing this behavior can have consequences on existing plugins on scripts, relying on the fact that new lines are always added at the end, so an impact analysis must be done first.

[flashcode]

Could be related to #1279.