4.13.0~alpha1: change of behaviour of [@tailcall] annotation under sequencing operators

[craigfe]

The following program emits a pair of wrong-tailcall-expectation warnings when compiled with ocamlc.4.13.0~alpha:

let rec f a k = k |> (f [@ocaml.tailcall]) a
let rec g a k = (g [@ocaml.tailcall]) a @@ k

; ocamlc --version
4.13.0~alpha1

; ocamlc -g -w @a test.ml
File "test.ml", line 1, characters 21-44:
1 | let rec f a k = k |> (f [@ocaml.tailcall]) a
                         ^^^^^^^^^^^^^^^^^^^^^^^
Error (warning 51 [wrong-tailcall-expectation]): expected tailcall
File "test.ml", line 2, characters 16-39:
2 | let rec g a k = (g [@ocaml.tailcall]) a @@ k
                    ^^^^^^^^^^^^^^^^^^^^^^^
Error (warning 51 [wrong-tailcall-expectation]): expected tailcall

On ocamlc.4.12.1, the program is accepted without warnings instead.

As far as I can tell, this isn't an intended feature of 4.13 – hence this issue – though it may be considered an improvement by some. Neither f nor g is syntactically a tail-call, but the implementation of [@tailcall] is already non-syntactic in other regards (e.g. it observes inlining differences #9510, and may soon account for register spilling #10424). Personally, I think of [@tailcall] as an assertion of a safety property (something like "this call doesn't require additional stack frames") and so preferred the 4.12 behaviour.

Some details:

• From a scan through the changelog, this looks potentially related to Typecheck x|>f as (f x) #10081.
• It seems that ocamlopt is not affected (neither 4.12 nor 4.13 emit a warning).
• As spotted by @yallop, it's possible to get the entire matrix ( ocaml{c,opt}.4.{12,13}) to emit warnings by first aliasing the sequencing operators:

let ( |> ) = ( |> )
let ( @@ ) = ( @@ )

let rec f a k = k |> (f [@ocaml.tailcall]) a
let rec g a k = (g [@ocaml.tailcall]) a @@ k

[lthls]

It's likely an unintended consequence of #10379. I've created PR #10508 to discuss a potential fix.

[lthls]

After a bit more investigating, I found that the change is not caused by #10379 but by #10081.
It's actually a trivial detail: there were several places were successive applications could be collapsed into a single one.
One of them is during translation of normal applications, in Translcore.transl_apply. This one doesn't collapse applications in bytecode when in debug mode, because debugging events are inserted around applications.
One other was in Simplif.simplify_exits (it has been removed in #10379, but at that point it was almost never used anyway). This one took care of also matching debug events, so that even in bytecode and in debug mode, the applications would get collapsed.
And one of the consequences of #10081 is that the %apply primitives are now translated early into normal applications, so the first translation scheme is used, whereas before 4.13 the second scheme triggers.

To sum up:

• let rec g a k = (g [@ocaml.tailcall]) a @@ k gives a warning in 4.13, not in 4.12
• let rec g a k = (g [@ocaml.tailcall]) a k gives no warning in both versions
• let rec g a k = ((g [@ocaml.tailcall]) a) k gives a warning in both versions

I think the 4.13 behaviour is more consistent than the 4.12 one, but it would make even more sense for all three cases to produce the same behaviour.

[Octachron]

The third case only raises a warning when Levent are inserted (by ocamlc -g ), isn't ?

[lthls]

Yes; without debug events, none of the cases raise a warning (both in 4.12 and 4.13), I think.

[Octachron]

It is bit disturbing that some of those warnings only exists while compiling with ocamlc -g. What would be the effect of discarding Levent in all cases (beyond getting right of the warnings)?

[lthls]

If I'm not mistaken, this will remove the ability to stop the debugger after the function was applied to its first argument but before it's applied to the second one. I haven't used ocamldebug that much, so I don't know how much of an issue it is.
There's also some code in bytegen.ml that seems to remove the events that would prevent tailcall optimisations, so maybe it's always better to discard the events in the middle of applications.
I'd welcome the advice of someone more familiar with the bytecode compiler though.

[Octachron]

My summary of the issue is that before 4.13 |> and @@ were accidentally better behaved than standard applications when compiling with ocamlc -g.

It seems fine to align the behavior between application and application operators in 4.13: we can wait for 4.14 for an eventual improvement of this behavior.

[xavierleroy]

If I'm not mistaken, this will remove the ability to stop the debugger after the function was applied to its first argument but before it's applied to the second one

Compiling in -g mode is not supposed to change multiple applications into several successive single applications. An application f 1 2 will always push 1 and 2, then jump to f. If f is defined as let f x y = ..., the next stopping point is after the = sign. If f is defined as let f x = ...; fun y -> ..., there will be a stopping point after the = sign and another after the -> sign. This works well in practice and avoids degrading bytecode performance too much in -g mode.

If we agree that application operators should produce the same bytecode as normal applications, including the multiple-application optimization, this should also be true in -g mode.

[github-actions]

This issue has been open one year with no activity. Consequently, it is being marked with the "stale" label. What this means is that the issue will be automatically closed in 30 days unless more comments are added or the "stale" label is removed. Comments that provide new information on the issue are especially welcome: is it still reproducible? did it appear in other contexts? how critical is it? etc.