Syntax proposal: let punning

[stedolan]

OCaml supports "punning": when a record field / named argument / object method has the same name as a variable, it need not be written twice. For instance, instead of:

let x = { foo = foo; bar = fn ~arg:arg () }

one can write the more concise:

let x = { foo; bar = fn ~arg () }

This patch extends punning to work with let bindings, allowing let x as shorthand for let x = x.

This is much more useful that it might first appear.

Anonymous functor parameters

When using Map, Set and similar container types, let compare = compare is a common pattern, which let-punning shortens:

module IntMap = Map.Make (struct type t = int let compare end)

Binding operators

When writing monadic or applicative code using binding operators, the pattern let* foo = foo is common, representing a monadic bind / applicative map of a variable. (Without binding operators, this is usually written as foo >>= (fun foo -> ...) or foo |> map (fun foo -> ...)). Let punning means that this example from the OCaml manual:

open ZipSeq
let sum3 z1 z2 z3 =
  let+ x1 = z1
  and+ x2 = z2
  and+ x3 = z3 in
    x1 + x2 + x3

can become:

open ZipSeq
let sum3 z1 z2 z3 =
  let+ z1 and+ z2 and+ z3 in
  z1 + z2 + z3

(This application was the original motiviation for this feature)

Re-exporting values

Modules which re-export most of their contents from another module can be more concisely written with let-punning. For instance:

module StringList = struct
  type t = string list
  let length = List.length
  let nth = List.nth
  let filter = List.filter
  let mem = List.mem
end

can become:

module StringList = struct
  type t = string list
  open List
  let length and nth and filter and mem
end

[gasche]

Indeed: Olivier Martinot and myself would have a use of "let punning" when manipulating applicative functors with let operators, we use an "unpack" idiom and+ x = x very often. See our ML workshop submission: Quantified Applicatives – API design for type-inference constraints.

The reason why this pattern is so common is that it emulates "box" rules for modalities. The modal typing rule:

x:t1, y:t2 |- e : t
-----------------------------------
x:Box t1, y:Box t2 |- box e : Box t

is expressed in OCaml as

let+ x = x and+ y = y in e

[ivg]

An alternative approach that could be applied for binding applicative/monadic parameters would be extending the pattern syntax to enable monadic unpacking of the function parameters:

let add +x +y = 
  x + y

which unfolds to

let add x y = 
  let+ x = x and+ y = y in
  x + y

This will move back abstraction/application on par with let binding, as the latter became superior to the former after the introduction of the binding operators.

Since the change is on the pattern level it will also affect the let-bindings itself, so that let +x = x in b will be unfolded to let+ x = x in b, as well as work with matches and other places where patterns are used.

I tend to believe that this approach addresses the essence of the problem of tautological bindings and fits more or less naturally into the language semantics, cf. the lazy pattern, e.g., let lazy x = x that is already in the language.

Finally, if we will allow spaces between the monadic operator and the pattern itself, e.g., let + x = x in b this approach will become a natural extension of the already existing binding operators.

[alainfrisch]

Another use case, similar to the third one, for bringing an explicit set of values from another module into scope:

open struct open M let x and y and z end

[gasche]

I'm not sure +p makes sense in general:

1. I don't see what it means in-depth within a pattern; for example how would we desugar let (+x1, +x2) = (v1, v2) in e, without a-priori knowledge of the operator and+? (a-priori knowledge would be against the spirit of the current desugaring rules)
2. I don't see what it means in non-exhaustive scenarios. For example what would be the meaning of match v with | +p1 -> e1 | p2 -> e2, when the second pattern is not itself a plus-pattern? If all patterns have a toplevel +, there is a clear meaning (let+ x = v in match x with p1 -> ...), but I don't see a reasonable extension to heterogeneous situations where some alternatives are plus-patterns and some are not.

[stedolan]

I think the +p idea could probably be made to work despite the issues @gasche points out: we could desugar +p to a mixture of (let+) and (and+), and always treat +p as an exhaustive pattern, giving exhaustivity/redundancy warnings if used in alternatives.

But (a) this is a much larger and trickier feature than let-punning, and (b) even if we had this feature, I'd still want let-punning to avoid having to write let +x = x.

[ivg]

I'm not sure +p makes sense in general:

Indeed, introducing it on the pattern level is probably too ambitious. My original idea, which I prematurely generalized, was to introduce it on the parameter level or even higher, on the fun level, with the following simple rewritting rule,

fun [<bind-kind>] <parameter> -> <expr>
---------------------------------------------------------------------
fun <parameter> -> let<bind-kind> <parameter> = <parameter> in <expr>

This will keep patterns pure and will minimize surprise as we already have something equal with the ?(<label>=<expr>) rewriting happening on the same level.

[stedolan]

I don't think that rewriting would do what you expect, because of OCaml's pervasive currying. The two-argument function let f p q = ... is in fact equivalent to let f = fun p -> fun q -> ..., so the rewriting would convert:

let f +x +y = ...

into

let f x = let* x = x in fun y -> let* y = y in ...

where the first let* happens outside the binding of y, so you end up with a function of type 'a -> ('b -> _ t) t rather than a 'a -> 'b -> _ t. To make this work in the expected way, we'd need to have some sort of uncurried functions, where let f +x +y = ... means something different from let f +x = fun +y -> ....

[psteckler]

In my current project, we have a ppx:

[%%define_locally M.(x,y,z)]

which expands to

let x,y,z = M.(x,y,z)

With the proposed syntax, could you write:

open M
let x,y,z

to achieve the same re-exporting result, instead of having to separate the bindings by and?

[gasche]

No,

include struct
  open M let x and y and z
end

would be the shortest expansion. We could possibly allow

include M.(struct let x and y and z end)

[stedolan]

Allowing , as an alternative to and here doesn't seem worthwhile: it saves a few characters, but opens a can of worms in its interaction with binding operators.

(I do like @gasche's M.(struct ... end) syntax though. I think the M.( stuff ) syntax is great, we should support it in more places. See also #9484)

[samwgoldman]

I see the benefit for let+ (and let lazy), although on my team I think many would consider it bad style to use the same variable name for 'a t and 'a in the first place, making the punning less useful. For example, we might prefer to use the names foo_opt and foo.

I'm also curious about the user experience if someone accidentally writes let x instead of let x = e. The compiler might point them to an error elsewhere in the code, and it might be difficult to see what is wrong. This is purely speculation, but I wonder if there is any way to get a sense for this kind of thing? I've worked with many people who are new to OCaml at my work, and it's already fairly difficult for people to get from compiler error to the necessary change.

[stedolan]

I'm also curious about the user experience if someone accidentally writes let x instead of let x = e. The compiler might point them to an error elsewhere in the code, and it might be difficult to see what is wrong. This is purely speculation, but I wonder if there is any way to get a sense for this kind of thing? I've worked with many people who are new to OCaml at my work, and it's already fairly difficult for people to get from compiler error to the necessary change.

If someone writes let x in ... by forgetting to write the definition of x, then the most likely outcome is that instead of a syntax error reported on in, the compiler reports an unbound value error on x. Neither of these is a particularly accurate description of the problem, but I don't think it makes things any worse.

If there happens to be an x in scope, then the compiler will now accept let x in .... If the programmer had really intended to write = e after x, then the most likely result is a type error on the first use of x. This could of course be far away from the definition, but a type error on a use of x is a good hint to look at the definition of x, so I don't think this will be too confusing.

[stedolan]

The consensus at a recent developer meeting was that the let* x in syntax is useful but the let x in syntax is a bit too weird. So, I've updated the PR to only allow let-punning on let*/and*. (in passing, also fixing a minor bug where the syntax accepted to the right of and* used to differ from that accepted after let*)

[gasche]

The implementation looks fine to me -- and I also like the proposed syntax.

[gasche]

I guess we need to refine this to "binding-operator punning". Could you include an actual example (and its desugaring)?

[ivg]

Also, are you sure that it is let-punning not let-prunning?

[gasche]

The name we use really is "punning". I think of it as a "pun" in that we use something for something else (the name of a declaration as its definition). The naming "record field punning" is well-established and more or less (well, remotely) consistent with type punning for example.

[ivg]

haha, I was always under impression that I was using type pruning)) Good day I have learned something new)

[stedolan]

It was pointed out to me that lots of binding-operator-style code that could benefit from this feature doesn't use the let* x = e in ... syntax, but instead uses the extension-node syntax let%foo x = e in ... (e.g. ppx_let, ppx_lwt). With the removal of support for let-punning on plain let, this usecase breaks.

So, I've just pushed a version of this patch that re-adds the support for let-punning on plain let, but triggers a syntax error if no extension node is present. This means that let x in ... is still a syntax error (exactly the same error as trunk), but let%foo x in ... is accepted and works like let* x in .... (This change is only relevant if ppxes are in use, as all %foo extension nodes are rejected by the default compiler).

[gasche]

I we are going to have this wart in the codebase, let's turn it into a strength that will make ocamlformat people happy: use it also for binding operators, and use this boolean when reprinting the AST.

[stedolan]

I'm not quite sure what you're suggesting here? The type let_binding is not part of the AST, but rather a temporary structure used by parser.mly to share code between various constructs that have similar syntax.

[gasche]

We should avoid these false blemishes in our semantic actions with the following setup:

let_binding_body:
| let_binding_body_with_punning { let (p,e) = $1 in (p, e, true) }
| let_binding_body_no_pun { let (p, e) = $1 in (p, e, false) }

[stedolan]

That is better, I'll change to something along those lines. (I don't think let_binding_body_with_punning is necessary as a separate nonterminal, though, as there's only one case)

[gasche]

The code looks okay and I understand what it does and why it is this way. Approved.

[gasche]

Nitpick: I suspect that if you had used a punned_binding rule for this case, you could have shared it with the letop_binding_body grammar. (The %prec annotation would stay here, I guess?)

[gasche]

Remark: if I understand correctly, we have pretty-printing support for the binding operator form, but not for the extension form. This is just fine as far as I am concerned.

[gasche]

If I'm not mistaken, let punning has not been added to the OCaml manual. This needs to be done before the feature is released. @stedolan, could you take care of this? (I think it would naturally be a new section in the "Language extensions" chapter; I considered extending the "Binding operators", but it doesn't really make sense if it also applies to extension-lets.)

[stedolan]

Done in #10202. I added it to "Binding operators" and cross-referenced from extension-lets.