Inclusion criteria

[gilch]

Hissp's repository seems to be several projects rolled into one:

• The Sphinx docs, with separate licensing.
  • API docs
  • Tutorials
  • Quick Start
  • Style Guide
  • FAQ
  • The Lissp ReST directive.
  • The Lissp lexer.
• setup.py
  • the README
• The Hissp compiler
• The Lissp reader
  • REPL
  • CLI
  • munger
• The bundled macros
• The testing machinery
  • Sybil parser
  • workflows

Some of these components seem a lot more stable than others, and that's an argument for separating them. The stable parts could get a 1.0 release, without the unstable parts holding them up. I'm obviously not going to break up all of these pieces. Some really do belong together.

Some of the documentation/testing machinery, the Sybil parser, Lissp ReST directive, and the Lissp lexer, at least, would be useful in other Lissp projects. That's an argument for adding them to the hissp package itself, but they have dependencies (Sybil, Sphinx), and I don't want hissp to have any dependencies. I could maybe make those dependencies optional, or I could split them off into their own package, which still seems a bit premature until Lissp stabilizes.

Hissp and Lissp maybe don't need to be together. Readerless mode is usable in its own right. Hebigo is already separate. I feel like I'm close to being able to implement an EDN-based Hissp as well. Lissp turned out to be a bit more complex than I wanted, mostly around the "reader macros", which is an argument for simplifying them, and munging. But currently, the docs and tests for Hissp proper depend on Lissp being there. Separating them seems premature, but it's something I'm considering.

That leaves the bundled macros. Having them implemented in Lissp complicates packaging a bit. Hebigo's macros are in readerless mode, so that's an option, but it's dogfooding Lissp/Hissp at least a little, which I feel is important. That said, this is one of the least stable areas, and one most in danger of bloat. Every addition needs tests and docs, which also have to change with changes. Those are strong arguments for separating them, but Lissp feels kind of crippled without them, so I'm reluctant to give them up. On the other hand, most of the early tutorials get by without them, and macros can really only expand to code that you could write yourself, but some of them seem to really help a lot.

Therefore, the bundled macros should be very minimal, but where do we draw the line? It's been my own case-by-case judgement so far, without too much thought for the whole. I've changed my mind several times. I've removed things and tried different approaches. That's why I want to develop some more objective criteria for what gets included and what doesn't.

[gilch]

Hissp compiles to a very restricted subset of Python. It completely removes statements, and restricts expressions to just basic literal types (not collections or bytes in Lissp), calls, identifier and method attribute access, lambda expressions, tuples of literals, and pickles, which could have been made with the other components.

It is possible to inject arbitrary code, which pretty much works anywhere if it's an expression, and can still work at the top level for statements. This is technically already how it handles identifiers, and how Lissp handles string literals. I've mostly discouraged other uses of injections for Lissp, but no so much for Hebigo, which lacking Lissp's munger, needs them for operators.

Python was not designed for such restrictions. The fact that there are workarounds at all is what makes Hissp possible, but it does make Lissp a lot less pleasant to use. Some macros are necessary to replace what was lost.

Therefore, the tier 1 strongest candidates for inclusion are macro replacements for statement types Python has had for a long time which have no easy expression alternative. These can't even be injected when not at the top level.

The later a statement type was added, the less essential it seems. Python was usable without them before.

The tier 2 next strongest candidates for inclusion are macro replacements for expression types Python has had for a long time, which have no easy functional alternative. These can at least be injected, but it's better if you don't have to. Again, older is stronger.

Beyond that, tier 3 candidates worth considering are macros, special forms, and read syntax core to Clojure, Arc, Scheme, or possibly Common Lisp. Arc and Scheme are kind of minimal, though in different ways. Clojure is probably the mainstream Lisp I know best, and was designed with the benefit of hindsight (as was Arc, really). It's also more functional than the other Lisps, which might make it a better fit for the restricted subset I'm targeting. Forms common to all of these Lisps might be more important.

And finally, tier 4 is anything new that's so useful that it's worth it.

These tiers may overlap to some degree, making an important item in a lower tier more important than an unimportant item in a higher tier. Something easy enough to write yourself might not deserve a macro, but if it's too hard, then a macro would also be hard to implement, especially with the restriction that the bundled macros can have no dependencies on hissp in their expansions.

[gilch]

I want to enumerate the candidates, but this is getting too big for an issue comment. Moving to the wiki: https://github.com/gilch/hissp/wiki/Bundled-macro-candidates

[gilch]

OK, most of the candidates are implemented. I will probably end up dropping a few of these that don't seem worth it, but I expect to keep most of them. The rest need docs/tests. There are a few more advanced threading macros in Clojure that I probably won't be implementing this pass (if ever), although I'm still seriously considering ...

I did get generators (yield) working via a smallish prelude class. That one was giving me trouble for a long time. It took a lot of iterations once I figured out how to do it, but I think I've found a balance I'm happy with. As a part of the micro-prelude that might be at the top of every file, a terse implementation was a high priority, but the result still had to be usable. I hand-minified (golfed?) it down to eight lines, by far the most complex object defined in the prelude, but much shorter than the Pythonic style would have been.

[gilch]

#155 is ready.

[gilch]

Merged.

[gilch]

The missing candidates still up for consideration are

• Arc ssyntax :, !, ~, .. Anarki has more now. A bundled reader macro could expand these. This is getting too complex. I can see this approaching APL-level terseness, like J's hooks and forks. Where does it end? : seems especially useful, but kinda built into Hebigo already, and trailing parens aren't that hard with parinfer. Maybe belongs in an external library. Leaning against it.
• Clojure/Hy ../.. Hebigo uses !of:, which seems a bit less useful in Lissp. I want getattr, getitem, and call somehow. It would be nice if it could do slice notation too. -> can already do it via calls, but it's pretty verbose. Not sure how this should work yet, but it's a loose end.
• as->. Seems much less important than ->/->>. Python's argument ordering isn't always as consistent as Clojure's, but methods (at least) work well with ->. An alternative might be a let1 or let-in that has (value name *body) form so it fits in -> and can adapt functions that don't fit. But X# pretty much works already. Leaning against.
• cond->/cond->>. Not in Python or the other Lisps, but they're more imperative than Clojure. It's hard to call this essential. Leaning against.
• some->/some->>. Similar story. Leaning against.
• case-lambda. Arity matching. This is built into Clojure's defn. One would usually have default arguments in Python instead, but this might work a little better for macros. Given case and specialized use cases, it's hard to call this essential. Leaning against.

I'm also considering a bench or timeit macro that can run a snippet repeatedly. time# doesn't have the resolution to distinguish fast cases.

[gilch]

Adding @#foo as an abbreviation for (operator..attrgetter 'foo).

It's a bit strange that spam.foo in Python is easily expressed the same way in Lissp, and spam-expr().foo() is still easily expressed as (.foo (spam-expr)), but something like spam-expr().foo required awkward workarounds, like (getattr (spam-expr) 'foo).

Now it can be expressed simply as (@#foo (spam-expr)), perhaps similar to Clojure's (.-foo (spam-expr)) syntax.

It also works in ->, so (-> (spam-expr) (@#foo) (@#bar)) works like spam-expr().foo.bar.

[gilch]

BTW, attrgetter names can have internal dots:

#> (-> (object) (@#__class__.__class__.__name__))
>>> # Qz_QzGT_
... # hissp.macros..QzMaybe_.Qz_QzGT_
... __import__('operator').attrgetter(
...   '__class__.__class__.__name__')(
...   object())
'type'

Really leaning against .. now.

[gilch]

Adding get# as an abbreviation for operator..itemgetter (single-arg version only). That makes getitem work in ->.

First (car) is now get#0, which is really nice; implementation is the best name. Don't expect this to work on iterators, but let-from can pull those apart.

Rest (cdr) is now (get#(slice 1 None) foo). Still not nearly as nice as foo[1:]. Python's notation seems unassailable here. Nothing I've come up with beats an inject: .#"foo[1:]". Of course, that's only for simple cases.

[gilch]

Added a tutorial on making a macro for easier slice notation. I don't know that I want to bundle this one. It works best with a helper class. It's small enough to maybe inline, but that seems too heavyweight for how it's likely to be used. Maybe it's no worse than the itemgetter we're already using. __slots__ would probably make it even lighter, even if it does take a bit more code to define. I could certainly process literals at read time, but that's slightly less useful. It's also a string-injection macro. I have mostly avoided those so far, but this case wouldn't be too bad if I only supported literals. There's always slice as a fallback. I'm considering it. Definitely tier 4.

[gilch]

I figured out how to do it without the helper class. Added in #166. New version also added to macro tutorial.

[gilch]

Added decorators and removed deftype@ in #167. There's kind of a conflict between the @# attr getters and the decorator macros, which would most naturally have the same name. The attr getters seem more important, and also share the @ mnemonic with set@ and zap@. So decorators are @@#!foo instead of @#!foo for now. The extra is already enough to make them distinguishable, so I could just overload the name, but this is a bit awkward to document and implement. setat, zapat, at# would give up the symmetry with set! and zap!, but be more similar to get#.

[gilch]

Overloaded @# to be both attr getter and decorator (as @#! ) in #168. Wasn't that hard to implement, but does make the docstring a little strange.

[gilch]

I'm eliminating the attergetter- overload for @#. #171. It saved all of two characters vs. X#X.. If I had thought of that earlier, I probably wouldn't've implemented @# in the first place. Because others might not think of it either, I kept the examples in the docs.