Code generation of environment values

[jbouwman]

This is a question about the interaction of code generation and the global environment, in the presence of native .coalton files, and involves some degree of 'thinking aloud' -- I'd appreciate any feedback.

A comment in typechecker/environment.lisp describes how the global environment is handled by the coalton-toplevel form: a list of calls to environment update functions (set-value-type, set-name, set-code, etc.) is prepended to compiler output, so that loading a fasl puts the global environment in the same state as evalling source does. This relies on implementations of the standard make-load-form generic function for any structure that appears as an environment value. For the case of a single integer constant, the source:

(defpackage #:six
  (:use #:coalton))

(cl:in-package #:six)

(named-readtables:in-readtable coalton:coalton)

(coalton-toplevel
  (define i 6))

expands to something like the following, trimmed for legibility:

(let ((env coalton-impl/entry:*global-environment*))
  (setf env
        (coalton-impl/typechecker/environment:set-value-type
         env 'i 'integer))
  (setf env
        (coalton-impl/typechecker/environment:set-name
         env 'i
         '#s(coalton-impl/typechecker/environment:name-entry
             :name i
             :type :value
             :docstring nil
             :location "coalton-toplevel (nil)")))
  (setf env
        (coalton-impl/typechecker/environment:set-code
         env 'i
         '#s(coalton-impl/codegen/ast:node-direct-application
             :type integer
             :rator-type (integer → integer)
             :rator coalton-library/math/num::|instance/num integer-coalton-library/classes:fromint|
             :rands (#s(coalton-impl/codegen/ast:node-literal
                        :type integer
                        :value 6)))))
  (setf coalton-impl/entry:*global-environment*
        env))

(coalton-impl/global-lexical:define-global-lexical i integer)

(setf i
  (coalton-library/math/num::|instance/num integer-coalton-library/classes:fromint|
                                             6))

(setf (documentation 'i 'variable)
                  "i :: integer")

The embedded constant struct values are dumped by the compiler to the output fasl.

A functionally identical Coalton version might look like:

(package six)

(define i 6)

A question then arises: is it necessary or desirable to produce lisp source code that representing environment values? That is, given 'six.coalton', generate 'six.lisp' that looks like:

;; Generated from tests/test.coalton Tue May  7 19:53:19 UTC 2024

(eval-when (:load-toplevel)
  (unless (eq (coalton-impl/settings:coalton-release-p) nil)
    (error "~A was compiled in development mode but loaded in release." nil)))

(defpackage #:six
  (:use #:coalton))

(in-package #:six)

;; set-value-type i

(global-env:set-value-type! 'i
  (... source form that evaluates to an instance of ty-scheme with value 'coalton:integer, like:
    (make-ty-scheme ...))

;; set-name i

(global-env:set-name! 'i
  (... evaluates to coalton-impl/typechecker/environment:name-entry))

;; set-code i

(global-env:set-code! 'i
  (... evaluates to coalton-impl/codegen/ast:node-direct-application))

(declaim (sb-ext:muffle-conditions sb-kernel:redefinition-warning))

(coalton-impl/global-lexical:define-global-lexical six::i integer)

(setf six::i
        (coalton-library/math/num::|INSTANCE/NUM INTEGER-COALTON-LIBRARY/CLASSES:FROMINT|
         6))

(setf (documentation 'six::i 'variable) "I :: INTEGER")

(declaim (sb-ext:unmuffle-conditions sb-kernel:redefinition-warning))

Are there alternatives? make-load-form in this case is bound to implementation-specific structures, in the form of the allocation and init forms returned by make-load-form-saving-slots, and I didn't see a way to adapt those that wasn't sbcl-specific. Adding a new generic ("make-source-form") has proven to work, for the most part, but maybe there is a different approach.

Perhaps expanding the question above: ought it always be possible to produce a functionally identical .lisp file from any given .coalton file?

[stylewarning]

I think the answer is "yes" since (re-)loading FASLs should be enough to allow later/further Coalton code to be type checked.

A problem we were having before was that FASLs would get cached (without the env updates) by ASDF and when new Coalton code was loaded, old cached FASLs would be loaded, but the Coalton compiler wouldn't know anything, and lead to errors.

We should be more rigorous with load form definitions.