Skip to content

Commit

Permalink
Move the CLI into its own crate (#1351)
Browse files Browse the repository at this point in the history 
* Fetch the wasm-bindgen-cli version from Cargo.lock instead of Cargo.toml

* Move the Nickel CLI to a separate crate

* Fix snapshot tests

* Rename the nickel flake output to nickel-lang-cli for consistency

* Move the nickel-lang crate into a subdirectory

* Update documentation for the new crate split

* Tweak snapshot test instructions

* Use project_root instead of ad-hoc environment variables

* Remove the library from nickel-lang-cli for now

* Rename nickel-lang to nickel-lang-lib and nickel-lang-cli to nickel-lang

* Apply suggestions from code review

Co-authored-by: Yann Hamdaoui <yann.hamdaoui@tweag.io>

* Tweak documentation for nickel_lang_utilities::project_root

* Use MacOS 13 github runner because of cachix/install-nix-action#183

---------

Co-authored-by: Yann Hamdaoui <yann.hamdaoui@tweag.io>
  • Loading branch information
@vkleen @yannham
vkleen and yannham committed Jun 16, 2023
1 parent 6f424fd commit 3e03cc7
Show file tree
Hide file tree
Showing 453 changed files with 434 additions and 374 deletions.
5 changes: 3 additions & 2 deletions .github/workflows/continuous-integration.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,14 @@ jobs:
matrix:
os:
- ubuntu-latest
- macos-latest
# MacOS 13 because of https://github.com/cachix/install-nix-action/issues/183
- macos-13
rust_channel:
- stable
include:
- os: ubuntu-latest
system: x86_64-linux
- os: macos-latest
- os: macos-13
system: x86_64-darwin
continue-on-error: true

Expand Down
24 changes: 18 additions & 6 deletions Cargo.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

128 changes: 10 additions & 118 deletions Cargo.toml
View file
Original file line number Diff line number Diff line change
@@ -1,127 +1,19 @@
[package]
name = "nickel-lang"
version = "1.0.0"
authors = ["Nickel team"]
license = "MIT"
readme = "README.md"
description = "Programmable configuration files."
homepage = "https://nickel-lang.org"
repository = "https://github.com/tweag/nickel"
keywords = ["configuration", "language", "nix"]
edition = "2021"

[[bin]]
name = "nickel"
path = "src/bin/nickel.rs"
bench = false

[lib]
bench = false

[features]
default = ["markdown", "repl", "doc"]
markdown = ["termimad"]
repl = ["rustyline", "rustyline-derive", "ansi_term"]
repl-wasm = ["wasm-bindgen", "js-sys", "serde_repr"]
doc = ["comrak"]

[build-dependencies]
lalrpop = "0.19.9"

[dependencies]
lalrpop-util = "0.19.9"
regex = "1"
simple-counter = "0.1.0"
clap = { version = "4.3", features = ["derive"] }
codespan = "0.11"
codespan-reporting = "0.11"
logos = "0.12"
serde = { version = "1.0.154", features = ["derive"] }
serde_json = "1.0.94"
serde_yaml = "0.9.19"
toml = { version = "0.7.2", features = ["parse"] }
void = "1"
sha-1 = "0.10.0"
sha2 = "0.10.6"
md-5 = "0.10.5"
directories = "4.0.1"
unicode-segmentation = "1.10.1"
indoc = "2"

termimad = { version = "0.23.0", optional = true }
ansi_term = { version = "0.12", optional = true }

rustyline = { version = "11.0", optional = true}
rustyline-derive = { version = "0.8.0", optional = true }

# The `wasm-bindgen` version is pinned (`=`) because it must be a version
# available in Nixpkgs.
wasm-bindgen = { version = "=0.2.83", optional = true, features = ["serde-serialize"] }
serde-wasm-bindgen = "0.5.0"
js-sys = { version = "0.3", optional = true }
serde_repr = { version = "0.1", optional = true }
pretty = "0.11.3"

comrak = { version = "0.17.0", optional = true, features = [] }
once_cell = "1.17.1"
typed-arena = "2.0.2"
malachite = {version = "0.3.2", features = ["enable_serde"] }
malachite-q = "0.3.2"
indexmap = {version = "1.9.3", features = ["serde"] }
strip-ansi-escapes = "0.1.1"

[dev-dependencies]
pretty_assertions = "1.3.0"
assert_matches = "1.5.0"
criterion = "0.4"
pprof = { version = "0.11.1", features = ["criterion", "flamegraph"] }
nickel-lang-utilities = {path = "utilities", version = "1.0.0"}
similar = "2.2.1"
test-generator = "0.3.1"
insta = { version = "1.28.0", features = ["filters"] }

[workspace]
members = [
".",
"nickel-lang-lib",
"nickel-lang-cli",
"lsp/nls",
"lsp/lsp-harness",
"utilities",
"nickel-wasm-repl",
"pyckel",
]

# Enable this to use flamegraphs
# [profile.release]
# debug = true

[[bench]]
name = "numeric"
harness = false

[[bench]]
name = "functions"
harness = false

[[bench]]
name = "arrays"
harness = false

# [[bench]]
# name = "records"
# harness = false

[[bench]]
name = "serialization"
harness = false

[[bench]]
name = "mantis"
harness = false

[[bench]]
name = "stdlib"
harness = false

[[bench]]
name = "typecheck-nixpkgs-lib"
harness = false
[workspace.package]
version = "1.0.0"
authors = ["The Nickel Team <nickel-lang@protonmail.com>"]
license = "MIT"
edition = "2021"
keywords = ["configuration", "language", "nix", "nickel"]
repository = "https://github.com/tweag/nickel"
homepage = "https://nickel-lang.org"
95 changes: 35 additions & 60 deletions HACKING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,12 @@ long.

## Content

The Nickel repository consist in 3 crates:
The Nickel repository consist of 4 crates:

- `nickel-lang` (path: `.`). The main crate containing the interpreter as a
library as well as the `nickel` binary.
- `nickel-lang-lsp` (path: `lsp/nls/`). the Nickel Language Server (NLS), an LSP
- `nickel-lang-lib` (path: `nickel-lang-lib`). The main crate containing the interpreter
as a library.
- `nickel-lang` (path: `nickel-lang-cli`). The `nickel` binary.
- `nickel-lang-lsp` (path: `lsp/nls/`). The Nickel Language Server (NLS), an LSP
server for Nickel.
- `nickel-lang-utilities`: (path: `utilities/`). An auxiliary crate regrouping
helpers for tests and benchmarks. Not required to build `nickel` itself.
Expand Down Expand Up @@ -57,10 +58,16 @@ nickel-lang-lsp 0.1.0

### Nickel

To only build the main crate `nickel-lang`, run:
To only build the main crate `nickel-lang-lib`, run:

```console
cargo build -p nickel-lang-lib
```

To build the interpreter CLI, run:

```shell
$ cargo build
$ cargo build -p nickel-lang
$ ./target/debug/nickel --version
nickel-lang 0.1.0
```
Expand All @@ -82,7 +89,7 @@ nickel-lang-lsp 0.1.0
There is a WebAssembly (WASM) version of the REPL, which is used for the online
playground on [nickel-lang.org][nickel-lang.org]. To ease the build, we use the
`nickel-repl` located in `nickel-wasm-repl`, which just wraps and re-export
the `nickel-lang` with the right settings for building to WebAssembly.
the `nickel-lang-lib` with the right settings for building to WebAssembly.

The Nix flake has also an output to do the whole build, but incremental
compilation is not as good as with direct usage of `cargo`.
Expand Down Expand Up @@ -117,71 +124,39 @@ LICENSE package.json nickel_lang_bg.js nickel_lang_bg.wasm [..]
Tests are run via `cargo test`. They are two types of tests:

- Unit tests, located directly in the corresponding module.
- Integration tests, located in the dedicated crate `tests/integration`.
- Integration tests, located in the dedicated crate `nickel-lang-lib/tests/integration`.
- Snapshot tests, located in `nickel-lang-cli/tests/smapshot`.

### Happy-path testing
### Test annotations

Tests are annotated with an expected result in a comment using TOML syntax that
must be located at the very beginning of the file. See the implementation in
`utilities/src/annotated_test.rs` for details. These annotations are also used
to mark examples, in the top-level subdirectory `examples`, with an expected
failure condition if necessary.

Tests for the happy path - i.e., valid Nickel programs which do not raise errors
are generally written in standalone Nickel files in the `tests/integration/pass`
are generally written in standalone Nickel files in the `nickel-lang-lib/tests/integration/pass`
directory. All `.ncl` files in this directory are automatically converted into
Rust integration tests, which run the file and assert that no errors were
raised during evaluation.

Each of these `.ncl` files is structured as an array of `Bool` expressions, which
is ultimately passed to a `check` function defined in
`tests/integration/pass/lib/assert.ncl`. This function applies an `Assert` contract
to each value in the array, which checks that the value it is applied to evaluates
to `true`. The benefit of using a contract for this is that if a test fails we
can simply run the file directly using Nickel, which gives better error messages
than the ones we get by default from `cargo test`.

### Testing failures

Tests which are expected to fail are predominantly written in Rust, because that
makes it possible to additionally check which particular error was raised. For
example, ([`tests/integration/records_fail.rs`](./tests/integration/records_fail.rs)):

```rust
#[test]
fn non_mergeable() {
assert_matches!(
eval("({a=1} & {a=2}).a"),
Err(Error::EvalError(EvalError::MergeIncompatibleArgs(..)))
);
assert_matches!(
eval("({a | default = false} & {a | default = true}).a"),
Err(Error::EvalError(EvalError::MergeIncompatibleArgs(..)))
);
}
```
`nickel-lang-lib/tests/integration/pass/lib/assert.ncl`.
This function applies an `Assert` contract to each value in the array, which
checks that the value it is applied to evaluates to `true`. The benefit of using
a contract for this is that if a test fails we can simply run the file directly
using Nickel, which gives better error messages than the ones we get by default
from `cargo test`.

The exception to the above is the suite of tests which check that the programs in
`examples/` behave as expected. These tests are also auto-generated from the example
programs, but since not every example is expected to pass, these files are annotated
with a comment declaring the expected behaviour. For example
([`examples/simple-contracts/simple-contract-div.ncl`](./examples/simple-contracts/simple-contract-div.ncl)):

```nickel
# test: blame
let Even = fun label value =>
if builtin.is_number value && value % 2 == 0 then
value
else
contract.blame label in
let DivBy3 = fun label value =>
if builtin.is_number value && value % 3 == 0 then
value
else
contract.blame label in
# Will cause an error! 4 is not divisible by 3.
(4 | Even
| DivBy3)
```
Tests which are expected to fail may be written in Rust in `nickel-lang-lib/tests/integration`.
However, simple failure test cases can make use of the test annotation support
and are located in `nickel-lang-lib/tests/integration/fail`.

### Snapshot testing

The project also contains a suite of snapshot tests in the `tests/snapshot`
The project also contains a suite of snapshot tests in the `nickel-lang-cli/tests/snapshot`
directory. Here, `.ncl` files written in the subdirectories of the `input`
directory are run against the last-built Nickel binary, and their output is
compared to the last-known output.
Expand All @@ -190,7 +165,7 @@ Failures of these tests do not necessarily mean that anything is wrong. Rather
it should be seen as an opportunity to review the diffs and either accept
any changes, or fix any issues introduced.

See the [README] in the snapshot testing crate for more detailed guides on
See `README.md` in the snapshot testing crate for more detailed guides on
working with snapshot tests.

## Benchmarking
Expand Down
4 changes: 2 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Please follow the getting started guide for Nickel users on the [nickel-lang
website](https://nickel-lang.org/getting-started). The instructions below are
either reproduced for this document to be self-contained or because
they are aimed toward hacking on the Nickel interpreter itself (e.g. building
the `nickel-lang` crate documentation).
the `nickel-lang-lib` crate documentation).

### Run

Expand All @@ -91,7 +91,7 @@ the `nickel-lang` crate documentation).
`nix profile add github:tweag/nickel`. The `nickel` command is then in your
`$PATH` and is available anywhere.
- Without Nix, you can use `cargo run` after [building](#build), passing
arguments with an extra `--` as in `cargo run -- -f program.ncl`.
arguments with an extra `--` as in `cargo run --bin nickel -- -f program.ncl`.

2. Run your first program:

Expand Down

0 comments on commit 3e03cc7

Please sign in to comment.