feat(lookup): exposing vector lookup logic through wasm

[dhable]

chore: bring cargo version inline with repo tags

There were a number of manual tag/releases created in our fork repo to release changes that could be picked up in the vector fork. With Jenkins, we now need to reset the version in the Cargo.toml files to match the tag set so things don't get weird.

Ref: LOG-16869

---

feat(lookup): exposing vector lookup logic through wasm

Expose two simple methods - one for parsing VRL paths into segments and one for joining segments into a VRL path. This accounts for for fields, indexes and coalesce. The code will also handle quoting and minimizing quoting on field to be more human friendly.

These two functions could be used to perform richer validation on user input and simplify some of the more complex path construction in the pipeline service and/or front end UI.

Ref: LOG-17092, LOG-16869

[dhable]

Ugh, I had to make a version tag v0.3.0 and then reset the version in Cargo.toml files to that because cargo doesn't think a 4 component version number is a valid semver (and it's not according to the RFC). Just more semver rage for Dan.

[c-nixon]

Mostly nits, one perf note and a few questions

[c-nixon]

Feels a bit like this would be better as a method on Segment, or as a From<Segment> for JsValue impl somewhere since we're already touching the segment file.

[dhable]

Addressed at https://github.com/answerbook/vrl/pull/12/files#diff-a35fa3fd1a9b06a65e5a0b37ce02e015a90db55d305c79308f2adbd5b8a0a834R39-R69.

I didn't want to leak JsValue into the rest of the code base so instead of implementing the method directly on Segment, I made an implementation of From on JsValue.

[penick]

Is the goal to use this in the pipeline-service, etc.? If that the case then this should not live in this repo. It should likely be its own crate that uses the vrl crate.

[penick]

I wonder if it makes sense to only set requires_quoting when it contains invalid chars a construction time:

diff --git a/lib/lookup/src/lookup_buf/segmentbuf.rs b/lib/lookup/src/lookup_buf/segmentbuf.rs
index fc803fb36..c2abcb713 100644
--- a/lib/lookup/src/lookup_buf/segmentbuf.rs
+++ b/lib/lookup/src/lookup_buf/segmentbuf.rs
@@ -38,8 +38,9 @@ impl From<String> for FieldBuf {
             // So we have to take a slice and clone it.
             let len = name.len();
             name = name[1..len - 1].to_string();
-            requires_quoting = true;
-        } else if !field::is_valid_fieldname(&name) {
+        }
+
+        if !field::is_valid_fieldname(&name) {
             requires_quoting = true
         }

Note: Same change required in FieldBuf.

[dhable]

I had started down this route and ended up running into a number of test cases that asserted that parsing and formatting resulted in the same string. For example, parsing top."middle".child and then formatting that segment would result in top.middle.child. Both are semantically the same but we then lose the ability to recreate the input from a parsed representation. From what I can tell of the path syntax, this is fine but the rippling number of changes to test code was concerning and why I opt-ed for localized implementation.

We could add an additional boolean into the struct to denote if the original field was quoted, along with the flag if quoting is required. Then the default formatting could retain the existing behavior while we add a new method to display with reduced quoting. I decided against this to avoid changing things alongside upstream too much. It's worth considering.

[dhable]

Storing a boolean if the original was quoted or not does allow the default to_string() method to maintain the existing behavior while adding a new to_humanized_string() method that does the reduced quoting.

[c-nixon]

All makes sense to me!