How to pass a Document to CST.visit

[gee4vee]

Following the docs, I have this code snippet which is showing an error in CST.visit. It doesn't like doc as an argument to visit. Is this the correct way to get a CST Document to use for visit?

  const fileContents = fs.readFileSync(filePath).toString();
  const tokens = new Parser().parse(fileContents);
  const doc = new Composer().compose(tokens);
  CST.visit(doc, (item, path) => {
    if (!CST.isCollection(item.value)) return;
    const scalar = CST.resolveAsScalar(item.value);
    if (scalar?.value === 'bar') {
      const parent = CST.visit.parentCollection(doc, path);
      const idx = path[path.length - 1][1];
      const { indent } = item.value;
      parent.items.splice(idx, 0, {
        start: item.start.slice(),
        value: CST.createScalarToken('bing', { end: [], indent })
      });
      return idx + 2;
    }
  });

Error:

Argument of type 'Generator<Parsed<ParsedNode>, void, unknown>' is not assignable to parameter of type 'Document | CollectionItem'.

[eemeli]

That would be because compose() returns a generator of parsed AST nodes, and you're looking for a single CST document. You probably want to iterate on the parse() results until you find one.

I would recommend reviewing the relevant docs section: https://eemeli.org/yaml/#parsing-yaml. If some part of that is not clear, please do say so, and hopefully we can improve the documentation. It might also be a good idea to play around with the Node.js interactive shell to see what the data looks like at each stage?

[eemeli]

In case you didn't notice it, this is the first line of the example that you posted:

const [doc] = new Parser().parse('[ foo, bar, baz ]')

Note the left-hand side of that assignment; the [...] there is destructuring the assigned value, a generator, and taking doc as its first emitted value.

Fundamentally, the base entity in YAML is not a document, but a stream which may include directives, comments and documents. This is represented by the Parser providing a lazily-evaluated generator of such entities.

[gee4vee]

I had tried that approach and that gives me a similar error:

      const { file } = argv;
      console.log(`Parsing Blueprint ${file}...`);
      const fileContents = fs.readFileSync(file).toString();
      const [doc] = new Parser().parse(fileContents);
      CST.visit(doc, (item, path) => {
        if (!CST.isCollection(item.value)) return;
        // console.log(`visiting item: ${JSON.stringify(item.value?.items[0])}`);

        // if (item.source? === 'workitems') {
        //   console.log(`found workitems: ${JSON.stringify(item)}`);
        // }
      });

Error:

Argument of type 'Token' is not assignable to parameter of type 'Document | CollectionItem'.
  Type 'SourceToken' is not assignable to type 'Document | CollectionItem'.
    Property 'start' is missing in type 'SourceToken' but required in type 'Document'.

[eemeli]

Have you tried yet what I recommended in my first reply, namely this?

You probably want to iterate on the parse() results until you find [a single CST document].

With your example as above, that translates to something like:

const cst = new Parser().parse(fileContents)
for (const doc of cst) {
  if (doc.type === 'document') {
    CST.visit(doc, (item, path) => {
      ...

[gee4vee]

Yes that works thx! One thing I'm wondering is how I can access properties like source which I can see when I print out a value, e.g.

      const tokens = new Parser().parse(fileContents);
      for (const token of tokens) {
        if (token.type === 'document') {
          CST.visit(token, (item, path) => {
            if (!CST.isCollection(item.value)) return;
            console.log(`visiting item: ${JSON.stringify(item.value?.items[0])}`);
          });
        }
      }

If I look at items[0], I can access properties like key and sep, but not source.

visiting item: {"start":[],"key":{"type":"scalar","offset":4,"indent":0,"source":"blueprint"},"sep":[{"type":"map-value-ind","offset":13,"indent":0,"source":":"}...

[gee4vee]

I think I got it now... this lets me find something by it's source, from which I can get the offset, and using the offset I can get the line number using LineCounter. Thanks for the help!

      const tokens = new Parser().parse(fileContents);
      for (const token of tokens) {
        if (token.type === 'document') {
          CST.visit(token, ({ key, value }) => {
            for (const tk of [
              key,
              value
            ]) {
              if (CST.isScalar(tk) && tk.source === 'workitems') {
                console.log(`workitems: line/col: ${JSON.stringify(lc.linePos(tk.offset))}`);
              }
            }
          });
        }
      }