New Code Node Proposal

[aarthificial]

The following is a proposal for a new Code node that's meant to eventually replace the current CodeBlock.
For the sake of simplicity, the proposal uses the present tense, but none of this has actually been implemented.
Nothing is set in stone and feedback is most welcome.

Generic Tag Function

A new generic tag function - CODE - is used to declare and store source code.
It operates in a similar way to code-fns tags with two major differences:

1. It does not require information about the language - the same tag function is used for everything.
2. The returned value is an array that can be reused in other tag functions:

   // the following code:
   yield* codeRef().edit(0.6)`export const ${modify('Example', 'Hello')} = 7;`;
   
   // could be written as:
   const source = CODE`export const ${modify('Example', 'Hello')} = 7;`;
   yield* codeRef().edit(0.6)`${source}`;

First-Class Signal Support

Fragments of code can be represented by a new dedicated signal created as follows:

const codeFragment = Code.createSignal('Example');

The value of this signal can be provided as either a string or the array returned by CODE:

const codeFragment = Code.createSignal(CODE`\
interface Color {
  alpha: number;
}`);

It's possible to nest signal values inside of template strings.
This includes the new code signals:

const source = CODE`\
export class ${codeFragment} {
  // implementation
}`;

As well as standard functions:

const position = Vector2.createSignal([200, 300]);
const source = CODE`\
  const position = new ${() => position().toString()};
`;
// const position = new Vector2(200, 300);

The contents of a Code node are also represented by a code signal.
The following is a valid example:

const className = Code.createSignal('Example');

view.add(
  <Code>
    {CODE`
    export class ${className} {
      public constructor(props: ${className}Props) {
        super(props);
      }
    }`}
  </Code>,
);

Modifying Code Signals

append

// add immediately:
fragment.append('code to append at the end');
// animate
yield* fragment.append('code to append at the end', 0.3);
// animate with tag
yield* fragment.append(0.3)`code to append at the end`;

prepend

// add immediately:
fragment.prepend('code to prepend at the beginning');
// animate
yield* fragment.prepend('code to prepend at the beginning', 0.3);
// animate with tag
yield* fragment.prepend(0.3)`code to prepend at the beginning`;

insert

Inserts the code at a given position.

// add immediately
fragment.insert([0, 12], 'Red');
// animate
yield* fragment.insert([0, 12], 'Red', 0.3);

modify

Replaces the code at a given range.
The first argument is a range meaning that selection helper functions could also be used here.

// change immediately
fragment.modify([[0, 12], [0, 16]], 'Yellow');
// animate
yield* fragment.modify(word(0, 12, 4), 'Yellow', 0.3);

remove

Removes the given range.

// remove immediately
fragment.remove([[0, 12], [0, 16]]);
// animate
yield* fragment.remove(word(0, 12, 4), 0.3);

bulk

Takes an array of code modifications and applies them at the same time.

// apply immediately
fragment.bulk([
  [word(0, 12, 12), 'replaced text'],
  [word(1, 11, 0), 'inserted text'],
  [word(2, 1, 7), ''],
]);
// animate
yield* fragment.bulk(0.3, [
  [word(0, 12, 12), 'replaced text'],
  [word(1, 11, 0), 'inserted text'],
  [word(2, 1, 7), ''],
]);

Default tween

The default signal tween animates from the current code to the new one using a diff algorithm:

yield* fragment(
  CODE`\
const variable = 'completely new code';
console.log('hello');`,
  0.3,
);

Modifying Code Nodes

All of the above methods are available in the Code node itself:

yield* codeRef().code.prepend(0.6)`\
import {Red} from '../colors';
import {Style} from '../styles';`;

On top of that, the node provides the traditional edit API:

yield* codeRef().edit(0.6)`\
export class ${modify('Example', 'Hello')}() {
  public constructor() {
    ${insert(`console.log('hello')`)}
  }
}`;

Putting it all Together

The following is a complete example of using code signals together with the Code node:

const className = Code.createSignal('Example');
const classBody = Code.createSignal('');

view.add(
  <Code ref={codeRef}>
    {CODE`
    export class ${className} {
      public constructor(props: ${className}Props) {
        super(props);
      }${classBody}
    }`}
  </Code>,
);

yield* all(
  className('Hello', 0.3);
  classBody(0.3)`
      public greet() {
        console.log('hello!');
      }`;
);

yield* classBody.append(0.3)`
      protected calculate() {
        return 3 * 4;
      }
`;

Full UTF8 support + Ligatures

The following types of characters should just work:

• CJK characters
  (Example font: "Noto Sans Traditional Chinese"):

  柯灰柯灰柯灰
  

  

• Code ligatures
  (Example font: "JetBrains Mono"):

  => !=
  

  

• Emojis:

  ❤️ 😊 😀
  

Highlighting

The Code node uses Lezer for parsing and highlighting the text.
It's the same parser powering CodeMirror 6. The @codemirror/legacy-modes package is used to provide support for all the languages supported by CodeMirror 5.

Specifying the Language

The language used by the Code node needs to be explicitly imported and passed to the node.
This ensures that all required languages are known at compile time and tree shaking can do its job.

For the most popular languages, @motion-canvas/2d provides proxies, making it as easy to use as possible:

import {tsx} from '@motion-canvas/2d/lib/languages';

view.add(<Code parser={tsx}/>);

More niche languages can still be used, either by directly importing the corresponding Lezer parser:

import {parser} from '@lezer/julia';

view.add(<Code parser={parser}/>);

Or, using the legacy mode:

import {csharp} from '@codemirror/legacy-modes/mode/clike';

view.add(<Code parser={createParser(csharp)}/>);

Styles

Styles are created using the same HighlightStyle mechanism CodeMirror uses. This makes it possible to use preexisting CodeMirror themes and easily define custom ones. Styles are passed directly to the Code node:

const MyHighlightStyle = HighlightStyle.define([
  {tag: tags.keyword, color: "#fc6"},
  {tag: tags.comment, color: "#f5d"}
])

view.add(<Code parser={tsx} style={MyHighlightStyle} />);

Decorations

The Code node provides a flexible API for creating custom decorations that can be applied at specific text ranges.
These can be used to implement things like selections, error highlights, and underlines. A detailed discussion about this feature can be found in #258.

Automatic selection

The old CodeBlock automatically selected newly inserted text to make it stand out.
In the new implementation, whether new insertions and edits are selected should be controlled by a separate property:

// This line won't get selected:
yield* codeRef().code.append('const a = 3', 0.3);

codeRef().selectOnEdit(true);
// This line will get selected:
yield* codeRef().code.append('const b = 7', 0.3);

Release Strategy

The Code node will be introduced as a separate class from CodeBlock, exported from @motion-canvas/2d/components.
The idea is to let us slowly experiment and work on the new API while keeping CodeBlock around as the recommended solution.
Once Code is ready, the original CodeBlock node will be marked as deprecated and scheduled to be removed in v4.

[hhenrichsen]

I like this, especially hooking into the highlighting library so that we don't have to worry about supporting each language at a library (or code-fns) level.

It might also be good to look into different diffing strategies. Maybe we can even hook into git. As a user of the library, my ideal would be to just have text blocks that keep track of what I want the code to be at a certain point; an API like this:

const myCode = createRef<Code>();
myCode().parser(python);

// Set the initial code value
myCode().edit("print('Hello, world!')");

// Add the code for the function definition over 2 seconds
yield* myCode().edit("main()

def main():
    print('Hello, world!')
", 2);

If I want more control over what that transition looks like, I can use the more low-level editing functions.

Another proposal: it'd be good to have some sort of API for getting the bounding boxes of CodeRanges, and perhaps even CodeRanges from text. I think this is one of the most commonly-asked-for features when working with CodeBlocks.

With regard to the decoration API, here's what I was looking at for the individual decoration interface. I don't think that this needs to be the solution there, but I did a dive into what some of the decorations would need and this is what I landed on:

export interface CodeDecoration {
  /**
   * The range that this decoration should affect.
   */
  range: CodeRange[] | SignalValue<CodeRange[]>;
  /**
   * If this decoration handles drawing the text itself. Multiple decorations of this type on the
   * same range are mutually exclusive.
   */
  needsToDrawText: boolean;
  /**
   * Text is drawn at 0;
   * Anything that needs to manipulate the context or draw before text should be negative.
   * Anything that needs to draw on top of text should be positive.
   */
  priority: number;

  /**
   * Draw the decoration on the text, or if needsToDrawText is set, draws the text.
   */
  draw(
    node: CodeBlock,
    position: Vector2,
    progress: number,
    char: string,
    lineHeight: number,
    width: number,
    context: CanvasRenderingContext2D,
  ): void;

  /**
   * Called on negative-priority decorations in inverse order once the text has been drawn.
   */
  cleanUp(
    node: CodeBlock,
    position: Vector2,
    char: string,
    lineHeight: number,
    width: number,
    context: CanvasRenderingContext2D,
  ): void;
}

[Ross-Esmond]

I don't think you want to use diffing as much as you might think. I experimented with this a lot when I started code-fns. All diffing algorithms are going to regularly misinterpret the intent of the video maker, because there's no way to determine intent.

When you use a template with a signal, you know what changed. Same with inserting, replacing, and deleting. Using a diff would involve a more verbose, less predictable API. You have to pass the full new string whenever you want to update only part of it, and you have to hope that the diffing algorithm gets the change right. Brackets are frequently assigned incorrectly in every algorithm I've managed to find.

[hhenrichsen]

The most frequent question I've seen on discord related to CodeBlocks is how to go from one source state to another.

The first is a bit more intuitive to those unfamiliar with template literals and more readable in multiline scenarios than the second, in my opinion.

code().edit(`print("Hello, world")`);
code().edit(
`def main():
    print("Hello, world")`, 2) 

code().edit`print("Hello, world")`
code().edit`${insert('def main():\n    ')}print("Hello, world")`

I'm not proposing that this is the only way:

If I want more control over what that transition looks like, I can use the more low-level editing functions.

[Ross-Esmond]

Yeah. Part of the problem is that code-fns was built to work differently than how the final API worked in MC.

The diff API is likely to be the most common solution for people, since it's the simplest to get started with. The moment you use a handful of brackets, though, it will fail. I suspect that you could actually make an algorithm that gets the brackets right, but I can't find anyone that's done it yet.

If diffing becomes the norm, the next most common question will likely be "how do I get this bracket to go to the right place". (Or I find out I'm the only person that cares)

I'm saying I think the diffing approach should actually be downplayed for anything other than the simplest transition (no brackets).

This was more targeted at the proposal in general, rather than your comment. You just brought up the subject of diffing.

[aarthificial]

To address the final API part: the edit method is just a helper function - the way code is tweened works (I believe) the way code-fns was designed to. I added edit when working on the Motion Canvas release video to speed up the process.
The thing about code-fns is that it requires both states to have the same structure, i.e. the same amount of ${} expressions.
So, say you start with something like this:

view.add(
  <CodeBlock
    ref={ref}
    code={`export class Example {
  public constructor() {}    
}`}
  />,
);

In order to animate another method appearing, I need to first set the same code with the correct structure and then tween to the new version:

ref().code(tsx`export class Example {
  public constructor() {}${''}
}`);

yield* ref().code(
  tsx`export class Example {
  public constructor() {}${`
  public render() {}`}
}`,
  0.6,
);

edit is a helper method for doing exactly this:

yield* ref().edit(0.6)`export class Example {
  public constructor() {}${insert(`
  public render() {}`)}
}`;

Of course, in this simple example, I could just define the CodeBlock immediately with the correct structure, but when you have dozens of tweens adding and removing bits of the code it's practically impossible to do. Maybe I'm misunderstanding the way code-fns is meant to be used and there's something I don't realize, I'm happy to be corrected.

When it comes to diffing, I think a good solution would be to use ${} as hints as to which parts of the code should stay the same.
If the following example:

view.add(
  <Code
    ref={ref}
    code={`export class Example {
  public constructor() {
    // This is a constructor
  }    
}`}
  />,
);

yield* ref().code(
  `export class Example {
  public constructor() {
    // This is a constructor
  }
  public render() {
    // This is a constructor
  }
}`,
  0.6,
);

Produces an incorrect tween like this:

  export class Example {
    public constructor() {
      // This is a constructor
+   }
+   public render() {
+     // This is a constructor
    }
  }

You could wrap the newly added method in an expression:

yield* ref().code(
  `export class Example {
  public constructor() {
    // This is a constructor
  }${`
  public render() {
    // This is a constructor
  }`}
}`,
  0.6,
);

The diff algorithm could then use the individual parts, try to find them in the previous state, and if they are present, make keeping them the same a priority. (Without the need for the previous code to have the same structure though)

[Ross-Esmond]

Maybe I'm misunderstanding the way code-fns is meant to be used and there's something I don't realize, I'm happy to be corrected.

I don't think so. The MC API just wound up having a different trade off than I was expecting when I wrote code-fns.

I expected to use it with a function that generates the template which you always called for every change. So like

function makeCode(name, call, arg) {
  return tsx`
    function ${name}() { ${call}(${arg}); }
  `;
}

// ...

myCodeBlock.code(makeCode('foo', 'bar', 'true'));
myCodeBlock.code(makeCode('foo', 'baz', 'true'));
myCodeBlock.code(makeCode('foo', 'baz', 'false'));

This would create:

function foo() { bar(true); }

function foo() { baz(true); }

function foo() { baz(false); }

The complexity of the generator function makeCode depends on the number of places in the code where you want to be able to make changes, rather than the amount of times you want to update the code. The MC API you built required that you respecify all of the code on every update so that it matched up, but also allowed you to change whatever you wanted throughout the code, while keeping the template simple, so it was just different.

myCodeBlock.code(`function foo() { bar(true); }`);
myCodeBlock.code(`function foo() { ${replace('bar', 'baz')}(true); }`);
myCodeBlock.code(`function foo() { baz(${replace('true', 'false')}); }`);

Of course, signals are better than either of those solutions, and are more "on brand" anyways. Mine is more of a functional approach whereas yours is more of an imperative approach, and they both wind up doing roughly the same thing. My API, however, really struggled with adding a sequence of things one by one. Like if you wanted to reveal 10 consecutive lines of code one at a time. I can't recall if I ever added a solution for that, but you just have append and it's fine.

Of course, hard setting the code property would be fine if there existed a text diffing algorithm that got brackets right. I might try to hack that out, just to see if I can.

[Ross-Esmond]

The reason for using tsx`true` as opposed to just CODE`true` is that you can get some editors to syntax highlight the code in the first option. Vim with tree-sitter would do it for me but VS code has some plugins for it as well. In code-fns I used a proxy object to let you use whatever token you wanted so that the editor knew which language highlight to use.

[Ross-Esmond]

I should probably note (before someone else does) that you can always just alias the function. const tsx = CODE Nothing really needs to change here. It could make it into the documentation though.

[Ross-Esmond]

Using numbers to identify locations will make changes to the in-video code difficult. Imagine if you have 30+ code transitions and you decide to add a line. You'd have to visit all the modifications and edit the line number to fix their locations.

I think the API should allow for string matching. Something like

insert(['func(', ')'], 'param')
replace('param', 'arg')
remove('arg')
// Or, for extra context
replace(['func(', 'param', ')'], 'arg')
remove(['func(', 'arg', ')'])

If the target string is not found or is found to be "ambiguous" (multiple matches) you throw an error. This has a few advantages for the developer.

• Coming up with a target string is easier than trying to count lines and characters.
• It allows for search and replace when you change the target string.
• It's less likely to break in general. (If I add a line, every following line number always changes.)
• When it does break due to an edit, that problem is usually brought to the attention of the developer, as opposed to silently being wrong.

[aarthificial]

I personally don't like this API (using ranges) at all, I added it because it was requested on Discord a handful of times.
Using string matching is an interesting idea but it's worth noting that it's more of an expansion to the range methods.
As in, it can be easily implemented using them:

fragment.modify(fragment.findRange('param'), 'arg');

A little side note to this part though:

Imagine if you have 30+ code transitions and you decide to add a line

In my experience, this never happens. And if it does, it's virtually impossible to design the animation so that you can make a change in only one place. It would require writing the whole thing explicitly in terms of insertions/removals which may be worth it if your animation is meant to change often, but for the purpose of a one-off video, I don't think anyone would find that worthwhile.

[Ross-Esmond]

If the findRange API exists or can easily be tacked on later than I'm good. I think the documentation should push it before numbers (you don't know you need a refactor until you do) but that can be discussed later with the docs.

[Ross-Esmond]

Eventually you should steal code-fns's indentation correction. Whenever you have a string like

    const calls = CODE`
      someCode();
      otherCode();
    `;

It would have its indentation removed. Then if you inserted it into

    CODE`
      function () {
        ${calls}
      }
    `;

the indentation on every line would be replaced with the indentation of ${calls}.

I don't know how well this lines up with your proposal, but it was a huge improvement whenever I added it to code-fns.

The indentation was corrected whenever you started the string with a new line. So you wouldn't correct

    CODE`function () {
        ${calls}
      }
    `;

[gk-patel]

Just as a note - I would reference this issue here
#815

It would be good if emojis are supported.

Thank you all for the great tool.

[aarthificial]

Thanks for the note!
Added a "Full UTF8 support + Ligatures" section to the proposal describing this.

[mpaxson]

This looks extremely Slick.

I saw https://github.com/hhenrichsen/motion-canvas-code-node-proposal and was curious if there's plans to incorporate this?

[hhenrichsen]

The <Code> node that I proposed has already been merged to main and will be released as an experimental feature sometime in early March.

[jasonjmcghee]

Out of curiosity, why does the developer need to think about insert, edit, remove etc. at all?

I understand wanting / providing an interface for manual control (and I'm not suggesting to remove that), but my guess is often people don't need that level of control.

Using diffWords in https://github.com/kpdecker/jsdiff under the hood and parsing the output to figure out insert, edit, remove, and allowing specifying custom timing / effects etc, could simplify many use cases.

Imagine:

const source = Code.createSignal(`\
interface Color {
  alpha: number;
}`);
yield* source.edit(0.6)`\
interface Color {
  red: number;
  blue: number;
  green: number;
}`)

And that's it.

It knows the current state and knows how to perform the necessary operation. (could be either insert + delete or edit)

Another example would be:

yield* source.edit(0.6)`\
interface Color {
  red: number;
  blue: number;
  green: number;
  alpha: number;
}`)

It performs the insert operation as defined in the docs / new api above.

You could allow customizing that with a high level api too in a number of ways (animate / delay per line, etc.).

---

A code snippet to demonstrate the idea (which doesn't work in the existing API, but paints a picture):

import { diffWords } from "diff";

class CodeToBeModified {
  constructor(public code: string) {
    this.code = code;
  }

  set(value: string) {
    const before = this.code;
    this.code = value;
    return this.buildEdit(before, this.code);
  }

  /**
   * Build up a list of the changes and non-changed strings
   * by executing `diffWords` and iterating through to find
   * every operation / piece.
   */
  buildEdit(before: string, after: string): string[] {
    const diffOut = diffWords(before, after);

    const out = [];

    for (let i = 0; i < diffOut.length; i++) {
      if (diffOut[i].added && !diffOut[i+1].removed) {
        out.push(insert(diffOut[i].value));
      } else if (diffOut[i].removed && diffOut[i-1].added) {
        out.push(edit(diffOut[i].value, diffOut[i-1].value));
      } else if (diffOut[i].removed) {
        out.push(remove(diffOut[i].value));
      } else {
        out.push(diffOut[i].value);
      }
    }

    return out;
  }
}

Which would require something like:

const code = CodeToBeModified(`\
interface Color {
  red: number;
  blue: number;
  green: number;
  alpha: number;
}`)

yield* source.edit(0.6, code.set(`\
interface Color {
  red: number;
  blue: number;
  green: number;
}`))

Which would obviously be better with proper abstraction.

[hhenrichsen]

Thanks for taking the time to write this out! Finding a solution like this was important to me as well when I was starting on the proposed solution, so I found something like this. It's also why there's a note about a Default Tween in the proposal.

I'm using the current state of the new node in this repo if you wanted to take a peek. If you need finer control over tweening, we still support the edit API (insert, replace, remove, etc.), but we also provide a defaultTweener where the code to tween something looks like this. I opted to use the patienceDiff (which was then shifted to work on tokens rather than lines thanks to Jacob's work) because it did a better job of respecting the whitespace, and there were implementations out there that I could base it on, and I've been satisfied with the results.

[jasonjmcghee]

Ah. Missed that in the code proposal and/or didn't understand.
Very cool. I'll show myself out!

[aarthificial]

Also, in case you're interested, the translation you described - from the output of a diff algorithm to a series of inserts, replaces and removes - is defined using a CodeDiffer type:
https://github.com/motion-canvas/motion-canvas/blob/main/packages/2d/src/lib/code/CodeDiffer.ts
Right now the implementation is hardcoded to use the defaultDiffer but it will be possible to define a custom one, in case one would want to use jsdiff, for example.

[jasonjmcghee]

Appreciate that. It seems to do exactly what I was hoping for. No particular affinity to jsdiff- mostly using it as an example to communicate my thoughts.