ci: Introduce Nx agents

[ZackDeRose]

Playing with Nx agents here.

Here's what I'm seeing:

• We can do a bit better by bumping up the agent count from 3 => 12 (free and not as restrictive as github seems to be when asking for a bunch of machine), and reducing the parallel option from 5 => 3 to not overload any one machine. Looks like now we're getting 3.5 minutes for a full run. We might try for more agents here, it's worth exploring!

• A downside from the legacy DTE setup to Nx agents is in legacy, we could setup workers and main machines all in parallel, now we have to setup the main job and then setup all other agents. We're losing about 20s here with cached node_modules - looks like this is made up for by bumping up agent counts though. When we launch Nx Workflows, we should be able to get that 20s back.

• There's some more optimizations we could try later, adding in small/med/large agent loads could help here for getting the full swarm when we need to run everything, and not get overly excited for smaller PRs

• Potentially we can add in the format and knip tasks to the swarm. This is probably trivial for perf, but it could help in the sense that all tasks can be run the same way (instead of setting up specific machines for the format and knip tasks), and we could see it in the nx cloud task list instead of spelunking github action logs!

• Nx Agents can be used for the main job as well - looks like those workflows are growing up to ~10 minutes now. I think now it's solid enough that we can use it reliably for the releases, while using the --skip-cache option to ensure we're running all checks before publishing!

@lachlancollins / @TkDodo let me know thoughts here, I'll clean this up a bit so we can merge if you like!

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
query	⬜️ Ignored (Inspect)	Visit Preview		Feb 10, 2024 11:22pm

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 1fdf144. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

---

✅ Successfully ran 1 target

• nx affected --targets=test:eslint,test:lib,test:types,test:build,build --parallel=3

---

Sent with 💌 from NxCloud.

[codesandbox-ci]

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit 1fdf144:

Sandbox	Source
@tanstack/query-example-angular-basic	Configuration
@tanstack/query-example-react-basic-typescript	Configuration
@tanstack/query-example-solid-basic-typescript	Configuration
@tanstack/query-example-svelte-basic	Configuration
@tanstack/query-example-vue-basic	Configuration

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (7a51c2a) 41.82% compared to head (899075b) 41.82%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #6863   +/-   ##
=======================================
  Coverage   41.82%   41.82%           
=======================================
  Files         178      178           
  Lines        7015     7015           
  Branches     1416     1416           
=======================================
  Hits         2934     2934           
  Misses       3718     3718           
  Partials      363      363           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ZackDeRose]

@lachlancollins / @TkDodo we're in a merge-able state now

[lachlancollins]

@ZackDeRose regarding the optimisations you mentioned above, how difficult would it be to implement agents for the ci.yml workflow? As you mentioned, CI times are now 9+ mins as we're testing examples too. I don't think it would be necessary to skip cache if it's already been tested in the exact same setup in a PR.