Dev server HMR and *.server.ts naming convention doesn't work with monorepos using shared routes

[jrestall]

What version of Remix are you using?

v1.18

Are all your remix dependencies & dev-dependencies using the same version?

• Yes

Steps to Reproduce

Minimal Reproduction: https://github.com/jrestall/remix-nx-monorepo/

1. Create a monorepo using Nx or Turborepo such as Logan's example at https://github.com/mcansh/remix-shared-routes/ that has shared route packages outside the app directory.
2. Setup the new dev server.
3. Modify files outside the main app directory and note HMR doesn't work.
4. Add files outside the main app directory that are used by shared routes that follow the *.server.ts pattern and note they don't get replaced with an empty module.

Expected Behavior

HMR and *.server.ts naming convention works with code outside the app dir that uses shared routes.

Actual Behavior

Following are the areas of code that limit HMR and *.server conventions to only work with routes in the app dir. I'm happy to submit a PR to fix this if a better approach can be agreed.

HMR

remix/packages/remix-dev/compiler/js/plugins/hmr.ts

Lines 118 to 122 in 40a4d7d

	!fs.existsSync(args.path) ||
	!args.path.startsWith(config.appDirectory))
	) {
	return undefined;
	}

remix/packages/remix-dev/compiler/js/plugins/hmr.ts

Lines 134 to 136 in 40a4d7d

	args.path.startsWith(config.appDirectory)
	? fs.statSync(args.path).mtimeMs
	: undefined

If the goal here is to not add react refresh to node_modules code, could we perhaps explicitly check whether the file is in node_modules instead?

Naming Convention

remix/packages/remix-dev/compiler/plugins/emptyModules.ts

Lines 24 to 28 in 40a4d7d

	path
	.resolve(args.resolveDir, args.path)
	.startsWith(config.appDirectory)
	) {
	return { path: args.path, namespace: "empty-module" };

Instead of limiting this convention to the app dir, if the goal is to exclude node_modules, could we change it to something like the following?

    !args.path.includes("node_modules")

[omakise]

I had the same problem

[DanielSchiavini]

HMR is working fine for me with NX monorepo

[jrestall]

HMR is working fine for me with NX monorepo

Are you using typescript path aliases to share code in the monorepo or a separate build for the shared packages? I think path aliases are causing the issue since it means the linked source code is outside the app dir.

We use path aliases to share feature routes and code between multiple remix/react-router apps. Would be great to hear what your Nx setup with Remix looks like.

[DanielSchiavini]

I have a few typescript libs in de libs/ folder, they are imported via the (automatically configured) aliases.
None of the libraries is buildable or publishable.
All the routes are in the remix app, though, with loaders, the implementation is in the libs.

[jrestall]

All the routes are in the remix app, though, with loaders, the implementation is in the libs.

Thanks, this is the key. If the route files are shared outside the app dir then HMR and naming conventions don't work.

I've created a minimal reproduction repo at https://github.com/jrestall/remix-nx-monorepo/ which shows the issue is with sharing routes that exist outside the app dir. Editing a shared route file or any file referenced by the shared route will not work with HMR or *.server.ts conventions.

I'm using mostly the same approach to share routes between many apps as Logan does here except using the flat routes package as configured here. Moving routes to each of our apps isn't an option as it would duplicate 100's of file-based routing configs that would get out of sync, even if the route module's implementation was shared.

[Jarrodsz]

Same problem preventing code move to mono repro packages. Seems common isecase can we please have a fix for this asap?🙏

[Jarrodsz]

I want to move routes to packages too to prevent duplication. Maybe as a bonus this van be made possible

[HerrBertling]

I can already see me running into this, so yeah, fixing this would make future me very happy 😃

[ryanflorence]

Talked with @pcattori about this. There are two concepts here that we need to be able to tell the compiler:

a. what to watch for changes for HMR
b. to apply some of our file conventions (*.{server|client}.ts) to sources outside of appDirectory

For (a) we already have watchPaths.

For (b) we have appDirectory but it is limited to a single directory and applies other file conventions like root, entry.server, etc.

I'm thinking we rename watchPaths to sourcePaths and then do both (a) and (b) with those directories.

We also discussed that this is something we can add after v2 ships.

Thoughts?

[jrestall]

Hi @ryanflorence and @pcattori, thanks for looking into this, your proposed solution looks elegant and I believe would solve the issue.

Tooling like Vite uses complex logic to determine the workspace root for such cases but reusing the watched source files seems nice and simple.

[jrestall]

This issue seems to be almost fixed if using the vite plugin, there's just one bit of code that incorrectly assumes routes are always under the app directory.

remix/packages/remix-dev/vite/plugin.ts

Line 1300 in c821253

if (!file.startsWith(vite.normalizePath(pluginConfig.appDirectory))) return;

@markdalgleish @pcattori

[jrestall]

Closing as completed by #8795 🎉

The Vite plugin now has very good monorepo support, can load routes from anywhere in the workspace and doesn't require any patches, the presets functionality is especially useful for configuring multiple apps without duplicating config. The *.server.ts naming convention also works well in the monorepo.

[markdalgleish]

@jrestall Oh, that's cool you're using the presets feature too! Out of interest, do you mind sharing your preset? I'd love to see it!

[jrestall]

We are using presets to avoid duplicating the way shared library routes are defined. Each monorepo app can reference shared functionality which can contain remix routes, so we use Nx (createGlobPatternsForDependencies) to get the route folders for each of the app's workspace dependencies and then remix-flat-routes to define them since it supports multiple route folders. Not all libraries are shared but we also structure the repo in this way to enforce separation between features as the banking domain is very large, so this enables us to scale the Remix site to very many teams and features whilst enforcing modularity.

The preset is written in JS rather than TypeScript due to vitejs/vite#5370.

// remixPreset.mjs

import fs from "fs";
import path from "path";
import { createGlobPatternsForDependencies } from "@nx/react/tailwind.js";
import { flatRoutes } from "remix-flat-routes";

/** @type {(base: string, appdir: string) => import('@remix-run/dev').Preset} */
export function remixPreset(base, appdir) {
  async function defineRoutes(defineRoutes) {
    const appFolder = path.join(appdir, "app");

    const routeDirs = createGlobPatternsForDependencies(
      appdir,
      "/src/routes"
    )
      .filter((routePath) => fs.existsSync(routePath))
      .map((routePath) => path.relative(appFolder, routePath));

    return flatRoutes(["routes", ...routeDirs], defineRoutes, {
      basePath: base,
      ignoredRouteFiles: ["**/*.test.{js,jsx,ts,tsx}", "**/*.snap"],
    });
  }

  return {
    name: "default-monorepo-preset",
    remixConfig: () => ({
      // ignore all files in routes folder
      ignoredRouteFiles: ["**/.*"],
      // add routes from all referenced lib projects
      routes: defineRoutes,
    }),
  };
}

[kiliman]

and then remix-flat-routes to define them since it supports multiple route folders

😍 I knew that feature would come in handy

Also, since Remix now supports basename directly, you probably want to use that instead of the basePath option since this only adds the prefix to all top-level routes.

One more thing. Your ignoredRouteFiles config should be [**/*]. You're currently only ignoring dot files.

[jrestall]

Thanks so much @kiliman! Your work on remix-flat-routes made the modular monorepo architecture possible with how flexible your package is. Your support for basePath was also really useful.

Appreciate the tips, we definitely plan to move to basename but it'll take a while as there's a lot of custom code to rollback that handled redirects and links under a base path. Nice catch on the ignoredRouteFiles thanks, the comment should just mention dot files rather than all files so I'll remove that config now that #8801 is in.