Next.js cannot compile with turbo when lowlight is used

[kalvin807]

Link to the code that reproduces this issue

https://codesandbox.io/p/devbox/blazing-wood-go8s7s?file=%2Fapp%2Fpage.tsx%3A3%2C10
https://codesandbox.io/p/devbox/old-resonance-lv6j3x

To Reproduce

1. Follow instruction to install lowlight
2. Register a syntax highlight

import {createLowlight} from 'lowlight'
import xml from 'highlight.js/lib/languages/xml'

const lowlight = createLowlight()

lowlight.register({xml})

// Note: `html` is an alias for `xml`.
console.log(lowlight.highlight('html', '<em>Emphasis</em>'))

3. Start dev server with turbo (--turbo defined in package.json) npm run dev
4. Server crashes with error Failed to compile

Module not found: Can't resolve 'highlight.js/lib/languages/typescript'
> 1 | import typescript from 'highlight.js/lib/languages/typescript'
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  2 | import {common, createLowlight} from 'lowlight'
  3 |
  4 | const lowlight = createLowlight()

Current vs. Expected behavior

I expect it compile without error, because it works correctly when dev server is run without turbopack (without --turbo flag). But it is currently failing with module not found. I noticed --turbo works if I include highlight.js as dependency.

Provide environment information

Operating System:
  Platform: linux
  Arch: x64
  Version: #1 SMP PREEMPT_DYNAMIC Sun Aug  6 20:05:33 UTC 2023
  Available memory (MB): 4102
  Available CPU cores: 2
Binaries:
  Node: 20.9.0
  npm: 9.8.1
  Yarn: 1.22.19
  pnpm: 8.10.2
Relevant Packages:
  next: 14.3.0-canary.26 // There is a newer canary version (14.3.0-canary.28) available, please upgrade! 
  eslint-config-next: N/A
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.1.3
Next.js Config:
  output: N/A
 ⚠ There is a newer canary version (14.3.0-canary.28) available, please upgrade! 
   Please try the latest canary version (`npm install next@canary`) to confirm the issue still exists before creating a new issue.
   Read more - https://nextjs.org/docs/messages/opening-an-issue

Which area(s) are affected? (Select all that apply)

Turbopack

Which stage(s) are affected? (Select all that apply)

next dev (local)

Additional context

The mvp is based on the instruction of MantineRichEditor Example:
https://mantine.dev/x/tiptap/#code-highlight

_PACK-3017

[github-actions]

We cannot recreate the issue with the provided information. Please add a reproduction in order for us to be able to investigate.

Why was this issue marked with the please add a complete reproduction label?

To be able to investigate, we need access to a reproduction to identify what triggered the issue. We prefer a link to a public GitHub repository (template for App Router, template for Pages Router), but you can also use these templates: CodeSandbox: App Router or CodeSandbox: Pages Router.

To make sure the issue is resolved as quickly as possible, please make sure that the reproduction is as minimal as possible. This means that you should remove unnecessary code, files, and dependencies that do not contribute to the issue. Ensure your reproduction does not depend on secrets, 3rd party registries, private dependencies, or any other data that cannot be made public. Avoid a reproduction including a whole monorepo (unless relevant to the issue). The easier it is to reproduce the issue, the quicker we can help.

Please test your reproduction against the latest version of Next.js (next@canary) to make sure your issue has not already been fixed.

If you cannot create a clean reproduction, another way you can help the maintainers' job is to pinpoint the canary version of next that introduced the issue. Check out our releases, and try to find the first canary release that introduced the issue. This will help us narrow down the scope of the issue, and possibly point to the PR/code change that introduced it. You can install a specific version of next by running npm install next@<version>.

I added a link, why was it still marked?

Ensure the link is pointing to a codebase that is accessible (e.g. not a private repository). "example.com", "n/a", "will add later", etc. are not acceptable links -- we need to see a public codebase. See the above section for accepted links.

What happens if I don't provide a sufficient minimal reproduction?

Issues with the please add a complete reproduction label that receives no meaningful activity (e.g. new comments with a reproduction link) are automatically closed and locked after 30 days.

If your issue has not been resolved in that time and it has been closed/locked, please open a new issue with the required reproduction.

I did not open this issue, but it is relevant to me, what can I do to help?

Anyone experiencing the same issue is welcome to provide a minimal reproduction following the above steps. Furthermore, you can upvote the issue using the 👍 reaction on the topmost comment (please do not comment "I have the same issue" without reproduction steps). Then, we can sort issues by votes to prioritize.

I think my reproduction is good enough, why aren't you looking into it quicker?

We look into every Next.js issue and constantly monitor open issues for new comments.

However, sometimes we might miss one or two due to the popularity/high traffic of the repository. We apologize, and kindly ask you to refrain from tagging core maintainers, as that will usually not result in increased priority.

Upvoting issues to show your interest will help us prioritize and address them as quickly as possible. That said, every issue is important to us, and if an issue gets closed by accident, we encourage you to open a new one linking to the old issue and we will look into it.

Useful Resources

• How to Contribute to Open Source (Next.js)
• How to create a Minimal, Complete, and Verifiable example
• Reporting a Next.js bug
• Next.js Triaging issues

[timneutkens]

Seems you've copy-pasted the sandbox and didn't add any code into it, in the future please double check that the provided reproduction actually reproduces the issue.

[timneutkens]

Here's a sandbox of the reproduction steps working fine on next@latest: https://codesandbox.io/p/devbox/sleepy-bassi-stw4yq?file=%2Fapp%2Fpage.tsx%3A7%2C1&workspaceId=062b139a-c054-40fe-bbf5-0210d48b9dd0. Also verified next@canary.

[kalvin807]

Sorry for the wrong code sandbox link.
Here is the updated one: https://codesandbox.io/p/devbox/old-resonance-lv6j3x
The reproduction steps only working fine if highlight.js is specified as dependencies in package.json.

Without adding highlight.js in package.json. highlight.js is still installed in node_module and resolved in lockfile. next dev --turbo will fail to compile, but next dev can.

[timneutkens]

Thank for the reproduction! Was able to figure out what the problem is based on that.

So the problem is that webpack resolving in this particular case is incorrect, it allows you to import modules from a place that your application code shouldn't have access to, namely the node_modules of a nested dependency. Turbopack has a more correct resolving that matches closer to what Node.js does. If you try running the Node.js repl you'll notice that import('highlight.js/lib/languages/typescript')

Looking at the docs for the package you'll want to do something like this instead as lowlight includes the highlight.js data:

import { createLowlight, common } from "lowlight";

const lowlight = createLowlight();
lowlight.register({ typescript: common.typescript });

Lowlight also exports all for all of the languages, keep in mind those are all bundled, best to keep them in a server component.