Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rollup-plugin] - Sourcemaps failing to upload when directory containing them has been deleted prior to upload happening #1053

Closed
joekrump opened this issue Mar 29, 2023 · 16 comments · Fixed by #1059
Assignees
Labels
rollup/vite @honeybadger-io/rollup-plugin

Comments

@joekrump
Copy link

What are the steps to reproduce this issue?

  1. Set up an app which runs vite build in a Heroku Nodejs buildpack
  2. Have the build command run in the buildpack

What happens?

  • Most sourcemap files that are output are uploaded
  • Sourcemap files that are found in the ephemeral tmp/build_... directory are not uploaded because they no longer exist, even though their entries show up in the bundle object which this plugin reads entries from. The build fails because this results in an Error being thrown.

What were you expecting to happen?

  • No issues. Sourcemaps are uploaded successfully.

Any logs, error output, etc?

image

Any other comments?

What versions are you using?

Operating System:
Building on the Heroku-20 stack
-----> Using buildpacks:
1. heroku/nodejs
2. heroku/ruby
Package Name: @honeybadger-io/rollup-plugin
Package Version: 5.1.5
Browser Version: N/A

@joekrump
Copy link
Author

It would be great if I could set something like an ignorePath regex or glob which I could set to "tmp/build_*" in the plugin config options. That would then be able to be used in isSourcemap: https://github.com/honeybadger-io/honeybadger-js/blob/a10f68655e4949e32f0fbe0ed8cdb763232d6497/packages/rollup-plugin/src/rollupUtils.ts#LL21C25-L21C25

@joekrump joekrump reopened this Mar 29, 2023
@BethanyBerkowitz
Copy link
Contributor

Hi @joekrump, thanks for reaching out! Just to clarify, you believe the files in tmp/ should not be uploaded, correct? I want to understand why they're even being generated since I don't see that happening when building locally with Vite. When I build, it looks like I just get assets in /dist/assets. Do you know if this is a Heroku thing?

Your suggestion of setting an ignorePath seems very reasonable, I'm just wondering if dumping files into /tmp and then deleting them is expected behavior.. if so, perhaps we should exclude those files by default rather than adding additional config.

Are you able to tell what folder those assets that were successfully uploaded were located in?

Also, would you mind sharing your vite.config.js just in case there's anything relevant?

@subzero10 subzero10 added the rollup/vite @honeybadger-io/rollup-plugin label Mar 31, 2023
@joekrump
Copy link
Author

joekrump commented Apr 11, 2023

Hey @BethanyBerkowitz sorry for the delay in getting back to you. I think the reason things are ending up in tmp is because the build is happening in a Heroku buildpack which is ephemeral and by the time the upload attempts to happen, it is no longer available.

It's a similar problem as to what is described in this SO post: https://stackoverflow.com/questions/73552104/cant-access-heroku-tmp-build-sha-build-files-in-github-action-once-build-has

.js.map files that were successfully uploaded existed in my app's public/vite/assets directory.

So I think that what is happening is that because the build happens in this ephemeral Heroku buildpack, that in the output bundle file is created (which is read by this rollup plugin), it lists that there are .js.map files within the tmp directory (which there were at the time of the build), however, by the time the sourcemap upload happens, those files and that entire directory have been cleaned up and only the files in public/vite/assets remain.

vite.config.ts:

import { defineConfig } from 'vite';
import rubyPlugin from 'vite-plugin-ruby';
import vue from '@vitejs/plugin-vue';
import svgLoader from 'vite-svg-loader';
import fullReload from 'vite-plugin-full-reload';
import { brotliCompress } from 'zlib';
import { promisify } from 'util';
import gzipPlugin from 'rollup-plugin-gzip';
import honeybadgerRollupPlugin from '@honeybadger-io/rollup-plugin';

const HONEYBADGER_API_KEY = JSON.stringify(process.env.HONEYBADGER_FRONTEND_API_KEY)?.trim();
const HONEYBADGER_REVISION = JSON.stringify(process.env.SOURCE_VERSION)?.trim();
const ENVIRONMENT = JSON.stringify(process.env.ENVIRONMENT)?.trim();
// eslint-disable-next-line no-relative-import-paths/no-relative-import-paths, import/extensions
import { RESOLUTION_ALIASES } from './config/vite';

const plugins = [
  // Do a full reload when changes are made to files with paths that match these.
  fullReload(['app/views/**/*']),
  rubyPlugin(),
  svgLoader(),
  vue(),
];

const rollupPlugins = [];

// Only compress assets in non-development environments.
if (process.env.VITE_RUBY_MODE && process.env.VITE_RUBY_MODE !== 'development') {
  const honeybadgerPluginOptions = {
    apiKey: process.env.HONEYBADGER_FRONTEND_API_KEY,
    assetsUrl:
      process.env.ENVIRONMENT === 'staging'
        ? 'https://staging-asset-url.com'
        : 'https://production-asset-url.com',
    revision: process.env.SOURCE_VERSION,
  };
  // used for uploading sourcemaps to honeybadger to allow for better debugging
  rollupPlugins.push(honeybadgerRollupPlugin(honeybadgerPluginOptions));

  const brotliPromise = promisify(brotliCompress);
  plugins.push(
    gzipPlugin({
      customCompression: (content) => brotliPromise(Buffer.from(content)),
      fileName: '.br',
    })
  );
}

export default defineConfig({
  build: {
    sourcemap: process.env?.VITE_RUBY_MODE === 'development' ? 'inline' : 'hidden',
    rollupOptions: {
      plugins: rollupPlugins
    }
  },
  resolve: {
    alias: RESOLUTION_ALIASES,
  },
  define: {
    HONEYBADGER_API_KEY,
    ENVIRONMENT,
    HONEYBADGER_REVISION,
  },
  plugins,
});

@subzero10
Copy link
Member

Hey @joekrump, thanks for the update and the config file.

I have a question:

  • Those files that are referenced in the tmp folder, aren't they necessary files to be uploaded to Honeybadger? I mean, if we went ahead with an ignorePath option or ignored files in tmp folder by default, you would be missing source maps, correct?

Additionally, a comment on your vite.config.ts file:

  • You define HONEYBADGER_API_KEY and HONEYBADGER_REVISION, but I think they are irrelevant since you set apiKey and revision in honeybadgerPluginOptions. Is this intended?

Finally, I think I have all I need to try this on my side and see if I can reproduce your issue and debug further.

@joekrump
Copy link
Author

Re vite.config.ts. I've defined HONEYBADGER_API_KEY, etc because they are used elsewhere in some other js code where I'm calling Honeybadger.configure() when my vue app first mounts.

@joekrump
Copy link
Author

The files in tmp get copied over into my public directory and so all the files will end up existing there.

@subzero10
Copy link
Member

Hey @joekrump, I created a very simple vite app and deployed on Heroku. I couldn't reproduce the errors shown above.

In any case, I will go ahead and create an ignorePath configuration option, so that you can continue using the plugin without these errors.

@joekrump
Copy link
Author

joekrump commented Apr 24, 2023

Thanks @subzero10! Any chance you can make that sample app open source that sample vite app? It might help me out and others in the future.

@subzero10
Copy link
Member

Thanks @subzero10! Any chance you can make that sample app open source that sample vite app? It might help me out and others in the future.

Here it is.
Note that it doesn't even create a static page. I just added the nodejs buildpack on Heroku and made sure that the build step is executed:

-----> Build
       Running build
       
       > vite-js-1053@0.0.1 build
       > vue-tsc && vite build
       
       vite v4.3.1 building for production...
       transforming...
       ✓ 18 modules transformed.
       rendering chunks...
       computing gzip size...
       dist/index.html                  0.45 kB │ gzip:  0.30 kB
       dist/assets/vue-5532db34.svg     0.50 kB │ gzip:  0.31 kB
       dist/assets/index-c322ae43.css   1.30 kB │ gzip:  0.67 kB
       dist/assets/index-2424b2d8.js   54.54 kB │ gzip: 22.02 kB │ map: 565.12 kB
       Successfully uploaded assets/index-2424b2d8.js.map to Honeybadger
       1 sourcemap file(s) successfully uploaded to Honeybadger
       ✓ built in 1.54s
       
-----> Caching build
       - npm cache
       
-----> Pruning devDependencies
       
       up to date, audited 22 packages in 287ms
       
       2 packages are looking for funding
         run `npm fund` for details
       
       found 0 vulnerabilities
       npm notice 
       npm notice New minor version of npm available! 9.5.1 -> 9.6.5
       npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.6.5>
       npm notice Run `npm install -g npm@9.6.5` to update!
       npm notice 
       
-----> Build succeeded!
 !     This app may not specify any way to start a node process
       https://devcenter.heroku.com/articles/nodejs-support#default-web-process-type
-----> Discovering process types
       Procfile declares types     -> (none)
       Default types for buildpack -> web
-----> Compressing...
       Done: 45.7M
-----> Launching...
       Released v4
       https://vite-js-1053.herokuapp.com/ deployed to Heroku

@joekrump
Copy link
Author

joekrump commented Apr 24, 2023

@subzero10 something I just caught regarding the upload errors.

Error output:

 Error: ENOENT: no such file or directory, open '/tmp/build_40af8e8b/public/vite/assets/errors.f2c021a1.js'
 Error: ENOENT: no such file or directory, open '/tmp/build_40af8e8b/public/vite/assets/quill.32217bd5.js'
 Error: ENOENT: no such file or directory, open '/tmp/build_40af8e8b/public/vite/assets/styleguide.11183cda.js'
 Error: ENOENT: no such file or directory, open '/tmp/build_40af8e8b/public/vite/assets/users_search.b38a9299.js'
 Error: ENOENT: no such file or directory, open '/tmp/build_40af8e8b/public/vite/assets/application.31b9c4cb.js'
 Error: ENOENT: no such file or directory, open '/tmp/build_40af8e8b/public/vite/assets/devise.8cbd75cd.js'

Those failed files all seem to correspond to the .sass files I have:

image

I can see that the .js files in the directory had sourcemaps uploaded successfully:

Successfully uploaded assets/quill.b96c7531.js.map to Honeybadger
Successfully uploaded assets/application.1d96dddf.js.map to Honeybadger

In my build output dir I do NOT have any devise.<hash>.js.map file or devise.<hash>.sass.map file or any other like that so I wonder if there is a bug in

export function extractSourcemapDataFromBundle (

Update

When comparing sourcemap.source between what shows up for the .sass file and the .js file I see that the sourcempa entry in the bundle for the sass file has most of its properties set to [] or ''. I think this could be used to determine whether the file is a legit sourcemap or not.

Sourcemap JSON for quill.sassSourcemap JSON for quill.js
{
  version: 3,
  file: 'quill-14abea49.js',
  sources: [],
  sourcesContent: [],
  names: [],
  mappings: ''
}
{
  version: 3,
  file: 'quill-5b69f169.js',
  sources: [
    '../../../node_modules/quill/dist/quill.js',
    '../../../node_modules/quill-paste-smart/dist/quill-paste-smart.js',
    '../../../node_modules/quill-mention/dist/quill.mention.esm.js',
    '../../../app/frontend/entrypoints/quill.js'
  ],
  sourcesContent: [
    '/*!\n' +
      ' * Quill Editor v1.3.7\n' +
      ' * https://quilljs.com/\n' +
      ' * Copyright (c) 2014, Jason Chen\n' +
      ' * Copyright (c) 2013, salesforce.com\n' +
      ' */\n' +
      '(function webpackUniversalModuleDefinition(root, factory) {\n' +
      "\tif(typeof exports === 'object' && typeof module === 'object')\n" +
      '\t\tmodule.exports = factory();\n' +
      "\telse if(typeof define === 'function' && define.amd)\n" +
      '\t\tdefine([], factory);\n' +
      "\telse if(typeof exports === 'object')\n" +
      '\t\texports["Quill"] = factory();\n' +
      '\telse\n' +
      '\t\troot["Quill"] = factory();\n' +
      "})(typeof self !== 'undefined' ? self : this, function() {\n" +
      'return /******/ (function(modules) { // webpackBootstrap\n' +
      'REDACTED FOR BEVITY'
    ],
    names: ['REDACTED FOR BEVITY'],
    mappings: 'REDACTED FOR BEVITY'
  }
}

@joekrump
Copy link
Author

So I'm starting to think that the root cause of the issue might be something to do with what I've outlined in this other issue: ElMassimo/vite_ruby#358

@joekrump
Copy link
Author

joekrump commented Apr 24, 2023

If that's the case, the ignorePath could still be helpful, but what might be even more helpful might be something like an ignoreSourcemapFn or something like that which would allow someone to run some custom check on a sourcemap before determining it's something that should be uploaded or not. What do you think?

In my case, I'd want to have a guard that did something like JSON.parse(sourcemap.source).sourcesContent !== []

@subzero10
Copy link
Member

Hey @joekrump, this is great info! I am working already on adding support for ignorePath and I will also work on your latest findings. However, I am thinking that if sourcesContent is empty, we should probably ignore those files by default.

@subzero10 subzero10 self-assigned this Apr 25, 2023
subzero10 added a commit that referenced this issue Apr 25, 2023
@joekrump
Copy link
Author

joekrump commented Apr 25, 2023

I am thinking that if sourcesContent is empty, we should probably ignore those files by default.

💯 That would be a great default to add and would fix my issue I think.

@subzero10
Copy link
Member

Hey @joekrump, we just released v5.4.0 of the @honeybadger-io/rollup-plugin with an ingorePaths option and an algorithm improvement to skip source maps that have empty sourcesContent.

Thank you for all your help on this!

cc @BethanyBerkowitz

@joekrump
Copy link
Author

🙌🏻 Amazing! Thanks for you work on this @subzero10 and @BethanyBerkowitz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rollup/vite @honeybadger-io/rollup-plugin
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants