Run code once per node server startup

[bhavin7181]

We basically want to initialize few application/site level parameters on the server which can be reused (at server) on each request and updated on set interval. There is a way to do this when the node server starts but as we are using next.js built-in server, it doesn't seem straightforward.

getInitialProps gets invoked for each SSR request in the _app.js. If I want to initialize few configs/parameters at the server startup level, what is the correct way to do it without writing a custom server?

Thank you

[Maczuga]

Hey, did you find any solution for that? I am stuck with the same issue.

[hbel]

Me too.

[Uranbold]

Same here.

[Anwesh43]

Think you need to create a custom server. Fetch the settings in the prepare method and store in a json file and use the configs in getInitialProps.

https://nextjs.org/docs/advanced-features/custom-server

[kostenickj]

anyone have ideas on how to do this WITHOUT a custom server?

[hbel]

I solved this by using some kind of memo function to access the object that needs initialization within the server-side code - the initialization code will only run once, afterwards, the initialized object gets returned. This works, but is not as elegant as it could be. But it's still better than building a custom server. It also suffers from the fact that next.js' recompile on code changes does not offer the ability to reset the state of this code which quickly leads to unwanted behaviour. (this also applies to the situation where a custom server is used. This is a general problem) Which in turn makes me wonder whether next.js really is ready for applications that go beyond simple crud with their backend.

[Shrivallabh]

Even with fast-memo, the memoized function was getting called 7 times. I wonder if it had something to do with concurrency configured for the node instance which got spawned.

[finom]

The only idea I've got is to make a route which is going to be pinged by curl, but that sounds ridiculous. Custom server would really suck there.

[hbel]

I worked around that by completely separating the data access layer from the route handlers and by making sure everything is properly initialized before any data access even occurs. But in retrospective, once again, I stumbled over the fact that next is a little too opinionated for the things my apps are doing.

[pgayvallet]

I worked around that by completely separating the data access layer from the route handlers and by making sure everything is properly initialized before any data access even occurs

Yea, I did the same, but not being able to check the DB connection at startup without being forced to use a custom server just for that seems ridiculous... That forces to perform a request to check if the server is properly configured, which is a pain for CIs or even for deployment.

[dafanasiev]

I think that you can set environment key NODE_OPTIONS with --require ./some_module.js. So, nextjs pass this to nodejs cli and nodejs "start" this module.

[destroyer22719]

Hi there, I've did a simple console.log in the file and it only runs on build, but when I run dev or start it never outputted anything to the console

[Shrivallabh]

I think this question is even more pertinent when we want to make a website that is pre-rendered using export functionality. There should be a way to do this easily.

[khromov]

We have started to use a singleton access pattern to get around this. The resource is not initialized until you call it for the first time. Example with node-postgres adapter:

import { Pool } from 'pg';
import getConfig from '../config';

let pool: Pool|null = null;

export async function query(incomingQuery, params = [], config:any = {}) {
  if (!pool) {
    console.log('🐘 Initializing Postgres connection!');
    pool = new Pool({
      connectionString: getConfig().databaseUrl,
      max: getConfig().databaseClients ?? 10,
    });
  }

  const results = await pool.query(incomingQuery, params);

  return results;
}

export function getPool() {
  return pool;
}

export async function disconnect() {
  if (pool !== null) {
    console.log('😵 Disconnecting from Postgres!');
    return pool.end();
  }

  return new Promise(() => {});
}

To use, simply run this at any place in your application:

import { query, disconnect } from './postgres';
const result = await query('...');
console.log(result);
await disconnect();

[khromov]

For resources that need to be initiated in a single API route, you can use the fact that code outside of the API function export will run one time when the file is imported, so you can initialize resources there, for example:

// pages/api/hello.js

// Code you place here will run only once

export default (req, res) => {
  // Code you place here runs on every request
  // ...
}

[hbel]

I ran into massive problems with singleton usage when using development mode (next dev) because native resouces (e.g. db connections, open files) were not properly being destroyed on reloading after code changes. In production, this works and is exactly what we did, too.

[assetcorp]

The problem with this is that for a task that needs to wait (something that returns a promise), the request might proceed before the promise resolves. So If I wanted to do some database initialization before continuing the request, it would mostly fail in this case.

[meelash]

#11686

[hbel]

This isn't a solution, either. A bunch of hacks that work around next.js don't solve the original Problem that next.js should be able to provide a proper solution in the first place.

The only solution I found that really works consistently by now is providing the actual backend not via next.js, but in a separate nodejs microservice. Not what we intended to do originally, but at least that doesn't break anything in dev.

[FreddyLB]

Is there any limitation that prevents expose functions for startup and cleanup the server?

pages/_startup.js

export default async function startup({ server }) {
    /// Start-up logic, database connections, add middlewares to the server, etc...
   console.log("Starting server...");
}

pages/_cleanup.js

export default function cleanup() {
    /// Clean-up logic
   console.log("Shutting down server...");
}

[alexweininger]

This would be so nice

[alvitoraidhy]

Would love to have something like this. Or something like _hooks.js that would attach exported startup, shutdown, etc listener functions to the corresponding events.

[chuckwu0]

Me too.

[karurosux]

Did anyone found out how to initialize stuff? All the solutions around involve using a custom server or other hacky stuff...

[ConsoleTVs]

It's a shame that we can't deal with low level stuff like initialisation / destruction in a low level abstraction of a http server

[mizozobu]

Not at the startup, but maybe at the first request as a workaround? It's now possible with middleware. Probably the same issue about destorying connections described in #15341 (reply in thread) exists in middleware, too. If you don't care about deinit, this could solve your problem.

pages/_middleware.ts

import type { NextFetchEvent, NextRequest } from 'next/server';

const firstRequestedAt = new Date();

export function middleware(req: NextRequest, ev: NextFetchEvent) {
  console.log(firstRequestedAt.toISOString());
}

This outputs the same ISO string at every request.

2022-02-20T04:50:04.824Z
2022-02-20T04:50:04.824Z
2022-02-20T04:50:04.824Z
2022-02-20T04:50:04.824Z
2022-02-20T04:50:04.824Z

---

However, once you move the logic to a different file to use module cache(which most of you are going for), it does not work as expected.

util.ts

const firstRequestedAt = new Date();
const getFirstRequestedAt = () => firstRequestedAt;

export { getFirstRequestedAt };

pages/_middleware.ts

import type { NextFetchEvent, NextRequest } from 'next/server';
import { getFirstRequestedAt } from 'path to util.ts';

export function middleware(req: NextRequest, ev: NextFetchEvent) {
  console.log(getFirstRequestedAt().toISOString(), 'middleware');
}

pages/some-page.tsx

import { getFirstRequestedAt } from 'path to util.ts';

export async function getStaticProps(context) {
  console.log(getFirstRequestedAt().toISOString(), 'getStaticProps');
  return {
    props: {},
    revalidate: 1,
  };
};

This outputs a different ISO string for middleware and getStaticProps.

2022-02-20T05:15:45.403Z middleware
2022-02-20T05:15:52.716Z getStaticProps
2022-02-20T05:15:45.403Z middleware
2022-02-20T05:15:52.716Z getStaticProps

Correct me if I was wrong, but this must be because middleware and getStaticProps does not share the same module cache since middleware(a.k.a. Edge Function) runs as a separate process (or even a separate network).

---

⚠️ Note: Be careful with the middleware if you are deploying it to Vercel. A middleware(edge function) gets created every so often, so you should not put code to be executed only once outside of the middleware function.

import type { NextFetchEvent, NextRequest } from 'next/server';

// This gets executed at every middleware initialization, not every request but quite often.
const firstRequestedAt = new Date();

export function middleware(req: NextRequest, ev: NextFetchEvent) {
  console.log(firstRequestedAt.toISOString());
}

[majelbstoat]

A specific use-case for this is tracing with open telemetry. this must be initialized before anything else. A full custom-server to support this is overkill, and comes with many downsides. A workaround is to use node --require preload.js, but this has a number of issues:

• the initialization of open telemetry is asychronous and you cannot do a top-level await for it to finish in a CommonJS module.
• scripts included with node --require must be CommonJS.
• As a result, the tracing library is typically not initialized before the first request and the next server reports it has started before tracing is ready.

Even allowing for this, preload.js is not included as part of next's outputStandalone file tracing, which means none of the node_modules require'd in it are included in the standalone bundle. So, we must also use @vercel/nft ourselves to ensure we copy the extra node_modules across.

As others have noted, it would be far, far simpler to have a _startup.ts or _init.ts file that was a well-known part of running next, where all this complexity could be avoided.

[leerob]

Hey folks, wanted to mention this solution. I know it's not perfect, but it should unblock you from running code before the Next.js server starts without needing to eject out of the default server.

node --require ./server-preload.js ./node_modules/.bin/next start
#    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#    Load our preload before running `next start`

Source and credit: https://jake.tl/notes/2021-04-04-nextjs-preload-hack

[dora-gt]

What about async procedure?

[gregg-cbs]

what about if you need context of the next server environment?

[gregg-cbs]

Also by the looks of things next dev runs the imported script twice:
"dev": "node --require ./boot.cjs ./node_modules/next/dist/bin/next dev",

so any database connections/watchers/services would be instantiated/run twice.

[gregg-cbs]

and it wont be possible to use any packages or functions from the project in the boot script because the boot script only allows require and not import.

[dora-gt]

Really important.

[gnikoloff]

nextjs is so nice to work with until the moment when it's not

[alexandros-megas]

Please consider upvoting this RFC #45628

[gregg-cbs]

This is the second most upvoted discussion in the past year, as seen here:
https://github.com/vercel/next.js/discussions?discussions_q=is%3Aopen+sort%3Atop+created%3A%3E%3D2022-04-14

Not sure why its getting no response.

[anthonyalayo]

I think the only thing we can do is get enough votes gathered to be in first place at this point...

[hassanKhademi]

if want run code after server is run
just write code in next.config.js file

//next.config.js
(async function () { 
  if (process.argv.includes('build')) {
    console.log("\n\n$$$$$$$$$$ THIS CODE RUN BEFORE SERVER IS UP (start build) 🏗️👷👷‍♀️🧑‍🏭  $$$$$$$$$$$$\n\n");
  }
  if (process.argv.includes('start')) {
    console.log("\n\n########### THIS CODE RUN AFTER SERVER IS UP (end build) ###########"); 
  } 
  // can use async function 
})();

In this case, can anyone tell me if there is a problem?

@alexandros-megas
@bhavin7181

[GitVex]

I'd also like to know if there could be any problem from using it this way

[jalovatt]

Doesn't work in standalone mode; it evaluates your next.config.js at build time and then serializes it, so you can't actually run code, import anything, etc.

As of 12.something your Next config can be module.exports = (phase) => { ... } and it will let you make things conditional by checking against:

// next/types
export declare const PHASE_EXPORT = "phase-export";
export declare const PHASE_PRODUCTION_BUILD = "phase-production-build";
export declare const PHASE_PRODUCTION_SERVER = "phase-production-server";
export declare const PHASE_DEVELOPMENT_SERVER = "phase-development-server";
export declare const PHASE_TEST = "phase-test";

(Standalone only gets as far as PHASE_PRODUCTION_BUILD, as mentioned above)

[Angeschossen]

None, if you dont want to use Typescript for the part you're calling. Otherwise, it gets a bit more complicated, but possible. I wouldn't use NextJS again, because its pretty inconvenient.

…

On Fri, Apr 14, 2023, 10:07 AM GitVex ***@***.***> wrote: I'd also like to know if there could be any problem from using it this way — Reply to this email directly, view it on GitHub <#15341 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF5QXIRQ3KJT5SHYWHDPLMDXBEAUDANCNFSM4PCWE5GQ> . You are receiving this because you commented.Message ID: <vercel/next. ***@***.***>

[gregg-cbs]

Massively inconvenient. Its almost absurd that this hasnt been brought into existence, even by some sort of patch. It completely f's me over.

[roneymaia]

After a lot of agony to get this working with modules, typescript and running scripts in the same nodejs process. I took advantage of some of the workarounds found here and made this hack that may work for more people just like it did for me.

"next": "^13.3.0"

boot.ts

const bootedServices = {
  example1: false,
  example2: false,
};

const bootHandler = async () => {
  // Example1 service...
  if (!bootedServices.example1) {
    console.log('[pages/api/_boot.ts] => EXAMPLE1');

    bootedServices.example1 = await new Promise<boolean>(resolve => {
      setTimeout(() => {
        resolve(true);
      }, 3000);
    });
  }

  // Example2 service...
  if (!bootedServices.example2) {
    console.log('[pages/api/_boot.ts] => EXAMPLE2');

    bootedServices.example2 = await new Promise<boolean>(resolve => {
      setTimeout(() => {
        resolve(true);
      }, 1000);
    });
  }

  return bootedServices;
};

export default bootHandler;

pages/api/_boot.ts

import type { NextApiRequest, NextApiResponse } from 'next';
import bootHandler from '@/boot';

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse,
) {
  const bootedServices = await bootHandler();

  res.status(200).json({ bootedServices });
}

next.config.js

const {
  PHASE_DEVELOPMENT_SERVER,
  PHASE_PRODUCTION_SERVER,
  PHASE_PRODUCTION_BUILD
} = require('next/constants');

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
};

const bootServices = async () => {
  return fetch('http://localhost:3000/api/_boot')
  .then(async (res) => {
    const resJson = await res.json();

    return JSON.stringify(resJson.bootedServices);
  }).catch(() => false);
}

module.exports = async (phase, { defaultConfig }) => {
  if (process.argv.includes('dev') && phase === PHASE_DEVELOPMENT_SERVER) {
    console.log('[ next.config.js (dev) ]');

    const bootedServices = await bootServices();

    console.log(`[ next.config.js (dev) ] => bootedServices: ${bootedServices}`);
  } else if (process.argv.includes('start') && phase === PHASE_PRODUCTION_SERVER) {
    console.log('[ next.config.js (start) ]');

    // Timeout start
    setTimeout(async () => {
      const bootedServices = await bootServices();
      console.log(`[ next.config.js (start) ] => bootedServices: ${bootedServices}`);
    }, 1000);
  } else if (process.argv.includes('build') && phase === PHASE_PRODUCTION_BUILD) {
    console.log('[ next.config.js (build) ]');

    // Boot into static pages? getStaticProps ?
    // pages/staticpage.tsx
    // import bootHandler from '@/boot';
    // const bootedServices = await bootHandler();
  }

  return nextConfig;
};

Terminal output

next dev

ready - started server on 0.0.0.0:3000, url: http://localhost:3000
info  - Loaded env from C:\...\.env
event - compiled client and server successfully in 651 ms (174 modules)
[ next.config.js (dev) ]
wait  - compiling /api/_boot (client and server)...
event - compiled successfully in 50 ms (44 modules)
[pages/api/_boot.ts] => EXAMPLE1
[pages/api/_boot.ts] => EXAMPLE2
[ next.config.js (dev) ] => bootedServices: {"example1":true,"example2":true}

next start

ready - started server on 0.0.0.0:3000, url: http://localhost:3000
info  - Loaded env from C:\...\.env
[ next.config.js (start) ]
[pages/api/_boot.ts] => EXAMPLE1
[pages/api/_boot.ts] => EXAMPLE2
[ next.config.js (start) ] => bootedServices: {"example1":true,"example2":true}

[Aaqu]

@nikola-lenert which next you are using? 13 or 14

EDIT:
I rebuilt this example for next 14, it required for dev setimeout from PHASE_PRODUCTION_SERVER and change next.config.mjs to next.config.js - seems to work

[nikola-lenert]

@Aaqu I am using Next 14

[Aaqu]

this example works with docker? For me: NO

[nikola-lenert]

I don't run Next via Docker so wouldn't know, maybe someone else can chime in.

[Aaqu]

I already understand, phases (next config function) are launched at next start, but not with node server.js

[anthonyalayo]

I believe...there is now a solution?

Last week, version 13.3.2 made changes for:
Add support for async instrumentation register: #48575

Looking at the history, version 13.2.0 added the initial support:
feat: add instrumentation hook: #46002

Checking out the docs from last week:
https://nextjs.org/docs/pages/building-your-application/optimizing/instrumentation

If you export a function named register from this file, we will call that function whenever a new Next.js server instance is bootstrapped. When your register function is deployed, it will be called on each cold boot (but exactly once in each environment).

Note: This feature is experimental. To use it, you must explicitly opt in by defining experimental.instrumentationHook = true; in your next.config.js.

I haven't tried it myself yet, but this may be what we were looking for.

[Aaqu]

after full day of tests instrumentation not work on build it is normal?

[jviall]

Even when using this approach:

  if (typeof window === "undefined") {
    setTimeout(async () => {
      await fetch(process.env.SITE_URL + "/api/queues/init")
    }, 2000)
  }

I am seeing multiple calls being made to my equivalent /init api route, at least locally. It doesn't cause me much grief if I check for it, but worth noting.

However, when deploying to a Vercel preview, I'm not seeing it getting called at all--is there something about Vercel deployments that need their own special treatment for this experimental feature? I'm able to get my use case working locally, but it's been a non-starter in a deployment. @leerob

@dongryphon @anthonyalayo any updates on what either of your experiences have been?

I'm also curious why the setTimeout is necessary--I see that it doesn't seem to work without it, but I can't reason about why, and I'm uncomfortable with an arbitrary wait being the solution, it doesn't feel reliable.

[jviall]

Seems like my issue stems from using standalone mode, which is logged as an ticket here: #49897

[jviall]

Thought... I'm not sure Vercel deployments respect standalone output? considering it's zero-config... But maybe the mode: 'standalone' setting in my next.config.js has broader impact...

[theDanielJLewis]

I hate having to make an API route handler (and then handle its security) only for running some initialization functions, but at least it works with the instrumentation feature.

Thanks for pointing this out!

[oyatek]

what about simply adding curl https://... call of an API endpoint into npm run start command?

[leerob]

We have a solution for this now! instrumentation

// instrumentation.ts

import { init } from 'package-init'
 
export function register() {
  init()
}

Let me know if this solves your use case.

Related: #45628

[galaxynova1999]

I think it should be A, but apparently it wasn't implemented that way, it's B now

[alecmerdler]

This bug is currently preventing us from using the new instrumentation.ts file to write to our database on startup.

[setoelkahfi]

I'm trying out this and it seems like the register() gets called whenever I call an API endpoint inside the api folder.

[mihilt]

I don't know why but 'next dev' works like 'a' and 'next start' works like 'b'

[gregg-cbs]

probably because next dev makes the requests from the start of the server because it opens up sockets for hot relaoding... so it working like B but it looks like A.

[WillCalderwood]

Any idea if it'd be ok to run operations on an interval from instrumentation.ts?

For example:

export function register() {
  console.log("This should run once");
}

setInterval(function () {
  try {
    console.log("Do something...")
  } catch (error) {
    console.log(error);
  }
}, 1000);

[WillCalderwood]

This doesn't work as Vercel kills it after 10 seconds (Hobby account).

[leerob]

Hey @WillCalderwood – could you share a bit more about what you're hoping to use this for on Vercel?

[WillCalderwood]

I wanted a service to poll the database and dispatch unsent emails rather than them being sent as part of the API due to the SMTP server being slow. It then kills two birds with one stone. Improves the speed of the API response and can try to resend on failure.

[WillCalderwood]

The generic feature request would be a way to schedule serverless functions to perform tasks.

[yaman3bd]

I want to use instrumentation for creating a realtime messaging client using Ably and I want to create the connection on server startup:

export async function register() {
  if (process.env.NEXT_RUNTIME === "nodejs") {
    await import("./lib/ably");
  }
}

but IDK why it runs twice when the server starts, I tried it on both with: npm run dev, npm run build && npm run start

I even tried to use this:

node --require ./server-preload.js ./node_modules/.bin/next start
#    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#    Load our preload before running `next start`

same thing it runs the script more than once IDK why:

it is ok to run the script more than once, but when I subscribe to the realtime channel:

const channel = realtime.channels.get("private:channel");

channel.subscribe(function (message) {
  console.log(message);
});

it subscribes more than once, which is smth I do not need, I only need to subscribe to the channel only once, on server startup.

Is this an expected behavior or is there smth I am missing in the setup? is there anyone experiencing the same issue and found a workaround?

[rechenberger]

Hey everybody. Looks like instrumentation.ts is not called when running server actions. 😭 Is this intentional or a bug?

[rechenberger]

To clarify. I want to do something like this in instrumentation.ts:

globalThis.myConfig = 123

and get the value back in the server action like

async () => {
  'use server'
  console.log(globalThis.myConfig)
}

it only works sometimes (when it happens to be the same lambda i guess) on vercel

[rechenberger]

update: writing and reading globalThis.myConfig actually seems to work
just having a let variable does not 🤔

type Config = {
  someStuff: number
}

let localConfig: Config

// Called in instrumentation.ts
export const setConfig = (config: Config) => {
  ;(globalThis as any).myConfig = config
  localConfig = config
}

// Called in ServerAction
export const getConfigGlobal = () => {
  return (globalThis as any).myConfig // ✅ works
}
export const getConfigLocal = () => {
  return localConfig // ❌ does not work
}

[Artsu]

Noticed the same issue. I have a shared rest client that I want to initialize before use but doing so in instrumentation.ts doesn't work since I'm using let singleton in the client. Can't use globalThis either since the other application using the client isn't next app

[birdzai]

instrumentation.ts is run only once on server startup as part of server initialization as far as I’m aware.

…

On Tue, Feb 27, 2024 at 10:10 AM Tristan Rechenberger < ***@***.***> wrote: Hey everybody. Looks like instrumentation.ts is not called when running server actions. 😭 Is this intentional or a bug? — Reply to this email directly, view it on GitHub <#15341 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEIG6K5UNBYOXHHJGCTKSO3YVYHPFAVCNFSM4PCWE5G2U5DIOJSWCZC7NNSXTOKENFZWG5LTONUW63SDN5WW2ZLOOQ5TQNRQHAZDCMI> . You are receiving this because you commented.Message ID: <vercel/next. ***@***.***>

[canadaduane]

It is run once per runtime (nodejs, edge) and there is an additional bug where it us run again if middleware is defined: #51450 (comment)

[oyatek]

We use instrumentation.ts successfully in our project, but with some limitations:

1. The environment (we use BlitzJS framework) is not fully initialized when instrumentation.ts gets called. So we can't do requests to Prisma, etc. Instead we have to do a call to our own API endpoint via axios. This is important to keep in mind.
2. In dev mode, the instrumentation.ts gets called multiple times. In production - 1 time.