Support tsconfig-paths and specify tsconfig*.json

[nhammond101]

Use case description

A change to load tsconfig-paths in supoort of serverless.ts definition files and secondly to optionally pass a different tsconfig file to the serverless command.

Primary use case is to allow the typescript compiler to use different paths when compiling the serverless.ts definition file.

Monorepos such as nx rely heavily on multiple modules referenced via the paths but currently serverless.ts won't be able to utilise them without loading via tsconfig-paths.

By specifying a different tsconfig file (the default is tsconfig.json), specific serverless tsconfig can be used solely for the serverless.ts compilation, allowing separation from the application code and the deployment code.

Proposed solution

1. Update the ts-node register command call with two optional parameters - require and project - require to implement the tsconfig-paths functionality and project for the tsconfig to use
2. Support the passing of --ts-config-path as a parameter into serverless, store in config and pass to the parseConfigurationFile function to help with reading the serverless.ts definition

[pgrzesik]

Thank you @nhammond101 for that proposal. I'm not a TypeScript expect by any means, so it's hard for me to evaluate it - @medikoo what is your opinion here? It looks like it might be touching some of the parts of the codebase that you were working on recently.

[nhammond101]

I've got a PR open, but trying to create tests is proving tricky so any help appreciated, specifically the testing of node modules required.

One thing I'm concerned about is documenting the peer dependencies required for typescript support that I'm proposing. It'll need typescript, ts-node (as it does currently), with my change also needing tsconfig-paths.

[medikoo]

@nhammond101 thanks for raising this issue.

To avoid increase of the maintenance cost on our side we would prefer to no host any further TypeScript specific solutions in context of https://github.com/serverless/serverless/ repository.

Note that support for serverless.ts files was opened via minimal means, without taking any TS related dependency onboard, and otherwise all TypeScript specific discussions and functionalities are hosted externally in context of https://github.com/serverless/typescript/. Ideally also all TS related matters should also be discussed over there, therefore let me move this issue there.

@fredericbarthelet do you understand the issue? Do you think it's possible to address the use case externally, without touching Framework core internals?

[fredericbarthelet]

Thanks @nhammond101 and @CorentinDoue for reporting the issue. It would indeed be of great value to TS developers to be able to take advantages of tsconfig-paths.
Thanks @medikoo and @pgrzesik for transfering it to this repository.

With Serverless current process, handling TS related configuration requires to update the service file reading script located in lib/configuration/read.js (see exemple PR from @nhammond101 serverless/serverless#8931) . Indeed, to my understanding:

• Any sls command triggers the execution of bin/serverless.js, which subsequently calls scripts/serverless.js
• scripts/serverless.js reads the configuration file using lib/configuration/read.js - the code responsible for parsing js, json, yml or ts configuration file.
• scripts/serverless.js then creates a new Serverless instance (from lib/Serverless.js), passing the read configuration as a constructor argument
• scripts/serverless.js then calls run method of the Serverless instance which iterates over various hooks registered by different plugins, passing along the Serverless instance containing all options specified in the service file within the service property

In order to be able to handle configuration parsing script other than lib/configuration/read.js, like for the intend to use language-specific feature, and therefore address this externally to the core framework codebase, it would require the addition of a similar hook mechanism to take place very early in the command handling process.

@medikoo, if you'd like to address this specific TS feature outside of serverless/serverless scope, it would require some heavy lifting, moving the configuration reading process away from scripts/serverless.js and init method of the lib/Serverless.js class and add a specific lifecycle event on every command configuration:read, which plugin could hook into in order to provide service file extension specific script, like to serverless.ts files. This development would also pave the way for other languages to provide their own means of specifying language-specific configuration.

@medikoo @pgrzesik, would you like me to propose such a solution where configuration parsing would actually be part of the command lifecycle or does this seem to be too big of a change to be undertaken at this stage ?

[nhammond101]

A bit more digging:

1. Setting TS_NODE_PROJECT=./tsconfig.app.json as an environment variable, allows me to specify the tsconfig file to use.
2. Adding

"ts-node": {
  "require": ["tsconfig-paths"],
}

to the tsconfig.app.json should work (i've traced the require all the way through to ts-node), but i'm still seeing cannot find module x errors that should be accessible if paths are being followed

[nhammond101]

So minor step forward - changing

"ts-node": {
  "require": ["tsconfig-paths"],
}

to

"ts-node": {
  "require": ["tsconfig-paths/register"],
}

does seem to help. I need to some more digging as the serverless-webpack plugin is throwing an error about a missing baseUrl in the tsconfig compilerOptions

Serverless:  
Serverless: Invoke webpack
Serverless: Invoke webpack:validate
Failed to load /Users/nick/Projects/apps/basic-auth/tsconfig.app.json: Missing baseUrl in compilerOptions
Serverless: Invoke webpack:compile
Serverless: Bundling with Webpack...
tsconfig-paths-webpack-plugin: Found no baseUrl in tsconfig.json, not applying tsconfig-paths-webpack-plugin

[fredericbarthelet]

@nhammond101 you can try your config with TS 4.1. Using a baseUrl is not required anymore: https://devblogs.microsoft.com/typescript/announcing-typescript-4-1/#paths-without-baseurl

[nhammond101]

i'm using 4.1 but dividab/tsconfig-paths#143 is causing the issue.

The workaround is adding delete process.env.TS_NODE_PROJECT; into the webpack.config.js

i'll create a fork of the typescript example and try and recreate the scenario. i think the base example works fine as the default tsconfig.json is being set and not overridden by the TS_NODE_PROJECT environment variable.

[nhammond101]

I'm wondering if using the webpack Environment plugin to unset TS_NODE_PROJECT would work within the webpack config instead

[nhammond101]

I've pushed a example repo - https://github.com/nhammond101/serverless/tree/master/lib/plugins/create/templates/aws-nodejs-typescript

• added a tsconfig.app.json
• updated webpack.config.js to remove the TS_NODE_PROJECT once it's been used
• added a build script for demo purposes

[medikoo]

@medikoo @pgrzesik, would you like me to propose such a solution where configuration parsing would actually be part of the command lifecycle or does this seem to be too big of a change to be undertaken at this stage ?

@fredericbarthelet I think that's a very good call.

Still, as it's about adding a hook at the very early stage of command processing, where we do not have config parsed (so have no means to know what plugins are intended to be loaded), it cannot be really done via plugins.

A solution that comes to my mind is.

1. Introduce support for optional serverless settings namespace in package.json
2. Recognize serverless.configResolver option as set to package.json, which simply may point a path (resolved with Node.js` module resolution rules) to module that exports a function, which when invoked will resolve (if needed asynchronously) a configuration object.

I think given configResolver if setup, should replace logic that resolves both configuration path and configuration content.

Having that users, may take advantage of full TS support, simply by configuring package.json as follows:

{
  "serverless": {
    "configResolver": "@serverless/typescript/config-resolver"
  },
  "dependencies": {
     "@serverless/typescript": "1"
  }
}

(@serverless/typescript/config-resolver may be simplified to @serverless/typescript if that makes sense)

Having that we could also move out current built-in TS resolution support from the Framework (ofc with next major), and then TS users may take advantage of unrestricted TS support which would be maintained in and provided purely in context of @serverless/typescript.

What do you think?

(cc @pgrzesik)

[fredericbarthelet]

@nhammond101 thanks for your awesome work :) ! Do you think you can provide a PR to the template to :

• add dependency to tsconfig-path
• add require option for ts-node in tsconfig.json like you did in https://github.com/nhammond101/serverless/commit/9bb6a86f76877c49333f90d3112eeb6a0dfccd40
• add a use case of serverless.ts actually using one path (@functions refering to src/functions maybe ?) for import

I'm not sure about the requirement for 2 separate tsconfig.json (one for ts-load from serverless/serverless config reader and of for ts-loader from webpack for ts file transpiling) as this might be quite cumbersome for most users. Do you believe this could instead be part of the template documentation readme for most advanced configuration ?

@medikoo noted, would be glad to work on such feature. I'll give it a go and see what comes through :)

[medikoo]

@medikoo noted, would be glad to work on such feature. I'll give it a go and see what comes through :)

That's very welcome!

I've realized that while we support .ts config file, we also should support .ts files in ${file(..)} variables (see serverless/serverless#8071). Technically that is possible via plugins, but separating support for .ts config file and .ts file in ${file to be configured via two different means, doesn't look good. Therefore maybe we should take more future proof and generic approach to extending the Framework.

Idea comes to my mind is to support extensions (core-level extensions, when plugins can be considered as high level functionality extensions) via package.json setting, as already proposed above, but with more generic layout:

{
  "serverless": {
    "extensions": ["@serverless/typescript"]
  }
}

And having that @serverless/typescript may export configResolver and e.g. fileImportExtension configuration through which functionality of ${file.. resolution could be extended

Currently I'm in a process of rewriting from scratch variables resolver, so to reduce implementation cost I'd expect the latter one to work only with this new resolver (let's not invest time at this point to try to adapt it into current implementation)

What do you think?

[pgrzesik]

I think your suggestion makes a lot of sense @medikoo - it would also allow for a more modular approach to the Framework in general I believe. One question I have - what, in your opinion, should be exported by such extension? Should we have a generic API for what might be exported by them?

[medikoo]

One question I have - what, in your opinion, should be exported by such extension?

I think it should support whatever extensions we have use case for.

I'd start with supporting this use case, so simply configResolver, and then probably fileImportExtension. Support for both I believe should be added with separatre PR's (with first one also driving extensions support).

I honestly hope that we will end at that point. Ideally, all that can be addressed via plugins is better if addressed via plugins

[nhammond101]

Do you believe this could instead be part of the template documentation readme for most advanced configuration ?

@fredericbarthelet I think so, I've created a PR and added a small advanced section in the README that could form the basis for advanced configurations.

[medikoo]

@fredericbarthelet @pgrzesik @nhammond101 I've opened issue with implementation spec on Framework side serverless/serverless#9311 This should bring us closer to have that implemented.

Let me know what you think

[ChristopherGillis]

I know a lot has changed in the last year, but I still can't seem to find a solution to this open issue. Does anyone know a workaround?

[severi]

bumped into this issue as well just now and hoping to find some solution to this.

[ChristopherGillis]

@severi Any chance you found a solution?

[severi]

@severi Any chance you found a solution?

I found two options for separating the serverless.ts tsconfig from application code:

1. place the serverless.ts file to a separate ”deployment” folder with its own tsconfig.json file

./api
     tsconfig.json (used for application code)
     src/
        <code here>
     deployment/
        serverless.ts
        tsconfig.json

2. if using serverless-bundle plugin: u can define separate tsconfig-filename that is used during build time (i.e serverless.ts uses tsconfig.json and the application code is build with tsconfig.functions.json for instance)
   https://github.com/AnomalyInnovations/serverless-bundle#typescript

[LBRDan]

Hello there,
just wanted to share a useful workaround about the multiple tsconfig*.json and usage of custom tsconfig paths
As serverless uses ts-node to load serverless.ts (and it should always be compiled as CJS as ts-node could be required at runtime in non-ESM node process), I used the ts-node env variables itself to load a specific tsconfig file.

example dev command:

export SLS_DEBUG=* TS_NODE_PROJECT='./tsconfig.serverless.json' && serverless offline -s dev

I prefer to keep my ts-paths inside a separated tsconfig.paths.json file, then extend both my main tsconfig.json and my tsconfig.serverless.json to let me use them anywhere I need to. This allows me also to have decent support from my code editor giving a generic tsconfig.json file

I'm in the situation where the "ts-node" key inside my tsconfig.json is already used by some custom ts-node based scripts inside my repository

A custom tsconfig.json solves even problems related to an ESM configured project (because ts-node loads the tsconfig.json file from the project root if any, unless told to skip it via options/args/envs), then the serverless.ts gets compiled in ESM and therefore it cannot be "required" (literally require(configPath) here)

I've also tried to author a feature that allowed to conventionally load an arbitrary tsconfig.serverless.json file within the same serverless.ts folder. It works, but I'm pretty sure that serverless folks are not inclined to accept such a feature regards their direction in serverless/serverless#9311

[bogdan-panteleev]

just installing "tsconfig-paths" package in the same level where the tsconfig.app.json lies helped me

[Chrissmith1991]

I'm having an issue with my serverless.ts not picking up my type declarations for my custom configuration.
Im able to get it working by using an export/import but im unable to figure out why its not getting the declarations. Im using a custom path so its getting the right tsconfig.json and tsconfig.paths.json.

so I'm unsure wether there is a related issue with .d.ts files

[ChristopherGillis]

Just confirming, adding

  "ts-node": {
    "require": ["tsconfig-paths/register"]
  },

To my tsconfig.base.json worked for me.

I'm using NX monorepo. serverless.ts is in an "app" that has a tsconfig.json which extends the tsconfig.base.json in the root dir. The base file has the paths in it. Now I can use proper nx library imports in my serverless.ts file.

[LuisAzeved]

I have the exact same use case as @ChristopherGillis and adding the above worked perfectly for me! Thanks

[cjnoname]

We are facing the same issue. Is there any official solution on this?