Client/Server code sharing in Typescript monorepos

With the popularization of node.js since 10+ years ago, it’s becoming more and more usual to find the same programming language in both sides of web applications. Using node.js in the server side allows for re-using a good amount of logic but, how can we effectively share code between client and server in Typescript monorepos? Let’s have a look at different alternatives.

Most modern web applications are characterized by the following two traits when it comes to code repetition:

• The same validation logic is executed on both client and server side. On the client side, to detect validation errors without having to communicate with the server, and on the server side, to protect ourselves from API faulty calls.
• Very similar data models are used in both client and server side. Since we deal with the same application data on both ends, it’s only logic to define symmetric data models.

In node.js monorepos we can easily extract the duplicated code away and require it using relative paths. But the extraction comes with some challenges when Typescript is in the mix. Let’s use a sample Typescript web app to better illustrate those challenges and how to resolve them effectively. We can call it Weather Now.

Weather Now UI

On one hand we have a React UI (i.e. weather-client) that fetches weather data for a given city from a web API and displays it in a simple layout. On the other hand we have an Express web API (i.e. weather-server) with a single endpoint that returns weather data (i.e. /api/weather) and serves the static React UI files in the root url.

And, to fit the purpose of this article, the monorepo sure has some duplicated code: validateCityName, the function which validates that a city name has been provided, and the Validation, Weather and WeatherIcons types.

Sample duplicated code in Weather Now

Now that we have a concrete example, let’s proceed to extract that duplicated code into a common folder (e.g. weather-common) and require it through relative path imports (see base-code-extraction branch).

Extraction of duplicated code to common folder

Here is where the challenges start: Typescript does compile successfully but node fails to start the server.

$ npm run start:server

> start:server
> cd weather-server && npm run start

> start
> node distribution/index.js

internal/modules/cjs/loader.js:905
  throw err;
  ^

Error: Cannot find module '/.../weather-monorepo/weather-server/distribution/index.js'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
    at Function.Module._load (internal/modules/cjs/loader.js:746:27)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:75:12)
    at internal/main/run_main_module.js:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}

In Javascript monorepos, where the code files maintain their relative location at runtime, this type of code extraction works straight away. In Typescript repositories however, where the compiled files generally change their location (i.e. the distribution folder), further configuration is necessary to perform the code extraction, since we are now importing code from outside the Typescript rootDir .

It is possible to keep the compiled files in the same relative location as the Typescript files (i.e. by removing the outDir property in tsconfig.json), but we generally want to compile them to a separate location, so we only include the distribution files when deploying the code to production.

Importing code from outside the scope of a Typescript project forces the Typescript compiler to replicate the file system hierarchy inside the distribution folder, so it can guarantee that the referenced files will be compiled and available from within the distribution folder (note that Typescript never changes the relative paths in compiled files). In Weather Now this causes the index.js server file’s location to change, causing the start npm script to fail.

.
···
├─ weather-common
|     ···
|     utils.ts
└─ weather-server
   ├─ distribution
   |  ├─ weather-common
   |  |     ···
   |  |     utils.js
   |  └─ weather-server
   |        ···
   |        index.js
   └─ source
         ···
         index.ts
      ···
      package.json
      tsconfig.server.json
···
package.json
tsconfig.base.json

Fortunately there are several ways of fixing this problem 👍 Let’s have a look at different solutions.

1. Relative paths fixes

The quick and dirty approach to get things working again. We just need to adapt the code to support the changes in the distribution folder. For Weather Now this means changing the references to compiled files in package.json (i.e. main and start npm script) as well as the paths to certain resources in the code files (see 1-path-fixes branch).

Adaptations for the new distribution file system hierarchy

Changing those paths might not look like a major deal but it adds unnecessary complexity to our code. Moreover, it forces us to introduce additional logic to, for example, run the code without compiling (e.g. using nodemon or ts-node). For Weather Now this means introducing an environment variable and setting the express.static path dynamically.

It works, but we can do better. Why changing the code to match the distribution file system hierarchy when we can change the distribution file system hierarchy to match the code?

2. common npm project

By turning the common folder into an npm project itself, Typescript will expect the compiled files to exist in the referenced paths and it will stop replicating the file system hierarchy inside the distribution folder 💪 Let’s go ahead and add both package.json and tsconfig.json files to the common folder, move the code files into a source subfolder and define an npm script to compile the Typescript code (see 2-npm-project branch).

Extraction of duplicated code to common npm project

Note that as a result of extracting the code to a separate project:

• We are introducing a build dependency. We will always need to compile the common project separately before compiling the projects that depend on it (i.e. update the build npm script on the root package.json).

Modifications to build npm script to compile common project

• Typescript looses access to the types definitions. We will need to generate type declaration files for the common project (i.e. setting the declaration property to true in tsconfig.json) and let Typescript know where to locate those files (i.e. setting the types property in the common package.json).

VSCode types definition resolution through declaration files

Generating type declaration files allows Typescript to compile but will as well cause the IDE symbols navigation to go the declaration files instead of the source code. Unfortunately we can’t fix this issue when using relative path imports.

• Hot reloading will no longer detect changes happening outside the project source folder. We will need to compile the common project in watch mode as well as tweaking nodemon (through nodemon.json) to watch for changes in the common project’s source folder.

Hot reloading necessary changes after extracting duplicated code

Now the compiled common code will remain inside the common project, and the distribution folders of the other projects will remain unchanged 🎉 Much better than 1. Relative paths fixes, but still can be improved.

.
···
├─ weather-common
|  ├─ distribution
|  |     ···
|  |     utils.js
|  └─ source
|        ···
|        utils.ts
|     ···
|     package.json
|     tsconfig.common.json
└─ weather-server
   ├─ distribution
   |     ...
   |     index.js
   └─ source
         ···
         index.ts
      ···
      package.json
      tsconfig.server.json
···
package.json
tsconfig.base.json

3. common npm project + npm local dependencies

After turning the common folder into a separate npm project we can use npm local dependencies (natively supported since npm 2.0) to install the common npm project as a dependency of the other npm projects, get rid of the relative paths in the cross-project import statements* and let npm natively resolve the dependency instead.

All we need to do is install the common project as a local dependency in all the projects that are importing code from it in the following fashion. This will register the local dependency in both package.json and package-lock.json in the dependent projects.

cd weather-client && npm instal --save ../weather-common
cd ../weather-server && npm install --save ../weather-common

Under the hood npm creates symbolic links to the local dependency folder inside the dependent projects’ node_modules folder:

node.js generated symbolic links

We can now go ahead and replace all the relative path imports with regular npm dependency imports (see 3-local-dependency branch, or 3-local-dependency-namespace if you prefer to namespace your packages).

npm local dependency import

Another advantage from this alternative is that it allows us to restore the IDE source code navigation we lost when extracting the duplicated code to a shared folder. We can use the Typescript paths property to tell the IDE where to look for type definitions. For Weather Now this is:

{
  "compilerOptions": {
    "paths": {
      "weather-common": ["./weather-common/source"]
    }
  },
  "extends": "./tsconfig.base.json"
}

VSCode types definition resolution through Typescript paths

Note that using Typescript paths to resolve the relative path imports without registering the local npm dependencies will result in runtime errors since, as previously mentioned, Typescript does not modify the imports path on compilation.

Wrong usage of Typescript paths

4. common npm project + npm workspaces

After turning the common folder into a separate npm project, we can use npm workspaces (introduced in npm 7.0) to simplify the management of the different projects in the monorepo. The main advantage of using npm workspaces is being able to run, directly from the root folder, npm scripts defined in the package.json of any project in the monorepo.

In order to enable workspaces in an existing npm monorepo we need to move the different projects into a specific folder (e.g. projects) and specify that folder through the workspaces property in the root package.json. npm workspaces come with a set of dependencies so we will need to re-install npm dependencies after updating the package.json, which will update package-lock.json as well.

Enabling npm workspaces in package.json

Note that after the re-install, apart from new npm workspaces specific dependencies (e.g. @nodelib/fs.scandir), the package-lock.json file will also contain each of the existing projects in the workspace folder as dependencies. Additionally, npm will create symbolic links in the root node_modules folder.

node.js generated symbolic links, through workspaces

We can now install the common project as a local dependency in all the projects that are importing code from it, this time using workspaces (i.e. the --workspace, or -w, argument).

npm install --save ./projects/weather-common -w weather-client
npm install --save ./projects/weather-common -w weather-server

This will register the local dependency in the root package.json and the dependent project’s package.json. It should update the dependent projects’s package-lock.json as well, but there is currently an npm known issue which causes the following install command to fail, leaving the file unchanged.

$ npm install --save ./projects/weather-common -w weather-client
npm ERR! Cannot set properties of null (setting 'dev')

npm ERR! A complete log of this run can be found in:
npm ERR     /.../.npm/_logs/2022-05-30T05_57_51_827Z-debug.log

Fortunately there is a simple workaround to that issue: re-installing the dependencies of the dependent project (e.g. running an npm install in the project folder). We can after replace all the relative paths in the cross-project import statements* with regular npm dependency imports (see 4-npm-workspaces branch, or 4-npm-workspaces-namespace if you prefer to namespace your packages).

npm local dependency import, through workspaces

Just like when using npm local dependencies, we can as well restore the IDE source code navigation we lost when extracting the duplicated code to a shared folder by using Typescript paths property to tell the IDE where to look for type definitions. For Weather Now this is:

{
  "compilerOptions": {
    "paths": {
      "weather-common": ["./projects/weather-common/source"]
    }
  },
  "extends": "./tsconfig.base.json"
}

VSCode types definition resolution through Typescript paths

One more advantage of having npm workspaces in place is that we can simplify, or even remove, the “shortcut” npm scripts. “shortcut” scripts are a common practice in node.js monorepos: since npm doesn’t detect scripts defined in “nested” folders, the only way to run “nested” scripts from the root folder is to define additional scripts which change the folder and then run the corresponding npm script:

Root folder “shortcut” npm scripts

Since we have landed npm workspaces to our monorepo we can replace the cd instructions in our root package.json npm scripts with the corresponding -w argument, slightly speeding up the scripts execution time:

Simplification of root folder npm scripts through npm workspaces

Note that certain npm scripts must remain intact. We need, for example, to build the projects in a certain order (i.e. weather-common before weather-server), so using the --ws(all workspaces) for the build script could run into compilation errors. Or, trying to use npm install -ws in the postinstall script would result into an endless install loop (the workspaces arguments only apply on install when adding/updating a dependency), so we need to keep the original postinstall script.

* What’s the fuss with cross-project relative paths?

Relative paths in cross-project import statements are not necessarily a problem. The value of replacing them is in making the dependent projects agnostic of the common code file system hierarchy. This way, if we were to make changes in the common npm project, the dependent projects would remain unchanged (as long as the common code API is kept the same).

Not that, while both 3. npm local dependencies and 4. npm workspaces allow for relative paths removal, they introduce a potential minor drawback: not being able to install public packages with the same name as the local dependencies. For example, naming a project common would prevent us from installing the common package from the npm registry.

A convenient way of working around these conflicts is to namespace the packages. In fact, you might have noticed that some popular packages use the @organization/package format on their names: @types/node, @react-native-community/slider,@angular/core, etc. By using that convention is easier to avoid name conflicts, since your local dependencies will have very specific names.

To namespace your packages you need to:

• Move the packages to a @namespace subfolder.
• Update paths to files outside the project folders accordingly (e.g. the extends property in tsconfig.json files referencing the root tsconfig.base.json).
• Update root package.json npm scripts that contain project paths.
• Modify the name property in the package.json file of your npm projects to include the namespace, and update package-lock.json by running npm install.
• In case you are renaming a project (e.g. from weather-common to @weather/common), you will need to rename the project folder and update the corresponding relative import paths as well as potential npm scripts in the root package.json.

See the corresponding example branch in each section for specific implementation details.

Conclusions

Quoting the popular reference to feline taxidermy, “there is more than one way to skin a cat”. The four described approaches will help you removing duplicated code. The more implementation effort, the more advantages you get. Give them a try and decide which one works better for you. Happy coding! ⌨️