Jest fails to load jest.config.ts in a ESM project using ts-node 10

[felipeplets]

🐛 Bug Report

In a project using TypeScript, Jest and setup as ESM (the output of the transpiler is ESM so Node will run ESM instead of CJS) JEST is failing with ts-node 10 but works with ts-node 9.

Error:

Error: Jest: Failed to parse the TypeScript config file /projects/ts-jest-ts-node-10/jest.config.ts
  Error: Must use import to load ES Module: /projects/ts-jest-ts-node-10/jest.config.ts
require() of ES modules is not supported.
require() of /projects/ts-jest-ts-node-10/jest.config.ts from /projects/ts-jest-ts-node-10/node_modules/ts-node/dist/index.js is an ES module file as it is a .ts file whose nearest parent package.json contains "type": "module" which defines all .ts files in that package scope as ES modules.
Instead change the requiring code to use import(), or remove "type": "module" from /projects/ts-jest-ts-node-10/package.json.

    at readConfigFileAndSetRootDir (/projects/ts-jest-ts-node-10/node_modules/jest-config/build/readConfigFileAndSetRootDir.js:118:13)
    at readConfig (/projects/ts-jest-ts-node-10/node_modules/jest-config/build/index.js:216:18)
    at readConfigs (/projects/ts-jest-ts-node-10/node_modules/jest-config/build/index.js:405:26)
    at runCLI (/projects/ts-jest-ts-node-10/node_modules/@jest/core/build/cli/index.js:220:59)
    at Object.run (/projects/ts-jest-ts-node-10/node_modules/jest-cli/build/cli/index.js:163:37)

error Command failed with exit code 1

To Reproduce

Steps to reproduce the behavior:

• Create a new folder
• Run npm init -y
• Run npm i -D ts-node typescript jest
• Add "type": "module" to your package.json
• Create a jest.config.ts file with the content below:

import type { Config } from '@jest/types';

const config: Config.InitialOptions = {
    verbose: true
};

export default config;

• Run npx jest

Expected behavior

The config file should be loaded.

Link to repl or repo (highly encouraged)

I have reproduced the issue in a StackBlitz:

• ts-node 9: https://stackblitz.com/edit/ts-jest-ts-node-9
• ts-node 10: https://stackblitz.com/edit/ts-jest-ts-node-10

In order to run the code please run npm test in the console and the error above will be raised.

Both projects are identical aside from the ts-node version.

Keep in mind that this project is using StackBlitz WebContainers and Turbo package manager (which is not NPM, though it have an alias) if you want to tweak my example but not go to learn about specifics about Turbo or WebContainers it may be better to just clone the repo.

envinfo

This is my local environment:

  System:
    OS: macOS 11.3.1
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  Binaries:
    Node: 16.2.0 - ~/.nvm/versions/node/v16.2.0/bin/node
    Yarn: 1.22.10 - ~/.nvm/versions/node/v16.2.0/bin/yarn
    npm: 7.14.0 - ~/.nvm/versions/node/v16.2.0/bin/npm
  npmPackages:
    jest: ^27.0.1 => 27.0.1 

[cspotcode]

Hi, I'm the ts-node maintainer and someone pointed this out on our issue tracker as well. I can explain what's happening here, and I want to share my proposed solution to see if the jest team agrees.

In node, if you try to require() a .js file in an ESM package (package.json "type": "module") node will throw ERR_REQUIRE_ESM Must use import to load ES Module

ts-node 10 aligns with node's behavior, so if you try to require() a .ts file in an ESM package, you'll get the same error, because .ts corresponds to .js. (Release notes mention this under ERR_REQUIRE_ESM)

I see that when jest installs ts-node, it overrides compilerOptions.module = commonjs. I believe that jest would like to have a mode for ts-node that forces commonjs, so that your jest.config.ts is executed as commonjs, ignoring node's ESM behavior.

This could be exposed as an API flag, used like this:

registerer = require('ts-node').register({
  compilerOptions: {
    module: 'CommonJS',
    allowCommonjsInEsmPackage: true // <-- name of this option TBD; this is an example.  Name could be allowLoadEsmAsCommonjs or allowRequireEsm or something like that
  },
});

https://github.com/facebook/jest/blob/00888027257e5a751ffb7002805248b1fc758681/packages/jest-config/src/readConfigFileAndSetRootDir.ts#L84-L91

If it were possible to programmatically install --loaders, I would suggest that we allow jest.config.ts to be executed as ESM, but I don't think that would be ergonomic.

Do you think it's important to support when someone is using --loader ts-node/esm? I don't think so, since it's only for the config file. But I'm not sure if projects commonly import other ESM files from their jest.config.*

[cspotcode]

Another option is a config that says "treat files matching these globs as .cjs"

Something like "moduleTypes": {"jest.config.ts": "cjs"} or "moduleTypes": {".": "cjs"}

This might be more intuitive, since it maps cleanly to a node concept. We will behave as if those .ts files compile into .cjs so that node runs them like CommonJS.

[karlhorky]

Workaround for anyone running into this with ESM + TypeScript + Jest v27 (until this issue is addressed):

1. Change jest.config.ts to jest.config.cjs
2. Rewrite the file contents to CommonJS + JSDoc

[cspotcode]

If anyone from the jest team has a minute to offer some feedback on the solution proposed in #11453 (comment) and #11453 (comment), I would greatly appreciate it.

See also: TypeStrong/ts-node#1342 tracking this feature on the ts-node side.

[SimenB]

I wonder if we should just spawn ts-node, somehow to load the config, execute the exported function (if any), then return that to the main process rather than using the programmatic API.

A flag allowing us to truly force CJS is all well and good until the user imports an ESM native package. I assume we cannot use import() like we do for ESM for TS as we'll need the loader, right?

[cspotcode]

Yeah, that works. You are correct that import(pathToTsFile) fails without the loader, since node's ESM resolver does not know how to classify the .ts file extension.

I think it depends on philosophy: in the general sense, I want ts-node to be user-configurable so that tools such as gulp, webpack, and jest can support it without too much code, and users can set the necessary flags in their tsconfig.json. For those tools, require('ts-node').register() will pull all ts-node options from tsconfig, so users have full control and the CLI tool doesn't need to configure anything. We can put docs and sample tsconfigs on our website so users will know how to setup tsconfig so that ts-node uses CJS for their config files. (Gulpfile.ts, webpack.config.ts, etc)

On the other hand, if jest is already overriding module: commonjs then spawning a subprocess fits jest's philosophy here, since jest wants to control lifecycle. And it doesn't preclude ts-node from adding the flag I proposed to support webpack, gulp, etc. Jest is an interesting case because it wants to control the bootstrapping of ts-node but not the version of ts-node being used.

When jest loads the config, is that happening in the very first CLI process? Or has jest already spawned a subprocess at that point?

This might technically be possible in a worker_thread, too? I think worker_threads let you add --loader but I'm not sure.

[SimenB]

This is essentially the first thing that happens yeah. I'm happy to keep using the programmatic API, I'm just afraid as mentioned that it's just pushing the issue in front of us (like ESM source JS, which will break if loaded via some transpiled require). After evaluating the config file (including potentially calling some exported function), everything in the exported config must be serializable, so I thought just loading outside of the main process should be fine since we can return a JSON.stringify from the other process. Will impact startup time of Jest, but if you've written your config in TS you've pretty much already gone down that route and it won't impact anybody else

[cspotcode]

Yeah, it won't impact startup much if users are smart and enable transpileOnly and if jest keeps the amount of code executing in the child process to a minimum.

To get a sense for timing:

ts-node --transpile-only -e 'console.log("hello world" as const)'

[SimenB]

We can probably pass --transpileOnly ourselves if we're spawning the binary

[cspotcode]

For --loader, you'll need to do something like this:

TS_NODE_TRANSPILE_ONLY=true node --loader ts-node/esm ...
# or
node --loader ts-node/esm-transpile-only ...

[SimenB]

Semi-related: microsoft/TypeScript#44501. How that shakes out might affect how ts-node does things and also how we should deal with it in Jest?