typeCheck is too slow

[jbedard]

The typeCheck option makes builds extremely slow because the Program is recreated for each file.

gulp-tslint solves this by forcing the user to pass the Program in as part of the configuration. I think the same could be done here by replacing the typeCheck with a program option? Or maybe allow passing the Program instead of a boolean to the typeCheck? Otherwise it will have to be cached somehow across files.

[sonicoder86]

Is it realistic to have more than one loader linting inside a Webpack configuration?

[jbedard]

Are you suggesting just caching the Program globally?

You can easily have multiple webpack configurations running at the same time, with different lint options etc.

[sonicoder86]

Maybe caching based on the webpack instance is ok.

[jbedard]

If the webpack instance is recreated per webpack config that should work...

[garethflowers]

We're seeing this issue too. Simply enabling the typeCheck in our project has increased build time from 2 minutes to 40 minutes!!

Using tslint-loader 3.5.3, tslint 5.2.0, webpack 2.5.1.

[sanex3339]

same here

[szebrowski]

same here

[simonvizzini]

We also have performance issues with tslint-loader. Linting is much slower compared to the tslint CLI, even without the typeCheck option (with typeCheck it's even slower).

Now I wanted to create my own tslint watcher and so I was playing around with the tslint API and chokidar. I found out during my testing that it's not a good idea to cache the Program instance, because then tslint will always report the same errors, even if they have been fixed afterwards. Creating the Program instance for every lint seems to be necessary to get correct results.

I never used gulp-tslint, but I took a look at the code and I have no idea why caching the Program instance works there... I'm a bit baffled by this.

[simonvizzini]

For reference, this is the example code I tested this with:

const linterConfigFile = path.resolve(watchDir, "tslint.json");
const tsConfigFile = path.resolve(watchDir, "tsconfig.json");

// Tried to store program instance here, but linter gives wrong results then.
// const tsProgram = Linter.createProgram(tsConfigFile);

const lintFile = (filePath) => {
    // necessary to create tsProgram for every lint...
    const tsProgram = Linter.createProgram(tsConfigFile);
    const linter = new Linter({ formatter: "codeFrame" }, tsProgram);

    const config = Configuration.findConfiguration(linterConfigFile, filePath).results;
    const fileContent = tsProgram.getSourceFile(filePath).getFullText();

    linter.lint(filePath, fileContent, config);
    console.log(linter.getResult().output);
};

chokidar.watch(watchDir, chokidarOptions)
    .on("change", (filePath) => {
        console.log("changed:", filePath);
        lintFile(path.resolve(watchDir, filePath));
    })
    .on("error", console.error.bind(console));

[sonicoder86]

@simonvizzini Can you confirm it is also on the pull request to address this issue?

[simonvizzini]

Ah, I'm sorry, I totally missed the pull request. This is strange, I'm sure @mtraynham has tested this, and I guess it worked for him. And if it works in gulp-tslint as well then why not here. Not really sure why it didn't work in my minimal sample. I guess this could be a case of "doesn't work on my machine" (work machine which runs Win10).

I'm at home now and I won't be able to investigate this further until monday, but I'll do a quick test later today on my linux machine at home.

[simonvizzini]

I haven't had much time to investigate this further but I found this interesting note in the gulp-tslint README regarding the typeCheck feature.

// NOTE: Ensure 'Linter.createProgram' is called inside the gulp task else the contents of the files will be cached
// if this tasks is called again (eg. as part of a 'watch' task).

This is exactly what I'm experiencing when I cache the program instance, I get back cached results. And if I understand gulp correctly then the gulp task is executed every time for every file change in watch mode, and thus creating a new Program instance every time. So as I see it it's not possible to cache the Program instance and so I don't think the related pull request will work at all. Or I'm still missing something...

[mtraynham]

@simonvizzini Yeah, that does look like a problem with caching the Program object with only the loader. Although, I assume this is only a problem because you are using Dev Server or some variant of watch mode. That PR may be ok for simple builds.

Alternatively, adding a compiler plugin for this loader which supports events for run & watch-run, we could use those event hooks to recreate the Program object for a "run".

The awesome-typescript-loader does something similar, so it only compiles changed files.