-
Notifications
You must be signed in to change notification settings - Fork 12.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add API for incremental compilation followed by watching changes and updating buildinfo #41611
Comments
My attempt to implement this by providing a createProgram function to createWatchCompilerHost and have that createProgram invoke createIncrementalProgram is not working: prepareIncrementalProgram(
program: BuilderProgramType,
buildInfoFile: string
) {
const diagnostics = [
...program.getConfigFileParsingDiagnostics(),
...program.getSyntacticDiagnostics(),
...program.getOptionsDiagnostics(),
...program.getGlobalDiagnostics(),
...program.getSemanticDiagnostics(), // Get the diagnostics before emit to cache them in the buildInfo file.
];
const writeIfBuildInfo = (
fileName: string,
data: string,
writeByteOrderMark: boolean | undefined
): boolean => {
if (fileName === buildInfoFile) {
info(`Writing ${getRelativeFileName(buildInfoFile)}`);
ts.sys.writeFile(fileName, data, writeByteOrderMark);
return true;
}
return false;
};
/**
* "emit" without a sourcefile will process all changed files, including the buildinfo file
* so we need to write it out if it changed.
* Then we can also tell which files were recompiled and put the data into the cache.
*/
const emitResult = program.emit(
undefined,
(fileName, data, writeByteOrderMark, onError, sourceFiles) => {
if (!writeIfBuildInfo(fileName, data, writeByteOrderMark)) {
if (sourceFiles && sourceFiles.length > 0) {
const relativeSourceFilePath = getRelativeFileName(
sourceFiles[0].fileName
);
if (fileName.match(/\.js$/)) {
info(`Compiling ${relativeSourceFilePath}`);
this.numCompiledFiles++;
this.addJavascriptToCache(relativeSourceFilePath, {
fileName,
source: data,
});
}
if (fileName.match(/\.map$/)) {
this.cache?.addSourceMap(relativeSourceFilePath, data);
}
}
}
}
);
this.writeDiagnostics(diagnostics);
}
createWatcher(directory: string): WatcherInstance {
info(`Creating new Typescript watcher for ${directory}`);
const configPath = ts.findConfigFile(
/*searchPath*/ "./",
ts.sys.fileExists,
"tsconfig.json"
);
if (!configPath) {
throw new Error("Could not find a valid 'tsconfig.json'.");
}
const buildInfoFile = ts.sys.resolvePath(
`${this.cacheRoot}/buildfile.tsbuildinfo`
);
if (!process.env.METEOR_TYPESCRIPT_CACHE_DISABLED) {
this.cache = new CompilerCache(
ts.sys.resolvePath(`${this.cacheRoot}/v1cache`)
);
}
const optionsToExtend: ts.CompilerOptions = {
incremental: true,
tsBuildInfoFile: buildInfoFile,
noEmit: false,
sourceMap: true,
};
const createProgram: ts.CreateProgram<BuilderProgramType> = (
rootNames = [],
options = {},
providedHost,
oldProgram,
configFileParsingDiagnostics,
projectReferences
) => {
const innerCreateProgram: ts.CreateProgram<BuilderProgramType> = (
...args
) => {
return ts.createEmitAndSemanticDiagnosticsBuilderProgram(...args);
};
const standardHost = ts.createIncrementalCompilerHost(options);
const host: ts.CompilerHost = {
...standardHost,
readFile: (fileName) => {
return standardHost.readFile(fileName);
},
};
const program = ts.createIncrementalProgram({
rootNames,
options,
configFileParsingDiagnostics,
projectReferences,
host,
createProgram: innerCreateProgram,
});
return program;
};
const watchHost = ts.createWatchCompilerHost(
configPath,
optionsToExtend,
ts.sys,
createProgram,
(diagnostic) => this.writeDiagnostics([diagnostic]),
(...args) => this.reportWatchStatus(...args),
watchOptionsToExtend
);
watchHost.afterProgramCreate = (program) => {
// The default implementation is to emit files to disk, which we absolutely do not want
this.prepareIncrementalProgram(program, buildInfoFile); // this method calls program.emit
};
const watch = ts.createWatchProgram(watchHost); While the compilation does work, the first compilation does not take advantage of the buildinfo file but recompiles all files, and subsequent compilations (using a cached "watch") generate a new program every time watch.getProgram() is invoked even if no source file has been changed, so it does not seem that watch mode is working. |
I’m closing this now as I believe the issues I see are bugs, not missing features. I will create a standalone minimal example that shows that when part of a watch build, incremental builds don’t work properly (maybe just under some circumstances that I have yet to figure out). |
Any update here? Noticed that you created a repro in a repo, but can't see that you've created an issue with it? |
I filed a separate issue for the bug that made me think this was not possible in the current API #41690 |
Search Terms
incremental watch compiler api
Suggestion
Support incremental + watch compilation in the compiler API
Use Cases
When using the compiler API to build a typescript compiler plugin for a build system that supports "developer mode" (a mode where changes are made incrementally and expected to quickly turn around with transpiled files for changes) it becomes necessary to support both incremental compilation (loading buildinfo from disk at startup and writing changes back) and watch mode (to not rescan the entire source tree for every change since this takes tens of seconds or even minutes).
It does not seem to be possible using the existing compiler API when I study it, and a request to show an example of this has gone unanswered (microsoft/TypeScript-wiki#260).
Examples
This would be used by me in my typescript compiler for meteorjs to keep a watching build instance around for a directory for subsequent compilations:
https://github.com/Meteor-Community-Packages/meteor-typescript-compiler/blob/master/meteor-typescript-compiler.ts
Checklist
My suggestion meets these guidelines:
The text was updated successfully, but these errors were encountered: