Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add directory watcher for tsserver and tsc #5127

Merged
merged 18 commits into from
Oct 15, 2015
Merged

Add directory watcher for tsserver and tsc #5127

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
0bc5c14
Change fileWatcher in sys for node 4
zhengbli Oct 1, 2015
5daa100
unify the node filewatcher in sys.ts and server.ts
zhengbli Oct 1, 2015
4dcf8c7
bug fixes
zhengbli Oct 1, 2015
98eaeba
temp save
zhengbli Oct 2, 2015
7fa26ad
Redesigned directory watchers
zhengbli Oct 5, 2015
9db53f2
Add directory watcher to tsc
zhengbli Oct 5, 2015
7741ec0
Use fs.watch for all directory watchers and some bug fixes
zhengbli Oct 5, 2015
17f0cce
Update comments
zhengbli Oct 5, 2015
f7e35d5
Incorporating changes from #3780
zhengbli Oct 5, 2015
3b6f283
Merge branch Microsoft/master
zhengbli Oct 5, 2015
8171ded
Lint fixes and test fixes
zhengbli Oct 6, 2015
9ed5b4c
more test fixes
zhengbli Oct 6, 2015
002f0c0
CR feedback
zhengbli Oct 14, 2015
def268c
Fix issues with removing roots
zhengbli Oct 14, 2015
62664fd
Add timer for batch processing directory changes
zhengbli Oct 14, 2015
6013968
Address build errors
zhengbli Oct 14, 2015
c754999
Fix rwcRunner
zhengbli Oct 14, 2015
fcfc25e
Fix lint errors
zhengbli Oct 14, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions src/compiler/commandLineParser.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -389,15 +389,15 @@ namespace ts {
catch (e) {
return { error: createCompilerDiagnostic(Diagnostics.Cannot_read_file_0_Colon_1, fileName, e.message) };
}
return parseConfigFileText(fileName, text);
return parseConfigFileTextToJson(fileName, text);
}

/**
* Parse the text of the tsconfig.json file
* @param fileName The path to the config file
* @param jsonText The text of the config file
*/
export function parseConfigFileText(fileName: string, jsonText: string): { config?: any; error?: Diagnostic } {
export function parseConfigFileTextToJson(fileName: string, jsonText: string): { config?: any; error?: Diagnostic } {
try {
return { config: /\S/.test(jsonText) ? JSON.parse(jsonText) : {} };
}
Expand All @@ -412,7 +412,7 @@ namespace ts {
* @param basePath A root directory to resolve relative path entries in the config
* file to. e.g. outDir
*/
export function parseConfigFile(json: any, host: ParseConfigHost, basePath: string): ParsedCommandLine {
export function parseJsonConfigFileContent(json: any, host: ParseConfigHost, basePath: string): ParsedCommandLine {
let errors: Diagnostic[] = [];

return {
Expand Down
25 changes: 25 additions & 0 deletions src/compiler/core.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -700,6 +700,9 @@ namespace ts {
}

export function getBaseFileName(path: string) {
if (!path) {
return undefined;
}
let i = path.lastIndexOf(directorySeparator);
return i < 0 ? path : path.substring(i + 1);
}
Expand All @@ -723,6 +726,18 @@ namespace ts {
*/
export const supportedExtensions = [".ts", ".tsx", ".d.ts"];

export function isSupportedSourceFileName(fileName: string) {
if (!fileName) { return false; }

let dotIndex = fileName.lastIndexOf(".");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

for(extension of supportedExtensions) {
   if (!fileExtensionIs(fileName, extension)) {
       return false;
   }
}

return true;

if (dotIndex < 0) {
return false;
}

let extension = fileName.slice(dotIndex, fileName.length);
return supportedExtensions.indexOf(extension) >= 0;
}

const extensionsToRemove = [".d.ts", ".ts", ".js", ".tsx", ".jsx"];
export function removeFileExtension(path: string): string {
for (let ext of extensionsToRemove) {
Expand Down Expand Up @@ -817,4 +832,14 @@ namespace ts {
Debug.assert(false, message);
}
}

export function copyListRemovingItem<T>(item: T, list: T[]) {
let copiedList: T[] = [];
for (var i = 0, len = list.length; i < len; i++) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use for-of

if (list[i] != item) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

!==

copiedList.push(list[i]);
}
}
return copiedList;
}
}
147 changes: 131 additions & 16 deletions src/compiler/sys.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ namespace ts {
write(s: string): void;
readFile(path: string, encoding?: string): string;
writeFile(path: string, data: string, writeByteOrderMark?: boolean): void;
watchFile?(path: string, callback: (path: string, removed: boolean) => void): FileWatcher;
watchFile?(path: string, callback: (path: string, removed?: boolean) => void): FileWatcher;
watchDirectory?(path: string, callback: (path: string) => void, recursive?: boolean): FileWatcher;
resolvePath(path: string): string;
fileExists(path: string): boolean;
directoryExists(path: string): boolean;
Expand All @@ -20,6 +21,12 @@ namespace ts {
exit(exitCode?: number): void;
}

interface WatchedFile {
fileName: string;
callback: (fileName: string, removed?: boolean) => void;
mtime: Date;
}

export interface FileWatcher {
close(): void;
}
Expand Down Expand Up @@ -192,6 +199,103 @@ namespace ts {
const _path = require("path");
const _os = require("os");

// average async stat takes about 30 microseconds
// set chunk size to do 30 files in < 1 millisecond
function createWatchedFileSet(interval = 2500, chunkSize = 30) {
let watchedFiles: WatchedFile[] = [];
let nextFileToCheck = 0;
let watchTimer: any;

function getModifiedTime(fileName: string): Date {
return _fs.statSync(fileName).mtime;
}

function poll(checkedIndex: number) {
let watchedFile = watchedFiles[checkedIndex];
if (!watchedFile) {
return;
}

_fs.stat(watchedFile.fileName, (err: any, stats: any) => {
if (err) {
watchedFile.callback(watchedFile.fileName);
}
else if (watchedFile.mtime.getTime() !== stats.mtime.getTime()) {
watchedFile.mtime = getModifiedTime(watchedFile.fileName);
watchedFile.callback(watchedFile.fileName, watchedFile.mtime.getTime() === 0);
}
});
}

// this implementation uses polling and
// stat due to inconsistencies of fs.watch
// and efficiency of stat on modern filesystems
function startWatchTimer() {
watchTimer = setInterval(() => {
let count = 0;
let nextToCheck = nextFileToCheck;
let firstCheck = -1;
while ((count < chunkSize) && (nextToCheck !== firstCheck)) {
poll(nextToCheck);
if (firstCheck < 0) {
firstCheck = nextToCheck;
}
nextToCheck++;
if (nextToCheck === watchedFiles.length) {
nextToCheck = 0;
}
count++;
}
nextFileToCheck = nextToCheck;
}, interval);
}

function addFile(fileName: string, callback: (fileName: string, removed?: boolean) => void): WatchedFile {
let file: WatchedFile = {
fileName,
callback,
mtime: getModifiedTime(fileName)
};

watchedFiles.push(file);
if (watchedFiles.length === 1) {
startWatchTimer();
}
return file;
}

function removeFile(file: WatchedFile) {
watchedFiles = copyListRemovingItem(file, watchedFiles);
}

return {
getModifiedTime: getModifiedTime,
poll: poll,
startWatchTimer: startWatchTimer,
addFile: addFile,
removeFile: removeFile
}
}

// REVIEW: for now this implementation uses polling.
// The advantage of polling is that it works reliably
// on all os and with network mounted files.
// For 90 referenced files, the average time to detect
// changes is 2*msInterval (by default 5 seconds).
// The overhead of this is .04 percent (1/2500) with
// average pause of < 1 millisecond (and max
// pause less than 1.5 milliseconds); question is
// do we anticipate reference sets in the 100s and
// do we care about waiting 10-20 seconds to detect
// changes for large reference sets? If so, do we want
// to increase the chunk size or decrease the interval
// time dynamically to match the large reference set?
let watchedFileSet = createWatchedFileSet();

function isNode4OrLater(): Boolean {
return parseInt(process.version.charAt(1)) >= 4;
}

const platform: string = _os.platform();
// win32\win64 are case insensitive platforms, MacOS (darwin) by default is also case insensitive
const useCaseSensitiveFileNames = platform !== "win32" && platform !== "win64" && platform !== "darwin";
Expand Down Expand Up @@ -284,25 +388,36 @@ namespace ts {
readFile,
writeFile,
watchFile: (fileName, callback) => {
// watchFile polls a file every 250ms, picking up file notifications.
_fs.watchFile(fileName, { persistent: true, interval: 250 }, fileChanged);
// Node 4.0 stablized the `fs.watch` function on Windows which avoids polling
// and is more efficient than `fs.watchFile` (ref: https://github.com/nodejs/node/pull/2649
// and https://github.com/Microsoft/TypeScript/issues/4643), therefore
// if the current node.js version is newer than 4, use `fs.watch` instead.
if (isNode4OrLater()) {
// Note: in node the callback of fs.watch is given only the relative file name as a parameter
return _fs.watch(fileName, (eventName: string, relativeFileName: string) => callback(fileName));
}

let watchedFile = watchedFileSet.addFile(fileName, callback);
return {
close() { _fs.unwatchFile(fileName, fileChanged); }
close: () => watchedFileSet.removeFile(watchedFile)
};

function fileChanged(curr: any, prev: any) {
// mtime.getTime() equals 0 if file was removed
if (curr.mtime.getTime() === 0) {
callback(fileName, /* removed */ true);
return;
}
if (+curr.mtime <= +prev.mtime) {
return;
},
watchDirectory: (path, callback, recursive) => {
// Node 4.0 `fs.watch` function supports the "recursive" option on both OSX and Windows
// (ref: https://github.com/nodejs/node/pull/2649 and https://github.com/Microsoft/TypeScript/issues/4643)
return _fs.watch(
path,
{ persisten: true, recursive: !!recursive },
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

persistent (per fs.watch docs)

(eventName: string, relativeFileName: string) => {
// In watchDirectory we only care about adding and removing files (when event name is
// "rename"); changes made within files are handled by corresponding fileWatchers (when
// event name is "change")
if (eventName === "rename") {
// When deleting a file, the passed baseFileName is null
callback(!relativeFileName ? relativeFileName : normalizePath(ts.combinePaths(path, relativeFileName)));
};
}

callback(fileName, /* removed */ false);
}
);
},
resolvePath: function (path: string): string {
return _path.resolve(path);
Expand Down
Loading