Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

node: add ReadStream and createReadStream #1435

Merged
merged 51 commits into from
Mar 24, 2022
Merged
Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
08e112c
chore: mock more node/ modules for vite
bartlomieju Oct 19, 2021
85b4008
more polyfills, debug log
bartlomieju Oct 20, 2021
d56a9bd
more polyfills2
bartlomieju Oct 20, 2021
f74ef8d
binary encoding in child_process
lucacasonato Oct 20, 2021
0aa6403
fix createReadableFromReader
lucacasonato Oct 20, 2021
bcc3ac5
some fixes
lucacasonato Oct 20, 2021
9140f1c
dont encode uint8array...
lucacasonato Oct 20, 2021
07b39a0
more mocks
lucacasonato Oct 20, 2021
12b87da
Merge branch 'main' into modules_for_vite
bartlomieju Oct 20, 2021
2bcd50c
readline, some HTTP server methods
bartlomieju Oct 21, 2021
944bfc0
Merge branch 'main' into modules_for_vite
bartlomieju Oct 21, 2021
c131279
Merge branch 'main' into modules_for_vite
bartlomieju Oct 21, 2021
66fa566
ServerResponse::hasHeader()
AaronO Oct 21, 2021
74ee2f1
http fixes
AaronO Oct 21, 2021
62a59b1
basic createReadStream
AaronO Oct 21, 2021
08b21dc
fix(node/http): no body responses
AaronO Oct 21, 2021
0c34958
fix(node/http): allow setting ServerResponse.statusCode
AaronO Oct 21, 2021
f5d6fe5
handle querystring.parse(undefined)
AaronO Oct 21, 2021
2e10f59
mock error
bartlomieju Oct 22, 2021
74f9d50
mock URL search params
bartlomieju Oct 22, 2021
3ee05d8
temp
bartlomieju Oct 23, 2021
6d56f72
Merge branch 'main' into modules_for_vite
bartlomieju Oct 27, 2021
ac03388
Merge branch 'main' into modules_for_vite
bartlomieju Oct 28, 2021
45f8b5b
remove _readline
bartlomieju Oct 28, 2021
910806c
fix merge conflict
bartlomieju Oct 28, 2021
8f83f12
polyfill process.execArgv
bartlomieju Oct 28, 2021
7bad583
Merge branch 'main' into modules_for_vite
bartlomieju Oct 29, 2021
581c33a
polyfill fork
bartlomieju Oct 29, 2021
17e6e56
Merge branch 'main' into pr/1435
AaronO Oct 29, 2021
2ac479a
mock more apis for eslint and parcel
bartlomieju Oct 31, 2021
90d20ed
add more Module APIs to polyfill
bartlomieju Oct 31, 2021
993a52e
Merge branch 'main' into modules_for_vite
bartlomieju Oct 31, 2021
341cf58
polyfill process.arch
bartlomieju Oct 31, 2021
7ad9c15
Merge branch 'main' into modules_for_vite
bartlomieju Nov 1, 2021
2774817
Merge branch 'main' into modules_for_vite
bartlomieju Nov 2, 2021
a033acb
Merge branch 'main' into modules_for_vite
bartlomieju Nov 2, 2021
7d39654
uncomment
bartlomieju Nov 2, 2021
aad04a0
Add process.memoryUsage()
AaronO Nov 2, 2021
fbcaa5e
revert changes to fs APIs
bartlomieju Nov 9, 2021
891350b
Merge branch 'main' into modules_for_vite
bartlomieju Nov 10, 2021
f7dd0f0
remove double memoryUsage
bartlomieju Nov 10, 2021
3602d4b
Merge branch 'main' into modules_for_vite
bartlomieju Nov 17, 2021
540052d
Merge branch 'main' into modules_for_vite
bartlomieju Dec 1, 2021
c69ef30
Merge branch 'main' into modules_for_vite
bartlomieju Feb 3, 2022
e3ff5af
fix ulr
bartlomieju Feb 3, 2022
e5716a7
Revert changes
bartlomieju Mar 23, 2022
5b55cd8
Merge branch 'main' into modules_for_vite
bartlomieju Mar 23, 2022
85ca230
add readstream
bartlomieju Mar 23, 2022
9dc2806
Merge branch 'main' into modules_for_vite
kt3k Mar 23, 2022
d8f51c3
Merge branch 'main' into modules_for_vite
kt3k Mar 24, 2022
01e9d1e
fix: lint and type errors
kt3k Mar 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion node/_fs/_fs_access.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,6 @@ export function access(
// TODO(bartlomieju) 'path' can also be a Buffer. Neither of these polyfills
// is available yet. See https://github.com/denoland/deno/issues/3403
export function accessSync(_path: string | URL, _mode?: number): void {
notImplemented("Not yet available");
// TODO(lucacasonato): this is not the right syscall, but it shoudl work for now sorta.
Deno.lstatSync(_path);
}
10 changes: 9 additions & 1 deletion node/_fs/_fs_stat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,14 @@ export function statSync(
path: string | URL,
options: statOptions = { bigint: false },
): Stats | BigIntStats {
const origin = Deno.statSync(path);
let origin;
try {
origin = Deno.statSync(path);
} catch (e) {
if (e instanceof Deno.errors.NotFound) {
e.code = "ENOENT";
}
throw e;
}
return CFISBIS(origin, options.bigint);
}
48 changes: 48 additions & 0 deletions node/_fs/_fs_streams.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
// Copyright 2018-2021 the Deno authors. All rights reserved. MIT license.
import { notImplemented } from "../_utils.ts";
import { fromFileUrl } from "../path.ts";
import { Buffer } from "../buffer.ts";
import { Readable as NodeReadable } from "../stream.ts";

class ReadStream extends NodeReadable {
public path: string;

constructor(path: string | URL, opts: Record<string, unknown>) {
path = path instanceof URL ? fromFileUrl(path) : path;
const hasBadOptions = opts && (
opts.fd || opts.start || opts.end || opts.fs
);
if (hasBadOptions) {
notImplemented();
}
const file = Deno.openSync(path, { read: true });
const buffer = new Uint8Array(16 * 1024);
super({
autoDestroy: true,
emitClose: true,
objectMode: false,
read: async function (_size) {
try {
const n = await file.read(buffer);
this.push(n ? Buffer.from(buffer.slice(0, n)) : null);
} catch (err) {
this.destroy(err as Error);
}
},
destroy: (err, cb) => {
try {
file.close();
} catch {}
cb(err);
},
});
this.path = path;
}
}

export function createReadStream(
path: string | URL,
options: ReadStreamOptions,
): ReadStream {
return new ReadStream(path, options);
}
9 changes: 8 additions & 1 deletion node/_fs/_fs_unlink.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,12 @@ export function unlink(path: string | URL, callback: (err?: Error) => void) {
}

export function unlinkSync(path: string | URL) {
Deno.removeSync(path);
try {
Deno.removeSync(path);
} catch (e) {
if (e instanceof Deno.errors.NotFound) {
e.code = "ENOENT";
}
throw e;
}
bartlomieju marked this conversation as resolved.
Show resolved Hide resolved
}
20 changes: 17 additions & 3 deletions node/child_process.ts
Original file line number Diff line number Diff line change
Expand Up @@ -168,11 +168,11 @@ export class ChildProcess extends EventEmitter {
}

ref(): void {
notImplemented("ChildProcess.ref()");
// notImplemented("ChildProcess.ref()");
}

unref(): void {
notImplemented("ChildProcess.unref()");
// notImplemented("ChildProcess.unref()");
}

private async _waitForChildStreamsToClose(): Promise<void> {
Expand Down Expand Up @@ -318,7 +318,21 @@ export function spawn(
return new ChildProcess(command, args, options);
}

export default { spawn };
export function fork(
modulePath: string,
argsOrOptions?: string[] | SpawnOptions,
maybeOptions?: SpawnOptions,
): ChildProcess {
const args = Array.isArray(argsOrOptions) ? argsOrOptions : [];
const options = !Array.isArray(argsOrOptions) && argsOrOptions != null
? argsOrOptions
: maybeOptions;
args.push(modulePath);
console.log("fork", Deno.execPath, args, options);
return new ChildProcess(Deno.execPath(), args, options);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wrong, it should actually imply deno run --compat --unstable --allow-read

}

export default { spawn, fork };

function ensureClosed(closer: Deno.Closer): void {
try {
Expand Down
10 changes: 9 additions & 1 deletion node/fs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import { realpath, realpathSync } from "./_fs/_fs_realpath.ts";
import { rename, renameSync } from "./_fs/_fs_rename.ts";
import { rmdir, rmdirSync } from "./_fs/_fs_rmdir.ts";
import { stat, statSync } from "./_fs/_fs_stat.ts";
import { createReadStream } from "./_fs/_fs_streams.ts";
import { symlink, symlinkSync } from "./_fs/_fs_symlink.ts";
import { truncate, truncateSync } from "./_fs/_fs_truncate.ts";
import { unlink, unlinkSync } from "./_fs/_fs_unlink.ts";
Expand All @@ -35,6 +36,11 @@ import { writeFile, writeFileSync } from "./_fs/_fs_writeFile.ts";

import * as promises from "./fs/promises.ts";

realpath.native = realpath;
realpathSync.native = realpathSync;
function read() {
}

export default {
access,
accessSync,
Expand All @@ -49,6 +55,7 @@ export default {
constants,
copyFile,
copyFileSync,
createReadStream,
Dir,
Dirent,
exists,
Expand All @@ -74,6 +81,7 @@ export default {
open,
openSync,
promises,
read,
readdir,
readdirSync,
readFile,
Expand Down Expand Up @@ -141,6 +149,7 @@ export {
open,
openSync,
promises,
read,
readdir,
readdirSync,
readFile,
Expand All @@ -162,7 +171,6 @@ export {
unlink,
unlinkSync,
utimes,
utimesSync,
watch,
watchFile,
writeFile,
Expand Down
2 changes: 1 addition & 1 deletion node/internal_binding/handle_wrap.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ export class HandleWrap extends AsyncWrap {
}

unref() {
notImplemented();
// notImplemented();
}

// deno-lint-ignore no-explicit-any
Expand Down
4 changes: 4 additions & 0 deletions node/module_all.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ export default {
fs,
"fs/promises": fsPromises,
http,
https: {},
"internal/readline/utils": internalReadlineUtils,
net,
os,
Expand All @@ -69,8 +70,11 @@ export default {
timers,
"timers/promises": timersPromises,
tty,
tls: {},
url,
"worker_threads": {},
util,
v8: {},
vm,
zlib: {},
} as Record<string, unknown>;
3 changes: 3 additions & 0 deletions node/process.ts
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,9 @@ Object.defineProperty(stdin, "isTTY", {
return Deno.isatty(Deno.stdin.rid);
},
});
stdin.setRawMode = () => {
console.log("called setRawMode");
};

/** https://nodejs.org/api/process.html#process_process_stdout */
export const stdout = createWritableStdioStream(Deno.stdout);
Expand Down
Loading