Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor!: better cross runtime support #97

Merged
merged 89 commits into from
Jul 20, 2022
Merged
Show file tree
Hide file tree
Changes from 75 commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
0569933
refactor: remove dependancies
Wykerd Jul 11, 2022
34cb9da
refactor!: commonjs to es6
Wykerd Jul 11, 2022
1e7a6f0
refactor!: NToken and Signature TS files
Wykerd Jul 11, 2022
6f4d96f
feat: cross platform cache (WIP)
Wykerd Jul 12, 2022
f622448
feat: EventEmitter polyfill
Wykerd Jul 12, 2022
29fe937
refactor: remove events
Wykerd Jul 12, 2022
5035172
feat: HTTPClient based on Fetch API (WIP)
Wykerd Jul 12, 2022
27965ec
refactor!: parsers refactor (WIP)
Wykerd Jul 12, 2022
3726359
refactor!: parsers refactor (WIP)
Wykerd Jul 12, 2022
dd207df
refactor!: parser refactor
Wykerd Jul 12, 2022
37b8f70
fix: some missed parsers while refactoring
Wykerd Jul 12, 2022
189f152
fix: better type inferance for parseResponse
Wykerd Jul 12, 2022
ac83bad
feat(TS): typesafe YTNode casts
Wykerd Jul 13, 2022
53515fb
feat: more type safety in YTNode and Parser
Wykerd Jul 13, 2022
5a2e465
refactor: VideoInfo download with fetch & TS (WIP)
Wykerd Jul 13, 2022
8fb1924
fix: LiveChat in VideoInfo
Wykerd Jul 13, 2022
76d19b4
refactor!: more typesafety in parser
Wykerd Jul 13, 2022
5c49385
refactor!: VideoInfo almost completed
Wykerd Jul 13, 2022
8f873a5
refactor!: player and session refactors
Wykerd Jul 13, 2022
0392751
refactor!: move auth logic to Session (WIP)
Wykerd Jul 13, 2022
23258dc
refactor: TS port for Actions and Innertube
Wykerd Jul 13, 2022
fc1a403
refactor: NavigationEndpoint TS
Wykerd Jul 13, 2022
c9db56f
refactor!: VideoInfo compiles without errors
Wykerd Jul 13, 2022
e265305
chore: delete old player
Wykerd Jul 13, 2022
e1cae96
fix: import errors
Wykerd Jul 13, 2022
882f471
fix: Utils import fixes
Wykerd Jul 13, 2022
093aebc
fix: several runtime errors
Wykerd Jul 13, 2022
42577ae
fix: video streaming
Wykerd Jul 14, 2022
a851516
chore: remove console.log debugging
Wykerd Jul 14, 2022
98d318a
chore: remove old unused dependencies
Wykerd Jul 14, 2022
59be455
fix: typescript errors
Wykerd Jul 14, 2022
297c856
refactor: TS feed
Wykerd Jul 14, 2022
8cb28ad
chore: delete old Feed
Wykerd Jul 14, 2022
40cb10a
refactor: move streamToIterable into Utils
Wykerd Jul 14, 2022
558027b
refactor: AccountManager TS
Wykerd Jul 14, 2022
8586626
refactor: FilterableFeed to TS
Wykerd Jul 14, 2022
1d60d72
refactor: InteractionManager to TS
Wykerd Jul 14, 2022
061903d
refactor: PlaylistManager to TS
Wykerd Jul 14, 2022
ba9e349
refactor: TabbedFeed to TS
Wykerd Jul 14, 2022
29046f7
refactor: Music to TS (WIP)
Wykerd Jul 14, 2022
3f04c3f
fix: getting the tests to pass (6/12)
Wykerd Jul 14, 2022
50d6ee8
fix: tests (7/12)
Wykerd Jul 14, 2022
4a11ea8
fix: download tests (8/12)
Wykerd Jul 14, 2022
21a37ad
fix: tests (9/12)
Wykerd Jul 14, 2022
b0b4117
feat: key based type validation for parsers
Wykerd Jul 14, 2022
e429471
fix: comments tests pass (10/12)
Wykerd Jul 14, 2022
4b7aa96
refactor: type safety checks removing @ts-ignore
Wykerd Jul 14, 2022
21b87a3
fix: playlist tests pass (11/12)
Wykerd Jul 14, 2022
c7240db
fix: all tests pass for node 🎉
Wykerd Jul 14, 2022
4741b3e
build: working Deno bundle
Wykerd Jul 14, 2022
e888cb5
docs: update deno example to download video
Wykerd Jul 14, 2022
e4b62b2
refactor: MusicResponsiveListItem to TS
Wykerd Jul 14, 2022
5c532ce
docs: TSDoc for Parser helpers
Wykerd Jul 14, 2022
10a96ad
docs: Parser documentation for TS
Wykerd Jul 14, 2022
f597f57
docs: add note about parseItem and parseArray
Wykerd Jul 14, 2022
0f64611
test: remove browser tests since they're identical
Wykerd Jul 14, 2022
d051511
feat: browser support and proxy example
Wykerd Jul 15, 2022
4863c1c
fix: merge conflicts
Wykerd Jul 15, 2022
b3b5f21
fix: PlaylistManager TS after merge
Wykerd Jul 15, 2022
9701b92
feat: in-browser video streaming
Wykerd Jul 15, 2022
f39e73f
refactor: cleanup the Dash example
Wykerd Jul 15, 2022
6507ad2
feat: allow custom fetch implementations
Wykerd Jul 16, 2022
0bafe05
feat: fetch debugger
Wykerd Jul 16, 2022
51bf764
fix: OAuth login
Wykerd Jul 16, 2022
cab9b6c
refactor: remove file extensions from imports
Wykerd Jul 16, 2022
559eacd
refactor: build scripts
Wykerd Jul 16, 2022
9124854
fix: CustomEvent on node
Wykerd Jul 17, 2022
f6ff7c2
fix: LiveChat
Wykerd Jul 17, 2022
0a37abb
fix: linting
Wykerd Jul 17, 2022
2eee3e1
fix: liniting in build-parser-json
Wykerd Jul 17, 2022
679b827
chore: update test workflow
Wykerd Jul 17, 2022
a8a4d7f
fix: NToken errors after lint fixes
Wykerd Jul 17, 2022
76b9927
fix: codacy complaints
Wykerd Jul 17, 2022
77d985d
docs: update to reflect changes
Wykerd Jul 17, 2022
1b2570f
refactor: cleanup imports/exports
Wykerd Jul 17, 2022
79c6ca8
fix: browser example
Wykerd Jul 17, 2022
55b983f
fix: cache on node
Wykerd Jul 17, 2022
d51271b
fix: stupid mistake
Wykerd Jul 17, 2022
fa7d414
refactor: Session#signIn to wait untill success
Wykerd Jul 17, 2022
682261a
refactor: freeze Constants
Wykerd Jul 17, 2022
9682676
refactor: cleanup HTTPClient Request
Wykerd Jul 17, 2022
6b8249b
refactor: debugFetch readability
Wykerd Jul 17, 2022
f4be045
chore: lint
Wykerd Jul 17, 2022
1873f5a
refactor: replace jsdoc with tsdoc eslint plugin
Wykerd Jul 18, 2022
14be811
fix: bunch of liniting warnings
Wykerd Jul 18, 2022
3854d81
refactor: better inference on YTNode#is
Wykerd Jul 18, 2022
e772b86
fix: linting warnings
Wykerd Jul 18, 2022
96e3ed7
revert: undici import
Wykerd Jul 20, 2022
4974a06
refactor: rename `list_type` to `item_type`
Wykerd Jul 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
11 changes: 6 additions & 5 deletions .eslintrc.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
plugins:
[ jsdoc ]
[ jsdoc, '@typescript-eslint' ]
env:
commonjs: true
es2021: true
node: true
extends: [ eslint:recommended, plugin:jsdoc/recommended ]
globals:
BROWSER: readonly
extends: [ eslint:recommended, plugin:jsdoc/recommended, 'plugin:@typescript-eslint/recommended' ]
parser: '@typescript-eslint/parser'
settings:
jsdoc:
mode: 'typescript'
Expand All @@ -31,6 +30,8 @@ rules:
jsdoc/no-undefined-types: 'off'
jsdoc/require-returns: 'off'

'@typescript-eslint/ban-types': 'off'

no-template-curly-in-string: error
no-unreachable-loop: error
no-unused-private-class-members: 'off'
Expand All @@ -42,7 +43,7 @@ rules:
no-implied-eval: error
arrow-spacing: error
no-invalid-this: error
no-lone-blocks: error
no-lone-blocks: warn
no-new-func: error
no-new-wrappers: error
no-new: error
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/node.js.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:

strategy:
matrix:
node-version: [ 12.x, 14.x, 16.x ]
node-version: [ 16.x, 18.x ]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/

steps:
Expand Down
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,11 @@ pnpm-lock.yaml

# Temporary files for testing
tmp/

# Build output
dist/
bundle/*.js.*
bundle/*.js

# MacOS
.DS_Store
122 changes: 111 additions & 11 deletions README_v2.0.0WIP.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,17 +88,19 @@ Innertube is an API used across all YouTube clients, it was created to simplify[

And huge thanks to [@gatecrasher777][gatecrasher] for his research on the workings of the Innertube API!

If you have any questions or need help, feel free to contact us on our chat server [here](https://discord.gg/syDu7Yks54).

<!-- GETTING STARTED -->
## Getting Started

### Prerequisites
- [NodeJS][nodejs] v14 or greater
YouTube.js runs on Node.js, Deno and in modern browsers.

To verify things are set up
properly, run this:
```bash
node --version
```
It requires a runtime with the following features:
- [`fetch`](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API)
- On Node we use [undici]()'s fetch implementation which requires Node.js 16.8+. You may provide your own fetch implementation if you need to use an older version. See [providing your own fetch implementation](#custom-fetch) for more information.
- The `Response` object returned by fetch must thus be spec compliant and return a `ReadableStream` object if you want to use the `VideoInfo#download` method. (Implementations like `node-fetch` returns a non-standard `Readable` object.)
- [`EventTarget`](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget) and [`CustomEvent`](https://developer.mozilla.org/en-US/docs/Web/API/CustomEvent) required.

### Installation
- NPM:
Expand All @@ -114,15 +116,113 @@ yarn add youtubei.js@latest
npm install git+https://github.com/LuanRT/YouTube.js.git
```

**TODO: Deno install instructions (esm.sh possibly?)**

<!-- USAGE -->
## Usage

Create an Innertube instance (or session):
```js
// const Innertube = require('youtubei.js');
import Innertube from 'youtubei.js';
const youtube = await new Innertube({ gl: 'US' });
```ts
// const { Innertube } = require('youtubei.js');
import { Innertube } from 'youtubei.js';
const youtube = await Innertube.create();
```

## Browser Usage
To use YouTube.js in the browser you must proxy requests through your own server. You can see our simple reference implementation in Deno in [`examples/browser/proxy/deno.ts`](https://github.com/LuanRT/YouTube.js/tree/main/examples/browser/proxy/deno.ts).

You may provide your own fetch implementation to be used by YouTube.js. Which we will use here to modify and send the requests to through our proxy. See [`examples/browser/web`](https://github.com/LuanRT/YouTube.js/tree/main/examples/browser/web) for an simple example using [Vite](https://vitejs.dev/).

```ts
// Pre-bundled version for the web
import { Innertube } from 'youtubei.js/bundle/browser';
await Innertube.create({
fetch: async (input: RequestInfo | URL, init?: RequestInit) => {
// Modify the request
// and send it to the proxy

// fetch the url
return fetch(request, init);
}
});
```

### Streaming
YouTube.js supports streaming of videos in the browser by converting YouTube's streaming data into a MPEG-DASH manifest.

The example below uses [`dash.js`](https://github.com/Dash-Industry-Forum/dash.js) to play the video.

```ts
import { Innertube } from 'youtubei.js';
import dashjs from 'dashjs';

const youtube = await Innertube.create({ /* setup - see above */ });

// get the video info
const videoInfo = await youtube.getInfo('videoId');

// now convert to a dash manifest
// again - to be able to stream the video in the browser - we must proxy the requests through our own server
// to do this, we provide a method to transform the urls before writing them to the manifest
const manifest = videoInfo.toDash(url => {
// modify the url
// and return it
return url;
});

const uri = "data:application/dash+xml;charset=utf-8;base64," + btoa(manifest);

const videoElement = document.getElementById('video_player');

const player = dashjs.MediaPlayer().create();
player.initialize(videoElement, uri, true);
```

Our browser example in [`examples/browser/web`]() provides a full working example.


<a name="custom-fetch"></a>

## Providing your own fetch implementation
You may provide your own fetch implementation to be used by YouTube.js. This may be useful in some cases to modify the requests before they are sent and transform the responses before they are returned (eg. for proxies).

```ts
// provide a fetch implementation
const yt = await Innertube.create({
fetch: async (input: RequestInfo | URL, init?: RequestInit) => {
// make the request with your own fetch implementation
// and return the response
return new Response(
/* ... */
);
}
});
```

<a name="caching"></a>

## Caching
To improve performance, you may wish to cache the transformed player instance which we use to decode the streaming urls.

Our cache uses the `node:fs` module in Node-like environments, `Deno.writeFile` in Deno and `indexedDB` in browsers.

```ts
import { Innertube, UniversalCache } from 'youtubei.js';
// By default, cache stores files in the OS temp directory (or indexedDB in browsers).
const yt = await Innertube.create({
cache: new UniversalCache()
});

// You may wish to make the cache persistent (on Node and Deno)
const yt = await Innertube.create({
cache: new UniversalCache(
// Enables persistent caching
true,
// Path to the cache directory, will create the directory if it doesn't exist
'./.cache'
)
});
```

## API

## Innertube : `object`
Expand Down
12 changes: 12 additions & 0 deletions browser.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
// Deno and browser runtimes

// Polyfill buffer
import { Buffer } from 'buffer';
if (!Reflect.has(globalThis, 'Buffer')) {
Reflect.set(globalThis, 'Buffer', Buffer);
}

import Innertube from './lib/Innertube';
export { default as Innertube } from './lib/Innertube.js';
export * from './lib/utils';
export default Innertube;
2 changes: 0 additions & 2 deletions build/browser.d.ts

This file was deleted.

Loading