Skip to content
This repository has been archived by the owner on May 22, 2024. It is now read-only.

feat: detect V2 API using static analysis #1429

Merged
merged 14 commits into from
May 15, 2023
Merged

feat: detect V2 API using static analysis #1429

merged 14 commits into from
May 15, 2023

Conversation

eduardoboucas
Copy link
Member

@github-actions
Copy link
Contributor

github-actions bot commented May 12, 2023

⏱ Benchmark results

Comparing with 569ebf4

largeDepsEsbuild: 2.2s

⬆️ 5.71% increase vs. 569ebf4

^                                                                   2.9s     3s                           
│                                                   2.8s    2.8s    ┌──┐    ┌──┐                          
│                                           2.6s    ┌──┐    ┌──┐    |  |    |  |                          
│   2.5s                                    ┌──┐    |  |    |  |    |  |    |  |                          
│ ──┌──┐────────────────────────────────────┼──┼────┼──┼────┼──┼────┼──┼────┼──┼────2.4s──────────────────
│   |  |    2.2s    2.2s                    |  |    |  |    |  |    |  |    |  |    ┌──┐            2.2s  
│   |  |    ┌──┐    ┌──┐    2.1s     2s     |  |    |  |    |  |    |  |    |  |    |  |    2.1s    ┌──┐  
│   |  |    |  |    |  |    ┌──┐    ┌──┐    |  |    |  |    |  |    |  |    |  |    |  |    ┌──┐    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
└───┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴──>
    T-12    T-11    T-10    T-9     T-8     T-7     T-6     T-5     T-4     T-3     T-2     T-1      T    
Legend

largeDepsNft: 7.7s

⬆️ 3.85% increase vs. 569ebf4

^                                                                          11.2s                          
│                                                  10.5s   10.6s   10.9s    ┌──┐                          
│                                          10.1s    ┌──┐    ┌──┐    ┌──┐    |  |    9.9s                  
│                                           ┌──┐    |  |    |  |    |  |    |  |    ┌──┐                  
│    9s                                     |  |    |  |    |  |    |  |    |  |    |  |                  
│ ──┌──┐────────────────────────────────────┼──┼────┼──┼────┼──┼────┼──┼────┼──┼────┼──┼──────────────────
│   |  |    7.9s    7.7s    7.6s            |  |    |  |    |  |    |  |    |  |    |  |            7.7s  
│   |  |    ┌──┐    ┌──┐    ┌──┐            |  |    |  |    |  |    |  |    |  |    |  |    7.4s    ┌──┐  
│   |  |    |  |    |  |    |  |    6.9s    |  |    |  |    |  |    |  |    |  |    |  |    ┌──┐    |▒▒|  
│   |  |    |  |    |  |    |  |    ┌──┐    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
└───┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴──>
    T-12    T-11    T-10    T-9     T-8     T-7     T-6     T-5     T-4     T-3     T-2     T-1      T    
Legend

largeDepsZisi: 14.9s

⬆️ 2.01% increase vs. 569ebf4

^                                                                  20.7s                                  
│                                                  19.3s   19.9s    ┌──┐   19.9s                          
│                                          18.9s    ┌──┐    ┌──┐    |  |    ┌──┐                          
│  17.2s                                    ┌──┐    |  |    |  |    |  |    |  |   17.4s                  
│ ──┌──┐────────────────────────────────────┼──┼────┼──┼────┼──┼────┼──┼────┼──┼────┌──┐──────────────────
│   |  |   15.1s   15.2s                    |  |    |  |    |  |    |  |    |  |    |  |                  
│   |  |    ┌──┐    ┌──┐   14.9s   14.2s    |  |    |  |    |  |    |  |    |  |    |  |   14.6s   14.9s  
│   |  |    |  |    |  |    ┌──┐    ┌──┐    |  |    |  |    |  |    |  |    |  |    |  |    ┌──┐    ┌──┐  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
│   |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |  |    |▒▒|  
└───┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴────┴──┴──>
    T-12    T-11    T-10    T-9     T-8     T-7     T-6     T-5     T-4     T-3     T-2     T-1      T    
Legend

const iscExports = mainExports
const { defaultExport, handlerExports } = getExports(ast.body, getAllBindings)

if (featureFlags.zisi_functions_api_v2 && handlerExports.length === 0 && defaultExport !== undefined) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now the feature flag is used to control whether the detection mechanism kicks in. We should be able to roll it out safely to everyone and remove the flag entirely as a quick follow-up.

@eduardoboucas eduardoboucas marked this pull request as ready for review May 14, 2023 12:21
@@ -76,6 +76,7 @@ export const parseExpression = ({
resolveDir: string
}) => {
const { program } = parse(rawExpression, {
plugins: ['typescript'],
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we're now parsing the source before bundling/transpiling.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseExpression is only used in esbuild when parsing non-static imports. Do we need the ts plugin there? At least it wasn't necessary up until now.

@eduardoboucas eduardoboucas requested a review from danez May 14, 2023 18:00
@eduardoboucas eduardoboucas requested a review from Skn0tt May 15, 2023 08:40
@@ -67,6 +59,17 @@ const zipFunction: ZipFunction = async function ({
return { config, path: destPath }
}

const inSourceConfig = await findISCDeclarationsInPath(mainFile, name, featureFlags)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to call out this change, since we're now parsing ISC before bundling. This means we'll start parsing ESM, CJS and TS rather than the bundled/transpiled CJS. I don't anticipate any problems with this, but it's not something I can easily wrap with a feature flag, so something to keep an eye on.

Copy link
Contributor

@danez danez May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to update the babel-parser in the future though, as we otherwise do not support the latest ES standard. It is currently locked at an old version because of previous performance problems. Last time I checked beginning of this year or end of last, the performance problems were still there I think.

But I think the problems are because of the zisi bundler and not necessarily the isc parsing.

// 2. Default function export
// 3. Named `config` object export
export const getExports = (nodes: Statement[], getAllBindings: BindingMethod) => {
const handlerExports: ISCExport[] = []
Copy link
Contributor

@Skn0tt Skn0tt May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just out of curiosity - could you give an example for a file that has multiple exports named handler? I thought that's impossible, but maybe i'm wrong. Or is this an array to make the code for CJS/ESM easier?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

exports.handler = () => {}
module.exports.handler = () => {}

Or even one of these variants repeated multiple times. It's true that only one will actually be evaluated, but at this stage we're just processing the AST so multiple declarations will lead to multiple nodes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, so this can only happen for CJS code. I'm wondering if it makes sense to throw / print a warning if this occurs - it'd make our code a tad easier + help catch user bugs. But maybe it'd break weird bundles that some frameworks emit 🤔

Copy link
Contributor

@Skn0tt Skn0tt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! left a non-blocking question.

@eduardoboucas eduardoboucas merged commit 113b929 into main May 15, 2023
@eduardoboucas eduardoboucas deleted the feat/v2-isc branch May 15, 2023 09:29
@@ -55,6 +55,7 @@
},
"dependencies": {
"@babel/parser": "7.16.8",
"@babel/plugin-syntax-typescript": "^7.21.4",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need to install this. This plugin is for babel itself, but we are only using the parser, which has no concept of plugin packages. For the parser plugins are simply strings that enable features.

@@ -67,6 +59,17 @@ const zipFunction: ZipFunction = async function ({
return { config, path: destPath }
}

const inSourceConfig = await findISCDeclarationsInPath(mainFile, name, featureFlags)
Copy link
Contributor

@danez danez May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to update the babel-parser in the future though, as we otherwise do not support the latest ES standard. It is currently locked at an old version because of previous performance problems. Last time I checked beginning of this year or end of last, the performance problems were still there I think.

But I think the problems are because of the zisi bundler and not necessarily the isc parsing.

return parseObject(node.declaration.declarations[0].init)
}

return {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we return null or undefined here? Then the check above could be easier

-if (Object.keys(config).length !== 0) {
+if (config) {

// - number
// - object
// - string
const parseObject = (node: ObjectExpression) =>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we could at some point add support for template literals, but that is not high prio.

@@ -76,6 +76,7 @@ export const parseExpression = ({
resolveDir: string
}) => {
const { program } = parse(rawExpression, {
plugins: ['typescript'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseExpression is only used in esbuild when parsing non-static imports. Do we need the ts plugin there? At least it wasn't necessary up until now.

Skn0tt pushed a commit to netlify/build that referenced this pull request May 21, 2024
…#1429)

* feat: detect V2 API using static analysis

* chore: add test case

* feat: propagate runtime API version

* feat: support `config` object

* refactor: simplify code

* chore: improve comments

* refactor: safely parse source

* refactor: rename property

* chore: update lock file

* chore: update tests

* chore: update tests

* chore: re-enable V2 tests

* refactor: bail when default export is found
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants