Adds a sapper extract CLI command, which scrapes the server to run …
#66
Conversation
…as a static website starting at the site's root. TESTED=Basic unit test ensuring relevant routes are added.
lib/utils/extract.js
Outdated
| * @param {number=} extractionDir The directory in which to place the extracted | ||
| * output. | ||
| */ | ||
| module.exports = async function(includeUrls = null, excludeUrls = null, |
There was a problem hiding this comment.
Super cool stuff! There is an issue to move away from async/await. Maybe it's best to preemptively not use it?
There was a problem hiding this comment.
Thanks! Hmm, there are a lot of await calls in Sapper already. I'm happy to change out my await and async calls, but do you think the onus of not being able to run on platforms like AWS Lambda because of the Node version is on the platforms to fix?
There was a problem hiding this comment.
@Rich-Harris has more insight on that than me, but I can definitely see the appeal of supporting lower versions of Node.
I just have all the async/await in test.js left to replace in a PR, so no worries!
There was a problem hiding this comment.
I've removed async/await in my changes to keep you from having to do the work :)
test/common/test.js
Outdated
| 'api/blog/how-can-i-get-involved/index.html', | ||
|
|
||
| 'blog/how-is-sapper-different-from-next/index.html', | ||
| 'api/blog/how-is-sapper-different-from-next/index.html', |
|
Just realized this is currently not producing a navigable website in the EDIT: fixed and added unit test clauses to cover it |
…ests for extracted client pages that match a regular expression
Also clarifies some texts and documentation.
|
FWIW, my PR fails within Travis the same way & it didn't change anything in relation to that test. Maybe it's off? |
|
If you look back in the pull request history, Travis is mostly failing, so I agree something seems off. I'm confident my PR doesn't contribute new failures. (On that note, there seems to be some amount of non-deterministic behavior in the tests. When my computer processor was overloaded, some of the tests sporadically failed, maybe since they're based on timeouts?) |
|
Thanks very much for this, and apologies for leaving it unattended a while. I'm actually having a few issues getting it to work correctly — if you serve the
Will keep chipping away at it (I added some commits to this PR already, mostly so that it doesn't try to extract without first building the site in prod mode), but noting it in here in case it you already know the answer 😀 |
|
I'll have time to look into it more in a few hours. I suspect it may depend on the static router. I think |
|
I'm using |
|
^ I'm working on that locally. Looks good but I think have some stuff missing. The labor of being 3 months out of date with Svelte |
|
That would be great. There is no reason to spin up a server (while researching this I was shocked there's no non-hacky way to get Express to feed you pages without consuming a port). For this PR, I figured getting a messy approach first that can be iteratively improved would be a good starting point. Still need to figure a good solution for having the extract pipeline know about server routes that are called in JS without any extra config options (like the |
|
I actually rescind my earlier comments... server routes are a real challenge if you don't have, well, a server. My current line of attack is to replace |
|
Just to be clear, would the server actually run frontend JS code to figure out what it needs? Let me know if I can help |
|
My current thinking is that we wouldn't run client-side code, because that opens a big can of worms. Instead, we rely on pages specifying their data dependencies in In time, we could probably add a way of configuring additional resources to export, on top of the ones that are automatically discovered. I opened a new PR on top of this one, since that was the easiest way to make additional changes — #88 |
…as a static website starting at the site's root. Resolves #9
TESTED=Basic unit test ensuring relevant routes are added.