-
-
Notifications
You must be signed in to change notification settings - Fork 164
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
provide a way to avoid re2 installation #630
Comments
It is a peer dependency of https://github.com/spamscanner/url-regex-safe, turns out. |
Is it worth evaluating just using a more basic URL regex?
|
It looks like metascraper could get away with simple using |
Yes, it's used 🙂 If r2 installation fails, still |
@Kikobeats But is it needed given that you can achieve the same check using |
Unfortunately it's needed. Maybe we can find a mechanism to skip re2 installation, like using an environment variable. |
Could we update metascraper API such that users can provide their own RegExp constructor and instruct them to use re2 if possible?, i.e. const SafeRegExp = (() => {
try {
const RE2 = require('re2');
return typeof RE2 === 'function' ? RE2 : RegExp;
} catch {
return RegExp;
}
})();
const metascraper = createMetascraper([
createMetascraperTitleRule(),
createMetascraperDescriptionRule(),
createMetascraperImageRule(),
], {RegExp: SafeRegExp}); |
Can you paste the installation error you are getting? Theoretically it should be work even re2 installation failed. |
It is not the installation error, it is the
RE 1: If you create an Dockerfile with: FROM node:19-bullseye
RUN npm install --global pnpm@^7.30.3 turbo@^1
RUN pnpm install re2 then just the RE 2: We use |
what if you use pnpm override to avoid install it? https://pnpm.io/package_json#pnpmoverrides I don't feel comfortable removing re2 since it's there for a good reason, but I have to explore a mechanism to opt-out if that's the thing the user wants. To me, something like |
It is not ideal, but we can. I say it is not ideal because overriding internal dependencies tends to introduce very hard to debug issues long-term. Been there a few times. |
I would make this more explicit, like |
@Kikobeats I was confused why we are still installing it, but it looks like re2 is actually a hard dependency of
If this can at least be made a peer dependency, then we can just skip installing it. |
This was painful but hopefully helps others uhop/node-re2#163 (comment) |
Yes, it's a hard dependency. I want to keep it unless you explicit opt-out passing |
is there a proper way to fix re2? im using nextjs 13 and get re2.node error when compiled |
Looking forward to that |
I was having the same issue with next 13. i tried excluding re2 from the webpack bundle but that didn't help. I just built my own solution using Cheerio. Just grab the html from your request url, load it with cheerio, and search for the specific metadata you need. Ofc, add in the appropriate fallbacks where necessary to improve accuracy. It's pretty simple and you wont have to deal with that re2 node error. Here is an example of what i did:
|
+1 |
+1 just started having a related issue building a node backend service that uses metascraper
it seems re2 is failing and it is trying to build locally (the build worked before). |
for info... seems the issue above with re2 and node alpine started with 1.20.1 - see uhop/node-re2#180 |
hope this can help #656 |
Did anyone here find a workable solution to exclude re2 from the build? #656 only helps at runtime, so it doesn't fix the issue for compiled apps (next, webpack, esbuild, vite, etc). Is this patch the best option? |
Unless I am missing something,
re2
is added as a dependency but it is not actually being used anywhere.The text was updated successfully, but these errors were encountered: