A program written to be touched as little as possible
This small program simply proxies websites while also rewriting the urls to be a passed in through itself (e.g. https://example.com/
-> https://pelicargo.com/mirror/example/
). This probably wouldn't catch every case - especially not for actual web usage, but it works just fine for NPM repos.
Unlike everything else in this company, this is written in nim, for two reasons: low overhead and minimal daemons. This is accomplished by time-traveling to 1993 and using the ancient technique known as CGI.
Also it's not a mirror, it's a proxy. ¯\(ツ)/¯
The binary on util-2
was built using nim 1.6.14, however it shouldn't be too version dependent. The following command will build the binary:
nim c --gc:orc -d:release -d:ssl --opt:speed mirrorer.nim
For some reason unknown to me, NGINX doesn't support a 30 year old spec that's been largely surpassed by numerous technologies. Install fcgiwrap
from the repos (they all have it and under the same name), then add the following to your NGINX server directive:
location /mirror {
gzip off;
fastcgi_pass unix:/var/run/fcgiwrap.socket;
include /etc/nginx/fastcgi_params;
fastcgi_param SCRIPT_FILENAME /path/to/mirrorer/binary;
fastcgi_param DOCUMENT_ROOT /path/to/mirrorer/directory;
}
Then adjust the configuration:
protocol="https"
[examplerepo]
url="https://npm.example.dev/"
auth="password in however the server accepts it"
protocol
- Set tohttps
if your server is configured for HTTPS, otherwisehttp
[examplerepo]
- Set this to something to identify what you're proxying, as this is part of the url that clients will connect tourl
- Site to proxy. Needs a slash at the endauth
- Auth headers to include. This would be the same value as_auth
in your.npmrc
file
Repeat the last three lines as many times as you'd like for different websites.
The examples would be accessible at: https://testcargo.com/mirror/examplerepo
If you're getting timeouts with big installs, you can edit /usr/lib/systemd/system/fcgiwrap.service
to
- ExecStart=/usr/sbin/fcgiwrap ${DAEMON_OPTS}
+ ExecStart=/usr/sbin/fcgiwrap -c6 ${DAEMON_OPTS}
This will allow multiple threads to be able to handle the requests