-
-
Notifications
You must be signed in to change notification settings - Fork 158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: dockerize, fix sign in loop #11
Conversation
I'll try to reduce the size by experimenting this https://github.com/Zenika/alpine-chrome, but without patching the binaries I think it won't work on newer version of Playwright. |
Nice, thanks! Just tried it. Build took 2m 21s on an M1 MBA, GUI says 1.73GB as image size: $ docker build --tag free-games-claimer .
docker build --tag free-games-claimer .
[+] Building 138.9s (14/14) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.93kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 108B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:focal 2.3s
=> [auth] library/ubuntu:pull token for registry-1.docker.io 0.0s
=> [internal] load build context 0.3s
=> => transferring context: 4.64MB 0.2s
=> [1/8] FROM docker.io/library/ubuntu:focal@sha256:bea6d19168bbfd6af8d77c2cc3c572114eb5d113e6f422573c93cb605a0e2ffb 3.6s
=> => resolve docker.io/library/ubuntu:focal@sha256:bea6d19168bbfd6af8d77c2cc3c572114eb5d113e6f422573c93cb605a0e2ffb 0.0s
=> => sha256:bea6d19168bbfd6af8d77c2cc3c572114eb5d113e6f422573c93cb605a0e2ffb 1.42kB / 1.42kB 0.0s
=> => sha256:fe2e84436d23f1aa22c44036d3aef84c2493c0c60c1bc814df74eaa9f53a30e0 529B / 529B 0.0s
=> => sha256:e784f03641c948e19855ca4741e6f4b7ebfbfe7e8b53672083e52efa465d1e97 1.48kB / 1.48kB 0.0s
=> => sha256:57d0418fe9dcc5262d8c4fcb06c852ad2d0407b541c64d0f5f2e6ec01fd411ba 27.17MB / 27.17MB 2.7s
=> => extracting sha256:57d0418fe9dcc5262d8c4fcb06c852ad2d0407b541c64d0f5f2e6ec01fd411ba 0.7s
=> [2/8] RUN apt-get update && apt-get install -y curl wget && curl -sL https://deb.nodesource.com/setup_16. 22.5s
=> [3/8] RUN apt-get update && apt-get install --no-install-recommends --no-install-suggests -y xvfb xau 12.9s
=> [4/8] WORKDIR /fgc 0.0s
=> [5/8] COPY package.json . 0.0s
=> [6/8] RUN export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true && npm install && npx playwright install-deps 85.9s
=> [7/8] COPY . . 0.2s
=> [8/8] RUN mv ./docker/entrypoint.sh /usr/local/bin/entrypoint && chmod +x /usr/local/bin/entrypoint && mv 0.3s
=> exporting to image 10.9s
=> => exporting layers 10.9s
=> => writing image sha256:7c359b1fa87a1ca4ea7fe70d09f921450aa47d0f6d4d196dcb707ddf613f25a8 0.0s
=> => naming to docker.io/library/free-games-claimer 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them $ docker run --rm -it \
-p 5900:5900 \
-v "$(pwd)/userDataDir:/fgc/userDataDir" \
--name free-games-claimer \
free-games-claimer
31/03/2022 14:56:31 passing arg to libvncserver: -rfbport
31/03/2022 14:56:31 passing arg to libvncserver: 5900
31/03/2022 14:56:31 passing arg to libvncserver: -passwd
31/03/2022 14:56:31 x11vnc version: 0.9.16 lastmod: 2019-01-05 pid: 25
Failed to read: session.ignoreBorder
...
Not signed in anymore. Please login and then navigate to the 'Free Games' page.
31/03/2022 14:57:07 Got connection from client 172.17.0.1
...
31/03/2022 15:07:26 destroyed xdamage object: 0x400031
^Ccaught signal: 2X connection to :60 broken (explicit kill or server shutdown).
31/03/2022 15:07:32 deleted 45 tile_row polling images.
extra[1] signal: -1
XIO: fatal IO error 4 (Interrupted system call) on X server ":60"
after 269258 requests (269258 known processed) with 0 events remaining.
node:internal/process/esm_loader:94
internalBinding('errors').triggerUncaughtException(
^
page.waitForNavigation: Navigation failed because page was closed!
=========================== logs ===========================
waiting for navigation to "https://store.epicgames.com/store/en-US/free-games" until "load"
navigated to "https://www.epicgames.com/id/login/epic?redirect_uri=https%3A%2F%2Fstore.epicgames.com%2Fstore%2Fen-US%2Ffree-games"
navigated to "https://www.epicgames.com/id/login/mfa?redirect_uri=https%3A%2F%2Fstore.epicgames.com%2Fstore%2Fen-US%2Ffree-games"
navigated to "https://store.epicgames.com/en-US/free-games"
navigated to "https://store.epicgames.com/en-US/free-games"
============================================================
at file:///fgc/epic-games.js:47:14 Logged in, but then it did not continue. Seems like the URL changed in the meantime, the Started it again, but it said it got a captcha (VNC not connected): ...
Signed in.
Number of free games: 2
Current free game: City of Brass
Not in library yet! Click GET.
page.waitForSelector: Timeout 20000ms exceeded.
=========================== logs ===========================
waiting for selector "text=Thank you for buying" to be visible
============================================================
at file:///fgc/epic-games.js:98:18 {
name: 'TimeoutError'
}
Saved a screenshot of hcaptcha challenge to screenshots/2022-03-31T15:14:56.170Z.png
Got hcaptcha challenge. To avoid it, get a link from https://www.hcaptcha.com/accessibility
Current free game: Total War: WARHAMMER
Already in library! Nothing to claim. Started it again and tried to connect with VNC while it was running, but then it said the game was already claimed. Other things:
|
I think we will pipe it to null.
Fluxbox is just for convenient usage inside VNC, we will drop it.
Doesn't work for me either, but after logging in, the scrollbar will work. Maybe maximizing the browser will help.
No we should not map the root. We can map Currently the image will rebuild very fast if you have built it before. I have made it so modifying I just ran the script in Docker, it claimed 2 games without any captcha. Somehow when claiming Total War, it said "Got hcaptcha" but it still claimed successfully.
|
* correct freegames url * skip downloading browsers in docker * remove fluxbox * remove stdout for vnc & xvfb
@vogler updated the PR.
The image size is now 1.17GB |
Sounds good.
You want to drop VNC as well then?
I assume
Ok, maybe it doesn't matter. Just thought for development I don't want to rebuild it after every .js change.
Yes, claimed City of Brass for me, just timed out waiting for |
IIRC you can't do that because the data dir for Chromium inside the container might be different from your local Chrome data dir. It will invalidate everything and make you sign in again.
You're right about
You still need to rebuild but it will be very fast if you have built it before. I think the script works for now, but certainly we need to refactor & handle more exceptions. Also we need to improve the locators as it might easily be broken. But for now, it works. My IP gets blacklisted now I guess, it displays captcha for even normal browser. But I'm sure everything works fine now in Docker. |
Updated TODO:
|
I had to delete it in the beginning because of the change
I don't normally use docker. Why would I have to rebuild it if it just runs the .js file from the share?
Yes. The text-based locators are probably the safest bet, but there are still some DOM/CSS-based ones.
👍 |
Not sure if I get what you mean. But if you don't use Docker then you don't need to build anything 😄 . This PR won't affect non-docker user in anyway. |
I meant if I'm testing |
I see. It builds fast though, but you can use this so you don't need to rebuild: docker run --rm -it \
-p 5900:5900 \
-v "$(pwd)/userDataDir:/fgc/userDataDir" \
-v "$(pwd)/screenshots:/fgc/screenshots" \
-v "$(pwd)/epic-games.js:/fgc/epic-games.js" \
--name free-games-claimer \
free-games-claimer
|
That's why I asked why not just share |
hey thanks and a little feedback.
now you can use vnc in the browser and login. now i try that prime-gaming and epic-games both work. |
That's the default procedure. Without rights to a repo, you fork it, commit to a new branch and then open a PR on the target.
We'll mention it in the readme that it should be in the same directory.
Nice, that would be more convenient. But maybe we won't need VNC at all. |
ah okay i dont know gh. i will have a look.
it's only for my personal organisation. the little monk inside me.
as long as it needed we could install it. makes things easier. just a suggestion. |
@viperdriver2000 nice one, I think noVNC is great to use. @vogler I think implementing noVNC is nice, because if we meet captcha, we can send a notification to our device (maybe use apprise?). Then user can open noVNC to control the browser, solve the captcha & claim the game. It has password built in so it's fine to forward the port. Maybe we can use ngrok to tunnel it as well. |
Sounds good. apprise is in Python and I didn't find any JS bindings or an alternative project for nodejs. ngrok docs look like you need an API key to use it? Should we merge this PR and continue in a new one?
Probably best to split into topics:
|
I fixed the bug "timed out waiting for Now it works perfectly for me, without any captcha. I also found this project https://github.com/claabs/epicgames-freegames-node, claabs did a great job in documenting & handling all stuff that we are missing (alpine docker, logging in, 2fa, refreshing token using cookies, advanced free game searching, tunneling...). There's only one problem, this project is using Puppeteer and it will display captcha in everything (including logging in). I suspect that hcaptcha can detect puppeteer but not Playwright. I'll finish this soon, maybe after that, we can collaborate with claabs to fix his captcha issue. |
@vogler Let's merge it for now if you don't mind, it has some bug fixes and a basic Dockerfile for now. We will improve it later :).
|
Just tried it on my RPi4 (SSD): docker build took 6m53s. Launching the browser failed: $ uname -m
aarch64
$ dpkg --print-architecture
armhf
$ docker run --rm -it \
-p 5900:5900 -p 6080:6080 \
-v "$(pwd)/userDataDir:/fgc/userDataDir" \
-v "$(pwd)/screenshots:/fgc/screenshots" \
-v "$(pwd)/epic-games.js:/fgc/epic-games.js" \
--name free-games-claimer \
free-games-claimer
node:internal/process/esm_loader:94
internalBinding('errors').triggerUncaughtException(
^
browserType.launchPersistentContext: Browser closed.
==================== Browser output: ====================
<launching> /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=ImprovedCookieControls,LazyFrameLoading,GlobalMediaControls,DestroyProfileOnBrowserClose,MediaRouter,AcceptCHFrame,AutoExpandDetailsElement,CertificateTransparencyComponentUpdater --allow-pre-commit-input --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --no-service-autorun --export-tagged-pdf --no-sandbox --disable-session-crashed-bubble --restore-last-session --user-data-dir=/fgc/userDataDir --remote-debugging-pipe about:blank
<launched> pid=68
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: y: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: ?: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: cannot open DLS]dq}����������������
[pid=68][err] $-5DTY_hr: No such file
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: ������������: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: @GKRahqx�������������
9@MZp: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: ������������: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: cannot create �W@xO@8
@$@@@�����
[pid=68][err] W�
[pid=68][err] W�
[pid=68][err] W�W�Wлл��7
��7
��7
y�)�
�Y�
�Y�
��ypfyP�td�F��F��F��7�7Q�td���XX/lib64/ld-linux-x86-64.so.2GNU INFOCrashpad4�
GNU&+�U9���Ba p � �������9: Directory nonexistent
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 1: ELF: not found
[pid=68][err] /root/.cache/ms-playwright/chromium-978106/chrome-linux/chrome: 17: Syntax error: word unexpected (expecting ")")
[pid=68] <process did exit: exitCode=2, signal=null>
[pid=68] starting temporary directories cleanup
=========================== logs ===========================
... same as above
============================================================
at async file:///fgc/epic-games.js:13:17 {
name: 'Error'
} |
I have tried to implement this in Docker, it works nicely inside the container now without captcha (sometimes it displays captcha because I login too many times, but it will go away).
What changed:
chrome
channel because it doesn't work for Linux ARM64You can build with
docker build --tag free-games-claimer .
Run with:
Then you can connect to the GUI with VNC Viewer on
127.0.0.1:5900
. Default password issecret
.#3 should be fixed, but not working for Raspberry 32 bit (<4)
#9 should be closed
TODO:
Some further info:
amd64
andarm64
for now. This means the docker image will not work for Raspberry <4.armhf
won't work even if we use executablePath. If you can find a way for it to work withexecutablePath
then we can consider but I doubt it.