TimeoutError: Error while deploying on google cloud run

[kairon007]

YOUR BUG DESCRIPTION HERE

If relevant, specify:

• OS: LINUX
• Kobweb libs version: 0.14.1
• Kobweb CLI version: 0.9.13

---

To ensure this bug gets triaged / prioritized quickly, please...

While trying to use Google cloud run, there's a com.microsoft.playwright.TimeoutError thrown while navigating to "http://localhost:8080/admin/create?_kobwebIsExporting=true", waiting until "load"

Use the default free cloud machine with ubuntu

Following is my dockerfile
`#-----------------------------------------------------------------------------

Variables shared across multiple stages (they need to be explicitly opted

into each stage by being declaring there too, but their values need only be

specified once).

ARG KOBWEB_APP_ROOT="site"

FROM eclipse-temurin:17 as java

FROM java as export

#-----------------------------------------------------------------------------

Create an intermediate stage which builds and exports our site. In the

final stage, we'll only extract what we need from this stage, saving a lot

of space.

ENV KOBWEB_CLI_VERSION=0.9.13
ARG KOBWEB_APP_ROOT

ENV NODE_MAJOR=20

Copy the project code to an arbitrary subdir so we can install stuff in the

Docker container root without worrying about clobbering project files.

COPY . /project

Update and install required OS packages to continue

Note: Node install instructions from: https://github.com/nodesource/distributions#installation-instructions

Note: Playwright is a system for running browsers, and here we use it to

install Chromium.

RUN apt-get update
&& apt-get install -y ca-certificates curl gnupg unzip wget
&& mkdir -p /etc/apt/keyrings
&& curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg
&& echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list
&& apt-get update
&& apt-get install -y nodejs
&& npm init -y
&& PLAYWRIGHT_DOWNLOAD_CONNECTION_TIMEOUT=120000 npx playwright install --with-deps chromium

Fetch the latest version of the Kobweb CLI

RUN wget https://github.com/varabyte/kobweb-cli/releases/download/v${KOBWEB_CLI_VERSION}/kobweb-${KOBWEB_CLI_VERSION}.zip
&& unzip kobweb-${KOBWEB_CLI_VERSION}.zip
&& rm kobweb-${KOBWEB_CLI_VERSION}.zip

ENV PATH="/kobweb-${KOBWEB_CLI_VERSION}/bin:${PATH}"

WORKDIR /project/${KOBWEB_APP_ROOT}

Decrease Gradle memory usage to avoid OOM situations in tight environments

(many free Cloud tiers only give you 512M of RAM). The following amount

should be more than enough to build and export our site.

RUN mkdir ~/.gradle &&
echo "org.gradle.jvmargs=-Xmx256m" >> ~/.gradle/gradle.properties

RUN kobweb export --notty

#-----------------------------------------------------------------------------

Create the final stage, which contains just enough bits to run the Kobweb

server.

FROM java as run

ARG KOBWEB_APP_ROOT

COPY --from=export /project/${KOBWEB_APP_ROOT}/.kobweb .kobweb

ENTRYPOINT .kobweb/server/start.sh`

The dockerfile works well on render.com

[bitspittle]

Unfortunately, I'm not sure there is anything I can do in the Kobweb codebase. How many times have you tried redeploying?

Personally I found working with Google Cloud Run (cheapest tier) very frustrating and temperamental and I often had to redeploy more than once to get something that stuck.

[kairon007]

Unfortunately, I'm not sure there is anything I can do in the Kobweb codebase. How many times have you tried redeploying?

Personally I found working with Google Cloud Run (cheapest tier) very frustrating and temperamental and I often had to redeploy more than once to get something that stuck.

Tried it a lot of times using a tweaked docker and even the same one. When I tried deploying my container image via docker push it throws an error "site folder not found"

[bitspittle]

site folder not found feels like a misconfigured Docker file? Basically an earlier image should create a valid Kobweb site build and the last image should copy over just those bits, which will be found living under your site/.kobweb/site directory. Just to double-check, do you have a top-level site folder in your project's root?

I often run Docker locally to verify that the image I wrote worked. Are you able to do that? That way we could verify the issue isn't the Dockerfile but is instead GCR somehow. (You said the file worked on render, though, right?)

[kairon007]

Yes absolutely the same docker worked on render and locally as well. I have the site folder too. Somehow gcr is doing something weird.

…

On Fri, Nov 24, 2023, 03:52 David Herman ***@***.***> wrote: site folder not found feels like a misconfigured Docker file? Basically an earlier image should create a valid Kobweb site build and the last image should copy over just those bits. Do you have a top-level site folder in your project's root? I often run Docker locally to verify that the image I wrote worked. Are you able to do that? That way we could verify the issue isn't the Dockerfile but is instead GCR somehow. (You said the file worked on render, though, right?) — Reply to this email directly, view it on GitHub <#382 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADXEZGSO6YJ3OT2XZHQKI3YF7EDVAVCNFSM6AAAAAA7WZO7J6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRUHE3DMNRWGA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bitspittle]

That is so weird.

Let me convert this issue to a discussion, so we can keep it going, because it doesn't feel right to close it.

[kairon007]

Thanks, I believe playwright timeout might be causing issues on gcr. Any suggestions on how we could do this?
I'm using https://github.com/stevdza-san/BlogMultiplatform and trying to deploy it to gcr . I have high hopes with kobweb,if someone could help resolve this @bitspittle

[bitspittle]

Ah huge apologies, not sure why I missed a notification for this.

So what I'm really curious about is what GCR is doing with the npx playwright install --with-deps chromium line in the dockerfile.

I would almost be tempted to create a DOCKERFILE that all it does is try to install playwright and then quit, and then check the logs.

Worst case, you can give up on the DOCKERFILE, use GitHub to export your site (see: https://github.com/varabyte/kobweb#exporting-your-site-in-a-github-workflow), and then maybe have GCR grab those files somehow, or upload the site files directly into your repo and have a DOCKERFILE that just unzips the site and deploys it. It's a bit more manual, but it could be a decent workaround if you need to get unblocked.

I would also search the internet for other users talking about running playwright on GCR.

Apologies since I don't have that much bandwidth to explore this further. Hopefully some of the suggestions help you with some ideas on how to either get over or around the problem.

[kairon007]

Hey, thanks a lot for getting back. Github workflow worked well for static export and deployment but failed with the same timeout when it wasnt static. What do you think should be the next step

[bitspittle]

FYI, looking here at your workflow file: https://github.com/kairon007/BlogMultiplatformKmp/blob/234eaad46479c4cb38c0dc22b293bbbcf55ec8eb/.github/workflows/export-site.yaml#L58

You are specifying --layout static there but as you are defining server API routes (under your src/jvmMain folder), you should leave this out (or set --layout kobweb instead). You can read more here if you want to learn more.

This might not fix your timeout but it's worth a shot.

[bitspittle]

So I just ran an export locally and got this:

Snapshotting html for "/admin/login"...
Snapshot finished in 28811ms.

That's awfully close to the 30 second timeout being managed by playwright.

Give me some time to look into this tomorrow, it's my end of the day here. Either something is happening that /admin/login is doing that should be fixed. Alternately, maybe I should allow customizing the export timeout to extend it.

[kairon007]

Awesome, thanks a lot for the update

…

On Tue, 28 Nov 2023 at 12:48, David Herman ***@***.***> wrote: So I just ran an export locally and got this: Snapshotting html for "/admin/login"... Snapshot finished in 28811ms. That's awfully close to the 30 second timeout being managed by playwright. Give me some time to look into this tomorrow, it's my end of the day here. Either something is happening that /admin/login is doing that should be fixed. Alternately, maybe I should allow customizing the export timeout to extend it. — Reply to this email directly, view it on GitHub <#384 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADXEZFBVHT3YEQ4M6ELQV3YGWF4XAVCNFSM6AAAAAA74V55AKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TMOBYHEZDO> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bitspittle]

I have NO idea why the /admin/login page would take ~30 seconds to export. Maybe I'm just too tired to see it right now.

If you have time today (I'm not sure what your timezone is), you might try disabling the page for now (just put a return at the top of the composable), get things to export, and start introducing things back in until you see what line is causing the huge performance spike.

I have a friend using Windows who isn't seeing the issue, while I myself on Linux am, and the GitHub CI machine is also Linux. If you're a Windows user, then that could be why things are working for you but not for the CI.

[kairon007]

I'm on mac, also trying to comment out login page entirely, will keep you posted, I'm in +5:30 Timezone, India. Thanks

…

On Tue, 28 Nov 2023 at 13:01, David Herman ***@***.***> wrote: I have NO idea why the /admin/login page <https://github.com/kairon007/BlogMultiplatformKmp/blob/master/site/src/jsMain/kotlin/com/example/blogmultiplatform/pages/admin/Login.kt> would take ~30 seconds to export. Maybe I'm just too tired to see it right now. If you have time today (I'm not sure what your timezone is), you might try disabling the page for now (just put a return at the top of the composable), get things to export, and start introducing things back in until you see what line is causing the huge performance spike. I have a friend using Windows who isn't seeing the issue, while I myself on Linux am, and the GitHub CI machine is also Linux. If you're a Windows user, then that could be why things are working for you but not for the CI. — Reply to this email directly, view it on GitHub <#384 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADXEZAK4MBOZWM6UWKWLD3YGWHLXAVCNFSM6AAAAAA74V55AKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TMOBZGAZDQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bitspittle]

FYI I just tried that locally and it didn't seem to matter -- it still took ~30 seconds to export even though it was an empty method...

Something is fishy. Now I'm wondering if the timing logic is wrong.

It's crazy late here, I'll investigate more tomorrow. Meanwhile, the strategy here isn't horrible -- just start commenting out multiple pages until you find which one lets the GitHub export happen.

If you have other things to work on, you can wait a day, as I'll be digging into this first thing tomorrow.

[kairon007]

Each page I comment, it times out on the other one, I think it's some other issue, I'll wait for your inputs when you're back

[bitspittle]

I'm still investigating this. However, I've got a workaround for you for now so you're not blocked. You'll need to set your Kobweb version to 0.15.3-SNAPSHOT however (to grab code I just added today), and this unfortunately introduces some compile errors into Stevdza-san's project.

Fortunately, I've created a commit for Stevdza-san's upstream project that fixes them, which you can apply as well to your own project.

Steps

• Apply these changes to your own code: stevdza-san/BlogMultiplatform@3e0e63c
  • The above changes set Kobweb to 0.15.2 in libs.version.toml. Set it to 0.15.3-SNAPSHOT in yours.
• Skip over exporting all the admin pages. (You probably don't need those showing up in SEO anyway :P)

  // In site/build.gradle.kts...
  kobweb {
     app {
        export {
           filter.set { !route.startsWith("/admin/") }
        }
     }
  }

I'll still update here when I figure out the root cause. However, this change is relatively non-hacky, and it should get you past your timeout issues. Let me know if this works for you or not!

[kairon007]

Thanks a lot. Much appreciated

[kairon007]

@bitspittle Used the fix above and I got the same error for
Snapshotting html for "/search/query"...
and the same timeout issue. I guess the real timeout will have to be fixed somehow. Please let me know if I can somehow help/assist you in making this easier. Thanks

[bitspittle]

Yeah, I just dropped a huge text dump which tries to explain the real issue. My workaround wasn't foolproof in the end, which I just only now realized, as surely your main Index.kt page is making some API calls itself, so even without admin pages, it's a ticking time bomb.

[bitspittle]

OK, looks like I figured out the gist of what's going on, but I still don't understand the full details.

You're calling API routes that are hanging for a while (e.g. "/api/readmainposts" and "/api/readmyposts"). I'm a bit confused about this because MongoDB APIs are themselves suspendable, but I'm guessing there's at least one call which is not, and is essentially locking up a thread on the server.

I would put a bunch of logging in your MongoDB class to see if you can identify if anything in there is blocking the current thread instead of suspending it. (A bunch of ctx.logger.debug("Before X"); X(); ctx.logger.debug("After X") should do it! You might also be able to use remote debugging as well)

At first, the client doesn't care that it doesn't get results from the server -- it just finishes the page without the returned data, takes a snapshot, and closes.

However, the server is still trying hard to serve this value. This is going to be a problem soon, which is why you often felt like you would remove one page from the export only for another one to fail instead.

The export process is basically visiting page after page without taking a break. So eventually you get to a new page that wants to ask the server for some resource (it doesn't have to be anything extremely complex; for example, I had it get stuck on serving me logo.svg before). The server tries to use a coroutine running on a thread to serve that resource, and it gets stuck, because that thread is still choking on the previous page's API request.

My guess, not 100% sure, is that some part of MongoDB has a default timeout of 30 seconds, so after it fails to connect, it aborts some call, which in turn frees up the thread, which then finally serves the resource to the waiting page about 30 seconds later.

If you go to ApiFunctions.kt and stub everything out by returning early, your exports will work quickly (but of course you don't want to ship code with stubbed API functions)

On my end, I'm still investigating what I should be doing. For sure, I'm going to be adding the ability to create trace files when you export (which is a feature provided by Playwright and what I used locally to debug this). I should also probably have a way for the server to know it's in export mode, so maybe you can update your API routes to return / use a fake DB instance in that case.

On your end, I'm not familiar enough to give confident advice, but perhaps one (or more) of:

• Use rememberPageContext().isExporting on the client to test if you're exporting, and don't call API calls in that case
• See if you can use logging to detect where MongoDB is locking up.
• Configure MongoDB so that it works even on CI.
• Use the kobweb { app { export { filter... } } } callback to basically disable all pages that rely on API calls. (Although it's currently an error to disable all pages; I should probably allow that for non-static layout exports, which yours is)

I'll keep in touch as features land on my end. I think the idea of having a dummy database running on the server in export mode is the cleanest solution.

[kairon007]

Understood, will be looking into mongodb

[bitspittle]

You actually might want to try this right now (disables all exporting):

kobweb { app { export { filter.set { false } } } }

It will report an error (which will become a warning in the future, I just submitted a fix but Gradle caches snapshots for 24 hours), but I think it's an error that doesn't actually block anything.

[kairon007]

w: No pages were found to export. This may be because your build script's kobweb.app.export.filter is filtering out all pages.

BTW, I've already been returning a local version of MongoDb, although it would be an empty one

[bitspittle]

Great! I think you should be able to run on GCR now?

[kairon007]

Still unable to do so getting Users First seen Last seen KobwebException: No site subfolders found. If you ran `kobweb export --layout static`, you should run `kobweb run --env prod --layout static` instead. <https://console.cloud.google.com/errors/CLrXg-PZ1ZONkgE?time=P1D&service=blogmultiplatformkmp&authuser=1&hl=en&project=afineday-service-sms> Also github pages also not showing the prod version just the static once I used last time.

…

On Wed, 29 Nov 2023 at 1:05 PM, David Herman ***@***.***> wrote: Great! I think you should be able to run on GCR now? — Reply to this email directly, view it on GitHub <#384 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADXEZHQQ5ZCHERKHAEQRNTYG3QS7AVCNFSM6AAAAAA74V55AKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TOMBRGA4DI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bitspittle]

I can't access your Google Cloud logs, so I'm not sure what's going on. However, do not switch to --layout static in your case, that's definitely a misleading message for your project.

BTW, what you're trying to do is NOT compatible with GHP. GHP is for static layout sites only, but what you're creating (a fullstack application) requires running a server. I was suggesting you using GitHub CI to build your site, and then have GCR copy those files over and serve them, instead of having GCR doing the export.

[bitspittle]

No idea @kairon007 if you're still around or if you moved on after the frustrating experience with GCR, but we just released 0.15.3, which honestly was a bunch of changes inspired by your difficulties here. You may want to upgrade to that and see if things are better for you. We definitely fixed an issue with the whole export process dying if any of the individual exports timed out.

You can review the release notes here: https://github.com/varabyte/kobweb/releases/tag/v0.15.3

[kairon007]

Thanks for the support guys, the last project was a poc and is currently on hold. Not sure if that'd resume but I'm definitely open to trying it out again soon in hopefully a new project. Thanks again for the support

[bitspittle]

No worries. Your reports helped make the product better, and we appreciate it!

Good luck with whatever direction you end up going in, and if you ever come back to try Kobweb again, we'll be here!

[kairon007]

Thanks, I'll check it out!

…

On Wed, 13 Dec 2023 at 4:17 AM, David Herman ***@***.***> wrote: No idea @kairon007 <https://github.com/kairon007> if you're still around or if you moved on after the frustrating experience with GCR, but we just released 0.15.3, which honestly was a bunch of changes inspired by your difficulties here. You may want to upgrade to that and see if things are better for you. We definitely fixed an issue with the whole export process dying if any of the individual exports timed out. You can review the release notes here: https://github.com/varabyte/kobweb/releases/tag/v0.15.3 — Reply to this email directly, view it on GitHub <#384 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADXEZATODJ772X5UC6B4M3YJDNHHAVCNFSM6AAAAAA74V55AKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQMZWGEZDQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>