Attempting to use Gmail Account with oAuth Verification causes Calibre-Web to become unresponsive

[delize]

Describe the bug/problem
Calibre-Web interface stops responding when navigating to https://domain/calibre-web/admin/mailsettings when attempting to use Google oAuth interface.

To Reproduce
Steps to reproduce the behavior:

1. Go to https://domain.org/calibre-web/admin/mailsettings
2. Change Server Type from Standard E-mail account to Gmail account with oAuth
3. Click Setup Gmail Account as E-Mail Server
4. Calibre-web crashes and does not open the oAuth tab.

Logfile
This is with debug context started:

[2022-01-03 22:43:10,223]  INFO {cps:134} Starting Calibre Web...
[2022-01-03 22:43:10,329]  INFO {cps.server:204} Starting Gevent server on [::]:8083
[2022-01-04 12:10:27,001] DEBUG {cps.ub:94} Found stored session: db57a252b3e3ec703e12a0ab4d0e8834ef28afc39f0cb57201c92a1301d93b19093359a606889c4b6dc356972541f88e3b410ea7c2495b775a5d26f83257a720
[2022-01-04 12:10:27,002] DEBUG {cps.ub:94} Found stored session: db57a252b3e3ec703e12a0ab4d0e8834ef28afc39f0cb57201c92a1301d93b19093359a606889c4b6dc356972541f88e3b410ea7c2495b775a5d26f83257a720
[2022-01-04 12:10:27,002] DEBUG {cps.web:1567} You are now logged in as: 'andrew'
[2022-01-04 12:10:31,201] DEBUG {cps.updater:332} Stable version: {'version': '0.6.14'}
[2022-01-04 12:11:17,306]  INFO {cps.server:272} webserver stop (restart=False)
[2022-01-04 12:11:35,294]  INFO {cps:134} Starting Calibre Web...
[2022-01-04 12:11:35,400]  INFO {cps.server:204} Starting Gevent server on [::]:8083
[2022-01-04 12:14:30,623]  INFO {cps.server:272} webserver stop (restart=False)
[2022-01-04 12:14:46,921]  INFO {cps:134} Starting Calibre Web...
[2022-01-04 12:14:47,032]  INFO {cps.server:204} Starting Gevent server on [::]:8083
[2022-01-04 12:14:52,936] DEBUG {cps.updater:332} Stable version: {'version': '0.6.14'}
[2022-01-04 12:21:23,386]  INFO {cps.server:272} webserver stop (restart=False)
[2022-01-04 12:21:39,553]  INFO {cps:134} Starting Calibre Web...
[2022-01-04 12:21:39,658]  INFO {cps.server:204} Starting Gevent server on [::]:8083
[2022-01-04 12:23:24,121] DEBUG {cps.updater:332} Stable version: {'version': '0.6.14'}
[2022-01-04 12:23:33,084] DEBUG {cps.updater:332} Stable version: {'version': '0.6.14'}

Expected behavior
Open up Google oAuth flow to validate the Service Account

Environment (please complete the following information):

• OS: [e.g. Windows 10/Raspberry Pi OS]

cat /etc/issue
Ubuntu 20.04.3 LTS \n \l

• Python version: [e.g. python2.7]

$python3 --version
Python 3.8.10

• Calibre-Web version: 0.6.14 - 4da64ce - 2021-10-31T11:31:53+01:00
• Docker container: LinuxServer
• Special Hardware: N/A
• Browser: Chrome: Version 96.0.4664.110 (Official Build) (x86_64)

Additional context

#2179 - Seems related to the issue
I have attempted to clear the app.db file and start from scratch - this does not seem to help resolve the issue.

[OzzieIsaacs]

This is a problem in the docker container. The gmail authentification workflow is supposed to open a random port, an start a server there for receiving the gmail answer, this doesn't work in the container (is block and not forseable as it's a random port)

[delize]

So want to claim that I am not an expert here, just trying to explore other ideas to get this to work.

There are also alternatives for oAuth flows:
https://stackoverflow.com/questions/47269124/gmail-oauth-from-within-docker-container
https://stackoverflow.com/questions/37243862/send-mail-via-google-apps-gmail-using-service-account-domain-wide-delegation-in
https://stackoverflow.com/questions/61665467/how-to-authenticate-with-gmail-api-from-node-container

It seems like other developers are able to work around this from within the a docker container. This doesn't seem like a blocker or technical limitation due to using containers.

In google's oauth doc: https://developers.google.com/identity/protocols/oauth2/web-server#python_1
You specify the webserver, so I am not sure why you are stating it is random.

Unless I am looking at the wrong file, is this the location where you are specifying the random port?
https://github.com/janeczku/calibre-web/blob/master/cps/services/gmail.py#L61

In which case, you could easily make this a static port - or if running in a container environment, specify a static port:
https://stackoverflow.com/questions/43878953/how-does-one-detect-if-one-is-running-within-a-docker-container-within-python/43879407

If I look at the guide doc here:
https://google-auth-oauthlib.readthedocs.io/en/latest/reference/google_auth_oauthlib.flow.html#google_auth_oauthlib.flow.InstalledAppFlow.run_local_server

It seems like the port isn't random, you can specify a specific port to run the auth on.

I do see this:
googleapis/google-auth-library-python-oauthlib#87

Which describes what you are referring to. That would be a way to possibly run the output to stdout/stderr, which would show in the Docker containers logs. However, it also looks like this potential limitation was resolved last year.

[kryndis]

So I was in a similar boat as @delize and I think I figured out a (very convoluted) workaround.

In my case, I'm running from a TrueNAS FreeBSD jail, but the symptoms of the problem were identical. My first clue came from noticing the text of a Google login page had been recorded in the nohup.out file. That's when I realized the server itself was trying to complete the authentication.

So, assuming you're running from an environment where you don't have (or want) a GUI and therefore don't have a graphical browser, here's how I got it to work. Note that you WILL need some sort of browser, text-based is fine. My jail had w3m available and working by default, so that's what I used.

1. Open a terminal to your server and launch calibre-web in the foreground with "python /path/to/cps.py"
2. On your normal machine, navigate to the mail settings and click the button. At this point it will appear to hang as always, but if you check your terminal, you should find a Google login page open in whatever browser is your default. Now if you could login here, life would be peachy, but unfortunately captchas and text browsers don't mix, so go ahead and quit out of the browser.
3. You should be presented with a very long Google authentication URL (this was what was just open in the text browser.) Go ahead and copy this into your browser on your desktop in a separate tab (make sure to leave the calibre-web page up) and you should be able to perform oauth authentication the same as usual.
4. At this point it will attempt to forward you to "http://localhost:<random_port>/<long_string_of_nonsense>" which obviously isn't going to work. No worries, go ahead and copy that URL.
5. Open a second terminal to your calibre-web server and open that URL you copied in a browser there. Note that if you're using w3m, it doesn't know how to deal with a URL without the http prefix that modern browsers like to use, so you'll need to add it.
6. You should see a simple "authentication successful" message on the browser in the second terminal and the hung calibre-web page will suddenly spring back to life with a properly working oauth setup!
7. At this point you can go ahead and kill the foregrounded calibre-web server process and restart it the way you normally would. You should be all set!

I'd be curious if this method works for anyone else that has a similar problem.

I think what I've done here is more or less invented a bastardized version of the run_console strategy found here: https://google-auth-oauthlib.readthedocs.io/en/latest/reference/google_auth_oauthlib.flow.html by subverting the run_local_server strategy that is currently used in calibre-web. Not sure if it would be worth it to someone to look into adding official support for a run_console strategy or not.

[rubenlagus]

Kudos @kryndis , that helped a lot.

Steps I followed were similar to yours but I don't have an browser, I managed just to use CURL for it and worked:

curl "http://localhost:XXXXX/?state=YYYYYYYY&scope=email%20openid%20https://www.googleapis.com/auth/userinfo.email%20https://www.googleapis.com/auth/gmail.send&authuser=0&prompt=consent"

[henkisdabro]

Is there a working solution for us running the docker-container version? I can't understand @kryndis steps above (how i would run the python command, in the foreground, within a docker container)

[Orgjvr]

@henkisdabro Here is what I did to get my docker container working:

1. Follow this procedure to create an oauth token: https://github.com/janeczku/calibre-web/wiki/Setup-Mailserver#Gmail
2. Add above result file to docker app with -v /app/calibre-web/gmail.json:/mnt/user/appdata/calibre-web/gmail.json (change paths as needed)
3. Inside docker app: get PID of current calibre app: ps -C "python3 /app/calibre-web/cps.py" -o pid=
4. Run the following command to kill the current calibre-web and restart it to get realtime output in terminal: kill PID FROM STEP && python3 /app/calibre-web/cps.py
5. Click on "Setup gmail account as email server" button in Calibre-Web
6. Copy link in terminal and open in any browser. Follow prompts and Allow access.
7. Copy failure link from browser and paste in docker container: curl "LOCALHOST LINK FROM BROWSER" Keep the double quotes around the link.
8. Thats it. Now you can exit and restart your container and test it.

Hope it helps.

[henkisdabro]

Thanks @Orgjvr very much for the detailed steps and full commands, so helpful for us newbies! This worked really well and email sending is now working!

[delize]

Can confirm that this is a solution to the original issue. :)

Really appreciate everyone coming together here to help us on Docker. :)

[IHSten]

Really helpful @Orgjvr thank you! Only thing I found to do differently is for step 4, use the command

kill PID && python3 /app/calibre-web/cps.py &

So you can keep the same shell while still getting terminal output and do the full authentication flow without opening another shell into the docker container, seemed to make it a lot smoother for me. Thanks again!

[Plaidstallion]

@Orgjvr thank you so much for the help.

[dr400]

Thx guys for the great job ... BUT : on step 4, application keeps on restarting by itself (port 8083 is already in use ...). Terminla won't show realtime output. Any idea ?

[Plaidstallion]

Thx guys for the great job ... BUT : on step 4, application keeps on restarting by itself (port 8083 is already in use ...). Terminla won't show realtime output. Any idea ?

So make sure you are not running two different commands for kill <pid_number> && python3 /app/calibre-web/cps.py Ensure you run it as one command with the && in between.

If you run those two commands separately s6-supervise will restart it in the linuxserver.io container before you can manually restart it.

[dr400]

Thx guys for the great job ... BUT : on step 4, application keeps on restarting by itself (port 8083 is already in use ...). Terminla won't show realtime output. Any idea ?

So make sure you are not running two different commands for kill <pid_number> && python3 /app/calibre-web/cps.py Ensure you run it as one command with the && in between.

If you run those two commands separately s6-supervise will restart it in the linuxserver.io container before you can manually restart it.

Thanks for the quick reply but it is the way I've done it ... but still nothing comes out into the terminal with regards to steps 5 & 6 Log is showing :

2022-06-17T15:21:12.544899340Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:18.109544367Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:23.663564926Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:29.208855181Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:35.016975544Z Error starting server: [Errno 98] Address already in use: ('', 8

[ricewillwin]

Thanks for the quick reply but it is the way I've done it ... but still nothing comes out into the terminal with regards to steps 5 & 6 Log is showing :

2022-06-17T15:21:12.544899340Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:18.109544367Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:23.663564926Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:29.208855181Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:35.016975544Z Error starting server: [Errno 98] Address already in use: ('', 8

Same issue is occurring for me even if using the build from around the time the original solution was posted. I don't know if this helps narrow down the problem but I am running this on arm64 (raspberry pi).

[dr400]

Sorry to insist but any help is more than welcome. My system is docker with calibre-web latest image. When open, the terminal will show nothing on calibre-web gmail provider information button press. All processes on google side are done. Last but not least, before all this, clicking the Oauth2 button was not freezing the application, but pressing the button below ("configuration of gmail account for mail server") fails and says : "no valid gmail.json files with Oauth information found"). Hope this helps a bit further ... Help is appreciated here ...

[Orgjvr]

Sorry to insist but any help is more than welcome. My system is docker with calibre-web latest image. When open, the terminal will show nothing on calibre-web gmail provider information button press. All processes on google side are done. Last but not least, before all this, clicking the Oauth2 button was not freezing the application, but pressing the button below ("configuration of gmail account for mail server") fails and says : "no valid gmail.json files with Oauth information found"). Hope this helps a bit further ... Help is appreciated here ...

@dr400 I think you might have an error in step 2. Either the volume was not added correctly to the docker container, or the files does not exists in the mounted path.

Take note that in step 2 the parameters to the -v are 2 paths which are separated by a colon. The first one needs to be the same as in step 2, while the second path needs to point to the file on your host system. In other words, you need to be able to do "ls /app/calibre-web/gmail.json" inside the docker container, and also need to be able to "ls /mnt/user/appdata/calibre-web/gmail.json" outside your container on the host system. This second path might be different for you. It would depend on where you saved the gmail.json file. Just make sure that you use the path where you saved your file, as the path after the colon in step 2.

Hope it helps.

[dr400]

@dr400 I think you might have an error in step 2. Either the volume was not added correctly to the docker container, or the files does not exists in the mounted path.

Take note that in step 2 the parameters to the -v are 2 paths which are separated by a colon. The first one needs to be the same as in step 2, while the second path needs to point to the file on your host system. In other words, you need to be able to do "ls /app/calibre-web/gmail.json" inside the docker container, and also need to be able to "ls /mnt/user/appdata/calibre-web/gmail.json" outside your container on the host system. This second path might be different for you. It would depend on where you saved the gmail.json file. Just make sure that you use the path where you saved your file, as the path after the colon in step 2.

Hope it helps.

Hi Orgjvr and thanks for helping.

Here below is the docker command I use :

docker run --name=calibre-web1 -e PUID=1026 -e PGID=100 -e TZ=Europe/Paris -e DOCKER_MODS=linuxserver/calibre-web:calibre -e OAUTHLIB_RELAX_TOKEN_SCOPE=1 -p 8081:8083 -v /volume1/docker/calibre-web:/config -v /volume1/docker/calibre/Calibre_Library/Calibre_Library:/books -v /volume1/docker/Localtime/localtime:/etc/localtime -v /volume1/docker/calibre-web/gmail.json:/mnt/user/appdata/calibre-web/gmail.json --restart unless-stopped lscr.io/linuxserver/calibre-web:latest

Mount is correct. Both ways are available.
Can't figure out what is wrong ...

[dr400]

I now know what was wrong ... path is not the correct one for my settings (docker run behind a synology server). gmail.json as to be copied into /app/calibre-web/ and not the mnt/user/appdata/calibre-web/ folder ... Thanks Orgjvr for pointing me in the right heading !

[dr400]

Thanks for the quick reply but it is the way I've done it ... but still nothing comes out into the terminal with regards to steps 5 & 6 Log is showing :
2022-06-17T15:21:12.544899340Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:18.109544367Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:23.663564926Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:29.208855181Z Error starting server: [Errno 98] Address already in use: ('', 8083) 2022-06-17T15:21:35.016975544Z Error starting server: [Errno 98] Address already in use: ('', 8

Same issue is occurring for me even if using the build from around the time the original solution was posted. I don't know if this helps narrow down the problem but I am running this on arm64 (raspberry pi).

looks like you may run into the same trouble ... in your docker compose, mnt wherever you're gmail.json is into /app/calibre-web/gmail.json :

-v /volume1/docker/calibre-web/gmail.json:/app/calibre-web/gmail.json

instead of

-v /volume1/docker/calibre-web/gmail.json:/mnt/usr/appdata/calibre-web/gmail.json

Hope this helps

[Plaidstallion]

The problem that I am having now is that with the linuxserver.io container, every week or so the access stops working and the books fail to send. I have to redo the entire process and then the send feature begins working for a time. Has anyone else experienced this problem? If so, have you found a fix?

[IHSten]

The problem that I am having now is that with the linuxserver.io container, every week or so the access stops working and the books fail to send. I have to redo the entire process and then the send feature begins working for a time. Has anyone else experienced this problem? If so, have you found a fix?

I experienced this as well and am hoping that I've fixed it by "publishing" my google cloud application. By default, applications that are in the "testing" state only have valid credentials for 7 days. You can "publish" without verifying, you'll just get an extra warning screen in the OAuth flow. Hopefully I'll remember to update this post if it works or not in a week.

[Plaidstallion]

The problem that I am having now is that with the linuxserver.io container, every week or so the access stops working and the books fail to send. I have to redo the entire process and then the send feature begins working for a time. Has anyone else experienced this problem? If so, have you found a fix?

I experienced this as well and am hoping that I've fixed it by "publishing" my google cloud application. By default, applications that are in the "testing" state only have valid credentials for 7 days. You can "publish" without verifying, you'll just get an extra warning screen in the OAuth flow. Hopefully I'll remember to update this post if it works or not in a week.

Thanks so much for the response. I just published and set up auth again based on your advice. Fingers crossed it solves the issue. It seems we can't be the only ones having this problem.

[naronick]

I've run into a new issue on this one - previously the instructions by Orgjvr worked just fine and got me up and running. I ran into the same one week expiration that Plaidstallion and IHSten had, but now when I try to reauthorize the google oauth page gives me the following error:
Error 400: redirect_uri_mismatch
with explanation
redirect_uri: http://localhost:44115/

I've added "http://localhost" to my allowed redirect URIs, but no that's not working either.

Anyone have a similar issue and able to get around it?

[sheldon-es-MRM]

@henkisdabro Here is what I did to get my docker container working:

1. Follow this procedure to create an oauth token: https://github.com/janeczku/calibre-web/wiki/Setup-Mailserver#Gmail
2. Add above result file to docker app with -v /app/calibre-web/gmail.json:/mnt/user/appdata/calibre-web/gmail.json (change paths as needed)
3. Inside docker app: get PID of current calibre app: ps -C "python3 /app/calibre-web/cps.py" -o pid=
4. Run the following command to kill the current calibre-web and restart it to get realtime output in terminal: kill PID FROM STEP && python3 /app/calibre-web/cps.py
5. Click on "Setup gmail account as email server" button in Calibre-Web
6. Copy link in terminal and open in any browser. Follow prompts and Allow access.
7. Copy failure link from browser and paste in docker container: curl "LOCALHOST LINK FROM BROWSER" Keep the double quotes around the link.
8. Thats it. Now you can exit and restart your container and test it.

Hope it helps.

Thanks, it worked for me. I've calibre-web installed in a Raspberry and had the same problem. I only had to change the cps route.

[DarrenYOW]

Thanks! Worked perfectly for me.

[CultusMechanicus]

Just as an FYI, on the linuxserver.io docker image I had to do: export CALIBRE_DBPATH=/config in the docker container before the kill, otherwise it didn't actually use my existing config.

[Plaidstallion]

Also, I recently found out you can use app codes with gmail to achieve the same functionality we had with simple auth before. After setting up a 2FA device one time it will give you essentially a username and password to send emails from the account with.

https://ugtechmag.com/enable-basic-authentication-gmail-account/

https://support.google.com/mail/answer/185833?hl=en

[Plaidstallion]

I couldn't get that to work, I set up an app password and set the server as smtp.gmail.com, TLS, port 527 etc but email tasks just ran forever and never set anything. They didn't out right fail, but they didn't work either.

Interesting, ok. I guess as I have mail send working with Google OAuth, I won't mess with it. I was able to use app codes for other services but it seems not to be an option here. Thanks for the info.

[DarrenYOW]

Actually, that's what I did in the end. After my "temp" OAuth expired, I wasn't able to get it to work again with a regenerated token, so I replaced it with SMTP using the app code. Been working fine, so I'm not touching it. Sent from Proton Mail mobile

…

-------- Original Message --------

On May 24, 2023, 5:21 AM, CultusMechanicus wrote: > Also, I recently found out you can use app codes with gmail to achieve the same functionality we had with simple auth before. After setting up a 2FA device one time it will give you essentially a username and password to send emails from the account with. > > https://ugtechmag.com/enable-basic-authentication-gmail-account/ > > https://support.google.com/mail/answer/185833?hl=en I couldn't get that to work, I set up an app password and set the server as smtp.gmail.com, TLS, port 527 etc but email tasks just ran forever and never set anything. They didn't out right fail, but they didn't work either. — Reply to this email directly, [view it on GitHub](#2254 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AW4FGJKBV2BQXANY6FEPJGDXHXHKHANCNFSM5LIDLRZA). You are receiving this because you commented.Message ID: ***@***.***>

[frizat82]

@DarrenYOW same issues i'm having - what SMTP service do you use? I'd prefer to move off the gmail token anyway it's a hassle

[Plaidstallion]

@DarrenYOW same issues i'm having - what SMTP service do you use? I'd prefer to move off the gmail token anyway it's a hassle

It's a bit of a pain to setup but I use AmazonSES for some other services that didn't have options for oath.

[DarrenYOW]

I'm using Gmail, as I already had the app password/token set up for other containers. Not ideal, but it is working for now. I don't anticipate hitting the send/attachment limit. Added to the "sh#t I gotta fix later" pile. Sent from Proton Mail mobile

…

-------- Original Message --------

On May 24, 2023, 7:53 PM, Frizat wrote: ***@***.***(https://github.com/DarrenYOW) same issues i'm having - what SMTP service do you use? I'd prefer to move off the gmail token anyway it's a hassle — Reply to this email directly, [view it on GitHub](#2254 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AW4FGJIF7MBV6P3S4P33BCTXH2NPZANCNFSM5LIDLRZA). You are receiving this because you were mentioned.Message ID: ***@***.***>

[rickfmn]

I have the gmail.json placed at /app/calibre-web/gmail.json.
I can terminal in and cat the file fine, however I just get this note in the logs:
[2023-07-10 21:21:07,666] ERROR {cps.admin:1289} Found no valid gmail.json file with OAuth information

I updated my copy of gmail.py to return the full path of the gmail.json file being requested and found it was looking for it at:
/config/gmail.json

So perhaps this comment may help someone else trying to configure this and failing.

[rickfmn]

-- FIXED --
Ok still running into issues here.. I have the gmail.json loading, I have then done these steps:

• Export config: export CALIBRE_DBPATH=/config
• get service id: ps -C "python3 /app/calibre-web/cps.py" -o pid=
• Kill and Start CPS again: kill [PID] && python3 /app/calibre-web/cps.py &
• Clicked the Setup Gmail button, gotten the URL from the terminal, pasted into new browser.
• That allows me to log in and click allow, however after that I am presented with a URL back to localhost on a non-opened port. I think this step is needed to finish the config. I have tried to change the port to match 8083 (or whatever port calibre-web is on), but it just spins.

What am I missing?
-- EDIT

I missed the last step

• curl "localhost address"

Working now (at least for now)

[kaibagley]

-- FIXED -- Ok still running into issues here.. I have the gmail.json loading, I have then done these steps:

* Export config: export CALIBRE_DBPATH=/config

* get service id: ps -C "python3 /app/calibre-web/cps.py" -o pid=

* Kill and Start CPS again: kill [PID] && python3 /app/calibre-web/cps.py &

* Clicked the Setup Gmail button, gotten the URL from the terminal, pasted into new browser.

* That allows me to log in and click allow, however after that I am presented with a URL back to localhost on a non-opened port.  I think this step is needed to finish the config.  I have tried to change the port to match 8083 (or whatever port calibre-web is on), but it just spins.

What am I missing? -- EDIT

I missed the last step

* curl "localhost address"

Working now (at least for now)

Can confirm this works, as long as the gmail.json is in /config, rather than /mnt/...

[jt196]

Folks, after pulling my hair out a bit trying to get this set up, I gave up and fiddled around with some App Specific Password settings and got it working OK. There was an issue with the security settings that may have been stopping some folks.

1. Generate an app-specific password as per the instructions
2. In admin > Edit Email Server Settings enter:

• Email Account Type - Standard Email Account
• SMTP Hostname - smtp.gmail.com
• SMTP Port - 587
• Encryption - STARTTLS (this is the reason mine failed)
• SMTP login - your email address
• SMTP Password - your new app specific password
• From Email - your email address
• Attachment size limit - I entered 50 here

Here's the sending success:

Perhaps folks can enlighten me as to why I shouldn't do it this way, but the set up is a helluva lot easier!

[NotAGizzard]

hey guys i was and still am having an issue with calibre web saying it found no valid gmail.json file with oauth information. i have followed all of the steps for setting up a mail server including the docker steps. i also have the oauth file from google downloaded and bound (at least I'm pretty sure its bound, I'm using portainer to manage my containers) to /app/calibre-web/gmail.json.

[jt196]

hey guys i was and still am having an issue with calibre web saying it found no valid gmail.json file with oauth information. i have followed all of the steps for setting up a mail server including the docker steps. i also have the oauth file from google downloaded and bound (at least I'm pretty sure its bound, I'm using portainer to manage my containers) to /app/calibre-web/gmail.json.

Not to harp on about it but try the app-specific approach I outlined above. Much simpler and is working on my setup.

[SercanKaraoglu]

I've run into a new issue on this one - previously the instructions by Orgjvr worked just fine and got me up and running. I ran into the same one week expiration that Plaidstallion and IHSten had, but now when I try to reauthorize the google oauth page gives me the following error: Error 400: redirect_uri_mismatch with explanation redirect_uri: http://localhost:44115/

I've added "http://localhost" to my allowed redirect URIs, but no that's not working either.

Anyone have a similar issue and able to get around it?

On google OAuth app type should be Desktop app not web server . Change that and it should work

[delize]

@OzzieIsaacs / /@janeczku

Has this process been improved at all?

Google has mentioned that they are deprecating several methods of email that do not use oAuth - so oAuth is going to be a preferred route in the future, and as I am setting this up again from scratch - this is becoming a struggle with the latest image and the most recent oAuth instructions found in this issue.

[drizuid]

@OzzieIsaacs / /@janeczku

Has this process been improved at all?

Google has mentioned that they are deprecating several methods of email that do not use oAuth - so oAuth is going to be a preferred route in the future, and as I am setting this up again from scratch - this is becoming a struggle with the latest image and the most recent oAuth instructions found in this issue.

there is no process for ozzie to improve, it works fine per his standard instructions outside of a container. That said, the instructions are pretty close to perfect, the only note is that you must do what was stated here #2254 (comment) otherwise, calibre-web will load the default database which is not the correct database in our container. I am considering doing a symlink or something to keep those databases the same, as it would also reduce the number of support requests for password resets, but I will have to find time to do so.

dev-box ~ # docker exec -it calibre-web bash

root@9518e931617d:/# ps -C "python3 /app/calibre-web/cps.py" -o pid=
    355
    
root@9518e931617d:/# export CALIBRE_DBPATH=/config && kill 355 && python3 /app/calibre-web/cps.py &
[1] 454

--click the button in the webui

root@9518e931617d:/# Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=<removed>

root@9518e931617d:/# curl "http://localhost:44673/?state=<removed>&scope=email%20https://www.googleapis.com/auth/gmail.send%20openid%20https://www.googleapis.com/auth/userinfo.email&authuser=1&prompt=consent"

The authentication flow has completed. You may close this window.
root@9518e931617d:/#

[wisie]

export CALIBRE_DBPATH=/config && kill 355 && python3 /app/calibre-web/cps.py &

This did the trick for me. Thank you.

[latipun7]

1. Follow this procedure to create an oauth token: [Wiki: Setup Mailserver (Gmail)](https://github.com/janeczku/calibre-web/wiki/Setup-Mailserver#Gmail)

2. Add above result file to docker app with -v /app/calibre-web/gmail.json:/mnt/user/appdata/calibre-web/gmail.json (change paths as needed)

3. Inside docker app: get PID of current calibre app: `ps -C "python3 /app/calibre-web/cps.py" -o pid=`

4. Run the following command to kill the current calibre-web and restart it to get realtime output in terminal: `kill `PID FROM STEP ` && python3 /app/calibre-web/cps.py`

5. Click on  "Setup gmail account as email server" button in Calibre-Web

6. Copy link in terminal and open in any browser. Follow prompts and Allow access.

7. Copy failure link from browser and paste in docker container: `curl "LOCALHOST LINK FROM BROWSER"` Keep the double quotes around the link.

8. Thats it. Now you can exit and restart your container and test it.

Hope it helps.

I tried following steps from @Orgjvr but encountered error on step 7.5.

After I run curl, the terminal did give me The authentication flow has completed. You may close this window.
But, on calibre-web page, returned this message instead of success: (mismatching_state) CSRF Warning! State not equal in request and response.

How to resolve that problem?

Here's my step to reproduce:

• my linux server at 192.168.1.1, run ~/calibre-web/bin/cps & via ssh terminal from my laptop (my calibre-web installed with pip)
• access calibre-web (http://192.168.1.1:8083) from my laptop's browser at 192.168.1.2
• run step 5-7 above, curl from ssh terminal
• page refreshed and got CSRF error.
• trying to send email or access register page, saying email server not setup.
  

[OzzieIsaacs]

It‘s a docker only problem

[chen8160]

Following the steps here and got "could not locate runnable browser " in the calibre-web UI and no links in the terminal which I ran the export CALIBRE_DBPATH=/config && kill [pid] && python3 /app/calibre-web/cps.py &.

Any idea why this is happening?

[thankarezos]

Following the steps here and got "could not locate runnable browser " in the calibre-web UI and no links in the terminal which I ran the export CALIBRE_DBPATH=/config && kill [pid] && python3 /app/calibre-web/cps.py &.

Any idea why this is happening?

I am getting the same error using linux.io server docker image

[ofostier]

Same problem for me using linux.io image
url doesn't happend in the terminal
it is like 2 process run at the same time

[Akadiuss]

Following the steps here and got "could not locate runnable browser " in the calibre-web UI and no links in the terminal which I ran the export CALIBRE_DBPATH=/config && kill [pid] && python3 /app/calibre-web/cps.py &.

Any idea why this is happening?

I have been looking for hours now, same errors and I'm not getting the link.

[tamuseanmiller]

Same issue here, would love to reopen this as it seems to be a regression or something.

[sargunv]

I also got "could not locate runnable browser" and worked around it with apt install lynx in the container. I didn't actually use lynx; just needed it present to get past that error and set up gmail auth with the usual export CALIBRE_DBPATH=/config && kill [pid] && python3 /app/calibre-web/cps.py

[math-eww]

Hey all - just went through the docker setup myself and got it working (thanks to all in this thread!) and wanted to share a simple clear way to get going

I set up a browser intercept so we don't need to install lynx (which has some issues with JS in the google auth flow) - this will just print the url you need to open directly

# Set up browser intercept
echo -e '#!/bin/bash\necho "Browser open attempt with URL: \$1" >> /tmp/oauth_url_log.txt\necho "Open the following URL manually in a browser:"\necho \$1' > /usr/local/bin/log_browser.sh && chmod +x /usr/local/bin/log_browser.sh
export BROWSER="/usr/local/bin/log_browser.sh"

# Get PID
ps -C "python3 /app/calibre-web/cps.py" -o pid=

# Set env, kill using PID from previous step, and relaunch calibre
export CALIBRE_DBPATH=/config && kill [PID]  && python3 /app/calibre-web/cps.py

# Click the setup gmail button in webui, a 'browser' will be opened with the OAuth authorization URL - instead we will print it to console

# Open the link in any browser, go through the login flow
# - Make sure to tick the send email permission if its not ticked

# The last step will fail in your browser (redirect to localhost) - copy that URL and place it in the curl command below
# - Use a second shell to do this (open a new docker exec) as the first one is still running calibre
curl "LOCALHOST LINK FROM BROWSER"

[stayupthetree]

export CALIBRE_DBPATH=/config && kill [PID] && python3 /app/calibre-web/cps.py

You are the real MVP, this worked brilliantly

[flywithdragons]

Hey all - just went through the docker setup myself and got it working (thanks to all in this thread!) and wanted to share a simple clear way to get going

I set up a browser intercept so we don't need to install lynx (which has some issues with JS in the google auth flow) - this will just print the url you need to open directly

# Set up browser intercept
echo -e '#!/bin/bash\necho "Browser open attempt with URL: \$1" >> /tmp/oauth_url_log.txt\necho "Open the following URL manually in a browser:"\necho \$1' > /usr/local/bin/log_browser.sh && chmod +x /usr/local/bin/log_browser.sh
export BROWSER="/usr/local/bin/log_browser.sh"

# Get PID
ps -C "python3 /app/calibre-web/cps.py" -o pid=

# Set env, kill using PID from previous step, and relaunch calibre
export CALIBRE_DBPATH=/config && kill [PID]  && python3 /app/calibre-web/cps.py

# Click the setup gmail button in webui, a 'browser' will be opened with the OAuth authorization URL - instead we will print it to console

# Open the link in any browser, go through the login flow
# - Make sure to tick the send email permission if its not ticked

# The last step will fail in your browser (redirect to localhost) - copy that URL and place it in the curl command below
# - Use a second shell to do this (open a new docker exec) as the first one is still running calibre
curl "LOCALHOST LINK FROM BROWSER"

I spent hours trying everything under the Sun. Was sure I'm doing everything correctly, and yet kept on getting the "could not locate runnable browser" error. I should have just read all the way to the end first. This solution is the only one that worked. Thank you!