doesn't comply with Google's OAuth 2.0 policy for keeping apps secure

[matthewlmcclure]

I followed the instructions at https://vdirsyncer.pimutils.org/en/stable/config.html#google to configure synchronization between my Google Contacts and my local filesystem.

$ vdirsyncer --version
vdirsyncer, version 0.18.0

$ uname -mprsv
Darwin 20.6.0 Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64 x86_64 i386

$ cat ~/.config/vdirsyncer/config
[general]
status_path = "~/.vdirsyncer/status/"

[pair matt_contacts]
a = "matt_contacts_local"
b = "matt_contacts_remote"

collections = ["from a", "from b"]

metadata = ["displayname"]

[storage matt_contacts_local]
type = "filesystem"
path = "~/.contacts/"
fileext = ".vcf"

[storage matt_contacts_remote]
type = "google_contacts"
token_file = "~/.matt_contacts_remote.vdirsyncer.token_file"
client_id = "..."
client_secret = "..."

$ vdirsyncer discover

I found the following information indicating that the flow vdirsync is using is deprecated.

OAuth out-of-band (oob) flow will be deprecated
OAuth out-of-band (OOB) is a legacy flow developed to support native clients which do not have a redirect URI like web apps to accept the credentials after a user approves an OAuth consent request. The OOB flow poses a remote phishing risk and clients must migrate to an alternative method to protect against this vulnerability. New clients will be unable to use this flow starting on Feb 28, 2022.

What do I need to do
Determine if your app is using the OOB flow
You can inspect your app code or the outgoing network call (in case your app is using an OAuth library) to determine if the Google OAuth authorization request your app is making has the following values for “redirect_uri” parameter.

redirect_uri=urn:ietf:wg:oauth:2.0:oob or urn:ietf:wg:oauth:2.0:oob:auto or oob

I have not been able to reproduce the success noted in the similar issue #970 (comment) .

[elshize]

I'm having the same problem. I tried setting up creds several times, but not working, unfortunately.

[WhyNotHugo]

This is yet-another scenario of Google deprecating without offering an alternative. They often deprecate APIs and forgets to offer any working replacement.

There's nothing we can do here. Google discourages usage of this flow, but does not offer any alternative. For reference, here's what the linked page states:

Feb 28, 2022 - new OAuth usage will be blocked for the OOB flow
Sep 5, 2022 - a user-facing warning message may be displayed to non-compliant OAuth requests
Oct 3, 2022 - the OOB flow is deprecated for existing clients

So as of 28th of February, it won't be possible for new users to use vdirsyncer. As of 3rd of October, existing setups will stop working.

I'll leave this issue open in case anything changes. For the moment, nothing can be done. See below.

[WhyNotHugo]

Maybe using the Web Server OAuth API would work, although it assumes that vdirsyncer is running in a context where it can expose a web-server to the person running it (which it not true when you're SSH'ing into another machine or alike).

[elshize]

@WhyNotHugo Thanks for looking into it. Google blows, no two ways about that. Really need to switch to something better...

[zarybnicky]

The service account flow also seems to be feasible - create a service account and share each calendar with the service account's email address. (https://stackoverflow.com/a/26067547/3403345 - edit: while that answer is ancient, the service flow still works the same, IIRC)

Other libraries seem to be switching to the above-mentioned WebServer OAuth API - without too many changes to the relevant code

And for the remote-server workflow, some applications seem to recommend simply copying a token file from a local installation.

[satnerd]

@WhyNotHugo took me a while to read through google's doc and understand whats needed.
One needs to select «TV and Limited-Input Device Applications» when creating the credentials.
The quickstart.py script need also some tiny tweeks. Find the needed changes in attached run-locl-server.py.txt
run-local-server.py.txt

[WhyNotHugo]

Other libraries seem to be switching to the above-mentioned WebServer OAuth API - without too many googleapis/google-auth-library-python-oauthlib#186

The problem is that this assumes that vdirsyncer can expose a port to the same machine where the user is running their browser for authentication. This is the case when running locally on your machine, but not when running on some remote device.

---

One needs to select «TV and Limited-Input Device Applications» when creating the credentials.

One bit caveat:

Important note: This OAuth 2.0 flow supports a limited set of scopes.

---

I think a helper to fetch a token locally is probably best. I'll try making a tiny easy-to-move-around binary that can be used for that.

[satnerd]

I run the script on a NAS (=no local browser) and use my browser on a different machine. Redirection works great.
Not sure what google means by writing about limited scopes.
I copied some more from quickstart.py into a new version of run-local-server.py.txt
run-local-server.py.txt
I could retrieve contact information.

[hoxu]

On 2022-05-04 Google sent an email with subject "[Action Required] Migrate your OAuth out-of-band flow to an alternative method before Oct. 3, 2022" - including the subject here to help people find this issue.

Is anyone working on a PR?

[nekr0z]

Any chance this warrants a point release? Looks rather important use-wise.

[WhyNotHugo]

Yeah, I've been preparing for a point release. Some changes around the tls settings need to be finalised in master first, but it'll be soon.

[hoxu]

I guess this hasn't made it into a release yet?

[nekr0z]

Actually, there seems to be a more elegant solution that does not require opening extra ports.

It is possible to keep the old authentication flow, but use the new "web app" credentials with a random localhost redirect_uri (i.e. something like http://127.0.0.1:12345). There's no need to actually listen to that port!

After the user gets authenticated by Google and allows the app to use the appropriate scopes, the user's browser gets redirected to redirect_uri. Since nothing is there, it displays some kind of the browser's built-in 404 page, but the address bar will contain a full URL with the code query param that we actually need to finish the auth process. We can either ask the user to copy that param, or (more user-friendly) just copy the whole URL to the terminal; vdirsyncer can then parse the URL and get the code that it needs to proceed.

[WhyNotHugo]

It seems they're deprecating the existing API without a suitable replacement. Not sure what we _can_ do.

[nekr0z]

It seems they're deprecating the existing API without a suitable replacement. Not sure what we can do.

As I pointed out, the API they still have left allows for a hack that requires minimal changes from out side. Basically, here is the solution (assuming the starting point is the behaviour in 0.18.0):

1. Change redirect_uri from urn:ietf:wg:oauth:2.0:oob to http://localhost:12345
2. Advise user to create new credentials of type "web application" with "https://localhost:12345" as redirect_uri (should match exactly, including scheme and trailing slashes).
3. Advise user to make sure nothing is running at localhost:12345
4. Have user perform authenitification.
5. When user's browser redirects to non-existing page (http://localhost:12345?query), ask user to copy the URL from the browser and paste it.
6. Parse the URL and extract code parameter from query. Use it to further obtain token.
7. Save token and use it as before. I will be of type "web" (used to be of type "installed"), but otherwise works exactly the same.

Yes, I know, I should learn Python and make a pull request...

[nekr0z]

FWIW, I can confirm that I made vdirsyncer version 0.18.0 (Debian package version 0.18.0-6.1) work with my google calendars today.
Here's what I did:

1. logged into my Google Developers Console
2. selected my vdirsyncer project
3. went to Credentials -> Create Credentials -> OAuth Client ID
4. selected "Web Application"
5. under "Authorised redirect URIs" added http://127.0.0.1:8088
6. pressed "Create"
7. ssh'd into my CalDAV server where vdirsyncer is supposed to be running regularly to sync some of my calendars to Google
8. edited my .vdirsyncer/config [storage google] section to have the new client_id and client_secret (those that I obtained after step 6).
9. edited /usr/lib/python3/dist-packages/vdirsyncer/storage/google.py and changed line 64 from

   redirect_uri="urn:ietf:wg:oauth:2.0:oob",

   to

   redirect_uri="http://127.0.0.1:8088",

   (i.e. exactly the same URI as on step 5)
10. run vdirsyncer discover my_calendar
11. opened the link in my browser (on my desktop machine)
12. proceeded with Google authentication until "Firefox can not connect to 127.0.0.1:8088." was displayed.
13. from the browser's address bar that looked like:

    http://127.0.0.1:8088/?state=SOMETHING&code=HERECOMESTHECODE&scope=https://www.googleapis.com/auth/calendar
    

    copied the "HERECOMESTHECODE" part (the actual code)
14. pasted the code into the ssh session where vdirsyncer was running
15. PROFIT!!!

This required no local server, no changes to my server's firewall, virtually no jumping through hoops, nothing. I suggest the one-line change be incorporated, and the steps to authenticate documented.

[lyz-code]

Thanks @nekr0z your solution worked for me too

[tomaszwostal]

Thanks @nekr0z the workaround works like a charm.

[hajonnes]

How do I

ssh into my CalDAV server where vdirsyncer is supposed to be running regularly to sync some of my calendars to Google

? I.e. number 7 in the instructions form nekr0z

[amcooper]

@hajonnes If it's any help, I run vdirsyncer from my local machine, so I didn't have to ssh into anything.

[mareeekkkk]

@nekr0z Got the Google part working but i am stuck at the part where you paste the code back to the ssh session (Point 14). Where do you paste it? Pasting it in bash results in no such directory..

[nekr0z]

@nekr0z Got the Google part working but i am stuck at the part where you paste the code back to the ssh session (Point 14). Where do you paste it? Pasting it in bash results in no such directory..

You must be using a version of vdirsyncer that is different from what's on my machine. The flow I described works for the exact version. In your case, other changes may be needed in google.py.

[mareeekkkk]

@nekr0z Yes it looks like our versions differ... I got the request to google working, by matching the uri to just http://127.0.0.1 without specifing a port.
But now i am stuck at redirecting the response back to my vdirsyncer instance. I am running in a docker on my syno NAS.
Is there a way to generate the token file via cli on the machine by using the code from the redirect uri?

[mschilli87]

For me, adding http://127.0.0.1 did the trick.

[drselump14]

I can confirm that adding http://127.0.0.1 on the Authorised redirect_uri page
works for me

[benasocj]

It seems that in my case I had to open the link in a private browsing web browser window without any plugins running because after opening the Google web page and logging in, I always got a "something went wrong" error on the next Google web page. Just in case someone else is having that problem.

[tromlet]

Yeah, pasting it in didn't work for me. :(

[jill88]

@nekr0z Got the Google part working but i am stuck at the part where you paste the code back to the ssh session (Point 14). Where do you paste it? Pasting it in bash results in no such directory..

Yea, same here. I have the code now but I don't have any way to paste it anywhere. I am using the docker version as well. I pasted it in the same ssh I had running, but it doesn't do anything.