Skip to content

Data Source: Google Photos

Jump to bottom
Matt Holt edited this page Dec 23, 2020 · 9 revisions

This data source adds your photos, videos, and albums from Google Photos.

It supports both the Google Photos API for regular automated pulls, and also importing from Google Takeout archives. You can also do both, but you must enable soft merging if you do, otherwise you risk duplicating your entire library locally!

Google Photos
Site https://photos.google.com
Listing Mode ✅ Download via API
✅ Import from file
Data Source ID google_photos
Authentication OAuth2
OAuth2 Provider ID google
OAuth2 App Config 
[oauth2.providers.google]
client_id = "YOUR_APP_ID"
client_secret = "YOUR_APP_SECRET"
auth_url = "https://accounts.google.com/o/oauth2/auth"
token_url = "https://accounts.google.com/o/oauth2/token"

API vs. Takeout

To fill your timeline, you can use the Google Photos API, Google Takeout, or both.

Using the API:

  • Can schedule regular pulls with get-latest for reliable, efficient downloading
  • Does NOT have location data (coordinates are also stripped from EXIF)

Using Google Takeout:

  • Manual process of downloading archive and using import
  • Does NOT have IDs (soft merging required if using with the API!)
  • Includes location data

Using both:

  • Regular automated pulls with the API, then occasional manual imports from Google Takeout archives
  • You get all the metadata from the API as well as location data from Takeout
  • SOFT MERGING REQUIRED

To use both, you must enable soft merging then and forever after, otherwise your entire library may be duplicated!

For API commands, I like to use -merge=soft,id,file,meta because I prefer the API's IDs (Takeout does not give any IDs at all), data file (it is compressed but still high-fidelity), and metadata (it is richer).

For import commands, I use -merge=soft because I only use it to add the missing location data.

Note how both commands require soft merging, not just one or the other. If you forget to enable soft merging, you could duplicate your library!

Adding your photos and videos via the API

You will need to create OAuth2 app credentials and then you can schedule regular, automated pulls.

Creating OAuth2 App Credentials

See instructions here for creating a Google Developers project, getting app credentials, and enabling the API.

Once you have written your config file, then add your Google Photos account (only needs to be done once):

$ timeliner add-account google_photos/you@gmail.com

Pulling from the API

NOTE: Be sure to configure soft merging if you have also used Google Takeout to import from Google Photos.

To get all your content:

$ timeliner get-all google_photos/you@gmail.com

Timeliner will concurrently get all your photos and videos from your feed in reverse-chronological order while iterating all albums and grouping items into them on your timeline. You can use flags to customize this behavior.

Alternatively, to get only the new items that are at least 10 days old (giving you time to delete unwanted photos before they are saved locally):

$ timeliner -end -240h get-latest google_photos/you@gmail.com

Caveats with the API

  • Location data is stripped. Photos are downloaded with their full EXIF metadata except for latitude and longitude. This is unfortunate. I recommend augmenting your timeline with Google Location History to add location data OR use the Google Takeout method of importing your photos, but be sure you enable soft merging if you do! Google Takeout archives include files uploaded in original quality and original location metadata.

  • If photos are changed on Google Photos after being added to your timeline, there is no way to download those changes without a full -reprocess flag. Even after a photo is edited in Google Photos, only the current edit is available. No version information is provided.

  • Album enrichments are not available.

Adding your photos and videos via Google Takeout

NOTE: Be sure to configure soft merging if you have also used the Google Photos API to fill your timeline.

  1. Go to https://takeout.google.com/
  2. Click "Deselect all" then scroll down to Google Photos.
  3. Enable Google Photos. You can select which albums you want to include.
  4. Scroll down and click Next.
  5. The other options are up to you. Click "Create export".

Once your archive is ready, go ahead and download it. Then import it into your timeline:

$ timeliner import /path/to/takeout.zip google_photos/you@gmail.com

This will use quite a bit of CPU and probably take several minutes.

Caveats with Takeout

  • Takeout does not provide any IDs, so we have to invent our own when storing them on your timeline. This means Timeliner cannot deduplicate across the API and Takeout, so you have to enable soft merging if you use both!

  • Limited metadata is available via Takeout, however full EXIF data (as far as I can observe) has been preserved in the downloaded photos.

  • Google can change the Takeout archive format at any time, breaking this implementation. Please help maintain this feature if you use it!

  • The importer does not currently support all variants of a file, for example those with names suffixed with -edited or (1) before the extension. It works by finding metadata JSON files, then looking for the version of the filename without the .json extension. So if the filename differs more than that, it won't be imported.

Note: Timeliner does not duplicate the functionality of Google Photos. It is a read-only API client that aggressively caches the photos for quick, accessible rendering on the account owner's computer so that the application can work offline. Timeliner encourages use of the Google Photos platform because it results in a richer projection of the user's data.

Clone this wiki locally