Use db-ip lite database for countries

[ukutaht]

Changes

See discussion in #219. With this PR, a basic lite IP-to-Country database is included automatically with no need for extra configuration. Unlike Maxmind there seem to be no restrictions around packaging the database as part of our self-hosted offering as long as credit is given. This will greatly simplify development setups and self-hosted configuration.

Plausible Cloud will keep using MaxMind's database temporarily so we don't have to put up a license notice. But I think it would be a good idea to move to db-ip using their commercial (more accurate) database.

Db-ip also has a free and paid city databases which is great for the future.

Tests

• Automated tests have been added

Changelog

• Entry has been added to changelog

Documentation

• Docs have been updated

[bhavin192]

This is awesome! I was wondering about following things:

• How frequently we are planning to update the checked in mmdb binary file? (I think binary files from git take up more space, as each file is saved in Git even when we replace it). Git LFS is there but feels like too much of hassle for one small file.
  • We can add a step to download latest file as part of container build (I can help with that, if we decide to do so).
• The self-hosted instances will get the new mmdb file whenever there is a new release, this means the file will be 5-6 months old before it gets updated. Not sure if it is a problem though. One can have similar mechanism as the Maxmind's container image, which downloads new file every month, if they want a latest file.
• I was going through this thread https://www.mail-archive.com/debian-legal@lists.debian.org/msg37375.html But did not get a satisfactory answer on if we can distribute the database with AGPL / GPL code. There have been changes in CC-BY-4.0 which make it compatible with GPL, the mail thread does not cover that, 4.0 was not there yet at that time.
  • Seems like we should be fine, I just skimmed through following links (I will read those later in detail).

    Version 4.0 of CC’s Attribution-ShareAlike (BY-SA) license is one-way compatible with the GNU General Public License version 3.0 (GPLv3). This compatibility mechanism is designed for situations in which content is integrated into software code in a way that makes it difficult or impossible to distinguish the two. There are special considerations required before using this compatibility mechanism. Read more about it here.

  • https://creativecommons.org/faq/#can-i-apply-a-creative-commons-license-to-software
  • https://wiki.creativecommons.org/wiki/ShareAlike_compatibility:_GPLv3
  • https://www.fsf.org/blogs/licensing/creative-commons-by-sa-4-0-declared-one-way-compatible-with-gnu-gpl-version-3

[ukutaht]

We can add a step to download latest file as part of container build (I can help with that, if we decide to do so).

I think that would be best. I was planning to update it manually every time we tag a new release but I think automatically downloading it each time Docker build runs is better.

Of course the ideal solution would be to download it in the background every month as it's updated. Even if we release new versions periodically it's not guaranteed that people will upgrade their plausible versions. I'm OK with having a manual update (like we do for User-Agent database and Referrer database) for now.

But did not get a satisfactory answer on if we can distribute the database with AGPL / GPL code.

I've emailed the Creative Commons organization asking for clarification about this. Let's see what they say.

[ukutaht]

License question is cleared up. Here's the response from db-ip people themselves:

Hi Uku,
Thank you for contacting us !
You are absolutely free to redistribute the DB-IP Lite databases along with your AGPLv3 project.

@bhavin192 happy to accept your contribution to download the DB whenever the Docker image is built. If you don't have time I can probably do it myself as well. Let me know.

[bhavin192]

That's great! I will take care of the container image changes.

[ukutaht]

@bhavin192 I just added a task to download the latest DB from db-ip. I also rebased the history, the main point of this was to remove the actual file from the git history so it doesn't weight the repo down in the future. I'll merge this now and await for container updates from you. Thanks!

[ukutaht]

@bhavin192 sorry I'm in a bit of a rush to get this version out so I added a download step to Dockerfile myself: 4fd5cdf

[bhavin192]

@ukutaht ah, no worries, I got stuck with something else, that's why I was not able to do it yesterday.