Speed-up dramatically proj.db build time. #4280
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
(follow-up / improvement of #4279)
Current proj.db build time is typically 50 to 60 seconds (and up to 7.5 hours on arm64 cross-compilation with full emulation!). Most of it is due to running consistency checks. Those checks actually only need to run once each time when we update the content of the database. When skipping them, the build time is cut to 3 seconds or so.
So in data/CMakeLists.txt, let keep track of an expected md5sum resulting from the concatenation of the data/sql/*.sql files. When building proj.db, we check if the got and expected md5sum match. If they do build proj.db by inserting the consistency check triggers after having inserted data record. If there's a mismatch, do a one time build with the triggers inserted before the data records, check that proj.db builds fine with that, and if so, emit a CMake error message indicating to the user that they must update the PROJ_DB_SQL_EXPECTED_MD5 variable in data/CMakeLists.txt with the provided value. Next runs will go through the fast build path, until content is updated again.
Demo of first time with outdated value in PROJ_DB_SQL_EXPECTED_MD5: