Add .qpj and .qmd Extensions to Shapefile Handling #8134

[Saixel]

What this PR does / why we need it:

This PR includes .qpj and .qmd in the list of file extensions handled by ShapefileHandler, ensuring that QGIS project files and metadata are correctly processed and included in the zipped shapefile packages. The documentation has been updated to reflect these changes, and a new test has been added to confirm the proper handling of these file types.

Which issue(s) this PR closes:

• Closes ".qpj" file taken out of QGIS Shapefile zip #8134

Special notes for your reviewer:

Please note that .qpj files are crucial for QGIS users as they contain projection information.

Suggestions on how to test this:

1. Upload a zipped shapefile containing .qpj and .qmd files.
2. Ensure the files remain within the zip after upload and are not extracted.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

N/A

Is there a release notes update needed for this change?:

Yes, it has been noted that Dataverse now retains .qpj and .qmd files in zipped shapefile uploads.

Additional documentation:

https://dataverse-guide--10305.org.readthedocs.build/en/10305/developers/geospatial.html

[coveralls]

coverage: 20.139% (-0.002%) from 20.141%
when pulling aa94532 on 8134-add-qpj-qmd-extensions
into 08cf9a8 on develop.

[pdurbin]

@Saixel this is a great start. Next, can you please review this page and consider if any changes should be made? https://dataverse-guide--10305.org.readthedocs.build/en/10305/developers/geospatial.html . As I may have mentioned, I'm not especially familiar with this feature so we'll need to learn together about it. 😅

The source is doc/sphinx-guides/source/developers/geospatial.rst

Also, once we're sure of the direction we're taking, we'll want to add a release note snippet. Please read this: https://preview.guides.gdcc.io/en/develop/developers/version-control.html#writing-a-release-note-snippet

Unfortunately, I don't see any tests being run by Jenkins (this is a recent problem where the webhook isn't delivered). Maybe if we push another commit, they will start. We'll want to check that the tests are passing. Speaking of tests, can you add any assertions of the new behavior to existing tests?

Finally, please add this PR to https://github.com/orgs/IQSS/projects/34 . For now the "in progress" column is a good place. You can assign yourself. Once that's done, you can remove the issue from the board (we do this to avoid duplicates).

Thanks!

[qqmyers]

Re: standup discussion: The code is just rezipping when a zip is a shapefile, but it is only including files with a known extension in doing so, so I agree that the one commit should do it.

FWIW: I think at one point SEAD had a shapefile previewer that showed the contents on a map ala the geoJson previewer. I'm not sure if something like that would be useful for the group with these files or not (or whether opensource code to preview shapefiles would understand the qpj file, etc.)

[pdurbin]

@Saixel I'd say you can safely switch this from draft to non-draft.

The tests are passing, which is great: https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-10305/3/testReport/

Do you plan to add or modify any tests?

[Saixel]

@Saixel I'd say you can safely switch this from draft to non-draft.

The tests are passing, which is great: https://jenkins.dataverse.org/job/IQSS-Dataverse-Develop-PR/job/PR-10305/3/testReport/

Do you plan to add or modify any tests?

Yes! I was thinking of adding one for the specific case, do you think it's worth it? @pdurbin

[pdurbin]

@Saixel yes, I think it would be great to add a test, especially if it doesn't involve adding any large test files.

[github-actions]

📦 Pushed preview images as

ghcr.io/gdcc/dataverse:8134-add-qpj-qmd-extensions

ghcr.io/gdcc/configbaker:8134-add-qpj-qmd-extensions

🚢 See on GHCR. Use by referencing with full name as printed above, mind the registry name.

[pdurbin]

The code looks good. API tests are passing.

I also tested it. I used the shape_with_qpj.zip file provided by @mathieu-massaviol at #8134 (comment)

When you unzip this file, its contents are:

shape_with_qpj/test.cpg
shape_with_qpj/test.dbf
shape_with_qpj/test.prj
shape_with_qpj/test.qmd
shape_with_qpj/test.qpj
shape_with_qpj/test.shp
shape_with_qpj/test.shx

Here's the before and after when uploading this file to Dataverse

before: https://demo.dataverse.org server running Dataverse 6.1

If you download test.zip and unzip it, it contains:

shape_with_qpj/test.cpg
shape_with_qpj/test.dbf
shape_with_qpj/test.prj
shape_with_qpj/test.shp
shape_with_qpj/test.shx

after: this pull request

If you download test.zip and unzip it, it contains:

shape_with_qpj/test.cpg
shape_with_qpj/test.dbf
shape_with_qpj/test.prj
shape_with_qpj/test.qmd
shape_with_qpj/test.qpj
shape_with_qpj/test.shp
shape_with_qpj/test.shx

That is to say, the .qpj and .qmd files stay in the zip. Seems like a good fix. Approved!

[pdurbin]

Looks great! See above for my writeup while testing.