-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
pypidb #674
Comments
Yes, this project is winding down; with <150 packages remaining it's getting to the point where having a database is unnecessary. And I already update it less and less often. I'm not sure what you are asking here. Do you want help mapping Fedora packages to SCM URLs, or Fedora packages to PyPI packages? Or do you want some help with one of the possible projects you mention? |
Hi @encukou , Help mapping Fedora packages to PyPI packages would be very helpful, if the team here are able to use their knowledge or tools to address that, and find it beneficial to have that mapping. Besides that, I created the issue to see if there is interest here in using pypidb to improve the linkage from Fedora packages to the source repositories for any reason that might be on the roadmap for Fedora. While this project is nearing completion, it surely has spawned other side-projects for improving Fedora Python that would also require systematic tasks that this project has facilitated. |
Binary RPMs provide
Currently don't really have a task where a mapping to PyPI names or SCM URLs would help. |
Seems you havent looked at the list I provided yet, and dont seem to be interested, so I'll close it. |
Do you mean https://github.com/jayvdb/pypidb/blob/ba7740e/tests/datasets.py#L46 and https://github.com/jayvdb/pypidb/blob/master/tests/test_fedora.py#L50 ?
Linking PyPI names or SCM URLs doesn't sound useful to me, currently. But I'll keep pypidb in mind if I find a project where it's needed. |
They are manually maintained because they are names which cant be automatically matched using the semi-automated matching algorithms in What would be really great is if Fedora undertook a project to rename their packages to match PyPI name so there is a consistent naming convention, or even only the subset of cases where the Fedora name clashes with a different PyPI project with the same name, which is very confusing. Also working with upstream projects to have them published on PyPI. I havent been doing that with openSUSE projects which were on the equivalent openSUSE list, and most upstream projects are happy to publish onto PyPI but often would like someone to help fix/test the setup.py changes needed. |
Thanks for explaining! I see more clearly what you're trying to say. Even though my answer doesn't change, I can at least explain a bit more. Both projects you mention would be nice. I'll certainly encourage packagers to use predictable component names. But I can't commit to drive a project to rename/publish them all. Especially the renaming would be a much project project than it might seem. Some name clashes come with backwards compatibility issues. Keep in mind that Fedora is older than PyPI. Two examples to think about:
While systems with ad-hoc rules and exceptions might be useful to get general overviews (like portingdb), it would be hard to build on top of them in the long term, and it'll be hard to maintain them. (portingdb sure is pretty bad, but at least it's going away after Python 2 is gone.) |
@encukou , one of the automated matching algorithms I use is adding/removing Redhat is older than PyPI, but Fedora isnt :P
This highlights another way this can help. After portingdb, where might I obtain a good list of Fedora Python package names. I originally fetched all rpm specs and obtained the names from that, however that is a huge download for the purposes of obtaining a list of 3000 names. I have built the list of adhoc data about Fedora package name mismatches, however that is not where I expect it to end, and this issue is part of trying to plot a path forward. But before finding where to maintain the data, the data needs to be checked. You can see in my adhoc data lists I often mention where the Fedora .spec has incorrectly used a Where packages cant be published on PyPI in a reasonable timeframe, Fedora might like to create dummy packages on PyPI with appropriate names, to prevent joe-jobs. This is the rationale I used with great effect to encourage package maintainers to get onto PyPI, as the adverse security implications are quite understandable and motivating.
So we do have shared objectives. All of the names on my list will almost certainly be wrong data emitted by |
That depends on what you consider a Python package. Something that is written entirely/mostly in Python? Has some Python script? Depends on Python directly/transitively? Installs an importable Python module? Has PyPA metadata? Needs Python always/optionally to build/run? One of these commands might do the job:
For But you're right that the python3dist name should match the PyPI package, or be blocked on PyPI. Otherwise we'll get mismatches between what the RPM and pip metadata means. |
Huh. There is absolutely no way for the packaging automation to know the Python package name is not registered on PyPI or that it is clashing with another software. What do you propose the guidelines should say? Something like:
Where do X can be something like:
Also note that when the RPM is installed, |
Yes. I was thinking about the first two Xs.
Yes, and that's a problem: the names in |
New paragraph: The name is derived from the Python distribution package name (e.g. the PS Should we reopen this or take it elsewhere? PS2 I've noticed the guidelines also say "Using a fictional module named 'example', the subpackage containing the Python 3 version must provide python3-example." Which obviously is not followed at all (see pkg_resources, rest_framework, etc...). |
|
I am primarily interested in importable Python modules, as they are dependencies that other packages may need.
Perfect; thanks. fwiw, the openSUSE naming policy does enforce the PyPI name, but with some exceptions which might help guide any Fedora policy around certain possible problems, but I find it doesnt answer the hair naming issues like |
Hello again! We had some private discussions on topics like this. Sorry for silence. As for dots vs. dashes, those can be put into a canonical form, which PyPI uses, using an algorithm described in PEP 503. All automated tools should convert to that form. |
Hi,
It looks like this project is almost complete, which is great to see.
A heads up I am using the fedora.json as a dataset for a list of Fedora python packages to use in testing of https://github.com/jayvdb/pypidb , which locates the SCM for PyPI packages. The methodology is to not rely on the links provided in the PyPI metadata, and it also doesnt include explicit mappings which would be brittle.
There are some packages I couldnt/didnt map to PyPI names in https://github.com/jayvdb/pypidb/blob/ba7740e/tests/datasets.py#L46 , and I am only currently looking at packages prefixed
python[23]-*
andpy*
. There is a mapping of Fedora names to PyPI names directly above that, which may be incomplete.The list of those PyPI packages in Fedora that I do not yet find an SCM for is at https://github.com/jayvdb/pypidb/blob/master/tests/test_fedora.py#L50 . I suspect some of those will be missing mappings from Fedora names to PyPI names, which the team here might be able to spot quickly.
The resulting mappings to SCM might be helpful for this project in various ways, e.g.
The text was updated successfully, but these errors were encountered: