Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug fix for potcar parsing #2910

Merged
merged 4 commits into from
Apr 4, 2023
Merged

Conversation

jmmshn
Copy link
Contributor

@jmmshn jmmshn commented Mar 25, 2023

bug fix for potcar parsing

2023.3.10 rewrote the PotcarSingle.from_file function slightly.
This caused a parsing error in the tests for pymatgen-analysis-defects.

I think this small change gives the intended behavior.

Otherwise, the POTCAR in this folder cannot be parsed.
https://github.com/materialsproject/pymatgen-analysis-defects/tree/main/tests/test_files/v_Ga/ccd_0_-1/optics

Without the fix:

pc = Potcar.from_file("./POTCAR")
{d.header for d in pc}

Gives:

{'PAW_PBE Ga_d 06Jul2010', ''}

with the fix, you get:

{'PAW_PBE Ga_d 06Jul2010', 'PAW_PBE N 08Apr2002'}

@jmmshn
Copy link
Contributor Author

jmmshn commented Mar 26, 2023

@janosh @shyuep, I think I have tests are failing due to some hash mismatches.
I'm not familiar with the hashing setup but since I'm only removing blank lines I think it should have been OK.

@janosh
Copy link
Member

janosh commented Mar 26, 2023

I'm not familiar with the hashing setup but since I'm only removing blank lines I think it should have been OK.

Agreed.

@jmmshn
Copy link
Contributor Author

jmmshn commented Mar 27, 2023

So this is still failing, I really hope this is somewhat connected to the massive amount of warnings I always get during POTCAR reading so w/e fixes this will fix that as well. Maybe we have to redo the hashes and be more consistent about trimming? Again I'm not too familiar with that part of the code.

@shyuep shyuep enabled auto-merge March 27, 2023 20:00
@shyuep
Copy link
Member

shyuep commented Mar 27, 2023

I have merged this first. But I really really dislike the whole hash system. Unless someone gives me a good reason why we are spending so much effort on this, I am disabling the tests and in future, I will disable the hash checks for good as well.

@shyuep shyuep disabled auto-merge March 27, 2023 20:01
@shyuep shyuep enabled auto-merge March 27, 2023 20:01
@janosh
Copy link
Member

janosh commented Mar 27, 2023

Unless someone gives me a good reason why we are spending so much effort on this, I am disabling the tests and in future, I will disable the hash checks for good as well.

I don't see a reason. Just to be clear, you mean keep all the Potcar tests and just disable those that relate to the hashes? I think that would be good.

@jmmshn
Copy link
Contributor Author

jmmshn commented Mar 27, 2023

@shyuep I think 99% of people only rely on the headers. I recall there were "secret" custom PAWs with the same headers as standard ones that people used (which later got released under a different header). But I feel like if you are using those then it's on the user to sort that out.

However, I assume the hashing was added to get to make sure the build process is a bit more careful (and might be used by the builders) so you might need to ping @munrojm before any major changes.

@shyuep
Copy link
Member

shyuep commented Mar 27, 2023

I agree. So why don't you comment out the offending tests in the PR to let it pass. We will then merge. I think the hash checks can throw warnings like they do currently, but I am really tired of these tests failing because of some minor missing new line or space somewhere.

@munrojm
Copy link
Member

munrojm commented Mar 27, 2023

As long as the hashing functionality remains in the code, we should be okay. We currently use hash checking as part of the calculation validation pipeline.

@@ -2275,19 +2275,19 @@ def from_file(filename: str):

functionals = []
for p in fdata.split("End of Dataset"):
if p.strip():
single = PotcarSingle(p + "End of Dataset\n")
if p_strip := p.strip():
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooo walrus is in 3.8? nice!
image

@shyuep
Copy link
Member

shyuep commented Mar 27, 2023

@munrojm I will leave the hash generation in. But if we are disabling the tests, I make no guarantees that future hashes generated will be valid. In other words, the hash is just being generated and we do not test for correctness. As I said, this is far too much effort to maintain for a functionality that very few people care about.

@munrojm
Copy link
Member

munrojm commented Mar 27, 2023

@shyuep, that is fair.

@janosh janosh added io Input/output functionality fix Bug fix PRs labels Mar 28, 2023
@jmmshn
Copy link
Contributor Author

jmmshn commented Apr 3, 2023

Can this get merged, please? I think the failing tests are from an unrelated issue.

auto-merge was automatically disabled April 3, 2023 05:58

Head branch was pushed to by a user without write access

@jmmshn jmmshn force-pushed the patch_potcar_bug branch from ffa0dd2 to e4a9fc2 Compare April 3, 2023 06:01
@janosh
Copy link
Member

janosh commented Apr 3, 2023

I agree. So why don't you comment out the offending tests in the PR to let it pass.

I think this is waiting on commenting out the failing POTCAR hash tests as @shyuep suggested.

@jmmshn
Copy link
Contributor Author

jmmshn commented Apr 4, 2023

@janosh I modified to test and now it just attempts parsing withouth any checks. I actually wasn't sure if it was proper etiquette for someon other than the maintainers to delete tests which is why I didn't do it originally.

@janosh
Copy link
Member

janosh commented Apr 4, 2023

@jmmshn Thanks! The remaining errors here come from breaking changes in pandas v2 should be fixed by #2935. Merging now.

@janosh janosh merged commit 1e9119a into materialsproject:master Apr 4, 2023
@jmmshn jmmshn deleted the patch_potcar_bug branch April 4, 2023 02:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix Bug fix PRs io Input/output functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants