Drop Hartree Slater cross GOS?

[ericpre]

From a discussion on gitter, it seems that the cross section formatting in recent version of Digital Micrograph have changed and they don't work anymore:

https://matrix.to/#/!duqsgMFcTyjkCrkoPq:gitter.im/$NvMinl5Q0Zvt-2of6Y5kooyA0v_25dGw13XXjItveqQ

[avira7]

I am having several issues with using Hartree-Slater GOS tables. Firstly, I had to change the import line in 'hartree_slater_gos.py' to the following: from exspy._defaults_parser import preferences. Secondly, as @ericpre mentioned, Digital Micrograph has changed their encoding so exspy can no longer use the Digital Micrograph H-S data tables.

[ericpre]

Thanks for the reporting the import issue, which clearly is a left over from the split. I guess this bug fall through the cracks since we are not allow to distribute the Hartree-Slater GOS tables and therefore we can't test it easily...

That being said, now there are two GOS tables that works out of the box (see https://hyperspy.org/exspy/user_guide/eels.html#generalised-oscillator-strengths and create_model documentation) and are possibly better (more edge, more accurate?). maybe it is time to drop support for the Digital Micrograph GOS?

1. they don't always work out of the box, the path may be to set manually by the user if installation is not standard - pain for the user,
2. they need maintenance but it is not possible to test - issue with distribution.

@avira7, do you have a reason to prefer the GMS GOS in favour of the default one (previously "gosh" and renamed to "DFT" in the last version)? These are downloaded automatically from https://zenodo.org/records/7645765.
In the last version, you can also use the "dirac" GOS, which are downloaded from https://zenodo.org/records/12800856.

[avira7]

My goal is to obtain a very accurate quantification of my EELS data. I am testing my quantification code on a test dataset, and I would like to see how much the at% vary with different models. It seems like the Hartree-Slater model is commonly used so it would have been a nice comparison.

I did try to use the Dirac data tables from https://zenodo.org/records/12800856 using the following code:

import pooch
GOSH10 = pooch.retrieve(
    url="doi:10.5281/zenodo.12800856/Dirac_GOS.gosh",
    known_hash="md5:02fb22ab55e39e51eb03c08dbf699545",
)

m = s_cl.create_model(low_loss=s_zlp, auto_background=True,auto_add_edges=True, \
                      GOS = "gosh", gos_file_path=path+"062f7f5481618c921ce78390fcab3c2b-Dirac_GOS.gosh")

I got the exact same quantification results when using the default tables, which suggests that it wasn't actually pulling from the Dirac GOS. How do I change the model so that it is using the custom GOS?

For my dataset, I saw a variation between the default "gosh" and "hydrogenic". The at% varied by ~5 at%. This is the first time that I am analyzing EELS data, so it is very possible that I am making a mistake with my quantification. Do you usually see a difference of ~5-7 at% when changing between models?

[ericpre]

With the latest version (0.3), use GOS="dft" or GOS="dirac" (see create_model for more information).

It is not surprising that you observe a difference between hydrogenic and the other GOS and it would be depends on a couple of parameters. If you want to know more about it and understand the differences, you should read the Egerton book on EELS (https://link.springer.com/book/10.1007/978-1-4419-9583-4).

For the "dft" and "dirac" GOS, these are more recent and it may be worth looking at their corresponding publication. If you want to understand the context, you can read this issue: hyperspy/hyperspy#2815.

[ericpre]

I move this thread to a discussion and rename the title to "Drop Hartree Slater cross GOS?" because I think that this is a valid question, as mentioned above.
Regarding the maintenance issue, maybe we should get in touch with Gatan to ask them to make these GOS freely available?

[gguzzina]

I had a quick chat about this with Ray Twesten of Gatan, about a year ago.

What actually happened is that they completely replaced their old “Peter Rez” GOS tables with a newer one, featuring an expanded list of edges and better sampling.
Contextually they also transitioned to the obfuscated binary format. This was something that was requested by whoever produced the dataset they licensed in order to “protect their intellectual property”.

So I find it extremely unlikely that Gatan will give you access to any of that.

But maybe they might be persuaded into releasing the old Peter Rez H-S dataset under a permissive license? This would be good for historical reasons (those were used for decades), and it's so small it might even be included directly in the exspy source code.

It may be possible to also ask Prof. Rez himself.

[gguzzina]

In general, I think that if possible it would be good to keep compatibility with the H-S table. Many people have copies lying around, and it may be nice to have if you want to repeat an old analysis and want to get the same result, or to perform comparisons as @avira7 is trying to do.

[ericpre]

Thank you @gguzzina, this is useful information. If the old Peter Rez H-S dataset had a permissive license, we could save them into the gosh format! 😃