Add flop count perf job back in

[charleskawczynski]

This PR adds the flop counting perf job back into CI.

I was a bit too impatient waiting for triscale-innov/GFlops.jl#49 to be fixed. I tried messaging the author on slack, it's been months since the issue was opened, and users still can't update to Julia 1.10. So, I made CountFlops.jl.

I think @dennisYatunin developed his own version, too. We could combine / extend it.

I think this is a very useful tool for simple, pointwise functions, like Thermodynamics, CloudMicrophysics, and SurfaceFluxes.

[nefrathenrici]

Looks good! It would be good to add similar tests to microphysics.

[dennisYatunin]

This looks good overall. The tool I'm developing will likely end up looking more similar to AllocCheck than to GFlops (maybe also with a few pieces from Cthulhu), but there might be some room for combining it with CountFlops because its overall scope is related. That being said, it would probably be easier to just merge your PR into GFlops, instead of maintaining a copy/fork. We can try using CountFlops for now, though, since that PR has already been open for a while.

As you've mentioned before, the main downside of the pure Cassette-based implementation in GFlops/CountFlops is that it does not account for compiler optimizations like constant propagation, which means that it can only give an upper bound on the number of times some function gets called. This is why I've been looking into how AllocCheck analyzes the IR with GPUCompiler, since we probably need to account for the compiler's IR transformations to estimate quantities like the register pressure of a kernel. Something akin to CountFlops.@count_ops should be okay for testing the simple functions in Thermodynamics, but it might not be very helpful for the more complex kernels in ClimaAtmos.

If CountFlops.@count_ops can be turned into a test for Float32/Float64 type stability, we could potentially use that test instead of the restrictions on FT throughout this repo. The resulting code would be more generic and readable, and we wouldn't need to sacrifice any type safety. Such a test should work despite the aforementioned optimization issue, since it does not need to depend on the exact number of Float32/Float64 operations.