-
Notifications
You must be signed in to change notification settings - Fork 440
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add logging #2512
add logging #2512
Conversation
I'm okay with this (once tests pass) but do want to see the docs. I especially need to know how developers decide what's worth logging. |
ok, docs and hopefully a fix for efp is up.
I don't have a good answer. On the DDD side, stuff goes in the log that no longer appends nicely to the output (because its run in a separate process), so logging is on the generous side. In general, the stuff we want to log doesn't map evenly to the (critical=50, error=40, warn=30, info=20, debug=10) levels that Python sets up. Something more like https://www.ibm.com/docs/en/cognos-analytics/10.2.2?topic=SSEP7J_10.2.2/com.ibm.swg.ba.cognos.ug_rtm_wb.10.2.2.doc/c_n30e74.html with "debug-low" etc. might be better. After all, geometry of the molecule is pretty essential, but it's not "progress of the job", so I called it debug. For the moment, log contents are a free-for-all, so I've tried to make that clear in the docs. |
Can you elaborate on how DDD affects our ability to output to file? |
Sure. For everything but analytic single points (effectively, any composite, finite difference, or many-body expansion), the individual pieces are sent to qcengine as a qcschema atomicinput job that returns a qcschema atomicresult (that has the usual output file printout as a field). So all the usual output is collected, but some of the most important stuff (the atomicresult) is in a dict, which would be weird for an output file. Also, the psi4 output file isn't a wonderfully behaved object for being opened and closed and reset, potentially for dozens of jobs running at the same time. It looks like the individual pieces outputs are being accumulated in the outfile now in DDD (see below). But logging is a natural object for accumulating the results of potentially multiple libraries into one file or stream, so it seems the right trajectory to get started along for distributed calcs.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the right call is to get this in, and then once DDD is in, encourage devs to start experimenting with logging. We can try to standardize when the devs have more experience.
Generally I'd like the default log level to be more verbose and spam the log file instead of having something crucial missing and the user needs to repeat a calculation. That said, I haven't used the internal python logger object and these high numbers are and multiple categories unexpected and for a compchem program a bit confusing to be honest. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for breaking DDD into these smaller PRs!
-------------------------- | ||
|
||
A log file is generated alongside the output file and has extension | ||
``.log``. Selected key information is sent to it. The log file contents |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there anything more that can be said about the contents of the .log file than "selected key information"? I understand that logging is new, and there might not be a concrete answer until DDD is fully merged.
However, I think it would be really good to differentiate between the purpose and contents of the .log and .out files: Are the two files redundant? What kind of information should users expect to find in each file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question! I've added a little more, but both DDD and I are unclear on the answer. :-)
* add logger * logging and efp * Apply suggestions from code review * Update build_planning.rst
* add logger * logging and efp * Apply suggestions from code review * Update build_planning.rst
Description
DDD in #1351 is nominally ready, but looking through the diff, I don't think I can answer close questioning on all the pieces. So I'm going to try to pick off some topics for coherent discussion.
This is No. 1 of the DDD series, #1351.
Todos
psi4.set_output_file
function to both set up a logger and callpsi4.core.set_output_file
Below is an example of output set at DEBUG with an extra
gradient()
call added to tu1>>> psi4 ../tests/tu1-h2o-energy/input.dat --loglevel=10 && cat ../tests/tu1-h2o-energy/input.log
Questions
Checklist
Tests added for any new featuresStatus