Feature rework install log processing #240
Conversation
… into feature_rework_checkin
|
Also, I should add that this removes the assumption that base64 decoding will have to occur during report checkins for the main checkin view and the install log; but it still will base64 decode if called for. @grahamgilbert is going to update sal-scripts to only bz2 compress the reports. base64 encoding results in a size gain of about 33%, plus the processing time, so this should be a minor improvement in checkin speed. |
Since we're delegating this task to other functions, we can just get with a default of an empty list, and all clients will be fine.
This is pending a decision about whether to try to optimize the non-postgres situations (instead of delete all, create all, do get_or_create and clear out missing).
There's no good way to do all of this with a single model until at the very least we get Subquery/OuterRef in Django2. At least here I've got it optimized and factored into separate functions. Next I'll remove the duplication.
sheagcraig
force-pushed
the
feature_rework_checkin
branch
from
abcff85
to
47116cd
Compare
This adds the feature that UpdateHistory will create an initial item if there isn't one to begin with. This may not indeed be the actual install date, but we don't have that information anyway.
If an UpdateHistory is no longer a managed item (created every checkin run from the Munki ManagedInstalls report), put it into a list for later processing. Then, for each of those, if it has no history items, delete it. Also, if the most recent history item is older than our `HISTORICAL_DAYS` setting, delete it.
|
Okay, done adding stuff. Please test and review! To summarize, this PR updates the machine checkin procedure, both to address performance concerns, and to clean up code that has aged. As suggested, Sal will generate Several now-unused methods on the The
If an UpdateHistory has been created, checkin will now create an initial UpdateHistoryItem with whichever entry in the report triggered the UpdateHistory creation. If an UpdateHistory ever has zero UpdateHistoryItems associated with it at the end of this function, they will be removed. Likewise, items that no longer are considered ManagedInstalls and have not had an UpdateHistoryItem created since There's a little timezone fix for plugin data submissions to add a timezone if parsing the one submitted results in a naive datetime. Checking for request method == POST now uses Django decorators. Wherever possible, raw_delete and bulk_create have been used to speed up checkin. |
|
Cool. |
This reverts commit 77a2e36.
To keept his from being another giant one, let's call it here.
Unless someone has some Install.log entries to add to the test, the regex I wrote should grab all of the relevant data from a line and simplifies the processing.
This also adds some TZ info in a few different places so that Django doesn't complain about naive datetimes.