You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As noted by @jameskerr in #1786 (comment), the Brim app has changed its behavior when a user attempts to import a mix of valid and invalid data, and we should clean this up.
The first video looks back at the behavior as of the last GA Brim release v0.24.0. Here a combination of a valid Zeek weird log and invalid macOS /etc/passwd are simultaneously dragged into the app.
v0.24.0.mp4
Here we observe:
The valid weird records are successfully imported into a newly-created Space.
The invalid /etc/passwd data is rejected.
The user is shown the warnings returned by the Zed backend from the failed attempt to auto-detect the bad input, but there's no indication of which filename triggered the warning, so this information is admittedly a bit lacking.
The second video captures the current behavior as of Brim commit 185344d, with the same pair of files being dragged into the app.
Repro.mp4
Here we observe:
The entire import operation is aborted, such that the Pool is deleted and even the valid records are not imported.
As in v0.24.0, the error message lacks reference to named files, making it difficult to make use of this error info if multiple invalid files were being imported.
Our approach to mixing valid and invalid data changed with the addition of the Preview & Load workflow in #2834. As shown in the attached video, now the use of zq for preview and shaping gives an early error about invalid data before the final load operation is attempted. The error only mentions the first invalid input file encountered, but this gives the user the ability to review the problem and see if it could be corrected (e.g., select an explicit input format if it's an auto-detect failure) or drop it from the import list entirely. In this case I intentionally used two copies of the /etc/passwd data to show that the file referenced in the error output changed after I dropped the first one and then after dropping the second we see just the valid data that remains for potential shaping and load.
Verify.mp4
This approach takes the place of the previously-envisioned idea of quietly importing just valid data and reporting invalid data after the fact. It seems a good improvement since it gives the user more up-front control and eliminates the potential for a partial load being undesirable and therefore requiring cleanup of the pool after the fact.
As noted by @jameskerr in #1786 (comment), the Brim app has changed its behavior when a user attempts to import a mix of valid and invalid data, and we should clean this up.
The first video looks back at the behavior as of the last GA Brim release
v0.24.0
. Here a combination of a valid Zeekweird
log and invalid macOS/etc/passwd
are simultaneously dragged into the app.v0.24.0.mp4
Here we observe:
weird
records are successfully imported into a newly-created Space./etc/passwd
data is rejected.The second video captures the current behavior as of Brim commit 185344d, with the same pair of files being dragged into the app.
Repro.mp4
Here we observe:
v0.24.0
, the error message lacks reference to named files, making it difficult to make use of this error info if multiple invalid files were being imported.In #1786 (comment), @jameskerr made his proposal of what should be done:
...and to this I'd add my own suggestion:
The text was updated successfully, but these errors were encountered: