Enable redline CSV support in upload #642
Conversation
| redline_intersection = set( | ||
| redline_fields).intersection(set(csv_header)) | ||
| if len(redline_fields) == len(redline_intersection): | ||
| return u'redline' |
There was a problem hiding this comment.
If I understand what you're trying to achieve, I think set(redline_fields).issubset(set(csv_header)) has the same effect and is more readable.
| timesketch_intersection = timesketch_fields.intersection( | ||
| set(csv_header)) | ||
| if len(timesketch_fields) == len(timesketch_intersection): | ||
| return u'timesketch' |
| # check if it is the right redline format | ||
| mandatory_fields = [u'Alert', u'Tag', u'Timestamp', u'Field', u'Summary'] | ||
| Returns: | ||
| Generator of event rows |
There was a problem hiding this comment.
Would also be useful to know what type event rows are.
| yield row_to_yield | ||
| csv_dialect = get_csv_dialect(csv_header) | ||
|
|
||
| if u'redline' in csv_dialect: |
There was a problem hiding this comment.
it seems that get_csv_dialect returns a string, not an array of strings. I think you want to use csv_dialect == 'redline'. Also, this will fail if csv_dialect is None.
| continue | ||
| yield row | ||
| else: | ||
| raise RuntimeError(u'Unknown CSV format') |
There was a problem hiding this comment.
You can turn this into a guard statement before the other checks
if csv_dialect is None:
raise RuntimeError()
if csv_dialect == 'timesketch':
...
if csv_dialect == 'redline':
...
|
This PR got lost in some other updates. This PR is partly obsolete and the validation of CSVs are being reworked. I'm dropping this one as thing are changing soon. Will enable redline uploads soon. |
Refactor the CSV importer to support Redline CSV as well.