logging: Perform filtering on arrays of strings (where possible)

[francislavoie]

Thanks to @IndeedNotJames for reporting this problem.

When trying to filter HTTP headers in the logs, many filters don't work at all because they didn't handle fields that are array values, which all headers are because they can have multiple values.

These changes are mainly for HTTP headers, but it'll work in general for any LoggableStringArray type being logged.

Now, the regexp, hash and ip_mask filters will correctly iterate over each string in the LoggableStringArray and apply their filtering on those.

Additionally, for ip_mask, we now split on commas and mask each segment that parses as an IP address. This should be useful for X-Forwarded-For type headers which may be a list of comma separated IPs.

And finally, I fixed a bug with rename where it would set the zap field type to String, which would totally clobber any array typed fields with an empty string instead. That was just a total oversight when I implemented that filter, because it was basically copy-paste from the replace filter.

As for the other filters, I didn't touch them for various reasons. replace doesn't iterate because it could be a valid usecase to change the type of a field to string... but also maybe not. I dunno about that one, you can probably just use regexp in its place. And for delete, it's not possible to delete a particular entry in an array; we'd need a new filter to do that, which is totally possible, but I'm not sure the usecase (when is there ever a guarantee that a particular index exists?). Also cookie and query already do their job; they're very specifically targeted at the Cookie header (and iterates itself just fine), and the uri field.

[mholt]

Wow, thank you for this fix and for the tests. I haven't tried it myself but I trust it works, given the evidence and who's contributing it :)

LGTM, thanks a bunch!

[pauljeannot]

Hi!

Thanks for the work you did there that is specifically very useful in my workflow today 😊
However I am facing an issue which is not in line with your last assertion:

Also cookie and query already do their job; they're very specifically targeted at the Cookie header (and iterates itself just fine), and the uri field.

Indeed, the query filter is quite useful also in other fields, like the Referer which is basically a uri. As the Referer is part of the Headers, the current implementation does not iterate on its values and thus the query filter does not work as I could have expected it.

I guess that's not a common usecase. The only workaround I see there is to use regexp, which makes it quite heavy.
Do you see any other workaround that would be easier ? Also, Is there an improvement on that part planned soon ?

Thanks a lot!

[francislavoie]

Ah I didn't think of the Referer header. Good point. If you want to contribute a fix, PRs are welcome.

[pauljeannot]

Alright. I will try to contribute asap!

[mholt]

Thanks for your contribution!