[FileBeat] IIS module: support for X-Forwarded-For #19142
Conversation
|
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
1 similar comment
|
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
botelastic
bot
added
the
needs_team
|
💚 CLA has been signed |
❕ Build Aborted
Expand to view the summary
Build stats
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
ycombinator
changed the title
|
run tests |
botelastic
bot
removed
the
needs_team
|
Pinging @elastic/integrations-platforms (Team:Platforms) |
|
Hi! We're labeling this issue as |
|
Hi! |
|
Please merge this PR. We also have many IIS webservers behind a load balancer and thus really need the X-Forwarded-For information otherwise the logs are totally useless. |
|
Same case as @jeffrysleddens' , this should be merged. |
|
I agree with @jeffrysledden and @nguyeni95, this should be merged! |
|
Ok, I am reopening this, but we would need to add some example test files, @marcosdiez could you add a example log file for testing? Thanks! |
jsoriano
added
the
Team:Services
|
Pinging @elastic/integrations-services (Team:Services) |
|
Is there any update on when we might see this implemented? this would be very useful at my organisation too. From my investigations though this is not going to work for all scenarios as you may find that you have more than one IP.. Update: ((?:-)|(?<source.xforwardfor>(%{IP}([,+]+)?)+))?$ to support the null - field x.x.x.x |
|
@MrBones757 thanks for your feedback, could you provide some example log lines that we could use for testing? |
|
Jamie,
See attached standard IIS log with X-Forward-For field enabled. as last attribute on applicable lines.
[iis_log_sample.log](https://github.com/elastic/beats/files/5348993/iis_log_sample.log)
|
|
Please find attached a sample of IIS logs with all options enabled and X-Forwarded-For added as last attribute. Where 10.24.129.162 is the IP address of our webserver and 10.24.136.240 is the IP address of our loadbalancer. And a screenshot of the IIS log settings used to produce these logs:
|
|
We have been using this modified version of the grok patterns of the IIS ingest pipeline to allow for the X-Forwarded-For field. Which was based on this pull request. We only swapped around the network.forwarded_ip and source.address fields as that seems to be more in line with the ECS definition of those fields. |
|
@NoelProf @jeffrysleddens wow, thanks a lot for sharing your logs and the configurations you are using! Would any of you like to open a pull request with them? If not, no problem, I will wrap-up all the info you have provided and prepare the change. |
|
This is a tricky one, because there is also still a discussion going on about how to handle proxy/loadbalancers in the ECS (elastic/ecs#938) and the network.forwarded_ip field (elastic/ecs#523) |
|
Hi, I finally went back to this. I have opened #24436 to replace this PR. It includes the code of this PR, but making the presence of the Thanks all for your help with this, very appreciated! Please take a look to the new PR if you have some time.
@jeffrysleddens regarding this, in my PR I keep using |
Type of Change: Enhancement
What does this PR do?
On FileBeat, for the Windows IIS logs, this PR adds an extra optional IP address field in the end of every access log line. This IP address is saved as
network.forwarded_ipOn IIS, if one adds the X-Forwarded-For header, it's appended to the end of the logs. This PR is to pick that. Very useful if your windows machine is behind a HTTP proxy/load balancer/firewall
Why is it important?
There is no downside on this and it makes life easier for those who need to capture the X-Forwarded-For header
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Author's Checklist
How to test this PR locally
Related issues
Use cases