Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Better IPv6 support #33

Open
lennartkoopmann opened this issue Mar 26, 2017 · 2 comments
Open

Better IPv6 support #33

lennartkoopmann opened this issue Mar 26, 2017 · 2 comments
Labels
bug triaged

Comments

@lennartkoopmann
Copy link
Contributor

  • Support IPv6 for all existing lookup sources
  • Avoid throwing huge exceptions in methods that only support IPv4 because a user might throw garbage in there.
@lennartkoopmann lennartkoopmann added this to the 1.0 milestone Mar 26, 2017
@lennartkoopmann lennartkoopmann mentioned this issue Mar 26, 2017
Closed
@joschi joschi removed this from the 1.0 milestone Sep 26, 2017
@jalogisch
Copy link
Contributor

This is supported by all backends except alien vault already.

@jalogisch
Copy link
Contributor

The current implementation gives big stack traces for every IPv6 Lookups in the server.log

2018-01-05T16:23:57.747+01:00 WARN  [GuavaLookupCache] Loading value from data adapter failed for key LookupCacheKey{prefix=59ead1dc1e0db84685307b36, key=2a00:1798::1:121}, returning empty result
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Could not parse [2a00:1798::1:121]
	at org.graylog2.lookup.caches.GuavaLookupCache$InstrumentedCache.get(GuavaLookupCache.java:243) ~[graylog.jar:?]
	at org.graylog2.lookup.caches.GuavaLookupCache.get(GuavaLookupCache.java:104) ~[graylog.jar:?]
	at org.graylog2.lookup.LookupTable.lookup(LookupTable.java:72) ~[graylog.jar:?]
	at org.graylog2.lookup.LookupTableService$Function.lookup(LookupTableService.java:534) ~[graylog.jar:?]
	at org.graylog.plugins.threatintel.functions.spamhaus.SpamhausIpLookupFunction.evaluate(SpamhausIpLookupFunction.java:43) ~[?:?]
	at org.graylog.plugins.threatintel.functions.spamhaus.SpamhausIpLookupFunction.evaluate(SpamhausIpLookupFunction.java:16) ~[?:?]
	at org.graylog.plugins.threatintel.functions.global.AbstractGlobalLookupFunction.lambda$matchEntityAgainstFunctions$2(AbstractGlobalLookupFunction.java:44) ~[?:?]
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_151]
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_151]
	at java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet.lambda$entryConsumer$0(Collections.java:1575) ~[?:1.8.0_151]
	at java.util.Iterator.forEachRemaining(Iterator.java:116) [?:1.8.0_151]
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) [?:1.8.0_151]
	at java.util.Collections$UnmodifiableMap$UnmodifiableEntrySet$UnmodifiableEntrySetSpliterator.forEachRemaining(Collections.java:1600) [?:1.8.0_151]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) [?:1.8.0_151]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) [?:1.8.0_151]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) [?:1.8.0_151]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [?:1.8.0_151]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) [?:1.8.0_151]
	at org.graylog.plugins.threatintel.functions.global.AbstractGlobalLookupFunction.matchEntityAgainstFunctions(AbstractGlobalLookupFunction.java:48) [graylog-plugin-threatintel-2.4.0.jar:?]
	at org.graylog.plugins.threatintel.functions.global.GlobalIpLookupFunction.evaluate(GlobalIpLookupFunction.java:61) [graylog-plugin-threatintel-2.4.0.jar:?]
	at org.graylog.plugins.threatintel.functions.global.GlobalIpLookupFunction.evaluate(GlobalIpLookupFunction.java:23) [graylog-plugin-threatintel-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.ast.expressions.FunctionExpression.evaluateUnsafe(FunctionExpression.java:63) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.ast.expressions.Expression.evaluate(Expression.java:41) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.ast.statements.VarAssignStatement.evaluate(VarAssignStatement.java:33) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.ast.statements.VarAssignStatement.evaluate(VarAssignStatement.java:22) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.processors.PipelineInterpreter.evaluateStatement(PipelineInterpreter.java:377) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.processors.PipelineInterpreter.executeRuleActions(PipelineInterpreter.java:364) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.processors.PipelineInterpreter.evaluateStage(PipelineInterpreter.java:305) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.processors.PipelineInterpreter.processForResolvedPipelines(PipelineInterpreter.java:263) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.processors.PipelineInterpreter.process(PipelineInterpreter.java:143) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog.plugins.pipelineprocessor.processors.PipelineInterpreter.process(PipelineInterpreter.java:99) [graylog-plugin-pipeline-processor-2.4.0.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.handleMessage(ProcessBufferProcessor.java:114) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.dispatchMessage(ProcessBufferProcessor.java:100) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:77) [graylog.jar:?]
	at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.onEvent(ProcessBufferProcessor.java:42) [graylog.jar:?]
	at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:143) [graylog.jar:?]
	at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66) [graylog.jar:?]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: java.lang.IllegalArgumentException: Could not parse [2a00:1798::1:121]
	at org.apache.commons.net.util.SubnetUtils.toInteger(SubnetUtils.java:287) ~[?:?]
	at org.apache.commons.net.util.SubnetUtils.access$400(SubnetUtils.java:27) ~[?:?]
	at org.apache.commons.net.util.SubnetUtils$SubnetInfo.isInRange(SubnetUtils.java:125) ~[?:?]
	at org.graylog.plugins.threatintel.adapters.spamhaus.SpamhausEDROPDataAdapter.lambda$doGet$1(SpamhausEDROPDataAdapter.java:157) ~[?:?]
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174) ~[?:1.8.0_151]
	at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) ~[?:1.8.0_151]
	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_151]
	at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:270) ~[?:1.8.0_151]
	at java.util.stream.StreamSpliterators$WrappingSpliterator.tryAdvance(StreamSpliterators.java:302) ~[?:1.8.0_151]
	at com.google.common.collect.CollectSpliterators$1WithCharacteristics.tryAdvance(CollectSpliterators.java:60) ~[graylog.jar:?]
	at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126) ~[?:1.8.0_151]
	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:498) ~[?:1.8.0_151]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:485) ~[?:1.8.0_151]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_151]
	at java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:152) ~[?:1.8.0_151]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_151]
	at java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:464) ~[?:1.8.0_151]
	at org.graylog.plugins.threatintel.adapters.spamhaus.SpamhausEDROPDataAdapter.doGet(SpamhausEDROPDataAdapter.java:158) ~[?:?]
	at org.graylog2.plugin.lookup.LookupDataAdapter.get(LookupDataAdapter.java:123) ~[graylog.jar:?]
	at org.graylog2.lookup.LookupTable.lambda$lookup$0(LookupTable.java:72) ~[graylog.jar:?]
	at org.graylog2.lookup.caches.GuavaLookupCache$InstrumentedCache.get(GuavaLookupCache.java:239) ~[graylog.jar:?]
	... 37 more

@lennartkoopmann lennartkoopmann added this to the 3.0.0 milestone Jan 5, 2018
@jekelundh jekelundh mentioned this issue Feb 28, 2018
Closed
@bernd bernd removed this from the 3.0.0 milestone Nov 16, 2018
kroepke added a commit that referenced this issue Mar 23, 2020
@kroepke
reduce error log noise by suppressing the stacktrace
2c75595 
extended the ip subnet check to ignore all IPv6 addresses be returning "false" for all of them, even unique local addresses
subnet check is not faster
only non-IP addresses will lead to a log message now, IPv6 checks will be silent

fixes #156
fixes Graylog2/graylog2-server#4624
related to #33
@kroepke kroepke mentioned this issue Mar 23, 2020
Merged
mpfz0r pushed a commit that referenced this issue Aug 17, 2020
@kroepke @mpfz0r
reduce error log noise by suppressing the stacktrace
abc08c0 
extended the ip subnet check to ignore all IPv6 addresses be returning "false" for all of them, even unique local addresses
subnet check is not faster
only non-IP addresses will lead to a log message now, IPv6 checks will be silent

fixes #156
fixes Graylog2/graylog2-server#4624
related to #33
mpfz0r pushed a commit that referenced this issue Aug 17, 2020
@kroepke @mpfz0r
reduce error log noise by suppressing the stacktrace
02d5626 
extended the ip subnet check to ignore all IPv6 addresses be returning "false" for all of them, even unique local addresses
subnet check is not faster
only non-IP addresses will lead to a log message now, IPv6 checks will be silent

fixes #156
fixes Graylog2/graylog2-server#4624
related to #33

(cherry picked from commit abc08c0)
@mpfz0r mpfz0r mentioned this issue Aug 17, 2020
Closed
mpfz0r pushed a commit that referenced this issue Oct 14, 2020
@kroepke @mpfz0r
Extend in_private_net to check for unique local addresses in IPv6
73bf9ea 
This reduces error log noise by suppressing the stacktrace

fixes #156
fixes Graylog2/graylog2-server#4624
related to #33

(cherry picked from commit abc08c0)

extend `in_private_net` to check for unique local addresses in IPv6

(cherry picked from commit 6c3f617)

use Graylog's IpSubnet class instead of ancient netty one

suppress harmless API stability warning

(cherry picked from commit 91d4f64)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug triaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@bernd @lennartkoopmann @joschi @jalogisch