Initial Commit

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,33 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: "[Bug]"
+labels: bug, todo
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**System (please complete the following information):**
+ - Splunk version heavy forwarder: [e.g. 7.1.2]
+ - Splunk version deployment server: [e.g. 7.1.2]
+ - Splunk cluster: [e.g. yes, no]
+ - TA version [e.g. 1.0.beta]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,29 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: "[Feat] "
+labels: enhancement, to discuss, todo
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+# Why: Why you want the feature
+A clear and concise description of why you want this feature. 
+
+# What: What you want to happen
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+
+**Additional context**
+Add any other context or screenshots about the feature request here.
+
+
+# How: How it is implemented
+- [ ] Task 1

CHANGELOG.md

@@ -0,0 +1,27 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+### Added (for new features)
+- Published app in Splunkbase
+- Extend logging capabilities
+### Changed (for changes in existing functionality)
+\-
+### Deprecated (for soon-to-be removed features)
+- Removed seq.json file and switch to Splunk checkpoints
+### Removed (for now removed features)
+\-
+### Fixed (for any bug fixes)
+\-
+### Security (in case of vulnerabilities)
+\-
+
+## [1.0.0] - 2019-03-12
+### Added (for new features)
+- Published Splunk technical addon in version 1.0.0 at github.com
+
+
+[Unreleased]: https://github.com/dcso/TIE-Splunk-TA/compare/v1.0.0...HEAD

LICENSE

@@ -0,0 +1,29 @@
+BSD 3-Clause License
+
+Copyright (c) 2015, 2019, Deutsche Cyber-Sicherheitsorganisation
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+
+* Neither the name of the copyright holder nor the names of its
+  contributors may be used to endorse or promote products derived from
+  this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README.md

@@ -0,0 +1,61 @@
+DCSO Threat Intelligence Engine (TIE) Technical Add-on for Splunk
+==================================================================
+Splunk technical add-on (TA) for DCSO Threat Intelligence Engine (TIE).
+
+Copyright (c) 2015, 2019, DCSO Deutsche Cyber-Sicherheitsorganisation GmbH
+
+# 1. Prerequisites and Installation
+* The default python major version of 
+  * 7.1.2 is Python 2.7.x
+  * 7.2.4 is Python 2.7.x
+* All required python packages are pre-installed from Splunk itself. 
+  * If any package is missing please open an issue to us and try to do: `pip install -r requirements.txt --no-cache`
+
+## 1.1 Prerequisites
+* Splunk
+* Customer for the DCSO TI-Aggregation Package
+* Generate an Token in the settings page of [TIE web interface](https://tie.dcso.de) with the following privileges:
+  * tie
+  * tie:pingback
+* Firewall Requirements
+
+    | Source                           | Destination | Protocol | Port | Comment    |
+    | -------------------------------- | ----------- | -------- | ---- | ---------- |
+    | \<Your Splunk server IP with the installed TA\> | tie.dcso.de | TCP      | 443  | API access |
+
+
+## 1.2 Installation
+This app must be installed on a **Heavy Forwarder** with an internet connection to reach the  [API](https://tie.dcso.de). 
+
+# 2. Configuration
+
+## 2.1 Splunk Setup Page
+An access token is required for the API access. If you are already a customer and do not have one, please do not hesitate to contact us. If you are not a customer yet, please feel free to contact us for a demo account.
+
+Contact Mail: ti-support [a] dcso.de
+
+The token has to be configured in the setup page of the technical add-on on the Splunk HF. You also have to enable the script by `unchecking` the "tie2index.py" box. There are also options for the schedule and the Index where the IoC's are stored. The Index must be known on the HF.
+
+## 2.2 Standard Filter
+
+The default settings for the filter you find in default/dcso_tie_setup.conf
+
+
+# 3. Usage
+
+## 3.1 Getting the IoCs
+
+### 3.1.1 tie2index
+
+The input script tie2index.py will automatically start with the oldest IoC in a 30 day range. From that it will iterate and index all updates made. The intervall is by default 10 minutes. All IoC and their update will be stored in an index (default: dcso_app_tie-api). We recommend at least 180 days as retention time for this index. From this index all lookups and files can be derived.
+
+To limit the used licence volume we only index IoCs within specified confidence and severity ranges. The ranges in the filter mentioned above are default.
+
+
+# Contact
+Mail: ti-support [a] dcso.de
+
+Website: https://dcso.de
+
+# License
+Please have a look at the LICENSE file included in the repository.

bin/dcso_tie_filter_handler.py

@@ -0,0 +1,93 @@
+# Copyright (c) 2017, 2019, DCSO GmbH
+
+import splunk.admin as admin
+import splunk.entity as en
+import re
+# import your required python modules
+
+'''
+Copyright (C) 2005 - 2010 Splunk Inc. All Rights Reserved.
+Description:  This skeleton python script handles the parameters in the configuration page.
+
+      handleList method: lists configurable parameters in the configuration page
+      corresponds to handleractions = list in restmap.conf
+
+      handleEdit method: controls the parameters and saves the values
+      corresponds to handleractions = edit in restmap.conf
+
+'''
+
+class ConfigApp(admin.MConfigHandler):
+  '''
+  Set up supported arguments
+  '''
+  def setup(self):
+    if self.requestedAction == admin.ACTION_EDIT:
+      for arg in ['ip_confidence', 'ip_severity', 'dom_confidence', 'dom_severity', 'url_confidence', 'url_severity', 'email_confidence', 'email_severity', 'confidence', 'severity']:
+        self.supportedArgs.addOptArg(arg)
+
+  '''
+  Read the initial values of the parameters from the custom file
+      myappsetup.conf, and write them to the setup page.
+
+  If the app has never been set up,
+      uses .../app_name/default/myappsetup.conf.
+
+  If app has been set up, looks at
+      .../local/myappsetup.conf first, then looks at
+  .../default/myappsetup.conf only if there is no value for a field in
+      .../local/myappsetup.conf
+
+  For boolean fields, may need to switch the true/false setting.
+
+  For text fields, if the conf file says None, set to the empty string.
+  '''
+
+  def handleList(self, confInfo):
+    confDict = self.readConf("dcso_tie_setup")
+    if None != confDict:
+      for stanza, settings in confDict.items():
+        for key, val in settings.items():
+          if key in ['ip_confidence', 'ip_severity', 'dom_confidence', 'dom_severity', 'url_confidence', 'url_severity', 'email_confidence', 'email_severity', 'confidence', 'severity'] and val in [None, '']:
+            val = ''
+          confInfo[stanza].append(key, val)
+
+  '''
+  After user clicks Save on setup page, take updated parameters,
+  normalize them, and save them somewhere
+  '''
+  def handleEdit(self, confInfo):
+    name = self.callerArgs.id
+    args = self.callerArgs
+
+    if self.callerArgs.data['ip_confidence'][0] is None:
+      self.callerArgs.data['ip_confidence'][0] = ''
+    if self.callerArgs.data['ip_severity'][0] is None:
+      self.callerArgs.data['ip_severity'][0] = ''
+    if self.callerArgs.data['dom_confidence'][0] is None:
+      self.callerArgs.data['dom_confidence'][0] = ''
+    if self.callerArgs.data['dom_severity'][0] is None:
+      self.callerArgs.data['dom_severity'][0] = ''
+    if self.callerArgs.data['url_confidence'][0] is None:
+      self.callerArgs.data['url_confidence'][0] = ''
+    if self.callerArgs.data['url_severity'][0] is None:
+      self.callerArgs.data['url_severity'][0] = ''
+    if self.callerArgs.data['email_confidence'][0] is None:
+      self.callerArgs.data['email_confidence'][0] = ''
+    if self.callerArgs.data['email_severity'][0] is None:
+      self.callerArgs.data['email_severity'][0] = '' 
+    if self.callerArgs.data['confidence'][0] is None:
+      self.callerArgs.data['confidence'][0] = '' 
+    if self.callerArgs.data['severity'][0] is None:
+      self.callerArgs.data['severity'][0] = ''
+
+    '''
+    Since we are using a conf file to store parameters,
+write them to the [setupentity] stanza
+    in app_name/local/myappsetup.conf
+    '''
+
+    self.writeConf('dcso_tie_setup', 'filter', self.callerArgs.data)
+
+# initialize the handler
+admin.init(ConfigApp, admin.CONTEXT_NONE)

bin/dcso_tie_proxy_handler.py

@@ -0,0 +1,83 @@
+# Copyright (c) 2017, 2019, DCSO GmbH
+
+import splunk.admin as admin
+import splunk.entity as en
+import re
+# import your required python modules
+
+'''
+Copyright (C) 2005 - 2010 Splunk Inc. All Rights Reserved.
+Description:  This skeleton python script handles the parameters in the configuration page.
+
+      handleList method: lists configurable parameters in the configuration page
+      corresponds to handleractions = list in restmap.conf
+
+      handleEdit method: controls the parameters and saves the values
+      corresponds to handleractions = edit in restmap.conf
+
+'''
+
+class ConfigApp(admin.MConfigHandler):
+  '''
+  Set up supported arguments
+  '''
+  def setup(self):
+    if self.requestedAction == admin.ACTION_EDIT:
+      for arg in ['host','port','user','password']:
+        self.supportedArgs.addOptArg(arg)
+
+  '''
+  Read the initial values of the parameters from the custom file
+      myappsetup.conf, and write them to the setup page.
+
+  If the app has never been set up,
+      uses .../app_name/default/myappsetup.conf.
+
+  If app has been set up, looks at
+      .../local/myappsetup.conf first, then looks at
+  .../default/myappsetup.conf only if there is no value for a field in
+      .../local/myappsetup.conf
+
+  For boolean fields, may need to switch the true/false setting.
+
+  For text fields, if the conf file says None, set to the empty string.
+  '''
+
+  def handleList(self, confInfo):
+    confDict = self.readConf("dcso_tie_setup")
+    if None != confDict:
+      for stanza, settings in confDict.items():
+        for key, val in settings.items():
+          if key in ['host','port','user','password'] and val in [None, '']:
+            val = ''
+          confInfo[stanza].append(key, val)
+
+  '''
+  After user clicks Save on setup page, take updated parameters,
+  normalize them, and save them somewhere
+  '''
+  def handleEdit(self, confInfo):
+    name = self.callerArgs.id
+    args = self.callerArgs
+
+    if self.callerArgs.data['host'][0] is None:
+      self.callerArgs.data['host'][0] = ''
+    if self.callerArgs.data['port'][0] is None:
+      self.callerArgs.data['port'][0] = ''
+    if self.callerArgs.data['user'][0] is None:
+      self.callerArgs.data['user'][0] = ''
+    if self.callerArgs.data['password'][0] is None:
+      self.callerArgs.data['password'][0] = ''
+
+
+
+    '''
+    Since we are using a conf file to store parameters,
+write them to the [setupentity] stanza
+    in app_name/local/myappsetup.conf
+    '''
+
+    self.writeConf('dcso_tie_setup', 'proxy', self.callerArgs.data)
+
+# initialize the handler
+admin.init(ConfigApp, admin.CONTEXT_NONE)