Skip to content

Commit

Permalink
add istio installation
Browse files Browse the repository at this point in the history
  • Loading branch information
@ekneg54
ekneg54 committed Jul 15, 2024
1 parent 9389bcc commit 000b5b2
Show file tree
Hide file tree
Showing 5 changed files with 179 additions and 52 deletions.
39 changes: 39 additions & 0 deletions doc/source/examples/minikube.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ with the following commands:
:caption: Configure and start minikube
minikube config set driver docker
minikube config set cpus 16
minikube config set memory 16GB
minikube start
Deploy the example
Expand All @@ -70,6 +72,43 @@ Then you have to update and build the helm subcharts repository:
helm dependencies update ./examples/k8s
helm dependencies build ./examples/k8s
Then install istio (for details see: `https://istio.io/latest/docs/setup/install/helm/`_. ):

.. code-block:: bash
:caption: Create the istio-system namespace
kubectl create namespace istio-system
.. code-block:: bash
:caption: Install istio
helm repo add istio https://istio-release.storage.googleapis.com/charts
helm repo update
helm install istio-base istio/base -n istio-system --set defaultRevision=opensiem --wait
helm install istiod istio/istiod -n istio-system --wait
.. code-block:: bash
:caption: Install istio ingress gateway
kubectl create namespace istio-ingress
helm install istio-ingress istio/gateway -n istio-ingress
.. code-block:: bash
:caption: Verifiy the istio installation
❯ helm ls -n istio-system
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
istio-base istio-system 1 2024-07-15 14:54:54.029747408 +0200 CEST deployed base-1.22.2 1.22.2
istiod istio-system 1 2024-07-15 14:57:41.496783572 +0200 CEST deployed istiod-1.22.2 1.22.2
❯ kubectl get deployments -n istio-system --output wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
istiod 1/1 1 1 24m discovery docker.io/istio/pilot:1.22.2 istio=pilot
❯ kubectl get pods -n istio-ingress
NAME READY STATUS RESTARTS AGE
istio-ingress-7f5f6f58b8-sv6gk 1/1 Running 0 16m
Next you can install the opensiem example using:

Expand Down
6 changes: 6 additions & 0 deletions examples/k8s/Chart.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,12 @@ dependencies:
repository: file://../../charts/logprep
version: 13.1.0
alias: connector
- name: logprep
repository: file://../../charts/logprep
version: 13.1.0
- name: kafka
repository: https://charts.bitnami.com/bitnami
version: 29.3.7
- name: opensearch
repository: https://charts.bitnami.com/bitnami
version: 1.2.6
77 changes: 77 additions & 0 deletions examples/k8s/templates/_helpers.tpl
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "opensiem.name" -}}
{{- printf "%s" .Chart.Name | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
{{- end }}

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "opensiem.fullname" -}}
{{- printf "%s-%s" .Release.Name .Chart.Name | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "opensiem.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{- define "opensiem.release" -}}
{{- printf "%s" .Release.Name | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
{{- end }}

{{/*
Common labels
*/}}
{{- define "opensiem.labels" -}}
helm.sh/chart: {{ include "opensiem.chart" . }}
{{ include "opensiem.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
app.kubernetes.io/application: {{ include "opensiem.name" . }}
{{- range $key, $value := .Values.extraLabels }}
{{ $key}}: {{ $value | quote }}
{{- end }}
{{- end }}

{{/*
{{- end }}

{{/*
Selector labels
*/}}
{{- define "opensiem.selectorLabels" -}}
app.kubernetes.io/name: {{ include "opensiem.fullname" . }}
app.kubernetes.io/instance: {{ include "opensiem.release" . }}
{{- end }}

{{/*
{{- end }}

{{/*
{{- end }}

{{/*
Create the name of the service account to use
*/}}
{{- define "opensiem.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "opensiem.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}

7 changes: 7 additions & 0 deletions examples/k8s/templates/gateway.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: gateway
spec:
selector:
istio: ingressgateway
102 changes: 50 additions & 52 deletions examples/k8s/values.yaml
View file
Original file line number Diff line number Diff line change
@@ -1,31 +1,15 @@
connector:
# The replica count
image:
tag: py3.11-13.0.0
replicas: 1

# The pod resources
resources:
limits:
memory: "2Gi"
cpu: "1"
requests:
memory: "2Gi"
cpu: "250m"

# Optional secrets that will be mounted into the pod
# Listed secrets are handled specially by the logprep deployment.
# Additional secrets will be mounted as usual.
# secrets:
# certificates:
# name: ca-cert # Name of the secret containing the ca certificate (or chain) in one data block
# credentials:
# name: logprep-credentials # Name of the secret containing the logprep credentials file
# imagePullSecret:
# name: logprep-image-pull-secret # Name of the secret containing the image pull secret
secrets: {}

# Logprep logging configuration.
# See: https://logprep.readthedocs.io/en/latest/user_manual/configuration/index.html#configuration-file-structure
# for available configuration options.
logger:
level: DEBUG
input:
Expand All @@ -46,7 +30,6 @@ connector:
/lab/123/(ABC|DEF)/pl.*: plaintext
/lab/123/ABC/auditlog: jsonl
/health: plaintext

output:
type: confluentkafka_output
topic: consumer
Expand All @@ -58,39 +41,6 @@ connector:
compression.type: gzip
statistics.interval.ms: "60000"

# Additional logprep configurations
# Note: The configuration keys 'logger', 'metrics', 'input' and 'output' will overwrite the
# corresponding keys in these configurations as they will be merged into
# the final configuration, after the configurations that are specified here.
# for configurations with data from an external API put the url in the name field and leave the data field undefined
#
# Example:
#
# configurations:
# - name: logprep-config
# data:
# pipeline: []
# - name: https://rule-server.de
#
# See https://logprep.readthedocs.io/en/latest/user_manual/configuration/index.html#configuration-file-structure
# for available configuration options.
configurations:
- name: logprep-config
data:
pipeline: []

# Additional artifacts needed by the logprep deployment. These follow the same mechanic
# as the configurations. The artifacts will be mounted into the pod under the name given
# in the artifacts configuration
# Beware, that configMaps are limited to 1MiB in size.
# Example:
# artifacts:
# - name: admin-list
# data: |
# alfalfa
# admin
# admin2
artifacts: []
## for additional configurations see: `https://github.com/bitnami/charts/blob/main/bitnami/kafka/values.yaml`
kafka:
controller:
Expand All @@ -102,3 +52,51 @@ kafka:
topics:
- consumer
- producer

logprep:
image:
tag: py3.11-13.0.0
input:
type: confluentkafka_input
topic: consumer
kafka_config:
bootstrap.servers: opensiem-kafka:9092
group.id: cgroup3
enable.auto.commit: "true"
auto.commit.interval.ms: "10000"
enable.auto.offset.store: "false"
queued.min.messages: "100000"
queued.max.messages.kbytes: "65536"
statistics.interval.ms: "60000"
preprocessing:
version_info_target_field: Logprep_version_info
log_arrival_time_target_field: event.ingested
hmac:
target: <RAW_MSG>
key: "thisisasecureandrandomkey"
output_field: Full_event
output:
type: opensearch_output
hosts:
- opensiem-opensearch:9200
default_index: processed
error_index: errors
message_backlog_size: 10000
timeout: 10000
flush_timeout: 60
max_retries: 3
parallel_bulk: false
user: admin
secret: admin
## for additional configurations see: `https://github.com/bitnami/charts/blob/main/bitnami/opensearch/values.yaml`
opensearch:
dashboards:
enabled: true
ingest:
replicaCount: 1
master:
replicaCount: 1
data:
replicaCount: 1
coordinating:
replicaCount: 1

0 comments on commit 000b5b2

Please sign in to comment.