Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(experimental): kick off inventory project #808

Merged
merged 7 commits into from
Dec 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/experimental-inventory-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: CI - experimental - inventory

on:
push:
branches: [main]
paths:
- experimental/license-inventory/**
pull_request:
branches: [main]
paths:
- experimental/license-inventory/**

permissions:
pull-requests: write

jobs:
build:
runs-on: ubuntu-latest

strategy:
matrix:
node-version: [18.x]
mongodb-version: [4.4]

steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4
with:
fetch-depth: 0

- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}

- name: Start MongoDB
uses: supercharge/mongodb-github-action@1.11.0
with:
mongodb-version: ${{ matrix.mongodb-version }}

- name: Install dependencies
working-directory: ./experimental/license-inventory
run: npm ci

- name: Test
working-directory: ./experimental/license-inventory
run: |
MONGO_URI="mongodb://localhost:27017/inventory" npm run test
27 changes: 27 additions & 0 deletions .github/workflows/experimental-inventory-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: experimental-inventory - Publish to NPM
on:
push:
tags:
- 'license-inventory-*'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4

# Setup .npmrc file to publish to npm
- uses: actions/setup-node@v4
with:
node-version: '18.x'
registry-url: 'https://registry.npmjs.org'

- run: npm ci
working-directory: ./experimental/license-inventory

- run: npm run build
working-directory: ./experimental/license-inventory

- run: npm publish --access=public
working-directory: ./experimental/license-inventory
env:
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
2 changes: 2 additions & 0 deletions experimental/license-inventory/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules
dist
31 changes: 31 additions & 0 deletions experimental/license-inventory/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<br />
<div align="center">
<a href="https://github.com/finos/git-proxy">
<img src="../../docs/img/logo.png" alt="Logo" height="95">
</a>

<br />
<strong>git-proxy: license-inventory</strong>
<br />

<p align="center">
<strong>experimental</strong> project to provide license data and attach said license data to projects for fueling policy evaluation.
<br />
</p>

<br />

[![FINOS - Incubating](https://cdn.jsdelivr.net/gh/finos/contrib-toolbox@master/images/badge-incubating.svg)](https://community.finos.org/docs/governance/Software-Projects/stages/incubating)
[![NPM](https://img.shields.io/npm/v/@finos/git-proxy-license-inventory?colorA=00C586&colorB=000000)](https://www.npmjs.com/package/@finos/git-proxy-license-inventory)
[![Build](https://img.shields.io/github/actions/workflow/status/finos/git-proxy/experimental-inventory-ci.yml?branch=main&label=CI&logo=github&colorA=00C586&colorB=000000)](https://github.com/finos/git-proxy/actions/workflows/experimental-inventory-ci.yml)
[![git-proxy](https://api.securityscorecards.dev/projects/github.com/finos/git-proxy/badge)](https://api.securityscorecards.dev/projects/github.com/finos/git-proxy)
<br />
[![License](https://img.shields.io/github/license/finos/git-proxy?colorA=00C586&colorB=000000)](https://github.com/finos/git-proxy/blob/main/LICENSE)
[![Slack](https://img.shields.io/badge/_-Chat_on_Slack-000000.svg?logo=slack&colorA=00C586)](https://app.slack.com/client/T01E7QRQH97/C06LXNW0W76)

</div>
<br />

This is an **experimental** project to provide license data and attach said license data to projects for fueling policy evaluation.

Please consider all REST API paths, inputs, and outputs in flux during `v0`. Additionally do not rely import paths to be stable.
43 changes: 43 additions & 0 deletions experimental/license-inventory/dev/grafana/datasources.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
apiVersion: 1

datasources:
- name: Loki
type: loki
uid: loki
access: proxy
url: http://loki:3100
basicAuth: false
version: 1
editable: false
isDefault: true

- name: Prometheus
type: prometheus
uid: prometheus
access: proxy
orgId: 1
url: http://prometheus:9090
basicAuth: false
isDefault: false
version: 1
editable: false
jsonData:
httpMethod: GET

- name: Tempo
type: tempo
uid: tempo
access: proxy
orgId: 1
url: http://tempo:3200
basicAuth: false
isDefault: false
version: 1
editable: false
jsonData:
tracesToLogsV2:
datasourceUid: loki
filterByTraceID: true
filterBySpanID: true
tracesToMetrics:
datasourceUid: prometheus
49 changes: 49 additions & 0 deletions experimental/license-inventory/dev/loki.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
auth_enabled: false

server:
http_listen_port: 3100
grpc_listen_port: 9096

common:
instance_addr: 127.0.0.1
path_prefix: /tmp/loki
storage:
filesystem:
chunks_directory: /tmp/loki/chunks
rules_directory: /tmp/loki/rules
replication_factor: 1
ring:
kvstore:
store: inmemory

frontend:
max_outstanding_per_tenant: 2048

pattern_ingester:
enabled: true

limits_config:
max_global_streams_per_user: 0
ingestion_rate_mb: 50000
ingestion_burst_size_mb: 50000
volume_enabled: true

query_range:
results_cache:
cache:
embedded_cache:
enabled: true
max_size_mb: 100

schema_config:
configs:
- from: 2020-10-24
store: tsdb
object_store: filesystem
schema: v13
index:
prefix: index_
period: 24h

analytics:
reporting_enabled: false
52 changes: 52 additions & 0 deletions experimental/license-inventory/dev/otel-collector-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
extensions:
health_check:
pprof:
endpoint: 127.0.0.1:1777
zpages:
endpoint: 127.0.0.1:55679

receivers:
otlp:
protocols:
http:
endpoint: 0.0.0.0:4318

# collect own metrics
prometheus:
config:
scrape_configs:
- job_name: otel-collector
scrape_interval: 10s
static_configs:
- targets:
- 127.0.0.1:8888

exporters:
loki:
endpoint: http://loki:3100/loki/api/v1/push
prometheus:
endpoint: 127.0.0.1:8889
namespace: otel-collector
send_timestamps: true
metric_expiration: 180m
enable_open_metrics: true
add_metric_suffixes: false
resource_to_telemetry_conversion:
enabled: true
otlphttp:
endpoint: http://tempo:4318

processors:
batch:

service:
pipelines:
traces:
receivers: [otlp]
exporters: [otlphttp]
metrics:
receivers: [otlp]
exporters: [prometheus]
logs:
receivers: [otlp]
exporters: [loki]
20 changes: 20 additions & 0 deletions experimental/license-inventory/dev/prometheus.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
global:
scrape_interval: 15s
evaluation_interval: 15s

scrape_configs:
- job_name: prometheus
static_configs:
- targets: [ 'localhost:9090' ]
- job_name: tempo
static_configs:
- targets: [ 'tempo:3200' ]
- job_name: promtail
static_configs:
- targets: [ 'promtail:9080' ]
- job_name: otel-collector
static_configs:
- targets: [ 'otel-collector:8889' ]
- job_name: otel-collector-meta
static_configs:
- targets: [ 'otel-collector:8888' ]
14 changes: 14 additions & 0 deletions experimental/license-inventory/dev/promtail.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
positions:
filename: /tmp/positions.yaml

clients:
- url: http://loki:3100/loki/api/v1/push

scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: mongo
__path__: /var/log-mongo/mongo.log*
40 changes: 40 additions & 0 deletions experimental/license-inventory/dev/tempo.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
stream_over_http_enabled: true
server:
http_listen_port: 3200

distributor:
receivers:
otlp:
protocols:
http:

compactor:
compaction:
block_retention: 1h

metrics_generator:
registry:
external_labels:
source: tempo
cluster: docker-compose
storage:
path: /var/tempo/generator/wal
remote_write:
- url: http://prometheus:9090/api/v1/write
send_exemplars: true
traces_storage:
path: /var/tempo/generator/traces

storage:
trace:
backend: local
wal:
path: /var/tempo/wal
local:
path: /var/tempo/blocks

overrides:
defaults:
metrics_generator:
processors: [service-graphs, span-metrics, local-blocks]
generate_native_histograms: both
Loading
Loading