Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(observability): Vector tap subcommand #6871

Merged
merged 113 commits into from
Apr 6, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
6427ee5
tap v2
leebenson Feb 19, 2021
a2c16a8
Merge branch 'master' into leebenson/tap2
leebenson Feb 19, 2021
a785f1a
Merge branch 'master' into leebenson/tap2
leebenson Feb 22, 2021
56aeaef
wip
leebenson Feb 22, 2021
ba81014
more wip
leebenson Feb 23, 2021
4a5cbcc
two-way control
leebenson Feb 25, 2021
fffd53d
add component name
leebenson Feb 25, 2021
8f05a84
attach sans detach wip
leebenson Feb 27, 2021
348a30d
fix atomic
leebenson Feb 27, 2021
ff36221
survive reloads
leebenson Mar 1, 2021
49f5585
Merge branch 'master' into leebenson/tap2
leebenson Mar 1, 2021
5d7f29e
componentNames
leebenson Mar 2, 2021
ef24c3f
persistent subscription
leebenson Mar 2, 2021
d2a2865
component not matched
leebenson Mar 3, 2021
5c4cbce
Merge branch 'master' into leebenson/tap2
leebenson Mar 3, 2021
fe83a8b
sample
leebenson Mar 3, 2021
b4585bd
head/tail/range
leebenson Mar 3, 2021
e70295e
wip
leebenson Mar 4, 2021
2b049ff
Merge branch 'master' into leebenson/tap2
leebenson Mar 4, 2021
108012b
refactor for limit
leebenson Mar 4, 2021
9252ac0
bounded chan
leebenson Mar 4, 2021
7055c6c
await
leebenson Mar 4, 2021
df1cea5
fix api tests
leebenson Mar 4, 2021
beaaf09
typos, etc
leebenson Mar 4, 2021
9aaa3f6
period
leebenson Mar 4, 2021
6665414
period
leebenson Mar 4, 2021
1500705
some tests
leebenson Mar 5, 2021
97fd8b5
wildcards
leebenson Mar 5, 2021
6ee7af7
comments
leebenson Mar 5, 2021
5e84552
diff
leebenson Mar 5, 2021
3c79d2a
remove from topology
leebenson Mar 6, 2021
453fad3
tweaks
leebenson Mar 6, 2021
6321a3e
param swap
leebenson Mar 6, 2021
4eb56ac
Merge branch 'master' into leebenson/tap2
leebenson Mar 6, 2021
8fe8508
combine use
leebenson Mar 6, 2021
33fbdf7
Merge branch 'master' into leebenson/tap2
leebenson Mar 10, 2021
53b6658
dedupe
leebenson Mar 10, 2021
3505e6a
clean up
leebenson Mar 10, 2021
4346512
-inspect
leebenson Mar 10, 2021
0fc1311
Merge branch 'master' into leebenson/tap2
leebenson Mar 10, 2021
e2f8102
merge
leebenson Mar 10, 2021
6c8f96b
improvements
leebenson Mar 11, 2021
fba4817
version merge
leebenson Mar 11, 2021
3a90dc9
clippy
leebenson Mar 11, 2021
2e351bd
Algorithm R
leebenson Mar 12, 2021
70a4176
fix batch
leebenson Mar 12, 2021
b4390d1
comments
leebenson Mar 12, 2021
677ada5
Merge branch 'master' into leebenson/tap2
leebenson Mar 15, 2021
edf610f
simplify
leebenson Mar 16, 2021
55face4
commentary
leebenson Mar 16, 2021
ef4cc1f
TapResult -> TapPayload
leebenson Mar 16, 2021
28bfc93
debug
leebenson Mar 17, 2021
f715353
Merge branch 'master' into leebenson/tap2
leebenson Mar 17, 2021
9a09620
clean up
leebenson Mar 19, 2021
fc21512
Merge branch 'master' into leebenson/tap2
leebenson Mar 19, 2021
d960b48
tests
leebenson Mar 19, 2021
8136a17
tweaks
leebenson Mar 19, 2021
1819ad7
happy clippy
leebenson Mar 19, 2021
fd0aab4
Merge branch 'master' into leebenson/tap2
leebenson Mar 22, 2021
e73fe82
Merge branch 'master' into leebenson/tap2
leebenson Mar 23, 2021
6659a3f
fix reload
leebenson Mar 23, 2021
9761daa
Merge branch 'master' into leebenson/tap2
leebenson Mar 24, 2021
a566229
graphql schema
leebenson Mar 24, 2021
3ea76b4
tap cli wip
leebenson Mar 22, 2021
e465aaf
client
leebenson Mar 22, 2021
cd181f1
client healthcheck
leebenson Mar 22, 2021
9130d29
tap subscription
leebenson Mar 22, 2021
1f61152
add missing connection_init
leebenson Mar 22, 2021
1cb59ac
Update mod.rs
leebenson Mar 23, 2021
7885c9e
commentary
leebenson Mar 24, 2021
59fdc7d
todo
leebenson Mar 24, 2021
db94e32
help
leebenson Mar 24, 2021
ef012ff
check-fmt
leebenson Mar 24, 2021
e34e105
Merge branch 'master' into leebenson/tap2
leebenson Mar 25, 2021
19aa7cd
debug
leebenson Mar 25, 2021
c2d335e
SendError
leebenson Mar 25, 2021
4fe7111
check-fmt
leebenson Mar 25, 2021
55df937
infallible log string
leebenson Mar 25, 2021
a5c1008
Merge branch 'leebenson/tap2' into leebenson/tap-cli
leebenson Mar 25, 2021
64d4ef2
gen schema
leebenson Mar 25, 2021
4a0378a
Merge branch 'leebenson/tap2' into leebenson/tap-cli
leebenson Mar 25, 2021
9659cfe
log event string
leebenson Mar 25, 2021
8469896
debug
leebenson Mar 25, 2021
7285387
usize sort
leebenson Mar 26, 2021
4f89bdf
event union
leebenson Mar 26, 2021
6a9c861
event encoding
leebenson Mar 26, 2021
412abe6
commentary
leebenson Mar 26, 2021
d121814
schema update
leebenson Mar 26, 2021
36b297f
Merge branch 'master' into leebenson/tap2
leebenson Mar 26, 2021
5400a57
Merge branch 'master' into leebenson/tap-cli
leebenson Mar 26, 2021
e0e1895
Merge branch 'leebenson/tap2' into leebenson/tap-cli
leebenson Mar 26, 2021
323e297
merge tap
leebenson Mar 26, 2021
10df625
payload union
leebenson Mar 26, 2021
22f9865
Merge branch 'leebenson/tap2' into leebenson/tap-cli
leebenson Mar 26, 2021
9d4048d
schema
leebenson Mar 26, 2021
ab0252b
fix shutdown
leebenson Mar 29, 2021
04bd6a9
Merge branch 'master' into leebenson/tap2
leebenson Mar 29, 2021
25b9d72
Merge branch 'leebenson/tap2' into leebenson/tap-cli
leebenson Mar 29, 2021
a6b30e3
Merge branch 'master' into leebenson/tap-cli
leebenson Mar 29, 2021
d648777
merge client
leebenson Mar 29, 2021
7ac960c
-EventEncodingType
leebenson Mar 31, 2021
6d80f8e
TapSink sink
leebenson Mar 31, 2021
b77c5f5
SinkExt in test
leebenson Mar 31, 2021
2c3bef2
Merge branch 'master' into leebenson/tap2
leebenson Apr 1, 2021
43c10ef
Merge branch 'leebenson/tap2' into leebenson/tap-cli
leebenson Apr 1, 2021
7054dfb
reqwest
leebenson Apr 1, 2021
5392ca3
Merge branch 'master' into leebenson/tap2
leebenson Apr 2, 2021
ab69183
update for tokio 1.x; non-blocking
leebenson Apr 6, 2021
c12f618
Merge branch 'master' into leebenson/tap2
leebenson Apr 6, 2021
378aa27
Merge branch 'leebenson/tap2' into leebenson/tap-cli
leebenson Apr 6, 2021
9d28265
merge tokio bump
leebenson Apr 6, 2021
3d6f445
fix test
leebenson Apr 6, 2021
578fc43
Merge branch 'leebenson/tap2' into leebenson/tap-cli
leebenson Apr 6, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -150,8 +150,8 @@ goauth = { version = "0.9.0", optional = true }
smpl_jwt = { version = "0.6.1", optional = true }

# API
async-graphql = { version = "2.5.9", optional = true }
async-graphql-warp = { version = "2.5.9", optional = true }
async-graphql = { version = "=2.5.9", optional = true }
async-graphql-warp = { version = "=2.5.9", optional = true }
itertools = { version = "0.10.0", optional = true }

# API client
Expand Down Expand Up @@ -242,6 +242,7 @@ twox-hash = "1.6"
url = "2.2.1"
uuid = { version = "0.8", features = ["serde", "v4"], optional = true }
warp = { version = "0.3.0", default-features = false, optional = true }
cfg-if = "1.0.0"

# For WASM
async-stream = "0.3.0"
Expand Down
3 changes: 2 additions & 1 deletion lib/vector-api-client/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "vector-api-client"
version = "0.1.1"
version = "0.1.2"
authors = ["Vector Contributors <vector@timber.io>"]
edition = "2018"
publish = false
Expand Down Expand Up @@ -34,3 +34,4 @@ chrono = { version = "0.4.6", features = ["serde"] }
url = "2.2.1"
uuid = { version = "0.8", features = ["serde", "v4"] }
weak-table = "0.3.0"
indoc = "1.0.3"
267 changes: 267 additions & 0 deletions lib/vector-api-client/graphql/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -816,6 +816,95 @@
"name": "ErrorsTotal",
"possibleTypes": null
},
{
"description": "Encoding format for the event",
"enumValues": [
{
"deprecationReason": null,
"description": null,
"isDeprecated": false,
"name": "JSON"
},
{
"deprecationReason": null,
"description": null,
"isDeprecated": false,
"name": "YAML"
}
],
"fields": null,
"inputFields": null,
"interfaces": null,
"kind": "ENUM",
"name": "EventEncodingType",
"possibleTypes": null
},
{
"description": "A notification regarding events observation",
"enumValues": null,
"fields": [
{
"args": [],
"deprecationReason": null,
"description": "Name of the component associated with the notification",
"isDeprecated": false,
"name": "componentName",
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "SCALAR",
"name": "String",
"ofType": null
}
}
},
{
"args": [],
"deprecationReason": null,
"description": "Event notification type",
"isDeprecated": false,
"name": "notification",
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "ENUM",
"name": "EventNotificationType",
"ofType": null
}
}
}
],
"inputFields": null,
"interfaces": [],
"kind": "OBJECT",
"name": "EventNotification",
"possibleTypes": null
},
{
"description": "Event notification type",
"enumValues": [
{
"deprecationReason": null,
"description": "A component was found that matched the provided pattern",
"isDeprecated": false,
"name": "MATCHED"
},
{
"deprecationReason": null,
"description": "There isn't currently a component that matches this pattern",
"isDeprecated": false,
"name": "NOT_MATCHED"
}
],
"fields": null,
"inputFields": null,
"interfaces": null,
"kind": "ENUM",
"name": "EventNotificationType",
"possibleTypes": null
},
{
"description": null,
"enumValues": null,
Expand Down Expand Up @@ -1627,6 +1716,88 @@
"name": "LoadAverageMetrics",
"possibleTypes": null
},
{
"description": "Log event with fields for querying log data",
"enumValues": null,
"fields": [
{
"args": [],
"deprecationReason": null,
"description": "Name of the component associated with the log event",
"isDeprecated": false,
"name": "componentName",
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "SCALAR",
"name": "String",
"ofType": null
}
}
},
{
"args": [],
"deprecationReason": null,
"description": "Log message",
"isDeprecated": false,
"name": "message",
"type": {
"kind": "SCALAR",
"name": "String",
"ofType": null
}
},
{
"args": [],
"deprecationReason": null,
"description": "Log timestamp",
"isDeprecated": false,
"name": "timestamp",
"type": {
"kind": "SCALAR",
"name": "DateTime",
"ofType": null
}
},
{
"args": [
{
"defaultValue": null,
"description": null,
"name": "encoding",
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "ENUM",
"name": "EventEncodingType",
"ofType": null
}
}
}
],
"deprecationReason": null,
"description": "Log event as an encoded string format",
"isDeprecated": false,
"name": "string",
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "SCALAR",
"name": "String",
"ofType": null
}
}
}
],
"inputFields": null,
"interfaces": [],
"kind": "OBJECT",
"name": "Log",
"possibleTypes": null
},
{
"description": "Host memory metrics",
"enumValues": null,
Expand Down Expand Up @@ -1963,6 +2134,27 @@
"name": "NetworkMetrics",
"possibleTypes": null
},
{
"description": "An event or a notification",
"enumValues": null,
"fields": null,
"inputFields": null,
"interfaces": null,
"kind": "UNION",
"name": "OutputEventsPayload",
"possibleTypes": [
{
"kind": "OBJECT",
"name": "Log",
"ofType": null
},
{
"kind": "OBJECT",
"name": "EventNotification",
"ofType": null
}
]
},
{
"description": "Information about pagination in a connection",
"enumValues": null,
Expand Down Expand Up @@ -3502,6 +3694,81 @@
"description": null,
"enumValues": null,
"fields": [
{
"args": [
{
"defaultValue": null,
"description": null,
"name": "componentNames",
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "LIST",
"name": null,
"ofType": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "SCALAR",
"name": "String",
"ofType": null
}
}
}
}
},
{
"defaultValue": "500",
"description": null,
"name": "interval",
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "SCALAR",
"name": "Int",
"ofType": null
}
}
},
{
"defaultValue": "100",
"description": null,
"name": "limit",
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "SCALAR",
"name": "Int",
"ofType": null
}
}
}
],
"deprecationReason": null,
"description": "A stream of events emitted from matched component(s)",
"isDeprecated": false,
"name": "outputEvents",
"type": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "LIST",
"name": null,
"ofType": {
"kind": "NON_NULL",
"name": null,
"ofType": {
"kind": "UNION",
"name": "OutputEventsPayload",
"ofType": null
}
}
}
}
},
{
"args": [],
"deprecationReason": null,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
subscription OutputEventsSubscription(
$componentNames: [String!]!, $limit: Int!, $interval: Int!, $encoding: EventEncodingType!){
outputEvents(componentNames: $componentNames, limit: $limit, interval: $interval) {
__typename
... on Log {
componentName
message
timestamp
string(encoding: $encoding)
}
... on EventNotification {
componentName
notification
}
}
}
Loading