feat(repo): Added ws streamer service #345
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
0xterminator
force-pushed
the
feat/eugene/wss
branch
2 times, most recently
from
664e241 to
c2a9661
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
2 times, most recently
from
7f70e72 to
58ff6a3
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
10 times, most recently
from
1572aee to
5a3b10d
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
from
5a3b10d to
e0a2e92
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
2 times, most recently
from
7a2c032 to
53bb2ba
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
from
53bb2ba to
56df498
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
5 times, most recently
from
c3814f2 to
b876801
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
from
bbf2921 to
a342601
Compare
0xterminator
requested review from
luizstacio,
pedronauck,
Jurshsmith and
cue8it
as code owners
0xterminator
force-pushed
the
feat/eugene/wss
branch
from
57cf755 to
776137f
Compare
0xterminator
changed the title
0xterminator
force-pushed
the
feat/eugene/wss
branch
2 times, most recently
from
98f8d89 to
7ab920a
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
2 times, most recently
from
9e99e6c to
3fdf014
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
from
3fdf014 to
4384a27
Compare
0xterminator
force-pushed
the
feat/eugene/wss
branch
from
7a73ca2 to
a99cc84
Compare
0xterminator
added a commit
that referenced
this pull request
Dec 23, 2024
* feat(publisher): Extend NATS publishing by storing payloads in S3 (#348) * feat(publisher): Publish streams to S3 * feat(publisher): add Localstack for testing s3 locally * feat(publisher): Ignore localstack data * feat(publisher): Fix build errors from integrating S3Client * feat(publisher): Ensure published objects can be streamed Also update tests * feat(publisher): Consistently hash static payload for s3 path * feat(publisher): Use testnet or mainnet buckets based on the env * feat(publisher): Distinct env vars for S3 * fix(publisher): Remove unused cli arg (#353) * fix(publisher): Remove trailing hyphen in bucket name (#354) * fix(publisher): Don't defer error logging (#355) * build(repo): Update Cargo.lock * feat(repo): Added ws streamer service (#345) * feat(repo): Added draft ws version * feat(repo): Fixes after AJs merge * feat(repo): Added docker-compose script for creating S3 bucket * fix(repo): Added better deserialization * fix(publisher): re-allow ctrl c shutdown publisher process * refactor(repo): Disallow nested Cargo.lock * refactor(publisher): Include default S3 bucket for local development * feat(repo): Improved serializations * feat(repo): Removed binary for ws-streamer * feat(repo): Pass any wildcard to stream filtering * feat(repo): Added deliver policy to ws pattern for historical data * feat(repo): Switched to tokio-tungestenite * feat(repo): Small change to websocket subscriptions --------- Co-authored-by: AJ <jurshsmith@gmail.com> * feat(repo): Added webserver service and added docker build step * fix(repo): use cli instead of config.toml * fix(ws): remove duplicated nats clients * fix(ws): use nats_url instead of FuelNetwork * feat(repo): Removed unused args by infra for ws streamer * feat: Adjusts to release S3 + WebServer (#361) * fix(repo): Solving publisher slowness (#352) * feat(repo): add sv-emitter package * build(repo): adjust run-publisher * feat(repo): first version of consumer * feat(repo): add the rest of payloads * refactor(repo): reuse executors * fix(repo): tests * build(repo): configure devops to emitter/consumer * build(repo): update chart version * fix(repo): consumer cli argument * build(repo): docker action * build(repo): fix consumer chart * build(repo): fix consumer chart * build(repo): fix consumer chart * build(repo): chart * fix(repo): general stuff * build(repo): docker action * ci(repo): fix * fix(repo): chain config * fix(repo): docker images * feat(repo): improve docker build * refactor(repo): a lot of improvements * fix(repo): fix helm chart nats config * feat(repo): worked using mirror * fix(repo): adjust nats cluster config * build(repo): adjust github action * build(repo): update chart version * build(repo): fix docker images * build(repo): lint warnings * ci(repo): adjust docker action * build(repo): fix docker file * fix(repo): stream creation * build(repo): fix tls for nats * build(repo): fix chart * build(repo): fix chart * build(repo): fix chart * build(repo): fix chart * fix(repo): use sts instead of deployment for consumer * fix(repo): fix chart * fix(repo): remove initcontainer * fix(repo): update chart * fix(repo): fix resources on consumer * fix(repo): bump chart * fix(repo): bump chart * fix(repo): bump chart * build(repo): fix makefile * build(repo): add publisher into one single Dockerfile * ci(repo): fix docker publish action * build(repo): fix build docker * ci(repo): fix docker action * ci(repo): fix tests * ci(repo): fix ci.yaml * build(repo): adjust fuel dependencies features * build(repo): Add DNS/TLS configuration for Nats (#357) * build(repo): Add DNS/TLS configuration for Nats * build(repo): bump chart version * build(repo): fix chart * build(repo): fix chart * build(repo): fix secretName on nats client websocket config * build(repo): Update NATS dependency on chart (#358) * build(repo): Update NATS dependency on chart * build(repo): Bump chart version * refactor(repo): remove FuelNetwork from nats * refactor(repo): remove unused benches * refactor(repo): general improvements * fix(repo): general fixes * build(repo): adjust chart for new architecture * build(repo): adjust chart * build(repo): fix chart * build(repo): fix chart * build(repo): fix chart * build(repo): fix chart * build(repo): docker image * build(repo): bump chart * build(repo): fix emitter Dockerimage * build(repo): adjust workflow dispatch for docker * build(repo): docker action * build(repo): fix publisher config * build(repo): bump chart * build(repo): general fixes and adjustments * build(repo): bump chart * fix(repo): s3 client workaround * refactor(repo): rename crates to be consistent * build(repo): bump chart * build(repo): some adjusts * fix(consumer): remove max_messages * refactor(repo): change s3 path * build(repo): bump chart * refactor(repo): perf improvements * refactor(webserver): send decoded payload direct on websocket * feat(repo): integrate rust sdk with websockets * build(repo): fix tests * docs(repo): adjust main readme --------- Co-authored-by: AJ <34186192+Jurshsmith@users.noreply.github.com> Co-authored-by: Pedro Nauck <pedronauck@gmail.com> Co-authored-by: AJ <jurshsmith@gmail.com>
pedronauck
added a commit
that referenced
this pull request
Dec 23, 2024
* feat(publisher): Extend NATS publishing by storing payloads in S3 (#348) * feat(publisher): Publish streams to S3 * feat(publisher): add Localstack for testing s3 locally * feat(publisher): Ignore localstack data * feat(publisher): Fix build errors from integrating S3Client * feat(publisher): Ensure published objects can be streamed Also update tests * feat(publisher): Consistently hash static payload for s3 path * feat(publisher): Use testnet or mainnet buckets based on the env * feat(publisher): Distinct env vars for S3 * fix(publisher): Remove unused cli arg (#353) * fix(publisher): Remove trailing hyphen in bucket name (#354) * fix(publisher): Don't defer error logging (#355) * build(repo): Update Cargo.lock * feat(repo): Added ws streamer service (#345) * feat(repo): Added draft ws version * feat(repo): Fixes after AJs merge * feat(repo): Added docker-compose script for creating S3 bucket * fix(repo): Added better deserialization * fix(publisher): re-allow ctrl c shutdown publisher process * refactor(repo): Disallow nested Cargo.lock * refactor(publisher): Include default S3 bucket for local development * feat(repo): Improved serializations * feat(repo): Removed binary for ws-streamer * feat(repo): Pass any wildcard to stream filtering * feat(repo): Added deliver policy to ws pattern for historical data * feat(repo): Switched to tokio-tungestenite * feat(repo): Small change to websocket subscriptions --------- Co-authored-by: AJ <jurshsmith@gmail.com> * feat(repo): Added webserver service and added docker build step * fix(repo): use cli instead of config.toml * fix(ws): remove duplicated nats clients * fix(ws): use nats_url instead of FuelNetwork * feat(repo): Removed unused args by infra for ws streamer * feat: Adjusts to release S3 + WebServer (#361) * fix(repo): Solving publisher slowness (#352) * feat(repo): add sv-emitter package * build(repo): adjust run-publisher * feat(repo): first version of consumer * feat(repo): add the rest of payloads * refactor(repo): reuse executors * fix(repo): tests * build(repo): configure devops to emitter/consumer * build(repo): update chart version * fix(repo): consumer cli argument * build(repo): docker action * build(repo): fix consumer chart * build(repo): fix consumer chart * build(repo): fix consumer chart * build(repo): chart * fix(repo): general stuff * build(repo): docker action * ci(repo): fix * fix(repo): chain config * fix(repo): docker images * feat(repo): improve docker build * refactor(repo): a lot of improvements * fix(repo): fix helm chart nats config * feat(repo): worked using mirror * fix(repo): adjust nats cluster config * build(repo): adjust github action * build(repo): update chart version * build(repo): fix docker images * build(repo): lint warnings * ci(repo): adjust docker action * build(repo): fix docker file * fix(repo): stream creation * build(repo): fix tls for nats * build(repo): fix chart * build(repo): fix chart * build(repo): fix chart * build(repo): fix chart * fix(repo): use sts instead of deployment for consumer * fix(repo): fix chart * fix(repo): remove initcontainer * fix(repo): update chart * fix(repo): fix resources on consumer * fix(repo): bump chart * fix(repo): bump chart * fix(repo): bump chart * build(repo): fix makefile * build(repo): add publisher into one single Dockerfile * ci(repo): fix docker publish action * build(repo): fix build docker * ci(repo): fix docker action * ci(repo): fix tests * ci(repo): fix ci.yaml * build(repo): adjust fuel dependencies features * build(repo): Add DNS/TLS configuration for Nats (#357) * build(repo): Add DNS/TLS configuration for Nats * build(repo): bump chart version * build(repo): fix chart * build(repo): fix chart * build(repo): fix secretName on nats client websocket config * build(repo): Update NATS dependency on chart (#358) * build(repo): Update NATS dependency on chart * build(repo): Bump chart version * refactor(repo): remove FuelNetwork from nats * refactor(repo): remove unused benches * refactor(repo): general improvements * fix(repo): general fixes * build(repo): adjust chart for new architecture * build(repo): adjust chart * build(repo): fix chart * build(repo): fix chart * build(repo): fix chart * build(repo): fix chart * build(repo): docker image * build(repo): bump chart * build(repo): fix emitter Dockerimage * build(repo): adjust workflow dispatch for docker * build(repo): docker action * build(repo): fix publisher config * build(repo): bump chart * build(repo): general fixes and adjustments * build(repo): bump chart * fix(repo): s3 client workaround * refactor(repo): rename crates to be consistent * build(repo): bump chart * build(repo): some adjusts * fix(consumer): remove max_messages * refactor(repo): change s3 path * build(repo): bump chart * refactor(repo): perf improvements * refactor(webserver): send decoded payload direct on websocket * feat(repo): integrate rust sdk with websockets * build(repo): fix tests * docs(repo): adjust main readme --------- Co-authored-by: AJ <34186192+Jurshsmith@users.noreply.github.com> Co-authored-by: Pedro Nauck <pedronauck@gmail.com> Co-authored-by: AJ <jurshsmith@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.