Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Roadmap Docs #306

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from
Draft

Roadmap Docs #306

wants to merge 24 commits into from

Conversation

expede
Copy link
Member

@expede expede commented Sep 11, 2023

Preview 📚


Just stashing here for now

Roadmap

Dependency Graph

Phase 0: MVP

The MVP focuses on getting the basics running.

  • Wit / Ipld
    • Bidirectional Interpreter ⚙️
  • Networking
  • Execution
    • Deterministic Wasm runtime
    • Receipt reuse / short-circuit
    • Graceful (crash-oblivious) resumption
    • Wasm fuel consumption
  • Scheduler
    • Peer probe
    • Computing a Workflow around Promises
  • Application layer
  • Packaging and Dissemination
    • Package for self-hosting
      • Available on brew
      • Available for download (cross-arch,cross-comp) on GitHub downloads
    • Package for deployment to managed services (i.e. Docker)
      • Docker container build (cross-arch)
      • Release on GitHub registry, Docker hub ⚙️

Phase 1:

Make it easy for people to actually use and for developers to iterate on.

  • Services
    • User Account ⚙️
  • Spec Integration
    • Implement cause
    • Workflow exceptions, i.e. catch
    • Injection of Ucan prf/chain from top-level workflow
    • Global workflow and task settings ⚙️
  • DX
    • Rust Wasm (guest) template
    • Onboarding docs ⚙️
    • UI / integration App (React)?
    • SDK(s)
  • Networking
    • Multi-node testing
      • Integration testing and simulation for libp2p swarm, gossipsub, Iroh peers
    • JSON-RPC (or similar) for server interaction (e.g. Node, Rails, Flask) and over
      WebSocket(s) for Browser applications
  • Data and State Effect
    • Effect WASI/component plugin system
    • Host blockstore
    • Guest Wasm blockstore interface/component
    • Content handle injection
  • Resource limits
    • Time ⚙️
    • Memory ⚙️
    • Disk
    • Fuel Metering (testing, edge cases) ⚙️
  • Observability
    • Process metrics (also used for "capability" discovery
    • Workflow and Runner instrumentation and traces
    • libp2p metrics ⚙️
  • Task Permissions
    • UCAN validation
    • Varsig implementation
  • Deployment
    • (One-step?) Deploy to Cloud Service1
    • (Fission) Infrastructure
      • Loki Stack: Tempo + Grafana + Prometheus

Phase 2

  • DX
  • "Distributed" Scheduling
    • Coordinator "service" and matchmaking
      • Active/Passive coordinators for workflow intake
      • Compute node load-balancing and partitioning
    • Workflow re-queuing (waiting on instruction from another workflow) and
      validation
  • Networking
    • Iroh integration, esp. NAT traversal
  • Observability
    • iroh metrics
  • 1st-party Effects
    • HTTP
    • Idempotent Randomness Oracle
  • Services
    • Task registry
    • Dashboard
  • Settlement
    • Stripe
  • Deployment
    • Add IPVM support to Kubo (kubo run)
    • Homestar per-node observability app/readme

Phase 3

  • Browser (JS) Host runtime, i.e. homestar-js

Phase?

  • JS Wasm (guest) model & template

Footnotes

  1. Still in discussion.

@expede expede changed the title Skeleton Roadmap Docs Sep 11, 2023
@expede expede self-assigned this Sep 11, 2023
@codecov
Copy link

codecov bot commented Sep 12, 2023

Codecov Report

Merging #306 (90c35c6) into main (1fad8c4) will decrease coverage by 0.10%.
Report is 3 commits behind head on main.
The diff coverage is n/a.

❗ Current head 90c35c6 differs from pull request most recent head 54f6447. Consider uploading reports for the commit 54f6447 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #306      +/-   ##
==========================================
- Coverage   73.09%   72.99%   -0.10%     
==========================================
  Files          70       67       -3     
  Lines        6935     6706     -229     
==========================================
- Hits         5069     4895     -174     
+ Misses       1866     1811      -55     

see 6 files with indirect coverage changes

@expede
Copy link
Member Author

expede commented Sep 12, 2023

LOL thanks Codecov

plan/ROADMAP.md Outdated Show resolved Hide resolved
@zeeshanlakhani
Copy link
Contributor

One thing the diagram doesn't really cover is individual node execution, and scheduling it self. Maybe that's captured in deterministic Wasm, but it really is work onto itself, without the embedded wasmtime for example.

@expede
Copy link
Member Author

expede commented Sep 19, 2023

Not sure that we need .md files counting against our Codecov metrics, or any of the tests to run when only writing prose 🤔

@zeeshanlakhani
Copy link
Contributor

Not sure that we need .md files counting against our Codecov metrics, or any of the tests to run when only writing prose 🤔

oh they are not counting on metrics. But will add a filter for things rust/toml specific. It's just a little finicky to get the right cross-section.

@expede
Copy link
Member Author

expede commented Sep 19, 2023

It seems to think it does

Merging #306 (90c35c6) into main (f388dbf) will decrease coverage by 0.05%.

@expede
Copy link
Member Author

expede commented Sep 19, 2023

(Anyhow, nothing that has to change today or something. Just noting it)

Signed-off-by: Brooklyn Zelenka <hello@brooklynzelenka.com>
Signed-off-by: Brooklyn Zelenka <hello@brooklynzelenka.com>
Signed-off-by: Brooklyn Zelenka <hello@brooklynzelenka.com>
Signed-off-by: Brooklyn Zelenka <hello@brooklynzelenka.com>
Signed-off-by: Brooklyn Zelenka <hello@brooklynzelenka.com>
Signed-off-by: Brooklyn Zelenka <hello@brooklynzelenka.com>
Signed-off-by: Brooklyn Zelenka <hello@brooklynzelenka.com>
Signed-off-by: Brooklyn Zelenka <hello@brooklynzelenka.com>
Signed-off-by: Brooklyn Zelenka <hello@brooklynzelenka.com>
@zeeshanlakhani
Copy link
Contributor

zeeshanlakhani commented Sep 19, 2023

Q @expede:

  • Package for deployment to managed services? Docker count here? We need to push it to registries (hub, gh), but wondering if it counts as the package needed?

@zeeshanlakhani zeeshanlakhani self-assigned this Sep 19, 2023
@expede
Copy link
Member Author

expede commented Sep 19, 2023

A @zeeshanlakhani:

Package for deployment to managed services? Docker count here?

That's part of it, but if we can do a one-click deploy on AWS, then even better.

@zeeshanlakhani
Copy link
Contributor

A @zeeshanlakhani:

Package for deployment to managed services? Docker count here?

That's part of it, but if we can do a one-click deploy on AWS, then even better.

Yep. I'll define this more toward infrastructure and add some sub components.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants