Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Curio - Easy Migration #11617

Merged
merged 51 commits into from
Mar 15, 2024
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
f6f4d0e
feat: lp mig - first few steps
snadrus Feb 12, 2024
229645b
lp mig: default tasks
snadrus Feb 13, 2024
5a61b87
code comments
snadrus Feb 13, 2024
17ffd12
docs
snadrus Feb 13, 2024
0c8969d
lp-mig-progress
snadrus Feb 20, 2024
aa049ba
Merge branch 'master' into easyMigration
snadrus Feb 20, 2024
6765ac1
shared
snadrus Feb 20, 2024
d47b2f7
comments and todos
snadrus Feb 26, 2024
443b05c
fix: curio: rename lotus-provider to curio (#11645)
LexLuthr Feb 29, 2024
b686823
lp: channels doc
snadrus Feb 29, 2024
86f8624
Merge branch 'easyMigration' of https://github.com/filecoin-project/l…
snadrus Feb 29, 2024
5c2a86b
finish easy-migration TODOs
snadrus Mar 2, 2024
345dd9c
out generate
snadrus Mar 2, 2024
94e2a37
Merge branch 'master' into easyMigration
snadrus Mar 2, 2024
ef9e3f2
merging and more renames
snadrus Mar 2, 2024
af19d34
avoid make-all
snadrus Mar 3, 2024
590537e
minor doc stuff
snadrus Mar 5, 2024
3ff1e4d
cu: make gen
snadrus Mar 6, 2024
ea1dd33
make gen fix
snadrus Mar 6, 2024
0c3d0ce
conflict-resolve
snadrus Mar 6, 2024
dedad07
Merge branch 'master' into easyMigration
snadrus Mar 6, 2024
5470aec
make gen
snadrus Mar 7, 2024
6505b90
tryfix
snadrus Mar 7, 2024
9fa63fb
go mod tidy
snadrus Mar 7, 2024
4c7130e
minor ez migration fixes
snadrus Mar 7, 2024
5086714
ez setup - ui cleanups
snadrus Mar 8, 2024
11f21a0
better error message
snadrus Mar 8, 2024
9d4ed63
guided setup colors
snadrus Mar 8, 2024
ea0b09e
better path to saveconfigtolayer
snadrus Mar 8, 2024
b94a75c
loadconfigwithupgrades fix
snadrus Mar 8, 2024
795bc6d
readMiner oops
snadrus Mar 8, 2024
01886bc
guided - homedir
snadrus Mar 8, 2024
97cb2d0
err if miner is running
snadrus Mar 8, 2024
d520d5c
prompt error should exit
snadrus Mar 8, 2024
1e27ea7
process already running, miner_id sectors in migration
snadrus Mar 8, 2024
ded8122
dont prompt for language a second time
snadrus Mar 8, 2024
500c92b
check miner stopped
Mar 8, 2024
aab78c6
unlock repo
Mar 8, 2024
f03c7d3
render and sql oops
snadrus Mar 8, 2024
0daa23d
Merge branch 'easyMigration' of https://github.com/filecoin-project/l…
snadrus Mar 8, 2024
2e4711e
curio easyMig - some fixes
snadrus Mar 12, 2024
01eae29
easyMigration runs successfully
snadrus Mar 12, 2024
daa2df3
lint
snadrus Mar 12, 2024
c3c82a3
review fixes
snadrus Mar 15, 2024
81947a6
fix backup path
Mar 15, 2024
6b128ff
Merge branch 'master' into easyMigration
Mar 15, 2024
37590ee
Merge branch 'master' into easyMigration
Mar 15, 2024
226c5bc
fixes1
snadrus Mar 15, 2024
dc3316a
fixes2
snadrus Mar 15, 2024
40f3bf5
fixes 3
snadrus Mar 15, 2024
40828a4
Merge branch 'easyMigration' of https://github.com/filecoin-project/l…
snadrus Mar 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1018,7 +1018,7 @@ workflows:
requires:
- build
suite: utest-unit-rest
target: "./blockstore/... ./build/... ./chain/... ./conformance/... ./gateway/... ./journal/... ./lib/... ./markets/... ./paychmgr/... ./provider/... ./tools/..."
target: "./blockstore/... ./build/... ./chain/... ./conformance/... ./curiosrc/... ./gateway/... ./journal/... ./lib/... ./markets/... ./paychmgr/... ./tools/..."
resource_class: 2xlarge
- test:
name: test-unit-storage
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
/lotus-chainwatch
/lotus-shed
/lotus-sim
/lotus-provider
/curio
/lotus-townhall
/lotus-fountain
/lotus-stats
Expand Down
8 changes: 4 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ COPY --from=lotus-builder /opt/filecoin/lotus-wallet /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-gateway /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-miner /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-worker /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-provider /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/curio /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-stats /usr/local/bin/
COPY --from=lotus-builder /opt/filecoin/lotus-fountain /usr/local/bin/

Expand All @@ -118,21 +118,21 @@ RUN mkdir /var/lib/lotus
RUN mkdir /var/lib/lotus-miner
RUN mkdir /var/lib/lotus-worker
RUN mkdir /var/lib/lotus-wallet
RUN mkdir /var/lib/lotus-provider
RUN mkdir /var/lib/curio
RUN chown fc: /var/tmp/filecoin-proof-parameters
RUN chown fc: /var/lib/lotus
RUN chown fc: /var/lib/lotus-miner
RUN chown fc: /var/lib/lotus-worker
RUN chown fc: /var/lib/lotus-wallet
RUN chown fc: /var/lib/lotus-provider
RUN chown fc: /var/lib/curio


VOLUME /var/tmp/filecoin-proof-parameters
VOLUME /var/lib/lotus
VOLUME /var/lib/lotus-miner
VOLUME /var/lib/lotus-worker
VOLUME /var/lib/lotus-wallet
VOLUME /var/lib/lotus-provider
VOLUME /var/lib/curio

EXPOSE 1234
EXPOSE 2345
Expand Down
54 changes: 27 additions & 27 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ CLEAN+=build/.update-modules
deps: $(BUILD_DEPS)
.PHONY: deps

build-devnets: build lotus-seed lotus-shed lotus-provider
build-devnets: build lotus-seed lotus-shed curio
.PHONY: build-devnets

debug: GOFLAGS+=-tags=debug
Expand Down Expand Up @@ -97,14 +97,14 @@ lotus-miner: $(BUILD_DEPS)
.PHONY: lotus-miner
BINS+=lotus-miner

lotus-provider: $(BUILD_DEPS)
rm -f lotus-provider
$(GOCC) build $(GOFLAGS) -o lotus-provider ./cmd/lotus-provider
.PHONY: lotus-provider
BINS+=lotus-provider
curio: $(BUILD_DEPS)
rm -f curio
$(GOCC) build $(GOFLAGS) -o curio ./cmd/curio
.PHONY: curio
BINS+=curio

lp2k: GOFLAGS+=-tags=2k
lp2k: lotus-provider
cu2k: GOFLAGS+=-tags=2k
cu2k: curio

lotus-worker: $(BUILD_DEPS)
rm -f lotus-worker
Expand All @@ -124,22 +124,22 @@ lotus-gateway: $(BUILD_DEPS)
.PHONY: lotus-gateway
BINS+=lotus-gateway

build: lotus lotus-miner lotus-worker lotus-provider
build: lotus lotus-miner lotus-worker curio
@[[ $$(type -P "lotus") ]] && echo "Caution: you have \
an existing lotus binary in your PATH. This may cause problems if you don't run 'sudo make install'" || true

.PHONY: build

install: install-daemon install-miner install-worker install-provider
install: install-daemon install-miner install-worker install-curio

install-daemon:
install -C ./lotus /usr/local/bin/lotus

install-miner:
install -C ./lotus-miner /usr/local/bin/lotus-miner

install-provider:
install -C ./lotus-provider /usr/local/bin/lotus-provider
install-curio:
install -C ./curio /usr/local/bin/curio

install-worker:
install -C ./lotus-worker /usr/local/bin/lotus-worker
Expand All @@ -156,8 +156,8 @@ uninstall-daemon:
uninstall-miner:
rm -f /usr/local/bin/lotus-miner

uninstall-provider:
rm -f /usr/local/bin/lotus-provider
uninstall-curio:
rm -f /usr/local/bin/curio

uninstall-worker:
rm -f /usr/local/bin/lotus-worker
Expand Down Expand Up @@ -260,13 +260,13 @@ install-miner-service: install-miner install-daemon-service
@echo "To start the service, run: 'sudo systemctl start lotus-miner'"
@echo "To enable the service on startup, run: 'sudo systemctl enable lotus-miner'"

install-provider-service: install-provider install-daemon-service
install-curio-service: install-curio install-daemon-service
mkdir -p /etc/systemd/system
mkdir -p /var/log/lotus
install -C -m 0644 ./scripts/lotus-provider.service /etc/systemd/system/lotus-provider.service
install -C -m 0644 ./scripts/curio.service /etc/systemd/system/curio.service
systemctl daemon-reload
@echo
@echo "lotus-provider service installed. Don't forget to run 'sudo systemctl start lotus-provider' to start it and 'sudo systemctl enable lotus-provider' for it to be enabled on startup."
@echo "Curio service installed. Don't forget to run 'sudo systemctl start curio' to start it and 'sudo systemctl enable curio' for it to be enabled on startup."

install-main-services: install-miner-service

Expand All @@ -286,10 +286,10 @@ clean-miner-service:
rm -f /etc/systemd/system/lotus-miner.service
systemctl daemon-reload

clean-provider-service:
-systemctl stop lotus-provider
-systemctl disable lotus-provider
rm -f /etc/systemd/system/lotus-provider.service
clean-curio-service:
-systemctl stop curio
-systemctl disable curio
rm -f /etc/systemd/system/curio.service
systemctl daemon-reload

clean-main-services: clean-daemon-service
Expand Down Expand Up @@ -366,7 +366,7 @@ docsgen-md-bin: api-gen actors-gen
docsgen-openrpc-bin: api-gen actors-gen
$(GOCC) build $(GOFLAGS) -o docgen-openrpc ./api/docgen-openrpc/cmd

docsgen-md: docsgen-md-full docsgen-md-storage docsgen-md-worker docsgen-md-provider
docsgen-md: docsgen-md-full docsgen-md-storage docsgen-md-worker docsgen-md-curio

docsgen-md-full: docsgen-md-bin
./docgen-md "api/api_full.go" "FullNode" "api" "./api" > documentation/en/api-v1-unstable-methods.md
Expand All @@ -375,8 +375,8 @@ docsgen-md-storage: docsgen-md-bin
./docgen-md "api/api_storage.go" "StorageMiner" "api" "./api" > documentation/en/api-v0-methods-miner.md
docsgen-md-worker: docsgen-md-bin
./docgen-md "api/api_worker.go" "Worker" "api" "./api" > documentation/en/api-v0-methods-worker.md
docsgen-md-provider: docsgen-md-bin
./docgen-md "api/api_lp.go" "Provider" "api" "./api" > documentation/en/api-v0-methods-provider.md
docsgen-md-curio: docsgen-md-bin
./docgen-md "api/api_curio.go" "Curio" "api" "./api" > documentation/en/api-v0-methods-curio.md

docsgen-openrpc: docsgen-openrpc-full docsgen-openrpc-storage docsgen-openrpc-worker docsgen-openrpc-gateway

Expand All @@ -401,16 +401,16 @@ gen: actors-code-gen type-gen cfgdoc-gen docsgen api-gen circleci

jen: gen

snap: lotus lotus-miner lotus-worker lotus-provider
snap: lotus lotus-miner lotus-worker curio
snapcraft
# snapcraft upload ./lotus_*.snap

# separate from gen because it needs binaries
docsgen-cli: lotus lotus-miner lotus-worker lotus-provider
docsgen-cli: lotus lotus-miner lotus-worker curio
python3 ./scripts/generate-lotus-cli.py
./lotus config default > documentation/en/default-lotus-config.toml
./lotus-miner config default > documentation/en/default-lotus-miner-config.toml
./lotus-provider config default > documentation/en/default-lotus-provider-config.toml
./curio config default > documentation/en/default-curio-config.toml
.PHONY: docsgen-cli

print-%:
Expand Down
2 changes: 1 addition & 1 deletion api/api_lp.go → api/api_curio.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (
"github.com/filecoin-project/lotus/storage/sealer/storiface"
)

type LotusProvider interface {
type Curio interface {
Version(context.Context) (Version, error) //perm:admin

AllocatePieceToSector(ctx context.Context, maddr address.Address, piece PieceDealInfo, rawSize int64, source url.URL, header http.Header) (SectorOffset, error) //perm:write
Expand Down
6 changes: 3 additions & 3 deletions api/client/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ import (
"github.com/filecoin-project/lotus/lib/rpcenc"
)

// NewProviderRpc creates a new http jsonrpc client.
func NewProviderRpc(ctx context.Context, addr string, requestHeader http.Header) (api.LotusProvider, jsonrpc.ClientCloser, error) {
var res v1api.LotusProviderStruct
// NewCurioRpc creates a new http jsonrpc client.
func NewCurioRpc(ctx context.Context, addr string, requestHeader http.Header) (api.Curio, jsonrpc.ClientCloser, error) {
var res v1api.CurioStruct

closer, err := jsonrpc.NewMergeClient(ctx, addr, "Filecoin",
api.GetInternalStructs(&res), requestHeader, jsonrpc.WithErrors(api.RPCErrors))
Expand Down
8 changes: 4 additions & 4 deletions api/docgen/docgen.go
Original file line number Diff line number Diff line change
Expand Up @@ -430,10 +430,10 @@ func GetAPIType(name, pkg string) (i interface{}, t reflect.Type, permStruct []r
i = &api.GatewayStruct{}
t = reflect.TypeOf(new(struct{ api.Gateway })).Elem()
permStruct = append(permStruct, reflect.TypeOf(api.GatewayStruct{}.Internal))
case "Provider":
i = &api.LotusProviderStruct{}
t = reflect.TypeOf(new(struct{ api.LotusProvider })).Elem()
permStruct = append(permStruct, reflect.TypeOf(api.LotusProviderStruct{}.Internal))
case "Curio":
i = &api.CurioStruct{}
t = reflect.TypeOf(new(struct{ api.Curio })).Elem()
permStruct = append(permStruct, reflect.TypeOf(api.CurioStruct{}.Internal))
default:
panic("unknown type")
}
Expand Down
Loading