move testground/lotus-soup testplan from oni to lotus #4727

nonsense · 2020-11-04T17:24:32Z

This PR is moving the lotus-soup testplan from https://github.com/filecoin-project/oni to Lotus, so that we maintain all Lotus related testplans near the APIs they are using, as well as add more testplans in the future and run them as part of CircleCI in Lotus.

TODO:

0. copy over filecoin-project/oni/master to Lotus, and preserve history of commits and authors (thanks to @magik6k for git subtree pointer)
1. make sure tests pass on TaaS
2. move and run graphsync testplan to Lotus as well (I am not sure this belongs to Lotus, @raulk @magik6k I leave it up to you to decide)
3. update .circleci rules to trigger tests only on master prior to merge
4. stop running testplans from filecoin-project/oni when this is approved and merged

FOLLOW UP:

5. add github status checks from TaaS (need repo:status github token from filecoin-helper or filecoin-ci user)
6. fix underlying code and run deals stress test as well as part of Lotus CI
7. fix TestSyncCheckpointHead - appears to be flaky
8. move docker images to lotus docker registry
9. release and pick an official version of the Testground client (and not https://gist.github.com/nonsense/5fbf3167cac79945f658771aed32fc44/raw/2e17eb0debf7ec6bdf027c1bdafc2c92dd97273b/testground-d3e9603)
10. deprecate the filecoin-project/oni repo

7 is not related to this PR, just something I noticed while triggering CI multiple times.
5, 9 and 10 should be addressed very soon if this is approved.
8 is to be decided by the team - at the moment I am using the iptestground org on Docker Hub.
6 is somewhat related to this PR - we have a test that exercises the broken logic (and it fails regularly), but we have to fix the underlying code, before merging the test - related issue is #4395

get seq num based on role, not based on group

fix compositions

Option to mine in natural time

- Each role now has its own file (role_*.go). - Options have their own file (lotus_opts.go). - Sync service constructions have their own file (sync.go). - Utilities are functionally grouped in files ({deals,retrieval}.go).

nonsense · 2020-11-05T13:49:44Z

Marking this as ready for review, because it is already running tests on TaaS, and overall shows the general direction of adding the testplans to Lotus. I've added @vyzo also as a reviewer, as I imagine he is most familiar with these tests from the Lotus team (together with @raulk and myself).

magik6k · 2020-11-06T15:36:04Z

One thing that would have been nice is to keep history of who committed what to Project Oni, rather than me just copying over all the files from master now. I couldn't think how to do that (and I particularly don't care about it), but I know some people might, so if you have ideas on how to preserve Git history for these testplans / testkit library, etc. let me know!

I did this with sector-storage / storage-fsm already, this look something like:

git remote add -f oni https://github.com/filecoin-project/[...].git
git merge -s ours --no-commit --allow-unrelated-histories oni/master
git read-tree --prefix=extern/oni sector-storage/master
git commit "Merge oni subtree"

https://docs.github.com/en/free-pro-team@latest/github/using-git/about-git-subtree-merges

…pdate go.mod and references

nonsense · 2020-11-06T20:08:38Z

Thanks @magik6k !

Commit N-2 and N-1 - I did a subtree merge and copied oni/master to testplans, and removed it from extern/oni
Commit N (last commit) - contains all the changes I am making so that we integrate Oni to Lotus:

Update Dockerfiles (due to new dynamic libraries, such as hwloc).
Update go.mod since now we reference lotus and other dependencies from different places.
Added a README.md more appropriate for Lotus, while still keeping the previous README.md from filecoin-project/oni

In its current form, we can outright remove the filecoin-project/oni if this is merged, as we now have a complete copy in testplans/ directory.

We have to decide if we want to keep everything under testplans/ and whether it belongs there - such as composer, dashboards, notes, old README.md, etc. Basically oni repo doesn't fully map to just Testground testplans/... in any case this is mostly a minor decision that we can address later as well.

…testplans-to-lotus

magik6k · 2020-11-20T14:27:10Z

testplans/composer/Dockerfile

@@ -0,0 +1,29 @@
+FROM golang:1.14.4-buster as tg-build


Needs updating to 1.15

magik6k

Looks good, just need to fix build-lotus-soup, and likely upgrade to go 1.15, then ship it

(May want to add Oni people to CODEOWNERS for testplans/)

nonsense · 2020-11-23T19:06:04Z

Thanks @magik6k for review. I updated .circleci/config.yml so that trigger-testplans jobs is run only on merge to master.

nonsense · 2020-11-23T19:46:10Z

Synced with master and updated go.mod for lotus-soup to use filecoin-project/lotus module from ../../.

magik6k

🚢

…se/integrate-testplans-to-lotus move testground/lotus-soup testplan from oni to lotus

nonsense and others added 30 commits June 29, 2020 19:02

Merge pull request #77 from filecoin-project/seq-num-for-all-miners

b3b89b2

get seq num based on role, not based on group

Merge pull request #78 from filecoin-project/fix-compositions

6c8abeb

fix compositions

option to turn on natural time mining

da85855

add composition for natural time test

cced69e

record shutting down message in a more pertinent place

79246e6

update composition-natural.toml

6b44e6d

Merge pull request #76 from filecoin-project/feat/mine-natural-time

24730e1

Option to mine in natural time

remove useless error argument from retrieveData

62f63a2

deal stress test

36f82ab

manifest and composition for deal stress test

4595ded

working client rpc

eec5c23

wip

204714a

resolved merge conflicts

33e60df

fixup

954ad1f

extract starting of rpc servers

63bce1a

upgrade go-sdk

b62878c

more logging for serial execution

903b1ea

extract withApiEndpoint

e3d2d2e

extract startServer

0a9af6a

fixup

46beab3

register and export lotus metrics (#84)

af582af

Merge pull request #82 from filecoin-project/sector-analysis-tools

035bf8b

refactor lotus recipes to make them more manageable. (#86)

7f37165

- Each role now has its own file (role_*.go). - Options have their own file (lotus_opts.go). - Sync service constructions have their own file (sync.go). - Utilities are functionally grouped in files ({deals,retrieval}.go).

form testkit; separate test case logic from node templates.

436fe15

iterate on testkit library.

81f8451

rename testkit.{Run=>Handle}DefaultRole.

e5d6751

go mod tidy.

de6d927

wip

e01b887

set drand_period=1s

8469f67

refactor drand node and implement halting

0b386f2

nonsense force-pushed the nonsense/integrate-testplans-to-lotus branch 3 times, most recently from 80da14b to f5a5f78 Compare November 5, 2020 13:46

nonsense marked this pull request as ready for review November 5, 2020 13:48

nonsense requested review from Kubuxu, magik6k and whyrusleeping as code owners November 5, 2020 13:48

nonsense requested review from raulk and vyzo November 5, 2020 13:48

nonsense force-pushed the nonsense/integrate-testplans-to-lotus branch from f5a5f78 to 14cc885 Compare November 6, 2020 19:40

nonsense added 3 commits November 6, 2020 21:02

merge filecoin-project/oni subtree in testplans/

1f678c1

remove oni submodule

f7f5a27

update Dockerfiles; integrate lotus-soup and graphsync in Lotus CI; u…

bf50071

…pdate go.mod and references

nonsense force-pushed the nonsense/integrate-testplans-to-lotus branch from 14cc885 to bf50071 Compare November 6, 2020 20:05

Merge remote-tracking branch 'origin/master' into nonsense/integrate-…

f0eac01

…testplans-to-lotus

magik6k reviewed Nov 20, 2020

View reviewed changes

testplans/composer/Dockerfile Outdated

@@ -0,0 +1,29 @@

FROM golang:1.14.4-buster as tg-build

Copy link

Contributor

magik6k Nov 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs updating to 1.15

magik6k approved these changes Nov 20, 2020

View reviewed changes

nonsense added 2 commits November 23, 2020 20:00

bump docker golang version to 1.15:buster

844c0a3

trigger testplans only on merge to master

3f15135

nonsense added 2 commits November 23, 2020 20:20

Merge branch 'master' into nonsense/integrate-testplans-to-lotus

fd64086

go mod replace lotus with ../../

52da5ac

magik6k approved these changes Nov 24, 2020

View reviewed changes

magik6k merged commit 7c83110 into master Nov 24, 2020

magik6k deleted the nonsense/integrate-testplans-to-lotus branch November 24, 2020 16:05

bibibong pushed a commit to EpiK-Protocol/go-epik that referenced this pull request Jan 8, 2021

Merge pull request filecoin-project#4727 from filecoin-project/nonsen…

fac31e3

…se/integrate-testplans-to-lotus move testground/lotus-soup testplan from oni to lotus

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

move testground/lotus-soup testplan from oni to lotus #4727

move testground/lotus-soup testplan from oni to lotus #4727

nonsense commented Nov 4, 2020 •

edited

Loading

nonsense commented Nov 5, 2020

magik6k commented Nov 6, 2020

nonsense commented Nov 6, 2020

magik6k Nov 20, 2020

magik6k left a comment

nonsense commented Nov 23, 2020

nonsense commented Nov 23, 2020

magik6k left a comment

move testground/lotus-soup testplan from oni to lotus #4727

move testground/lotus-soup testplan from oni to lotus #4727

Conversation

nonsense commented Nov 4, 2020 • edited Loading

nonsense commented Nov 5, 2020

magik6k commented Nov 6, 2020

nonsense commented Nov 6, 2020

magik6k Nov 20, 2020

Choose a reason for hiding this comment

magik6k left a comment

Choose a reason for hiding this comment

nonsense commented Nov 23, 2020

nonsense commented Nov 23, 2020

magik6k left a comment

Choose a reason for hiding this comment

nonsense commented Nov 4, 2020 •

edited

Loading