Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

* init decoupling-post-worker code #197

Merged

Conversation

lovel8
Copy link

@lovel8 lovel8 commented Aug 20, 2021

Merge based on:
filecoin-project/lotus#6943

@lovel8
Copy link
Author

lovel8 commented Oct 27, 2021

@jennijuju The functions of removing duplicate sectors also needs to be merged ( types.go file ) . This modification can improve efficiency

@lovel8 lovel8 force-pushed the feat/decoupling-post-worker branch 3 times, most recently from 413873a to d670ec9 Compare October 27, 2021 06:10
@lovel8 lovel8 force-pushed the feat/decoupling-post-worker branch from d670ec9 to a72c001 Compare November 4, 2021 02:21
@lovel8
Copy link
Author

lovel8 commented Nov 4, 2021

@jennijuju The code has been debugged and verified, please review the merge

types.go Outdated
@@ -131,3 +132,50 @@ type PrivateSectorInfo struct {
type AllocationManager interface {
Free()
}

func DeDuplicatePrivateSectorInfo(ctx context.Context, sortPrivSectors *SortedPrivateSectorInfo) (SortedPrivateSectorInfo, error) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at code in Lotus it looks like it should be possible to execute this and split operations before creating the SortedPrivateSectorInfo.
I strongly suggest changing it to that.
The SortedPrivateSectorInfo type should only be used for directly passing sector information into FFI not for carrying sector information around or operating on them.

Copy link

@Kubuxu Kubuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As requested in the comment, perform operations required first and then create an instance of SortedPrivateSectorInfo.

Changes in proofs.go and distributed.go look good.

@lovel8
Copy link
Author

lovel8 commented Nov 19, 2021

@Kubuxu Agree with you. We have a try

@lovel8
Copy link
Author

lovel8 commented Nov 23, 2021

@jennijuju @Kubuxu According to the review comments, the deduplication of SortedPrivateSectorInfo is performed in lotus, because the key member variables of SortedPrivateSectorInfo are private types and cannot be accessed at the lotus layer.
image

We tried to perform deduplication before calling ffi.NewSortedPrivateSectorInfo and tested it, but proof verification Fail.

Any suggestions for modification?

types.go Outdated
}
if !flag {
break
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The above code seems to sort sector infos by SectorNumber, but the original code sorts them by SealedCID, why the change?

Also if you want to sort, please use sort.Slice. See types.go#93

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kubuxu According to the logic before splitting,Split calculation results need to be sorted by SectorNumber
企业微信截图_16377349048939

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Kubuxu Any suggestions for modification?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the members of the SortedProvateSectorInfo structure are private and cannot be operated in lotus, we think the DeDuplicatePrivateSectorInfo interface is necessary, or are there any other implementation suggestions? @Kubuxu

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't passing sectors deduplicated here to NewSortedPrivateSectorInfo sort them correctly? If not, can we change NewSortedPrivateSectorInfo to sort correctly?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@magik6k We can change NewSortedPrivateSectorInfo to sort correctly, If It without affecting the logic of other functions

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@magik6k Modification has been submitted, please review

types.go Outdated Show resolved Hide resolved
types.go Outdated Show resolved Hide resolved
types.go Outdated Show resolved Hide resolved
types.go Outdated Show resolved Hide resolved
@lovel8
Copy link
Author

lovel8 commented Dec 21, 2021

@magik6k The code has been debugged and verified, please review and merge

Copy link
Contributor

@magik6k magik6k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't really understand the change in sorting - why sorting by SealedCID(CommR) worked before, and why we're changing it to sort by SectorNumber. Reading the rust code, the only thing I see is that the code doesn't really seem to care about sorting, at least in case of the monolith post methods.

Will merge this, if it's somehow broken it should be really easy to see that and fix

@magik6k magik6k requested a review from Kubuxu January 4, 2022 11:12
@magik6k magik6k merged commit 8e377f9 into filecoin-project:master Jan 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants