Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Carved/Subitems may be added again to case when resuming processing with --continue #792

Closed
lfcnassif opened this issue Oct 20, 2021 · 1 comment · Fixed by #937
Closed

Carved/Subitems may be added again to case when resuming processing with --continue #792

lfcnassif opened this issue Oct 20, 2021 · 1 comment · Fixed by #937
Labels
bug

Comments

@lfcnassif
Copy link
Member

Discovered this while testing #789. Not sure when this regression was introduced.
@lfcnassif lfcnassif added the bug label Oct 20, 2021
@lfcnassif lfcnassif mentioned this issue Oct 20, 2021
Merged
@lfcnassif lfcnassif changed the title Subitems added again to case when resuming processing with --continue (at least in master) Subitems may be added again to case when resuming processing with --continue (at least in master) Oct 30, 2021
@lfcnassif lfcnassif mentioned this issue Jan 25, 2022
Merged
lfcnassif added a commit that referenced this issue Jan 25, 2022
@lfcnassif
#792: skip non last text frags when looking for parents with lost child
33d1ca0
lfcnassif added a commit that referenced this issue Jan 25, 2022
@lfcnassif
#792: don't use ID_IN_SOURCE in trackID of carved/subitems:
907f4c9 
- could make subitems of embedded disks decoded again to be added twice
@lfcnassif lfcnassif changed the title Subitems may be added again to case when resuming processing with --continue (at least in master) Carved/Subitems may be added again to case when resuming processing with --continue (at least in master) Jan 25, 2022
@lfcnassif lfcnassif changed the title Carved/Subitems may be added again to case when resuming processing with --continue (at least in master) Carved/Subitems may be added again to case when resuming processing with --continue Jan 26, 2022
lfcnassif added a commit that referenced this issue Jan 26, 2022
@lfcnassif
#792: skip seen containers enqueued twice (b98dc52 regression)
f91e560
@lfcnassif
Copy link
Member Author

Given the fixes, I think this is happening since this feature was first released.

lfcnassif added a commit that referenced this issue Jan 26, 2022
@lfcnassif
#792: add numCarved counter per parent and add carvedId to carved items
11a47b9
lfcnassif added a commit that referenced this issue Jan 26, 2022
@lfcnassif
#784/#792 improve globalID formula, throw exception if something is bad
9abaeab
lfcnassif added a commit that referenced this issue Jan 26, 2022
@lfcnassif
#792: skip non last text frags when looking for parents with lost child
aa5484b
lfcnassif added a commit that referenced this issue Jan 26, 2022
@lfcnassif
#792: don't use ID_IN_SOURCE in trackID of carved/subitems:
ba97468 
- could make subitems of embedded disks decoded again to be added twice
lfcnassif added a commit that referenced this issue Jan 26, 2022
@lfcnassif
#792: skip seen containers enqueued twice (b98dc52 regression)
3a37b98
lfcnassif added a commit that referenced this issue Jan 27, 2022
@lfcnassif
#792: reset num carved/frags counters to avoid processing parents again,
e604805 
(and adding duplicated children after including carvedId in trackId)
lfcnassif added a commit that referenced this issue Jan 27, 2022
@lfcnassif
#792: fix 3a37b98: don't ignore parents being processed in later queues
6949699
lfcnassif added a commit that referenced this issue Feb 10, 2022
@lfcnassif
#792: add numCarved counter per parent and add carvedId to carved items
ecd0989 
Conflicts:
	iped-engine/src/main/java/dpf/sp/gpinf/indexer/process/task/BaseCarveTask.java
lfcnassif added a commit that referenced this issue Feb 10, 2022
@lfcnassif
#792: skip non last text frags when looking for parents with lost child
5b0defb 
Conflicts:
	iped-engine/src/main/java/dpf/sp/gpinf/indexer/process/task/ElasticSearchIndexTask.java
	iped-engine/src/main/java/dpf/sp/gpinf/indexer/process/task/IndexTask.java
	iped-engine/src/main/java/dpf/sp/gpinf/indexer/process/task/SkipCommitedTask.java
lfcnassif added a commit that referenced this issue Feb 10, 2022
@lfcnassif
#792: skip seen containers enqueued twice (b98dc52 regression)
7ed32a6 
Conflicts:
	iped-engine/src/main/java/dpf/sp/gpinf/indexer/process/task/SkipCommitedTask.java
lfcnassif added a commit that referenced this issue Feb 10, 2022
@lfcnassif
#792: reset num carved/frags counters to avoid processing parents again,
5f95eda 
(and adding duplicated children after including carvedId in trackId)
lfcnassif added a commit that referenced this issue Feb 10, 2022
@lfcnassif
#792: fix 3a37b98: don't ignore parents being processed in later queues
e11ca8d 
Conflicts:
	iped-engine/src/main/java/dpf/sp/gpinf/indexer/process/task/SkipCommitedTask.java
lfcnassif added a commit that referenced this issue Feb 10, 2022
@lfcnassif
#792: copy some changes from commit 9abaeab
d28f0dd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix --continue, improve and rename persistentID sepinf-inc/IPED
1 participant
@lfcnassif