Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Nested provenance cannot be resolved because SVN path is not found #6160

Closed
bennati opened this issue Dec 2, 2022 · 9 comments
Closed

Nested provenance cannot be resolved because SVN path is not found #6160

bennati opened this issue Dec 2, 2022 · 9 comments
Labels
bug Issues that are considered to be bugs downloader About the downloader tool

Comments

@bennati
Copy link
Contributor

bennati commented Dec 2, 2022 •
edited
Loading

When scanning https://github.com/apache/maven-doxia/tree/doxia-1.1.2/doxia-core
The scanner returns the following error:

Could not resolve nested provenance for package 'Maven:xml-apis:xml-apis:1.3.03': SVNException: svn: E160005: Target path '/xerces/xml-commons' does not exist svn: E175002: REPORT of '/repos/asf/!svn/vcc/default': 500 Internal Server Error (https://svn.apache.org)

Despite the path existing https://svn.apache.org/repos/asf/xerces/xml-commons/tags/xml-commons-external-1_3_03/

This is our curation, that has worked until now:

- id: "Maven:xml-apis:xml-apis:1.3.03"
  curations:
    ...
    vcs:
      type: "Subversion"
      url: "https://svn.apache.org/repos/asf/xerces/xml-commons"
      revision: "tags/xml-commons-external-1_3_03"
@sschuberth
Copy link
Member

500 Internal Server Error (https://svn.apache.org)

This sounds like a temporary issue with the server itself. Can you just try again @bennati?

@sschuberth sschuberth added downloader About the downloader tool needs info An issue where further information is required scanner About the scanner tool labels Dec 22, 2022
@bennati
Copy link
Contributor Author

bennati commented Dec 23, 2022

I tried again and the issue is still present.
You can verify it as well: Scanned revision bf37c3f8745e1e68524ff8184c69615bb4562125 of Git repository https://github.com/apache/maven-doxia.git

@sschuberth sschuberth removed the needs info An issue where further information is required label Dec 23, 2022
@bennati
Copy link
Contributor Author

bennati commented Dec 23, 2022

I debugged the scanner and found that the correct VCS_URL is passed to

ort/downloader/src/main/kotlin/vcs/Subversion.kt

Line 112 in 35d9252

clientManager.updateClient.doUpdate(

The issue seems to be inside org.tmatesoft.svn.core.wc.SVNClientManager, as it tries to split the VCS_URL into a base URL and path, where the base URL is https://svn.apache.org/repos/asf/!svn/vcc/default (which does not work)

Below is the stacktrace:

org.tmatesoft.svn.core.SVNException: svn: E160005: Target path '/xerces/xml-commons' does not exist
svn: E175002: REPORT of '/repos/asf/!svn/vcc/default': 500 Internal Server Error (https://svn.apache.org)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:70)
	at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:57)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.runReport(DAVRepository.java:1365)
	at org.tmatesoft.svn.core.internal.io.dav.DAVRepository.update(DAVRepository.java:859)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.updateInternal(SvnNgAbstractUpdate.java:263)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgAbstractUpdate.update(SvnNgAbstractUpdate.java:115)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:40)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgUpdate.run(SvnNgUpdate.java:18)
	at org.tmatesoft.svn.core.internal.wc2.ng.SvnNgOperationRunner.run(SvnNgOperationRunner.java:20)
	at org.tmatesoft.svn.core.internal.wc2.SvnOperationRunner.run(SvnOperationRunner.java:21)
	at org.tmatesoft.svn.core.wc2.SvnOperationFactory.run(SvnOperationFactory.java:1239)
	at org.tmatesoft.svn.core.wc2.SvnOperation.run(SvnOperation.java:294)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:311)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:291)
	at org.tmatesoft.svn.core.wc.SVNUpdateClient.doUpdate(SVNUpdateClient.java:387)
	at org.ossreviewtoolkit.downloader.vcs.Subversion.updateEmptyPath(Subversion.kt:116)
	at org.ossreviewtoolkit.downloader.vcs.Subversion.updateWorkingTree-BWLJW6A(Subversion.kt:138)
	at org.ossreviewtoolkit.downloader.VersionControlSystem.updateWorkingTree-BWLJW6A$default(VersionControlSystem.kt:361)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver$resolveNestedRepository$5.invoke(NestedProvenanceResolver.kt:83)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver$resolveNestedRepository$5.invoke(NestedProvenanceResolver.kt:82)
	at org.ossreviewtoolkit.scanner.utils.DefaultWorkingTreeCache.use(WorkingTreeCache.kt:65)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver.resolveNestedRepository(NestedProvenanceResolver.kt:82)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver.access$resolveNestedRepository(NestedProvenanceResolver.kt:47)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver$resolveNestedProvenance$1.invokeSuspend(NestedProvenanceResolver.kt:56)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at org.ossreviewtoolkit.scanner.provenance.DefaultNestedProvenanceResolver.resolveNestedProvenance(NestedProvenanceResolver.kt:56)
	at org.ossreviewtoolkit.scanner.Scanner$resolveNestedProvenances$duration$1$1$1$1.invokeSuspend(Scanner.kt:207)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
14:03:49.136 [DefaultDispatcher-worker-2] WARN  org.ossreviewtoolkit.downloader.vcs.Subversion - Failed to update the Subversion working tree at '/tmp/ort-DefaultWorkingTreeCache3204553652592340692' to revision '380970':
SVNException: svn: E160005: Target path '/xerces/xml-commons' does not exist
svn: E175002: REPORT of '/repos/asf/!svn/vcc/default': 500 Internal Server Error (https://svn.apache.org)

@bennati
Copy link
Contributor Author

bennati commented Jan 9, 2023

Any opinion on whether this is a bug in the SW or in the Apache SVN server?
Any suggestions on how to fix this?

@bennati
Copy link
Contributor Author

bennati commented Jan 25, 2023

any updates on this? @sschuberth

@sschuberth
Copy link
Member

No, I didn't yet find the time to look into this.

@sschuberth sschuberth self-assigned this Jan 25, 2023
@bennati
Copy link
Contributor Author

bennati commented Feb 14, 2023

any updated?

@sschuberth sschuberth mentioned this issue Feb 14, 2023
Open
@sschuberth sschuberth added bug Issues that are considered to be bugs and removed downloader About the downloader tool labels Feb 14, 2023
@sschuberth
Copy link
Member

sschuberth commented Feb 14, 2023 •
edited
Loading

I finally invested some of my personal time to look into this, and I was able to come up with a test to reproduce this. It's not related to the downloader, but to the (nested) provenance resolver. Looks like we're having a SVN-specific issue with deciding whether to update the working tree with a (numeric) revision, or a switch to a (symbolic) tag.

sschuberth added a commit that referenced this issue Feb 14, 2023
@sschuberth
test(Default*ProvenanceResolverFunTest): Add tests for Subversion
934bd0f 
Disable one of the tests as the underlying issue [1] still needs to be
fixed, but keep the test as a reproducer.

[1]: #6160

Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
sschuberth added a commit that referenced this issue Feb 14, 2023
@sschuberth
test(*ProvenanceResolverFunTest): Add tests for Subversion
73309ac 
Disable one of the tests as the underlying issue [1] still needs to be
fixed, but keep the test as a reproducer.

[1]: #6160

Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
sschuberth added a commit that referenced this issue Feb 14, 2023
@sschuberth
test(*ProvenanceResolverFunTest): Add tests for Subversion
c5e2789 
Disable one of the tests as the underlying issue [1] still needs to be
fixed, but keep the test as a reproducer.

[1]: #6160

Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
@sschuberth sschuberth removed their assignment Feb 27, 2023
@sschuberth sschuberth added downloader About the downloader tool and removed scanner About the scanner tool labels Feb 15, 2024
sschuberth added a commit that referenced this issue Apr 25, 2024
@sschuberth
fix(subversion): Make the peg revision newer than the operative revision
4166a72 
Ensure that always the first algorithm mentioned at [1] is applied. This
avoids errors e.g. when updating to a resolved revision of a tag path.

Fixes #6160.

[1]: https://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
@sschuberth
Copy link
Member

sschuberth commented Apr 25, 2024 •
edited
Loading

@bennati, could you please try whether #8572 fixes the issue for you? (Note, cloning that repo might actually take a long time...)

sschuberth added a commit that referenced this issue Apr 25, 2024
@sschuberth
fix(subversion): Make the peg revision newer than the operative revision
03e5fe7 
Ensure that always the first algorithm mentioned at [1] is applied. This
avoids errors e.g. when updating to a resolved revision of a tag path.

Fixes #6160.

[1]: https://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
sschuberth added a commit that referenced this issue Apr 26, 2024
@sschuberth
fix(subversion): Make the peg revision newer than the operative revision
a8f58fa 
Ensure that always the first algorithm mentioned at [1] is applied. This
avoids errors e.g. when updating to a resolved revision of a tag path.

Fixes #6160.

[1]: https://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
sschuberth added a commit that referenced this issue Apr 26, 2024
@sschuberth
fix(subversion): Make the peg revision newer than the operative revision
f5465f4 
Ensure that always the first algorithm mentioned at [1] is applied. This
avoids errors e.g. when updating to a resolved revision of a tag path.

Fixes #6160.

[1]: https://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Issues that are considered to be bugs downloader About the downloader tool
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sschuberth @bennati