current release has a signature for com.sun.jna.5.13.0.v20230812-1000.jar that is not valid?

[jcompagner]

Hi,

not sure where this really belongs but when i have a target file and i export that target file to a local disk

<target includeMode="feature" name="eclipse_sources" sequenceNumber="60">
    <locations>
        <location includeAllPlatforms="false" includeConfigurePhase="true" includeMode="planner" includeSource="true" type="InstallableUnit">
            <repository location="https://download.eclipse.org/eclipse/updates/4.29/"/>
            <unit id="org.eclipse.emf.ecore.feature.group" version="2.35.0.v20230829-0934"/>
            <unit id="org.eclipse.equinox.executable.feature.group" version="3.8.2200.v20230717-2134"/>
            <unit id="org.eclipse.platform.sdk" version="4.29.0.I20230903-1000"/>
        </location>
    </locations>
    <targetJRE path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-17"/>
</target>

Then the com.sun.jna is not in that target file, i tried to debug this through and the org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transferSingle() method

it tries to make a directory of that file (no idea why that is also) but i guess because something is wrong

then in the end when it wants to copy i get:

Status ERROR: org.eclipse.equinox.p2.artifact.repository code=0 Problems downloading artifact: osgi.bundle,com.sun.jna,5.13.0.v20230812-1000. children=[Status ERROR: org.eclipse.equinox.p2.artifact.repository code=0 The signature is invalid for current content]

and then the dir is also removed again..

in our previous release that is building on eclipse 2023.06 the com.sun.jna artifact is just a jar..

[merks]

I'm not sure what it means to "export a target file to a local disk".

It's clearly a jar here:

https://download.eclipse.org/eclipse/updates/4.29/R-4.29-202309031000/plugins/com.sun.jna_5.13.0.v20230812-1000.jar

It's full of native libraries so it will tend to get unpack to access those.

[jcompagner]

weird it is not extracted with a previous eclipse (in that export i just see it purely as a jar, so i guess something changed in the manifest? (i can have a look)

what i mean with export is, that in the target file editor there on the right side of the "Set as Active Platform" a button "Export"
we use that internally constantly..
To dump the whole target that we have on the local disk, and then we have a local.target file that points to that dir to develop against it.

This is because so many times (before we did this) we have problems that if 1 site is down that suddenly eclipse thinks it can't resolve the target and you just can't develop anymore..
(and yes i didn't change the target, this happens on stable targets, not sure if that is better now, but it was always very annoying that suddenly nothing compiles and run anymore because eclipse tried to get to remote sites that where down for a while)

But now suddenly we do that same export (with our new target file pointing to the 4.29 eclipse site
and we export it to disk, then when we try to launch the verify of the launch config suddenly says "verify failed, no com.sun.jna bundle found"
And to my surprise it really was not in the local target dir..
And i didn't see much in the various log files as far as i can see, only when i really debugged it i finally found the above message.

[jcompagner]

by the way we get around it by just dumping that exact file as a jar into the local_target\plugins dir. then it does work, so we can continue fine but it is a bit annoying that when we need to create a target folder (because something did change) we need to copy our self that file back in.

[jcompagner]

manifest files are pretty similar i think it is just rewritten a bit (it has a bit more chars per line)
and it version changed from: 5.13.0 to 5.13.0.v20230812-1000

so i guess that it now tries to extract it is something of p2 or equinox that now sees the Bundle-NativeCode: attribute and thinks let me extract it.

But extracting would be fine, that is not directly a problem (just a difference) the difference is that for some reason the signature that it generated in the p2 site? is not what it expects.

[merks]

I believe it gets unzipped because of the touchpoint instruction:

So it's just following instructions. Even looking in my shared bundle pool, it's a folder there too, and there are quite a number of bundles that are folders:

I don't think you can generally expect to turn an artifact pool into an update site except by using the p2 publisher.

The described problem (poor offline behavior) is why I very much prefer to use targlets where resolution support rollback:

https://wiki.eclipse.org/Oomph_Targlets

Note that the artifact is not jar signed, but rather only PGP signed:

https://download.eclipse.org/oomph/archive/reports/download.eclipse.org/eclipse/updates/4.30-I-builds/index/com.sun.jna_5.13.0.v20230812-1000.html

And I expect that PGPSignatureVerifier cannot verify the PGP signature of a folder.

So I don't think anything is wrong, though certainly inconvenient...

Perhaps you'd be better to mirror the repository to the local file system and use that mirror... The CBI p2 aggregator is also very handy for producing mirrors of compositions and subsets...

[jcompagner]

the problem is its not just that repository
our target consist of of many repository urls (and now also many mvn dependencies directly)
so its not about mirroring 1 site, we want a dump op the complete target on disk (for better offline support yes)

we do this already for years now and it always worked fine (why is there a nice export target button there anyway ;) )

but for some reason it can't verify the signature..

But its not that that is the only folder we have more of them. For example of the 809 plugins 10 of them are in a folder
(and 1 folder is even called "classes" which i find a bit weird)

(the first stuff)

but why that jna jar is suddenly so different and bombs out with a verify i still don't understand (and it is quite hard to debug through that, its a lot of code)

[merks]

It can't verify the signature because it's a PGP signature. It's possible to verify a jar-signature against a folder but that's not possible to do that for a PGP signature. I've never looked at what Export Target does or how it's implemented (and to be honest I had no idea there was such a button!). From what you've described, it probably should be smarter to not try to verify a PGP signature for anything that is nor a jar. (In general, when transferring artifacts that are already on disk, it doesn't make so much sense to verify them at all.)

So maybe it would be good if you created a PDE issue with a trivial test case (i.e., a target with just that one problematic bundle) that fails to export with the exception you've shown.

[laeubi]

@jcompagner just curious, have you tried using a Directory Location instead? Then P2 won't try to verify anything and it should be even faster...

[laeubi]

If I understand correctly, the export fails...

Then this might be related to:

• Rework the PDE Export Wizards for products, features, bundles and categories using P2 eclipse-pde/eclipse.pde#862

and another incarnation of newer features not fully supported by "old" Export wizards.

What's maybe not obvious to everyone is that there is a pool folder with everything in it that one could just copy and then use as you've suggested

I don't think this will work, as this doe only include everything from IU locations but not Maven or other Location types.

[merks]

I see! Then I'm not sure exactly what you mean. I thought the export fails. Are you suggesting an alternative to exporting?

[laeubi]

I thought it was a problem when reading the exported data back, but seems I got it wrong. Anyways I wonder if we simply can skip signature validation in P2 when the target is a folder? @jcompagner can you probably look if it is mentioned in the error log and copy the stack trace here?

[jcompagner]

so to make it more clear: we use directory but that is just a result of the export of the real target

the actual target file (that is also used by tycho to build our product):

https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/com.servoy.eclipse.target.target

and then the local target is one with just a directory target:

https://github.com/Servoy/servoy-eclipse/blob/master/launch_targets/eclipse_local.target

but the problem is now that the export of the first that is consumed by the last is not exporting suddenly with the new 2023.12 eclipse site the com.sun.jna plugin (That it now wants to extract, which it didn't before)

by the way, this is not directly a problem of the eclipse that is used to export it, thats the weird part here, yes i am running also 23.12 and exporting with that, but i have co workers that have exactly the same problem but they are running older eclipses..
So it is really purely the eclipse update site:

that now triggers something else...

(i didn't test it myself but i think if i would download eclipse 4.28 and then use the above simple target it still wouldn't also do that com.sun.jna.xxx.jar)

The thing is, there is no exception anywhere i only found this:

Status ERROR: org.eclipse.equinox.p2.artifact.repository code=0 Problems downloading artifact: osgi.bundle,com.sun.jna,5.13.0.v20230812-1000. children=[Status ERROR: org.eclipse.equinox.p2.artifact.repository code=0 The signature is invalid for current content]

as a status error when processing that file..

so here:

https://git.eclipse.org/c/equinox/rt.equinox.p2.git/tree/bundles/org.eclipse.equinox.p2.artifact.repository/src/org/eclipse/equinox/internal/p2/artifact/repository/MirrorRequest.java?h=master#n282

it makes the directory and then later on here:

https://git.eclipse.org/c/equinox/rt.equinox.p2.git/tree/bundles/org.eclipse.equinox.p2.artifact.repository/src/org/eclipse/equinox/internal/p2/artifact/repository/MirrorRequest.java?h=master#n291

it ends up in the next if because the status.isOK() is false and that is the above status.

but then that is collected somewhere but i don't really see any output or any warning later (in ui or in the log) that it really did go wrong.. It seems that that error status is just ignored..

[merks]

Yes, I expect all Eclipse versions will unzip the jar according to the touchpoint instruction to do that. I also expect all versions will have an issue doing some type of PGP verification of a folder while exporting.

I think the message is produced here:

https://github.com/eclipse-equinox/p2/blob/a4b04564e8f6b27bbec7571b175b96605bbd4d2e/bundles/org.eclipse.equinox.p2.artifact.repository/src/org/eclipse/equinox/internal/p2/artifact/processors/pgp/PGPSignatureVerifier.java#L249-L252

I think the thing that's wrong is that the export is doing verification in cases that can't possibly verify properly. I think that's the 3rd time I say that though, so I think we are back to this suggestion:

#1502 (reply in thread)

Or did you have something else in mind?

[merks]

I've investigated further and have opened the following issue:

eclipse-equinox/p2#374

In the end, PDE is just using p2, and it's simply not possible to PGP verify a folder's contents with the PGP signature associated with the original jar used to produce that folder so p2 should and must avoid that.