Produce checksums for artifacts

[jonahgraham]

Hello Tycho users!

The next version of Eclipse (2021-12) is will log if p2 is missing checksums. For example we are seeing this new log entry:

No digest algorithm is available to verify download of
binary,epp.package.java.executable.win32.win32.x86_64,4.22.0.20211007-1500.

I have found that our project (Eclipse EPP) does not, and never has, generated download.checksum properties.

Is there a setting that forces checksums on or off? I have tried the fix-artifacts-metadata-mojo but that didn't add them (maybe it only corrects ones that are already present?).

The issues in EPP is tracked by Bug 576528

[laeubi]

Tycho generates checksums by default, what tycho version + mojo-configuration are you using to generate the update-sites?
fix-artifacts-metadata should also update or add them, so maybe something in the setup is preventing the calculation of checksums? Do you see any errors/warning in the log?

There is an extension point org.eclipse.equinox.p2.artifact.repository.artifactChecksums that handles what checksums are available, but md5 should always be present.

[jonahgraham]

I don't see any errors/warnings. Is fix-artifacts-metadata-mojo designed to add checksums where there aren't any? Can it be used on the SimRel repo to update all the checksums without having to get all the out of date projects to rebuild?

https://bugs.eclipse.org/bugs/show_bug.cgi?id=576528#c5 has a list of the projects that currently contribute jars with missing sha256.

[laeubi]

fix-artifacts-metadata-mojo calls the RecreateRepositoryApplication that reads the repository and updates the hashes.
Technically you don't need Tycho for that and should be able to use a java debugger with conditional breakpoints to find out why a particular artifact is not updated.
P2 also supports storing artifacts in a .blobstore but I havent found details about that and maybe this kind of storage prevents the calculation of hashcodes. As fart as I can see this only should applicable for pack.gz files...

[mickaelistria]

Adding checksums usually comes from p2 publisher IIRC, I think it should be investigated in p2 the ProductPublisher or the subparts that take care of creating the "binary" artifacts. It's definitely a good thing we start logging such missing checksums errors, it unveils some existing flaws other approaches have failed at noticing so far.

On Saturday, October 9, 2021, Christoph Läubrich ***@***.***> wrote: Tycho generates checksums by default, what tycho version + mojo are you

using to generate the update-sites?

fix-artifacts-metadata should also update or add them, so maybe something

in the setup is preventing the calculation of checksums? Do you see any errors/warning in the log?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. <

https://ci5.googleusercontent.com/proxy/i11_EHK9pNYFJlv5L83JwXeuMnbXB0aZFUy2B-IUDc05EmC5UDpLpFn6Qn6LbEr7bR4XneIYPXeMC13RKCSkcLsifa53p-zYu_aMVtSAWT_Q0n-nueCwiJR7xULtN-8as1XTqJdyzoVrPbsdu8VdcOv-vBMtamnGRe1WF30BTBZ9xHxl6SJPIDE_GTEUMXOR9gvLdoeOPltNjNsH2D-b9Yk952vm1ZS4Ke6dtRO9V7_V5OBOC5Pm=s0-d-e1-ft#https://github.com/notifications/beacon/AADSQQQRFOIYBPTCS5DX3B3UF7ODPA5CNFSM5FUM7JWKYY3PNVWWK3TUL52HS4DFWFCGS43DOVZXG2LPNZBW63LNMVXHJKTDN5WW2ZLOORPWSZGOAALCL6A.gif

…

-- Mickael Istria Eclipse IDE <https://www.eclipse.org/eclipseide> developer, for Red Hat Developers <https://developers.redhat.com/>

[jonahgraham]

Is there a setting that forces checksums on or off?

If the mirroring source is the "Tycho internal repo" there are no checksums to mirror. This was the case in EPP because Bug 518965 "create the p2 mirror from Tycho internal repo (so we can skip the assemble repo)." the checksums were lost in the final output as the Tyhco internal repo does not contain the checksums.

There is probably an issue to be filed for Tycho in here, but not sure what it is yet (could be: "Tycho internal repos should contain checksum properties"?)

I have tried the fix-artifacts-metadata-mojo but that didn't add them (maybe it only corrects ones that are already present?)

I don't think I ever ran this properly, there is very little feedback in this mojo, but I discovered that I was not running it on the correct directory. I raised #348 to help the next person.