Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 395138

Summary: Different files of com.ibm.icu 4.4.2.v20110823 in Juno release repos
Product: Community Reporter: Anders Hammar <anders.g.hammar>
Component: Cross-ProjectAssignee: David Williams <david_williams>
Status: RESOLVED WONTFIX QA Contact:
Severity: normal    
Priority: P3 CC: daniel_megert, david_williams
Version: unspecified   
Target Milestone: ---   
Hardware: All   
OS: All   
Whiteboard:

Description Anders Hammar CLA 2012-11-27 04:56:25 EST
I just discovered that there are different versions of com.ibm.icu 4.4.2.v20110823 in the 201206270900 Juno release repo and the 201209280900 Juno release repo. Their checksum (MD5) differs which is also seen in the artifacts.xml files of respective repo.
This totally fools any proxying software like Nexus, which assumes that a release is stable and doesn't change. What happens is that Nexus caches the file on the first request (which in my case was after the initial Juno release, i.e. the 201206270900 repo) but uses the most recent metadata (artifact.xml, from the 201209280900 repo). This fails build software like Tycho as it verifies the checksum in the metadata with the downloaded file, which will then be different.

Would it be possible to correct this? Also, please make sure you don't use different builds of the same release in the future to prevent this issue again.
Comment 1 Dani Megert CLA 2012-11-27 04:58:59 EST
(In reply to comment #0)
> I just discovered that there are different versions of com.ibm.icu
> 4.4.2.v20110823 in the 201206270900 Juno release repo and the 201209280900
> Juno release repo.

You mean the complete bundle version is the same but the content is different? If so, do you know what's actually different?
Comment 2 Anders Hammar CLA 2012-11-27 05:24:22 EST
I've found yet another example of this I think:
org.apache.jasper.glassfish 2.2.2.v201205150955

My Tycho build is complaining of checksum error. Didn't do any further investigation though. Had to delete the cached file in Nexus to get around this. As an admin of the corp Nexus instance I have that option, but it's not an option for the developers.
Comment 3 Anders Hammar CLA 2012-11-27 05:27:18 EST
(In reply to comment #1)
> (In reply to comment #0)
> > I just discovered that there are different versions of com.ibm.icu
> > 4.4.2.v20110823 in the 201206270900 Juno release repo and the 201209280900
> > Juno release repo.
> 
> You mean the complete bundle version is the same but the content is
> different? If so, do you know what's actually different?

Yes, the version specified in artifacts.xml is the same but he files are different (different checksum). I had a quick look and see that some files in META-INF of the file in the 201209280900 repo have a newer timestamp (12-01-19).
Comment 4 David Williams CLA 2012-11-27 08:35:57 EST
My guess is these differ (only) in the certificate used when signing the bundle and I know there are those that feel strongly "that is still different, the qualifier should change" (but, not all of us :) ... see bug 317764). 

I peeked on the file system and computed md5sum for one of the bundles mentioned (icu4j) and can confirm it is the Juno SR0 version in the common repo that stands out ... different from what's in Orbit: 


Orbit

b0d01f0723b407b4018abeae78dcb068  ./tools/orbit/downloads/drops/R20120119162704/repository/plugins/com.ibm.icu_4.4.2.v20110823.jar
b0d01f0723b407b4018abeae78dcb068  ./tools/orbit/downloads/drops/R20120526062928/repository/plugins/com.ibm.icu_4.4.2.v20110823.jar

Common Repos

9de44d34d3e5b58ddb9b1dcf33ec9248  ./releases/juno/201206270900/plugins/com.ibm.icu_4.4.2.v20110823.jar
b0d01f0723b407b4018abeae78dcb068  ./releases/juno/201209280900/plugins/com.ibm.icu_4.4.2.v20110823.jar


What's in Common repo appears to follow what's in Eclipse (rather than what's in Orbit). 
9de44d34d3e5b58ddb9b1dcf33ec9248  ./eclipse/updates/3.8/R-3.8-201206081200/plugins/com.ibm.icu_4.4.2.v20110823.jar
9de44d34d3e5b58ddb9b1dcf33ec9248  ./eclipse/updates/4.2/R-4.2-201206081400/plugins/com.ibm.icu_4.4.2.v20110823.jar

b0d01f0723b407b4018abeae78dcb068  ./eclipse/updates/3.8/R-3.8.1-201209141540/plugins/com.ibm.icu_4.4.2.v20110823.jar
b0d01f0723b407b4018abeae78dcb068  ./eclipse/updates/4.2/R-4.2.1-201209141800/plugins/com.ibm.icu_4.4.2.v20110823.jar

Looking further back in time, in Eclipse repos, we seem different still for Indigo Release:
447d585e10e46d04773feb3ec3d8159f  ./eclipse/updates/3.7/R-3.7.2-201202080800/plugins/com.ibm.icu_4.4.2.v20110823.jar

This 3.7 one are in an Orbit repo, from what I can tell; This might have been before we started using the comparator, in Orbit? 

So, one conclusion to draw: the comparator is only as good as what you compare it too! But, there might have been
signs in comparator logs for Eclipse that were missed, not sure.

And, we do not use a comparator for common repo. Someone would have to invest in that to make that happen (either technology or change process, or both). 

Bottom line, I am very sympathetic, but am not sure what "we" can do to fix "your" repository. I say that knowing first hand of others besides you who have to "flush their cache" occasionally to pick up a new bundle that has not changed its qualifier. 

Suggestions/contributions welcome.
Comment 5 Anders Hammar CLA 2012-11-28 05:16:41 EST
Thanks for looking into this!

I'm not familiar enough with the Eclipse process (I had to lookup "Orbit" e.g.) to really suggest anything. I do know though that many Eclipse projects are switching to Tycho and as Tycho caches artifacts locally in the same way as Maven does, devs will run into issues if you have more than one version of an artifact of a specific qualifier. We've seen this problem in the Maven world, where I work, and learned the hard way to never change or delete a released artifact.

The drawback of being forced to flush the cache of repo managers (such as Nexus), is that eventually people will get tired of the issues and just stop caching. So instead of getting help reducing the load on your repos, you will get the full load of every Eclipse user.
Comment 6 David Williams CLA 2013-06-01 16:55:47 EDT
I'm not sure of current Juno's repo for ICU4J ... or Kepler's for that matter (but, do know there's some few bundles that differ in qualifier only) but there's not further action planned here, so marking as "won't fix".