Community
Participate
Working Groups
Build Identifier: I20110613-1736 when running p2.repo2runnable task fails ONLY on AIX. <target name="retrieve"> <echo>Mirroring from repo at http://artifacts.intranet/head-integration/lastSuccessful/archive.</echo> <p2.repo2runnable> <source> <repository location="http://artifacts.intranet/head-integration/lastSuccessful/archive/root/packaging.build/dist/repositories/base-platform"/> <repository location="http://artifacts.intranet/head-integration/lastSuccessful/archive/root/packaging.build/dist/repositories/packaging.distribution"/> </source> <destination location="file:/${work}"/> </p2.repo2runnable> </target> this always fails with: [java] BUILD FAILED [java] /mnt/data/hudson/workspace/head-AIX-externaltests/root/packaging.build/ant/run-externaltests.xml:392: The following error occurred while executing this line: [java] /mnt/data/hudson/workspace/head-AIX-externaltests/root/packaging.build/ant/run-externaltests.xml:403: Error occurred while transforming repository: An error occurred while collecting items to be installed. [java] at org.eclipse.equinox.p2.internal.repository.tools.tasks.Repo2RunnableTask.execute(Repo2RunnableTask.java:62) [java] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291) [java] at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37) The destination dir only contains some binaries but no features and no plugins. The same ant script runs perfect on windows, linux and os400. AND when running the same application but using the command it also works on AIX, ie: /usr/java6/bin/java -jar ../../../releng/com.id.releng.runtime/plugins/org.eclipse.equinox.launcher_1.2.0.v20110502.jar \ -application org.eclipse.equinox.p2.repository.repo2runnable \ -source http://artifacts.intranet/head-integration/lastSuccessful/archive/root/packaging.build/dist/repositories/base-platform \ -source http://artifacts.intranet/head-integration/lastSuccessful/archive/root/packaging.build/dist/repositories/packaging.distribution \ -destination work/eclipse -verbose -debug -clean tried with several versions of the IBM J9 JVM: Java(TM) SE Runtime Environment (build pap3260sr9fp1-20110208_03(SR9 FP1)) 32bit and 64bit Java(TM) SE Runtime Environment (build pap3260sr9fp2-20110627_03(SR9 FP2)) 32bit and 64bit I've enabled tracing for p2, and compared the logs from AIX and linux (sun java 1.6.0_26) and found not much difference expect the linux WILL start fetching the plugins Reproducible: Always Steps to Reproduce: 1. create 2 repositories and place them on a website 2. run ant with the p2.repo2runnable task to make a runnable from the 2 repositories
Does the operation fail on Sun's VM?
(In reply to comment #1) > Does the operation fail on Sun's VM? No, this only happens on AIX with the IBM j9 vm. There is no Sun's vm for AIX, so you can not switch vm on this system.
Created attachment 200334 [details] log with all p2 tracing enabled Attached a log with all p2 tracing enabled. I snipped the log a bit because it was 2MB. in .options I had the following enabled: org.eclipse.equinox.p2.engine/debug=true org.eclipse.equinox.p2.enginesession/debug=true org.eclipse.equinox.p2.profileregistry/debug=true org.eclipse.equinox.p2.core/debug=true org.eclipse.equinox.p2.core/reconciler=true org.eclipse.equinox.p2.core/debug=true org.eclipse.equinox.p2.core/generator/parsing=true org.eclipse.equinox.p2.core/engine/installregistry=true org.eclipse.equinox.p2.core/metadata/parsing=true org.eclipse.equinox.p2.core/artifacts/mirrors=true org.eclipse.equinox.p2.core/core/parseproblems=true org.eclipse.equinox.p2.core/planner/operands=true org.eclipse.equinox.p2.core/planner/projector=true org.eclipse.equinox.p2.core/engine/profilepreferences=true org.eclipse.equinox.p2.core/publisher=true org.eclipse.equinox.p2.core/reconciler=true org.eclipse.equinox.p2.core/core/removeRepo=true org.eclipse.equinox.p2.core/updatechecker=true org.eclipse.equinox.p2.garbagecollector=true org.eclipse.equinox.p2.repository/credentials/debug=true org.eclipse.equinox.p2.repository/transport/debug=true
Created attachment 200340 [details] log with all p2 tracing enabled of linux which succeeds attaching log file for the same ant script but on linux which successfully can create a valid repository.
Moving to p2 for now. The error here seems to be a problem with the repository: Error occurred while transforming repository: An error occurred while collecting items to be installed. At the very least, there could be a more useful error message indicating which repository and what the error was (perhaps a MD5 sum which doesn't match the artifact). Does the /mnt/data/hudson/workspace/head-AIX-externaltests/root/packaging.build/ant/data/.metadata/.log file mentioned at the end contain any additional information?
Created attachment 200364 [details] data/.metadata/.log no, it contains the same exceptions.
Can you point me to the source code where the MD5 is checked? This way I can make a simple testcase similar like in the code, so we can test/prove that the MD5 implementation on AIX IBM J9 jdk contains a bug.
The MD5 stuff happens in the org.eclipse.equinox.p2.artifact.repository bundle. The class is org.eclipse.equinox.internal.p2.artifact.processors.md5.MD5Verifier. It is an output stream, it updates the md5 on write and compares the result on close. There is also a system property "eclipse.p2.MD5Check" that can be set to "false" to disable the MD5 check.
(In reply to comment #8) > The MD5 stuff happens in the org.eclipse.equinox.p2.artifact.repository bundle. > The class is > org.eclipse.equinox.internal.p2.artifact.processors.md5.MD5Verifier. > > It is an output stream, it updates the md5 on write and compares the result on > close. > > There is also a system property "eclipse.p2.MD5Check" that can be set to > "false" to disable the MD5 check. tried with -Declipse.p2.MD5Check=false and it starts fetching plugins BUT half way through fetching the plugins from the base.platform, the same error occurs. so it's not the md5 check. I'm going to try to adjust the p2 source code and adding more e.printStacktrace(), because some important exceptions are clearly swallowed somewhere.
Found it: make sure that your temp dir is big enough!! I printed out the MultiStatus returned by org.eclipse.equinox.p2.internal.repository.tools.Repo2Runnable.run(IProgressMonitor). It contained exceptions with: java.io.IOException: There is not enough space in the file system. When downloading plugins, they are first stored in the temp dir before they are moved to the destination defined. The temp dir on our AIX is only 256MB, we have several big test plugins containing lots of test data (1 test plugin is 140MB). So for me this issue can be closed, or are marked invalid, or be used to improve logging.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
AIX is not supported for some years already.
