Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 265654 - Races between DownloadJobs?
Summary: Races between DownloadJobs?
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.5 M6   Edit
Assignee: Andrew Niefer CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 248602
  Show dependency tree
 
Reported: 2009-02-20 10:27 EST by Andrew Niefer CLA
Modified: 2009-03-25 16:18 EDT (History)
2 users (show)

See Also:


Attachments
Proposed fix (2.37 KB, patch)
2009-02-20 11:07 EST, John Arthorne CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew Niefer CLA 2009-02-20 10:27:36 EST
The test org.eclipse.pde.build/P2Tests#testBug265526_265524 attached to bug 265524 is failing intermittently.  The failure is caused by:
java.io.IOException: Failed to create directory C:\Dev\Platform\Workspaces\junit-workspace\pde.build\265526\outRepo2\plugins\b_1.0.0.
	at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getOutputStream(SimpleArtifactRepository.java:716)
	at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transferSingle(MirrorRequest.java:161)
	at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transfer(MirrorRequest.java:153)
	at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.perform(MirrorRequest.java:94)
	at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:502)
	at org.eclipse.equinox.internal.p2.artifact.repository.simple.DownloadJob.run(DownloadJob.java:64)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)


This corresponds to outputFile.mkdirs() failing in SimpleArtifactRepository#getOutputStream.  There is no obvious reason why this should fail.

I suspect a race between download jobs.

My class library has the following for File#mkdirs:
    public boolean mkdirs() {
1	if (exists()) {
	    return false;
	}
2	if (mkdir()) {
 	    return true;
 	}
        File canonFile = null;
        try {
            canonFile = getCanonicalFile();
        } catch (IOException e) {
            return false;
        }
	String parent = canonFile.getParent();
3       return (parent != null) && (new File(parent).mkdirs() &&
                                    canonFile.mkdir());
    }


Consider the following
JobA(repo/plugins/b_1.0.0) 
JobB(repo/plugins/a_1.0.0.jar)
repo/plugins does not previously exist.

JobA 1 (plugins/b_1.0.0)     exists() == false
JobA 2 (plugins/b_1.0.0)     mkdir() == false (plugins does not exist)

JobB 1 (plugins/a_1.0.0.jar) exists() == false
JobB 2 (plugins/a_1.0.0.jar) mkdir() == false (plugins does not exist)
JobB 3 (plugins/a_1.0.0.jar) new File(parent).mkdirs() (recurse to parent)
JobB 1 (plugins)             exists() == false
JobB 2 (plugins)             mkdir() == true

JobA 3 (plugins/b_1.0.0)     new File(parent).mkdirs() (recurse to parent)
JobA 1 (plugins)             exists() == true, return false
JobA 3 (plugins/b_1.0.0)     new File(parent).mkdirs() == false, never try
                             canonFile.mkdir()

The failure seems to go away when I previously create the plugins folder.
Comment 1 John Arthorne CLA 2009-02-20 10:44:56 EST
This turns out to be a known bug in the Sun VM. The IBM VM's don't seem to have the same problem.

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4742723

The suggested workaround is "write your own mkdirs()"
Comment 2 John Arthorne CLA 2009-02-20 11:07:28 EST
Created attachment 126311 [details]
Proposed fix

We may need to review other uses of mkdirs(), but this is the case that is known to be highly concurrent today where the problem was observed.
Comment 3 John Arthorne CLA 2009-02-20 11:08:59 EST
Moving to Andrew to verify.
Comment 4 Andrew Niefer CLA 2009-02-20 11:53:45 EST
This patch fixes my test problems.

For the record I was using Sun 1.4.2_17-b06