Bugzilla – Full Text Bug Listing
| Summary: | Project builders do not execute in specified order. | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | P Duffy <paduffy> | ||||||
| Component: | Ant | Assignee: | Darin Wright <darin.eclipse> | ||||||
| Status: | VERIFIED FIXED | QA Contact: | |||||||
| Severity: | blocker | ||||||||
| Priority: | P2 | CC: | bsimoneau, darin.eclipse, Darin_Swanson, dj.houghton, eclipse, john.arthorne | ||||||
| Version: | 3.0 | ||||||||
| Target Milestone: | 3.0.1 | ||||||||
| Hardware: | PC | ||||||||
| OS: | Windows 2000 | ||||||||
| Whiteboard: | |||||||||
| Attachments: |
|
||||||||
|
Description
P Duffy
Move to Platform/Core Further information on bogus build order. The order is actually 1, 3, 2, 1, 3...meaning all the ANT targets run, Java builder runs, followed AGAIN by all the ANT targets. Several teams are blocked from moving to Eclipse 3.0 until we can resolve this issue. And we must move to 3.0 soon for source control reasons. Please don't adjust the priority field, that is used by the developers for planning. Details and help for entering bugs can be found here: https://bugs.eclipse.org/bugzilla.html Can you: - copy /plugins/org.eclipse.core.resources/.options to the same directory as your eclipse executable - change the following lines: (there are other lines in there too if you would like more info) org.eclipse.core.resources/debug=true org.eclipse.core.resources/build/invoking=true - run eclipse with the -debug command-line arg - check the debug output (and post here if you can) - note that debug output will be printed to the console so you may have to specify the VM arg and point it to java.exe and not javaw.exe. (-vm /java/jre/bin/java) OK, here are the results per previous instructions. These are the results of a
Build Project that was run after the project was cleaned.
Install location:
file:/d:/temp/eclipse/
Configuration file:
file:/d:/temp/eclipse/configuration/config.ini loaded
Configuration location:
file:/d:/temp/eclipse/configuration/
Configuration file:
file:/d:/temp/eclipse/configuration/config.ini loaded
Framework located:
file:/d:/temp/eclipse/plugins/org.eclipse.osgi_3.0.0/
Loading framework classpath from:
file:/d:/temp/eclipse/plugins/org.eclipse.osgi_3.0.0/eclipse.properties
Framework classpath:
file:/d:/temp/eclipse/plugins/org.eclipse.osgi_3.0.0/core.jar
file:/d:/temp/eclipse/plugins/org.eclipse.osgi_3.0.0/console.jar
file:/d:/temp/eclipse/plugins/org.eclipse.osgi_3.0.0/osgi.jar
file:/d:/temp/eclipse/plugins/org.eclipse.osgi_3.0.0/resolver.jar
file:/d:/temp/eclipse/plugins/org.eclipse.osgi_3.0.0/defaultAdaptor.jar
file:/d:/temp/eclipse/plugins/org.eclipse.osgi_3.0.0/eclipseAdaptor.jar
Splash location:
d:\temp\eclipse\plugins\org.eclipse.platform_3.0.0\splash.bmp
Debug options:
file:/D:/temp/eclipse/.options loaded
Time to load bundles: 62
Starting application: 2890
Mon Aug 02 14:27:34 EDT 2004 - [Worker-1] Invoking (FULL_BUILD) on builder: Exte
rnalToolBuilder(csrc)
Mon Aug 02 14:27:54 EDT 2004 - [Worker-1] Builder finished: ExternalToolBuilder(
csrc) time: 20031ms
Mon Aug 02 14:27:54 EDT 2004 - [Worker-1] Invoking (FULL_BUILD) on builder: Java
Builder(csrc)
Mon Aug 02 14:32:07 EDT 2004 - [Worker-1] Builder finished: JavaBuilder(csrc) ti
me: 253551ms
Mon Aug 02 14:32:08 EDT 2004 - [Worker-1] Invoking (INCREMENTAL_BUILD) on builde
r: ExternalToolBuilder(csrc)
Mon Aug 02 14:32:38 EDT 2004 - [Worker-1] Builder finished: ExternalToolBuilder(
csrc) time: 30157ms
I watched the build run alongside this log and noted the following:
- Step 1 and 3 ran during the first FULL_BUILD ExternalToolBuilder. This
should have been only step 1.
- Step 2 ran during JavaBuilder. As expected.
- Step 1 and 3 ran again during the INCREMENTAL_BUILD on ExternalToolBuilder.
This should have been step 3 alone. No idea why this was an incremental versus
full build step here.
Please advise...projects are blocked on migration to 3.0 until this is resolved.
The problem with the full/incremental build is covered by bug 21029 and bug 39713. That being said, I've created a project with 2 external tool builders and they were run in the correct order. The builders are executed in the order that they are specified in the the project description's command/buildSpec list. Are you able to include your .project file (for the csrc project) and the xml files for the 2 external builders? Are the 2 external tool builders pointing to different files or a single file whose output is dependant on project state? Here are the .project and launch files. The launchers invoke different ANT
target in the same build.xml file.
The .project file...
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>csrc</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>org.eclipse.ui.externaltools.ExternalToolBuilder</name>
<arguments>
<dictionary>
<key>LaunchConfigHandle</key>
<value><project>/.externalToolBuilders/NewAntPreCompile.launch</va
lue>
</dictionary>
</arguments>
</buildCommand>
<buildCommand>
<name>org.eclipse.jdt.core.javabuilder</name>
<arguments>
</arguments>
</buildCommand>
<buildCommand>
<name>org.eclipse.ui.externaltools.ExternalToolBuilder</name>
<arguments>
<dictionary>
<key>LaunchConfigHandle</key>
<value><project>/.externalToolBuilders/NewAntPostCompile.launch</v
alue>
</dictionary>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>org.eclipse.jdt.core.javanature</nature>
</natures>
</projectDescription>
NewAntPreCompile.launch...
<?xml version="1.0" encoding="UTF-8"?>
<launchConfiguration type="org.eclipse.ant.AntBuilderLaunchConfigurationType">
<booleanAttribute key="org.eclipse.debug.ui.ATTR_LAUNCH_IN_BACKGROUND"
value="false"/>
<booleanAttribute key="org.eclipse.jdt.launching.DEFAULT_CLASSPATH"
value="true"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_RUN_BUILD_KINDS"
value="full,incremental,"/>
<stringAttribute key="org.eclipse.debug.core.ATTR_REFRESH_SCOPE"
value="${project}"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_LOCATION"
value="${workspace_loc:/csrc/dev/build.xml}"/>
<booleanAttribute key="org.eclipse.debug.core.appendEnvironmentVariables"
value="true"/>
<stringAttribute key="org.eclipse.jdt.launching.CLASSPATH_PROVIDER"
value="org.eclipse.ant.ui.AntClasspathProvider"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_ANT_TARGETS"
value="javaPreCompile,"/>
</launchConfiguration>
NewAntPostCompile.launch...
<?xml version="1.0" encoding="UTF-8"?>
<launchConfiguration type="org.eclipse.ant.AntBuilderLaunchConfigurationType">
<booleanAttribute key="org.eclipse.debug.ui.ATTR_LAUNCH_IN_BACKGROUND"
value="false"/>
<booleanAttribute key="org.eclipse.jdt.launching.DEFAULT_CLASSPATH"
value="true"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_RUN_BUILD_KINDS"
value="full,incremental,"/>
<stringAttribute key="org.eclipse.debug.core.ATTR_REFRESH_SCOPE"
value="${project}"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_LOCATION"
value="${workspace_loc:/csrc/dev/build.xml}"/>
<booleanAttribute key="org.eclipse.debug.core.appendEnvironmentVariables"
value="true"/>
<stringAttribute key="org.eclipse.jdt.launching.CLASSPATH_PROVIDER"
value="org.eclipse.ant.ui.AntClasspathProvider"/>
<stringAttribute key="org.eclipse.ui.externaltools.ATTR_ANT_TARGETS"
value="javaPostCompile,"/>
</launchConfiguration>
Can you attach your build.xml file too? Thanks. Moving to Platform/Ant to see if they have any insight... I can not provide the build.xml Paul, is that the complete console debug output? How are you initiating the build? These are the steps that I am doing and the output that I am getting: - create Java project - create HelloWorld program (so the Java builder has work to do) - create build.xml file with the following contents: <?xml version="1.0" encoding="UTF-8"?> <project name="aaa" default="init"> <target name="init"/> <target name="1"> <echo>hi1</echo> </target> <target name="2"> <echo>hi2</echo> </target> </project> - in the project properties, add a new builder named "one" which will run the "1" target from the build.xml file. - put "one" to run before the Java builder - in the project properties, add a new builder named "two" which will run the "2" target from the build.xml file. - put "two" to run after the Java builder - Project -> Clean - get the following output (deleted dates and thread for clarity): Invoking (CLEAN_BUILD) on builder: ExternalToolBuilder(bbb) Builder finished: ExternalToolBuilder(bbb) time: 10ms Invoking (CLEAN_BUILD) on builder: JavaBuilder(bbb) Builder finished: JavaBuilder(bbb) time: 10ms Invoking (CLEAN_BUILD) on builder: ExternalToolBuilder(bbb) Builder finished: ExternalToolBuilder(bbb) time: 0ms Invoking (FULL_BUILD) on builder: ExternalToolBuilder(bbb) Builder finished: ExternalToolBuilder(bbb) time: 1352ms Invoking (FULL_BUILD) on builder: JavaBuilder(bbb) Builder finished: JavaBuilder(bbb) time: 20ms Invoking (INCREMENTAL_BUILD) on builder: ExternalToolBuilder(bbb) Builder finished: ExternalToolBuilder(bbb) time: 10ms As you can see, the CLEAN is called on the project for each builder, and then a build is called after that. From what I understand, you are seeing the following behaviour from a build: - run external tool builder 1 - run external tool builder 2 - run Java builder - run external tool builder 1 - run external tool builder 2 Is it possible that you've mistaken the calls with the build-type of CLEAN_BUILD to be extra build runs? DJ, I can reproduce the problem (and you are as well I believe :-)) This is the interesting part: Invoking (FULL_BUILD) on builder: ExternalToolBuilder(bbb) Builder finished: ExternalToolBuilder(bbb) time: 1352ms Invoking (FULL_BUILD) on builder: JavaBuilder(bbb) Builder finished: JavaBuilder(bbb) time: 20ms Invoking (INCREMENTAL_BUILD) on builder: ExternalToolBuilder(bbb) Builder finished: ExternalToolBuilder(bbb) time: 10ms The FUll build of the External tool builder will trigger BOTH builds (target 1 and target 2 will execute). Then the incremental build of the External tool builder will trigger BOTH builds again. The External tool builder implementation as it stands now will only order correctly for the external tool builders and will not be correct in relation to other builders such as the Java builder. Either all external tool builders will run after the Java builder or all will run before (if any one builder is before the Java builder). I am going to investigate further but I believe I can come up with a fix for 3.0.1. A workaround should be possible by creating a dummy project that is dependant on the real project. The only purpose for the dummy project would be to run the post compile buildfile. The real project would run the pre compile before the Java builder. Ah, I see now. Yes, I get both runs both of the time...it is more obvious when I turn off auto-build, do a clean, remove debug history, and then do a ctrl-b. Looks like help is on the way. Great! Might not be relevant, but answer to the question re: the debug output I submitted: I did a clean, exited eclipse, restarted eclipse, then started the project build (to keep the clean results out of the debug log). Will this be repaired in 3.0.1 ot 3.1? Time estimate for each? Can you elaborate on the workaround? Cheers, I will likely propose some fix for 3.0.1 I believe 3.0.1 is out in Sept? but I would have to check the plans. 3.1 is slated for 2Q 2005 Workaround: Your goal is to have a builder run before the Java compile and then after the Java compile. In your current project, leave the NewAntPreCompile builder running before the Java builder. Remove the post builder. In a new "dummy" project, set up the NewAntPostCompile builder. Set this project to require the real project on its buildpath (so it is built after your real project). My quick testing shows that this works. If I provide a patched org.eclipse.ui.externaltools would you be able to help us out by testing the fix? Yes Thank you. I will attempt to have a patch by the end of the day. Any success with the workaround? Just a note that the problem was introduced with the fix for bug 63760. Created attachment 13752 [details]
External tools plugin with proposed patch
Replace the exisiting org.eclipse.ui.externaltools plugin with this one.
Eclipse will need to be restarted.
While I anticipate no problems, please ensure that you test in a workspace
where nothing critical can be lost.
I installed the new pluggin. Our build now appears to be properly functioning as it did under 2.1. Thanks! Let me know if we can be of any further assistance. Thanks for reporting back. Did you get a chance to try the workaround? I would love verification that this was a possible solution as well. The work around also appeared to work. *** Bug 68293 has been marked as a duplicate of this bug. *** Thanks for your support. With this patch the clean builders are working fine. Even with autobuild on the clean post build sees the output of the java builder, but the clean builder cycle is running twice. But with this patch no manual nor incremental builders are invoked. Thanks. I can reproduce Thomas' problem. The workaround of creating another project with the post-compile steps works for me, but the plugin patch causes the build to keep executing over and over until I run out of memory. *** Bug 71645 has been marked as a duplicate of this bug. *** I would appreciate if the parties involved in this bug report could evaluate another patch for the external tools plugin. Created attachment 13855 [details]
Patched version of the external tools plugin
The new patch looks like it does the build in the correct order and only once. I just tried this pluggin and our build appears to operate properly. My ANT pre compile buidler executes, followed by Eclipse Java builder, followed by my ANT post compile target. I do, however, appear to have a problem peforming a clean. Under 2.1...my tean perform a clean by calling an external tool that invoked an ANT target which simply blew away the root and all subdirectories of the local output folder. That was all we had to do for a clean. Simple and swift. Subsequent builds worked fine. Under 3.0...this is not working correctly. After I invoke our clean target, a subsequent project build will only execute the ANT pre and ANT post steps. The middle Java builder step is not invoked, causing the overall build to fail. So, now we are looking at using Project/clean instead of calling our external clean target. What exactly is project/clean doing that permits it to interop with the build correctly, yet our external clean target will not? Thanks! Thanks for reporting back...great to have more eyes verifying the correct functionality! Clean problem: are you refreshing the affected file hierarchies after you external clean? See the Refresh tab of the Ant launch configuration. And just to clarify...the clean problem is independant of the problems of this bug report? Glad you asked re: refresh on clean. When I turn on the external "clean" target refresh, execute the clean to blow away the output folder tree, then invoke the project build...the Eclipse Java builder will not run. Pre compile ANT target runs. No Java vuilder Post compile ANT target runs. The overall build fails. P.S. Even if I run the project/clean after having this problem with the external clean, the build fails. The project/clean appears to do nothing. Subsequent build only runs the ANT pre and post compilation steps. I have no way to restore a working build. ? What have you set to be refreshed? I have submitted this to be included in 3.1 M1 What did I refresh: project containing the selected resource. Also selected recursively include sub-folders. I do not know why the Java builder would not be triggered after a clean. Possibly the Core people could comment? I do not believe that there are any remaining issues for the Ant integration that are not described in other bug reports (bug 71438) I also don't know why your Java builder is not executing. Confirm that it is still in the project build spec and enabled by looking at the Builders property page for your project. Also check for build path errors in the problems view. If you still experiencing problems, I suggest entering a bug against JDT Core. Build spec was not changed at all. No build path errors. Same symptoms as my original problem report. 1. Invoke external ANT clean target, which simply blows away the entire build output folder and all sub folders. 2. Invoke project build...nothing happens. 3. Refresh the project. 3. Build the project. Runs ANT pre compile, does not run Java Builder, runs ANT post comple target. From build log... Fri Aug 13 16:00:37 EDT 2004 - [Worker-3] Invoking (INCREMENTAL_BUILD) on builde r: ExternalToolBuilder(csrc) Fri Aug 13 16:00:53 EDT 2004 - [Worker-3] Builder finished: ExternalToolBuilder( csrc) time: 15234ms Fri Aug 13 16:00:53 EDT 2004 - [Worker-3] Invoking (INCREMENTAL_BUILD) on builde r: JavaBuilder(csrc) Fri Aug 13 16:00:53 EDT 2004 - [Worker-3] Builder finished: JavaBuilder(csrc) ti me: 47ms This is the same description as my original problem. Project/clean of the project does not help. Subsequent project build executes ANT precompile, skips Java Builder, executes port compile. ???? If you are doing incremental builds then you are hitting bug 71438 which we cannot fix until but 21029 is fixed up (which is being worked on). For this to work you need to trigger a full build which is what we have attempted to fix in this bug report. My understanding is that this is triggered by doing Project>Clean to remove the build state. The next build will then, by definition be a full build. I seem to have stumbled across a workaround. Starting after a successful FULL build. - Call external ANT clean that blows away output folder. - DO NOT DO A PROJECT REFRESH - Project/build does nothing. - Project/clean - Project build then correctly does FULL build (ANT ext pre target, Java Builder, ANT ext port target). Does not like refresh after I flow away the output folder. Suggested action? File another bug? This bug, executing builders in project specified order, is not yet fixed (even with the pluggin patch) The following sequence fails the build, which worked fine under 2.1 - Call external ANT "clean" target (deletes output folder and all subfolders....thats all it does). We do this to blow away ALL build by- products, including a few java sources that are generated from syntax descriptions then built as part of the main java source build. - Refresh project. Note that without refresh, subsequent project/build does nothing. - Project/build does not execute builders in project specified order...build fails. It executes ANT pre compilation, ANT post compilation, Java Builder, ANt pre (again), ANT post (again). It should have executed ANT pre, Java builder, ANT post. Build is failing. This all worked fine under 2.1. Here is the debug log for the last build I described. The build was invoked, after a project refresh, via the project/build operation. Why is the log indicating incremental builds? Ant pre and post are running EACH time the external tool builder is run....pre ONLY should be run on the first external tool builder call, post ONLY on the second. ??? Mon Aug 16 13:42:52 EDT 2004 - [Worker-15] Invoking (INCREMENTAL_BUILD) on build er: ExternalToolBuilder(csrc) Mon Aug 16 13:43:10 EDT 2004 - [Worker-15] Builder finished: ExternalToolBuilder (csrc) time: 17485ms Mon Aug 16 13:43:10 EDT 2004 - [Worker-15] Invoking (INCREMENTAL_BUILD) on build er: JavaBuilder(csrc) Mon Aug 16 13:43:13 EDT 2004 - [Worker-15] Builder finished: JavaBuilder(csrc) t ime: 3204ms Mon Aug 16 13:43:13 EDT 2004 - [Worker-15] Invoking (INCREMENTAL_BUILD) on build er: ExternalToolBuilder(csrc) Mon Aug 16 13:43:26 EDT 2004 - [Worker-15] Builder finished: ExternalToolBuilder (csrc) time: 13110ms how I see it: your "blowing away of the build artifacts" is not resetting the build state of the builders (as a project Clean would do). Therefore you are getting an incremental build and this is bug 71438. If you preface your external clean with a Project>Clean, is the behavior correct? yes So you are seeing bug 71438. This bug is just going to cover "full" builds which I believe is fixed. Thanks for sticking with us on this. DarinW, please verify that Full build builders execute in the specified order. Released to both 3.0.1 and 3.1 Verified. |