Community
Participate
Working Groups
After fixing bug 484268 and bug 334281, the workaround (API filters) got removed via bug 477904. Since our build uses an older version of API Tools to generate the reports we now get compatibility warnings in one of our API Tools Verification Reports: http://download.eclipse.org/eclipse/downloads/drops4/N20160421-2000/apitools/analysis/html/org.eclipse.core.databinding.property/report.html The API Tools used during the build have to be from the latest I-build (or newer).
This is the commit that changes the version of Eclipse that is used during the build to create the .api_description files: http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=4fbbc8e39910798e35176f713fb86acf310b8707 There is another possibly relevant change needed in "getBasebuilderAndTools.xml" which is where we generate the "post-build" reports. Does the "reporter" need changing too? Or, just the "generation"?
Just to document it here ... and then later on a wiki, the version of "eclipse-run" and "apitools.iu" is in this version of getBaseBuilerAndTools.xml (there's two of them :/ ) http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/tree/eclipse.platform.releng.tychoeclipsebuilder/eclipse/getBaseBuilderAndTools.xml#n73 As can be seen there, we are currently using the version of "M3" to generate the reports. I should update this in any case, either to M6, or to the latest I-build, if needed. If anyone knows which (M6 or latest I-build) or can tell after the 10 AM rebuild today, let me know.
(In reply to David Williams from comment #1) > There is another possibly relevant change needed in > "getBasebuilderAndTools.xml" which is where we generate the "post-build" > reports. Does the "reporter" need changing too? Or, just the "generation"? The bugs mentioned in comment 0 were not related to API descriptions but API Tools API comparison (affects UI and reports). Hence we need to update the report generation to at least the latest I-build.
Here's the commit that updates the report generation tools to the latest I-build. http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=d40579f56a3deb091655d0c425fc09b0f299e1cd The commit is larger that it would normally have to be, since, as one of the "TODOs" in that file mentioned, "we should read 'eclipserun-repo' from [properties file]". I *think* we can do that right now with the changes in that commit, but this "install base and tools" method is called from several places, so I am not 100% positive the properties are defined at the time of the "first call" -- 90% positive, just not 100%. So I also included some debugging statements, and as "backup" set the value to the value we want, if it is not already set. Do you want another N-build today to confirm? Or will the 8 PM one suffice?
Something went wrong. There are two possible reasons: 1) The update was not correct 2) There is a bug in the latest version of the API Tools. While the bugs mentioned in comment 0 look good in the 'API Tools Verification Reports' report, we now have lots of invalid issues reported by http://download.eclipse.org/eclipse/downloads/drops4/N20160422-2000/apitools/freeze_report.html Adding Vikas to check whether changes in API Tools could cause this.
(In reply to Dani Megert from comment #5) > Something went wrong. I'll document what I know, but don't see an obvious reason for the results observed. The version of API tools installed, during the builds, since the change is org.eclipse.pde.api.tools_1.1.0.v20160418-1724.jar I'm assuming "4/18" is the right date? = = = = = There is a new error logged during the post-build processing (the first part is pasted below). That implies to me something is wrong with "api tools" (though, not necessarily, I guess?) This appears to be during the "last step" of the tasks that are done in api-tools-builder.xml. <apitooling.apideprecation_reportconversion xmlfile="${deprecation_report}" debug="true" htmlfile="${deprecation_html}" /> So, not sure if that would effect "freeze reports" per se, or if it just indicates something else is going wrong. = = = = = = = BUILD FAILED /shared/eclipse/builds/4N/siteDir/eclipse/downloads/drops4/N20160423-1500/eclipse.platform.releng.aggregator/eclipse.platform.releng.tychoeclipsebuilder/eclipse/buildScripts/api-tools-builder.xml:122: java.lang.NumberFormatException: For input string: "DEPRECATION" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:580) at java.lang.Integer.parseInt(Integer.java:615) at org.eclipse.pde.api.tools.internal.tasks.APIDeprecationReportConversionTask$ConverterDefaultHandler.startElement(APIDeprecationReportConversionTask.java:119) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1363) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2786) at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848) at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777) at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141) at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:649) at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:333) at javax.xml.parsers.SAXParser.parse(SAXParser.java:328) at org.eclipse.pde.api.tools.internal.tasks.APIDeprecationReportConversionTask.execute(APIDeprecationReportConversionTask.java:337) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
(In reply to David Williams from comment #6) > (In reply to Dani Megert from comment #5) > > Something went wrong. > > I'll document what I know, but don't see an obvious reason for the results > observed. > > The version of API tools installed, during the builds, since the change is > > org.eclipse.pde.api.tools_1.1.0.v20160418-1724.jar > > I'm assuming "4/18" is the right date? > > = = = = = > > There is a new error logged during the post-build processing (the first part > is pasted below). What are the others? > This appears to be during the "last step" of the tasks that are done in > api-tools-builder.xml. > > <apitooling.apideprecation_reportconversion > xmlfile="${deprecation_report}" > debug="true" > htmlfile="${deprecation_html}" /> > > So, not sure if that would effect "freeze reports" per se, or if it just > indicates something else is going wrong. No, this affects the deprecation report, which is no longer generated. From the last test result logs: No deprecation report. Nothing deprecated since 4.5.2. Filed bug 492337 to track that.
(In reply to Dani Megert from comment #5) > Something went wrong. There are two possible reasons: > > 1) The update was not correct > > 2) There is a bug in the latest version of the API Tools. I looked at the logs and verified that the correct baseline was used for the freeze report. I then looked closer and found the problem: The freeze report is now supposed to also list compatible changes (see bug 490770). I've reopened that bug and reverted the change for now.
(In reply to Dani Megert from comment #7) > > > > There is a new error logged during the post-build processing (the first part > > is pasted below). > > What are the others? That was the only one. I meant "new" as in "one I had never seen before". And by "first part" meant the "top" of the stack track. (the rest of it was pretty much more of the same). (In reply to Dani Megert from comment #8) > I looked at the logs and verified that the correct baseline was used for the > freeze report. I then looked closer and found the problem: The freeze report > is now supposed to also list compatible changes (see bug 490770). I've > reopened that bug and reverted the change for now. I assume this means once we get a good I-build, then we update the "eclipserun" value again to point to that I-build? I ask since I wonder if you have considered moving any of this function out to the "buildtools" repository? I know some of the API function must be in the SDK, but anything that is purely "production build reporting" might be better suited for that repository, since it can be built and tested relatively independent of the rest of the build and prepared "ahead of time" so we don't have to wait on an I-build to change the tools. That's the same repo where things like "generateIndex" is located. Just a thought for the future.
(In reply to David Williams from comment #9) > I assume this means once we get a good I-build, then we update the > "eclipserun" value again to point to that I-build? Yes. > I ask since I wonder if > you have considered moving any of this function out to the "buildtools" > repository? Sorry my ignorance, but I don't know what the buildtools repository is.
(In reply to Dani Megert from comment #10) > (In reply to David Williams from comment #9) > > I assume this means once we get a good I-build, then we update the > > "eclipserun" value again to point to that I-build? > > Yes. Updated with the 1300 I-build. http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=56c3a03af235782e8d9758ced63daaa766c87234 = = = = = = = Not sure when you plan to fix/implement bug 490770, but I suggest a new bug to "update tools" then it is. We will, of course, update 'eclipserun' after the milestone, to produce RC1, which is our custom. = = = = = = = > > I ask since I wonder if > > you have considered moving any of this function out to the "buildtools" > > repository? > > Sorry my ignorance, but I don't know what the buildtools repository is. Well, it's the repo where we keep some build tools. :) It is "manually" built, since changes seldom, and its p2 repo is kept on build.eclipse.org, since very specific to our build. Besides "Generate Index" ant task (which include the "drop file" and "test summaries"), there is also one to "verify compile", and "scan logs for comparator errors" -- and probably a few others. Again, just a long term thought, if these "reporting" parts of the API tools are of the same nature. http://git.eclipse.org/c/platform/eclipse.platform.releng.buildtools.git/
In closing, I also meant to mention there was a comparator error based on .api_descriptions changing, I assume due to this tool changing. http://download.eclipse.org/eclipse/downloads/drops4/I20160425-1300/buildlogs/comparatorlogs/buildtimeComparatorUnanticipated.log.txt Once we are comfortable all is correct, several bundles may need to be "touched" so their .api_descriptions are current?
(In reply to David Williams from comment #12) > In closing, I also meant to mention there was a comparator error based on > .api_descriptions changing, I assume due to this tool changing. > > http://download.eclipse.org/eclipse/downloads/drops4/I20160425-1300/buildlogs/comparatorlogs/buildtimeComparatorUnanticipated.log.txt > > > Once we are comfortable all is correct, several bundles may need to be > "touched" so their .api_descriptions are current? I only found one comparator error due to new API descriptions. Fixed with http://git.eclipse.org/c/platform/eclipse.platform.ui.git/commit/?id=0c5bda44287094258bd6b0f34f579d644dfada50
David, can you verify whether indeed I20160425-1300 was used to run the API Tools? The latest results are weird: - as expected, the deprecation report is back and looks OK - as expected, the freeze report is again empty - BUT: the latest API Tools should no longer report the following: http://download.eclipse.org/eclipse/downloads/drops4/I20160426-1615/apitools/analysis/html/org.eclipse.core.databinding.property/report.html I have verified in my workspace that API Tools does not issues those things (as expected). Olivier, is the analysis done differently for the report than in the workspace? Maybe using the .api_description file also for the current build and not just the baseline? Is there anything in the logs, e.g. an exception?
(In reply to Dani Megert from comment #14) > David, can you verify whether indeed I20160425-1300 was used to run the API > Tools? The latest results are weird: The log of installing the tools says Installing org.eclipse.pde.api.tools 1.1.0.v20160425-0845 Which I'd assume is the correct version (time "checked in" to Git) given the I-build date and time. > - BUT: the latest API Tools should no longer report the following: > http://download.eclipse.org/eclipse/downloads/drops4/I20160426-1615/apitools/ > analysis/html/org.eclipse.core.databinding.property/report.html That is the one that had a comparator error, where an "old bundle" was swapped in for the newly built one. So, I would suspect related to that, rather than the tools? > > I have verified in my workspace that API Tools does not issues those things > (as expected). > > Olivier, is the analysis done differently for the report than in the > workspace? Maybe using the .api_description file also for the current build > and not just the baseline? > > > Is there anything in the logs, e.g. an exception? No exceptions in error log, as before. The regular (sysout) log is too large to "inspect", but no occurances of [ERROR or "BUILD FAILED".
(In reply to David Williams from comment #15) > That is the one that had a comparator error, where an "old bundle" was > swapped in for the newly built one. So, I would suspect related to that, > rather than the tools? Well, I'm not sure it uses the .api_description of the stuff that's being built. At least in the IDE it only uses them for the baseline (4.5.2 in our case). And, when we first updated the tools, the errors were not there.
(In reply to Dani Megert from comment #16) > (In reply to David Williams from comment #15) > > That is the one that had a comparator error, where an "old bundle" was > > swapped in for the newly built one. So, I would suspect related to that, > > rather than the tools? > > Well, I'm not sure it uses the .api_description of the stuff that's being > built. At least in the IDE it only uses them for the baseline (4.5.2 in our > case). And, when we first updated the tools, the errors were not there. You might be able to spot something in the "publish_eclipse" log. For example, in http://download.eclipse.org/eclipse/downloads/drops4/I20160426-1615/buildlogs/mb080_publish-eclipse_output.txt From my casual reading, it appears the "win32.zip" even from "current build" is downloaded and used for analysis. .../downloads/drops4/I20160426-1615/eclipse-SDK-I20160426-1615-win32.zip
(In reply to Dani Megert from comment #14) > is the analysis done differently for the report than in the > workspace? Maybe using the .api_description file also for the current build > and not just the baseline? This is the case and confirmed. After having the correct .api_description file, the report got green!
