Community
Participate
Working Groups
The build currently checks for missing copyrights, out of date copyrights, etc., but it doesn't check for missing legal files such as about.html. When I worked on TPTP, I wrote a script that we could reuse. This script is packaged with some other scripts in a zip file: http://dev.eclipse.org/viewcvs/index.cgi/test-results/platform/org.eclipse.hyades.use.cases/resources/Doc_smoke.zip?root=TPTP_Project&view=co Now I notice that someone has updated the utilities since I left. I've sent him an email to find out what was changed and why. We may need to use his version (he created a copy) rather than the version that I created. I'll update this bugzilla to clarify this once I've heard back.
No changes were made other than to add a .bat file for convenience when running the scripts.
Upgrading to critical/P1 because we need to keep track of the third-party plug-ins and plug-ins with encryption. The scripts that I wrote in TPTP parse the about.html files in every plug-in, whether directory or JAR plug-in, and produces a list of plug-ins: * plug-ins with missing legal files * plug-ins with non-EPL content * plug-ins with encryption * plug-ins with the standard EPL license I would like for us to create a list of the approved third-party plug-ins that COSMOS is shipping. Then we should adapt that script to compare that known list against the results found when parsing the driver. If any plug-ins show up in the build that are not in the approved list we need to pull them from CVS immediately.
Because the build doesn't do this work today, it's more accurate to make this bugzilla an enhancement.
Created attachment 92811 [details] Updated version of the EPL checking tool I found that some assumptions that I made when I first created that tool were incorrect. In the COSMOS zips, there is no one "plugins" directory to check. Instead, there can be multiple "plugins" and "lib" directories. Also, I started to add support to the program to be able to check the built zip files directly instead of needing to unzip the zips to check the plug-ins directory. This work needs to be completed. I added a "// TODO" comment in the .java file to indicate the part of the code that needs to be updated. I am attaching the latest version of this utility. Note that I also updated the resources directory because some versions of about.html weren't in the resources directory, leading to some false positive "Unknown license" flags.
To summarize the conversation that I had with Jagmit, this is what we need to do for this enhancement. 1. FindNonEPLPlugins.java needs to be updated so that it can search the zip files directly instead of needing to unzip the drivers. 2. The build needs to run this tool as a post-build step on the built zips. This utility will check the built zips to ensure that all legal files are present and will flag unknown or special licenses (third party, cryptography, disassembler). 2.1 First, the build needs to check if any plug-ins are missing an about.html file. If a plug-in is found to be missing a legal file, an email needs to be sent to cosmos-mgmt to notify the Leads. (I expect that in the short term it'll be delegated to me to follow up, but the Leads still need to be notified.) 2.2 We need to flag any unexpected special licenses. It is expected that COSMOS has some special licenses, such as for Apache Muse, but we need to send an email to cosmos-mgmt when a new special license is found. That is, we need to maintain a list of plug-ins that have a special license and when this tool is completed it will compare the plug-ins that have a special license to the known list of special licenses and if any new special licenses are found an email needs to be sent to cosmos-mgmt. (Note that some of the COSMOS redistributed plug-ins, although we have IPZillas for them, don't have an about.html. That needs to be fixed first before we can implement this step.)
Now I can run this tools on the ziped files and get the output FindNonEPLPlugins.txt. But still get exceptions (Error while unzipping file. archive is: null The exception text is invalid CEN header (bad signature)) for few jars. Have to investigate, why this is happening.
Hi Jagmit, That's a known problem. If you look in the file you'll see a //TODO Jagmit line. That's where the file is created. Note that if there's another way of parsing an embedded JAR other than extracting it that would be better than the algorithm that I chose. (i.e., extracting the JAR from the driver ZIP.) It would be best if we didn't have to extract anything at all. FYI, I started to modify this program again to search distributions for source files. There will be updates. Can you contain your work on this just to figuring out how we can parse the content of a JAR file that's embedded in a zip? Thanks, Ruth.
Have made update to program. Now it can parse jars, within a zip file. As discussed before, have created the new wiki pages, which lists BadorMissingCopyright and MissingAbout.html. And linked these new pages from the current candidate wiki page (http://wiki.eclipse.org/COSMOS/COSMOS_i10_candidate).
Have made the update, so that FindNonEPLPlugins run part of every build, and the produced a report missing About.html. Here is the output http://download.eclipse.org/technology/cosmos/1.0.0/COSMOS-1.0.0-200806090849/FindNonEPLPlugins.txt I tried publishing the report to the download page, but the report section got broken on making this modification. I will publish this report after the end of current i11 iteration.
Created attachment 109182 [details] build script
Created attachment 109183 [details] missing about.html
Created attachment 109188 [details] build reports
Created attachment 109467 [details] build report_v2
Hi saurabh, can you apply these patches
Created attachment 110471 [details] build report_v3 build reports
saurabh: Can you apply this patch (build report_v3)
Patch is applied.
I see the report, which list the plugins with missing about.html at the download page http://download.eclipse.org/technology/cosmos/1.0.0/COSMOS-1.0.0-200808191109/FindNonEPLPlugins.txt Please let me if it is ok.
Marking this bug as fixed, please open if anything else is remaining to be completed
As of TPTP 4.6.0, TPTP is in maintenance mode and focusing on improving quality by resolving relevant enhancements/defects and increasing test coverage through test creation, automation, Build Verification Tests (BVTs), and expanded run-time execution. As part of the TPTP Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes), this enhancement/defect is verified/closed by the Project Lead since this originator of this enhancement/defect has an inactive Bugzilla account and considered to be fixed. If this enhancement/defect is still unresolved and reproducible in the latest TPTP release (http://www.eclipse.org/tptp/home/downloads/), please re-open.
This enhancement/defect was mistaken closed as part of the TPTP 4.6 Bugzilla housecleaning process (see http://wiki.eclipse.org/Bugzilla_Housecleaning_Processes) since the originator of this enhancement/defect has an inactive Bugzilla account. If this enhancement/defect is still unresolved and reproducible, please re-open.
