Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 350157 - "cleanup" sometimes cleans up too much
Summary: "cleanup" sometimes cleans up too much
Status: RESOLVED FIXED
Alias: None
Product: WTP Releng
Classification: WebTools
Component: releng (show other bugs)
Version: 3.10   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: webtools.releng CLA
QA Contact: Carl Anderson CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-23 11:00 EDT by David Williams CLA
Modified: 2018-06-29 15:14 EDT (History)
1 user (show)

See Also:


Attachments
patch to "touch" prereq files (785 bytes, patch)
2011-11-19 20:32 EST, David Williams CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2011-06-23 11:00:03 EDT
[Thought we had previous, similar bug reports, but can not find them right now.]

Our "cleanup" script has been a little too aggressive lately. I think. 

The symptoms of this are that some long standing "pre-reqs" were downloaded again, sometimes incorrectly. One was the jonas zip we use a little in unit testing. Another 3 were 
DITA-OT1.2.1_bin-ASL.zip
oagis_release_8.0_GA-withSP3.zip
WSDL_1.1_for%20OAGIS_8.0.zip

These last three were noticiable because they can not be downloaded automatically ... have some preliminary license webpages to read and agree with first. 

Since this has happened before, we actually have a small cache to save some the non downloadable precache files. It is a peer of prereqsCache and named saveNonDLprereqs. There is nothing automatic about copying these, but, when this happens, or, on a completely fresh system, the three files that are there can be copied over to prereqsCache so that next time the build trys to get them, it will find them in prereqsCache, and not bother trying to download them. 

But, this bug is mean to track why they were deleted in the first place. 

The clean up script is supposed to work be removing prereqs if they have not been used in a long time ... like 3 months or something. And, to make sure "used" includes simply reading these files, the build scripts are supposed to "touch" them, then ever it unzips them. 

But, something is obviously amiss, so will investigate. If can not reliably be fixed, we may want to avoid automatic clean up of prereqsCache and do it "by hand", like we used to. (Which is not a very good solution, as its normally a waste of space to keep unused pre-reqs around ... unused being some weekly I build of platform, and similar.
Comment 1 David Williams CLA 2011-11-19 20:32:03 EST
Created attachment 207271 [details]
patch to "touch" prereq files

I ran into a problem similar to this one in Orbit (bug 364156) and discovered I never did "hook up" the touch tasks to run. The attached patch hooks them in, in WTP scripts, where essentially, if already exists, it is "touched", if doesn't exist its fetched. 

Simple enough fix, but I'll let Carl decide when right time is to apply it ... after all, there is a chance something would break. 

One thing that could easily break, is that now the build will _fail_ if it can not change the "modified times" of the file (such as if wtpBuild id does not have permission to). Ant is supposed to "give a warning" according to its docs, but from what I've seen in Orbit, it causes the build to fail and stop. That might be what you want ... but figured when the scripts are finally hooked in, someone should "be around" to see if anything fails and needs intervention. 

HTH
Comment 2 Carl Anderson CLA 2011-11-30 07:28:16 EST
Committed to HEAD