Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 350157

Summary:

"cleanup" sometimes cleans up too much

Product:

[WebTools] WTP Releng

Reporter:

David Williams <david_williams>

Component:

releng

Assignee:

webtools.releng <webtools.releng-inbox>

Status:

RESOLVED FIXED

QA Contact:

Carl Anderson <ccc>

Severity:

normal

Priority:

CC:

ccc

Version:

3.10

Target Milestone:

---

Hardware:

OS:

Linux

Whiteboard:

Attachments:

Description	Flags
patch to "touch" prereq files	none

Description David Williams

2011-06-23 11:00:03 EDT

[Thought we had previous, similar bug reports, but can not find them right now.]

Our "cleanup" script has been a little too aggressive lately. I think.

The symptoms of this are that some long standing "pre-reqs" were downloaded again, sometimes incorrectly. One was the jonas zip we use a little in unit testing. Another 3 were
DITA-OT1.2.1_bin-ASL.zip
oagis_release_8.0_GA-withSP3.zip
WSDL_1.1_for%20OAGIS_8.0.zip

These last three were noticiable because they can not be downloaded automatically ... have some preliminary license webpages to read and agree with first.

Since this has happened before, we actually have a small cache to save some the non downloadable precache files. It is a peer of prereqsCache and named saveNonDLprereqs. There is nothing automatic about copying these, but, when this happens, or, on a completely fresh system, the three files that are there can be copied over to prereqsCache so that next time the build trys to get them, it will find them in prereqsCache, and not bother trying to download them.

But, this bug is mean to track why they were deleted in the first place.

The clean up script is supposed to work be removing prereqs if they have not been used in a long time ... like 3 months or something. And, to make sure "used" includes simply reading these files, the build scripts are supposed to "touch" them, then ever it unzips them.

But, something is obviously amiss, so will investigate. If can not reliably be fixed, we may want to avoid automatic clean up of prereqsCache and do it "by hand", like we used to. (Which is not a very good solution, as its normally a waste of space to keep unused pre-reqs around ... unused being some weekly I build of platform, and similar.

Comment 1 David Williams

2011-11-19 20:32:03 EST

Created attachment 207271 [details]
patch to "touch" prereq files

I ran into a problem similar to this one in Orbit (bug 364156) and discovered I never did "hook up" the touch tasks to run. The attached patch hooks them in, in WTP scripts, where essentially, if already exists, it is "touched", if doesn't exist its fetched. 

Simple enough fix, but I'll let Carl decide when right time is to apply it ... after all, there is a chance something would break. 

One thing that could easily break, is that now the build will _fail_ if it can not change the "modified times" of the file (such as if wtpBuild id does not have permission to). Ant is supposed to "give a warning" according to its docs, but from what I've seen in Orbit, it causes the build to fail and stop. That might be what you want ... but figured when the scripts are finally hooked in, someone should "be around" to see if anything fails and needs intervention. 

HTH

Comment 2 Carl Anderson

2011-11-30 07:28:16 EST

Committed to HEAD