Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 229989

Summary: [Indexer] Need a way to monitor indexer Job
Product: [Tools] CDT Reporter: Tim Kelly <tim.kelly>
Component: cdt-coreAssignee: Andrew Ferguson <andrew.ferguson>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: P3 CC: andrew.ferguson, cdtdoug, mober.at+eclipse
Version: 5.0   
Target Milestone: 5.0   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Bug Depends on:    
Bug Blocks: 397652    
Attachments:
Description Flags
proposed fix
none
patch applied to HEAD cdtdoug: iplog-

Description Tim Kelly CLA 2008-05-02 12:24:20 EDT 
Build ID: 5.0.0

Steps To Reproduce:
There is currently no way I get the progress monitor for an indexing job. For example if you want to know if an indexer job was canceled there is no way to do this.


More information:
The specific use case is when generating an external source index. For example, in GeneratePDOM#run(), the indexer is set up and then joined. 

Then without knowing whether or not the indexer was cancelled a PDOM is copied to a new destination, which may actually be incomplete. 

This is a continuation of discussion from bug 223521.
Comment 1 Tim Kelly CLA 2008-05-02 12:25:54 EDT 
Transfer pertinent comments from but 223521:

Tim:
> Maybe if there was a way to kick on the indexer directly it could be done with
> a progress monitor you have access to, but I didn't see any way to do this.

Andrew:
Yes, I'd like to get Markus's input in the new bug as there is a single job for
all indexing tasks - this means even if we do have access to the indexer
progress monitor, we could be looking at the progress for indexing several
projects (especially running with the workbench). For example, you might have

 (1) User-initiated PDOM generation task
 (2) incremental update of user project

and the user cancels during (2). As we have the progress for the single indexer
Job, we cannot distinguish which tasks finished and which were cancelled. With
that in mind, it might be better to generate additional events from the
indexer.
Comment 2 Markus Schorn CLA 2008-05-05 09:55:55 EDT 
As far as I understand the issue is, that there is no convenient way of testing whether the index for a project is complete or not.

There are multiple reasons why an index may not be complete, including:
- an indexer job (containing a task affecting the project in question) has 
  been cancelled by the user,
- the indexer policy is set to manual and index has not been updated manually,
- resource changes have been notified, while the CDT core plugin was not 
  activated.
- a previous session left an incomplete index for a project.

Note, that if there is a concurrent requests to rebuild an index the task related to the first request will be cancelled. Nevertheless, the index will be complete as soon as the indexer is idle.

Drawing the right conclusion out of a set of completed/cancelled indexer tasks will be unreliable and/or cumbersom. I suggest to add a method to the index manager that allows to check an index of a project for completeness. It would need to traverse the files of the project and compare the local timestamps of the files (timestamp cached by the resources plugin) to the timestamp stored in the index.
Comment 3 Tim Kelly CLA 2008-05-05 11:01:14 EDT 
(In reply to comment #2)
 It would
> need to traverse the files of the project and compare the local timestamps of
> the files (timestamp cached by the resources plugin) to the timestamp stored in
> the index.

This sounds like a good sanity check for completeness. But based on Andrew's comment in bug 221358 (https://bugs.eclipse.org/bugs/show_bug.cgi?id=221358#c4) I didn't think this was possible. It sounds like you are saying we can do a file to file comparison of timestamps. Or does the IResource in the project necessarily have to correspond with the project pdom?
Comment 4 Markus Schorn CLA 2008-05-06 09:13:07 EDT 
> This sounds like a good sanity check for completeness. But based on Andrew's
> comment in bug 221358 (https://bugs.eclipse.org/bugs/show_bug.cgi?id=221358#c4)
> I didn't think this was possible. It sounds like you are saying we can do a
> file to file comparison of timestamps. Or does the IResource in the project
> necessarily have to correspond with the project pdom?

I agree with what Andrew writes, the sanity check would only be possible for project-based PDOMs. In the specific use-case GeneratePDOM.run() creates a project and before the PDOM of this project is exported it could be verified.
Comment 5 Andrew Ferguson CLA 2008-05-06 11:26:08 EDT 
Created attachment 98858 [details]
proposed fix

thanks for the new direction. I've had a go at implementing this - Tim, could you verify this is what you're after? (It fits with my simple scenario of triggering pre-built content creation from an test button in the IDE).
Comment 6 Tim Kelly CLA 2008-05-06 16:45:03 EDT 
(In reply to comment #5)
> Created an attachment (id=98858) [details]
> proposed fix
> 
> thanks for the new direction. I've had a go at implementing this - Tim, could
> you verify this is what you're after? (It fits with my simple scenario of
> triggering pre-built content creation from an test button in the IDE).
> 

Looks OK except one small problem: the preinclude file (the one specified in the pdom generation arguments) gets added to the index twice so the check always fails for me in: PDOMManager#areSynchronized(...)

If I change the "file.length" comparison to allow 1 or 2 files to workaround it, the sanity check is successful:
-------------------
// pre-includes may be listed twice
if(file.length < 1 || file.length > 2)
	return false;  
-------------------
Thanks, Tim
Comment 7 Andrew Ferguson CLA 2008-05-08 05:41:43 EDT 
Created attachment 99249 [details]
patch applied to HEAD
Comment 8 Andrew Ferguson CLA 2008-05-08 05:42:09 EDT 
thanks, marking as FIXED