Description Carl Knight 2011-02-07 20:30:03 EST

Build Identifier: 3.6.1 M20100909-0800

[b]Background[/b]

We have multiple versions of our script engine. They use different grammas (newer building on the older). If you change interpretters, we kick off a clean and rebuild with the gramma associated with the new interpreter chosen if it's gramma differs from the currently selected.

[code]
if( hasOutstandingConfigurationChange() == true ){
    try {
        //TODO Figure out what to do about the problem view when switching grammars
        //     results in parse errors for one but not the other.
        ResourcesPlugin.getWorkspace().build(IncrementalProjectBuilder.CLEAN_BUILD, null);
        ResourcesPlugin.getWorkspace().build(IncrementalProjectBuilder.FULL_BUILD, null);
    } catch (CoreException e) {
        e.printStackTrace();
    } finally {
        reset();
    }
}
[/code]

What I noticed was that even when the gramma was switched, the errors were those of the last build.

[b]The suspect.[/b]

When I request a CLEAN_BUILD, as near as I can determine, the ModelManager cache of module information is never cleared. i.e. [code]ModelManager.getModelManager().getSourceModuleInfoCache().clear()[/code] never happens. This means that when I request a FULL_BUILD following the clean, the following happens inside ParserBuildParticipant.build(IBuildContext)...

[code]
		public void build(IBuildContext context) throws CoreException {
			IModuleDeclaration moduleDeclaration = (ModuleDeclaration) context
					.get(IBuildContext.ATTR_MODULE_DECLARATION);
			if (moduleDeclaration != null) {
				// do nothing if already have AST - optimization for reconcile
				return;
			}
[/code]
The following call to getSourceModuleInfoCache() retrieves stale module and problem data.
[code]
			// get cache entry
			final ISourceModuleInfo cacheEntry = ModelManager.getModelManager()
					.getSourceModuleInfoCache().get(context.getSourceModule());

			// check if there is cached AST
			moduleDeclaration = SourceParserUtil.getModuleFromCache(cacheEntry,
					context.getProblemReporter());
			if (moduleDeclaration != null) {
				// use AST from cache
				context.set(IBuildContext.ATTR_MODULE_DECLARATION,
						moduleDeclaration);
				return;
			}
[/code]
And we return (above), instead of proceeding with a reparse (below). If I have cleaned, I expect a build to require the reparsing of files, not the loading of stale cached modules and problem information.
[code]
			// create problem collector
			final ProblemCollector problemCollector = new ProblemCollector();
			// parse
			moduleDeclaration = parser.parse((IModuleSource) context
					.getSourceModule(), problemCollector);
			// put result to the cache
			SourceParserUtil.putModuleToCache(cacheEntry, moduleDeclaration,
					problemCollector);
			context.set(IBuildContext.ATTR_MODULE_DECLARATION,
					moduleDeclaration);
			// report errors to the build context
			problemCollector.copyTo(context.getProblemReporter());
		}
[/code]

[b]Solution (and by no means the right one)[/b]

Create a class that implements IResourceChangeListener and listen for requests to CLEAN (either by menu or programattically, works in either case) and call SourceParserUtil.clearCache(). As near as I can tell, this happens before anything else gets interested in the cache and so seems safe. Please correct me if I'm wrong.

[code]
	@Override
	public void resourceChanged(IResourceChangeEvent event) {
		if( (event.getType() & IResourceChangeEvent.PRE_BUILD) != 0 && event.getBuildKind() == IncrementalProjectBuilder.CLEAN_BUILD ){
			try{
				SourceParserUtil.clearCache();
			} catch (Throwable e) {
				e.printStackTrace();
			}
		}
	}
[/code]

Is this a bug in the DLTK or an issue with my use of it?

Thanks.

- Carl Knight.

Reproducible: Sometimes

Steps to Reproduce:
This depends upon the state of the DLTK ModuleDeclaration cache. See the details copy and pasted from my original forum post.

1. Request a clean and build.
2. Check where the ModuleDelcaration has come from (stale cache) see details above.
3. If it has come from the stale cache, the clean and build has effectively failed.