Comment 1 Jeff McAffer 2004-06-16 11:06:56 EDT

I just renamed the file back to plugin.xml and it seems that someone 
(configurator?) noticed and fixed things up.  This is strange.  Adding Dorian 
for comment

Comment 2 Dorian Birsan 2004-06-16 11:17:46 EDT

Jeff, is this in self hosting mode or in a regular eclipse ?

Comment 3 Jeff McAffer 2004-06-16 11:26:21 EDT

regular eclipse.

Comment 4 Dorian Birsan 2004-06-16 11:51:47 EDT

Here is my guess:

when the workbench starts, the workbench has an indirect dependency on lucene, 
via the help.ide plugin, and the plugin is probably verified by the runtime. 

The update configurator will not detect changes to plugin.xml.

Comment 5 Rafael Chaves 2004-06-16 12:32:49 EDT

I could reproduce this with the install location on NTFS, but not on FAT32.
After "disabling" lucene, I can run fine once, the second time lots of failures
will happen.

In ConfigurationActivator.canRunWithCachedData():

private boolean canRunWithCachedData() {
  return  
    !"true".equals(System.getProperty("osgi.checkConfiguration")) && 			   
System.getProperties().get("osgi.dev") == null && 
    lastTimeStamp==configuration.getChangeStamp() &&   
lastStateTimeStamp==Platform.getPlatformAdmin().getState(false).getTimeStamp();
}

lastTimeStamp was older than configuration.getChangeStamp(), causing lucene to
be uninstalled. refreshBundles will return abruptly (without doing anything)
because there seem to be no bundles to be resolved, so the problem will only be
noticed next time we run.

Comment 6 Jeff McAffer 2004-06-16 12:58:36 EDT

I have a strong suspicion that I have forgotten a case here but...

I thought that if checkConfiguration was NOT specified as "true", we should not 
check for changes to the plugin.xmls.  You can still check for new plugins but 
there is no real need to poll each plugin's timestamp.

Comment 7 Dorian Birsan 2004-06-16 13:29:30 EDT

Hmmm.... it looks like the configurator is doing more work than needed in this 
particular case. As per our earlier discussions, the configurator should run 
out of cached data when no plugins have been added or removed. To check for 
this, it is sufficient (not on FAT systems) to check the parent folder 
(plugins) for timestamp changes.
So, if you still want this, please move the bug to update and I recommend 
having it done for RC3.

Comment 8 Jeff McAffer 2004-06-16 13:59:03 EDT

+1 for RC3 from me.

One clarification, I think that update should do something if a plugin dir is 
added or removed.  If someone is messing with the contents of a plugin (as I 
was) it should be completely ignored unless osgi.checkConfiguration = true. 
Note that if osgi.dev != null we set osgi.checkConfiguration = "true" so you 
don't need to test that as well.

Comment 9 Dorian Birsan 2004-06-16 15:28:47 EDT

The problem seems to be in SiteEntry.computePluginsChangeStamp().
There, we are computing the plugins timestamp by looking at both the plugins 
directory, and each plugin directory. While this may improve plugin discovery 
on FAT file systems, it actually decreases the performance on the other file 
systems. Since FAT is already buggy, this code should be removed., or only 
executed when osgi.checkConfiguration is set.

Comment 10 Jeff McAffer 2004-06-16 15:34:57 EDT

sounds good.  

Comment 11 Dorian Birsan 2004-06-16 16:13:27 EDT

Created attachment 12326 [details]
patch to skip over computing timestamps for each plugin

Comment 12 Konrad Kolosowski 2004-06-16 18:17:41 EDT

I have looked at the patch, noticed two issues:
1. The line
 return  !"true".equals(System.getProperty("osgi.checkConfiguration"))
should probably not be commented out.
2. The computeFeaturesChangeStamp() methods has now two ways of computing the 
time stamp, depending on the presence of osgi.checkConfiguration.  This 
introduces inconsistency how the time stamp is computed, and may result in 
trying to compare timestamps calculated using different methods.
If the patch is to be released, the computeFeaturesChangeStamp() should 
probably be changed to never look at individual plug-ins directories.

It is a tough call to decide whether this code should be released.  The bug  
looks more like a performance improvements, and there is a chance of 
introducing regression for some scenarios.

With the changes to the patch I describe, I will give it +1 with a disclaimer 
that if we find out strange regressions for some scenarios we should undo the 
change, as there might not be time to react and stabilize before 3.0.

Comment 13 Dorian Birsan 2004-06-16 18:22:22 EDT

I also spoke with Kevin and he gave it a +1.

Konrad, thanks for review.
The first line was not meant to be commented out, it was there for testing 
only, the final patch should not have had it.

I also agree with our comment about not checking the plugin directories 
timestamps at all.
I will release this code later tonight, I will veirfy it again.

Comment 14 Jeff McAffer 2004-06-16 23:03:39 EDT

- Its a bummer that we do not always notice additions on FAT
- deletions are not generally a problem as OSGi will already have detected them 
and alerted update
- A very large number of our downloads are from 98 even though that is not an 
officially supported platform
- don't know the ratio of NTFS to FAT for NT, 2K and XP.
- Always checking every plugin dir is painful.  Especially for large systems or 
network filesystems

Possibilities:
- the plugin dir timestamp maxing could be changed to checksum the timestamps.  
Slightly more expensive but should catch all additions/removals.
- Detect FAT for each plugins dir.  If it is FAT, do the slow (check each 
plugin dir) approach
- If we can't find an efficient way to detect FAT, detect Win98 (there is a 
system property that will give the info) and do the slow thing
- On all non-FAT/Win98 systems, just check the parent dir (e.g., plugins or 
features) to detect additions.

Comment 15 Dorian Birsan 2004-06-17 09:19:16 EDT

We could write some jni code to use the windows api GetVolumeInformation to 
get the name of the file system. This would mean loading one more dll on 
windows, but it'll probably be the fastest.
In update.core we do have a jni dll to load, but this is needed in the 
configurator. If osgi/runtime have native code that we can use, that'll avoid 
loading extra stuff in update.configurator.

Comment 16 Jeff McAffer 2004-06-17 10:45:16 EDT

Using JNI would address the issue but also introduces issues with adding 
another thing to the osgi.bundles list etc etc.  We should avoid that at this 
point.  One possibility is to do it but put all the dlls in the 
update.configurator plugin.  This too feels like a bit too much at this point.

There is no JNI code elsewhere in the runtime or OSGi.

Comment 17 Dorian Birsan 2004-06-17 14:22:12 EDT

After more discussions, here is the latest proposal:

to do now:

- remove the code for checking timestamps on each plugin directory
- ensure that docs or README recommends people who manually unzip plugins to 
restart eclipse with osgi.checkConfiguration set to "true"

The net effect is that this will fix this bug. The runtime will have to ensure 
that it can correctly react to a cached plugin.xml being removed.
Also, the update configurator will start faster in the regular scenarios, as 
the timestamp is only verified on the "plugins" folder.
When running with osgi.checkConfiguration, the update configurator will not 
attempt to run with cached bundles, but instead it will collect all the 
plugins on a site (and exclude the appropriate ones, as directed by 
platform.xml). This will solve the problem of picking up newly unzipped 
plugins. 

Comment 18 Dorian Birsan 2004-06-17 14:24:21 EDT

to do later:

Konrad has found that this code:

File volumes[] = File.listRoots();
for(int v=0; v< volumes.length; v++)
	if(new File(volumes[v], "$MFT").exists())
		System.out.println(volumes[v]+" is NTFS");
	else
	        System.out.println(volumes[v]+" is not NTFS, possibly FAT");
		
	}

can tell us if a volume is ntfs (does not work for mounted drives or CDROM).
We could use this code to do extra work, as per Jeff's suggestion, when 
windows && !ntfs. This extra work could involve keeping track of what plugins 
are known to the config, making change discovery easier.

Comment 19 Jeff McAffer 2004-06-17 14:39:35 EDT

this looks good.  A couple of points re comment 17.

- the runtime does NOT detect the removal of plugin.xml unless 
checkConfiguration=true

- your README statement is conservative.  It is really only FAT people who need 
to set osgi.checkConfiguration=true if they unzip.  On NTFS and other file 
systems, the plugins dir timestamp changes and the new plugins are detected.

Comment 20 Dorian Birsan 2004-06-17 14:59:56 EDT

Jeff, when I test with the fix and renamed the plugin.xml in lucene to 
plugin.xml.1, the startup code had this error:

!SESSION Jun 17, 2004 14:55:10.641 --------------------------------------------
-
eclipse.buildId=unknown
java.version=1.4.2_03
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.651
!MESSAGE Application error
!STACK 1
java.lang.RuntimeException: Application "org.eclipse.ui.ide.workbench" could 
not be found in the registry. The applications available are: 
org.eclipse.ant.core.antRunner, org.eclipse.pde.build.BuildScriptGenerator, 
org.eclipse.pde.build.FetchScriptGenerator, 
org.eclipse.pde.junit.runtime.uitestapplication, 
org.eclipse.pde.junit.runtime.legacyUItestapplication, 
org.eclipse.pde.junit.runtime.coretestapplication, 
org.eclipse.update.core.standaloneUpdate.
	at org.eclipse.core.internal.runtime.PlatformActivator$1.run
(PlatformActivator.java:324)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run
(EclipseStarter.java:272)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run
(EclipseStarter.java:128)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.core.launcher.Main.basicRun(Main.java:185)
	at org.eclipse.core.launcher.Main.run(Main.java:638)
	at org.eclipse.core.launcher.Main.main(Main.java:622)

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.651
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.ant.ui_3.0.0/ [6] 
was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.661
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.compare_3.0.0/ [7] 
was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.671
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.debug.ui_3.0.0/ 
[17] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.671
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.help.base_3.0.0/ 
[19] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.671
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.help.ide_3.0.0/ 
[20] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.681
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.help.ui_3.0.0/ 
[21] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.681
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.help.webapp_3.0.0/ 
[22] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.691
!MESSAGE Bundle 
update@/d:/0616/eclipse/plugins/org.eclipse.jdt.debug.ui_3.0.0/ [25] was not 
resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.691
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.jdt.junit_3.0.0/ 
[30] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.701
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.jdt.ui_3.0.0/ [33] 
was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.701
!MESSAGE Bundle 
update@/d:/0616/eclipse/plugins/org.eclipse.ltk.ui.refactoring_3.0.0/ [38] was 
not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.711
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.pde.ui_3.0.0/ [48] 
was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.711
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.search_3.0.0/ [57] 
was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.711
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.team.cvs.ui_3.0.0/ 
[64] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.721
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.team.ui_3.0.0/ 
[65] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.721
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.ui.editors_3.0.0/ 
[70] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.731
!MESSAGE Bundle 
update@/d:/0616/eclipse/plugins/org.eclipse.ui.externaltools_3.0.0/ [71] was 
not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.731
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.ui.ide_3.0.0/ [73] 
was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.741
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.ui.win32_3.0.0/ 
[77] was not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.741
!MESSAGE Bundle 
update@/d:/0616/eclipse/plugins/org.eclipse.update.scheduler_3.0.0/ [85] was 
not resolved.

!ENTRY org.eclipse.osgi Jun 17, 2004 14:55:10.751
!MESSAGE Bundle update@/d:/0616/eclipse/plugins/org.eclipse.update.ui_3.0.0/ 
[86] was not resolved.

Comment 21 Dorian Birsan 2004-06-17 15:01:44 EDT

Created attachment 12415 [details]
patch, redone

Comment 22 Dorian Birsan 2004-06-17 15:15:20 EDT

I think the error is caused by having to have lucene during startup, see my 
comment #4.

Comment 23 Dorian Birsan 2004-06-17 15:33:10 EDT

fixed and released.