Community
Participate
Working Groups
Build Identifier: 3.4.2.R34x_v20080826-1230 The usage of java.io.File.DeleteOnExit() API in the framework (StorageManager, BaseStorage, so forth) creates a leak in java.io.DeleteOnExitHook as described in this post: http://www.jroller.com/javabean/entry/solving_an_outofmemoryerror_java_6 We are using OSGi for a long running server that does not expect to be shutdown for a very long time. And we are installing, uninstalling, and updating bundles as time progresses frequently (testing with 600 such operations in an hour). We are seeing a growth in java.io.DeleteOnExitHook using eclipse memory analyzer. Please consider changing the OSGi framework implementation to not rely upon deleteOnExit. Otherwise, we have to clean up the memory space using reflection. As is, using OSGi framework will run out of memory in our case really quickly. Thanks. Reproducible: Always Steps to Reproduce: 1. Start OSGi 2. Create an environment where a bundle is getting updated very frequently. 3. Observe a growth in memory in java.io.DeleteOnExitHook. Every call to deleteOnExit adds an entry to the LinkedHashSet inside the Hook class.
We can work around this by changing our usage of StorageManager for saving the .bundledata files. Can you confirm that the large number of files being marked for delete on exit are for the .bundledata files?
(In reply to comment #1) > We can work around this by changing our usage of StorageManager for saving the > .bundledata files. Can you confirm that the large number of files being marked > for delete on exit are for the .bundledata files? I have seen entries in heap dump that has many references to files in config area: <CONFIG>/org.eclipse.osgi/.bundledata1431170944290931637.tmp This is seen in java.io.DeleteOnExitHook object in the memory tree. The acutal .tmp file may be gone, but the entry remains since there was a call to File.deleteOnExit() for that temp file.
(In reply to comment #2) > (In reply to comment #1) > > We can work around this by changing our usage of StorageManager for saving the > > .bundledata files. Can you confirm that the large number of files being marked > > for delete on exit are for the .bundledata files? > > I have seen entries in heap dump that has many references to files in config > area: <CONFIG>/org.eclipse.osgi/.bundledata1431170944290931637.tmp > > This is seen in java.io.DeleteOnExitHook object in the memory tree. > > The acutal .tmp file may be gone, but the entry remains since there was a call > to File.deleteOnExit() for that temp file. Thanks, I understood that the leak happens even when we have actually deleted the file ourselves before the VM exists. This is because the DeleteOnExitHook is holding onto some extra data even when you have deleted the file yourself. Thanks for confirming the file name that is getting pinned in memory.
You could try setting the configuration option osgi.useReliableFiles=true to work around this issue. There is an issue with using this if your cache files are too big. I opened bug350453 to fix that. You may also need to set osgi.reliableFile.maxInputStreamBuffer to be a really big integer to handle .bundledata files larger than 128k
(In reply to comment #4) > You could try setting the configuration option osgi.useReliableFiles=true to > work around this issue. There is an issue with using this if your cache files > are too big. I opened bug350453 to fix that. You may also need to set > osgi.reliableFile.maxInputStreamBuffer to be a really big integer to handle > .bundledata files larger than 128k Looking at the fix for bug350453, it looks like ReliableFile has its own static cache: private static Hashtable cacheFiles = new Hashtable(20); Can you let me know if this cacheFiles will get be pruned regularly and not have the growth problem until JVM shutdown like java.io.DeleteOnExitHook? I am trying to backport this bug into the 3.4.2.R34x_v20080826-1230 release label.
(In reply to comment #5) > > Can you let me know if this cacheFiles will get be pruned regularly and not > have the growth problem until JVM shutdown like java.io.DeleteOnExitHook? I recommend you double check at runtime. But the cacheFiles should get pruned up when we close a ReliableFile after writing to it. BTW, another work around is to configure the framework to save its state at much larger intervals. Right now we default to saving the framework state after 30 seconds (30000 ms) of inactivity. You could bump that up to something much larger like 24 hours (or 86400000 ms). This would leak at most one File to the DeleteOnExit class. But if you install/update/uninstall at least one bundle a day then no file would get leaked because we would not save the state of the framework to disk until the framework was shutdown.
Sorry, I forgot to actually tell you how to set the save interval. You can use the following configuration property: eclipse.stateSaveDelayInterval=86400000
I removed the problematic call to deleteOnExit in commit: http://git.eclipse.org/c/equinox/rt.equinox.framework.git/commit/?id=25adc962ef117d879d9ca5713a8f64af2af4834a After some more review I decided it was best to simply do away with the call to deleteOnExit. In our usage of StorageManager output streams we always try to close the files in finaly blocks which will ultimately rename the file or delete it. So all we are really doing is allowing the VM to delete the file on exit if exit happens right in the middle of writing. This seems so rare and not much to gain by calling the deleteOnExit. So I decided to simply remove it.
