Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 376322 - StateWriter results in error on large attributes (greater than 64k)
Summary: StateWriter results in error on large attributes (greater than 64k)
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: Framework (show other bugs)
Version: 3.8.0 Juno   Edit
Hardware: All All
: P3 critical (vote)
Target Milestone: Juno M7   Edit
Assignee: Thomas Watson CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-04-09 10:06 EDT by Raymond Auge CLA
Modified: 2012-04-10 10:08 EDT (History)
2 users (show)

See Also:


Attachments
An extremely long list of exported packages (70.45 KB, text/plain)
2012-04-09 10:08 EDT, Raymond Auge CLA
no flags Details
Patch (2.60 KB, patch)
2012-04-09 10:12 EDT, Raymond Auge CLA
tjwatson: iplog+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Raymond Auge CLA 2012-04-09 10:06:35 EDT
Build Identifier: I20120405-1735

When the framework attempts to persist it's state, if any of the Framework attributes or bundle attributes are larger than 64k, persistence fails with the following error:

java.io.UTFDataFormatException: encoded string too long: 72142 bytes
	at java.io.DataOutputStream.writeUTF(DataOutputStream.java:364)
	at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323)
	at org.eclipse.osgi.internal.resolver.StateWriter.writeStringOrNull(StateWriter.java:686)
	at org.eclipse.osgi.internal.resolver.StateWriter.writePlatformProp(StateWriter.java:181)
	at org.eclipse.osgi.internal.resolver.StateWriter.saveState(StateWriter.java:130)
	at org.eclipse.osgi.internal.resolver.StateObjectFactoryImpl.writeState(StateObjectFactoryImpl.java:439)
	at org.eclipse.osgi.internal.baseadaptor.StateManager.writeState(StateManager.java:178)
	at org.eclipse.osgi.internal.baseadaptor.StateManager.update(StateManager.java:116)
	at org.eclipse.osgi.internal.baseadaptor.BaseStorage.saveStateData(BaseStorage.java:661)
	at org.eclipse.osgi.internal.baseadaptor.BaseStorage.saveAllData(BaseStorage.java:458)
	at org.eclipse.osgi.internal.baseadaptor.BaseStorage$StateSaver.run(BaseStorage.java:1304)
	at java.lang.Thread.run(Thread.java:679)

Reproducible: Always

Steps to Reproduce:
1.Set the OSGi Framework property org.osgi.framework.system.packages.extra to a value larger than 64k [attached file contents]

2. Start the framework
Comment 1 Raymond Auge CLA 2012-04-09 10:08:23 EDT
Created attachment 213748 [details]
An extremely long list of exported packages
Comment 2 Raymond Auge CLA 2012-04-09 10:12:20 EDT
Created attachment 213749 [details]
Patch

This is a patch that keeps backward compatibility for existing persisted values, and conserves the old behavior for short (less than 64k) values.
Comment 3 Thomas Watson CLA 2012-04-09 10:21:34 EDT
Thanks for the bug.  And thanks for the fix!  I took a quick look at the patch and it seems like good possible solution.  I need to update the state cache version (org.eclipse.osgi.internal.resolver.StateReader.STATE_CACHE_VERSION).
Comment 4 Raymond Auge CLA 2012-04-09 10:29:38 EDT
Thank you.

Regarding the cache version, I wasn't sure what to do about that. Thanks for pointing it out.

Also, I wasn't entirely confident that simply going to the next byte value as the type for LOBJECT was actually completely safe. But I couldn't identify anything that would obviate it being wrong either.

My first invocation of the fix was simply to write everything using the new pattern (since there is not really any additional cost). I wasn't actually sure if there was actually any compatibility maintained for state across Equinox versions.
Comment 5 Thomas Watson CLA 2012-04-10 09:54:44 EDT
(In reply to comment #4)
> Thank you.
> 
> Regarding the cache version, I wasn't sure what to do about that. Thanks for
> pointing it out.
> 
> Also, I wasn't entirely confident that simply going to the next byte value as
> the type for LOBJECT was actually completely safe. But I couldn't identify
> anything that would obviate it being wrong either.

I think this is fine for this purpose.  I renamed the constant to LONG_STRING.

> 
> My first invocation of the fix was simply to write everything using the new
> pattern (since there is not really any additional cost). I wasn't actually sure
> if there was actually any compatibility maintained for state across Equinox
> versions.

The new reader needs to continue to support the old format, otherwise an updated framework will not be able to read the cache files.  We could consider writing everything in the new format, but since we must support reading the old format it is not much work to continue writing "short" strings in the old format and it seems less risky to do so at this point.

I released the fix and test with this commit:

http://git.eclipse.org/c/equinox/rt.equinox.framework.git/commit/?id=d5060d2d1b368f06248e8440e3ef575b6c5da082

Notice that I used your extremely long list of exports in the testcase.
Comment 6 Thomas Watson CLA 2012-04-10 10:08:35 EDT
Comment on attachment 213749 [details]
Patch

Thanks for the contribution.