Community
Participate
Working Groups
With Code Recommenders being enabled by default and in more packages than before, we should try hard to avoid redundant model and index downloads. Thus, the local model repository should be shared by multiple Eclipse installs, probably being kept below ${user.home}/.eclipse somewhere. We need to make sure that multiple Eclipses running simultaneously do not corrupt the repository, though. Maybe we can see how m2e handles this situation when accessing ${user.home}/.m2/repository.
Adding webmasters to bring this proposed change for Mars to their attention.
(In reply to Andreas Sewe from comment #0) > We need to make sure that multiple Eclipses running simultaneously do not > corrupt the repository, though. Maybe we can see how m2e handles this > situation when accessing ${user.home}/.m2/repository. For the record, this is the implementation used by m2e: [1]. [1] <https://github.com/takari/takari-local-repository>
I am not sure what this means for us?
Short: Less traffic. Recommenders will be enabled by default in Java, JEE, RCP, and Committer package in Mars. This means that every user will download the Java Runtime Models + Model Search Index once per workspace. This makes roughly 5MB per user and per workspace. Since users create new workspaces from time to time, Recommenders would download the Java model and index again. According to the eclipse.org download stats of today we have ~80k model downloads per week. 80K x 5MB x 4 Weeks makes round about 1600 GB traffic per month. We hope to get rid of these repeated model downloads by changing the storage to a user-home based storage (instead of a workspace-based storage). Hope that explains what we do and why we think this is necessary for eclipse.org - and why you should know. FWIW, there is yet no code in place that can make use of mirrors. Thus, this traffic is served by eclipse.org servers at the moment (since you did not notice yet, it seems we are not causing too much traffic ;-) - but since you are discussing the size of content.xml/jar I think we should try to minimize our share as well...
> Short: Less traffic. Thank you.
I have investigated possible implementations, in particular using a SyncContext/SyncContextFactory similar to that used by [1]. It seems, however, that this complexity is unnecessary for our use case: org.eclipse.aether.connector.basic.PartialFile supports concurrent downloads by multiple processes out of the box, and within the same Eclipse process ModelRepository is already thread-safe. PartialFile creates a .lock file when the download starts (some time between TransferListener.transferInitiated and TransferListener.transferStarted). Other processes downloading the same file block and go straight to TransferListener.transferSucceeded when the downloading process is done. I tested this on Windows 7 and Mac OS X so far. FYI: I asked on aether-users about this [2]; as of this writing, the discussion is still ongoing. [1] <https://github.com/takari/takari-local-repository/blob/master/src/main/java/io/takari/aether/concurrency/LockingSyncContext.java> [2] <https://dev.eclipse.org/mhonarc/lists/aether-users/msg00572.html>
Change is merged. Please create new bugs for issues you may find.
