Bugzilla – Full Text Bug Listing
| Summary: | Need to detect OOM exceptions. | ||
|---|---|---|---|
| Product: | [Eclipse Project] Platform | Reporter: | Mike Wilson <Mike_Wilson> |
| Component: | Search | Assignee: | Platform-Search-Inbox <platform-search-inbox> |
| Status: | RESOLVED WORKSFORME | QA Contact: | |
| Severity: | enhancement | ||
| Priority: | P2 | CC: | daniel_megert, n.a.edgar |
| Version: | 3.0 | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
|
Description
Mike Wilson
I did a bit of experimenting. On windows, with a current Java VM, it looks like...
Runtime.getRuntime().freeMemory()
... actually is a reasonably accurate picture of how much memory is free. I
suggest you call this and stop if there is less than, say, 10Meg free. The
number should be large, both because you need to allow the UI enough memory to
display the result, and because seaching is lower priority than keeping the
system alive.
For the UI team, you should probably put in a similar check, with a much lower
number, say 500K free.
Of course, this needs to be tested across all the VMs. It's ok, if it returns an
inaccurate but large number, since this is no worse than we are now. It would be
bad if one of the VMs returned an inaccurate but low number.
Basically once you are near the end of free memory every operation (e.g. CVS sync, compare, save,...) can suffer this problem. Are you suggesting that each individual component should add some protection for this? The memory that will be used depends on the search type (Java, text, ...) which is contributed by (other) plug-ins and hence the basic search component has no control over a concrete search operation. Each search contributor would have to add such a mechanism. I realize that any plugin could cause you to run out of memory. Indeed, that's why I recommended that the UI plugin also catch OOM exceptions. The point is, search is the only one that has done it to me, and as a user I would find it perfectly acceptable to have a search plugin that said "Search aborted because of low memory. Partial result returned". If the search algorithms can use unbounded amounts of memory, then they have to deal with the case. I don't see why it's any harder for search plug-ins to watch for out of memory errors than it is for them to implement any other part of the "searching contract". Just let them know they need to call some API you provide that manages the query. >I don't see why it's any harder for search plug-ins to watch for out of memory
>errors than it is for
I did not state that it is hard/harder for them, did I ;-)? I only wanted to
point out that others will be affected as well.
Other examples where you can run into the same problem (unbound memory
consumption) are:
- building large hierarchies
- comparing files with lots of changes
I agree that those areas are also interesting, and need to be investigated. I just find it frustrating that we (all of Eclipse) are still handling low memory conditions the way we were before R1.0 shipped (i.e just go until we run out, then crash (and if you're lucky generate a log message)). To me, causing Eclipse to exit because some code we wrote used up all available memory is no different than causing it to exit because some code we wrote generated any other run time exception: It's a bug, and we should fix it. I've annotated UI's bug 33972 with a pointer to this one and marked it for M5. One problem with a catch-all at the workbench level is that we have no way of knowing how big a chunk another plugin will allocate to trigger an OutOfMemoryError. A comfort zone of ~500K is probably reasonable though. If a plugin is regularly allocating bigger chunks than that, it should be doing its own checking. It's clear that pathalogical situations could still cause OOM, but even handling the common ones would definately an improvement. I've asked to the Core and VCM teams to think about their memory usage patterns, to see if there are places where they need to be checking. This would actually have to be handled by the individual searches. Matches are allocated by clients (i.e Java search). In order to hande OutOfMemory, they would have to ensure that they're cleaning up cleanly. We need to plan this. No plans here. It seems to me that the default OOM handling of the workbench is good enough also for search. Not sure if it is a good idea if plugins intruduce their own handling. Setting to remind. Closing as WORKSFORME as we did not get OOME bugs against search for a long time now. |