Community
Participate
Working Groups
I have a java web project that contains 80,000 images in a non-source directory. Eclipse uses huge amounts of memory to track these and refreshes take ages. I would like to be able to exclude the images directory from the project path.
The recommended way of dealing with this is to create your project in a different location on disk, and then "link" the source folders from your large project tree into it. This is done from the Java Build Path property pgae, on the "Source" tab. Click "Add Folder", then "Create New Folder, then "Advanced", then choose the file system location for that source folder. In other words, we offer a selective inclusion model rather than selective exclusion. I realize this is sometimes less convenient, but it is a design trade-off we made. See this link for a more detailed discussion: http://www.eclipse.org/eclipse/development/inflexible-projects-problem.html
Please reopen and explain if this approach doesn't work for you.
The "large" folder is at src/www/images/business/. Using the recommended approach I would also have to link each individual file at every level below this directory. For example, I would need to link each of the html files in src/www/ (of which there are many) to a NEW parallel directory src/www. I could not link this directory itself in because that would include the directory that I am trying to exclude. Once I have created this new parallel folder, src/www, every new file that I created would go into this folder and not with the other html files in the original directory. So although it may be possible it is extremely difficult and is still not correct.
Thanks for the explanation. When you said "non-source directory" I thought the images were stored in a separate directory hierarchy from your source code. I assume you have a Java project, and "src" is the name of the source folder. If you haven't already done so, you should try setting an exclusion pattern on that source directory. Again from the "Source" tab of the Java Build Path property page, expand the "src" folder, select "Excluded", and click "Edit". From there you can add an exclusion pattern for sub-directories that should not be considered by the builder. I believe in your case this would be "www/images/business/**". This won't completely remove the directory from Eclipse, but will cause the Java builder to ignore that directory (it won't copy the images from there to the project output directory).
I have already done as you suggested but the problem is due to the large number of files consuming memory (and cpu when refreshing). Eclipse is using about 800MB RAM just to start up. The solution would be to allow resources to be filtered (excluded) at a lower level than JDT source (ie resource level).
I understand the request. I frequently do performing testing on a workspace with 75,000 resources, which doesn't seem to require nearly that much memory. Maybe I need to start using a bigger workspace ;) What's the total number of resources in your workspace? Do you have other plugins installed, or is it just a plain Eclipse install? Are you using CVS, or any other source control plugins? 32 or 64 bit Linux? Sorry for the barrage of questions, but support for "large-scale" workspaces is an important part of our 3.1 plan, so I'm eager to find out where that memory is going and where our scalability problems lie.
I am running on Linux 32 bit. I have 10 fairly large projects in the workspace (including Hibernate, Jetty, Lucene, Webwork ...) I have the WebTools plugin and the Subclipse Plugin installed and I use the CVS plugin for most of the third party libraries. Running "ps aux" shows that eclipse is using 1.13 GB of virtual memory. One of my projects contains 70,000 images. It was also be symlinked to another directory under the project root which may have compounded the problem. But I removed the symlink in case this was causing the problem. Eclipse crashes on startup with an OutOfMemoryError unless I use -Xmx600M.
Bigfoot indeed. Thanks for the details!
Adding my name to the cc list as we are now tracking performance issues more closely. Please remove the performance keyword if this is not a performance bug.
*** Bug 86787 has been marked as a duplicate of this bug. ***
*** Bug 109377 has been marked as a duplicate of this bug. ***
*** Bug 111616 has been marked as a duplicate of this bug. ***
Comments from bug 111616: This is proposal for adding filtering support at the project root, much like source folders do today, that would allow excluding resources/folders, and enabling multiple projects to be mapped to a similar physical root folder on disk. We would expect the project/resource api to exclude the specified content, and client would not be required to have knowledge of the filtered resources. Some areas of concern would be: 1) scheduling rules on these "shared" roots. 2) team support of these filtered projects 3) team support for groupings of projects that share a common physical root.
I haven't seen any additional activity on this request, so I'll add some additional details to the scenarios I hope can be solved by filtering at the project level. Given an example J2EE folder layout: sampleEAR/ EJB1/ META-INF/ ejb-jar.xml WAR1/ javasrc/ web-content/ WEB-INF/ web.xml META-INF/ Application.xml util.jar 1) Mapping "sampleEAR" to an EAR project (simple project). This directory has a mixture of archived content (util.jar), and nested module directories, and a metadata directory "META-INF" for the deployment descriptor. In addition to the existing file layout, additional eclipse/other resource files could be included in the project root(.project etc...) If project filters are introduced, an exclusion rule could be created to ignore the eclipse metadata files, along with the nested module directories. 2) The nested module directories mapped to "sibling" eclipse projects With project filtering, the "publishable" content of each j2ee module could be represented
*** Bug 122002 has been marked as a duplicate of this bug. ***
There is now technology in place to support exclusion filters in the workspace. You can create an exclusion filter in I20060110 by creating a linked resource to the "null" file system. The linked resource will hide any file system resource with the same name, thus acting as an exclusion mechanism. However, I am not currently convinced this is something that should be exposed to end-users for Eclipse 3.2 because I believe the implications are not fully understood. Repository plugins generally operate directly in the file system, and thus will typically not be able to interpret these filters. This is probably ok for John P's use case (comment #0 and comment #3), but Chuck's use case with overlapping projects is more complicated (comment #14). If projects are overlapping in the file system, there are different options: 1) each project is version-managed separately 2) parent project's version management takes care of the content in the sub-projects. In the case of 2) the repository tools should ignore the exclusion filters and manage the entire top-level project. In the case of 1), the repository tools must respect the exclusion filters or they would compete with each other and both attempt to manage the overlapping region. Since there are a large number of existing repository tools, some of which operate on IResource and others operate directly on the file system, approach 1) is going to break some of them. I'm currently leaning towards ensuring the technology is in place for 3.2, but not exposing it as an end-user feature until we understand what to do with VCM tools.
*** Bug 93073 has been marked as a duplicate of this bug. ***
*** Bug 144957 has been marked as a duplicate of this bug. ***
*** Bug 216794 has been marked as a duplicate of this bug. ***
Although this bug is much older, I'm going to mark this as a duplicate of bug 252996 because that bug has a working patch. Those still interested can track that bug instead. *** This bug has been marked as a duplicate of bug 252996 ***
