Community
Participate
Working Groups
This has probably been reported elsewhere, but the long directory paths blow past the Windows 256 character limit for files on a drive system, which results in an inability to delete folders from Explorer once created (the error message comes across as 'Cannot delete <dir>: folder is not empty'). This, in turn, comes because the folder contains a folder which goes past the end of the file system limit. This seems to have gotten worse since Virgo 2.1, hence raising it as a bug report. Can we condense the folders somehow? It doesn't help that the downloadable itself chews through a number of characters (virgo-web-server-3.0.0.M01) before you even get to the other parts. C:\virgo-web-server-3.0.0.M01\work\org.eclipse.virgo.kernel.deployer_3.0.0.M01\staging\com.example.app.par-1.0.0.0-SNAPSHOT\bundle\com.example.app.par-1.0.0.0-SNAPSHOT-com.example.bundle\1.0.0.0-SNAPSHOT\com.example.bundle-1.0.0.0-SNAPSHOT.jar\... By the time you get to this depth, Windows is already saying 'path name is too long' - and that's without the fact that the .jar directory appears to contain an uncompressed selection of the JAR's contents. You are then stymmied into not being able to do anything with this path - even delete it - and the only 'fix' is to start randomly renaming parent directories (and/or moving them) in order to shrink the path namespace down, so that you can delete it. Some observations: * You're repeating data here. ** The PAR name is repeated both in the directoryy 'staging' as well as the prefix of the 'bundle' subdirectory. It doesn't need both. ** The JAR name is repeated, both as a suffix of the 'com.example.app.par-1.0.0-SNAPSHOT-com.example.bundle\' directory name, and in the directory name two levels down ** The JAR version is repeated, both in the directory name and also as the suffix of the bundle direcotry itself * The kernel_deployer takes up a significant space; o.e.v.kernel.deployer_3.0.0M1 would likely be as unique and several parts shorter * Prefixes in the customer bundle can almost certainly be pruned. At the time of PAR deployment, you know the full set of bundles in any case. You could compress the first two parts (c.e.app.par) for example. * You could compress the directories further, using numbers (1,2,3) and then have a 'map' in each folder layer (com.example.app.par-1.0.0.0-SNAPSHOT=1). Then you wouldn't need to do any renaming as it would just be \staging\1\bundle\1\1, with \staging\map=="1=com.example.app.par-1.0.0.0-SNAPSHOT", \staging\1\bundle\1\map=="1=com.example.applpar..." etc. With the map in a cache, lookup to translate to directory names would likely be just as efficient as a direct path lookup is at the moment. Although no-one is seriously going to install Virgo (or any other production service...) on a Windows box, it is still (sadly) the choice of development platform for a number of corporate environments, and the inability to test/cleanup on these platforms is going to hurt Virgo's adoption.
Thanks for raising this old chestnut again. Clearly this is a significant issue for Windows users, although I am surprised that it has gotten worse since Virgo 2.1, especially since "2.1.0.RELEASE" is longer than "3.0.0.M01" and I am not aware of any structural changes in the deployer staging area. Some simple changes like you suggest will shorten the path, but the duplication you observe will probably push us up against the limit anyway, so a more radical overhaul seems to be necessary: something we have so far shied away from because of the complexity. For now, I'll simply make some comments. One of the early drivers for the layout was that the only way we could do bundle manifest transformation was to change the manifest on disk before installing into Equinox, but these days we do manifest transformations in memory, so there is less need to copy bundles around the file system. The other main reason for expanding bundles on disk was Tomcat resource loading and this may require more effort to avoid. Note that there is currently no limit in Virgo on the length of (a) filenames, (b) bundle symbolic names, (c) bundle versions, (d) application symbolic names, (e) application versions, all of which feature in the example you quote! Any one of these could at the moment legitimately be longer than 255 characters.
If you want to run Virgo on Windows I recommend Cygwin :) I use it every day to build and run Virgo (as well as for GIT and Ruby). I know this cannot be a serious solution, but it seems as the only way to delete paths longer than 256 characters without shortening them. The problem with Cygwin approach is that Java would still face the Windows limitations.
I have been looking at a pragmatic solution given that a full solution would be hard to implement and pretty disruptive. I unpacked Virgo Tomcat Server 3.0.0.M03 into a directory with a very long name (>100 chars). After starting and kill Virgo, it was then not possible to delete the install directory using the normal "rmdir /s /q" command - lots of messages about paths that cannot be found and directories not being empty. However the following workaround was successful. C:\>subst x: averylongdirectorynameindeedafgagfadadfasdfasdfadfadsfasdfadfasdfas dfasdfadsfasdfadsfadfadsfasdfadsfadsf C:\>x: X:\>rmdir virgo-tomcat-server-3.0.0.M03 /s /q X:\>c: C:\>subst x: /d C:\>dir averylongdirectorynameindeedafgagfadadfasdfasdfadfadsfasdfadfasdfasdfasd fadsfasdfadsfadfadsfasdfadsfadsf Volume in drive C has no label. Volume Serial Number is F8EF-AC9B Directory of C:\averylongdirectorynameindeedafgagfadadfasdfasdfadfadsfasdfadfas dfasdfasdfadsfasdfadsfadfadsfasdfadsfadsf 04/21/2011 12:10 PM <DIR> . 04/21/2011 12:10 PM <DIR> .. 0 File(s) 0 bytes 2 Dir(s) 6,076,203,008 bytes free C:\> Is this a reasonable workaround? Could we address this bug by wrapping this up in a batch file and documenting it in the user guide?
There are two other reasons for taking a pragmatic approach. I think producing cryptic, shortened directory names will not be helpful to people who hit certain kinds of problems and need to delve into the directory structure to work out what's going on. Also, since most production environments are UNIX-based, it would be a shame to allow a Windows restriction to dictate the directory layout there.
Oh a mistake earlier. The long directory name wasn't actually >100 chars (I wrongly assumed the command prompt was 80 chars wide), but it was unreasonably long IMO. I am happy to repeat the experiment if necessary with more extreme directory name length.
It's normal for a number of drive letters to be mapped to network drives. How do you propose deciding which one to pick to use for the temporary mapping? I've tested and for both subst on subst and subst on network so don't see a problem there. If java can handle the deletion (probably because it is using newer APIs) then can Java not be used to clean the install? Just a thought. On the other hand the existence of these names causes other problems eg backup software may not be able to read it and the like. The problem isn't just about deleting. You can't use Explorer to go into that folder (and thus, read any of the contents). Focusing on a single symptom doesn't affect the existence of other symptoms of the same underlying problem.
I would probably have the drive letter passed in as a parameter to the script. Yes, it could well be possible to code a utility in Java to avoid that step, if that would provide a satisfactory solution. However, you are right that this is really working around one symptom of the problem. But isn't that all we can really do in Virgo since the problem is in the OS and we would have to publish and enforce some pretty draconian restrictions on various names (see comment 1) to be certain of users not hitting the problem? Not sure where to go with this one. If a workaround isn't acceptable and we can't resource the necessary reengineering any time soon, then what options do we have? What's your feeling Alex? What would best help you as a user?
(In reply to comment #7) > I would probably have the drive letter passed in as a parameter to the script. > Yes, it could well be possible to code a utility in Java to avoid that step, if > that would provide a satisfactory solution. Clearly Java/Virgo is able to go past these so maybe it would work. > However, you are right that this is really working around one symptom of the > problem. But isn't that all we can really do in Virgo since the problem is in > the OS and we would have to publish and enforce some pretty draconian > restrictions on various names (see comment 1) to be certain of users not > hitting the problem? The problem seems prevalent in the staging directories which Virgo has control over naming. Whether the app or bundle name is long is irrelevant, since Virgo does not have to use them in the path name. It is entirely within Virgo's power to not use the app/bundle names as directory names. > Not sure where to go with this one. If a workaround isn't acceptable and we > can't resource the necessary reengineering any time soon, then what options do > we have? What's your feeling Alex? What would best help you as a user? I'd say have a map file of name-to-ids and then indirect via that to get to the contents. Either that or delete the staging directory after use - though if it's still needed by tomcat this isn't an option.
We have discussed this in the team and concluded that we cannot avoid long path names in the work directory without impacting (a) resource loading in web apps, and (b) self support. The cleanest solution is to add a -nostart option to the startup scripts. When used in combination with the -clean option, this will remove the work and serviceability directories without starting Virgo. This is a useful function on any platform when a user or administrator needs to clean out the current state but doesn't want to start Virgo just yet. In terms of the current bug, "-clean -nostart" can be used to get back to approximately the same set of directories that Virgo was installed with, which the user will then be able to delete using normal techniques (e.g. rmdir /s /q). This approach is safer than providing some kind of "delete Virgo installation" script. Users can easily write such a script (to drive "-clean -nostart") if they need one. As part of the fix, I will describe the new option in the User Guide and add a "known issue" to draw people's attention to the solution when they have trouble deleting under Windows.
Added the -noStart option to dmk.sh/dmk.bat.
Added "Cleaning Virgo Web Server without Starting it" section to the User Guide and a known issue "Problem Deleting Installation Directory under Windows" with cross-references between these two.
Spamming Eclipse.org is a waste of time
