Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 350143

Summary: Do not mess svn:externals order
Product: [Technology] Subversive Reporter: Julien HENRY <julien.henry>
Component: CoreAssignee: Igor Burilo <igor.burilo>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: P3 CC: a.gurov
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: All   
Whiteboard:
Attachments:
Description Flags
Alphabetic ordering, folders first
none
Wrong order for svn:externals
none
svn:externals are no more displayed after last update (0.7.9.I20110715-1700)
none
Crash dump
none
Correct order on google code
none
Wrong order on my company repo
none
Patch that fix the issue i.vinnykov: iplog+

Description Julien HENRY CLA 2011-06-23 10:03:06 EDT
Build Identifier: 20110609-2236

We are using svn:externals to link other folders. We tried to sort our svn:externals in alphabetical order to help users to find what they are looking for.
The problem is that subversive is not honoring the order of svn:externals definitions.
I think the fix is very easy. Just replace HashMap by LinkedHashMap in SVNUtility.java (line 173):

public static Map<String, SVNEntryRevisionReference> parseSVNExternalsProperty(String property, IRepositoryResource propertyHolder) {
 	Map<String, SVNEntryRevisionReference> retVal = new HashMap<String, SVNEntryRevisionReference>();

Thanks

Reproducible: Always
Comment 1 Alexander Gurov CLA 2011-07-10 03:16:03 EDT
Created attachment 199381 [details]
Alphabetic ordering, folders first

As you can see (see attachment) all svn:externals inclusions are correctly ordered without any relation to random initial order or internal data structures like HashMap etc. So, I don't see any issue here.
If I've incorrectly understood your request, please explain it a little more.
Comment 2 Julien HENRY CLA 2011-07-12 03:53:43 EDT
Created attachment 199471 [details]
Wrong order for svn:externals

Is it something that was fixed recently? As you can see on my screenshot, the order is not good. I don't see any major difference between your case and mine. Except the fact that the labels of your svn:externals are surrounded by double quotes. Could it be the reason?
Comment 3 Alexander Gurov CLA 2011-07-12 11:19:10 EDT
There were no changes for a long time in the code and when I made this screenshot of mine and posted my answer here there still weren't any changes. But after that I have found that there were two slightly different reodering codes working over the nodes, so I have refactored it and removed useless one (bug #351649). Now I wonder: is it possible that those two reordering codes conflicted with each other in your case?
Could you please specify which plug-in version you are using now that is causing this kind of issue?
Comment 4 Julien HENRY CLA 2011-07-12 11:40:44 EDT
Here is my version:

Subversive SVN Team Provider (Incubation)	0.7.9.I20110602-1700

Thanks for your time.
Comment 5 Alexander Gurov CLA 2011-07-12 11:49:55 EDT
It seems the version is pretty recent. So, it should be the same as of time when I took my screenshot. On this week I will publish new Subversive build which will include the fix for bug #351649. Then, if it is not too hard, please update your Subversive version and tell me here if the problem is gone or not. If the problem will still persists even after update then we will do further investigation on the case, because as for now I can't see where the flaw can be except for the fixed code parts. Thank you in advance.
Comment 6 Julien HENRY CLA 2011-07-12 11:58:47 EDT
No worries, I'll wait for the new version and be happy to test. Please ping me on this issue when it is released.
Comment 7 Alexander Gurov CLA 2011-07-15 19:34:38 EDT
(In reply to comment #6)

Hello,

As promised I've published new Subversive development build to the "Early Access" site. More information about the build itself is placed at the page http://www.eclipse.org/subversive/downloads.php

There are updates to the main plug-in and to the connectors as well (although connectors got minor changes only).

I hope this build will work well with svn:externals in your case. Also it has a fix to the really annoying issue with "Edit Conflicts" action under Eclipse 3.7.
Comment 8 Julien HENRY CLA 2011-07-18 03:30:15 EDT
Created attachment 199808 [details]
svn:externals are no more displayed after last update (0.7.9.I20110715-1700)

After updating plugin and restarting Eclipse, svn:externals folders are not displayed anymore! I'll try a bit more investigation, but do you have an idea of the reason of the regression? I have checked in Eclipse SVN settings and it seems everything concerning svn:externals is enabled.

Subversive SVN Team Provider (Incubation)	0.7.9.I20110715-1700
Comment 9 Julien HENRY CLA 2011-07-18 03:34:28 EDT
We are using post 1.5 svn:externals format (see http://svnbook.red-bean.com/en/1.5/svn.advanced.externals.html). Could it be the reason of the difference between your configuration and mine?
Comment 10 Julien HENRY CLA 2011-07-18 03:35:24 EDT
(In reply to comment #9)
> We are using post 1.5 svn:externals format (see
> http://svnbook.red-bean.com/en/1.5/svn.advanced.externals.html). Could it be
> the reason of the difference between your configuration and mine?

Sorry, I intended to say PRE 1.5 format.
Comment 11 Alexander Gurov CLA 2011-07-18 07:26:22 EDT
(In reply to comment #9)
> We are using post 1.5 svn:externals format (see
> http://svnbook.red-bean.com/en/1.5/svn.advanced.externals.html). Could it be
> the reason of the difference between your configuration and mine?

I have checked with both formats and it worked for me just fine. I'll think of the way to find the issue reasons (although I haven't touched any code that could svn:externals to be visible or not, only reordering code were modified).
Comment 12 Julien HENRY CLA 2011-07-18 07:50:42 EDT
There is an error message in the log when I open a folder containing svn:externals:

Message: L''opération de récupération des Sous-dossiers du Dossier du Référentiel a échoué.
(fr2en => Operation to retrieve sub-folders of repository folder failed)

java.lang.UnsupportedOperationException
	at java.util.AbstractList.add(AbstractList.java:131)
	at java.util.AbstractList.add(AbstractList.java:91)
	at org.eclipse.team.svn.core.operation.remote.GetRemoteFolderChildrenOperation.runImpl(GetRemoteFolderChildrenOperation.java:99)
	at org.eclipse.team.svn.core.operation.AbstractActionOperation.run(AbstractActionOperation.java:81)
	at org.eclipse.team.svn.core.utility.ProgressMonitorUtility.doTask(ProgressMonitorUtility.java:104)
	at org.eclipse.team.svn.core.operation.CompositeOperation.runImpl(CompositeOperation.java:95)
	at org.eclipse.team.svn.core.operation.AbstractActionOperation.run(AbstractActionOperation.java:81)
	at org.eclipse.team.svn.core.operation.LoggedOperation.run(LoggedOperation.java:39)
	at org.eclipse.team.svn.core.utility.ProgressMonitorUtility.doTask(ProgressMonitorUtility.java:104)
	at org.eclipse.team.svn.core.utility.ProgressMonitorUtility.doTaskExternal(ProgressMonitorUtility.java:90)
	at org.eclipse.team.svn.ui.utility.DefaultCancellableOperationWrapper.run(DefaultCancellableOperationWrapper.java:55)
	at org.eclipse.team.svn.ui.utility.ScheduledOperationWrapper.run(ScheduledOperationWrapper.java:37)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Comment 13 Julien HENRY CLA 2011-07-18 08:05:51 EDT
In GetRemoteFolderChildrenOperation.java line 88, may I suggest to replace:

List<IRepositoryResource> newTmp = Arrays.asList(tmp);

by

List<IRepositoryResource> newTmp = new ArrayList<IRepositoryResource>(tmp.length);
newTmp.addAll(Arrays.asList(tmp));


Indeed Arrays.asList(tmp) seems to be immutable.
Comment 14 Alexander Gurov CLA 2011-07-24 09:17:01 EDT
Hello Julien, 

Thank you for the stack trace and your thoughts. I've published a development build with the fix to this one issue.
Comment 15 Julien HENRY CLA 2011-07-25 06:19:01 EDT
I did the update and now it is even worse... Eclipse crash as soon as I try to expand a SVN folder containing svn:externals. There is nothing in .metadata/.log so I don't know what to suggest. Tell me if you need any information.
Comment 16 Julien HENRY CLA 2011-07-25 07:39:20 EDT
Created attachment 200269 [details]
Crash dump

Running eclipse from the command line I get the attached crash dump.
Comment 17 Julien HENRY CLA 2011-07-25 09:44:36 EDT
More investigation. The crash is not always reproducible. It always occur for a specific folder in our SVN, but not in other places where we are using svn:externals.

But I can say the order issue is NOT resolved. I still have a very strange order for externals sub folders.
For example:
  - prevoyance
  - homebanking
  - bureaux
  - socram

This is not a lexical order, nor the order declared in svn:external. It really look like the "random" order of a hashmap.

I will try to set up a public SVN repo to reproduce the issue. I think you are testing with "local" repo (file://) and maybe this is the reason of the difference.
Comment 18 Julien HENRY CLA 2011-07-25 10:28:06 EDT
I did additional testing and I am very confused. It seems the issue depends on which svn repo the svn:externals are created. Maybe because in my company we are using SVN server version 1.4?
See the two attached screenshots. I have created the same svn:externals configuration on:
  - a google code svn repo
  - my company repo
When I browse google repo from Eclipse, lexical order is correctly rendered. But when I browse my company repo, the order is wrong.
Comment 19 Julien HENRY CLA 2011-07-25 10:28:38 EDT
Created attachment 200285 [details]
Correct order on google code
Comment 20 Julien HENRY CLA 2011-07-25 10:29:04 EDT
Created attachment 200286 [details]
Wrong order on my company repo
Comment 21 Julien HENRY CLA 2011-07-25 10:51:55 EDT
(In reply to comment #18)
> I did additional testing and I am very confused. It seems the issue depends on
> which svn repo the svn:externals are created. Maybe because in my company we
> are using SVN server version 1.4?

In fact the version of my company repo is 1.5.5. I was not able to find version of google code repo.
Comment 22 Alexander Gurov CLA 2011-07-25 12:27:11 EDT
(In reply to comment #17)
> More investigation. The crash is not always reproducible. It always occur for a
> specific folder in our SVN, but not in other places where we are using
> svn:externals.
If crash is somehow related to the Subversive plug-in - then it could only be through the native JavaHL client. That's why for myself - I prefer to use the SVN Kit-based one. Regarding crash dump - unfortunately I can't say, it was possible for me to gain any useful information from there.

(In reply to comment #21)
> (In reply to comment #18)
> > I did additional testing and I am very confused. It seems the issue depends on
> > which svn repo the svn:externals are created. Maybe because in my company we
> > are using SVN server version 1.4?
I can't see how there could be a dependence from the server version. After all it works in a very simple manner:
1) get the list of real nodes
2) check if the parent folder has properties (it is cached information)
3) get svn:externals property if exists
4) parse the property, create a nodes and add them to the end of common list
5) reorder them with regards to external folders names
6) map reordered nodes to their UI representations 1-to-1

The only idea I've got is that the svn:externals property in your project contains some kind of invisible characters that weren't skipped by parsing and got recognized along the names as their parts. Is it possible?
Comment 23 Julien HENRY CLA 2011-07-26 03:16:29 EDT
(In reply to comment #22)
> If crash is somehow related to the Subversive plug-in - then it could only be
> through the native JavaHL client. 

I am using SvnKit. JavaHL connector is not even installed.

> I can't see how there could be a dependence from the server version. After all
> it works in a very simple manner:
> 1) get the list of real nodes
> 2) check if the parent folder has properties (it is cached information)
> 3) get svn:externals property if exists
> 4) parse the property, create a nodes and add them to the end of common list
> 5) reorder them with regards to external folders names
> 6) map reordered nodes to their UI representations 1-to-1
> 
> The only idea I've got is that the svn:externals property in your project
> contains some kind of invisible characters that weren't skipped by parsing and
> got recognized along the names as their parts. Is it possible?

As I said, I have created the same configuration (copy/paste) and I get different results depending on which server the svn:externals is "hosted". So no invisible character here... The sort algorithm looks very complex so I'm trying check if there is no corner case.
Comment 24 Julien HENRY CLA 2011-07-26 04:06:56 EDT
Concerning the crash, there is already a bug report and a workaround: Bug 319438 So this is not related to your code.

I still have to figure out why the order is not correct on our internal SVN server.
Comment 25 Julien HENRY CLA 2011-07-26 08:52:28 EDT
Tadaaaam I got it!

See the following test project: https://aubrane.googlecode.com/svn/branches/test-svn-externals-bug3

The issue happens when svn:externals folders are of type IRepositoryRoot. It occurs when the target location ends with "trunk", "tags" or "branches", and also when the target location is not on the same server. In this case a special sorting is done (I guess this is initially a "hack" to have trunk/tags/branches in this order). For externals I think the sort should always be done on the label, whatever is the target location.

I will submit a patch shortly.
Comment 26 Julien HENRY CLA 2011-07-26 09:05:13 EDT
Created attachment 200357 [details]
Patch that fix the issue
Comment 27 Alexander Gurov CLA 2011-07-29 13:11:19 EDT
Hello Julien, 

Thank you very much, I've completely missed this tricky situation while analyzing reordering code. I've applied your patch and also have made changes so that these nodes would not be colored as IRepositoryRoot(s).
Comment 28 Julien HENRY CLA 2011-08-01 10:02:52 EDT
Thanks for your time Alexander. Is there a new version available with the fix?
Comment 29 Alexander Gurov CLA 2011-08-01 10:06:39 EDT
For now there is no newer version , I want to work on few more issues before publishing new build.
Comment 30 Alexander Gurov CLA 2011-08-20 02:17:31 EDT
(In reply to comment #28)
> Thanks for your time Alexander. Is there a new version available with the fix?
Hello Julien, 

New version with the fix is available at "early access" site.
Comment 31 Julien HENRY CLA 2011-08-22 05:29:40 EDT
(In reply to comment #30)
> New version with the fix is available at "early access" site.

Thanks Alexander. I have tested and it works fine now. Should I update status to VERIFIED?
Comment 32 Alexander Gurov CLA 2011-08-22 06:11:15 EDT
(In reply to comment #31)
As you wish. The most important thing is that it works now, thanks to your help. :)
Comment 33 Igor Vinnykov CLA 2012-05-15 11:27:56 EDT
Comment on attachment 200357 [details]
Patch that fix the issue

Set iplog flag