Community
Participate
Working Groups
Adding sftp filesystem return 401 authentication error. When debugging it looks like each time we are trying to connect to the sftp filesystem server should request for credentials via Basic authentication, but this does not happen. Even when I hacked the Projects.prefs file and hardcoded the credentials in the content location URL I still cannot access the filesystem, because it ignores the URL credentials. When is more, I discovered while debugging that after doing GET on the sftp project SftpFileStore.childInfos() gets called over and over again.
I was wrong on the call earlier today. There is a way to add a remote sftp connection outside the old project page. 1. Go to editor, top folder 2. Select [+] action ("New") 3. Choose "Link to server" 4. In "Name" write anything, in "Server path" input the sftp path, like sftp://username@somedomain.net:22/home/username/sftphome/ (has to be a valid sftp connection) 5. Confirm RESUT: "Authentication required for: Orion File Service. Login and re-try the request" message shown, 401 response for POST workspace under the hood EXPECTED: New project created with connection to sftp server
Huh... The WWW-Authenticate header is added to the response. I don't know why I didn't check outher browsers earlier, but now I see that the authentication prompt shows in Chrome, but not in FF. But still when I add the correct credentials it doesn't work. First time it says the the folder already exists. Second time when credentials are already cached nothing happens.
There seems to be a bug in Firefox that does not display Basic Auth prompts in XHR calls: https://groups.google.com/forum/#!topic/mozilla.dev.ajax/wyfKEYBLeBI I checked FF 23.0.1 and FF 24.0 and both seem to have this problem.
In this commit: http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=2675b34374a9b60087940f1c763d7b6c292348c3 I fix two problems: 1. The problem that when we are trying to add a new sftp connection and we get "Project already exists" error after logging in 2. While fetching file info we may end in an infinite loop There are still two problems: 1. We don't get authentication prompt in FF (see comment 3) 2. Even if a link to server is added correctly it is not displayed in navigator Ad 2. The remote projects are not included in workspace's Children. To get remote folders we have to go to GET [workspace_url]/drive?depth=1. This was intentionally added in Bug 399637.
(In reply to comment #4) > 1. We don't get authentication prompt in FF (see comment 3) It looks like our problem may be with the hidden iframe. When I create file service directly in the code instead of doing it by plugin the basic auth prompt is displayed.
(In reply to comment #5) > It looks like our problem may be with the hidden iframe. When I create file > service directly in the code instead of doing it by plugin the basic auth prompt > is displayed. Yes, It's an iframe problem. When I remove iframe.sandbox the prompt is shown.
More information here: We support two content locations for a project, either file://..., which is a local content location, or sftp://..., which is a remote content location If you call http://localhost:8080/workspace/user-OrionContent , only local projects are returned. If you call http://localhost:8080/workspace/user-OrionContent/drive , only remote projects are returned. Pretty sure that the drive feature was added for the Orion 3.0 projects feature. As soon as I remove the "drive" logic from the WorkspaceResourceHandler, I can successfully get the full project list, access and edit files connected to the remote sftp project. To change a local project into a remote project, you just need to change the content location. As you have already mentioned above, this only works for chrome, it does not work for firefox.
We need to add "Create a project from a SFTP site" so we can debug this further. There is nothing blocking adding this command and it will work on Chrome.
(In reply to Anthony Hunter from comment #8) > We need to add "Create a project from a SFTP site" so we can debug this > further. There is nothing blocking adding this command and it will work on > Chrome. Anton was asking about this again. I will look at adding this project support again if I get a chance unless Gorza gets to it first.
Experience notes. I used Orion to make some new simple blog software for a personal project ( http://goo.gl/dGVPzD ). It's experimental in some ways - just a simple blog. So I reached a point where I wanted to put this on my server. So I downloaded it as a zip, and copied it into Coda on my mac. I won't use Orion again for editing this software. I'd prefer to. My analytics software tells me that ~ 600 unique requests have come to read my blog about Orion and SFTP. ~800 reads in total. 7 reads today. So, from that I derive there is interest in it as a feature. My hosting company tells users to use one of three options for FTP - Filezilla for example. It provides custom config files for its users. This is a real case. SFTP is still important to some users. It seems a shame that we're so close to providing it. It would be brilliant to make a reliable way of using it - both as a real time connection, and as a synched connection. But just one way would be great to start with.
*** Bug 422189 has been marked as a duplicate of this bug. ***
Created attachment 237824 [details] orion client patch to added "Create a project from a sftp site" I have pushed commit: git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=f19f2b913bde3a9152553683dcf3cba5ed7f63e7 This small change removes the drive support from the workspace handler so that all projects (remote = sftp:/ and local = file:/) are returned by the workspace handler. With this change, all we need to do is add the "Create a project from a sftp site" to the plus button. This attachment is a first try and it works on Chrome. Gorza, can you review my patch and confirm it is ok? I think something must be off as the git project command is now missing and I am not sure why yet.
Anthony, sftp connection works on Chrome, but does not work on FF. The problem is still that the basic auth prompt is not shown on FF due to sandboxing restriction. You can test in on the client side yourself, using [+] -> Link to Server in editor root.
Another problem is that I get error 500 when I try to remove the folder linked to sftp.
What are the steps for testing - do you both have a patch? I think it's fair to support on Chrome to begin with, and solve the problem for Firefox as an increment. Better supporting on one browser ( the most popular for our software ) than none.
You don't need a patch, this is exactly the scenario: [+] -> Link to Server then in the field paste the sftp address in the field, in normal format: sftp://user@server.com:22/path/on/the/server
Created attachment 237880 [details] Error on attempting to connect I tried to connect to my sftp site and saw the attached picture. The error is unhelpful - I don't know what part of my sftp credentials is faulty. I'd prefer a dialog to type fields into - ftp server name, user name, path If you look at Filezilla or Coda - that's what they ask for - the trouble with asking for the string is that you leave it very open for misinterpretation and subsequently frustration. In any case - what's wrong with this? Mark seems to think it's a server setting.
Maybe you can try "SFTP from..." or "SFTP to..." action with the same server and then if it works we'll know that this is solo sftp filesystem error, not general problem with connecting to the server.
There is a bug on the server... fix coming. Although I'm sure there is also more we can do on the client side with helping the user to enter the information. On the SFTP from/to command we have a popup with fields to fill out that is a bit easier to use. Maybe enter a separate bug for the usability and I will use this for the server bug.
(In reply to Anthony Hunter from comment #12) > Gorza, can you review my patch and confirm it is ok? I think something must > be off as the git project command is now missing and I am not sure why yet. Hi Gorza, were you able to confirm that the patch works on Chrome?
(In reply to John Arthorne from comment #19) > There is a bug on the server... fix coming. Although I'm sure there is also > more we can do on the client side with helping the user to enter the > information. On the SFTP from/to command we have a popup with fields to fill > out that is a bit easier to use. Maybe enter a separate bug for the > usability and I will use this for the server bug. The patch I added above is the start of such a create project action. We can expand, but want to make sure it is good for a start.
Do we plan to fix SFTP on FF? I mean the bug about not prompting for credentials?
http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=83126d59829351174c9a97f4a7830bef2cd0f262 With this fix I am able to create sftp-backed projects, at least on localhost. I know this worked in the past so I think this was a recent regression.
(In reply to Szymon Brandys from comment #22) > Do we plan to fix SFTP on FF? I mean the bug about not prompting for > credentials? Yes - absolutely we should - as far as I know that's something that can be rectified through an iframe? BUT - SFTP seems a difficult thing to get right even on Chrome ( without even focusing on the problem with FireFox ). Please let's not focus on the FF problem UNTIL we have a reliable way of connecting in one browser. It'll be much easier to fix it then - to compare and eliminate other problems. Plus - supporting it for a start in Chrome and Safari as at least a starting point, it is much better than not supporting it in any browser - at least it is an option for some, a choice they can make.
Problem from comment 0 is still valid. SFTP filesystem still does not work on FF unless it allows anonymous access. If we declare FF as supported browser I don't really think that we should do anything further with sftp until it starts working on FF.
(In reply to Anthony Hunter from comment #20) > (In reply to Anthony Hunter from comment #12) > > Gorza, can you review my patch and confirm it is ok? I think something must > > be off as the git project command is now missing and I am not sure why yet. > > Hi Gorza, were you able to confirm that the patch works on Chrome? Sorry, Hi Gosia, could you review the patch? It adds the "Create a project from a sftp site", you provide a name and URL (sftp://username@somedomain.net:22/home/username/sftphome/) .
(In reply to comment #26) > Sorry, Hi Gosia, could you review the patch? It adds the "Create a project from > a sftp site", you provide a name and URL > (sftp://username@somedomain.net:22/home/username/sftphome/) . Anthony, I applied your patch and I get a failure 500: {"HttpCode":500,"DetailedMessage":"Failureconnectingtosftp://gosiatomczyk:[password]@frs.sourceforge.net:22","Message":"Failureconnectingtosftp://gosiatomczyk:[password]@frs.sourceforge.net:22","Severity":"Error","Code":0} when I reload the page the top-level folder is created and it works, but it does not contain project.json, so it's not recognized as a project. BTW: do we accept that adding a project like this will create project.json on the remote site? And that we cannot add a project this way if we have readonly access? I think that adding SFTP dependecies is also important (or maybe more) and should not have those problems.
I tested again and do not have issues at all on Chrome, so I have committed the "Create a project from a sftp site" contribution to the Orion client. We now have a base for debugging.
(In reply to Anthony Hunter from comment #28) > I tested again and do not have issues at all on Chrome, so I have committed > the "Create a project from a sftp site" contribution to the Orion client. > > We now have a base for debugging. This just did not sound right, so google is your friend. Abbreviations that start with the consonants F, H, L, M, N, R, S and X attract an, because they start with vowel sounds and SFTP is an abbreviation, so capitalized. so it is "Create a project from an SFTP site"
(In reply to Anthony Hunter from comment #28) > I tested again and do not have issues at all on Chrome, so I have committed > the "Create a project from a sftp site" contribution to the Orion client. > > We now have a base for debugging. What about my problem with error 500? Did you manage to reproduce it?
(In reply to Malgorzata Janczarska from comment #30) > What about my problem with error 500? Did you manage to reproduce it? I tried it today using my SourceForge account. It didn't work. This is what I get. {"HttpCode":500,"DetailedMessage":"Failure connecting to sftp://sbrandys:jajeczko1@frs.sourceforge.net:22","Message":"Failure connecting to sftp://sbrandys:********@frs.sourceforge.net:22","Severity":"Error","Code":0} This SFTP server works using WinSCP for instance. Also when I refreshed the page I saw the new project, but could do nothing with it. What is even worse, when I tried to add a file/folder to it, I got an error status saying: Failure connecting to sftp://sbrandys:[password]@frs.sourceforge.net:22 where [password] was in open text. Not sure why you ignored Gosia's comment 27. I would at least double check with someone else before pushing.
(In reply to Szymon Brandys from comment #31) ... also could you disable "Create from SFTP" action on non-supporting browsers like FF, unless you are going to fix FF during M1?
(In reply to Malgorzata Janczarska from comment #30) > (In reply to Anthony Hunter from comment #28) > > I tested again and do not have issues at all on Chrome, so I have committed > > the "Create a project from a sftp site" contribution to the Orion client. > > > > We now have a base for debugging. > > What about my problem with error 500? Did you manage to reproduce it? I do not have any problems at all with the FTP account I am using.
(In reply to Anthony Hunter from comment #33) > I do not have any problems at all with the FTP account I am using. Could you create a sourceforge account and try it there? Both mine and Szymons failed.
(In reply to Anthony Hunter from comment #33) > (In reply to Malgorzata Janczarska from comment #30) > > (In reply to Anthony Hunter from comment #28) > > > I tested again and do not have issues at all on Chrome, so I have committed > > > the "Create a project from a sftp site" contribution to the Orion client. > > > > > > We now have a base for debugging. > > > > What about my problem with error 500? Did you manage to reproduce it? > > I do not have any problems at all with the FTP account I am using. I still see the same problem on orion.eclipse.org with the latest deployment as the image that I attached a screenshot for. Error on attempting to connect.
So sftp basically worked with my home site. However, adding it significantly reduced my main page load time. Now it takes up to 5 seconds to display. If I remove the sftp directory then it returns to normal. Can also say that a project was not really created, but the directory showed up.
(In reply to Ken Walker from comment #36) > However, adding it > significantly reduced my main page load time. Now it takes up to 5 seconds > to display. This is probably caused by getting some information while listing folders, that require contacting remote server, like folder size. In previous metastore this used to be addressed as far as I remember: the sftp folders where not included when listing workspace folders, but one had to make a request to well-known location for that. It's not a great solution though, because then sftp folders don't show up in your workspace. Another problem is that when you log in to Orion next time you'll be probably prompted for basic auth password to sftp server while opening workspace root.
I was adding the wrong format string for my server. Adding the correct string gets me a connection, but no content. Anthony can reproduce this. I think the first step is to develop some reliability in connecting from at least one browser, and then build better usability around it. I like the idea of limiting a server connection to a project at first - so that we can have a 'local' work area, and the remote connection. It would not be synchronized, but it would be workable to begin with, and it could then limit any kind of credential query until the project is accessed, whereby it should be acceptable to the person working with it. I'm worried about what Ken's seeing if we just add these servers to the main navigation. Just a thought.
Another note - Anthony just mentioned to me. For my server, it seems to only connect when I don't supply a path - rejects it even if I supply a path. It might be the same for other servers. Mine is a commercial North American ISP.
I have done some debugging using Anton's hickory site. With two tweaks it mostly works, although it is still unstable, as I will elaborate on below. As Anton says above, you cannot add a path or port to his SFTP server URL. Not sure what this means yet. With my SFTP server, I am depending on the path to isolate the Orion project to a specific folder on the SFTP site. Anton cannot do this, so you can only get his entire SFTP site as an Orion project. As a result of this limitation, performance degrades quickly for this project. The top root level folder for the hickory site has 46 entries. Due to the way to have coded the client, we have two explorers and so two calls to /file/{folder}/?depth=1, in addition to a bunch of other calls to the /workspace and /file servlets. Each of these makes separate calls to the SFTP server. In addition, sever side, the projectStore is called a bunch more times by decorators, etc. I reduced some calls by commenting out a call to projectStore.fetchInfo() to get a timestamp. Some other operation is making the same call and commenting this out does not result in anything missing in the UI. The end result is still that the root listing takes somewhere between 26 and 42 seconds to complete. FileZilla does the same operation in one second by calling "ls -l" and I guess parsing the results. Unfortunately, the timeout on the client for all server calls is 15 seconds. Since I knew these listings were taking as much as 42 seconds, I changed all the timeouts to three minutes and was able to get the site to work. There is still a further problem that on occasion, a call to the SFTP server results in a "Failed to authenticate to host" error. If this occurs in the right spot, the entire page fails to load and you get the error like in comment #31 So to get the hickory side to work: 1) Change all timeouts to three minutes 2) Eliminate extra SFTP server calls server side. So in conclusion, there is a lot of work needed to get this to work: 1) We need a better way to validate the SFTP URL the user provides. 2) We need to handle the case client side where remote projects could take longer than 15 seconds to respond. 3) Revisit how we do directory listings server side on remote projects. 4) Debug the "Failed to authenticate to host" error and unhide the exception server side. 5) Support Firefox.
Not sure how many of the changes are in the server now. I could go two folders deep, but not three. So close to making a css change on my website that I know I need to make! Also when I made a connection this time, I got an error message - 'an unknown error occurred' - yet the connection was made when I refreshed the page. So - for my case we're narrowing in on a success path. I think it would be better as a feature of projects - where we add a server as associated content of a project [ as in the design that I circulated ]. I'd be willing to work on some of the client side support in the next iteration.
We are getting better with sftp filesystem on Chrome, but there are still some issues I suggest we should hide this action for M1, so it's not consumed by other Orion-based applications using M1.
(In reply to Malgorzata Janczarska from comment #42) > We are getting better with sftp filesystem on Chrome, but there are still > some issues I suggest we should hide this action for M1, so it's not > consumed by other Orion-based applications using M1. Removed for 5.0 M1 with commit: http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/commit/?id=16eb22a0ffdacf905bd98ea7927aecceec979c5e
I am not sure if we still plan to follow through with this enhancement.
Closing as part of a mass clean up of inactive bugs. Please reopen if this problem still occurs or is relevant to you. For more details see: https://dev.eclipse.org/mhonarc/lists/orion-dev/msg03444.html
