Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 326489 - [tcf] Support recursive directory copy/delete
Summary: [tcf] Support recursive directory copy/delete
Status: RESOLVED FIXED
Alias: None
Product: TCF
Classification: Tools
Component: Core (show other bugs)
Version: unspecified   Edit
Hardware: PC Mac OS X - Carbon (unsup.)
: P3 normal (vote)
Target Milestone: 0.4.0   Edit
Assignee: Anna Dushistova CLA
QA Contact: Martin Oberhuber CLA
URL:
Whiteboard:
Keywords:
Depends on: 246987
Blocks:
  Show dependency tree
 
Reported: 2010-09-29 03:58 EDT by Anna Dushistova CLA
Modified: 2013-06-05 07:56 EDT (History)
5 users (show)

See Also:


Attachments
Supporting directory recursive copy/delete via TCF (11.07 KB, patch)
2010-09-30 09:05 EDT, Liping Ke CLA
no flags Details | Diff
Supporting directory recursive copy/delete via TCF (12.04 KB, patch)
2010-10-09 04:52 EDT, Liping Ke CLA
no flags Details | Diff
Supporting directory recursive copy/delete via TCF (9.03 KB, patch)
2010-10-15 03:53 EDT, Liping Ke CLA
anna.dushistova: iplog+
anna.dushistova: review+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Anna Dushistova CLA 2010-09-29 03:58:06 EDT
This bug is created per #comment 29 for #bug 246987.
Comment 1 Liping Ke CLA 2010-09-30 09:05:52 EDT
Created attachment 179948 [details]
Supporting directory recursive copy/delete via TCF

Supporting directory recursive copy/delete via TCF
Comment 2 Liping Ke CLA 2010-09-30 09:07:40 EDT
Hi, Anna

I paste the patch here.

Since we're on several day's national day leave, email access might be a little slower:)

Thanks a lot!
criping
Comment 3 Anna Dushistova CLA 2010-10-05 18:08:28 EDT
Liping,
I was not able to apply this patch right away. However, after applying it manually it doesn't seem to work: when I am trying to copy a directory, the "Paste" action is disabled.
Comment 4 Anna Dushistova CLA 2010-10-05 18:12:53 EDT
Delete works though. Also, could you please explain why you need
to change getServiceImplType in FileSubsystemConfiguration and ProcessSubsystemConfiguration to return ITCFSubsystem?
Comment 5 Liping Ke CLA 2010-10-07 21:19:15 EDT
Hi, Anna

I will try latest RSE code and verify the patch why paste is disabled. The UI problem is decided by RSE part.
And the reason why I change the getServiceImplType, is because we'd better let all sub-services including (process, file, shell, terminal) share the same TCF connection (Seems telnet and ssh did the same thing). Otherwise, the login password dialog will be prompted for several times.

How do you think?

After I verify the patch with latest upstream, I will resend the patch to you.

And the three patches should have apply orders. I will give the orders after resend the patch.


Thanks& Regards,
criping
Comment 6 Liping Ke CLA 2010-10-09 04:52:11 EDT
Created attachment 180543 [details]
Supporting directory recursive copy/delete via TCF

Hi, Anna
I rebased the patch with latest trunk. And I tested both copy and delete. It works. For copy, please make sure that the UI copy-paste action direction should be (remote -> remote). If you want to copy local files to remote, it will go different path (upload, download). So my ops sequence is:
1) new a TCF connection
2) run agent
3) click file service under TCF connection
4) select a folder under eg: tcf conn/Files/Root/test_folder, right click copy
5) select a folder under eg: tcf conn/Files/Root/tmp, right click paste.

If you still meet any problem, just let me know. I am using Helios (eclipse classic, 3.6) and tm version is http://download.eclipse.org/dsdp/tm/updates/3.2.

Thanks a lot for your help!

criping
Comment 7 Anna Dushistova CLA 2010-10-11 15:58:48 EDT
I am using RSE 3.2.0 for testing.
In my home directory, I created a tmp/ dir.
I am trying to copy /home/anna/<my dir with subfolders> to /home/anna/tmp.
I can click "Copy" on /home/anna/<my dir with subfolders>, but "Paste" on /home/anna/tmp is grayed out.
However, this operation(with the very same folders) works with the Linux/ssh RSE connection type.

(In reply to comment #6)
> Created an attachment (id=180543) [details]
> Supporting directory recursive copy/delete via TCF
> 
> Hi, Anna
> I rebased the patch with latest trunk. And I tested both copy and delete. It
> works. For copy, please make sure that the UI copy-paste action direction
> should be (remote -> remote). If you want to copy local files to remote, it
> will go different path (upload, download). So my ops sequence is:
> 1) new a TCF connection
> 2) run agent
> 3) click file service under TCF connection
> 4) select a folder under eg: tcf conn/Files/Root/test_folder, right click copy
> 5) select a folder under eg: tcf conn/Files/Root/tmp, right click paste.
> 
> If you still meet any problem, just let me know. I am using Helios (eclipse
> classic, 3.6) and tm version is
> http://download.eclipse.org/dsdp/tm/updates/3.2.
> 
> Thanks a lot for your help!
> 
> criping
Comment 8 Anna Dushistova CLA 2010-10-11 16:02:25 EDT
(In reply to comment #5)
> Hi, Anna
> 
> I will try latest RSE code and verify the patch why paste is disabled. The UI
> problem is decided by RSE part.
> And the reason why I change the getServiceImplType, is because we'd better let
> all sub-services including (process, file, shell, terminal) share the same TCF
> connection (Seems telnet and ssh did the same thing). Otherwise, the login
> password dialog will be prompted for several times.

Just looked at the RSE code to make sure - both ssh and telnet are working through service and not subsystem.
Comment 9 Liping Ke CLA 2010-10-11 21:38:05 EDT
(In reply to comment #7)
> I am using RSE 3.2.0 for testing.
> In my home directory, I created a tmp/ dir.
> I am trying to copy /home/anna/<my dir with subfolders> to /home/anna/tmp.
> I can click "Copy" on /home/anna/<my dir with subfolders>, but "Paste" on
> /home/anna/tmp is grayed out.
> However, this operation(with the very same folders) works with the Linux/ssh
> RSE connection type.

Hi, Anna
I am trying to reproduce your steps.
1) I am not using a remote target, but use TCF 127.0.0.1. Are you using remote target?
2) In your home directory, created a tmp/dir, you mean under 127.0.0.1_TCF->Files->Home, you create tmp? I found whatever folders I am creating (I select Home, right click the mouse, select new->folders, filling tmp in the UI), it will report "New resource will not be visible due to subsetting. Create it anyway?) If I continue, I found the tmp does exist, but not visible under view 127.0.0.1_TCF->Files->Home. I will check the latest 3.2 code why this happen.
3) If I create the folders tmp under root->home->lke, everything is fine. After creating, I can copy anything to root->/home/lke/tmp.

For the getServiceImplType, I recheck the code in org.eclipse.rse.subsystems.*.ssh (RSE 3.1), yes, you're right. it will all return service type ISshService.class, not subsystem type.
I remember the reason why I modify this code: If we use different service type,  each sub-service(shell, terminal, etc) will get a different connectorservice. After the connection is authenticated, the password prompt UI will be popped up several times. In order to make the four sub-services reuse the same connectorservice at the same time so that the password UI will not prompt several times, I made this change. (At first, I use service type for ssh, terminal too). Any suggestion for this?

Thanks a lot for your help!
criping
Comment 10 Liping Ke CLA 2010-10-11 21:48:38 EDT
Hi, Anna

Under my Home directory, I can't create any folder correctly. It will report ""New resource will not be visible due to subsetting. Create it anyway". Do you meet the same problem? I check the code in 
org.eclipse.rse.internal.files.ui.wizards.SystemNewFolderWizard, it is because meetsFilterCriteria fails. Do you meet the same problem?

I will down load RSE3.2 source code to do some debugging for this after I finished my release cycle this week.

Thanks& Regards,
criping
Comment 11 Liping Ke CLA 2010-10-12 02:50:26 EDT
Hi, Anna

I just identified a bug in TCFFileSubSystemConfiguration.java which caused my problem. It must set setIsUnixStyle(true). Yet current code did not set, so my linux system was identified as "windows style". For example, when I want to create test under Files->Home, the correct path should be /home/lke/test, yet since it is not set, absolute path will be given as (/home/lke\\test), so meetsFilterCriteria=false, then I meet the error.

But If I set the unixStyle, I will meet other errors, Home will not point to /home/lke, but the root. Seems I need sometimes to debug the root cause. I will fire a bug and send a fix to this problem firstly.

Thanks & Regards,
criping
Comment 12 Liping Ke CLA 2010-10-12 04:24:57 EDT
Oh, I found after setting setIsUnixStyle(true), all temp folders (in debugging plugin env, remove runtime-EclipseApplicationXXX) must be removed. Then everything is fine. I can create new folders under Files->Home now. I will open a new bug and assign it to you. And I will send you the fix patch too.

Thanks a lot!
criping
Comment 13 Liping Ke CLA 2010-10-12 04:46:23 EDT
And Anna, 
I filed a bug for the problem I mentioned above.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=327525.

Still after fix this bug, I did following things:
1) click tcf_local icon
2) click Files local icon
3) click Home icon
4) select Home, right click folder, New->Folder, filling the dialog box (test)
5) select any folders under Home, right click folder, click Copy
6) select test you just created, right click folder, click paste.

Everything is fine.

Could you tell me the UI procedure when you meet problems? I can't reproduce the problem here.
I download RSE 3.2.1 source code, use debug mode launching.

Thanks a lot!
criping
Comment 14 Anna Dushistova CLA 2010-10-12 06:15:58 EDT
(In reply to comment #9)
> (In reply to comment #7)
> > I am using RSE 3.2.0 for testing.
> > In my home directory, I created a tmp/ dir.
> > I am trying to copy /home/anna/<my dir with subfolders> to /home/anna/tmp.
> > I can click "Copy" on /home/anna/<my dir with subfolders>, but "Paste" on
> > /home/anna/tmp is grayed out.
> > However, this operation(with the very same folders) works with the Linux/ssh
> > RSE connection type.
> 
> Hi, Anna
> I am trying to reproduce your steps.
> 1) I am not using a remote target, but use TCF 127.0.0.1. Are you using remote
> target?

No, localhost is my target. My OS is OpenSuse 11.2.

> 2) In your home directory, created a tmp/dir, you mean under
> 127.0.0.1_TCF->Files->Home, you create tmp? I found whatever folders I am
> creating (I select Home, right click the mouse, select new->folders, filling
> tmp in the UI), it will report "New resource will not be visible due to
> subsetting. Create it anyway?) If I continue, I found the tmp does exist, but
> not visible under view 127.0.0.1_TCF->Files->Home. I will check the latest 3.2
> code why this happen.

No, I mean going to the shell and doing something like "cd; mkdir tmp".
And then doing refresh in Eclipse.
Comment 15 Anna Dushistova CLA 2010-10-12 06:18:55 EDT
(In reply to comment #9)
> For the getServiceImplType, I recheck the code in
> org.eclipse.rse.subsystems.*.ssh (RSE 3.1), yes, you're right. it will all
> return service type ISshService.class, not subsystem type.
> I remember the reason why I modify this code: If we use different service type,
>  each sub-service(shell, terminal, etc) will get a different connectorservice.
> After the connection is authenticated, the password prompt UI will be popped up
> several times. In order to make the four sub-services reuse the same
> connectorservice at the same time so that the password UI will not prompt
> several times, I made this change. (At first, I use service type for ssh,
> terminal too). Any suggestion for this?

You might want to take a look at the AbstractConnectorServiceManager and how it is used in RSE.
Comment 16 Liping Ke CLA 2010-10-12 21:11:52 EDT
(In reply to comment #14)
> No, I mean going to the shell and doing something like "cd; mkdir tmp".
> And then doing refresh in Eclipse.
Hi, Anna
I am using Ubuntu. And I tried your steps
1) launch eclipse debug mode with TCF RSE
2) Open TCF Files
3) mkdir dir /home/lke/test in terminal
4) Refresh TCF Files (F5 or click mouse). I found paste is disabled now.
5) copy a folder and now paste is enabled.
6) I can copy it successfully.

But I have a local fix for linuxstyle problem.

I will try to found an open suse machine to test whether the problems exist on that release and let you know the result.

Thanks a lot!

criping
Comment 17 Anna Dushistova CLA 2010-10-13 15:45:05 EDT
I made some more testing on that and this is what I found out. 
If I run an agent as "anna"(./agent) and login as "anna" user(which is picked by default anyways) in RSE,
it all works fine.
But if I run an agent as root(sudo ./agent) but login in RSE as "anna"(because it's default and I was lazy to change it), paste is disabled for me. Btw, in this case I am shown /root contents under home, which seems to be incorrect because I am logged in as "anna".

(In reply to comment #16)
> (In reply to comment #14)
> > No, I mean going to the shell and doing something like "cd; mkdir tmp".
> > And then doing refresh in Eclipse.
> Hi, Anna
> I am using Ubuntu. And I tried your steps
> 1) launch eclipse debug mode with TCF RSE
> 2) Open TCF Files
> 3) mkdir dir /home/lke/test in terminal
> 4) Refresh TCF Files (F5 or click mouse). I found paste is disabled now.
> 5) copy a folder and now paste is enabled.
> 6) I can copy it successfully.
> 
> But I have a local fix for linuxstyle problem.
> 
> I will try to found an open suse machine to test whether the problems exist on
> that release and let you know the result.
> 
> Thanks a lot!
> 
> criping
Comment 18 Liping Ke CLA 2010-10-13 20:29:07 EDT
Hi, Anna
Thanks for the useful information. I will try your steps.
Thanks& Regards,
criping
Comment 19 Liping Ke CLA 2010-10-13 21:30:48 EDT
(In reply to comment #17)
> I made some more testing on that and this is what I found out. 
> If I run an agent as "anna"(./agent) and login as "anna" user(which is picked
> by default anyways) in RSE,
> it all works fine.
> But if I run an agent as root(sudo ./agent) but login in RSE as "anna"(because
> it's default and I was lazy to change it), paste is disabled for me. Btw, in
> this case I am shown /root contents under home, which seems to be incorrect
> because I am logged in as "anna".

Hi, anna
Just had a quick look @ the problem, seems it's nothing to do with copy/delete problem. But the limitation of TCF itself.
After you apply a patch, set your system to be linux style, you will found your Home directory will show nothing instead of root directory, (showing root is caused by the bug i reported, linux style is not set).
I checked the code, in TCFFileService.java, getUserHome will ask ask TCF agent what's the current user. If you're running TCF as root, it will return root instead of anna. That's the key of inconsistency. I am not sure how to fix this problem.
How's your idea?

Thanks & Regards,
criping
Comment 20 Eugene Tarassov CLA 2010-10-14 13:20:00 EDT
(In reply to comment #19)
> I checked the code, in TCFFileService.java, getUserHome will ask ask TCF agent
> what's the current user. If you're running TCF as root, it will return root
> instead of anna. That's the key of inconsistency. I am not sure how to fix this
> problem.
> How's your idea?
> 
> Thanks & Regards,
> criping

The problem is that Terminals service is a wrong place to handle login. It only changes user account for bash that it starts for a terminal, but all services that the agent provides continue to use original account (e.g. root). Correct implementation of login would fork the agent, change user on the child agent, then re-route all client traffic to the new agent. This is better to be implemented as a separate service, e.g. Login service. Until it is done, it is better to assume that login is not supported at all, and a user has to start the agent under right user account before connecting to it.

Regards,
Eugene
Comment 21 Liping Ke CLA 2010-10-14 20:39:57 EDT
> The problem is that Terminals service is a wrong place to handle login. It only
> changes user account for bash that it starts for a terminal, but all services
> that the agent provides continue to use original account (e.g. root). Correct
> implementation of login would fork the agent, change user on the child agent,
> then re-route all client traffic to the new agent. This is better to be
> implemented as a separate service, e.g. Login service. Until it is done, it is
> better to assume that login is not supported at all, and a user has to start
> the agent under right user account before connecting to it.
> 
> Regards,
> Eugene

Hi, Eugene
Thanks for the useful explanation. We did not realize the problem at first because we only need shell services. Yes, when using authenticated connection, if file/process service still not switch user but using agent owner to decide filters , it will cause incompatible issues. We'll figure out a way to totally solve the problem and make all services consistent.
Thanks & Regards,
criping
Comment 22 Anna Dushistova CLA 2010-10-15 03:45:43 EDT
So, I can accept the current patch if changes to TCFFileSubSystemConfiguration.java
and TCFProcessSubSystemConfiguration.java are reverted and I suggest we track the login issues in a separate bug. How does that sound?

(In reply to comment #21)
> > The problem is that Terminals service is a wrong place to handle login. It only
> > changes user account for bash that it starts for a terminal, but all services
> > that the agent provides continue to use original account (e.g. root). Correct
> > implementation of login would fork the agent, change user on the child agent,
> > then re-route all client traffic to the new agent. This is better to be
> > implemented as a separate service, e.g. Login service. Until it is done, it is
> > better to assume that login is not supported at all, and a user has to start
> > the agent under right user account before connecting to it.
> > 
> > Regards,
> > Eugene
> 
> Hi, Eugene
> Thanks for the useful explanation. We did not realize the problem at first
> because we only need shell services. Yes, when using authenticated connection,
> if file/process service still not switch user but using agent owner to decide
> filters , it will cause incompatible issues. We'll figure out a way to totally
> solve the problem and make all services consistent.
> Thanks & Regards,
> criping
Comment 23 Liping Ke CLA 2010-10-15 03:52:30 EDT
(In reply to comment #22)
> So, I can accept the current patch if changes to
> TCFFileSubSystemConfiguration.java
> and TCFProcessSubSystemConfiguration.java are reverted and I suggest we track
> the login issues in a separate bug. How does that sound?
> 
Hi, Anna
That sounds good!
For the single connection and login issues, we will try to find a good solution which will need some time.
Regards,
criping
Comment 24 Liping Ke CLA 2010-10-15 03:53:06 EDT
Created attachment 180931 [details]
Supporting directory recursive copy/delete via TCF
Comment 25 Anna Dushistova CLA 2010-10-15 05:17:52 EDT
This patch is applied, thanks.
I created a bug 327865 to track login issues.
Comment 26 Doug Schaefer CLA 2011-05-17 10:50:18 EDT
Moving bugs to new home for IP log.
Comment 27 Martin Oberhuber CLA 2013-06-05 06:27:59 EDT
Bulk change: Marking all bugs from the TM era (until June 2011) target 0.3