Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 373443

Summary:	import from HTTP
Product:	[ECD] Orion	Reporter:	Susan McCourt <susan>
Component:	Server	Assignee:	John Arthorne <john.arthorne>
Status:	RESOLVED FIXED	QA Contact:
Severity:	enhancement
Priority:	P3	CC:	antonm, john.arthorne, mamacdon, simon_kaegi, Szymon.Brandys, tomasz.zarna
Version:	0.4
Target Milestone:	0.5 M1
Hardware:	PC
OS:	Windows 7
Whiteboard:
Bug Depends on:
Bug Blocks:	362067

Description Susan McCourt

2012-03-06 17:15:04 EST

As part of working on bug 362067, I'm trying to figure out how to use the Pixlr API to save an edited image back to Orion. I managed to wire everything up so that I can tell Pixlr what URL to call when it saves something. It comes back to my URL and I can successfully determine the URL where the image is saved.

Now the million dollar question is:
How could I take this URL and cause it to be saved back to the file client? Basically I want to be able to "copy" the bytes at the specified URL back to a known location in my fileClient.

Ideas that Simon and I discussed:
- an Orion servlet that does this. Downside: letting plugins/people download random stuff into Orion. What if it's the latest Harry Potter movie pirated? Simon thinks it's more exposure to Orion/the user if we are moving random bytes into Orion than if we have the user do it via browser extension, etc. At least the user presumably is taking on the risk if they explicitly install something that allows it.
- some kind of CORS service that would take this risk and then make content available to Orion from CORS
- a browser extension or flash solution

We need to figure this out.
It's a pretty common API where some web tool gives you the URL to whatever it created for you (images, css generators, etc.). We need to get over the hump of "well, I guess you have to download it and then import it back in."

If we aren't willing to take on the risk of cross domain byte slurping then we need to find some add-on that can do it.

Simon wants to think about it some....

Comment 1 Susan McCourt

2012-03-06 20:06:55 EST

Simon suggested that we may be able to do a totally hacko save by mapping the pixelr URL returned from save to a site mapping.  That would let us get the bytes via the self hosting site proxy. I'm not fully sure how this would work yet.  I think the pixlr hostname changes (api1, api2, etc.) so I think we'd have to read the hostname and add a mapping on the fly.  Mark, is that even possible?

Another alternative might be to make it the user's problem to map the urls used (when they try save) to their self hosting site.  I'm going to try this.

Comment 2 Susan McCourt

2012-03-07 13:26:44 EST

John/Simon - how do you feel about us supporting an "import from HTTP" command?  The user would have to initiate it.  If we had this command, i could make pixlr save work today.  I already hand it the saveurl that it should use and from that I figure out the source HTTP.

Then I'd just need to open a command slide out for "import from HTTP" specifying the source HTTP and the target resource location.

The user would always have to approve it.

We could put a link to the resource in the slideout so that the user could preview it in the browser if they didn't trust it.

Comment 3 Szymon Brandys

2012-03-07 13:40:28 EST

We already have apply patch from URL. It works via a server hook. I think it would be a similar thing.

Comment 4 Susan McCourt

2012-03-07 13:46:58 EST

John said he can look at this.
cc'ing Tomek because he's done something similar for "apply patch from URL"

Comment 5 Simon Kaegi

2012-03-07 13:49:00 EST

I like it a zillion times better than the proxy. Not sure if it would be better to be tied to a filesystem or a generic service that gets the bytes and then lets the user save them where they like.

Comment 6 John Arthorne

2012-03-07 14:23:45 EST

I think an "Add from URL" command would be quite reasonable. I have found need for this in copying resources from other web servers into my workspace. Anton is working on a single file import UI so maybe it can fit in with that on the client side.

Comment 7 Anton McConville

2012-03-07 14:38:45 EST

Have ideas on fitting in an image import ... though I want to finish this first part of the import UI revamp I'm doing.

Would we want to filter files to begin with - only allow images for instance?

Comment 8 Mark Macdonald

2012-03-07 14:57:26 EST

(In reply to comment #1)
> Simon suggested that we may be able to do a totally hacko save by mapping the
> pixelr URL returned from save to a site mapping.  That would let us get the
> bytes via the self hosting site proxy. I'm not fully sure how this would work
> yet.  I think the pixlr hostname changes (api1, api2, etc.) so I think we'd
> have to read the hostname and add a mapping on the fly.  Mark, is that even
> possible?

In theory this would work -- you could avoid the same-domain-xhr problem -- but there's no way to dynamically add a mapping. You'd have to save the pixelr image, get its URL, add a mapping for the URL to your self-hosting site config, and restart the site.

The HTTP import idea sounds a lot nicer...

Comment 9 Susan McCourt

2012-03-07 16:49:22 EST

(In reply to comment #7)
> Have ideas on fitting in an image import ... though I want to finish this first
> part of the import UI revamp I'm doing.
> 
> Would we want to filter files to begin with - only allow images for instance?

For the generic "import from URL" command I would say no.  No reason to limit to images.  This mechanism will be generally useful for any delegated editor to specify where it saved remote content.  Then we can use it go bring it back into the workspace.

Comment 10 John Arthorne

2012-03-15 15:51:09 EDT

I have added support for doing a PUT with arbitrary HTTP input:

http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=cc87de99efbdd349fed782aa75021ff9f2bce23a

Documentation:

http://wiki.eclipse.org/Orion/Server_API/File_API#Setting_file_contents_with_different_HTTP_input

I guess we need an equivalent method for file creation?

Comment 11 John Arthorne

2012-03-15 16:03:21 EDT

(In reply to comment #10)
> I guess we need an equivalent method for file creation?

In theory our current "copy file" POST request could be made to handle arbitrary source URLs. Currently the implementation makes the assumption that the source is on the same server. I have opened bug 374439 to fix this, but don't plan to work on it until we have a need for it. Client code may be easier and more consistent if it just uses POST followed by PUT for this case.