Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 389089 - Import a zip from a URL
Summary: Import a zip from a URL
Status: RESOLVED FIXED
Alias: None
Product: Orion
Classification: ECD
Component: Server (show other bugs)
Version: 0.5   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: 1.0 M2   Edit
Assignee: John Arthorne CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 369968 378749
  Show dependency tree
 
Reported: 2012-09-07 15:29 EDT by Susan McCourt CLA
Modified: 2012-09-18 10:27 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Susan McCourt CLA 2012-09-07 15:29:11 EDT
In bug 373443 the server implemented PUT with arbitrary HTTP input.
At the time John and I discussed whether it might be useful to handle PUTs differently for zip files.  We now need this feature.

Basically we need to stitch together the zip detection and unzip feature in our current "import from local computer" and the "put from arbitrary HTTP" done in bug 373443.  So I would provide a target folder and URL, and if the file at the URL were not a zip, it would be copied to the target folder.  If the file is a zip, its contents are expanded into the target folder.
Comment 1 Susan McCourt CLA 2012-09-07 15:32:56 EDT
John, would you be able to get this implemented for M2?
There are two scenarios needing it.

In bug 378749, I want to let plugins specify a URL where a zip file lives, so that someone could contribute a "Create New Content" button and point to some zip file on the web containing their project template.

In bug 369968, jjb was wanting to point at a zip file and import it into Orion.

I didn't even think about what the API might be on the server side or on the fileClient side.  I suppose it's best thought of as a "URL" equivalent to the "import from local computer" API.
Comment 2 Susan McCourt CLA 2012-09-07 15:55:55 EDT
(In reply to comment #1)
> John, would you be able to get this implemented for M2?
> There are two scenarios needing it.

Just saw bug 369968 comment 2.  
thanks....
Comment 3 John Arthorne CLA 2012-09-11 15:36:08 EDT
I think this should be implemented as a POST request on /xfer/import/file/path, exactly the same as local file import. The POST body would be empty, but either a header or query parameter on the URL would refer to the source URL. Currently this servlet uses an options HTTP header to specify if you want "raw" (not unzipped). If the header is missing then it will try to unzip.
Comment 4 John Arthorne CLA 2012-09-12 16:51:38 EDT
Implementation and tests pushed to master:

http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=a45786baf2fc9d68ff6e9b84f40e360ce0641d22

The syntax is like this:

/xfer/import/file/path?source=http://example.com/some/file.zip

I'll leave this open for now so you can try it out and we can iterate if it needs changes. I also need to add some wiki documentation.
Comment 5 John Arthorne CLA 2012-09-14 16:03:41 EDT
Some documentation here:

http://wiki.eclipse.org/Orion/Server_API/Transfer_API#Import_a_file_from_a_remote_URL
Comment 6 Susan McCourt CLA 2012-09-17 15:29:20 EDT
(In reply to comment #4)
> Implementation and tests pushed to master:
> 
> http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/
> ?id=a45786baf2fc9d68ff6e9b84f40e360ce0641d22
> 
> The syntax is like this:
> 
> /xfer/import/file/path?source=http://example.com/some/file.zip
> 
> I'll leave this open for now so you can try it out and we can iterate if it
> needs changes. I also need to add some wiki documentation.

An issue I'm hitting...
I'm specifying the target location for the folder, and this works fine when importing and unzipping.  But if I set the options header to "raw" I get a failure because I didn't specify the target file name.  Talked to John, and he thinks it's reasonable to have the server side assume the same file name from the URL if not specified by the client.  

We also discussed improving the documentation a bit to specify that the "raw" option is in the X-Xfer-Options. It used the vague word "options" which could have been additional options.  You could copy the explanation and perhaps an example like shown in the upload case.
Comment 7 John Arthorne CLA 2012-09-18 10:26:40 EDT
(In reply to comment #6)
> An issue I'm hitting...
> I'm specifying the target location for the folder, and this works fine when
> importing and unzipping.  But if I set the options header to "raw" I get a
> failure because I didn't specify the target file name.  Talked to John, and
> he thinks it's reasonable to have the server side assume the same file name
> from the URL if not specified by the client.  

I have made this change. It was following the same code path as the old POST import, in which we must be supplied with a filename because the POST body is just the file contents. I also added an extra JUnit for the non-zip case to verify:

http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=7de2754342c6c23c8491981b5a87a2b1d645d16d

> We also discussed improving the documentation a bit to specify that the
> "raw" option is in the X-Xfer-Options. It used the vague word "options"
> which could have been additional options.  You could copy the explanation
> and perhaps an example like shown in the upload case.

Done: http://wiki.eclipse.org/Orion/Server_API/Transfer_API#Import_a_file_from_a_remote_URL
Comment 8 John Arthorne CLA 2012-09-18 10:27:11 EDT
Marking fixed again.