Bug 368432

Comment 1 John Arthorne 2012-01-17 15:41:50 EST

I would think this is simple traditional key/value query parameters on the resource, like we do ?depth=1 on the file API.

Comment 2 Tomasz Zarna 2012-02-09 04:10:22 EST

Git resources are heterogeneous comparing to files. Each git response has different structure, so ?depth=1 won't work. What first came to my mind were two params: "expand" and "collapse". They would work like this:

Assume A and B:

=> GET /A
<= {
	aProperty : "...",
	BLocation : "/B/...",
	CLocation : "/C/..."
}

=> GET /B
<= {
	bProperty : "...",
	DLocation : "/D/..."
}

=> GET /C
<= {
	cProperty : "...",
	DLocation : "/D/..."
}

then with the "expand" param we could decide which location should be "expanded" in the response:

=> GET /A?expand=B
<= {
	aProperty : "...",
	B : {
		bProperty : "..."
		DLocation : "/D/...",
	},
	CLocation : "/C/..."
}

=> GET /A?expand=B,C
<= {
	aProperty : "...",
	B : {
		bProperty : "..."
		DLocation : "/D/...",
	},
	C : {
		cProperty : "...",
		DLocation : "/D/..."
	}
}

using "collapse" would tell the server which nodes keep "collapsed" and "expand" the rest

=> GET /A?collapse=C
<= {
	aProperty : "...",
	B : {
		bProperty : "..."
		DLocation : "/D/...",
	},
	CLocation : "/C/..."
}

This should also allow nesting like this:

=> GET /A?... <haven't thought about the URI yet>
<= {
	aProperty : "...",
	B : {
		bProperty : "..."
		D : {
			dProperty : "..."
		},
	},
	CLocation : "/C/..."
}

Comment 3 Tomasz Zarna 2012-02-15 05:51:58 EST

(In reply to comment #2)
> This should also allow nesting like this:
> 
> => GET /A?... <haven't thought about the URI yet>

Reading how GData tries to deal with this sounds like a good idea: http://code.google.com/apis/gdata/docs/2.0/reference.html#PartialResponse

Comment 4 Tomasz Zarna 2012-05-04 09:50:24 EDT

There is also an OSLC spec for specifying properties to be included in the response: http://open-services.net/bin/view/Main/OSLCCoreSpecQuery?sortcol=table;up=#oslc_select

Comment 5 Tomasz Zarna 2012-05-15 10:15:38 EDT

Updated orbit.map and server's feature with new dependency on ANTLR : http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=eb072fb4b5b903d55c05b26d80dfb20be9dc1e62. Available in builds >= I20120514-2230, any of them should be used as a target platform.

Comment 6 John Arthorne 2012-05-15 13:50:35 EDT

Did you enter a CQ for Antrl? We are long past the CQ deadline for June release but it is possible it will be accepted. Can you explain what Antlr is being used for here - it seems like overkill if it is just parsing URL arguments.

Comment 7 Tomasz Zarna 2012-05-16 04:33:12 EDT

(In reply to comment #6)
> Did you enter a CQ for Antrl? We are long past the CQ deadline for June release
> but it is possible it will be accepted. 

The CQ I found on Orbit download page [1] is CQ 4865. I assumed if the lib is in the repo it can be freely used. To I need to file a new CQ for using ANTLR in Orion? I guess if that's required I could easily piggyback on that CQ.

> Can you explain what Antlr is being used
> for here - it seems like overkill if it is just parsing URL arguments.

It is used for parsing the URL param representing selection query. It may seem like overkill at first, but:
* even now the query can have different forms from with simple comma separated lists of properties ie "A,B,C" to lists with nested properties and wildcards ie "A,B{*},C{C1,D{D1,D2}}"
* in the future we plan to add negations so excluding specific properties would be easier
* we also plan to add support for predefined, named sets of properties to be used instead of longer lists ie A,B{$name} not A,B{B1,B2,B4}

I agree that the grammar is now quite simple, but I would probably spend the same amount of time implementing my own parser for nested properties. Moreover, given the future plans adding few extra rules to the grammar should be easier than maintaining the homegrown parser. And last, but not least in the eventuality of supporting the OSLC syntax for the selection having their grammar in hand we would be halfway done.

[1] http://download.eclipse.org/tools/orbit/downloads/drops/S20120509234235/

Comment 8 Tomasz Zarna 2012-05-16 05:46:43 EDT

Server/Core changes + initial adoption of git resources from o.e.orion.server.git.objects.* pushed as http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=6d548a9cb873e04ba6baa7d6bbc6e81c6d54540c (yesterday).

Comment 9 Tomasz Zarna 2012-05-16 08:29:47 EDT

Added default resource shapes for git resources. #toJSON methods delegate now object serialization to JSONSerializer with the default resource shape. See http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=29ce754cf0af4c74a56f008910c3825220419a1e

The next step would be to handle custom selection queries and use them to serialize git objects.

Comment 10 Simon Kaegi 2012-05-16 09:05:05 EDT

Hey guys,

This looks like you're adding batch request support to the server. That's fine as I understand it might be critical for performance but I want to understand how/if that impacts the client service api. In particular I'm concerned that we will leak the details of request batching into the client-side apis. Can you describe it a little or point me at another relevant bug.

Comment 11 Tomasz Zarna 2012-05-16 09:24:01 EDT

Created attachment 215710 [details]
mylyn/context/zip

Comment 12 John Arthorne 2012-05-25 21:03:14 EDT

Although it would be nice if we could get rid of it, I have entered this CQ for Orion to use Antlr:

https://dev.eclipse.org/ipzilla/show_bug.cgi?id=6534

Comment 13 Szymon Brandys 2012-05-28 07:02:36 EDT

I talked to Simon about API being exposed by this change. It seems we agreed that each resource may have a set of predefined shapes. Each shape defines what properties/fields are part of the (JSON) response. For this we indeed do not need Antlr.

Comment 14 Tomasz Zarna 2012-05-30 05:58:43 EDT

Created attachment 216457 [details]
Patch removing the ANTLR dependency

Comment 15 Tomasz Zarna 2012-06-01 07:05:32 EDT

Created attachment 216639 [details]
Patch removing ANTLR dependency

Removed a dummy ResourceShapeDescription annotation introduced in the previous patch

Comment 16 Szymon Brandys 2012-06-01 07:09:40 EDT

(In reply to comment #15)
> Created attachment 216639 [details]
> Patch removing ANTLR dependency
> 
> Removed a dummy ResourceShapeDescription annotation introduced in the previous
> patch

Looks good. We need to open a bug for further work i.e. being able to use shapes other than the default one.

Comment 17 Tomasz Zarna 2012-06-01 07:44:37 EDT

(In reply to comment #14)
> Created attachment 216457 [details]
(In reply to comment #15)
> Created attachment 216639 [details]

Both patches applied as
http://git.eclipse.org/c/orion/org.eclipse.orion.server.git/commit/?id=4220523b2b4466ae149ef336be9a5fd4fabe5756

The ANTLR dep is gone. The code still contains a bunch of TODOs, but the work
can be continued on separate bugs like bug 381350 as Szymon suggested in the
previous comment.