Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.

Bug 344167

Summary: Consistent server API syntax for pagination
Product: [ECD] Orion Reporter: John Arthorne <john.arthorne>
Component: ServerAssignee: Project Inbox <orion.server-inbox>
Status: RESOLVED WONTFIX QA Contact:
Severity: normal    
Priority: P3 CC: simon_kaegi, Szymon.Brandys, tomasz.zarna
Version: 0.2   
Target Milestone: ---   
Hardware: PC   
OS: Windows 7   
Whiteboard:
Bug Depends on:    
Bug Blocks: 343506    

Description John Arthorne CLA 2011-04-28 13:30:46 EDT
We need to ensure that all operations that can return many results use consistent pagination syntax. Here's one example:

http://developer.github.com/v3/#pagination
Comment 1 John Arthorne CLA 2011-05-11 09:35:01 EDT
Another link on the subject:

http://open-services.net/bin/view/Main/OslcCoreSpecification?sortcol=table;up=#Resource_Paging
Comment 2 John Arthorne CLA 2011-05-11 09:47:55 EDT
My current thoughts on this:

- The most important thing is that our JSON objects that support pages should have "NextPage" and "PreviousPage" attributes (possibly also "FirstPage" and "LastPage"). On the client side the UI would just check for the existence of those attributes, and render the corresponding buttons. The rest of the work is all on the server.

- Pagination can become really complex. For example it is intertwined with sorting. We're unlikely to fully work this out in the short term. That's ok because clients shouldn't be impacted by changes to the server API syntax for paging.

- A common simple solution is to define query attributes for page size and page number. I don't really like this because it encourages the client to manually fiddle with the URL rather than simply traverse links - not very RESTful. It's also the least likely to produce a stable paging result. For example if I am on page 80, and a bunch of items get added on earlier pages, then a format of ?page=90&items=10 might return the same results again! 

- An even simpler solution is to have a single ?start=Blah attribute. This is slightly more stable because changes to previous pages won't affect it. Overall I like this single attribute approach best as a starting point (?start=whatever). It should be easy to implement and provide fairly stable results. It could also be used to implement fully stable paging where the "start=" attribute gives a special token for a particular client which guarantees that client will get a stable page result.
Comment 3 Szymon Brandys CLA 2011-05-11 12:02:34 EDT
(In reply to comment #2)
> My current thoughts on this:
> 
> - The most important thing is that our JSON objects that support pages should
> have "NextPage" and "PreviousPage" attributes (possibly also "FirstPage" and
> "LastPage"). On the client side the UI would just check for the existence of
> those attributes, and render the corresponding buttons. The rest of the work is
> all on the server.

Like in Atom Pub

<link rel="first" />
<link rel="next" />
<link rel="last" />

I wanted to use it for Git Log page.

> - A common simple solution is to define query attributes for page size and page
> number. I don't really like this because it encourages the client to manually
> fiddle with the URL rather than simply traverse links - not very RESTful. It's
> also the least likely to produce a stable paging result. For example if I am on
> page 80, and a bunch of items get added on earlier pages, then a format of
> ?page=90&items=10 might return the same results again!

If the state on the server was changed, the fact that ?page=90&items=10 returns now what ?page=80&items=10 used to return before is ok. Moreover using page and size attributes in URIs is the same thing as using paths in the file location. For file locations like /file/[path] the client can also easily change it manually.

> - An even simpler solution is to have a single ?start=Blah attribute. This is
> slightly more stable because changes to previous pages won't affect it. Overall
> I like this single attribute approach best as a starting point
> (?start=whatever). 

So for the git log, the 'start' attribute could be the commit id and for the user list, it would be the user id? What would happen if a user is deleted and <link rel="prev"> points at this deleted user?

It's not clear for me what "a special token for a particular client which guarantees that client will get a stable page result" is. Could you explain please based on the user list example?
Comment 4 John Arthorne CLA 2011-05-11 13:15:26 EDT
(In reply to comment #3)
> It's not clear for me what "a special token for a particular client which
> guarantees that client will get a stable page result" is. Could you explain
> please based on the user list example?

Say you wanted a user list API that returned a "stable" result. The client could navigate forwards and backwards and the list would never change. The first time the client makes a request to this API, the server makes a copy of the user list and assigns it an id, say "abcd". The list with that id will never change on the server - it is a read-only copy of the list. It could return a result with "next" and "previous" links of the form ...&start=50-abcd, which means start at position 50 of the list with id "abcd".

I don't think we'll actually need this in the near future. But, imagine we had a client that wanted to do something like "iterate over each user, and give them a $50 voucher for free hosting". Such a client would want a stable user list API so they don't give someone two vouchers.
Comment 5 John Arthorne CLA 2015-05-05 15:46:32 EDT
Closing as part of a mass clean up of inactive bugs. Please reopen if this problem still occurs or is relevant to you. For more details see:


https://dev.eclipse.org/mhonarc/lists/orion-dev/msg03444.html
Comment 6 John Arthorne CLA 2015-05-05 16:00:18 EDT
Closing as part of a mass clean up of inactive bugs. Please reopen if this problem still occurs or is relevant to you. For more details see:


https://dev.eclipse.org/mhonarc/lists/orion-dev/msg03444.html