cursor, order desc support for getpage

[guileen]

Tiedot is amazing, I'd like to use it in my projects, but it missing some key feature for my situation.

1. Think about a list view, show recently posts there. I need all the posts order by insert time desc. So I need getpage can order reverse.

2. When I view the first page on mobile phone, it show recent 20 document. And before I scrolling to next page, someone post a new document. The 2nd page of 20 document will contains 1 duplicate document with 1st page. Generally I solved this page issue in other db is pass id as cursor in every page. I pass the last id of previous page as current page cursor parameter, so current page knows where to start.

   /getpage?cursor=lastid&total=20&reverse=1

   If reverse, the 1st page cursor should be something like infinity or -1. if not reverse the 1st page cursor should be 0.

3. I notice the result of getpage is {id:doc, id:doc}, it make JSON parse difficult, and lost the sequence. I think [doc, doc], and id contains in doc, would be better.

[guileen]

I notice that I misunderstanding total parameter in getpage, it means total pages, not total items per page.

I think limit will be a good parameter for my requirement,

[HouzuoGuo]

Thank you very much for your feedback and I am really sorry that those features are currently missing. Please note that GetPage API returns documents in a more or less unpredictable order, therefore the latest documents do not always come last. tiedot is rather good at indexing discrete attribute values, but I am afraid that due to lack of a suitable index implementation, tiedot is not suited for sorting data.

I have been studying to make a cache friendly skip list, but it seems rather challenging.

[guileen]

Yes, sorting on persistent data is difficult, I can give some experience of using other NoSQL, but I don't know how is it works.

Leveldb and HBase can iterate by key order, and the key is client specified when insert, this simple feature make a big flexibility.

Think about the situation I mentioned, the post keys is like

post:000000001
post:000000002
post:...
post:000999999
post:001000000

I can just iterate keys start from post:00100000 end to post:00000000 limit by 20, will get latest 20 post, next page I start from cursor post:00999980 to post:00000000 limit by 20, will get 2nd page, and so on.

I maybe affect the core data structure of tiedot, just FYI.

[HouzuoGuo]

Thanks very much for your input, I'll keep it in mind and wait till the day I manage to implement a more efficient index.