State-Token

[Mbucari]

@rmcrackan

I thought State-Token looked a little too small to be a real Token. It's actually a unix timestamp with milliseconds. You can specify any time you want, not just the values returned in State-Token, and Audible will deliver only the changes to your library after that time. If an item was removed from your library, item["status"] is "Revoked".

I tested this with provided_review, and unfortunately Audible does not consider a change in provided_review to be a state change, so it will not return items that only have changed user ratings. It appears the only way to detect a provided_review change is to query the all items with that response group.

[rmcrackan]

That token being a timestamp makes sense. Actually a very clever way of doing that.

For Libation, I have no problem with a full library query. It's SIGNIFICANTLY faster than it was and it supports unlimited books. Thanks for researching this.

Obviously having a nearly instant scan would be sexy. Here's the rub: my most common day-to-day use for Libation is deciding on my next book because audible's site and app are godawful for this. My workflow is to rate every book I finish and use Libation's -israted flag. I could imagine also using one of audible's flags for is-finished, but such a flag is metadata that would surely have the same state-token limitation.

[Mbucari]

What I thought might work for Libation is:

1. Do initial library scan per usual, requesting all response groups for the entire library. Store the timestamp of the last library sync.
2. When re-scanning:
   2.1 Do a scan for books added/removed since the last scan, and store the last scan timestamp
   2.2 Scan entire library with only the provided_review response group and update the database.

This definitely more complicated though. It gets even more complicated if more than 100 books have changed since the last scan, because then you'd have to make successive calls to retrieve the rest of the changes using the Continuation-Token. Or, you'd just resolve to re-scan the full library if that happened.

But if you did this, the average scan time could be significantly reduced even more, despite the worst-case scan time increasing slightly.

[rmcrackan]

I'm missing something. Doing a quick state-token scan followed by a full library scan can't be quicker than only doing a full library scan.

[Mbucari]

Doing a full library scan with only the provided_review response group. That way you're not getting all other Item info that hasn't changed or that you don't care about, such as summaries, customer_reviews, and category_ladders. the returned JSONs will be much smaller.

[rmcrackan]

I see what you mean. For users who don't create reviews, the 2.2 scan could be negligible.

I don't love the complexity. Also, there's more edge-case fun: Audible occasionally updates other stuff like cover art and categories. If they update anything else in the future, that too will have to be shoe-horned in to the 2.2/update scan. At present, supporting new updates is just a simple tweak to the BookImporter because all of the data is already available from the scan.

[Mbucari]

Yeah, I'm with you on the complexity. Just typing it out here I was thinking to myself "this sounds like a pain in the ass to debug".

[Mbucari]

#93

Sorry for posting this here. I didn't see that this was already an Issue.

[mkb79]

@Mbucari
No problem at all. So the issue will be keep clean for documentation purposes.

[mkb79]

@rmcrackan

I thought State-Token looked a little too small to be a real Token. It's actually a unix timestamp with milliseconds. You can specify any time you want, not just the values returned in State-Token, and Audible will deliver only the changes to your library after that time. If an item was removed from your library, item["status"] is "Revoked".

I tested this with provided_review, and unfortunately Audible does not consider a change in provided_review to be a state change, so it will not return items that only have changed user ratings. It appears the only way to detect a provided_review change is to query the all items with that response group.

Must the state-token be created by Audible first and send to you or can you use any timestamp you want?

[rmcrackan]

It sounds like: any timestamp will do. Which makes sense. The result set is all added or removed books between a certain datetime and now. If I were coding such a feature, my only requirement would be a beginning timestamp. My guess is that the which token audible returns to you is so you know their internal time for your current request. However, I have not personally tested this.

[Mbucari]

My tests indicate that any timestamp will work.

[mkb79]

Thank you. With these knowledge some ideas come up in me!