Scanner RPC Design

[mpguerra]

Let's start brainstorming some of the RPC methods that we may want to implement for the Zebra Blockchain scanner

[mpguerra]

During yesterday's Zebra team sync we came up with the following list of potential RPC methods we may want to implement:

• RPC server
  • Get Info
  • Authenticate server
• Key Management
  • Register a new viewing key
  • Delete a previously registered viewing key
• User/Account Management
  • Register a new account/user
  • Log in
  • Refresh token
• Results
  • Retrieve results for a viewing key
  • Stop/Cancel scan for key

[upbqdn]

I think I don't follow the proof of the ownership of the key then. Would such proofs prevent the scenario where users try to delete publicly known scanning keys?

[arya2]

Would such proofs prevent the scenario where users try to delete publicly known scanning keys?

I don't think a proof of knowledge for the keys would anyway, you're right - it still needs a username for authorizing access to public viewing keys.

[mpguerra]

Let's keep it simple for now and forget about public keys and usernames/passwords

[upbqdn]

I think not having support for users doesn't block anything anyway. It should be possible to add that as the last bit. Are we even sure the RPCs will need to be public?

[arya2]

Let's keep it simple for now and forget about public keys and usernames/passwords

Adding usernames may be trivial, otherwise, I agree it should be left until the next version and will exclude it.

I think not having support for users doesn't block anything anyway. It should be possible to add that as the last bit.

Yeah, I don't think it'll block anything either.

Are we even sure the RPCs will need to be public?

I don't think they need to be public for exchanges, though I am hoping some individual users will run their own Zebra nodes at home to use with their mobile wallets.

They may need to be public later if Zcash adopts detection keys.

[upbqdn]

I thought of a trivial interface that would, apart from user management, essentially consist of only two RPCs:

• scan(Vec<Key>, Option<HeightInterval>) -> Stream<(TXID, Key)>
• clear_cache_for(Vec<Key>)

The persistent storage would be transparent for clients, meaning that Zebra would actually perform the scanning only if there were no results in the storage. Otherwise, it would return cached results from the storage.

[upbqdn]

But now I'm thinking this might not work for clients that want to initiate the scanning, then disconnect from the server, and fetch the results at a later time.

[arya2]

scan(Vec, Option) -> Stream<(TXID, Key)>

But now I'm thinking this might not work for clients that want to initiate the scanning, then disconnect from the server, and fetch the results at a later time.

I think this push interface works. If the client disconnects from the server, it can pick up where it left off when it reconnects to fetch the results.

Possible Design:

• start a scanner task
  • try_send results to a bounded channel for a small buffer
  • store results in the database if the channel is closed or full
• in the connection handler with the channel receiver:
  • check the database for any cached results since the last disconnection
  • try sending results to the client
  • delete results once the client has successfully received them
  • check the channel for any new results
  • drop results once the client has successfully received them
  • drain the channel to the database once the connection is dropped

Some clients may want to avoid streaming the results for simplicity, and I'm not sure if Tor works with http/2 or http/1.1 sockets, but this should work fine in mobile environments where background tasks are unreliable, and we could add another method later that works with regular http/1.1 requests.

Lightwalletd uses streams, I think it would be okay to provide only that interface for the first version.

[upbqdn]

To make the interface more specific:

• Passing an empty vec to clear_cache_for would implicitly remove all scanning keys and corresponding results from the storage.
• I don't understand the frameworks we'll use well enough, so I don't yet know how the clients will distinguish between "scanning finished: no results for the submitted scanning keys" and "no results, but scanning didn't finish yet".

If we go with these two RPCs, I don't see any other ones we would need.

[mpguerra]

This is the latest design:

Green items are already implemented
Lilac/Light blue items are work for current iterations/milestone
Dark Blue items are future, currently unplanned work