Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for RedisJSON #657

Merged
merged 29 commits into from Aug 25, 2022
Merged

Add support for RedisJSON #657

merged 29 commits into from Aug 25, 2022

Conversation

d3rpp
Copy link
Contributor

@d3rpp d3rpp commented Aug 2, 2022

Part 2 of #650

Changes over previous editions:

To take on board some of the feedback in #650 (specifically the result variant), I needed to make a few changes

Custom implementer macro:

This was done because the redisjson functions in implementation now return results that get unwrapped before the query, following the following structure

  • Did Serialise Correctly?
    • YES: Go on with Query -> Return Result of Query
    • NO: Return Result::Err with ErrorKind::Serialize (only active with json feature) and a description given by the serde_json error messages

This change will only affect some RedisJson Commands, the core commands are untouched by this

There will likely be debate over the removed call to ::std::mem::replace; however, this is only because since the items are now wrapped in Result's, the value is limited to the scope of the initial function, and when I try to use the main macro rustc complains that the value doesn't live long enough.

This also called for the introduction of the arg_take function on Cmd, which added a function that takes the value by value and returns the value (instead of &mut Cmd from arg.

Tests

It has all the same tests as the previous one except with an extra one to test that the serde_json errors are being handled correctly (at the top of test_json.rs); this attempts to serialise one of the few cases where serde_json fails to serialise a struct. (When a HashMap does NOT have a string-ifiable key (Strings and Numbers only)


I thought I added more but upon review I did not

d3rpp added 12 commits July 29, 2022 13:53
added extra newline

Replaced json_mget with condition in json_get

removed call to std::mem::replace in the macro, it wasn't doing anything and all tests still passs

added newlines

Add a Discord badge to README

hid internal function from docs

formatted
added extra newline

Replaced json_mget with condition in json_get

removed call to std::mem::replace in the macro, it wasn't doing anything and all tests still passs

added newlines

Add a Discord badge to README

hid internal function from docs

formatted

Ran cargo fmt

Add a Discord badge to README
Add a Discord badge to README
Add a Discord badge to README
Json support to local main
merging from redis-rs/redis-rs#main into d3rpp/redis-rs-tokio-drift#main
redis/src/cmd.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@djc djc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an initial round of review, this is looking much better.

README.md Outdated Show resolved Hide resolved
redis/src/commands/json.rs Outdated Show resolved Hide resolved
redis/src/commands/json.rs Outdated Show resolved Hide resolved
redis/src/commands/json.rs Outdated Show resolved Hide resolved
- Updated README to use correct module import
- Normalized spelling of Serialization
- implemented `From<serde_json::Error> for RedisError` to make implementation of some commands, cleaner.
- updated command names to be more readable (json_strappend -> json_str_append, etc.)
@d3rpp d3rpp requested a review from djc August 2, 2022 21:48
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
redis/tests/support/mod.rs Outdated Show resolved Hide resolved
redis/src/commands/json.rs Outdated Show resolved Hide resolved
redis/src/commands/json.rs Outdated Show resolved Hide resolved
redis/src/commands/json.rs Outdated Show resolved Hide resolved
redis/src/commands/json.rs Outdated Show resolved Hide resolved
redis/src/commands/json.rs Outdated Show resolved Hide resolved
redis/src/commands/json.rs Outdated Show resolved Hide resolved
redis/src/cmd.rs Outdated Show resolved Hide resolved
d3rpp and others added 10 commits August 5, 2022 07:05
Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
removing extra "s" due to me misreading the redis docs

Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
updated comment

TODO: RE-ADD START + STOP

Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
updated comments for `json_arr_insert`

Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
@d3rpp
Copy link
Contributor Author

d3rpp commented Aug 4, 2022

lol whoops

@d3rpp
Copy link
Contributor Author

d3rpp commented Aug 4, 2022

Still todo: add start and stop to the json_arr_index function

I know there is a trait somewhere where you can take T or Option::<T>::None but I cannot remember what is was

edit: it's Into<Option<T>>, I'll make it an isize since it can only be a number anyway
edit 2: it makes the macro panic, I'll just move it to a second function which takes them instead.

@d3rpp
Copy link
Contributor Author

d3rpp commented Aug 4, 2022

I will admit it is not the worlds cleanest solution but its cleaner than

json_arr_index("key", "path", &json!({"hello": "world"}), None, None)

// or

json_arr_index("key", "path", &json!({"hello": "world"}), 0, 0)

@d3rpp d3rpp requested a review from djc August 10, 2022 03:17
Copy link
Contributor

@djc djc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember, why did we want to have the JSON commands in separate traits (JsonCommands, AsyncJsonCommands) anyway?


/// Append the JSON `value` to the array at `path` after the last element in it.
fn json_arr_append<K: ToRedisArgs, P: ToRedisArgs, V: Serialize>(key: K, path: P, value: &'a V) {
Ok::<Cmd, RedisError>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you rewrite it like this:

let mut cmd = cmd("JSON.ARRAPPEND");
cmd.arg(key)
    .arg(path)
    .arg(serde_json::to_string(value)?);
Ok::<_, RedisError>(cmd)

I think you no longer need arg_take()? (Also note that we don't need to specify the Cmd type in the Ok turbofish.)

Also while looking this I noticed all the indentation in this file is tabs instead of spaces, should fix that up (VS Code has a Convert Indentation to Spaces thing.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll do some investigation into this, I think I tried something like this, but I may be wrong

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it for one command and it seemed to work? But maybe I got it wrong...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried it for one command and it seemed to work? But maybe I got it wrong...

sorry, I've been busy with a hackathon over the weekend, will look into this asap

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No hurry from my side! Hope you had fun at the hackathon.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you rewrite it like this:

let mut cmd = cmd("JSON.ARRAPPEND");
cmd.arg(key)
    .arg(path)
    .arg(serde_json::to_string(value)?);
Ok::<_, RedisError>(cmd)

I think you no longer need arg_take()? (Also note that we don't need to specify the Cmd type in the Ok turbofish.)

Also while looking this I noticed all the indentation in this file is tabs instead of spaces, should fix that up (VS Code has a Convert Indentation to Spaces thing.)

let mut cmd = cmd("JSON.ARRAPPEND");

cmd.arg(key)
	.arg(path)
	.arg(serde_json::to_string(value)?);

Ok::<_, RedisError>(cmd)

Does indeed seem to work, ill update the rest of them

I've run the command to use spaces instead of tabs

@d3rpp
Copy link
Contributor Author

d3rpp commented Aug 18, 2022

I don't remember, why did we want to have the JSON commands in separate traits (JsonCommands, AsyncJsonCommands) anyway?

If we don't it's 2 Separate traits with the same name so If you import both of them the compiler complains unless you alias one of them.

@djc
Copy link
Contributor

djc commented Aug 18, 2022

I meant, why can't we make these available as part of the normal command traits (but subject to the Cargo feature flag)?

@d3rpp
Copy link
Contributor Author

d3rpp commented Aug 18, 2022

I meant, why can't we make these available as part of the normal command traits (but subject to the Cargo feature flag)?

That's what I did in the First PR (#644)

Screen Shot 2022-08-19 at 10 16 04

which lead to the creation of separate modules, and to import them at the same time, it needs a different name.

@djc
Copy link
Contributor

djc commented Aug 22, 2022

Okay, let's keep the additional trait for now.

removed `arg_take`
Copy link
Contributor

@djc djc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some spit and polish, but I think this is almost ready to merge!

redis/src/commands/json.rs Outdated Show resolved Hide resolved
redis/src/commands/json_macro.rs Outdated Show resolved Hide resolved
redis/tests/support/mod.rs Outdated Show resolved Hide resolved
Moved json macro declaration to json.rs

Renamed RedisJson and RedisModule to Json and Module respectfully
Copy link
Contributor

@djc djc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, nice work! @jaymell did you want to take another look at this?

@jaymell
Copy link
Contributor

jaymell commented Aug 25, 2022

It looks good to me, though @d3rpp I'm going to request that if possible you squash this down to one commit and give it a nice commit message to save me a bit of work merging 😁 (and of course resolve conflicts)

@d3rpp
Copy link
Contributor Author

d3rpp commented Aug 25, 2022

It looks good to me, though @d3rpp I'm going to request that if possible you squash this down to one commit and give it a nice commit message to save me a bit of work merging 😁 (and of course resolve conflicts)

oooooh last time I tried that it was absolutely painful, is there not a squash checkbox next to the merge button?

@d3rpp
Copy link
Contributor Author

d3rpp commented Aug 25, 2022

image

umm, idk
is there a squash button in the drop-down next to the merge button?

this actually makes 0 sense to me

also, the repo is up-to-date, so there shouldn't be any conflicts

@djc djc merged commit be2e60e into redis-rs:main Aug 25, 2022
@djc
Copy link
Contributor

djc commented Aug 25, 2022

Yeah, I just squashed it at merge. @d3rpp thanks for persisting in getting this merged!

nonirosenfeldredis pushed a commit to nonirosenfeldredis/redis-rs that referenced this pull request Feb 13, 2023
Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
nonirosenfeldredis pushed a commit to nonirosenfeldredis/redis-rs that referenced this pull request Feb 26, 2023
Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
nonirosenfeldredis pushed a commit to nonirosenfeldredis/redis-rs that referenced this pull request Feb 26, 2023
Co-authored-by: Dirkjan Ochtman <dirkjan@ochtman.nl>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants