Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON serializability to new specs #1035

Open
j0rdanit0 opened this issue Nov 3, 2021 · 2 comments
Open

Add JSON serializability to new specs #1035

j0rdanit0 opened this issue Nov 3, 2021 · 2 comments
Labels
area/core Related to core module: events, entities, clients, specs enhancement Improvement over an existing feature help wanted Contributors welcome!
Milestone

Comments

@j0rdanit0
Copy link
Sponsor Contributor

Feature Description:
Currently, there is no supported way to serialize the new specs into JSON and deserialize them back from JSON.

Example use case: (Embed pagination)

  • A bot produces too much embed data for a single message and must divide content across multiple pages.
  • The embed content is all calculated up front in the form of multiple EmbedCreateSpec instances and stored in a persistence layer.
  • Buttons provide the user the ability to switch between the various pages of the content by querying the needed spec and editing the message with it.
  • Since the specs are deserialized from JSON, they do not have to be rebuilt every time.

Justification:
Besides supporting the above use case, Legacy specs already have this functionality and it should be ported over to the new specs.

  • Serialization is achieved via LegacySpec#asRequest
  • Deserialization is achieved via LegacyXSpec#from(XData) methods.

The new specs could implement this same behavior, or something else entirely. However, it would be nice to add this functionality before Legacy specs are removed.

@NovaFox161 NovaFox161 added enhancement Improvement over an existing feature area/core Related to core module: events, entities, clients, specs labels Nov 3, 2021
@NovaFox161 NovaFox161 added this to the 3.2.1 milestone Nov 3, 2021
@quanticc quanticc modified the milestones: 3.2.1, 3.2.2 Nov 11, 2021
@quanticc quanticc added the help wanted Contributors welcome! label Jan 23, 2022
@quanticc quanticc modified the milestones: 3.2.2, 3.2.x Backlog Jan 23, 2022
@quanticc quanticc modified the milestones: 3.2.x Backlog, 3.2.5 Jun 11, 2023
@quanticc
Copy link
Member

Looking into this after quite a while 😓 . Most specs could have their XSpecGenerator annotated with:

@JsonSerialize(as = XSpec.class)
@JsonDeserialize(as = XSpec.class)

Some considerations must be taken regarding EmbedCreateFields and GuildCreateFields classes that must be serializable too.

MessageCreateFields like file and fileSpoiler will be ignored as we cannot properly serialize those. This means some limitations are expected - those methods will be annotated with @JsonIgnore

@quanticc quanticc modified the milestones: 3.2.5, 3.2.6 Jun 14, 2023
@JasonTheKitten
Copy link
Contributor

JasonTheKitten commented Jun 29, 2023

EDIT: Stupid me, I just realized that discord4j's json module already has a custom serializer/deserializer for possibles. I wrote my own for nothing.

I was looking at this for a bit

I used ObjectMapper to convert between classes and JSON. ObjectMapper is part of json-databind, which is visible to discord4j's json module, but not to the core module that all of the specs are located in right now.

Anyways, I ran into, like, two problems:

  1. Jackson seems to pick up on a few methods that it is not supposed to. I fixed this by disabling AUTO_DETECT_IS_GETTERS.
  2. Discord4J's Possible is a bit weird. It uses generics, and Java has type erasure that makes doing magic with generics a bit harder. I ended up writing a custom serializer/deserializer that works by keeping track of the class name of the wrapped object if it is wrapping an object

I did do a bit of modification to a local copy of D4J, doing what @quanticc suggested with the JsonSerialize, JsonDeserialize, and JsonIgnore annotations. So the following code likely would not work with an actual current release of D4J.

The generated JSON is insanely long. Probably not practical

{"content":{"present":true,"cls":"java.lang.String","value":"Hey"},"tts":{"present":false},"allowedMentions":{"present":false},"components":{"present":true,"cls":"java.util.Arrays$ArrayList","value":[{"data":{"type":1,"value":{"present":false},"values":{"present":false},"url":{"present":false},"options":{"present":false},"min_length":{"present":false},"max_length":{"present":false},"label":{"present":false},"style":{"present":false},"components":{"present":true,"cls":"java.util.ArrayList","value":[{"type":2,"value":{"present":false},"values":{"present":false},"url":{"present":false},"options":{"present":false},"min_length":{"present":false},"max_length":{"present":false},"label":{"present":true,"cls":"java.lang.String","value":"Click me!"},"style":{"present":true,"cls":"java.lang.Integer","value":3},"components":{"present":false},"disabled":{"present":false},"emoji":{"present":false},"custom_id":{"present":true,"cls":"java.lang.String","value":"hey"},"channel_types":{"present":false},"required":{"present":false},"max_values":{"present":false},"min_values":{"present":false},"placeholder":{"present":false}}]},"disabled":{"present":false},"emoji":{"present":false},"custom_id":{"present":false},"channel_types":{"present":false},"required":{"present":false},"max_values":{"present":false},"min_values":{"present":false},"placeholder":{"present":false}},"children":[{"data":{"type":2,"value":{"present":false},"values":{"present":false},"url":{"present":false},"options":{"present":false},"min_length":{"present":false},"max_length":{"present":false},"label":{"present":true,"cls":"java.lang.String","value":"Click me!"},"style":{"present":true,"cls":"java.lang.Integer","value":3},"components":{"present":false},"disabled":{"present":false},"emoji":{"present":false},"custom_id":{"present":true,"cls":"java.lang.String","value":"hey"},"channel_types":{"present":false},"required":{"present":false},"max_values":{"present":false},"min_values":{"present":false},"placeholder":{"present":false}},"url":{"empty":true,"present":false},"label":{"empty":false,"present":true},"style":"SUCCESS","emoji":{"empty":true,"present":false},"customId":{"empty":false,"present":true},"type":"BUTTON"}],"type":"ACTION_ROW"}]},"messageReference":{"present":false},"embeds":{"present":false},"nonce":{"present":false}}

And then you can convert it back into the spec

MessageCreateSpec{content=Possible{Hey}, nonce=Possible.absent, tts=Possible.absent, embeds=null, files=[], fileSpoilers=[], allowedMentions=Possible.absent, messageReference=Possible.absent, components=[{data={type=1, value={present=false}, values={present=false}, url={present=false}, options={present=false}, min_length={present=false}, max_length={present=false}, label={present=false}, style={present=false}, components={present=true, cls=java.util.ArrayList, value=[{type=2, value={present=false}, values={present=false}, url={present=false}, options={present=false}, min_length={present=false}, max_length={present=false}, label={present=true, cls=java.lang.String, value=Click me!}, style={present=true, cls=java.lang.Integer, value=3}, components={present=false}, disabled={present=false}, emoji={present=false}, custom_id={present=true, cls=java.lang.String, value=hey}, channel_types={present=false}, required={present=false}, max_values={present=false}, min_values={present=false}, placeholder={present=false}}]}, disabled={present=false}, emoji={present=false}, custom_id={present=false}, channel_types={present=false}, required={present=false}, max_values={present=false}, min_values={present=false}, placeholder={present=false}}, children=[{data={type=2, value={present=false}, values={present=false}, url={present=false}, options={present=false}, min_length={present=false}, max_length={present=false}, label={present=true, cls=java.lang.String, value=Click me!}, style={present=true, cls=java.lang.Integer, value=3}, components={present=false}, disabled={present=false}, emoji={present=false}, custom_id={present=true, cls=java.lang.String, value=hey}, channel_types={present=false}, required={present=false}, max_values={present=false}, min_values={present=false}, placeholder={present=false}}, url={empty=true, present=false}, label={empty=false, present=true}, style=SUCCESS, emoji={empty=true, present=false}, customId={empty=false, present=true}, type=BUTTON}], type=ACTION_ROW}]}

Overall, I ended up with something like this

public class Main {

    public static void main(String[] args) {
        // I just copied this from the D4J docs
        MessageCreateSpec spec = MessageCreateSpec.create()
                .withContent("Hey")
                .withComponents(ActionRow.of(Button.success("hey", "Click me!")));

        // The actual interesting part
        ObjectMapper objectMapper = JsonMapper.builder()
            .disable(MapperFeature.AUTO_DETECT_IS_GETTERS)
            .disable(SerializationFeature.FAIL_ON_EMPTY_BEANS)
            .build();

        SimpleModule module = new SimpleModule();
        module.addSerializer(Possible.class, new PossibleSerializer());
        module.addDeserializer(Possible.class, new PossibleDeserializer());
        objectMapper.registerModule(module);

        String serialized = null;
        try {
            serialized = objectMapper.writeValueAsString(spec);
            System.out.println(serialized);
            System.out.println(objectMapper.readValue(serialized, MessageCreateSpec.class));
        } catch (JsonProcessingException e) {
            throw new RuntimeException(e);
        }
    }

}
// "Possible" has to be kept as a rawtype. The compiler will complain if you do "Possible<?>"
public class PossibleDeserializer extends JsonDeserializer<Possible> {
    @Override
    public Possible deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException, JacksonException {

        JsonNode node = jsonParser.getCodec().readTree(jsonParser);
        boolean present = node.get("present").asBoolean();
        if (present) {
            JsonNode value = node.get("value");
            String className = node.get("cls").asText();
            try {
                Class<?> cls = Class.forName(className);
                return Possible.of(jsonParser.getCodec().treeToValue(value, cls));
            } catch (ClassNotFoundException e) {
                throw new RuntimeException(e);
            }
        } else {
            return Possible.absent();
        }
    }
}
public class PossibleSerializer extends JsonSerializer<Possible> {

    @Override
    public void serialize(Possible possible, JsonGenerator jsonGenerator, SerializerProvider serializerProvider) throws IOException {
        jsonGenerator.writeStartObject();
        jsonGenerator.writeBooleanField("present", !possible.isAbsent());
        if (!possible.isAbsent()) {
            jsonGenerator.writeObjectField("cls", possible.get().getClass().getName());
            jsonGenerator.writeObjectField("value", possible.get());
        }
        jsonGenerator.writeEndObject();
    }

}

Right now I just have the code in it's own project because it was easier to test that way.

@quanticc quanticc modified the milestones: 3.2.6, 3.2.7 Sep 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/core Related to core module: events, entities, clients, specs enhancement Improvement over an existing feature help wanted Contributors welcome!
Projects
None yet
Development

No branches or pull requests

4 participants