What to expect from v12?

[kneth]

The team has been working on Realm JavaScript version 12 for a while. We cannot give a date for when we will release, but we would like to give you a brief introduction to what to expect. Our current plan is to release the first beta in the second half of February 2023, and have an extended period of time (likely months) to polish it before promoting it to GA. We will need your feedback on the beta releases and release candidates to validate our approach.

You might expect many breaking changes as we are bumping the major version. But actually we are planning to have as few breaking changes as possible. The reason is that the next major version is more breaking for us than you. In reality it is a complete rewrite of the SDK internals.

We plan to change our collection classes a bit. Today they derive from a common Collection class which is modeled over ReadonlyArray. It is problematic for Realm.Dictionary as there is no natural ordering. Furthermore, we are deprecating our namespaced API since we find it out of touch with modern TypeScript and JavaScript development. We are dropping support for Altas push notifications (they have been deprecated some time ago).

Other changes might come along during the development process and we will document them carefully.

The goal of the rewrite is to keep the API as it is, and change the internal implementation. To ensure that we are keeping API untouched, we are reusing all the tests we have written over the years. If all tests pass before and after the rewrite, we believe the external API is untouched.

Realm JavaScript is today a mixture of C++ and vanilla JavaScript. TypeScript definitions and API documentation have been added on the side. A good portion of the API does not touch a single line of JavaScript code, but goes directly to an implementation in C++. This makes it difficult to quickly add new functionality, as you have to decide if it can be implemented in JavaScript, C++ or a mixture of both. Moreover, you need to remember to update TypeScript definitions and API documentation. Consequently, over the years we have seen issues where either API documentation or TypeScript definitions are not consistent with the implementation.

Realm JavaScript builds on Realm Core which is composed of a storage engine, query engine and sync client connecting your client device with MongoDB Atlas. Realm Core is a C++ library, and the vast majority of Realm JavaScript’s C++ code calls into Realm Core. Another large portion of our current C++ code is interfacing with the different JavaScript engines we are supporting (currently using NAPI (Node.js and Electron) and JSI (JavaScriptCore and Hermes)).

Our rewrite will create two separated layers: i) a hand crafted SDK layer and ii) a generated binding layer. The binding layer is interfacing the JavaScript engines and Realm Core. It is generated code, and our code generator (or binding generator) will read a specification of the Realm Core API and generate C++ code and TypeScript definitions. The generated C++ code can be called from JavaScript or TypeScript.

On top of the binding layer, we implement a hand-crafted SDK layer. It is an implementation of the Realm JavaScript API as you know it. It is implemented by using classes and methods in the binding layer as building blocks. As the implementation language we have chosen to use TypeScript.

We see a number of benefits from this rewrite. First, our hypothesis is that we are able to deliver new functionality faster. We don’t have to write so much C++ boilerplate code as we have done in the past.

Second, we are implementing the SDK in TypeScript, which guarantees that the TypeScript definitions will be accurate and consistent with the implementation. If you are a TypeScript developer, this is for you. Likely your editor will guide you through integrating with Realm, and it will be possible to do static type checking and analysis before deploying your app in production. We are also moving from JSDoc to TSDoc so the API documentation will coexist with the SDK implementation. Again, it will help you and your editor in your day-to-day work, as well as eliminating the previously seen inconsistencies between the API documentation and TypeScripts definitions

Third, we are lowering the bar for you to contribute. In the past, you likely had to have a good understanding of C++ to open a pull request with either a bug fix or a new feature. Many features can now be implemented in TypeScript alone by using the building blocks found in the binding layer. We are looking forward to seeing contributions from you.

Last but not least, we hope to be able to generate more optimal code for the supported JavaScript engines. In the past we had to write C++ code which was working across multiple JavaScript engines. Our early measurements indicate that many parts of the API will be a little faster and in a few places it will be much faster.

The current plan is to come out with an early release candidate, and we hope that you will have some time to try it out and give us feedback. In the meantime, you can follow the development by checking out the bindgen branch.

[brunoducattii]

Congrats Realm Team. You can count on us from BR.

[kneth]

We have released v12.0.0-alpha.0 and v12.0.0-alpha.1 and we invite you to try it out and give us feedback. There is still a long way before the final v12.0.0 release, and we hope you will take the journey together with us.

[brunoducattii]

Where should we insert the feedback from V12 versions?

[kraenhansen]

Thanks for reaching out! This discussion would be a good place for general feedback and if you find specific issues we would appreciate if you create seperate issues with steps on how to reproduce including the version you're testing with.

[brunoducattii]

Roger that.

[RalissonMattias]

Hello guys, I have a new issue. Reported in #5666

[XHMM]

It's there a plan to provide a universal model/schema degisn so we can define schema once and use it in reactnative(js side),iOS,Android?

[kraenhansen]

Are you referring to reusing and declaring schema and reusing across Relam JS, Realm Kotlin and Realm Swift? Since these require source code interfacing with their respective platform (classes etc.) it'll unlikely ever be copy-paste between platforms.
If you're using Altas Device Sync, you can set the schema once and use the UI to download schema code for the various SDKs, as described here: https://www.mongodb.com/docs/atlas/app-services/sync/data-model/create-a-schema/#std-label-create-rom-from-schema

Alternatively you might be able to use our Realm JS SDK for iOS, Android and Node.js and reuse the schema declaring code. Do you see any issues in that approach?

[XHMM]

Thanks for your reply. Altas Device Sync looks good but it seems not suitable for offline-focused app.

Currently my app was written in react-native, realm js sdk works well for both platforms, but when I need to implement something like ios home widget, apple watch app, I have to write native code using swift/oc (becuase react-native not support these), so now I need to re-declare realm schema using swift/oc.

Defining schema using different sdk has many pitfalls and I cannot focus on my business logic: when my shchem changed, I have to change them respectively, or when sdk updated, I have to take a look at their changelog and update my code carefully to prevent it not working under different sdk.

If there is a "protocal" like what protobuf does, we define schema or writing query using unified language and it can be translated/parsed into different language code, things would be easier.

[kraenhansen]

Thanks for the explanation! I've shared this across our SDK teams and will report back if/when I have something to share 🙂

[sean-fletching]

@kneth Thank you for the information about this release. I am excited to see what the new TS experience is like; I feel the issue right now as I'm working on integrating it into my React/TS app. 🥂 Godspeed 😁