Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: ijl/orjson
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 3.9.10
Choose a base ref
...
head repository: ijl/orjson
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 3.9.11
Choose a head ref
  • 6 commits
  • 58 files changed
  • 1 contributor

Commits on Jan 18, 2024

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    ef675bf View commit details
  2. ahash compile-time-rng

    ijl committed Jan 18, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    5ae9b58 View commit details
  3. Escape str using SIMD

    ijl committed Jan 18, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    5205258 View commit details
  4. Simplify with SerializerState

    ijl committed Jan 18, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    a40f58b View commit details
  5. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    34507e4 View commit details
  6. 3.9.11

    ijl committed Jan 18, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    6ac06d9 View commit details
Showing with 2,449 additions and 2,293 deletions.
  1. +8 −9 .github/workflows/debug.yaml
  2. +1 −1 .github/workflows/lint.yaml
  3. +28 −20 .github/workflows/linux.yaml
  4. +8 −0 CHANGELOG.md
  5. +88 −56 Cargo.lock
  6. +5 −8 Cargo.toml
  7. +5 −5 README.md
  8. +3 −6 build.rs
  9. +6 −5 ci/azure-macos.yml
  10. +1 −1 ci/azure-pipelines.yml
  11. +1 −1 ci/azure-win.yml
  12. +0 −7 ci/config.toml
  13. +2 −2 integration/wsgi.py
  14. +7 −6 pyproject.toml
  15. +0 −3 requirements.txt
  16. +3 −3 script/develop
  17. +2 −2 script/graph
  18. +1 −3 script/lint
  19. +1 −3 script/pynumpy
  20. +4 −5 src/deserialize/cache.rs
  21. +1 −1 src/deserialize/deserializer.rs
  22. +1 −1 src/deserialize/json.rs
  23. +25 −24 src/deserialize/pyobject.rs
  24. +2 −2 src/deserialize/utf8.rs
  25. +3 −6 src/deserialize/yyjson.rs
  26. +0 −44 src/ffi/list.rs
  27. +0 −2 src/ffi/mod.rs
  28. +7 −9 src/lib.rs
  29. +1 −1 src/opt.rs
  30. +0 −1,330 src/serialize/json.rs
  31. +2 −1 src/serialize/mod.rs
  32. +108 −0 src/serialize/obtype.rs
  33. +42 −107 src/serialize/per_type/dataclass.rs
  34. +8 −4 src/serialize/per_type/datetime.rs
  35. +18 −38 src/serialize/per_type/default.rs
  36. +116 −142 src/serialize/per_type/dict.rs
  37. +1 −1 src/serialize/per_type/fragment.rs
  38. +1 −1 src/serialize/per_type/int.rs
  39. +102 −35 src/serialize/per_type/list.rs
  40. +2 −3 src/serialize/per_type/mod.rs
  41. +2 −2 src/serialize/per_type/none.rs
  42. +56 −68 src/serialize/per_type/numpy.rs
  43. +12 −35 src/serialize/per_type/pyenum.rs
  44. +0 −59 src/serialize/per_type/tuple.rs
  45. +2 −2 src/serialize/per_type/unicode.rs
  46. +4 −2 src/serialize/per_type/uuid.rs
  47. +46 −179 src/serialize/serializer.rs
  48. +59 −0 src/serialize/state.rs
  49. 0 src/serialize/{writer.rs → writer/byteswriter.rs}
  50. +232 −0 src/serialize/writer/escape.rs
  51. +451 −0 src/serialize/writer/formatter.rs
  52. +673 −0 src/serialize/writer/json.rs
  53. +12 −0 src/serialize/writer/mod.rs
  54. +149 −0 src/serialize/writer/simd.rs
  55. +23 −39 src/typeref.rs
  56. +28 −0 src/util.rs
  57. +5 −0 test/test_indent.py
  58. +81 −9 test/test_type.py
17 changes: 8 additions & 9 deletions .github/workflows/debug.yaml
Original file line number Diff line number Diff line change
@@ -7,24 +7,24 @@ jobs:
strategy:
fail-fast: false
matrix:
rust: [
{ version: "1.65" }, # MSRV
{ version: "nightly-2023-10-10" },
profile: [
{ rust: "1.65", features: "" },
{ rust: "1.65", features: "--features=yyjson" },
{ rust: "nightly-2024-01-17", features: "--features=yyjson,unstable-simd" },
]
python: [
{ version: '3.12', abi: 'cp312-cp312' },
{ version: '3.11', abi: 'cp311-cp311' },
{ version: '3.8', abi: 'cp38-cp38' },
]
env:
CC: "gcc"
CFLAGS: "-O2 -fno-plt"
CFLAGS: "-O2"
LDFLAGS: "-Wl,--as-needed"
CARGO_UNSTABLE_SPARSE_REGISTRY: "true"
steps:
- run: curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain ${{ matrix.rust.version }} --profile minimal -y
- run: curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain ${{ matrix.profile.rust }} --profile minimal -y

- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: '${{ matrix.python.version }}'
- run: python -m pip install --user --upgrade pip "maturin>=1,<2" wheel
@@ -36,9 +36,8 @@ jobs:
PATH="$HOME/.cargo/bin:$PATH" maturin build --release \
--out=dist \
--profile=dev \
--features=yyjson \
--interpreter python${{ matrix.python.version }} \
--target=x86_64-unknown-linux-gnu
--target=x86_64-unknown-linux-gnu ${{ matrix.profile.features }}
- run: python -m pip install --user dist/orjson*.whl
- run: python -m pip install --user -r test/requirements.txt -r integration/requirements.txt
2 changes: 1 addition & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@ jobs:
lint:
runs-on: ubuntu-22.04
steps:
- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- run: curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain=stable --profile=default -y
48 changes: 28 additions & 20 deletions .github/workflows/linux.yaml
Original file line number Diff line number Diff line change
@@ -62,17 +62,17 @@ jobs:
env:
PATH: /github/home/.local/bin:/github/home/.cargo/bin:/opt/python/${{ matrix.python.abi }}/bin:/opt/rh/gcc-toolset-12/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
CC: "clang"
CFLAGS: "-O2 -fno-plt -flto=thin"
LDFLAGS: "-O2 -flto=thin -fuse-ld=lld -Wl,--as-needed"
RUSTFLAGS: "-C linker=clang -C link-arg=-fuse-ld=lld"
CFLAGS: "-O2 -fstrict-aliasing -emit-llvm -flto=full"
LDFLAGS: "-O2 -fstrict-aliasing -emit-llvm -flto=full -fuse-ld=lld -Wl,--as-needed -Wl,-plugin-opt=also-emit-llvm"
RUSTFLAGS: "-C linker=clang -C linker-plugin-lto -C lto=fat -C link-arg=-fuse-ld=lld -Z mir-opt-level=4 -Z virtual-function-elimination -D warnings"
CARGO_UNSTABLE_SPARSE_REGISTRY: "true"
container:
image: quay.io/pypa/manylinux_2_28_x86_64:latest
options: --user 0
steps:
- run: yum install -y clang lld
- run: curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly-2023-10-10 --profile minimal -y
- run: rustup component add rust-src --toolchain nightly-2023-10-10-x86_64-unknown-linux-gnu
- run: curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain nightly-2024-01-17 --profile minimal -y
- run: rustup component add rust-src --toolchain nightly-2024-01-17-x86_64-unknown-linux-gnu
- uses: actions/checkout@v4

- name: build-std
@@ -84,7 +84,7 @@ jobs:
- run: |
maturin build --release --strip \
--out=dist \
--features=no-panic,yyjson \
--features=encoding_rs/simd-accel,no-panic,unstable-simd,yyjson \
--compatibility manylinux_2_17 \
--interpreter python${{ matrix.python.version }} \
--target=x86_64-unknown-linux-gnu
@@ -138,22 +138,22 @@ jobs:
uses: PyO3/maturin-action@v1
env:
CC: "gcc"
CFLAGS: "-O2 -fno-plt"
CFLAGS: "-O2"
LDFLAGS: "-O2 -flto -Wl,--as-needed"
RUSTFLAGS: "-C target-feature=-crt-static"
RUSTFLAGS: "-C target-feature=-crt-static -Z mir-opt-level=4"
CARGO_UNSTABLE_SPARSE_REGISTRY: "true"
with:
rust-toolchain: nightly-2023-10-10
rust-toolchain: nightly-2024-01-17
rustup-components: rust-src
target: ${{ matrix.platform.target }}
manylinux: musllinux_1_1
args: --release --strip --out=dist --features=no-panic,yyjson -i python${{ matrix.python.version }}
args: --release --strip --out=dist --features=encoding_rs/simd-accel,no-panic,unstable-simd,yyjson -i python${{ matrix.python.version }}

- name: Set up QEMU
if: matrix.platform.arch != 'x86_64'
uses: docker/setup-qemu-action@v3
with:
image: tonistiigi/binfmt:qemu-v7.0.0
image: tonistiigi/binfmt:qemu-v8.1.4
platforms: ${{ matrix.platform.platform }}

- name: Test
@@ -194,28 +194,35 @@ jobs:
target: [
{
arch: 'aarch64',
cflags: '-O2 -flto',
features: 'encoding_rs/simd-accel,no-panic,unstable-simd,yyjson',
rustflags: '-Z mir-opt-level=4 -D warnings',
target: 'aarch64-unknown-linux-gnu',
cflags: '-O2',
},
{
arch: 'armv7',
cflags: '-Os -flto -fstrict-aliasing',
features: 'no-panic,yyjson', # no SIMD
rustflags: '-C opt-level=s -Z mir-opt-level=4 -D warnings',
target: 'armv7-unknown-linux-gnueabihf',
cflags: '-Os -fstrict-aliasing',
},
{
arch: 'ppc64le',
cflags: '-O2 -flto',
features: 'no-panic,unstable-simd,yyjson',
rustflags: '-Z mir-opt-level=4 -D warnings',
target: 'powerpc64le-unknown-linux-gnu',
cflags: '-O2',
},
{
arch: 's390x',
cflags: '-O2 -flto -march=z10',
features: 'no-panic,unstable-simd,yyjson',
rustflags: '-Z mir-opt-level=4 -C target-cpu=z10 -D warnings',
target: 's390x-unknown-linux-gnu',
cflags: '-O2 -march=z10',
},
]
steps:
- uses: actions/checkout@v4

- name: build-std
run: |
mkdir .cargo
@@ -226,14 +233,15 @@ jobs:
env:
PYO3_CROSS_LIB_DIR: "/opt/python/${{ matrix.python.abi }}"
CFLAGS: "${{ matrix.target.cflags }}"
LDFLAGS: "${{ matrix.target.cflags }} -flto -Wl,--as-needed"
LDFLAGS: "${{ matrix.target.cflags }} -Wl,--as-needed"
CARGO_UNSTABLE_SPARSE_REGISTRY: "true"
RUSTFLAGS: "${{ matrix.target.rustflags }}"
with:
target: ${{ matrix.target.target }}
rust-toolchain: nightly-2023-10-10
rust-toolchain: nightly-2024-01-17
rustup-components: rust-src
manylinux: auto
args: --release --strip --out=dist --features=no-panic,yyjson -i python${{ matrix.python.version }}
args: --release --strip --out=dist --features=${{ matrix.target.features }} -i python${{ matrix.python.version }}

- uses: uraimo/run-on-arch-action@v2
name: Test
@@ -277,7 +285,7 @@ jobs:
- uses: actions/download-artifact@v3
with:
name: wheels
- uses: actions/setup-python@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: pip install pip "maturin>=1,<2"
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
# Changelog


# 3.9.11 - 2024-01-18

### Changed

- Improve performance of serializing. `str` is significantly faster. Documents
using `dict`, `list`, and `tuple` are somewhat faster.


## 3.9.10 - 2023-10-26

### Fixed
Loading