v0.13.2

Bugfix

Fix autodiff graph memory management strategy to improve performance (#1702 #1710) @louisfd
Fix matmul double broadcasting for ndarray (#1646 #1679) @lancelet

v0.13.1

Bugfix

Fix autodiff memory leak and improve performance with a new graph memory management strategy (#1698) @nathanielsimard @louisfd
Fix inplace fused operations (#1682) @nathanielsimard

Improvements

Linear 1D support, helpful for ONNX support (#1682) @nathanielsimard
Upgrade wgpu to 0.19.4 (#1692) @nathanielsimard

v0.13.0

The Burn Release 0.13 is a significant update introducing numerous new features and performance enhancements. One major change is the removal of the Sync trait implementation from most Burn types, see Core User APIs. Additionally, the release introduces several new tensor operations, module features, optimizers, as well as improvements to the autodiff backend. Notably, a new bridge mechanism facilitates runtime switching between backends, and significant work has been done on the Just-in-Time and Wgpu backends. The release also addresses numerous bug fixes, documentation improvements, infrastructure updates, CI enhancements, and miscellaneous changes to improve code quality and usability.

Core User APIs

A major change in this release is that most Burn types no longer implement the Sync trait, such as modules, optimizers, and tensors. This change should not impact users of the Learner struct for model training. However, it may affect those who implemented their own training loop and inference server. While modules, optimizers and tensors can be sent to other threads, they cannot be accessed concurrently by multiple threads. This aligns with Burn's workflow, where each tensor operation requires an owned version of the tensor. The change was made to safely reduce the number of locks needed when modifying the state of the autodiff graph, fusion state, allocation cache, and various other use cases. While not all locks have been removed, the type signature no longer poses a problem for follow-up optimizations. Note that the same tensor can still be sent to multiple threads without copying the underlying data. However it will require cloning before sending a tensor to a thread. (#1575) @nathanielsimard

Tensor

• Support signed value for Tensor::arange #1238 @Nikaidou-Shinku
• Add Tensor::unsqueeze_dims op (#1236) @skewballfox
• Add support for Any, All operations to Tensor (#1342) @ashdtu
• Add not_equal and not_equal_elem tensor ops (#1374) @laggui
• Element wise min/max between a pair of tensors (#1385) @boondocklabs
• Add is_close and all_close tensor operators (#1389) @antimora
• Interpolate tensor operation (Inference Only) (#1246) @Nikaidou-Shinku @antimora @ashdtu
• Autodiff/training support for Nearest Interpolation (#1414) @Nikaidou-Shinku @ashdtu @antimora
• Add argwhere and nonzero boolean tensor ops (#1394) @laggui
• Add bool() op for numerical tensor (#1402) @antimora
• Tensor permute operator (#1410) @antimora
• Add sign tensor operator (#1446) @antimora
• Rename diagonal to eye tensor op and add missing entry for diagonal to Book tensor section (#1449) @antimora
• Add prod and prod_dim tensor ops (#1460) @antimora
• Add tril_mask, triu_mask and diag_mask ops (#1479) @antimora
• Add flip tensor operator (#1468) @carrotflakes
• Add tensor sorting operations (#1488) (#1494) @laggui
• Add topk tensor operation (#1497) @laggui
• Tensor expand operator (#1508) @antimora
• Provide Tensor Padding Helpers (#960) (#1097) @jcmullwh @antimora
• Move log_sigmoid to activation ops (#1558) @laggui
• Add repeat autodiff and fusion support (#1600) @louisfd

Module

• Feature Addition: PRelu Module (#1328) @Arjun31415
• Implement Instance Normalization (#1321) @tushushu
• Add enum module support (#1337) @laggui
• Make the parameters of conv1d and conv2d public. (#1245) @Arjun31415
• Parameters are now lazy initialized, so you don't need to implement both the init and init_with(record) method for training/inference. (#1539) @nathanielsimard
• Support multilabel binary cross entropy (#1571)
• Implement Huber loss (#1444) @WorldSEnder
• Feat: Add Leaky Relu Model (#1467) @Arjun31415
• Feat/swiglu (#1507) @ashdtu
• Feat: transformer rotary positional encoding to transformer modules (#1604) @ashdtu

Optimizer

• Add linear learning rate scheduler (#1443) @astral4
• Exponential learning rate scheduler @1481 @rubenjr0
• Cosine Annealing learning rate scheduler with cold restarts @1481 @rubenjr0
• Add Rank0 variant to AdaptorRecordV1 and AdaptorRecordItemV1 (#1442) @carrotflakes

Train

• Add multi-label classification dataset and metric (#1572) @laggui
• Add learner training summary (#1591) @laggui

Backend

This release also introduces the backend bridge, a new mechanism for runtime switching between backends.
While an improvement, it remains compatible with previous methods of supporting mixed precision. (#1529) @nathanielsimard

JIT

Significant effort has been devoted over the past few months to refactor the previous Wgpu backend into a shader-agnostic Just-in-Time backend.
All lower-level dependencies have been abstracted into the Just-in-Time Runtime trait, requiring a compiler, compute server, and storage.
The bulk of this work was carried out by @nathanielsimard and @louisfd.

Commits: #1274 #1280 #1313 #1340 #1356 #1359 #1378 #1391 #1396 #1398 #1417 #1429 #1423 #1424 #1433 #1456 #1474 #1457 #1480 #1472 #1493 #1509 #1530 #1528 #1541 #1550 #1569

Wgpu

• Enable burn-fusion by default. (#1223) @nathanielsimard
• Feature/autotune int ops (#1136) @agelas
• Add runtime options in Wgpu init methods. (#1505) @nathanielsimard
• Decent speedup of transposed convolution @louisfd

Autodiff

Extensive work has also been undertaken on Burn's autodiff backend.
The backend now supports gradient checkpointing to reduce memory usage and has been refactored into a client/server architecture.
These updates result in significantly less blocking when tracking gradients, enhancing performance particularly on smaller models.
Furthermore, various bugs have been fixed where some graph nodes weren't used, potentially truncating the autodiff graph.
Overall, these changes make the autodiff process more reliable and efficient. (#1575) (#1358) @louisfd @nathanielsimard

Candle

• Upgrade to Candle 0.4.1. (#1382) @laggui

Data

• Add an image folder dataset implementation. (#1232) (#1132) @laggui
• Add burn::data::network::downloader. (#1283) @laggui

Import

• [PyTorchRecorder] Allow multiple pattern matches in chain. (#1269) @laggui
• [PyTorchRecorder] Pytorch config extraction (#1323) @antimora
• [PyTorchRecorder] Pass top-level key to extract state_dict (#1300) @antimora
• [PyTorchRecorder] print debug option (#1425) @antimora
• [PyTorchRecorder] Truncate debug display for NestedValue (#1428) @antimora
• [PyTorchRecorder] Support for non-contiguous indexes in PyTorchFileRecorder keys (#1432) @antimora
• [PyTorchRecorder] Add Enum module support (#1436) @antimora
• [ONNX] Parser rewrite (#1296) @skewballfox

Benchmarks

We have implemented a system that enables the comparison of backends across a variety of tasks.
Currently, most of these tasks consist of micro-benchmarks, but we plan to expand the range of benchmarks in the future.
To ensure Burn's portability and performance across different devices, the community can run and upload benchmarks! 🔥

• Created the burnbench CLI. (#1260) @syl20bnr
• Added GitHub authentication to the burnbench CLI. (#1285) @syl20bnr
• Updated GitHub App ID with the official application. (#1397) @syl20bnr
• Implemented benchmark upload functionality to the server. (#1381) @syl20bnr
• Compiled benchmarks in a dedicated target directory. (#1435) @syl20bnr
• Enhanced benchmark result presentation with a neat table and attempted to run every benchmark. (#1464) @akhildevelops
• Improved access token refreshing and displayed authenticated user name. (#1483) @syl20bnr
• Added system information to benchmark results. (#1495) @syl20bnr
• Included Operating System information in benchmark results. (#1531) @syl20bnr
• Fixed automatic fusion activation issue with Wgpu. (#1542) @syl20bnr
• Tweaked and added kinds to Gelu benchmark names. (#1533) @syl20bnr
• Ensured backend names in JSON reports match the burnbench CLI. (#1375) @errordeveloper @syl20bnr
• Added 'all' choice to --benches and --backends options. (#1567) @syl20bnr
• Revamped burnbench output for improved readability and compactness. (#1568) @syl20bnr
• Added URL to browse results on the burn.dev website. (#1573) @syl20bnr

Bug Fix

• Fix the pow backward pass when one of the tensor wasn't tracking the gradients. (#1225) (#1224) @nathanielsimard
• Fix batch norm on the LibTorch backend when the aggregation was on the same device. (#1226) @nathanielsimard
• Fix training dashboard metrics switch on Max OS & Linux (#1228) @nathanielsimard
• Fix a bug introduced in (#1138) where arithmetic could fail on usize type. (#1287) @louisfd
• [PyTorchRecorder] Fix out of memory bug (#1270) (#1286) @antimora
• [PyTorchRecorder] Fix chain pattern matching when multiple patterns are provided (#1273) @laggui
• Fix LogEventStore end epoch log (#1314) @laggui
• Huggingface dataset importer: check that pa_type is valid before checking if is_binary (#1354) @laggui
• Fix implicit casting of bool in wgpu backend (#1391) @louisfd
• Fix Switched arguments in reshape_args_usize check (#1409) @jackdarlison
• Fix tch view data corruption (#1434) @nathanielsimard
• Missing Debug derive for Group Norm Config (#1482) @Arjun31415
• Numerically stable log_sigmoid (#1548) @laggui
• Fix pytorch recorder adapt_linear when using autodiff backend (#1576) @laggui

Infrastructure

The minimum Rust version has been updated to 1.75. (#1297) @syl20bnr

Docs

• Improve the doc feature flags for docs.rs (#1212) @syl20bnr
• Include the backends in the documentation (#1229) @nathanielsimard
• Started the burn developer book. (#1184) @skewballfox @syl20bnr @antimora
• Update TORCH_CUDA_VERSION usage. (#1284) @laggui
• fix(book): add missing device parameter to mode.init(). (#1302) @apertureless
• fix(book): add missing second parameter to CrosEntropyLoss constructor (#1301) @apertureless
• docs(book-&-examples):...

v0.12.1

Bugfix

Fix wgpu performance issue: revert to wgpu 0.18.0 #1221 @nathanielsimard
Fix problem with batch norm on LibTorch backend #1226 @nathanielsimard
Fix docs build #1212 #1229 @syl20bnr @nathanielsimard
Fix training dashboard metrics switch #1228 @nathanielsimard

Chores

Put all dependencies versions in workspace #1210 @nathanielsimard

v0.12.0

This release highlights an optimized Wgpu Backend, clearer examples and documentation, and numerous bug fixes.
Notably, breaking changes in device management mandate explicit device specification to prevent potential bugs.
Additionally, the new PyTorch recorder simplifies model porting by enabling automatic import of PyTorch's weights.
We also put a lot of efforts into improving our CI infrastructure for enhanced reliability, efficiency, and scalability.

Changes

Tensor & Module API

• Added support for generic modules #1147 @nathanielsimard
• Added support for tuple modules #1186 @varonroy
• Enabled loading PyTorch .pt (weights/states) files directly to module's record, currently available on Linux & MacOS #1085 @antimora
• Added mish and softplus activation functions #1071 @pacowong
• Improved chunk performance in backends @1032 @Kelvinyu1117
• [Breaking] Added the device as an argument for tensor operations that require it, replacing the previous optional device usage #1081 #518 #1110 @kpot
  • Code update involves either using Default::default for the same behavior or specifying the desired device.
• Allowed raw tensors to be serialized/deserialized directly with serde #1041 @jmacglashan
• [Breaking] Forced the choice of the device for deserialization #1160 #1165 @nathanielsimard
• Added element-wise pow operation #1133 @skewballfox
• Refactored the tensor backend API names #1174 @skewballfox
• [Breaking] Changed the default recorder to NamedMpkFileRecorder #1161 #1151 @laggui
  • After a bit of exploration, we removed any type of compression because it adds to much overhead

Examples & Documentation

• Updated the text-classification example #1044 @nathanielsimard
• Fixed import and type redefinitions in mnist-web-inference #1100 @syl20bnr
• Fixed documentation of Tensor::stack #1105 @PonasKovas
• Fixed some typos in links in the burn-book #1127 @laggui
• Added an example for a custom CSV dataset #1129 #1082 @laggui
• Fixed missing ticks in Burn book and removed unused example dependency #1144 @laggui
• Added a new example for regression problems #1150 #1148 @ashdtu
• Added model saving and loading examples in the book #1164 #1156 @laggui
• Added Rust concept notes and explanations to the Burn Book #1169 #1155 @laggui
• Fixed jupyter notebook and ONNX IR example #1170 @unrenormalizable
• Added a custom mnist dataset, removing the Python dependency for running the guide and the mnist example #1176 #1157 @laggui
• Updated documentation and book sections on PyTorch import #1180 @antimora
• Updated burn-book with improved tensor documentation #1183 #1103 @ashdtu
• Updated burn-book with a new dataset transforms section #1183 #1154 @ashdtu
• Update CONTRIBUTING.md with code guidelines. #1134 @syl20bnr
• Fixed documentation of Multi Head Attention #1205 @ashdtu

Wgpu Backend

• Optimized the repeat operation with a new kernel #1068 @louisfd
• Improved reduce autotune by adding the stride to the autotune key #1070 @louisfd
• Refactored binary operations to use the new JIT compiler IR #1078 @nathanielsimard
• Added persistent cache for autotune #1087 @syl20bnr

Fusion

• Refactored burn-fusion, making it possible to eventually save the JIT state #1104 @nathanielsimard
• Improved fusion in the Wgpu backend with caching #1069 @nathanielsimard
• Supported fusing int operations with burn-fusion #1093 @nathanielsimard
• Supported automatic vectorization of operations fused with burn-fusion in WGPU #1123 #1111 @nathanielsimard
• Supported automatically executing in-place operations fused with burn-fusion in WGPU #1128 #1124 @nathanielsimard
• Heavily refactored burn-fusion to better reflect the stream optimization process #1135 @nathanielsimard
• Heavily refactored burn-fusion to save all execution plans for any trigger #1143 @nathanielsimard
• Supported multiple concurrent optimization streams #1149 #1117 @nathanielsimard
• Supported overlapping optimization builders #1162 @nathanielsimard
• Supported fusing ones, zeroes, and full operations #1159 @nathanielsimard
• Supported autotuning fused element-wise kernels #1188 #1112 @nathanielsimard

Infra

• Support testing accelerate(MacOS) on the burn-ndarray backend #1050 @dcvz
• Improved CI output by introducing groups #1024 @dcvz
• Updated scheduled CI tasks #1028 @Luni-4
• Added support for Windows Pipeline #925 @Luni-4
• Fixed CI for testing the wgpu backend by pinning versions #1120 @syl20bnr
• Fixed burn-compute build command with no-std #1109 @syl20bnr
• Temporarily disabled unnecessary steps on Windows runners to save CI time #1107 @syl20bnr
• Refactored serialization of backend comparison benchmarks #1131 @syl20bnr
• Fixed doc build on docs.rs #1168 @syl20bnr
• Added cargo xtask commands for dependencies and vulnerabilities checks #1181 #965 @syl20bnr
• Added cargo xtask command to manage books #1192 @syl20bnr

Chore

• Shared some properties across the cargo workspace #1039 @dcvz
• Formatted the codebase with nightly where the stable version falls short #1017 @AlexErrant
• Improved panic messages on the web @1051 @dcvz
• Used web-time in wasm #1060 @sigma-andex
• Refactored some tensor tests #1089 @nathanielsimard
• Made Embedding weights public #1094 @unrenormalizable
• Updated candle version and added support for slice_assign #1095 @louisfd
• Records no longer require Debug and Clone #1137 @nathanielsimard
• Removed cargo warning #1108 @syl20bnr
• Updated wgpu version to 0.19.0 #1166 @nathanielsimard
• Added tests for Slice assign vs Cat in LSTM backward #1146 @louisfd
• Updated xtask publish task #1189 @Luni-4
• Enable dependabot daily #1195 @Luni-4
• Updated Ratatui version #1204 @nathanielsimard
• Updated tch version #1206 @laggui

Bug Fixes

• Fixed a slice issue in the LibTorch backend that could corrupt tensors' data #1064 #1055 @nathanielsimard
• Fixed issues with tensor stack and reshape on ndarray #1053 #1058 @AuruTus
• Fixed multithread progress aggregation in dataloader #1083 #1063 @louisfd
• Resolved a numerical bug with tanh on MacOS with Wgpu #1086 #1090 @louisfd
• Fixed burn-fusion, where only operations followed by a sync were fused #1093 @nathanielsimard
• Removed the requirement for users to add serde as a dependency for Burn #1091 @nathanielsimard
• Fixed transformer prenorm on the residual path #1054 @Philonoist
• Fixed conv2d initialization by supporting fan_out #1138 @laggui
• Resolved the problem of sigmoid gradient generating NaN #1140 #1139 @wcshds
• Fixed FullPrecisionSettings type for integers #1163 @laggui
• Fixed batchnorm not working properly when training on multiple devices #1167 @wcshds
• Fixed powf function in WGPU, with new tests #1193 #1207 @skewballfox @louisfd
• Fixed regex in PyTorch Recorder #1196 @antimora

v0.11.1

Burn v0.11.1 fixes a few bugs in the recent v0.11.0

Bugfixes

Fix concurrency issue in burn-fusion, related to freeing tensors that are never read @nathanielsimard

Fix typos in the book @shanmo

Fix Readme @nathanielsimard

Fix docs build @dcvz

Thanks

Thanks to all aforementioned contributors

v0.11.0

The main feature of Burn v0.11.0 is automatic kernel fusion, which is still in active development but already usable. Many enhancement and new features have been added throughout the framework, for better efficiency and reliability.

Warnings:

• There are some breaking changes, see below.
• The organization has been renamed from burn-rs to tracel-ai.

Changes

Overall changes

• [Breaking] Refactor backend names @nathanielsimard

• [Breaking] Updated the feature flags of burn to improve usability @nathanielsimard

• Update of Burn's Readme @nathanielsimard @louisfd

Burn Fusion

• Innovative automatic kernel fusion algorithm @nathanielsimard

• Relative computation graph cache @nathanielsimard

Burn Core

• GroupNorm module @dcvz

• Allow for int and bool constant tensors in modules @nathanielsimard

• Quiet softmax in transformers @wbrickner

Burn Tensor

• New operators in tensor API: unsqueeze_dim, narrow, stack, chunk, tril, triu @dcvz

• Recip operation support on all backends @gzsombor

• Implement DoubleEndedIterator for DimIter @wcshds

Burn Compute

• Major Autotune refactor @louisfd

Burn Import

• ONNX Support for Gather @CohenAriel

• ONNX Support for Cos, Exp, Gelu, Log, Neg @antimora

• ONNX Support ConvTranspose2D @npatsakula, @antimora,

• ONNX Support for Sqrt @edmondop

• Support count_include_pad attr in avg_pool2d ONNX @antimora

Burn Train

• Add warmup consideration for estimated training time @nathanielsimard

Burn WGPU

• New Matmul kernels @louisfd

• New Reduce kernel @louisfd

• Add Autotune capabilities to Matmul and Reduce @louisfd

• Support of kernel fusion for element-wise operations @nathanielsimard @louisfd

Burn Candle

• Support conv_transpose_1d @louisfd

• Enable accelerate for MacOS CPU @dcvz

Backend Comparison

• Custom Gelu benchmarks @nathanielsimard

• Persistence of results in json @louisfd

Bugfixes

• Allow arbitrary precision threshold for float equality assertion @meteor-lsw

• Update serde_rusqlite to the new version with MIT/Apache2 license @antimora

• Fix SQLite database tests on Windows @syl20bnr

• Fix max_dim and min_dim tensor operations @gzsombor

• Fix inplace double binary broadcasting in the LibTorch backend @nathanielsimard

Documentation

• Add Python details in the Book's getting started @antimora

• Miscellaneous Book fixes @syl20bnr @mks-h

Continuous Integration

• Add test coverage @Luni-4

• Speedup typos check @Luni-4

• Dependency checks @Luni-4

• Vulnerability checks @Luni-4

Thanks

Thanks to all aforemetioned contributors.

v0.10.0

Burn v0.10.0 sees the addition of the burn-compute crate to simplify the process of creating new custom backends, a new training dashboard and the possibility of using the GPU in the browser along with a web demo. Additionally, numerous new features, bug fixes, and CI improvements have been made.

Warning: there are breaking changes, see below.

Changes

Burn Compute

• Introduction of burn-compute, a new Burn crate making it easier to create async backends with custom kernels. @nathanielsimard, @louisfd

• Add new memory management strategies @louisfd, @nathanielsimard

• Add autotune capabilities @louisfd

Burn Import

• Add more ONNX record types @antimora

• Support no-std for ONNX imported models @antimora

• Add custom file location for loading record with ONNX models @antimora

• Support importing erf operation to ONNX @AuruTus

Burn Tensor

• Add covariance and diagonal operations @ArvidHammarlund

• [Breaking] Reading operations are now async when compiling to wasm, except when wasm-sync feature is enabled. @nathanielsimard @AlexErrant

• [Breaking] Improved Clamp API @nathanielsimard

• Add unfold tensor operation @agelas, @nathanielsimard

• Improve tensor display implementation with ellipsis for large tensors: @macroexpansion

Burn Dataset

• Improved speed of SqLite Dataset @antimora

• Use gix-tempfile only when sqlite is enabled @AlexErrant

Burn Common

• Add benchmark abstraction @louisfd

• Use thread-local RNG to generate IDs @dae

Burn Autodiff

• Use AtomicU64 for node ids improving performance @dae

Burn WGPU

• Enable non-blocking reads when compiling to wasm to fully support WebGPU @nathanielsimard

• Add another faster matmul kernel @louisfd

• [Breaking] Massive refactor to use burn-compute @nathanielsimard

Burn Candle

• Candle backend is now available as a crate and updated with Candle advances @louisfd @agelas

Burn Train

• New training cli dashboard using ratatui @nathanielsimard

• [Breaking] Heavy refactor of burn-train making it more extensible and easier to work with @nathanielsimard

• Checkpoints can be customized with criteria based on collected metrics @nathanielsimard

• Add the possibility to do early stopping based on collected metrics @nathanielsimard

Examples

• Add image classifier web demo using different backends, including WebGPU, @antimora

Bugfixes

• Epoch and iteration were swapped. (#838) @daniel-vainsencher

• RNN (Gru & LSTM) were not generic over the batch size @agelas, @EddieMataEwy

• Other device adaptors in WGPU were ignored when best available device was used @chistophebiocca

Documentation

• Update book @nathanielsimard

• Doc improvements with std feature flag: @ArvidHammarlund

Chores

• Update all dependencies @antimora

• Lots and lots of CI Improvements with coverage information @Luni-4, @DrChat, @antimora, @dae, @nathanielsimard

Thanks

Thanks to all aforemetioned contributors and to our sponsors @smallstepman, @0x0177b11f and @premAI-io.

v0.9.0

Burn v0.9.0 sees the addition of the Burn Book, a new model repository, and many new operations and optimizations.

Burn Book

The Burn Book is available at https://burn-rs.github.io/book/

• Burn Book setup and plan @nathanielsimard @wdoppenberg @antimora
• Motivation & Getting started @louisfd @nathanielsimard
• Basic Workflow: from training to inference @nathanielsimard @louisfd
• Building blocks @nathanielsimard
• ONNX models @antimora
• Advanced sections @nathanielsimard

Model repository

The Model repository is available at https://github.com/burn-rs/models

• Setup @nathanielsimard
• Add SqueezeNet @antimora
• Multiple models made with Burn @Gadersd
  • Llama 2
  • Whisper
  • Stable Diffusion v1.4

Changes to Burn

Neural networks

• Three new optimizers
  • AdamW @wdoppenberg
  • AdaGrad @CohenAriel
  • RMSProp @AuruTus
• Custom initializer for transformer-related modules @wbrickner
• Cross Entropy with label smoothing and weights @ArvidHammarlund

Tensors

• Many new operators
  • cast @trfdeer @nathanielsimard
  • clamp, clamp_min, clamp_max @antimora
  • abs @mmalczak
  • max_pool1d, max_pool with dilation @caiopiccirillo
  • adaptive_avg_pool 1d and 2d @nathanielsimard
  • conv_transpose 1d and 2d, with backward @nathanielsimard
  • Not operator @louisfd
  • Dim iterator @ArvidHammarlund
• More tests for basic tensor ops @louisfd

Training

• New training metrics @Elazrod56
  • CPU temperature and use
  • GPU temperature
  • Memory use
• Custom training and validation metric loggers @nathanielsimard
• Migration from log4rs to tracing, better integration in a GUI app @dae
• Training interruption @dae
• New custom optimize method @nathanielsimard

Backends

• WGPU backend
  • Autotune @louisfd @nathanielsimard
    • Cache optimization @agelas
  • Pseudo-random number generator @louisfd
  • Fix configs @nathanielsimard
  • Matmul optimization @louisfd
• ndarray backend
  • Optimization of argmin/argmax @DrChat
  • Optimization of conv2d @DrChat
• Candle backend @louisfd
  • Support for all basic operations
  • Work in progress

Dataset

• Option for with or without replacement in dataset sampler @nathanielsimard

Import & ONNX

• Refactor, performance, tests and fixes @antimora @Luni-4 @nathanielsimard, @Gadersd
• New operators @Luni-4 @antimora @AuruTus
  • Reshape
  • Transpose
  • Binary operators
  • Concat
  • Dropout
  • Avg pool
  • Softmax
  • Conv1d, Conv2d
  • Scalar and constants
  • tanh
  • clip

Fix

• Hugging Face downloader Windows support @Macil
• Fix grad replace and autodiff backward broadcast @nathanielsimard
• Fix processed count at learning completion @dae
• Adjust some flaky tests @dae
• Ability to disable experiment logging @dae

Configuration

• Rewrite publish and checks scripts in Rust, with cargo-xtask @Luni-4 @DrChat
• Add Typos verification to checks @caiopiccirillo @antimora
• Checks for Python and venv environment @mashirooooo
• Feature flags for crates in different scenarios @dae

Documentation

• Configuration doc for vscode environment setup @caiopiccirillo
• Jupyter notebook examples @antimora
• Readme updated @louisfd

Thanks

Thanks to all aforemetioned contributors and to our sponsors @smallstepman and @premAI-io.

v0.8.0

In this release, our main focus was on creating a new backend using wgpu.
We greatly appreciate the meaningful contributions made by the community across the project.
As usual, we have expanded the number of supported operations.

Changes

Tensor

• Added Max/Minimum operation @nathanielsimard
• Added average pooling 1D operation @nathanielsimard
• Added Gather/Scatter operations @nathanielsimard
• Added Mask Where operation @nathanielsimard
• Refactor index related operations @nathanielsimard
  • index, index_assign => slice, slice_assign
  • index_select, index_select_assign => select, select_assign
• New syntax sugar for transpose @wbrickner
• Added SiLU activation function @Poxxy

Dataset

• Added a dataset using Sqlite for storage. Now used to store huggingface datasets. @antimora
• New speech command audio dataset. @antimora
• Create python virtual environment for huggingface dependencies. @dengelt

Burn-Import

• Big refactor to make it easier to support new operations. @nathanielsimard
• Support bool element type. @maekawatoshiki
• Added Add operator. @Luni-4
• Added MaxPool2d operator. @Luni-4
• Parse convolution 2D config. @Luni-4
• Added sigmoid operation. @Luni-4

Backend

• New burn-wgpu backend 🔥! @nathanielsimard @louisfd
  • Tile 2D matrix multiplication
  • All operations are supported
• Improve performance of repeat with the tch backend. @nathanielsimard

Neural Networks

• Added LSTM module. @agelas
• Added GRU module. @agelas
• Better weights initialization with added support for Xavier Glorot. @louisfd
• Added MSE loss. @bioinformatist
• Cleanup padding for convolution and pooling modules. @Luni-4
• Added sinusoidal positional embedding module. @antimora

Fix

• Deserialization of constant arrays. @nathanielsimard
• Concat backward with only one dim. @nathanielsimard
• Conv1d stride hardcoded to 1. @antimora
• Fix arange with the tch backend. @nathanielsimard

Documentation

• Improve documentation across the whole project ♥! @antimora

Thanks

Thanks to all contributors and to the sponsor @smallstepman.