v0.16.0

Added

• Spark 3.5 support #759 (@chris-twiner)

Changed

• Update mouse to 1.2.2 #776 (@typelevel-steward)
• Update cats-effect to 3.5.2 #760 (@typelevel-steward)
• Update cats-mtl to 1.4.0 #766 (@typelevel-steward)

v0.15.0

Changed

• Update spark-core, spark-mllib, spark-sql to 3.4.1 #731 (@typelevel-steward)
• Update cats-effect to 3.5.1 #732 (@typelevel-steward)
• Update refined to 0.11.0 #730 (@typelevel-steward)
• Update scala-library to 2.13.11 #725 (@typelevel-steward)
• Update scala-library to 2.12.18 #718 (@pomadchin)
• Make Literals foldable, ensure Parquet predicates pushdown #721 (@chris-twiner)

v0.14.1

This release adds Spark 3.4 support and makes it DBR 12.2 compatible, lots of thanks to @chris-twiner for doing this incredible work 🎉

With this release we drop Spark 3.1 artifacts publish, this is according to the original plan to maintain artifacts across the three latest Spark minor versions.

Added

• Spark 3.4.0 and DBR 12.2 support #699 (@chris-twiner)
• Add TypedEncoder for Set #710 (@cchantep)

Changed

• Update scala-library to 2.12.17 #659 (@typelevel-steward)
• Update cats-effect to 3.5.0 #713 (@typelevel-steward)
• Update cats-effect to 3.4.11 #711 (@typelevel-steward)
• Update cats-mtl to 1.3.1 #708 (@typelevel-steward)
• Update cats-effect to 3.4.10 #707 (@typelevel-steward)
• Update spark-core, spark-mllib, spark-sql to 3.2.4 #702 (@typelevel-steward)

v0.14.0

Changed

• Update refined to 0.10.3 #697 (@typelevel-steward)
• Add utility asJoinColValue to allow join on Value class dataset #695 (@cchantep)
• Refactor optional column operations #696 (@cchantep)
• Update spark-core, spark-mllib, spark-sql to 3.3.2 #690 (@typelevel-steward)
• Update cats-effect to 3.4.8 #691 (@typelevel-steward)
• Update cats-effect to 3.4.5 #685 (@typelevel-steward)
• Update cats-effect to 3.4.4 #680 (@typelevel-steward)
• Update spark-core, spark-mllib, spark-sql to 3.2.3 #677 (@typelevel-steward)
• Update cats-effect to 3.4.2 #678 (@typelevel-steward)
• Update cats-effect to 3.4.1 #676 (@typelevel-steward)
• Update alleycats-core, cats-core to 2.9.0 #673 (@typelevel-steward)
• Update shapeless to 2.3.10 #655 (@typelevel-steward)
• Update scala-library to 2.13.10 #668 (@typelevel-steward)
• Update spark-core, spark-mllib, spark-sql to 3.3.1 #670 (@typelevel-steward)
• Update scalatest to 3.2.13 #653 (@typelevel-steward)

v0.13.0

Note: this release is with a dropped Spark 3.0.x support.

Changed

• Update spark-core, spark-mllib, spark-sql to 3.3.0 #637 (@typelevel-steward)
• Update cats-effect to 3.3.13 #643 (@typelevel-steward)
• Update refined to 0.10.1 #644 (@typelevel-steward)
• Update cats-mtl to 1.3.0 #642 (@typelevel-steward)
• Update alleycats-core, cats-core to 2.8.0 #639 (@typelevel-steward)

v0.12.0

Added

• Support optional Value class as literal #623 (@cchantep)
• Implement java.time.{Duration, Instant, Period} type encoders #581 (@jgoday)
• Allow Frameless to explode Map[A,B] #488 (@ayoub-benali)

Changed

• Update cats ecosystem to CE3 #616 (@custommonkey, @Daenyth, @pomadchin)
• Update alleycats-core, cats-core to 2.7.0 #583 (@scala-steward)
• Update spark-core, spark-mllib, spark-sql to 3.1.3 #613 (@scala-steward)
• Update spark-core, spark-mllib, spark-sql to 3.2.1 #601 (@scala-steward)
• Update spark-core, spark-mllib, spark-sql to 3.0.3 #599 (@scala-steward)
• Resolve a possible explodeMap column names collision #582 (@pomadchin)

v0.11.1

Note: this release is the same as v0.11.0 but with a fixed Spark 3.1.x and Spark 3.0.x artifacts publish.

Fixed

• Fix Spark cross publish #580 (@pomadchin)

v0.11.0

Note: this release has a broken Spark 3.1.x and Spark 3.0.x artifacts publish. We fixed it in v0.11.1.

Added

• Establish Spark 3.x & Scala 2.13 cross publish #573 (@pomadchin)
• Support refined types #571 (@cchantep)
• Dotted syntax (via lambda) for TypedColumns #449 (@timvw)
• Get nested field for struct column #545 (@cchantep)

Changed

• Encode Value class field using the inner type #546 (@cchantep)

Fixed

• Map encoder with Value class #567 (@cchantep)

v0.10.1

• Docs: Better explanation on how to work with single column TypedDatasets
• Fixes a bug with the asCo() method
• Better support for optional columns (fixed issue #204)
• Upgrades to Spark 3.1.0
• Adds the “writeStream()” menthod

v0.9.0

• Spark 3.0 support