Hearth 0.4.0, Kindlings 0.3.0, scala-newtype-compat 0.1.1 and refined-compat 0.2.0 - new releases for cross-compilable macros
On Friday I've released a bunch of libraries that are based on meta programming:
What are they and what problem they solve?
Hearth is a library that was created to help you write robus macros without going insane. Why macros instread of Shapeless/Mirrors?
- faster compilation - you can get tighter feedback loop if your compiler returns result faster (because you don't treat typer as a naive Free monad)
- better runtime performance - it is much easier to generate the fast code when you control the result and not rely on cleverness of the compiler in handling intermediate structures
- better error messages - if you know how to provide good error messages to your users when you implement web services, you should be able to reuse that experience to users that call your macros
However, since juggling ASTs in even more of an ivory tower than the category theory, it helps if someone handled all the common cases with high-level utilities. Therefore Hearth.
(As a bonus, with Hearth, you can implement macros in a way that reuses code between Scala 2 and 3, so if your are still on 2.13 Hearth-based macros would not get in your way of migrating to Scala 3 eventually).
(I consider Hearth feature-complete at this point, so I am not planning the next release. It's not 1.0.0 only because I want to give some time to other people who could use it and give some feedback).
How can we tell if that would enable implementing macros in practice?
That's where Kindlings comes in. We reimplemented a lot of the existing Scala libraries (or their type class derivations) with Hearth:
- Avro (complete Avro4s reimplementation)
- Cats type class derivations (Kittens reimplementation, but with a few more types supported the last time I checked)
- Cats tagless derivations (cats-tagless derivation reimplementation)
- Circe sanely automatic derivation of codecs
- DI (dependency injection) (reimplementation of MacWire-like DI)
- DI Cats (reimplementation of MacWire-like DI but for Cats, but without assumption that F=cats.effect.IO)
- Diff (Diffx/Difficious-like reimplementation)
- Jsoniter (sanely automatic derivation of Jsoniter codecs)
- Mocking (ScalaMock-like mocking)
- Optics (MacWire-like optics)
- PureConfig and SConfig sanely-automatic derivation
- ScalaCheck sane automatic derivation
- Tapir Schema sanely-automatic derivation - uses the same config as Kindlings' Circe or Jsoniter, so less risk of mismatches!
Additionally, it adds:
- UBJson codecs with derivation
- Scala XML sanely-automatic derivation
- Scala YAML sanely-automatic derivation
- Jsoniter JSON that does NOT rely on Circe's JSON
- Tapir OpenAPI schema rendered that relies on Jsoniter without relying on Circe's JSON
As for the integrations it currently supports:
- Cats collections
- Iron
- Refined
without requiring you to import it in every file that needs it - macros can pick up the integrations automatically from the classpath!
And all of that while using the same API for both Scala 2 and Scala 3! (And supporting JVM, Scala.js and Native where possible).
What about the compat modules?
If you wrote DDD code with Scala Newtype and Refined, like Practical FP in Scala (the 2.13 version), you mighe be in trouble:
- Scala Newtype has no support for Scala 3, since Scala 3 has no support for macro annotations (at least the the scope that 2.13 supported them)
- Refined has support for Scala 3... but no support the macros to turn literals into Refined types during compilation (skipping runtime checks, where we see from the source code that predicate would be matched)
You could migrate by rewriting the code into one of dozen newtype libraries, but at this point you have to migrate at once: Scala version, newtype library and tests, which is not fun, especially if you have a few dozen repositories, that could not be easily cross-compiled during transition.
Scala Newtype compat solves the first problem by providing a compiler plugin that emulates the 2.13 macro annotations for Scala 3. You add it, and the plugin would rewrite the code on Scala 3, so that cross-compilation is possible again.
Refined compat solves the second problem, using Hearth - Scala 3 is missing eval which is like a REPL running in macros, but we can implement a weaker version of eval that treats AST as a recording of literals, and operations that we can re-play using runtime reflection. While not as powerful as eval, it's good enough for 80% of use cases.
With both of these, your Newtype/Refined-based codebase can cross-compile between Scala 2.13 and Scala 3 unblocking the migration. (And once you are migrated you can switch to some more modern library).
And Pipez?
It's more of a trivia, but if you ever missed using custom F[_] in Chimney, Pipez library is exactly about it. Actually, even more, it's about modelling transformations as Pipe[_, _], so in theory your Kleisli and ZIO should also work with it.
It came to be when I wondered if I'll be able to make Chimney cross-compilable between 2.13 and 3. Pipez spearheaded that effort of 1 codebase for 2 macro systems. Then Chimney refined it into chimney-macro-commons, so that Chimney maintainers could just work with higher-level abstraction and focus on the derivation logic. Finally, Hearth turned all of that research into a library that is not tied to any specific use case. And then Pipez was used again, to verify that Chimney can be ported to Hearth (Chimney 2.0.0 on Hearth confirmed!).
Hopefully, all of that would be enough to convince the community that macros have more potential for smooth developer experience than Shapeless/Mirrors, that writing macros can be engineered with as high standard any other Scala code, and that is not a hypothetical possibility, but something we can already have today.
(No ETA on tutorials, but I am thinking of writing some).