dtolnay / watt, Hacker News

Watt*is a runtime for executing Rust procedural macros compiled as WebAssembly.

^{*optional but entertaining pronunciation:Wahteetee}

[dependencies]watt=" (0.1)  "

Compiler support: requires rustc 1. 35

(Rationale)

• Faster compilation.By compiling macros ahead-of-time to Wasm, we save all downstream users of the macro from having to compile the macro logic or its dependencies themselves.

  Instead, what they compile is a small self-contained Wasm runtime (~ 3 seconds, shared by all macros) and a tiny proc macro shim for each macro crate to hand off Wasm bytecode into the Watt runtime (~ 0.3 seconds per proc-macro crate you depend on). This is much less than the 20 seconds it can take to compile complex procedural macros and their dependencies.

• Isolation.The Watt runtime is 100% safe code with zero dependencies. While running in this environment, a macro’sonly possible interaction with the worldis limited to consuming tokens and producing tokens. This is true regardless of how much unsafe code the macro itself might contain! Modulo bugs in the Rust compiler or standard library, it is impossible for a macro to do anything other than shuffle tokens around.

• Determinism.From a build system point of view, a macro backed by Wasm has the advantage that it can be treated as a purely deterministic function from input to output. There is no possibility of implicit dependencies, such as via the filesystem, which aren’t visible to or taken into account by the build system.

Getting started

Start by implementing and testing your proc macro as you normally would, using whatever dependencies you want (syn, quote, etc). You will end up with something that looks like:

externcrateproc_macro;useproc_macro::TokenStream;  # [proc_macro]pubFNmy_macro(input: TokenStream) ->TokenStream {     / * ... * /}

# [proc_macro_derive]and# [proc_macro_attribute]are supported as well; everything is analogous to what will be shown here for# [proc_macro].

When your macro is ready, there are just a few changes we need to make to the signature and the Cargo.toml. In your lib.rs, change each of your macro entry points to a no_mangle extern “C” function, and change the TokenStream in the signature from proc_macro to proc_macro2. Finally, add a call toproc_macro2 :: set_wasm_panic_hook ()at the top of your macro to ensure panics get printed to the console; this is optional but helpful!

It will look like:

useproc_macro2::TokenStream;  # [no_mangle]pubextern"C"fnmy_macro(input: TokenStream) ->TokenStream {     proc_macro2::set_wasm_panic_hook();      / * same as before * /}

Now in your macro’s Cargo.toml which used to contain this:

change it instead to say:

[lib]crate-type=["cdylib","rlib"]  [patch.crates-io]proc-macro2={git="https : //github.com/dtolnay/watt"}}

This crate will be the binary that we compile to Wasm. Compile it by running:

$cargo build --release --target wasm 32 - unknown-unknown

Next we need to make a small proc-macro shim crate to hand off the compiled Wasm bytes into the Watt runtime. In a new Cargo.toml, put:

[lib]proc-macro=true[dependencies]watt=" (0.1)  "

And in its src / lib.rs put:

externcrateproc_macro;useproc_macro::TokenStream;staticWASM:&[u8]=include_bytes!("my_macro.wasm");  # [proc_macro]pubfn(input: TokenStream)->TokenStream {     watt::proc_macro("my_macro", input, WASM) }

Finally, copy the compiled Wasm binary from target / wasm 32 – unknown -unknown / release / my_macro.wasm under your implementation crate, to the src directory of your shim crate, and it’s ready to publish!

Remaining work

• Performance.Watt compiles pretty fast, but so far I have not put any effort toward optimizing the runtime. That means macro expansion can potentially take longer than with a natively compiled proc macro.

  Note that the performance overhead of the Wasm environment is partially offset by the fact that our proc macros are compiled to Wasm in release mode, so Downstreamcargo buildwill be running a release-mode macro when it would have been running debug-mode for a traditional proc macro.

  There is a great deal of low-hanging fruit in the runtime; as I said, it hasn’t been optimized. In particular I am doing something silly with how strings are handed off one character at a time instead of in bulk in memory. I. would love contributions from people who have a better idea of ​​how this stuff is intended to work in Wasm in general.

  As another idea, maybe there could be some kind ofcargo install watt-runtimewhich installs an optimized Wasm runtime locally, which the Watt crate can detect and hand off code to if available. That way we avoid running things in a debug-mode runtime altogether.

• Tooling.The getting started section shows there are a lot of steps to building a macro for Watt, and a pretty hacky patching in of proc-macro2. Ideally this would all be more straightforward, including easy tooling for doing reproducible builds of the Wasm artifact for confirming that it was indeed compiled from the publicly available sources.

• RFCs.The advantages of fast compile time, isolation, and determinism may make it worthwhile to build first-class support for Wasm proc macros into rustc and Cargo. The toolchain could ship its own high performance Wasm runtime, which is an even better outcome than Watt because that runtime can be heavily optimized and consumers of macros don’t need to compile it.

This can’t be real

To assist in convincing you that this is real,here is serde_derive compiled to Wasm. It was compiled from the commitserde-rs / @ serde 217 BA (e) . Feel free to try it out as:

// [dependencies]// serde="1.0"// serde_json="1.0"// wa-serde-derive="0.1"usewa_serde_derive::Serialize;  # [derive(Serialize)]structWatt{     msg: &'static(str) , }FNmain() {     letw=Watt {msg:"hello from wasm!"};     println!("{}", serde_json::to_string(&  (w).unwrap()); }

Acknowledgments

The current underlying Wasm runtime is a fork of the Rust-WASM

project by Yoann Blein and Hugo Guiroux, a simple and spec-compliant WebAssembly interpreter.

(License)^{Everything outside of the `runtime` directory is licensed under either of}Apache License, Version 2.0orMIT licenseat your option. The `runtime` directory is licensed under theISC license.
_{Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.}