Decorator JITs - Python as a DSL
Spend enough time looking at Python programs and packages for machine learning,
and you'll notice that the "JIT decorator" pattern is pretty popular. For
example, this JAX snippet:
import jax.numpy as jnp
import jax
@jax.jit
def add(a, b):
return jnp.add(a, b)
# Use "add" as a regular Python function
... = add(...)
Or the Triton language
for writing GPU kernels directly in Python:
import triton
import triton.language as tl
@triton.jit
def add_kernel(x_ptr,
y_ptr,
output_ptr,
n_elements,
BLOCK_SI...
Read more at eli.thegreenplace.net