GitHub - trymirai/uzu: A high-performance inference engine for AI models
uzu
A high-performance inference engine for AI models on Apple Silicon. Key features:
Simple, high-level API
Hybrid architecture, where layers can be computed as GPU kernels or via MPSGraph (a low-level API beneath CoreML with ANE access)
Unified model configurations, making it easy to add support for new models
Traceable computations to ensure correctness against the source-of-truth implementation
Utilizes unified memory on Apple devices
Quick Start
First, add the uzu dependency to your Cargo.t...
Read more at github.com