GitHub - SearchSavior/OpenArc: Lightweight Inference server for OpenVINO
Welcome to OpenARC
OpenArc is a lightweight inference API backend for Optimum-Intel from Transformers to leverage hardware acceleration on Intel CPUs, GPUs and NPUs through the OpenVINO runtime using OpenCL drivers.
It has been designed with agentic use cases in mind.
OpenArc serves inference and integrates well with Transformers!
Under the hood it's a strongly typed fastAPI implementation of OVModelForCausalLM from Optimum-Intel. So, deploying inference use less of the same code, while reaping ...
Read more at github.com