News Score: Score the News, Sort the News, Rewrite the Headlines

GitHub - SearchSavior/OpenArc: Lightweight Inference server for OpenVINO

Welcome to OpenARC OpenArc is a lightweight inference API backend for Optimum-Intel from Transformers to leverage hardware acceleration on Intel CPUs, GPUs and NPUs through the OpenVINO runtime using OpenCL drivers. It has been designed with agentic use cases in mind. OpenArc serves inference and integrates well with Transformers! Under the hood it's a strongly typed fastAPI implementation of OVModelForCausalLM from Optimum-Intel. So, deploying inference use less of the same code, while reaping ...

Read more at github.com

© News Score  score the news, sort the news, rewrite the headlines