Fast inference engine | Nitro
Embeddable AIA fast, lightweight 3mb inference server to supercharge apps with local AI.curl -sfL https://raw.githubusercontent.com/janhq/nitro/main/install.sh | sudo /bin/bash -CopyOpenAI-CompatibleNitro is a drop-in replacement for OpenAI's REST API NitroPOSThttp://localhost:3928/v1/chat/completionsCopycurl http://localhost:3928/v1/chat/completions
-H "Content-Type: application/json"
-d '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant."
},
...
Read more at nitro.jan.ai
© News Score score the news, sort the news, rewrite the headlines