ChatGPT clone in 30 minutes on AWS Kubernetes - Cluster.dev - Medium
In this blog post I will demonstrate how employing Cluster.dev can streamline launching one of the Hugging Face LLMs with chat on AWS cloud, on top of a Kubernetes cluster, and make it production-ready.Hugging Face TGI and Chat-UIIn addition to models, datasets, and Python libraries, Hugging Face also provides Docker containers for local inference, including projects like Text Generation Inference (a Docker container to serve models) and Chat-UI (a Docker image for interactive chatting with mode...
Read more at medium.com