GitHub - rentruewang/bocoel: Bayesian Optimization as a Coverage Tool for Evaluating LLMs. 10 times faster and accurate evaluation with just a few lines of modular code.
☂️ BoCoEL
Bayesian Optimization as a Coverage Tool for Evaluating Large Language Models
🤔 Why BoCoEL?
Large language models are expensive and slow behemoths, and evaluating them on gigantic modern datasets only makes it worse.
If only there is a way to just select a meaningful (and small) subset of the corpus and obtain a highly accurate evaluation.....
Wait, sounds like Bayesian Optmization!
Bocoel works in the following steps:
Encode individual entry into embeddings (way cheaper / faster than...
Read more at github.com