# Lighteval

🤗 Lighteval is your all-in-one toolkit for evaluating LLMs across multiple
backends—whether it's
[transformers](https://github.com/huggingface/transformers),
[tgi](https://github.com/huggingface/text-generation-inference),
[inference providers](https://huggingface.co/docs/huggingface_hub/en/guides/inference),
[vllm](https://github.com/vllm-project/vllm), or
[nanotron](https://github.com/huggingface/nanotron)-with
ease. Dive deep into your model’s performance by saving and exploring detailed,
sample-by-sample results to debug and see how your models stack-up.

Customization at your fingertips: letting you effortlessly create [new
tasks](adding-a-custom-task) and
[metrics](adding-a-new-metric)
tailored to your needs, or browsing all our existing tasks and metrics.

Seamlessly experiment, benchmark, and store your results on the Hugging Face
Hub, S3, or locally.
