Evaluation metrics are the quantifiable measures used to assess how well a language model or other AI system performs on a given task, giving developers an objective way to compare models or track improvement over time rather than relying on subjective impressions alone. Common metrics include perplexity, which measures how well a model predicts a sample of text, accuracy, for classification-style tasks, F1 score, which balances precision and recall, and BLEU score, commonly used for translation quality.
No single metric captures everything that matters about a model's real-world usefulness, which is why teams typically track several metrics together and increasingly supplement automated scores with human evaluation, especially for open-ended tasks like conversational quality, helpfulness, and factual accuracy, where a purely numeric metric can miss important nuances a human reviewer would catch immediately.
