LLM Evaluation & Benchmarking QUIZ (MCQ QUESTIONS AND ANSWERS)

To test hardware

To evaluate models across multiple dimensions including accuracy, fairness, and robustness

To deploy models

To train models

How relevant the retrieved context is to the question

How fast context is retrieved

The size of the context

The number of contexts retrieved

How often the model answers

How well the answer aligns with the retrieved context

The speed of answering

The length of answers

An architecture benchmark

An API challenge

A benchmark testing scientific reasoning

A retrieval challenge

To test model robustness against intentionally challenging inputs

To test model speed

To compare models

To measure training efficiency

Corrupted test data

When test data appears in training data, inflating performance

Biased datasets

Outdated benchmarks

A simple benchmark

A summarization benchmark

A code benchmark

An advanced version of GLUE with harder tasks

SQL queries

Reading comprehension and question answering

Code quality

Translation

Evaluating the model itself vs. its performance on downstream tasks

Internal vs. external models

Fast vs. slow evaluation

Automated vs. manual evaluation

Testing humans

Evaluating human-written text

Having human judges assess model outputs for quality

Comparing humans and models

A basic benchmark

A hardware benchmark

A subset of challenging tasks from BIG-Bench

A biology benchmark

Tendency to prefer responses in certain positions when comparing outputs

Bias toward certain positions in text

Bias in model architecture

Geographic bias

Energy level optimization

Ranking models based on pairwise comparisons

Error level operation

Embedding length optimization

Matching model architectures

Matching training and test distributions

Exact parameter matching

The percentage of predictions that exactly match reference answers

Structural similarity

Syntactic similarity

How similar two texts are in meaning, not just words

Length similarity

An evaluation metric using BERT embeddings to measure semantic similarity

A score for BERT models only

The number of BERT parameters

BERT's accuracy

To remove the model

To understand the contribution of individual components by removing them

To reduce model size

To test deployment

A test of common sense reasoning through pronoun disambiguation

A code challenge

A translation task

A mathematical problem set

Based on data source

Whether they require ground truth references for comparison

Based on model type

Based on computation method

Counting total errors

Measuring error rate

Removing errors

Systematically examining and categorizing model mistakes

A machine translation benchmark

A mathematics test

A memory test

A multi-turn conversation benchmark evaluated by GPT-4

A metric measuring how well a model predicts a sample

The model's confusion level

The number of parameters

The training time

Worse performance

Faster inference

Smaller model size

Better performance in predicting text

To compress models

To quantify and compare model performance

To train models faster

To deploy models

Evaluation with zero errors

Evaluation without test data

Testing a model on tasks without specific training or examples

Testing with zero parameters

Testing with a small number of examples provided in the prompt

Evaluation with few parameters

Short evaluation time

Testing on few datasets

Model size

Inference speed

Training efficiency

Quality of machine translation by comparing to reference translations

Model color coding

Quality of text summarization

Code generation

Image quality

The speed of classification

The proportion of correct predictions

The model's confidence level

The number of classes

Commonsense reasoning about everyday situations

Mathematical reasoning

Code generation

Language translation