LLM APIs & Deployment QUIZ (MCQ QUESTIONS AND ANSWERS)

A way to paginate model outputs

A method for multi-page document processing

A caching strategy

An attention mechanism that manages memory like virtual memory in OS

Higher throughput always means lower latency

Optimizing for throughput (batch processing) may increase per-request latency

They are independent metrics

Lower throughput always means higher latency

A fast attention mechanism that reduces memory usage

A caching strategy

A type of model architecture

A deployment framework

To split a model across multiple devices when it's too large for one

To remove unnecessary model parts

To create multiple model versions

To compress the model

Using two different model versions

Deploying on different cloud providers

Using multiple GPUs

Maintaining two identical environments to enable zero-downtime deployment

Gradually rolling out a new version to a small percentage of users first

Deploying to all users at once

Testing internally before production

Deploying during off-peak hours

Automatically fine-tuning models

Automatically updating model weights

Dynamically adjusting computational resources based on demand

Scaling model size based on input

To train models

To store model weights

To manage, route, and secure API requests

To generate embeddings

Training a model before deployment

Pre-loading and initializing a model to reduce first-request latency

Fine-tuning on new data

Testing model performance

Guaranteed availability

Faster inference

Better model accuracy

Significantly lower cost for interruptible workloads

Serving multiple models simultaneously

Using multiple GPUs for one model

Dynamically loading multiple fine-tuned adapters on a single base model

Serving models in multiple regions

Processing batches continuously without stopping

Dynamically adding new requests to existing batches during processing

Training with continuous data streams

Batching requests over time windows

Model parallelism splits the model across devices, data parallelism splits data

They are the same thing

Model parallelism is for inference, data parallelism is for training

Data parallelism is faster than model parallelism

Guessing the next token before computation

Using a smaller model to generate draft tokens that a larger model verifies

Decoding multiple sequences simultaneously

Pre-computing possible outputs

Converting model code to executable format

Compressing model weights

Optimizing model computation graphs for specific hardware

Converting between frameworks

A security measure

Stopping requests to a failing service to prevent cascading failures

A load balancing technique

A rate limiting strategy

Faster computation and lower memory usage with minimal accuracy loss

Better accuracy than FP32

Smaller model size only

Works only on CPUs

Training with quantized data

Counting parameters during training

Training on quantitative data

Training a model to be robust to quantization during inference

A database server

A training framework

A data preprocessing tool

A platform for deploying and managing AI models at scale

Using multiple models

Serving multiple users or organizations from a single shared instance

Deploying across multiple cloud providers

Running multiple inference requests

Online serves real-time requests, offline processes batches asynchronously

Online is faster, offline is slower

Online uses GPUs, offline uses CPUs

Online is more expensive, offline is cheaper

The final layer of a neural network

A URL where an API can be accessed

A database connection string

A model checkpoint

General Processing Unit

Guided Parallel Utilization

Graphics Processing Unit

Global Performance Utility

Training multiple models simultaneously

Processing multiple requests together to improve throughput

Splitting a model across multiple GPUs

Storing multiple models in memory

Larger batches reduce latency but decrease throughput

Batch size doesn't affect latency

Larger batches always decrease both latency and throughput

Larger batches increase throughput but may increase latency

Counting the number of parameters

Increasing model size

Adding more layers to the model

Reducing model precision to lower memory and computational requirements

Removing unnecessary layers from a model

Training a smaller model to mimic a larger model's behavior

Compressing the training data

Fine-tuning on a smaller dataset

Model hallucinations

Data leakage

Model bias

Excessive requests that could overload the system

To train models faster

To store and reuse responses for repeated queries

To compress models

To prevent hallucinations

Starting a model without training data

Deploying at low temperatures

The delay when initializing a model for the first time

Testing a model before production