AI Training Glossary

Testing AI systems by deliberately trying to make them fail or behave badly. The goal is to find vulnerabilities before real users do.

Labeling or tagging data to help AI learn. Examples: drawing boxes around objects in images, categorizing text, or transcribing audio.

One of the most common entry-level AI tasks.

The process of making AI systems behave in ways that match human values and intentions. RLHF is a key alignment technique.

A rectangle drawn around an object in an image to identify and locate it. Common in computer vision annotation tasks.

Example: Drawing boxes around cars in street photos to train self-driving AI.

A technique where you show AI how to reason step-by-step instead of jumping to an answer. Helps models explain their thinking.

Example: Instead of just saying "42", show: "First we multiply 6×7=42, so the answer is 42."

Sorting data into categories. Examples: labeling emails as spam/not spam, or categorizing customer support tickets by topic.

How much text an AI can "remember" at once. Like short-term memory. Older models had small windows (a few paragraphs), newer ones can handle entire books.

The process of labeling raw data (images, text, audio) so AI can learn from it. Most entry-level AI training work falls under this category.

Someone with specialized knowledge in a specific field (medicine, law, engineering, etc.) who trains AI in that domain.

These roles pay the most because the expertise is rare and valuable.

An unusual or rare scenario that the AI might not handle well. Finding and documenting edge cases helps improve model robustness.

Example: "What happens if someone asks the AI for medical advice in emojis?"

Assessing AI performance by rating outputs, comparing responses, or testing for accuracy. A core part of RLHF work.

Teaching AI by showing it just a few examples. "Here are 3 examples of good customer emails — now write one yourself."

Taking a pre-trained AI model and training it further on specific data to specialize it for a particular task or domain.

Example: Fine-tuning a general language model on legal documents to make a legal AI.

An ideal, high-quality example of what the AI should have said. Think of it as the "gold standard" answer that the model learns to emulate.

Writing golden responses is a key task in RLHF training.

Connecting AI responses to real sources or facts instead of just generating plausible-sounding text. Helps reduce hallucinations.

When AI confidently generates false or made-up information. The model "hallucinates" facts, citations, or details that don't exist.

Example: Citing a non-existent research paper or inventing fake statistics.

A major focus of AI safety work.

AI systems that include humans in the decision-making process, either for training, validation, or quality control.

That's you — you're the human in the loop!

An AI's ability to understand and follow user instructions accurately. RLHF training heavily focuses on improving this.

Example: "Write a haiku about cats" → AI actually writes a haiku, not an essay about cats.

Tagging data with descriptive information. Similar to annotation but often refers to simpler categorization tasks.

Example: Marking images as "cat" or "dog".

AI models trained on massive amounts of text data to understand and generate human-like language. ChatGPT, Claude, and Gemini are all LLMs.

The AI system itself — the software trained on data to perform tasks. When people say "the model", they mean the AI you're training or evaluating.

The input or instruction you give to an AI. The question, command, or text that the model responds to.

Example: "Explain quantum physics in simple terms" is a prompt.

The art and science of crafting effective prompts to get better outputs from AI. A growing specialty in AI work.

Skills: Experimentation, creativity, understanding model behavior.

Ordering AI responses from best to worst. A common RLHF task where you compare multiple outputs and say which is better.

The AI's ability to think through problems logically and explain its thought process. A key focus of advanced AI training.

Deliberately trying to break AI systems by finding edge cases, safety failures, or ways to bypass restrictions. Named after military "red team" exercises.

Example tasks: Finding prompts that make the AI generate harmful content, leak training data, or behave unpredictably.

The primary method for training AI to be helpful, harmless, and honest. You rate AI outputs, and the model learns from your preferences.

How it works: AI generates responses → Humans rate them → Model learns what "good" looks like → Repeat.

This is the core of most AI training jobs.

Making sure AI doesn't generate harmful, biased, or dangerous content. A major focus of red teaming and evaluation work.

Example safety issues: Generating medical misinformation, creating malware, or producing hate speech.

Determining the emotional tone of text (positive, negative, neutral). A common annotation task.

Example: Labeling customer reviews as happy, angry, or neutral.

A setting that controls how creative or random AI responses are. Low temperature = predictable. High temperature = creative/chaotic.

You won't adjust this yourself, but you might see it mentioned in guidelines.

The information used to teach AI. Your annotations, golden responses, and ratings become training data.

High-quality training data = better AI. That's why your work matters.

When AI performs a task without any examples, just based on the instruction. "Translate this to French" with no translation examples.

Modern LLMs are surprisingly good at zero-shot tasks.