Training AI Models
Training AI Models
You now know that neural networks learn by adjusting weights and biases. But how does the full training process actually work? How do you know when a model has learned enough - or too much? In this lesson, we will walk through the complete training journey.
The Training Loop
Training an AI model follows a cycle that repeats over and over:
- Predict - Feed data through the model and get a prediction.
- Compare - Check how far the prediction is from the correct answer.
- Adjust - Update the weights to reduce the error.
- Repeat - Do it again with the next batch of data.
This loop runs thousands or even millions of times. Each repetition nudges the model slightly closer to the right answers.
Training GPT-4 reportedly cost over $100 million in computing power alone. The training loop ran across thousands of specialised chips for months.
Loss Functions: How Wrong Is the Model?
After each prediction, we need a way to measure how wrong the model was. This measurement is called the loss (or cost), and the formula that calculates it is the loss function.
- Low loss = the prediction was close to the correct answer.
- High loss = the prediction was far off.
Think of it like a dartboard. The bullseye is the correct answer. The loss is the distance from where your dart landed to the bullseye. The goal of training is to minimise that distance over time.
Common loss functions include:
- Mean Squared Error (MSE) - measures the average squared distance between predictions and actual values. Used for predicting numbers.
- Cross-Entropy Loss - measures how well predicted probabilities match the true categories. Used for classification tasks.
What does a loss function measure in AI training?
Discussion
Sign in to join the discussion