A scalar is just a single number — like your age, the temperature outside, or your bank balance. A vector is a number with a direction — like 'walk 5 km north'. That direction part is what makes vectors the language of machine learning.
Vectors in machine learning
Every data point in ML is a vector. A house with 3 features (size, rooms, price) is a vector with 3 numbers: [1400, 3, 250000]. An image of 28×28 pixels is a vector of 784 numbers. A word in an embedding model is a vector of 300 numbers.
Vector addition & scaling
Add two vectors by adding their components. Scale a vector by multiplying every component by the same number.
Addition: [1, 2] + [3, 1] = [4, 3]
Scaling: 2 × [1, 2] = [2, 4]
General: α·v = [α·v₁, α·v₂, α·v₃]Component-wise operations — the foundation of linear algebra
Vectors everywhere in ML
- Each training example is a vector of feature values: house = [1400 sqft, 3 beds, 2 baths, 15 yrs]
- An image (28×28 pixels) flattened into a vector of 784 numbers
- A word in Word2Vec is a 300-dimensional vector where meaning = position in space
- Neural network weights form vectors (and matrices) — everything learned is stored as numbers in vectors
- Distance between two data points = Euclidean distance between their vectors