Limits of the picture
A gradient is local information, not a guarantee that one step reaches a minimum. At a stationary point it is zero, but the point may be a saddle or maximum. If features use different scales, the steepest direction also depends on that coordinate geometry. The Jacobian and Hessian extend the derivative picture to vector outputs and curvature.
Use a directional derivative to predict a change
At a point x, the gradient ∇f collects the partial derivatives. For a small displacement h, f(x+h) is approximately f(x)+∇f(x)·h. If h is a unit direction, the directional derivative is ∇f·h; its maximum is ‖∇f‖ in the gradient direction. For f(x,y)=x²+3y² at (1,2), the gradient is (2,12), so the steepest local increase points mostly in the y direction. The steepest decrease points in the negative-gradient direction.
This statement assumes the ordinary Euclidean notion of distance. If coordinates use different units or a different geometry, “steepest” depends on how step length is measured; feature scaling can therefore change gradient descent behaviour. Partial derivatives hold the other inputs fixed, which may not match a real process where inputs move together. A zero gradient at a differentiable point indicates no first-order change in any direction, but it can be a minimum, maximum, or saddle. Use the Hessian to examine curvature, and remember that finite steps can overshoot the local approximation.
Try this above
Move in x and y independently and watch their partial derivatives. Then compare a step along the negative gradient with a step that is too large.
For f(x,y)=x²+3y² at (1,2), a small step h=(0.01,0) predicts a change of about (2,12)·(0.01,0)=0.02. A step of the same Euclidean length in the y direction predicts about 0.12. The approximation works for sufficiently small steps; for larger steps, curvature adds error. This dot-product interpretation explains why the gradient is useful for local optimisation and why the learning rate must be controlled. It also gives a numerical check by comparing predicted and observed changes for a small displacement.
A partial derivative is not the same as the rate along every path through a point. For f(x,y)=xy at (1,1), each partial derivative is 1, but moving along the path y=x changes f=x² at rate 2 with respect to x. The gradient combines partials so a chosen direction's rate is their dot product with that direction. Specify the path or direction when interpreting change in a multivariable system.