Does a charge always move along an electric field line?

The click

A field line shows the direction of the force on a positive charge, and so the direction of its acceleration (F = qE), at each point. It does not show the direction of velocity. A particle's path follows its velocity, so a field line and a trajectory are different things.

A charge released from rest starts off along the field line, because at first its velocity builds up only in the direction of the force. But on a curved field line the line bends away, while the charge, by inertia, tends to keep going in the direction it is already moving. The force then has a component perpendicular to the velocity, so the path does bend, but in general it does not bend in step with the field line, and the charge drifts off it. How far it drifts depends on the particle's speed and on how sharply the line curves.

The charge follows a field line exactly only in special cases: when the line is straight (a uniform field, or a radial line from a point charge) and the charge starts at rest or moves along that line.

A familiar comparison: a ball thrown horizontally. Gravity's field lines point straight down, yet the ball moves along a parabola, because its velocity and acceleration point in different directions. A charged particle fired sideways into the uniform field between capacitor plates does the same: it follows a parabola and crosses field lines.