What if we take the sum of squared errors
$$E(w) = \sum_{j=1}^{M} e_j^2$$
What if we just take the sum of the errors
$$E(w) = \sum_{j=1}^{M} e_j$$
A Sample:
You can use the red dots on the line to manipulate the line.
When you click somewhere on the graph a point will be added. The points used to calculate the error are the point clicked and the intersection of a vertical line through the clicked point with the reference line. These points are somewhat larger in diameter and colored green. They can be manipulated by dragging them.
If you create two points at a somewhat equal distance from the referene line, but one above and the other below, you will notice that the sum of errors will be close to zero. Because the points are on opposite sides of the line their error will add-up to zero.
What if we take the sum of squared errors
$$E(w) = \sum_{j=1}^{M} e_j^2$$
What if we take the mean sum of squared errors
$$E(w) = \frac{1}{M} \sum_{j=1}^{M} e_j^2$$
A Sample:
You can use the red dots on the line to manipulate the line.
When you click somewhere on the graph a point will be added. The points used to calculate the error are the point clicked and the intersection of a vertical line through the clicked point with the reference line. These points are somewhat larger in diameter and colored green. They can be manipulated by dragging them.
When adding points at both sides of the reference line the mean error will remain more or less constant, while the pure sum will only get bigger. Image what would happen when having thousands of points.