Support Vector Machines — Lecture series — Problem with examples on the negative side of a hyperplane

In the previous post, we spoke about how to choose the best hyperplane from all of the possible hyperplanes that can be generated by the perceptron learning algorithm. We stated that the best hyperplane is the one that has the greatest distance from its data points. If this concept is still a little unclear to you, I would suggest that you revisit the previous post. In this post, I will be talking about one potential pitfall to look out for in the selection of the optimum hyperplane.

Learning objective:

Understand the potential pitfall that can come from evaluating the distance between examples (on the negative side of the hyperplane) and the hyperplane.

Main question:

We know that to help us select the optimum hyperplane, we will first have to find the distance between the example data points and the hyperplane, and then pick the smallest distance and associate that to the hyperplane. So suppose that data point x1 and x2 are both on the negative side of the hyperplane (note: ‘negative side’ is the left hand side of the hyperplane and ‘positive side’ is the right hand side of the hyperplane) and they have the following respective positions -5 and -1, which of the 2 points has the smallest distance from the hyperplane?

In the question above, you might mistakenly think that because -5 is ‘smaller than’ -1, x1 must have the smallest distance to the hyperplane. Note that the minus sign (-) in-front of these numbers just signify the position of the numbers with respect to the hyperplane and have no effect on the magnitude values of these numbers. To put it another way, the minus sign just shows that these data points are on the left hand side of the hyperplane.

As a result of the explanation above, we can now correctly come to the conclusion that x2 has the smallest distance to the hyperplane since its magnitude value is 1 and that of x1 is 5.

I hope you got that. If it sounds unclear, please read through it again and see if it makes more sense. You can always reach out for further clarification.