Support Vector Machines — Lecture series — The update rule part 1
In the previous post, we spoke about the perceptron learning algorithm and how it can be used to discriminate between a given set of linearly separable data points. In this post, we will be talking a little bit more about the update rule that is within the perceptron learning algorithm.
Learning objective:
The main objective here is to gain understanding of the update rule within the perceptron learning algorithm and how it works.
Main question:
Suppose that in the process of implementing the perceptron learning algorithm, some data point x that is supposed to be classified as a -1 is misclassified as a +1, how will you correctly classify this data point through the use of the update rule?
Well, to answer this question, let’s take a quick look at the equation of the hyperplane that is doing the classification process, shall we?
where both w and x are vectors
In the equation of the hyperplane, you can see that we are taking the dot product of 2 vectors w and x. In one of our previous posts, I mentioned that the equation representing the dot product of two vectors can be represented as the product of the magnitudes of these two vectors divided by the cosine of the angle between the vectors.
Therefore, the dot product of w and x (i.e w.x) can be represented as the product of the magnitudes of the vectors w and x divided by the cosine of the angle between the vectors of w and x.
Now, from trigonometry, we know the cosine of any angle smaller than 90 degrees is a positive value and the cosine of any angle greater than 90 degrees is a negative value. So if we have a value that was classified as +1, then we know that the cosine of the angle between w and x, used in the computation of the dot product is smaller than 90 degrees. If this value was misclassified and we will like to change its value to its correct classification of -1, we will have to increase the angle between the vectors w and x to a value which is larger than 90 degrees.
In the next post we will complete our overview of the update rule.