Support Vector Machines — Lecture series — Dot product cont’d 3
In the previous post I spoke about how to geometrically compute the dot product of 2 given vectors. In this post, we would be talking about the proof for the geometric dot product formula that was given in the previous post and we would also be talking about why the result obtained from the geometric dot product is equivalent to the result obtained from the algebraic dot product.
Learning objective:
Understand the proof of the geometric dot product formula
The main question:
In the previous post, the two vectors in Fig. 1 were given and it was stated that the formula in Fig. 2 could be used to compute the geometric dot product of those vectors.
The question here is, how did we arrive at the formula in Fig. 2?
First, observe that we have a triangle ABC in Fig 1 and that we know the magnitudes (lengths) of the sides A and B but not C. Also observe that to be able to get to the direction that the vector A is pointing to, while moving through the direction of vector B, you will have to pass through the direction of vector C. In essence, the magnitude and the direction of B + the magnitude and the direction of C = the magnitude and the direction of A. Therefore, C = A — B. I hope this makes sense :)
However, there is also another formula relating C to A and B and that is the law of cosines. To find out more information about the law of cosines, please watch this video: https://www.youtube.com/watch?v=9CGY0s-uCUE
But basically all this law is saying is that:
Let’s reconcile the 2 formulas.
On the left-hand side of the law of cosines, we can further expand on C as:
If you should move around terms and cancel them, you will eventually end up with this:
Which after you divide through by 2 would give you the geometric dot product formula.
Note that the algebraic formula and the geometric formula for determining the dot product do give the same result. For instance consider the example below where we are trying to find the dot product between a vector A and itself:
The geometric approach produces:
And the algebraic approach produces the same result:
I hope you are now super comfortable with the idea of dot product. In the next post, we will be talking about the concept of linear separability.