Support Vector Machines — Lecture series — Sequential Minimal Optimization Part 2
In the previous post, I gave a general overview on what the Sequential Minimal Optimization algorithm was and how it works. In the next subsequent posts, we would be breaking down the concepts regarding the SMO algorithm into manageable chunks that you can easily assimilate, understand and enjoy. In this post, we would be stating the two components of the SMO algorithm.
Learning objective:
Knowing the 2 components of the SMO algorithm.
Main question:
We have a general sense of how the SMO algorithm works, but what are the 2 main components of the algorithm that power how it accomplishes its objectives?
The SMO algorithm is split up into two main tasks, the first task involves choosing which Lagrange multipliers to optimize and the second task involves implementing an analytic method to solve the two chosen Lagrange multipliers.
Note that the two Lagrange multipliers that are chosen must fulfil all of the constraints of the SVM optimization problem. In fulfilling these constraints, the inequality constraints cause the selected Lagrange multipliers to lie within a ‘box’, as demonstrated in Fig. 1, and the linear equality constraints cause the Lagrange multipliers to lie on a diagonal line, as also demonstrated in Fig. 1.
In the next post, we would be looking at the first aspect of the SMO algorithm which involves choosing which Lagrange multipliers to optimize.