Exploring ‘unpopular’ optimization method in Machine Studying
Beforehand, we leveraged gradient ascent to maximise the probability of predicting fraudulent transactions.
We’ll leverage Newton’s technique to realize the identical aim on this article.
Content material
- Newton’s technique — Mathematical theorem
- Hessian matrix
- Execs & cons in ML optimization
- Simulation — Binary logistic regression in fraud detection case
- Evaluation
Newton’s technique is a root-finding algorithm which produces approximations to the roots (or zeroes) of a real-valued perform such that:
Think about we wish to discover the x-intercept outlined a of calculus f(x).
We’ll begin with selecting a random place to begin x1 and take its tangent line to approximate the perform to get our first approximation of x-intercept: x2.
Take one other tangent line at x = x2, then we get the second approximation of x-intercept: x3 and so forth. If we repeat this course of, we’ll be capable of discover the goal intercept a as x_i will get nearer to it.
