SoftServe: A Scalable Quasi-Newton Method for Deep Learning
Quasi-Newton (QN) methods have long been among the most effective methods for large-scale unconstrained convex optimization. Two obstacles have limited their use in deep learning: non-convexity and enormous parameter sizes. We introduce SoftServe, a family of QN methods designed to overcome these obstacles without line searches or ad hoc curvature corrections. SoftServe derives positivedefinite…
We haven't written up this one. arXiv cs.AI has the full story — the link below goes straight to it.