TY - GEN
T1 - Using curvature information for fast stochastic search
AU - Orr, Genevieve B.
AU - Leen, Todd K.
N1 - Copyright:
Copyright 2014 Elsevier B.V., All rights reserved.
PY - 1997
Y1 - 1997
N2 - We present an algorithm for fast stochastic gradient descent that uses a nonlinear adaptive momentum scheme to optimize the late time convergence rate. The algorithm makes effective use of curvature information, requires only O(n) storage and computation, and delivers convergence rates close to the theoretical optimum. We demonstrate the technique on linear and large nonlinear back-prop networks.
AB - We present an algorithm for fast stochastic gradient descent that uses a nonlinear adaptive momentum scheme to optimize the late time convergence rate. The algorithm makes effective use of curvature information, requires only O(n) storage and computation, and delivers convergence rates close to the theoretical optimum. We demonstrate the technique on linear and large nonlinear back-prop networks.
UR - http://www.scopus.com/inward/record.url?scp=84898987060&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84898987060&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84898987060
SN - 0262100657
SN - 9780262100656
T3 - Advances in Neural Information Processing Systems
SP - 606
EP - 612
BT - Advances in Neural Information Processing Systems 9 - Proceedings of the 1996 Conference, NIPS 1996
PB - Neural information processing systems foundation
T2 - 10th Annual Conference on Neural Information Processing Systems, NIPS 1996
Y2 - 2 December 1996 through 5 December 1996
ER -