Optimal asymptotic learning rate: Macroscopic versus microscopic dynamics

Todd K. Leen, Bernhard Schottky, David Saad

Research output: Contribution to journalArticlepeer-review

6 Scopus citations

Abstract

We investigate the asymptotic dynamics of on-line learning for neural networks, and provide an exact solution to the network dynamics at late times under various annealing schedules. The dynamics is solved using two different frameworks: the master equation and order parameter dynamics, which concentrate on microscopic and macroscopic parameters, respectively. The two approaches provide complementary descriptions of the dynamics. Optimal annealing rates and the corresponding prefactors are derived for soft committee machine networks with hidden layers of arbitrary size.

Original languageEnglish (US)
Pages (from-to)985-991
Number of pages7
JournalPhysical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics
Volume59
Issue number1
DOIs
StatePublished - 1999

ASJC Scopus subject areas

  • Statistical and Nonlinear Physics
  • Statistics and Probability
  • Condensed Matter Physics

Fingerprint

Dive into the research topics of 'Optimal asymptotic learning rate: Macroscopic versus microscopic dynamics'. Together they form a unique fingerprint.

Cite this