Stochastic Manhattan learning: Time-evolution operator for the ensemble dynamics

Todd K. Leen, John E. Moody

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Typical theoretical descriptions of the ensemble dynamics of stochastic learning algorithms rely on a truncated expansion to approximate the time-evolution operator appearing in the master equation. In this paper we give an exact expression for the time-evolution operator for Manhattan learning, a variant of stochastic gradient-descent learning in which the weights are updated in proportion to the sign of the cost function gradient. This closed form for the time evolution captures the full nonlinearity of the problem without approximation, allowing exact study of the ensemble dynamics.

Original languageEnglish (US)
Pages (from-to)1262-1265
Number of pages4
JournalPhysical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics
Volume56
Issue number1 SUPPL. B
StatePublished - Jul 1997

Fingerprint

Time Operator
Evolution Operator
learning
Ensemble
Stochastic Gradient
operators
Gradient Descent
Stochastic Algorithms
Master Equation
Cost Function
Learning Algorithm
gradients
Closed-form
Proportion
descent
Nonlinearity
Gradient
proportion
Approximation
nonlinearity

ASJC Scopus subject areas

  • Mathematical Physics
  • Physics and Astronomy(all)
  • Condensed Matter Physics
  • Statistical and Nonlinear Physics

Cite this

Stochastic Manhattan learning : Time-evolution operator for the ensemble dynamics. / Leen, Todd K.; Moody, John E.

In: Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics, Vol. 56, No. 1 SUPPL. B, 07.1997, p. 1262-1265.

Research output: Contribution to journalArticle

@article{d738817d7de1475c80b3b7f58c7e63af,
title = "Stochastic Manhattan learning: Time-evolution operator for the ensemble dynamics",
abstract = "Typical theoretical descriptions of the ensemble dynamics of stochastic learning algorithms rely on a truncated expansion to approximate the time-evolution operator appearing in the master equation. In this paper we give an exact expression for the time-evolution operator for Manhattan learning, a variant of stochastic gradient-descent learning in which the weights are updated in proportion to the sign of the cost function gradient. This closed form for the time evolution captures the full nonlinearity of the problem without approximation, allowing exact study of the ensemble dynamics.",
author = "Leen, {Todd K.} and Moody, {John E.}",
year = "1997",
month = "7",
language = "English (US)",
volume = "56",
pages = "1262--1265",
journal = "Physical Review E",
issn = "2470-0045",
publisher = "American Physical Society",
number = "1 SUPPL. B",

}

TY - JOUR

T1 - Stochastic Manhattan learning

T2 - Time-evolution operator for the ensemble dynamics

AU - Leen, Todd K.

AU - Moody, John E.

PY - 1997/7

Y1 - 1997/7

N2 - Typical theoretical descriptions of the ensemble dynamics of stochastic learning algorithms rely on a truncated expansion to approximate the time-evolution operator appearing in the master equation. In this paper we give an exact expression for the time-evolution operator for Manhattan learning, a variant of stochastic gradient-descent learning in which the weights are updated in proportion to the sign of the cost function gradient. This closed form for the time evolution captures the full nonlinearity of the problem without approximation, allowing exact study of the ensemble dynamics.

AB - Typical theoretical descriptions of the ensemble dynamics of stochastic learning algorithms rely on a truncated expansion to approximate the time-evolution operator appearing in the master equation. In this paper we give an exact expression for the time-evolution operator for Manhattan learning, a variant of stochastic gradient-descent learning in which the weights are updated in proportion to the sign of the cost function gradient. This closed form for the time evolution captures the full nonlinearity of the problem without approximation, allowing exact study of the ensemble dynamics.

UR - http://www.scopus.com/inward/record.url?scp=0031188949&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0031188949&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:0031188949

VL - 56

SP - 1262

EP - 1265

JO - Physical Review E

JF - Physical Review E

SN - 2470-0045

IS - 1 SUPPL. B

ER -