Limit results on pattern entropy

Alon Orlitsky, Narayana P. Santhanam, Krishnamurthy Viswanathan, Junan Zhang

Research output: Contribution to journalArticlepeer-review

18 Scopus citations

Abstract

We determine the entropy rate of patterns of certain random processes including all finite-entropy stationary processes. For independent and identically distributed (i.i.d.) processes, we also bound the speed at which the per-symbol pattern entropy converges to this rate, and show that patterns satisfy an asymptotic equipartition property. To derive some of these results we upper bound the probability that the nth variable in a random process differs from all preceding ones. copy; 2006 IEEE.

Original languageEnglish (US)
Pages (from-to)2954-2964
Number of pages11
JournalIEEE Transactions on Information Theory
Volume52
Issue number7
DOIs
StatePublished - Jul 1 2006

Keywords

  • Asymptotic equipartition property
  • Entropy rate
  • Large alphabets
  • Patterns
  • Universal compression

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Fingerprint Dive into the research topics of 'Limit results on pattern entropy'. Together they form a unique fingerprint.

Cite this