Limit results on pattern entropy

Alon Orlitsky; Narayana P. Santhanam; Krishnamurthy Viswanathan; Junan Zhang

doi:10.1109/TIT.2006.876351

Limit results on pattern entropy

Alon Orlitsky, Narayana P. Santhanam, Krishnamurthy Viswanathan, Junan Zhang

Research output: Contribution to journal › Article › peer-review

18 Scopus citations

Abstract

We determine the entropy rate of patterns of certain random processes including all finite-entropy stationary processes. For independent and identically distributed (i.i.d.) processes, we also bound the speed at which the per-symbol pattern entropy converges to this rate, and show that patterns satisfy an asymptotic equipartition property. To derive some of these results we upper bound the probability that the nth variable in a random process differs from all preceding ones. copy; 2006 IEEE.

Original language	English (US)
Pages (from-to)	2954-2964
Number of pages	11
Journal	IEEE Transactions on Information Theory
Volume	52
Issue number	7
DOIs	https://doi.org/10.1109/TIT.2006.876351
State	Published - Jul 2006
Externally published	Yes

Keywords

Asymptotic equipartition property
Entropy rate
Large alphabets
Patterns
Universal compression

ASJC Scopus subject areas

Information Systems
Computer Science Applications
Library and Information Sciences

Access to Document

10.1109/TIT.2006.876351

Cite this

@article{c8bc4008029043b78760744443de484f,

title = "Limit results on pattern entropy",

abstract = "We determine the entropy rate of patterns of certain random processes including all finite-entropy stationary processes. For independent and identically distributed (i.i.d.) processes, we also bound the speed at which the per-symbol pattern entropy converges to this rate, and show that patterns satisfy an asymptotic equipartition property. To derive some of these results we upper bound the probability that the nth variable in a random process differs from all preceding ones. copy; 2006 IEEE.",

keywords = "Asymptotic equipartition property, Entropy rate, Large alphabets, Patterns, Universal compression",

author = "Alon Orlitsky and Santhanam, {Narayana P.} and Krishnamurthy Viswanathan and Junan Zhang",

note = "Funding Information: Manuscript received December 20, 2004; revised March 7, 2006. This work was supported by the National Science Foundation and the Ericsson Corporation.",

year = "2006",

month = jul,

doi = "10.1109/TIT.2006.876351",

language = "English (US)",

volume = "52",

pages = "2954--2964",

journal = "IEEE Transactions on Information Theory",

issn = "0018-9448",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "7",

}

TY - JOUR

T1 - Limit results on pattern entropy

AU - Orlitsky, Alon

AU - Santhanam, Narayana P.

AU - Viswanathan, Krishnamurthy

AU - Zhang, Junan

N1 - Funding Information: Manuscript received December 20, 2004; revised March 7, 2006. This work was supported by the National Science Foundation and the Ericsson Corporation.

PY - 2006/7

Y1 - 2006/7

N2 - We determine the entropy rate of patterns of certain random processes including all finite-entropy stationary processes. For independent and identically distributed (i.i.d.) processes, we also bound the speed at which the per-symbol pattern entropy converges to this rate, and show that patterns satisfy an asymptotic equipartition property. To derive some of these results we upper bound the probability that the nth variable in a random process differs from all preceding ones. copy; 2006 IEEE.

AB - We determine the entropy rate of patterns of certain random processes including all finite-entropy stationary processes. For independent and identically distributed (i.i.d.) processes, we also bound the speed at which the per-symbol pattern entropy converges to this rate, and show that patterns satisfy an asymptotic equipartition property. To derive some of these results we upper bound the probability that the nth variable in a random process differs from all preceding ones. copy; 2006 IEEE.

KW - Asymptotic equipartition property

KW - Entropy rate

KW - Large alphabets

KW - Patterns

KW - Universal compression

UR - http://www.scopus.com/inward/record.url?scp=33746887402&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33746887402&partnerID=8YFLogxK

U2 - 10.1109/TIT.2006.876351

DO - 10.1109/TIT.2006.876351

M3 - Article

AN - SCOPUS:33746887402

SN - 0018-9448

VL - 52

SP - 2954

EP - 2964

JO - IEEE Transactions on Information Theory

JF - IEEE Transactions on Information Theory

IS - 7

ER -

Limit results on pattern entropy

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this