Relative redundancy for large alphabets

Alon Orlitsky; Narayana Santhanam; Junan Zhang

doi:10.1109/ISIT.2006.262138

Relative redundancy for large alphabets

Alon Orlitsky, Narayana Santhanam, Junan Zhang

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.

Original language	English (US)
Title of host publication	Proceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006
Pages	2672-2676
Number of pages	5
DOIs	https://doi.org/10.1109/ISIT.2006.262138
State	Published - Dec 1 2006
Externally published	Yes
Event	2006 IEEE International Symposium on Information Theory, ISIT 2006 - Seattle, WA, United States Duration: Jul 9 2006 → Jul 14 2006

Publication series

Name	IEEE International Symposium on Information Theory - Proceedings
ISSN (Print)	2157-8101

Other

Other	2006 IEEE International Symposium on Information Theory, ISIT 2006
Country/Territory	United States
City	Seattle, WA
Period	7/9/06 → 7/14/06

ASJC Scopus subject areas

Theoretical Computer Science
Information Systems
Modeling and Simulation
Applied Mathematics

Access to Document

10.1109/ISIT.2006.262138

Cite this

Orlitsky, A, Santhanam, N & Zhang, J 2006, Relative redundancy for large alphabets. in Proceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006., 4036457, IEEE International Symposium on Information Theory - Proceedings, pp. 2672-2676, 2006 IEEE International Symposium on Information Theory, ISIT 2006, Seattle, WA, United States, 7/9/06. https://doi.org/10.1109/ISIT.2006.262138

@inproceedings{161daa0b49c0460187652e8a2c5e60d9,

title = "Relative redundancy for large alphabets",

abstract = "Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.",

author = "Alon Orlitsky and Narayana Santhanam and Junan Zhang",

year = "2006",

month = dec,

day = "1",

doi = "10.1109/ISIT.2006.262138",

language = "English (US)",

isbn = "1424405041",

series = "IEEE International Symposium on Information Theory - Proceedings",

pages = "2672--2676",

booktitle = "Proceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006",

note = "2006 IEEE International Symposium on Information Theory, ISIT 2006 ; Conference date: 09-07-2006 Through 14-07-2006",

}

TY - GEN

T1 - Relative redundancy for large alphabets

AU - Orlitsky, Alon

AU - Santhanam, Narayana

AU - Zhang, Junan

PY - 2006/12/1

Y1 - 2006/12/1

N2 - Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.

AB - Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.

UR - http://www.scopus.com/inward/record.url?scp=39049107079&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=39049107079&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2006.262138

DO - 10.1109/ISIT.2006.262138

M3 - Conference contribution

AN - SCOPUS:39049107079

SN - 1424405041

SN - 9781424405046

T3 - IEEE International Symposium on Information Theory - Proceedings

SP - 2672

EP - 2676

BT - Proceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006

T2 - 2006 IEEE International Symposium on Information Theory, ISIT 2006

Y2 - 9 July 2006 through 14 July 2006

ER -

Relative redundancy for large alphabets

Abstract

Publication series

Other

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this