Relative redundancy for large alphabets

Alon Orlitsky, Narayana Santhanam, Junan Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.

Original languageEnglish (US)
Title of host publicationIEEE International Symposium on Information Theory - Proceedings
Pages2672-2676
Number of pages5
DOIs
StatePublished - 2006
Externally publishedYes
Event2006 IEEE International Symposium on Information Theory, ISIT 2006 - Seattle, WA, United States
Duration: Jul 9 2006Jul 14 2006

Other

Other2006 IEEE International Symposium on Information Theory, ISIT 2006
CountryUnited States
CitySeattle, WA
Period7/9/067/14/06

Fingerprint

Redundancy

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Orlitsky, A., Santhanam, N., & Zhang, J. (2006). Relative redundancy for large alphabets. In IEEE International Symposium on Information Theory - Proceedings (pp. 2672-2676). [4036457] https://doi.org/10.1109/ISIT.2006.262138

Relative redundancy for large alphabets. / Orlitsky, Alon; Santhanam, Narayana; Zhang, Junan.

IEEE International Symposium on Information Theory - Proceedings. 2006. p. 2672-2676 4036457.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Orlitsky, A, Santhanam, N & Zhang, J 2006, Relative redundancy for large alphabets. in IEEE International Symposium on Information Theory - Proceedings., 4036457, pp. 2672-2676, 2006 IEEE International Symposium on Information Theory, ISIT 2006, Seattle, WA, United States, 7/9/06. https://doi.org/10.1109/ISIT.2006.262138
Orlitsky A, Santhanam N, Zhang J. Relative redundancy for large alphabets. In IEEE International Symposium on Information Theory - Proceedings. 2006. p. 2672-2676. 4036457 https://doi.org/10.1109/ISIT.2006.262138
Orlitsky, Alon ; Santhanam, Narayana ; Zhang, Junan. / Relative redundancy for large alphabets. IEEE International Symposium on Information Theory - Proceedings. 2006. pp. 2672-2676
@inproceedings{161daa0b49c0460187652e8a2c5e60d9,
title = "Relative redundancy for large alphabets",
abstract = "Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.",
author = "Alon Orlitsky and Narayana Santhanam and Junan Zhang",
year = "2006",
doi = "10.1109/ISIT.2006.262138",
language = "English (US)",
isbn = "1424405041",
pages = "2672--2676",
booktitle = "IEEE International Symposium on Information Theory - Proceedings",

}

TY - GEN

T1 - Relative redundancy for large alphabets

AU - Orlitsky, Alon

AU - Santhanam, Narayana

AU - Zhang, Junan

PY - 2006

Y1 - 2006

N2 - Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.

AB - Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.

UR - http://www.scopus.com/inward/record.url?scp=39049107079&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=39049107079&partnerID=8YFLogxK

U2 - 10.1109/ISIT.2006.262138

DO - 10.1109/ISIT.2006.262138

M3 - Conference contribution

SN - 1424405041

SN - 9781424405046

SP - 2672

EP - 2676

BT - IEEE International Symposium on Information Theory - Proceedings

ER -