Relative redundancy for large alphabets

Alon Orlitsky, Narayana Santhanam, Junan Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.

Original languageEnglish (US)
Title of host publicationProceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006
Pages2672-2676
Number of pages5
DOIs
StatePublished - Dec 1 2006
Event2006 IEEE International Symposium on Information Theory, ISIT 2006 - Seattle, WA, United States
Duration: Jul 9 2006Jul 14 2006

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
ISSN (Print)2157-8101

Other

Other2006 IEEE International Symposium on Information Theory, ISIT 2006
CountryUnited States
CitySeattle, WA
Period7/9/067/14/06

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Relative redundancy for large alphabets'. Together they form a unique fingerprint.

  • Cite this

    Orlitsky, A., Santhanam, N., & Zhang, J. (2006). Relative redundancy for large alphabets. In Proceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006 (pp. 2672-2676). [4036457] (IEEE International Symposium on Information Theory - Proceedings). https://doi.org/10.1109/ISIT.2006.262138