Relative redundancy for large alphabets

Alon Orlitsky, Narayana Santhanam, Junan Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Standard redundancy measures the excess number of bits required to encode a sequence of a given length when the underlying distribution is not known. Relative redundancy measures the same increase, but as a function of the sequence's minimum description length. We consider the relative redundancy of i.i.d. distributions over large alphabets and show that, like standard redundancy, relative redundancy too increases with the alphabet size. We then consider compression of patterns of i.i.d. strings. Again analogous to standard redundancy, we show that the relative redundancy of patterns of large, or even infinite alphabet i.i.d. distributions is negligible compared to the patterns' minimum description length.

Original languageEnglish (US)
Title of host publicationProceedings - 2006 IEEE International Symposium on Information Theory, ISIT 2006
Number of pages5
StatePublished - Dec 1 2006
Externally publishedYes
Event2006 IEEE International Symposium on Information Theory, ISIT 2006 - Seattle, WA, United States
Duration: Jul 9 2006Jul 14 2006

Publication series

NameIEEE International Symposium on Information Theory - Proceedings
ISSN (Print)2157-8101


Other2006 IEEE International Symposium on Information Theory, ISIT 2006
Country/TerritoryUnited States
CitySeattle, WA

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Modeling and Simulation
  • Applied Mathematics


Dive into the research topics of 'Relative redundancy for large alphabets'. Together they form a unique fingerprint.

Cite this