Universal coding of zipf distributions

Yoav Freund, Alon Orlitsky, Prasad Santhanam, Junan Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The universal coding of Zipf distributions are discussed. Coding schemes whose redundancy increases slower than n are known as universal. The universal coding scheme can be considered as an algorithm for combining expert advice whose code length is equal to the cumulative log loss and the redundancy is the difference between the loss of the combining algorithm and the loss of the best expert. When compressing natural-language text, it is reasonable to code the text a word at a time, thereby relying on the distribution of the words. One approach to reduce the redundancy in that case would be to restrict the collection of possible distributions over the words.

Original languageEnglish (US)
Title of host publicationLecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)
EditorsB. Scholkopf, M.K. Warmuth
Pages736-737
Number of pages2
Volume2777
StatePublished - 2003
Externally publishedYes
Event16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003 - Washington, DC, United States
Duration: Aug 24 2003Aug 27 2003

Other

Other16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003
CountryUnited States
CityWashington, DC
Period8/24/038/27/03

Fingerprint

Redundancy
Coding
Natural Language
Text

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Freund, Y., Orlitsky, A., Santhanam, P., & Zhang, J. (2003). Universal coding of zipf distributions. In B. Scholkopf, & M. K. Warmuth (Eds.), Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science) (Vol. 2777, pp. 736-737)

Universal coding of zipf distributions. / Freund, Yoav; Orlitsky, Alon; Santhanam, Prasad; Zhang, Junan.

Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science). ed. / B. Scholkopf; M.K. Warmuth. Vol. 2777 2003. p. 736-737.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Freund, Y, Orlitsky, A, Santhanam, P & Zhang, J 2003, Universal coding of zipf distributions. in B Scholkopf & MK Warmuth (eds), Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science). vol. 2777, pp. 736-737, 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, United States, 8/24/03.
Freund Y, Orlitsky A, Santhanam P, Zhang J. Universal coding of zipf distributions. In Scholkopf B, Warmuth MK, editors, Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science). Vol. 2777. 2003. p. 736-737
Freund, Yoav ; Orlitsky, Alon ; Santhanam, Prasad ; Zhang, Junan. / Universal coding of zipf distributions. Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science). editor / B. Scholkopf ; M.K. Warmuth. Vol. 2777 2003. pp. 736-737
@inproceedings{079e05e313eb400885b930fbf0fa95cb,
title = "Universal coding of zipf distributions",
abstract = "The universal coding of Zipf distributions are discussed. Coding schemes whose redundancy increases slower than n are known as universal. The universal coding scheme can be considered as an algorithm for combining expert advice whose code length is equal to the cumulative log loss and the redundancy is the difference between the loss of the combining algorithm and the loss of the best expert. When compressing natural-language text, it is reasonable to code the text a word at a time, thereby relying on the distribution of the words. One approach to reduce the redundancy in that case would be to restrict the collection of possible distributions over the words.",
author = "Yoav Freund and Alon Orlitsky and Prasad Santhanam and Junan Zhang",
year = "2003",
language = "English (US)",
volume = "2777",
pages = "736--737",
editor = "B. Scholkopf and M.K. Warmuth",
booktitle = "Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)",

}

TY - GEN

T1 - Universal coding of zipf distributions

AU - Freund, Yoav

AU - Orlitsky, Alon

AU - Santhanam, Prasad

AU - Zhang, Junan

PY - 2003

Y1 - 2003

N2 - The universal coding of Zipf distributions are discussed. Coding schemes whose redundancy increases slower than n are known as universal. The universal coding scheme can be considered as an algorithm for combining expert advice whose code length is equal to the cumulative log loss and the redundancy is the difference between the loss of the combining algorithm and the loss of the best expert. When compressing natural-language text, it is reasonable to code the text a word at a time, thereby relying on the distribution of the words. One approach to reduce the redundancy in that case would be to restrict the collection of possible distributions over the words.

AB - The universal coding of Zipf distributions are discussed. Coding schemes whose redundancy increases slower than n are known as universal. The universal coding scheme can be considered as an algorithm for combining expert advice whose code length is equal to the cumulative log loss and the redundancy is the difference between the loss of the combining algorithm and the loss of the best expert. When compressing natural-language text, it is reasonable to code the text a word at a time, thereby relying on the distribution of the words. One approach to reduce the redundancy in that case would be to restrict the collection of possible distributions over the words.

UR - http://www.scopus.com/inward/record.url?scp=9444297793&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=9444297793&partnerID=8YFLogxK

M3 - Conference contribution

VL - 2777

SP - 736

EP - 737

BT - Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)

A2 - Scholkopf, B.

A2 - Warmuth, M.K.

ER -