Multi-stream video fusion using local principal components analysis

Ravi K. Sharma, Misha Pavel, Todd K. Leen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

We present an approach for fusion of video streams produced by multiple imaging sensors such as visible-band and infrared sensors. Our approach is based on a model in which the sensor images are noisy, locally affine functions of the true scene. This model explicitly incorporates reversals in local contrast, sensor-specific features and noise in the sensing process. Given the parameters of the local affine transformations and the sensor images, a Bayesian framework provides a maximum a posteriori estimate of the true scene. This estimate constitutes the rule for fusing the sensor images. We also give a maximum likelihood estimate for the parameters of the local affine transformations. Under Gaussian assumptions on the underlying distributions, estimation of the affine parameters is achieved by local principal component analysis. The sensor noise is estimated by analyzing the sequence of images in each video stream. The analysis of the video streams and the synthesis of the fused stream is performed in a multiresolution pyramid domain.

Original languageEnglish (US)
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
EditorsB.F. Andresen, M. Strojnik
PublisherSPIE
Pages717-725
Number of pages9
Volume3436
Edition2
StatePublished - 1998
EventProceedings of the 1998 Conference on Infrared Technology and Applications XXIV. Part 1 (of 2) - San Diego, CA, USA
Duration: Jul 19 1998Jul 24 1998

Other

OtherProceedings of the 1998 Conference on Infrared Technology and Applications XXIV. Part 1 (of 2)
CitySan Diego, CA, USA
Period7/19/987/24/98

Fingerprint

principal components analysis
Principal component analysis
Fusion reactions
fusion
Image sensors
sensors
Sensors
Maximum likelihood
maximum likelihood estimates
Infrared radiation
Imaging techniques
estimates
pyramids
synthesis

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Condensed Matter Physics

Cite this

Sharma, R. K., Pavel, M., & Leen, T. K. (1998). Multi-stream video fusion using local principal components analysis. In B. F. Andresen, & M. Strojnik (Eds.), Proceedings of SPIE - The International Society for Optical Engineering (2 ed., Vol. 3436, pp. 717-725). SPIE.

Multi-stream video fusion using local principal components analysis. / Sharma, Ravi K.; Pavel, Misha; Leen, Todd K.

Proceedings of SPIE - The International Society for Optical Engineering. ed. / B.F. Andresen; M. Strojnik. Vol. 3436 2. ed. SPIE, 1998. p. 717-725.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sharma, RK, Pavel, M & Leen, TK 1998, Multi-stream video fusion using local principal components analysis. in BF Andresen & M Strojnik (eds), Proceedings of SPIE - The International Society for Optical Engineering. 2 edn, vol. 3436, SPIE, pp. 717-725, Proceedings of the 1998 Conference on Infrared Technology and Applications XXIV. Part 1 (of 2), San Diego, CA, USA, 7/19/98.
Sharma RK, Pavel M, Leen TK. Multi-stream video fusion using local principal components analysis. In Andresen BF, Strojnik M, editors, Proceedings of SPIE - The International Society for Optical Engineering. 2 ed. Vol. 3436. SPIE. 1998. p. 717-725
Sharma, Ravi K. ; Pavel, Misha ; Leen, Todd K. / Multi-stream video fusion using local principal components analysis. Proceedings of SPIE - The International Society for Optical Engineering. editor / B.F. Andresen ; M. Strojnik. Vol. 3436 2. ed. SPIE, 1998. pp. 717-725
@inproceedings{108bec095ec748de8a40eb5a54eca8ef,
title = "Multi-stream video fusion using local principal components analysis",
abstract = "We present an approach for fusion of video streams produced by multiple imaging sensors such as visible-band and infrared sensors. Our approach is based on a model in which the sensor images are noisy, locally affine functions of the true scene. This model explicitly incorporates reversals in local contrast, sensor-specific features and noise in the sensing process. Given the parameters of the local affine transformations and the sensor images, a Bayesian framework provides a maximum a posteriori estimate of the true scene. This estimate constitutes the rule for fusing the sensor images. We also give a maximum likelihood estimate for the parameters of the local affine transformations. Under Gaussian assumptions on the underlying distributions, estimation of the affine parameters is achieved by local principal component analysis. The sensor noise is estimated by analyzing the sequence of images in each video stream. The analysis of the video streams and the synthesis of the fused stream is performed in a multiresolution pyramid domain.",
author = "Sharma, {Ravi K.} and Misha Pavel and Leen, {Todd K.}",
year = "1998",
language = "English (US)",
volume = "3436",
pages = "717--725",
editor = "B.F. Andresen and M. Strojnik",
booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",
publisher = "SPIE",
edition = "2",

}

TY - GEN

T1 - Multi-stream video fusion using local principal components analysis

AU - Sharma, Ravi K.

AU - Pavel, Misha

AU - Leen, Todd K.

PY - 1998

Y1 - 1998

N2 - We present an approach for fusion of video streams produced by multiple imaging sensors such as visible-band and infrared sensors. Our approach is based on a model in which the sensor images are noisy, locally affine functions of the true scene. This model explicitly incorporates reversals in local contrast, sensor-specific features and noise in the sensing process. Given the parameters of the local affine transformations and the sensor images, a Bayesian framework provides a maximum a posteriori estimate of the true scene. This estimate constitutes the rule for fusing the sensor images. We also give a maximum likelihood estimate for the parameters of the local affine transformations. Under Gaussian assumptions on the underlying distributions, estimation of the affine parameters is achieved by local principal component analysis. The sensor noise is estimated by analyzing the sequence of images in each video stream. The analysis of the video streams and the synthesis of the fused stream is performed in a multiresolution pyramid domain.

AB - We present an approach for fusion of video streams produced by multiple imaging sensors such as visible-band and infrared sensors. Our approach is based on a model in which the sensor images are noisy, locally affine functions of the true scene. This model explicitly incorporates reversals in local contrast, sensor-specific features and noise in the sensing process. Given the parameters of the local affine transformations and the sensor images, a Bayesian framework provides a maximum a posteriori estimate of the true scene. This estimate constitutes the rule for fusing the sensor images. We also give a maximum likelihood estimate for the parameters of the local affine transformations. Under Gaussian assumptions on the underlying distributions, estimation of the affine parameters is achieved by local principal component analysis. The sensor noise is estimated by analyzing the sequence of images in each video stream. The analysis of the video streams and the synthesis of the fused stream is performed in a multiresolution pyramid domain.

UR - http://www.scopus.com/inward/record.url?scp=0032295996&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032295996&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:0032295996

VL - 3436

SP - 717

EP - 725

BT - Proceedings of SPIE - The International Society for Optical Engineering

A2 - Andresen, B.F.

A2 - Strojnik, M.

PB - SPIE

ER -