Smoothing problem (stochastic processes)

The Smoothing problem (not to be confused with smoothing in statistics, image processing and other contexts) refers to Recursive Bayesian estimation also known as Bayes filter is the problem of estimating an unknown probability density function recursively over time using incremental incoming measurements. It is one of the main problems defined by Norbert Wiener [1][2]

A smoother is an algorithm or implementation that implements a solution to such problem. Please refer to the article Recursive Bayesian estimation for more information. The Smoothing problem and Filtering problem are often considered a closely related pair of problems. They are studied in Bayesian smoothing theory.

Note: Not to be confused with blurring and smoothing using methods such as moving average. See smoothing.

Example smoothers

Some variants include:[3]

  • Rauch–Tung–Striebel (RTS) smoother
  • Extended RTS smoother (ERTSS)
  • Gauss–Hermite RTS smoother (GHRTSS)
  • Cubature RTS smoother (CRTSS)

The confusion in terms and the relation between Filtering and Smoothing problems

There are four terms that cause confusion: Smoothing (in two senses: estimation and convolution), and Filtering (again in two senses: estimation and convolution).

Smoothing (estimation) and smoothing (convolution) can mean totally different, but sound like they are apparently similar. The concepts are different and are used in almost different historical contexts. The requirements are very different.

Note that initially, the Wiener's filter was just a convolution, but the later developments were different: one was estimation and the other one was filter design in the sense of design of a convolution filter. This is a source of confusion.

Both the smoothing problem (in sense of estimation) and the filtering problem (in sense of estimation) are often confused with smoothing and filtering in other contexts (especially non-stochastic signal processing, often a name of various types of convolution). These names are used in the context of World War 2 with problems framed by people like Norbert Wiener.[1][2] One source of confusion is the Wiener Filter is in form of a simple convolution. However, in Wiener's filter, two time-series are given. When the filter is defined, a straightforward convolution is the answer. However, in later developments such as Kalman filtering, the nature of filtering is different to convolution and it deserves a different name.

The distinction is described in the following two senses:

1. Convolution: The smoothing in the sense of convolution is simpler. For example, moving average, low-pass filtering, convolution with a kernel, or blurring using Laplace filters in image processing. It is often a filter design problem. Especially non-stochastic and non-Bayesian signal processing, without any hidden variables.

2. Estimation: The smoothing problem (or Smoothing in the sense of estimation) uses Bayesian and state-space models to estimate the hidden state variables. This is used in the context of World War 2 defined by people like Norbert Wiener, in (stochastic) control theory, radar, signal detection, tracking, etc. The most common use is the Kalman Smoother used with Kalman Filter, which is actually developed by Rauch. The procedure is called Kalman-Rauch recursion. It is one of the main problems solved by Norbert Wiener.[1][2] Most importantly, in the Filtering problem (sense 2) the information from observation up to the time of the current sample is used. In smoothing (also sense 2) all observation samples (from future) are used. Filtering is causal but smoothing is batch processing of the same problem, namely, estimation of a time-series process based on serial incremental observations.

But the usual and more common smoothing and filtering (in the sense of 1.) do not have such distinction because there is no distinction between hidden and observable.

The distinction between Smoothing (estimation) and Filtering (estimation): In smoothing all observation samples are used (from future). Filtering is causal, whereas smoothing is batch processing of the given data. Filtering is the estimation of a (hidden) time-series process based on serial incremental observations.

See also

References
  1. 1942, Extrapolation, Interpolation and Smoothing of Stationary Time Series. A war-time classified report nicknamed "the yellow peril" because of the color of the cover and the difficulty of the subject. Published postwar 1949 MIT Press. http://www.isss.org/lumwiener.htm
  2. Wiener, Norbert (1949). Extrapolation, Interpolation, and Smoothing of Stationary Time Series. New York: Wiley. ISBN 0-262-73005-7.
  3. Simo Särkkä. Bayesian Filtering and Smoothing. Publisher: Cambridge University Press (5 Sept. 2013) Language: English ISBN 1107619289 ISBN 978-1107619289
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.