Kalman Assumption
in [DSP]
|
Prev: CPM
Next: "Correcting" output of a filter
From: Cagdas Ozgenc on 26 Apr 2010 05:04 Hello, In Kalman filtering does the process noise have to be Gaussian or would any uncorrelated covariance stationary noise satisfy the requirements? When I follow the derivations of the filter I haven't encountered any requirements on Gaussian distribution, but in many sources Gaussian tag seems to go together. Thanks
From: Rune Allnor on 26 Apr 2010 05:56 On 26 apr, 11:04, Cagdas Ozgenc <cagdas.ozg...(a)gmail.com> wrote: > Hello, > > In Kalman filtering does the process noise have to be Gaussian or > would any uncorrelated covariance stationary noise satisfy the > requirements? I don't know the Kalman filter in sufficient detail to have any opinions about how the model would cope with non-Gaussian noise, but there are some very good reasons for choosing a Gaussian noise model. Gaussian noise is fully described vy the first and second order moments, since all higher-odrer moments vanish. The Gaussian distribution is the only distribution for which this is true. So if you choose to use something else than a Gaussian distribution, you are responsible for two severely complicating factors: 1) Which distribution to use? 2) How to find the parameters for the distribution? So by making a *philosophical* simplification you avoid a lot of serious *technical* problems. And of course, the Gaussian model isn't that bad, even from a philosophical point of view: Noise (or rather, stochastic processes) appear twice in the Kalman model: As observation noise, and as the innovation process. The empirical experience is that the Gaussian model is rather good for modeling observation noise in quite a few different systems. The innovation process is an internal component to the Kalman filter, so you are freee to choose any kind of stochastic model you want. *Choosing* to use the simple and convenient Gaussian model relieves you from addressing a lot of boring technical details. Rune
From: Tim Wescott on 26 Apr 2010 12:40 Cagdas Ozgenc wrote: > Hello, > > In Kalman filtering does the process noise have to be Gaussian or > would any uncorrelated covariance stationary noise satisfy the > requirements? > > When I follow the derivations of the filter I haven't encountered any > requirements on Gaussian distribution, but in many sources Gaussian > tag seems to go together. The Kalman filter is only guaranteed to be optimal when: * The modeled system is linear. * Any time-varying behavior of the system is known. * The noise (process and measurement) is Gaussian. * The noise's time-dependent behavior is known (note that this means the noise doesn't have to be stationary -- just that it's time-dependent behavior is known). * The model exactly matches reality. None of these requirements can be met in reality, but the math is at its most tractable when you assume them. Often the Gaussian noise assumption comes the closest to being true -- but not always. If your system matches all of the above assumptions _except_ the Gaussian noise assumption, then the Kalman filter that you design will have the lowest error variance of any possible _linear_ filter, but there may be nonlinear filters with better (perhaps significantly better) performance. Here's a simple related example. It's not really Kalman filtering, but it should get the point across: Assume that you're measuring a slowly varying voltage, and for some reason some joker keeps inserting a dry cell battery in your lines, and he's known to be doing it -- on average -- exactly half the time. So for any given measurement the battery may be there, offsetting your measurement by +1.5V, or it may not be, thereby adding no offset at all. On top of this offset noise, there's some small Gaussian noise -- say 50mV worth. You measure your voltage 10 times, and you get: 10.04, 11.47, 11.55, 9.92, 10.10, 10.01, 11.53, 9.96, 10.03, 11.48 The 'linear filter' way to process this data would be to just average the results, then subtract 0.75V. This gives you a reading of 10.61 - 0.75 = 9.86V. But you're throwing a way a huge amount of information by doing this. Because you know that the noise has a huge two-valued component, and a much smaller Gaussian component, we can identify which data points are offset. By not adjusting these, we are allowing the "not quite one half" nature of the offset to skew our whole estimate -- and this is not necessary in this case. Just by looking at the data above, you can see the four events where the 1.5V offset was present. Take this offset out, and you get: 10.04, 9.97, 10.05, 9.92, 10.10, 10.01, 10.03, 9.96, 10.03, 9.98 The average of _this_ is 10.01. So we used an estimation process that's horridly nonlinear -- we picked out all of the "high" measurements and adjusted them, then averaged. But as a consequence we got a much more accurate measurement (at least I was thinking "10V" when I made my vector of measurements. There are filters that are related to the Kalman filter that can take this stuff into account. But the construction of them becomes infinitely more ad-hoc as you depart further from the assumptions I outlined above. The _nice_ thing about the above assumptions is that you can start with a system description, turn the crank on a Kalman design technique, and get one filter as a result. You may need to adjust the internals of the filter for better performance, but you are just selecting a different implementation of a filter that is mathematically identical to the filter that you originally designed. So you never have to ask "is this filter the best", just "is the way I'm implementing this the best". Extended Kalman filters, Unscented Kalman filters, Baysian filters, particle filters, etc., are all either rank approximations to the "truth" that you're trying to find, or are exceedingly mathematically intractable. You can get some really nice results, but you have to dig hard for them, and you have to dig even harder to make sure that what you're doing is robust. -- Tim Wescott Control system and signal processing consulting www.wescottdesign.com
From: HardySpicer on 26 Apr 2010 21:52 On Apr 27, 4:40 am, Tim Wescott <t...(a)seemywebsite.now> wrote: > Cagdas Ozgenc wrote: > > Hello, > > > In Kalman filtering does the process noise have to be Gaussian or > > would any uncorrelated covariance stationary noise satisfy the > > requirements? > > > When I follow the derivations of the filter I haven't encountered any > > requirements on Gaussian distribution, but in many sources Gaussian > > tag seems to go together. > > The Kalman filter is only guaranteed to be optimal when: > > * The modeled system is linear. > * Any time-varying behavior of the system is known. > * The noise (process and measurement) is Gaussian. > * The noise's time-dependent behavior is known > (note that this means the noise doesn't have to be stationary -- > just that it's time-dependent behavior is known). > * The model exactly matches reality. > > None of these requirements can be met in reality, but the math is at its > most tractable when you assume them. Often the Gaussian noise > assumption comes the closest to being true -- but not always. > > If your system matches all of the above assumptions _except_ the > Gaussian noise assumption, then the Kalman filter that you design will > have the lowest error variance of any possible _linear_ filter, but > there may be nonlinear filters with better (perhaps significantly > better) performance. Don't think so. You can design an H infinity linear Kalman filter which is only a slight modification and you don't even need to know what the covariance matrices are at all. H infinity will give you the minimum of the maximum error. Hardy
From: Peter K. on 26 Apr 2010 22:18
On 26 Apr, 21:52, HardySpicer <gyansor...(a)gmail.com> wrote: > Don't think so. You can design an H infinity linear Kalman filter > which is only a slight modification and you don't even need to know > what the covariance matrices are at all. > H infinity will give you the minimum of the maximum error. As Tim says, the Kalman filter is the optimal linear filter for minimizing the average estimation error. Reformulations using H- infinity techniques do not give an optimal linear filter in this sense. As you say, though, H-nifty (sic) give the optimal in terms of minimizing the worst case estimation error... which may or may not give "better" results than the Kalman approach. Depending on the application, neither "optimal" approach may give exactly what the user is after... their idea of "optimal' may be different from what the mathematical formulations give. Ciao, Peter K. |