Convolution Tutorial
in [DSP]
|
Prev: forward error correction capabilities?
Next: Position estimation using tri-axis accelerometer with gyroscope in Static standing
From: HardySpicer on 27 Dec 2009 20:21 On Dec 28, 10:50 am, Jerry Avins <j...(a)ieee.org> wrote: > HardySpicer wrote: > > ... > > > Agreed but it's pretty dammed close. I mean how linear is an R-C > > network? Try adding two sine waves and passing them through an RC > > network. > > What cross-spectral terms to you get percentage wise? I would be > > interested to know. > > Electrolytics are notoriously non-linear. High-value ceramic capacitors > -- calcium titanate, for example -- are also non-linear. Dielectric > absorption is a continuing problem with sample-and-hold capacitors. > > Non-linear resistors abound. Hewlwtt and Packard had the brilliance to > use a tungsten resistor to stabilize the amplitude of a Wien-bridge > oscillator and founded a company on the parent. (The movie Fantasia was > an early user of their tunable audio oscillators.) > > The components we commonly use are explicitly selected to be adeqquately > linear under their normal conditions of use. Most of the world doesn't > work that way. > > Jerry > -- > Engineering is the art of making what you want from things you can get. Don't for get that linear convolution is a special case of the non- linear problem which has an infinite series of nested convolutions. So convolution still holds (sort of). Hardy
From: robert bristow-johnson on 27 Dec 2009 21:55 On Dec 27, 8:21 pm, HardySpicer <gyansor...(a)gmail.com> wrote: > > Don't forget that linear convolution is a special case of the non- > linear problem which has an infinite series of nested convolutions. i dunno what you mean by "nested convolutions". even the convolution of a convolution is a linear operation which can be modeled with its own aggregate convolution. if what you mean that "linear convolution" is a special case of the more general Volterra series model for a general time-invariant, non- linear system with memory, then i agree with that. > So convolution still holds (sort of). Hardy, despite all the people picking on you, i'm more on your side. you need not back down too far... On Dec 27, 4:04 pm, HardySpicer <gyansor...(a)gmail.com> wrote: > On Dec 28, 8:31 am, Tim Wescott <t...(a)seemywebsite.com> wrote: > > > > > On Sun, 27 Dec 2009 10:01:07 +0000, invalid wrote: > > > "brent" <buleg...(a)columbus.rr.com> wrote in message > > > news:0fd6f825-e7ad-4642- > > > a5fe-83de8ff8f...(a)x18g2000vbd.googlegroups.com... > > > >>I have created a tutorial on the convolution integral. It uses an > > >> interactive flash program with embedded audio files. It is located > > >> here: > > >>http://www.fourier-series.com/Convolution/index.html > > > > You start off by saying that convolution is a mathematical operation, at > > > which point I switched off. > > > > Convolution is the way that real systems in the real world (such as > > > pianoforte strings) > > > respond to stimuli that are continuous (such as a sine wave from a > > > loudspeaker in close proximity) > > > Convolution is _not_ the way that real systems in the real world respond > > to stimuli of any sort. here, Tim, i might have to disagree. if the real system in the real world is sufficiently linear (operating within specs of which the nonlinearities lie outside), then convolution is *precisely* the way it responds to stimuli. the real system that is operating essentially linearly is responding (at the "present" time, t) to a stimulus t0 seconds ago (at time t-t0) with a scaling that is proportional to h(t0), how it would respond to an isolated impulse-like stimulus positioned at a time t0 seconds ago. and the real system ("sufficiently" linear) has superposition applying; how it responds to a stimulus with two components, one 5 ms ago and another 10 ms ago, will be the sum of what it would do for the two of them as isolated stimuli. for an RC circuit, that's what the *physics* says as long as we keep the Volume knob set to something below Arc-Weld (or "11" on Spinal Tap). > > Convolution is just a _mathematical operation_ > > that _approximates_ what real systems do. Sometimes it even does it well. > > > All real systems are nonlinear. even more generally, all systems deviate from whatever model we describe them with. that doesn't stop us from using models to describe systems. sometimes a linear model suffices. sometimes you can separate out a non-linear component from a linear subsystem with memory. when you can't, i guess there is Volterra series to fall back on. > > The convolution operation is one way to > > implement a linear model of a system. it's the *only* way, but there are multiple ways to implement convolution. but, physically, why real (analog) LTI systems themselves perform convolution as the primary way that they operate is because the primary manner that they operate is linear (superposition applies) and time-invariant. if, upon analysis of a real system, you conclude that it intrinsically behaves linearly with superposition and has no built-in clock (or someone twisting a knob), then you know it's LTI and it *naturally* is performing convolution in its intrinsic operation. > > Thus, the convolution operation > > does not model any real system with 100% accuracy. *only* because LTI doesn't. but if you stay out of saturation, if there aren't other little nonlinearities in the present state of the physical system (like crossover, hysteresis, etc), then that LTI system is *naturally* doing convolution. it is not (likely) doing convolution by continuously performing a Fourier Transform, multiplying in the frequency domain, and inverse Fouriering back, but for the RC circuit, as long as that capacitor is *accumulating* charge in an additive manner, and as long as the resistor isn't burning up, that RC filter is itself doing the convolution integral. > > As a model, the > > convolution operation is only as good as the fit between its bedrock > > assumption of linearity and the system's actual conformity to linear > > behavior. > > > For many systems, using convolution is a horribly indirect way to > > implement what should be a simple, limited-state, ordinary linear > > differential equation. actually implementing an IIR filter is just another way to do convolution where a recursive expression satisfies the model. it's recursive convolution. > > > and not just impulses (such as when hit > > > with a hammer). I had difficulty with Convolution for years until it was > > > explained to me in this practical way at which point it became > > > meaningful > > > instead of being some arcane mathematical operation which I did not > > > really trust. > > > > Unless you introduce the student to the practical basis of why you would > > > want to undertake such a weird operation, then you might as well give > > > up. > > > > Mathematical analysis should come after practical experience and not > > > before. > > > I do agree that mathematical analysis should be kept firmly in the > > context of what is real -- when I teach control systems I try to draw > > examples from the real world as often as possible, and I try to keep a > > clear distinction between the thing you're interested in and the > > mathematical model that you've made of it. > > > But then, you've already wandered away from reality if you're claiming > > that real systems convolve their input signals with unfailing accuracy. > > > In today's world I don't think you can ask for practical experience > > before theoretical knowledge, though -- with that assumption, engineering > > schools would only take technicians who had already been through an > > apprenticeship, which severely cuts down on the available candidate pool. > > Agreed but it's pretty dammed close. I mean how linear is an R-C > network? Try adding two sine waves and passing them through an RC > network. .... that's my story and i'm sticking to it. r b-j
From: steveu on 27 Dec 2009 23:46 >On Sun, 27 Dec 2009 10:01:07 +0000, invalid wrote: > >> "brent" <bulegoge(a)columbus.rr.com> wrote in message >> news:0fd6f825-e7ad-4642- >a5fe-83de8ff8f7f6(a)x18g2000vbd.googlegroups.com... >>>I have created a tutorial on the convolution integral. It uses an >>> interactive flash program with embedded audio files. It is located >>> here: >>> http://www.fourier-series.com/Convolution/index.html >> >> You start off by saying that convolution is a mathematical operation, at >> which point I switched off. >> >> Convolution is the way that real systems in the real world (such as >> pianoforte strings) >> respond to stimuli that are continuous (such as a sine wave from a >> loudspeaker in close proximity) > >Convolution is _not_ the way that real systems in the real world respond >to stimuli of any sort. Convolution is just a _mathematical operation_ >that _approximates_ what real systems do. Sometimes it even does it well. Convolution *is* the way many real systems behave. Its not some arcane mathematical trick. Its the direct mathematical representation of the underlying physical process. How well it fits reality is generally a matter of how much the system is affected by second order effects. This is pretty much like any other area of science and engineering. >All real systems are nonlinear. The convolution operation is one way to >implement a linear model of a system. Thus, the convolution operation >does not model any real system with 100% accuracy. As a model, the >convolution operation is only as good as the fit between its bedrock >assumption of linearity and the system's actual conformity to linear >behavior. You must absolutely loath the entire scientific education system. Almost everything is taught as if it obeys relatively simple relationships, and that's pretty much always a first order approximation. Often the higher order elements are so small you can largely ignore them. If you want accuracy, you'd better scrap Newton's laws of motion. If you really want to complain about people being taught about stuff like its an real accurate model, look at the real villans, like how capacitors are taught. The number of engineers who treat them like they are linear devices is truly sad. They demand that the latest silicon can do A/D conversion at high speed with >16 bits precision, and then surround them with tiny surface mount capacitors who's characteristics are bizarrely funky. Steve
From: Tim Wescott on 28 Dec 2009 02:37 On Sun, 27 Dec 2009 13:04:19 -0800, HardySpicer wrote: > On Dec 28, 8:31 am, Tim Wescott <t...(a)seemywebsite.com> wrote: >> On Sun, 27 Dec 2009 10:01:07 +0000, invalid wrote: >> > "brent" <buleg...(a)columbus.rr.com> wrote in message >> > news:0fd6f825-e7ad-4642- >> >> a5fe-83de8ff8f...(a)x18g2000vbd.googlegroups.com... >> >> >>I have created a tutorial on the convolution integral. It uses an >> >> interactive flash program with embedded audio files. It is located >> >> here: >> >>http://www.fourier-series.com/Convolution/index.html >> >> > You start off by saying that convolution is a mathematical operation, >> > at which point I switched off. >> >> > Convolution is the way that real systems in the real world (such as >> > pianoforte strings) >> > respond to stimuli that are continuous (such as a sine wave from a >> > loudspeaker in close proximity) >> >> Convolution is _not_ the way that real systems in the real world >> respond to stimuli of any sort. Convolution is just a _mathematical >> operation_ that _approximates_ what real systems do. Sometimes it even >> does it well. >> >> All real systems are nonlinear. The convolution operation is one way >> to implement a linear model of a system. Thus, the convolution >> operation does not model any real system with 100% accuracy. As a >> model, the convolution operation is only as good as the fit between its >> bedrock assumption of linearity and the system's actual conformity to >> linear behavior. >> >> For many systems, using convolution is a horribly indirect way to >> implement what should be a simple, limited-state, ordinary linear >> differential equation. >> >> > and not just impulses (such as when hit with a hammer). I had >> > difficulty with Convolution for years until it was explained to me in >> > this practical way at which point it became meaningful >> > instead of being some arcane mathematical operation which I did not >> > really trust. >> >> > Unless you introduce the student to the practical basis of why you >> > would want to undertake such a weird operation, then you might as >> > well give up. >> >> > Mathematical analysis should come after practical experience and not >> > before. >> >> I do agree that mathematical analysis should be kept firmly in the >> context of what is real -- when I teach control systems I try to draw >> examples from the real world as often as possible, and I try to keep a >> clear distinction between the thing you're interested in and the >> mathematical model that you've made of it. >> >> But then, you've already wandered away from reality if you're claiming >> that real systems convolve their input signals with unfailing accuracy. >> >> In today's world I don't think you can ask for practical experience >> before theoretical knowledge, though -- with that assumption, >> engineering schools would only take technicians who had already been >> through an apprenticeship, which severely cuts down on the available >> candidate pool. >> >> --www.wescottdesign.com > > Agreed but it's pretty dammed close. I mean how linear is an R-C > network? Try adding two sine waves and passing them through an RC > network. > What cross-spectral terms to you get percentage wise? I would be > interested to know. If arguments about capacitor nonlinearities are too subtle, try doing this with a 1000 ohm resistor, a 1 microfarad, 50V cap, then plug the assembly into a 120V, 60Hz wall socket. As a thought experiment, of course. -- www.wescottdesign.com
From: Tim Wescott on 28 Dec 2009 02:48
On Sun, 27 Dec 2009 22:46:35 -0600, steveu wrote: >>On Sun, 27 Dec 2009 10:01:07 +0000, invalid wrote: >> >>> "brent" <bulegoge(a)columbus.rr.com> wrote in message >>> news:0fd6f825-e7ad-4642- >>a5fe-83de8ff8f7f6(a)x18g2000vbd.googlegroups.com... >>>>I have created a tutorial on the convolution integral. It uses an >>>> interactive flash program with embedded audio files. It is located >>>> here: >>>> http://www.fourier-series.com/Convolution/index.html >>> >>> You start off by saying that convolution is a mathematical operation, > at >>> which point I switched off. >>> >>> Convolution is the way that real systems in the real world (such as >>> pianoforte strings) >>> respond to stimuli that are continuous (such as a sine wave from a >>> loudspeaker in close proximity) >> >>Convolution is _not_ the way that real systems in the real world respond > >>to stimuli of any sort. Convolution is just a _mathematical operation_ >>that _approximates_ what real systems do. Sometimes it even does it > well. > > Convolution *is* the way many real systems behave. Its not some arcane > mathematical trick. I agree. > Its the direct mathematical representation of the > underlying physical process. I disagree. The _direct_ mathematical representation of a mass being acted on by a force in one dimension in a Newtonian frame is a differential equation, with the order depending on whether you're interested in the mass's velocity or position. Note here that I am drawing the same distinction between 'respond' and 'behave' that I do between convolution (which _is_ an arcane mathematical construct, but it's a damn useful one and not one I sneer at), and a nice representative differential equation. The differential equation models the reality. The convolution integral solves the differential equation. That's two different things. If you don't believe me, I'll post a convolution integral, and you can reply with the differential equation -- including variable names -- that it was generated from. > How well it fits reality is generally a > matter of how much the system is affected by second order effects. This > is pretty much like any other area of science and engineering. > >>All real systems are nonlinear. The convolution operation is one way to > >>implement a linear model of a system. Thus, the convolution operation >>does not model any real system with 100% accuracy. As a model, the >>convolution operation is only as good as the fit between its bedrock >>assumption of linearity and the system's actual conformity to linear >>behavior. > > You must absolutely loath the entire scientific education system. Actually not. My issue here is narrow, and it is that the notion that one is working with ideal models, rather than real systems, is not expressed, much less stressed. Few engineering students grasp that the model is not the reality, and many stumble as a consequence of this lack. > Almost > everything is taught as if it obeys relatively simple relationships, and > that's pretty much always a first order approximation. Often the higher > order elements are so small you can largely ignore them. If you want > accuracy, you'd better scrap Newton's laws of motion. If I want accuracy I'll use Newton's laws of motion, but if I'm going over 1% of the speed of light and I want 14-bit accuracy I'll damn well brush up on my relativity. > If you really want to complain about people being taught about stuff > like its an real accurate model, look at the real villans, like how > capacitors are taught. The number of engineers who treat them like they > are linear devices is truly sad. They demand that the latest silicon can > do A/D conversion at high speed with >16 bits precision, and then > surround them with tiny surface mount capacitors who's characteristics > are bizarrely funky. You merely find a specific example of my overall complaint to harp upon, where I harp upon the underlying problem. -- www.wescottdesign.com |