Generating a continuous random variable with an arbitrary distribution [DSP]

Prev: Cross correlation in signal analysis
Next: Generating a continuous random variable with an arbitrarydistribution

From: Randy Yates on 18 Apr 2008 22:13

I recently examined the feasibility of generating noise for testing a
communication system and came to the disappointing conclusion that
generating a digital signal and then converting it to analog via an ADC
will never be capable of generating a stationary, continuous random
process (signal) with an arbitrary distribution.

Here's my reasoning - please correct me if I'm wrong.

Let's assume the "digital" samples are infinite resolution. This is the
best case since introducing a finite resolution limits the problem even
further. Let the digital samples be distributed with some arbitrary
(desired) pdf f(x).

When that signal is then converted to analog via an ADC, the action of
the reconstruction filter will be to scale and sum some number of the
digital samples together. The scalings and the number of samples summed
will probably both be a function of time. Then, in general, by the
Central Limit Theorem, these samples will tend toward Gaussian. Further,
the output process probably won't even be stationary. The precise
characterization depends heavily on the type of reconstruction filter.

An interesting situation occurs when the reconstruction filter is the
idea lowpass. At the sample points (t = n*T), the analog output consists
of just one digital sample, so at the sample points the analog output
will have a pdf f(x). However, between the sample points, various
numbers and amounts of the original digital samples will be scaled and
summed, and almost certainly the analog output at exactly halfway
between the sample points will be very Gaussian-like.

If we use a 1-bit converter and a 1-bit PN sequence (let's say it's
perfectly uncorrelated and uniformly distributed with

f(x) = 0.5 * delta(x-1) + 0.5 * delta(x+1),

and let's use the ideal lowpass filter for reconstruction. Then the
analog output will be particularly pathological, with a simple two-level
discrete PDF at the sample points, something very close to Gaussion
halfway between the sample points, and something in between in between
the sample points.

No? Yes? I thought this was a VERY interesting problem and I am
surprised (once it was asked) why it really isn't treated in the texts
(Papoulis, Leon-Garcia, etc.). The texts discuss the PSD of such
systems, but not the distribution of the resulting continuous-time
random process.

Comments?
--
% Randy Yates % "My Shangri-la has gone away, fading like
%% Fuquay-Varina, NC % the Beatles on 'Hey Jude'"
%%% 919-577-9882 %
%%%% <yates(a)ieee.org> % 'Shangri-La', *A New World Record*, ELO
http://www.digitalsignallabs.com

From: Randy Yates on 18 Apr 2008 23:13

Randy Yates <yates(a)ieee.org> writes:

> I recently examined the feasibility of generating noise for testing a
> communication system and came to the disappointing conclusion that
> generating a digital signal and then converting it to analog via an
> ADC

Doh! Substitue "DAC" for "ADC" in this entire post. Sorry!
--
% Randy Yates % "I met someone who looks alot like you,
%% Fuquay-Varina, NC % she does the things you do,
%%% 919-577-9882 % but she is an IBM."
%%%% <yates(a)ieee.org> % 'Yours Truly, 2095', *Time*, ELO
http://www.digitalsignallabs.com

From: glen herrmannsfeldt on 18 Apr 2008 23:31

Randy Yates wrote:

> I recently examined the feasibility of generating noise for testing a
> communication system and came to the disappointing conclusion that
> generating a digital signal and then converting it to analog via an ADC
> will never be capable of generating a stationary, continuous random
> process (signal) with an arbitrary distribution.

> Here's my reasoning - please correct me if I'm wrong.

> Let's assume the "digital" samples are infinite resolution. This is the
> best case since introducing a finite resolution limits the problem even
> further. Let the digital samples be distributed with some arbitrary
> (desired) pdf f(x).

I suppose I agree in the general case, but maybe not in specific cases.

For example 1/f noise falls of as, well, 1/f, and so could have an
effect at fairly large frequencies.

But if one, for example, wants to test the effects of noise on an audio
system then some reasonable factor above 20kHz should be enough.
Resolution might be important, but 24 bits shouldn't be too far off.
(What is state-of-the-art in DAC's?) One should sample fast enough
that the effects of the filter aren't too significant.

-- glen

From: Jerry Avins on 18 Apr 2008 23:31

From: dbd on 19 Apr 2008 04:50

On Apr 18, 7:13 pm, Randy Yates <ya...(a)ieee.org> wrote:
> I recently examined the feasibility of generating noise for testing a
> communication system and came to the disappointing conclusion that
> generating a digital signal and then converting it to analog via an ADC
> will never be capable of generating a stationary, continuous random
> process (signal) with an arbitrary distribution.
>
> Here's my reasoning - please correct me if I'm wrong.
>
> Let's assume the "digital" samples are infinite resolution. This is the
> best case since introducing a finite resolution limits the problem even
> further. Let the digital samples be distributed with some arbitrary
> (desired) pdf f(x).
>
> When that signal is then converted to analog via an ADC, the action of
> the reconstruction filter will be to scale and sum some number of the
> digital samples together. The scalings and the number of samples summed
> will probably both be a function of time. Then, in general, by the
> Central Limit Theorem, these samples will tend toward Gaussian. Further,
> the output process probably won't even be stationary. The precise
> characterization depends heavily on the type of reconstruction filter.
>
> An interesting situation occurs when the reconstruction filter is the
> idea lowpass. At the sample points (t = n*T), the analog output consists
> of just one digital sample, so at the sample points the analog output
> will have a pdf f(x). However, between the sample points, various
> numbers and amounts of the original digital samples will be scaled and
> summed, and almost certainly the analog output at exactly halfway
> between the sample points will be very Gaussian-like.
>
> If we use a 1-bit converter and a 1-bit PN sequence (let's say it's
> perfectly uncorrelated and uniformly distributed with
>
> f(x) = 0.5 * delta(x-1) + 0.5 * delta(x+1),
>
> and let's use the ideal lowpass filter for reconstruction. Then the
> analog output will be particularly pathological, with a simple two-level
> discrete PDF at the sample points, something very close to Gaussion
> halfway between the sample points, and something in between in between
> the sample points.
>
> No? Yes? I thought this was a VERY interesting problem and I am
> surprised (once it was asked) why it really isn't treated in the texts
> (Papoulis, Leon-Garcia, etc.). The texts discuss the PSD of such
> systems, but not the distribution of the resulting continuous-time
> random process.
>
> Comments?
> --
> % Randy Yates % "My Shangri-la has gone away, fading like
> %% Fuquay-Varina, NC % the Beatles on 'Hey Jude'"
> %%% 919-577-9882 %
> %%%% <ya...(a)ieee.org> % 'Shangri-La', *A New World Record*, ELOhttp://www.digitalsignallabs.com

I don't understand why so many people seem to lose track of their
basic knowledge when they contemplate noise in DSP. We know how to
calculate and measure quantization noise in ADCs and DACs. If
quantization noise is well under the signal level at frequencies of
interest, it doesn't represent any more problem for noise than for
determinate signals. Picking a 1 bit DAC may be a poor place to start,
but we can tell from an analysis of the quantization noise if it is a
problem.

Digital noise generators have finite power supplies that inherently
cause output clipping. Digital noise generators must use bandwidth
limiting filters to allow higher spectral levels while controlling
clipping. The noise generation process in digital noise generators has
inherent clipping because of the limited range of noise amplitude the
process can represented with. Now go back and substitute analog for
digital in the three previous sentences and you will have an accurate
description of our traditional analog noise generators. There are
enough limitations in the instrumentation that the digital question is
not a big one for the things we have been using noise generators for
(remember the GR1390?), if we remember to apply the DSP principles we
are used to here.

The real issue may be that we have actually believed the the revered
analog generators were doing something far beyond their actual
performance. It may be far beyond the capability of DSP to provide the
performance we can dream of (and believed we had) and knowledge of DSP
will not be a comfort here. It is also important to remember that no -
finite- sample of noise: analog, digital or ideal, accurately
represents the statistical measures of the distributions they are
drawn from.

DSP does allow a variety of parameters of noise generation to be
controlled precisely which is something that digital might do better.

Dale B. Dalrymple
http://dbdimages.com

| Next | Last
Pages: 1 2 3
Prev: Cross correlation in signal analysis
Next: Generating a continuous random variable with an arbitrarydistribution