11  Signals and Systems

The heart’s electrical signal is a continuous voltage varying over time. A computer, however, stores only a list of numbers. Somewhere between the electrode on the skin and the value on a computer, the continuous voltage must be turned into 0s ans 1s.

An analog-to-digital converter, or ADC, is the hardware component that sits between the voltage sensor and the computer. It takes the analog signal and turns it into a digital bitstream (Figure 11.1).

Figure 11.1: The analog-to-digital pipeline. A continuous analog voltage \(v(t)\) (left) is first sampled — read at a fixed rate \(f_s\) and held between readings — then quantized — each held value rounded to one of \(2^n\) allowed levels — and finally stored as a digital bitstream of \(n\)-bit numbers. Sampling and quantization together constitute the ADC. Drawn by scripts/gen_signals_pipeline.py.

Continuous Signals

A continuous signal is a real number that can take on any value within a range and is defined at every time \(t\) (Figure 11.2).

Figure 11.2: A continuous-time signal \(x(t)\). It has a value at every instant, so between any two moments there is always another moment at which the signal is defined. The notation takes round brackets, and \(t\) is a real number. Generated by images/01_measuring/11_signals_systems/gen_l5_ct_dt.py.

Discrete Signals

A discrete signal is a sequence of numbers, each representing the signal’s value at a specific point in time. Unlike a continuous signal, it is defined only at these discrete instants, typically spaced by a fixed sampling interval \(\Delta t\) (Figure 11.3).

Figure 11.3: The same signal as Figure 11.2, sampled: a discrete-time signal \(x[n]\), defined only at the instants \(t = nT\) and spaced by the sampling interval \(T\). The underlying continuous signal is drawn in grey for comparison, but the discrete signal consists of the marked values alone. The notation takes square brackets, and \(n\) is an integer. Generated by images/01_measuring/11_signals_systems/gen_l5_ct_dt.py.

To understand how to correctly sample a continuous signal, let’s review sinusoids. This is important because nearly any signal can be broken into a sum of sinusoids.

Why sinusoids?

Signals occur in unlimited variety. However any signal, however complicated, can be represented as a sum of sinusoids.

Let’s consider the signal in our earlier example. It rises to unequal peaks and has a small ripple riding on a slow swing — but it is nothing more than two sine waves added together: one of amplitude 1 at 2 Hz, and a smaller one of amplitude 0.45 at 5 Hz, shifted in phase by 0.8 rad (Figure 11.4).

import numpy as np

t = np.linspace(0, 1.0, 2000)

first  = 1.00 * np.sin(2 * np.pi * 2.0 * t)              # A = 1.00, f = 2 Hz
second = 0.45 * np.sin(2 * np.pi * 5.0 * t + 0.8)        # A = 0.45, f = 5 Hz, phi = 0.8

signal = first + second                                  # the continuous signal
Figure 11.4: The continuous signal of Figure 11.2 is the sum of two sinusoids. Top: a 2 Hz sine of amplitude 1. Middle: a 5 Hz sine of amplitude 0.45, shifted by \(\phi = 0.8\) rad. Bottom: their sum. Generated by images/01_measuring/11_signals_systems/gen_l5_ct_dt.py.

Real signals are typically approximated by many sinusoids, and a square wave needs infinitely many — its sharp corners are assembled from ever-higher harmonics. However, the general principle holds: the decomposition of a signal into sines and cosines of different frequencies is possible. You will learn this, and it’s mathmatical form when you discuss Fourier analysis in other classses. It is why sinusoids are the natural building blocks of signal processing.

The Sinusoid

An angle can be measured in degrees, but for waves the natural unit is the radian. One radian is the angle subtended at the centre of a circle by an arc exactly one radius long. Measuring an angle in radians therefore amounts to counting how many radii of arc it cuts off: an angle \(\theta\) subtends an arc of length

\[s = r\,\theta .\]

Rearranged, \(\theta = s/r\) is a length divided by a length, so an angle in radians is a pure number and carries no units of its own. Go once around the circle and the arc becomes the entire circumference, \(s = 2\pi r\), so the angle of a full revolution is

\[\theta = \frac{2\pi r}{r} = 2\pi .\]

The radius cancels, which is why a full turn is \(2\pi\) radians for a circle of any size. Figure 11.5

Figure 11.5: One radian is the angle whose arc is one radius long. A string exactly one radius long is lifted off the circle and bent — without changing its length — until it lies along the rim; the angle it subtends at the centre is one radian, and a grey spoke marks where it ends. Six such pieces almost close the circle, and a short final piece of \(0.283\,r\) completes it, so the circumference is \(2\pi = 6.283\) radii and a full turn is \(2\pi\) radians. Generated by images/01_measuring/11_signals_systems/radian_demo.py.

A sine wave is the height of a point travelling around a circle (Figure 11.6). The argument of the sine is therefore an angle: one full lap is \(2\pi\) radians.

Figure 11.6: Where a sine wave comes from. A point travels counter-clockwise around the circle on the left, and its height traces the wave on the right. The horizontal axis is the angle \(\theta\) the point has swept, measured in radians: one full lap of the circle is \(2\pi\) radians and one full period of the wave, so the two are the same statement in different words. No time appears anywhere — that comes next. Generated by images/01_measuring/11_signals_systems/sine_nomenclature.py.

The Sinusoid V(t)

A signal as a function of time can be written as

\[v(t) = A \sin(\omega t),\]

where:

  • \(A\) — the amplitude
  • \(\omega\) — the angular frequency, in radians per second

The angle inside the sine advances at \(\omega\) radians per second, and one full cycle takes \(T\) seconds and covers \(2\pi\) radians, so

\[\omega = \frac{2\pi}{T}.\]

The period is the reciprocal of the frequency, \(T = 1/f\), so this can equally be written

\[\omega = 2\pi f,\]

and the wave itself becomes

\[v(t) = A \sin(2 \pi f t).\]

Finally, the wave can be shifted left or right by \(\phi\), the phase:

\[v(t) = A \sin(2 \pi f t + \phi),\]

Move the sliders to see what each of the three numbers does to the wave. Watch the period \(T = 1/f\) reported beneath the plot as you change the frequency.

Sampling Rate

The rate at which samples are taken is the sampling rate (or sampling frequency) \(f_s\), measured in samples per second, called hertz. It sets the spacing between samples, the sampling interval:

\[\Delta t = \frac{1}{f_s}.\]

At \(f_s = 500~\text{Hz}\) an EKG yields one voltage reading every \(\Delta t = 1/500 = 2~\text{ms}\), so a 10-second strip becomes a list of \(500 \times 10 = 5000\) numbers.

Let’s consider how our EKG is recorded at different sampling rates

Figure 11.7 records the same four beats at different sampling frequencies. At 500 Hz the dots are dense enough to draw the waveform on their own. Going down the panels, the samples thin out: the T wave coarsens, then the P wave becomes a couple of dots, and by 25 Hz the QRS complex survives as one or two samples.

Look at the R peaks in the bottom panel. They are recorded at 1.05, 1.01, 0.76 and 0.45 mV, a spread of 603 µV, from the same patient, in the same three seconds. Below a sufficient rate, what you measure depends on where the samples happen to land.

Figure 11.7: The same four beats of one recording, sampled at four rates. A single sweep line crosses all four panels, so every panel is sampling the same instant at every moment and any difference between them is the rate alone. Recording: PTB Diagnostic ECG Database (PhysioNet), record patient104/s0306lre, lead II, healthy control. Generated by images/01_measuring/11_signals_systems/sampling_rates_panels.py.

Min sampling rate

Clearly how frequently we sample matters! But how frequent is enough? Again let’s return to the the fact that a signal can be approximated by a series of sine waves.

Sampling too slowly produces an effect more serious than the loss of fine detail: a high-frequency sinusoid becomes indistinguishable from a lower-frequency one. This effect is called aliasing and it is seen more clearly in Figure 11.8.

Figure 11.8: Aliasing. The same \(3~\text{Hz}\) sine wave (blue) sampled at four rates. At 10 and 4 samples per cycle the recording is faithful. At exactly 2 samples per cycle, the Nyquist rate, the samples alternate in sign at the correct frequency but no longer determine the wave: here they read ±0.84 against a true peak of 1, and had they fallen on the zero crossings every sample would read 0. Below it, at 1.33 samples per cycle, the samples fall on a slower curve entirely (orange dashed): a \(3~\text{Hz}\) signal is recorded as \(1~\text{Hz}\), and once digitized nothing distinguishes it from a genuine \(1~\text{Hz}\) wave. Generated by images/01_measuring/11_signals_systems/sampling_rate_sweep_hz.py.

The recorded frequency of an undersampled component is set by the signal frequency and the sampling rate together. A component of true frequency \(f_\text{signal}\) is recorded as

\[f_\text{alias} = \left| f_\text{signal} - n f_s \right|,\]

where:

  • \(f_\text{signal}\) — the true frequency of the component
  • \(f_s\) — the sampling rate
  • \(n\) — the integer that brings the result into the range 0 to \(f_s/2\), which is the nearest integer to \(f_\text{signal}/f_s\)

The bottom panel of Figure 11.8 shows this. There the \(3~\text{Hz}\) wave is sampled at 1.33 samples per cycle, so \(f_s = 4~\text{Hz}\) and the nearest integer to \(3/4\) is \(n = 1\), giving \(f_\text{alias} = |3 - 4| = 1~\text{Hz}\). Note that the alias is a real, well-defined frequency, which is what makes undersampling so problematic.

The Nyquist–Shannon sampling theorem

Let the highest frequency present in the signal be \(f_\text{max}\). To capture this signal accurately, the sampling rate must be greater than twice this maximum frequency:

\[f_s > 2 f_\text{max}.\]

The threshold \(2 f_\text{max}\) is the Nyquist rate. Sampling below it causes aliasing, in which high-frequency components masquerade as lower-frequency ones. Sampling exactly at it is not sufficient either: two samples per cycle can land anywhere on the wave, including on every zero crossing, in which case every sample reads zero.

This is visually demonstrated in the following example.

Figure 11.9: One 3 Hz sine wave, sampled at a rate that is walked steadily down from 10 samples per cycle to barely one. Top: the recorded samples (orange) against the true wave (blue), with two bars comparing the sampling interval \(\Delta t = 1/f_s\) to half the signal’s period \(T_0/2\); faithful sampling requires the sampling interval to be strictly shorter. Bottom: the frequency actually recorded, against sampling rate, with the Nyquist rate \(2f_0 = 6\) Hz marked by a dashed line. To the right of it the recording is faithful: 3 Hz in, 3 Hz out. To the left it fails: at \(f_s = 4.5\) Hz the wave is recorded as 1.5 Hz, and by \(f_s = 3.2\) Hz a 3 Hz signal is recorded as a slow 0.2 Hz drift that was never present. Generated by images/01_measuring/11_signals_systems/sampling_rate_sweep_hz.py.
NoteKey result: the Nyquist–Shannon sampling theorem

A signal that contains no frequency higher than \(f_{\max}\) is completely and faithfully captured by its samples if the sampling rate satisfies

\[\boxed{\; f_s > 2\,f_{\max} \;}\]

Sampling below twice the highest frequency present causes that frequency to alias into a lower one, corrupting the record irreparably; sampling at exactly twice it leaves the amplitude and phase undetermined. The threshold \(2 f_{\max}\) is the Nyquist rate, and half the sampling rate, \(f_s/2\), is the Nyquist frequency: a given ADC can represent only frequencies below it (Shannon 1949; Baraniuk and contributors 2009).

ImportantKnowledge check

A nerve signal contains frequencies up to \(3~\text{kHz}\). (a) What must the sampling rate exceed to capture it without aliasing? (b) If you instead sampled it at \(4~\text{kHz}\), what would happen to a \(3~\text{kHz}\) component?

  1. Nyquist requires \(f_s > 2 f_{\max} = 2 \times 3~\text{kHz} = \mathbf{6~kHz}\); you must sample faster than 6000 times a second.

  2. At \(f_s = 4~\text{kHz}\) the Nyquist frequency is only \(2~\text{kHz}\), below the \(3~\text{kHz}\) component, so that component aliases: it appears as a false frequency \(|3 - 4| = \mathbf{1~kHz}\), a signal that was never there.

Exercises

TipProblem 1

An engineer must digitize three signals: an EKG with content up to \(150~\text{Hz}\), a phonocardiogram (heart sounds) up to \(800~\text{Hz}\), and an audio recording up to \(20~\text{kHz}\).

  1. State the Nyquist rate for each, which the sampling rate must exceed.
  2. The AHA specifies \(500~\text{Hz}\) for a diagnostic ECG. By what factor does that exceed the Nyquist rate, and why is having margin desirable?

  1. Nyquist requires \(f_s > 2 f_{\max}\): for the EKG, \(f_s > 2(150) = \mathbf{300~Hz}\); for the phonocardiogram, \(f_s > 2(800) = \mathbf{1600~Hz}\); for the audio, \(f_s > 2(20{,}000) = \mathbf{40~kHz}\).

  2. \(500~\text{Hz}\) is \(500/300 = \mathbf{1.67\times}\) the Nyquist rate. Real signals are not perfectly band-limited and real anti-aliasing filters cannot cut off infinitely sharply, so sampling a comfortable margin above \(2 f_{\max}\) keeps any residual high-frequency content from aliasing into the band of interest.

TipProblem 2

A pure \(900~\text{Hz}\) tone is sampled at \(f_s = 1000~\text{Hz}\).

  1. Is the Nyquist criterion satisfied for this tone? What is the Nyquist frequency?
  2. Compute the apparent (alias) frequency using \(f_{\text{apparent}} = |f - n f_s|\) with \(n\) the nearest integer to \(f/f_s\).
  3. In one sentence, say what a listener or a computer would wrongly conclude about the tone.

  1. The Nyquist frequency is \(f_s/2 = \mathbf{500~Hz}\). The \(900~\text{Hz}\) tone is above it, so the criterion is violated and the tone will alias.

  2. \(f/f_s = 900/1000 = 0.9\), nearest integer \(n = 1\), so \(f_{\text{apparent}} = |900 - 1(1000)| = \mathbf{100~Hz}\).

  3. The \(900~\text{Hz}\) tone would be recorded and reproduced as a spurious \(100~\text{Hz}\) tone — a low pitch that was never present in the original.

TipProblem 3

A wheel with a single painted spoke rotates forward at exactly \(18~\text{revolutions per second}\). It is filmed by a camera running at \(f_s = 24~\text{frames per second}\) (so each frame is one sample of the spoke’s angle).

  1. Treating the rotation rate as a “frequency,” is it above or below the camera’s Nyquist frequency?
  2. The apparent rotation rate is \(|f - n f_s|\) with \(n\) the nearest integer to \(f/f_s\). Compute it.
  3. Between frames the spoke advances three-quarters of a turn. What does the wheel appear to do on film — and in which direction?

  1. The Nyquist frequency is \(f_s/2 = 12~\text{rev/s}\). The wheel spins at \(18~\text{rev/s}\), above it, so its motion aliases.

  2. \(f/f_s = 18/24 = 0.75\), nearest integer \(n = 1\), so the apparent rate is \(|18 - 1(24)| = \mathbf{6~rev/s}\).

  3. Between frames the spoke advances three-quarters of a turn forward — but the eye reads the smallest motion between two frames, and a three-quarter turn forward looks identical to a quarter turn backward. So the wheel appears to rotate slowly backwards at about \(6~\text{rev/s}\) — the wagon-wheel effect. (In signed terms the alias is \(f - n f_s = 18 - 24 = -6~\text{rev/s}\); the negative sign is the backward direction.)

References

Baraniuk, Richard, and contributors. 2009. Signals and Systems. OpenStax CNX / Rice University. https://eng.libretexts.org/Bookshelves/Electrical_Engineering/Signal_Processing_and_Modeling/Signals_and_Systems_(Baraniuk_et_al.)/10\%3A_Sampling_and_Reconstruction.
Shannon, Claude E. 1949. “Communication in the Presence of Noise.” Proceedings of the IRE 37 (1): 10–21. https://doi.org/10.1109/JRPROC.1949.232969.