Using Multimedia and the Web to Teach the Theory of Digital Multimedia Signals

Abstract

This paper is about using multimedia via the World Wide Web in a course that teaches the theory behind digital audio and video. New course materials accessible via the Web are now used by students to explore demos that were presented in class.

At Georgia Tech (and soon Rose-Hulman) a relatively new course that teaches the fundamentals of discrete signal processing (DSP) to sophomores is now required of computer engineering majors. Because DSP is involved in every aspect of multimedia information signals (coding, transmission, storage, playback, etc.) the numerous in-class demos and all of the labs relate to the creation or analysis of sounds or images via computer. The course is structured so that the classroom time is used to explain theory, which is then implemented in lab to explore a given concept (e.g., sampling rates) or carry out a small design (e.g., write a MATLAB program to produce a sequence of sinusoidal waveforms that when sent to a speaker will play a song such as Ramblin' Wreck).

Traditionally, reading and homework assignments provide a link between lectures and labs. In this course the lecture/lab gap is closed via follow-up demos that are run by the students via the Web. Labs are assigned that require the students to run demos that illustrate and reinforce concepts introduced in the lecture. The homework assignments are designed to help the students to make the transition from the overview/demo mode to the implementation/lab mode. This leads them to implement something via MATLAB to test their answer.
In the spirit of presenting via the Web, this paper is also a Web document which can be found at http://www.ee.gatech.edu/research/DSP/courses/ee2200/fie95/FIE1995paper.html.

Introduction

Multimedia is everywhere. Our students have probably seen it most of their lives. A few know how to use it effectively, but until recently computer engineering students had to wait until their junior or senior year to learn the theory behind coding, storage and playback of multimedia information signals. Now Computer Engineering students at Georgia Tech (and soon at Rose-Hulman) are learning the theory behind multimedia information signals during their sophomore year, even before taking circuits! This paper gives some background information about the sophomore course Introduction to Discrete Time Signal Processing and shows how multimedia via the Web is used to teach it. Since this paper is on the FIE 1995 CD-ROM it is an example of using multimedia to present a paper. If you are viewing the electronic version via a Web browser such as Mosaic or NetScape, feel free to click on the links to explore this paper in a way a student would explore our class.

Background

An overview of the sophomore DSP course is given in Listen and See with DSP, by Jim McClellan and Ron Schafer, presented in a special session at the International Symposium on Circuits and Systems, Seattle WA, May 1-3, 1995. Although papers from the session were not printed in the conference proceedings, you can click here and see an electronic version of the paper. Here is a PostScript version so you can print a copy of your own. This section is a summary of that paper with links to the sections being summarized.

Instead of teaching circuit theory as the lead-off course into the traditional signals/systems sequence, the sequence has been re-ordered to require an introductory course in applied signal processing, concentrating on digital signals. This course is followed by a conventional circuits course and then a course on transforms and Fourier analysis.

An essential part of the first course is a computer-based lab where students listen to sounds and see results of image processing in order to develop intuition for frequency content and filtering. This intuitive connection is central to our approach; it is our attempt to have students relate the physical attributes for certain signals to the mathematical formulas manipulated in system theory. The fact that the level of the course is basic and the processing methods intuitive is a strong motivation for students to take an active role in the lab - their processing systems actually do interesting things and the results can be seen and/or heard.

The objective of this course is to show students with strong interests in computers the role that mathematical system theory can play in the development of computer applications and products. We advocate that in the future this same course sequence be adopted for all EE majors where the objective would be to capture their attention for applications of computer processing to modern engineering systems.

The style of the course is "hands-on" with computers being used as an integral part of the classroom and lab environment. In-class demonstrations are used to relate simple sinusoids to the sounds they make. One lab has them synthesize a short musical passage such as the Georgia Tech "Ramblin' Reck" song. Here are some examples from the Fall of 1994, and here are samples from winter 1995. Another lab deals with imaging filtering. Though restricted to 1-D filtering of the rows and columns, the students observe interesting effects with simple low- and high-pass filters.

The following topics are covered in the course:

Definitions of signals and systems.
Introduction to a powerful software tool: MATLAB in our case. Students are assumed to have enough programming experience to absorb MATLAB syntax on the fly.
Review of some relevant mathematics: vector/matrix notation, complex numbers.
Sinusoidal signals: amplitude, phase and frequency.
Representing sinusoids with phasors.
Synthesizing sounds with sums of sinusoids.
Frequency content: combining sinusoids; harmonics, AM and FM signals.
Sampling, aliasing and quantization.
Filtering: the concept of smoothing data.
Block level description of systems.
FIR Filtering: generalize weighted averaging filters.
Recursive Filtering: difference equations with feedback terms.
Frequency response of simple filters: magnitude and phase response.
Simulation of dynamic response in the time domain; impulse response.
z-transform analysis with polynomials.
Synthesizing sounds via narrowband recursive filters.

The Web

The most effective way to tell you about the web pages is to show you. We'll present an overview of the goals of the pages here along with a few pictures (so those who don't have the CD-ROM will have some clue as to what is going on), but the best way to understand what we are doing is to play with the Web pages yourself.

Lost in the Web
Our first goal was to be sure the Web didn't provide yet another way to lose students (some students have been known to start wandering the Web and be lost for hours at a time) so we have provided an overview page for the whole course, a Master Table of Contents, and overviews of individual parts such as Labs and Demos

Click on any of the above figures to see the functioning pages.

The In-class Demos
The in-class demos are effective at showing an overview of a topic. Simply handing the demo to a student doesn't always work since the professor often gives additional information during the demo. We used the Web to supply this extra information in demos such as Tuning Forks, Reading Sinusoids, Rotating Phasors, Reading Spectrograms, and Mapping from the Z-plane to Frequency Response.

Here is a brief description of each:

Tuning Forks In class, the equation of the frequency of a tuning fork is derived. This demo shows how the mass and type of material affect the pitch of the tuning fork. Some QuickTime® movies of various tuning forks being played allow the student to see the different forks and hear the different pitches. Plots of the sound of one fork is also shown.

Reading Sinusoids This is a MATLAB program that drills the student on how to read a sinusoid. It uses the MATLAB graphic user interface so the student who hasn't learned MATLAB (yet) won't get lost in the details of MATLAB.

Rotating Phasors Is there a better way to learn about a rotating phasor than to watch one rotate? This uses some MATLAB generated QuickTime® movies to show phasors in action.

Reading Spectrograms Spectrograms are a great way to see what a signal is doing. This demo introduces the spectrogram and allows the student to both see and hear sounds.

Mapping from the Z-plane to Frequency Response The relationship between poles and zeros and the frequency response comes alive with this demo which flies the student by the mountains of a 3D pole-zero plot while they watch the unit circle being lifted from the plot and unrolled into a frequency response.

When these demos are presented in class they allow some student interaction. Indeed the student can influence the demo, but only through the professor. The demos also tend to hide many implementation details. This is good when the student is first exposed to a new topic, but often the goal of the demo is to motivate a connection between an idea and the concept that underlies it. An important example is the link between the frequency spectrum of a signal and the sound perceived when we hear it played through a speaker. This requires that the student be able to experiment, adjust parameters and even make mistakes. Putting the demos on the Web allows more interaction. Letting the students run the MATLAB code that generated the demos allows even more interaction, however the lab is where even more learning occurs.

The Labs
The computer based labs fill in the other end of the spectrum. Here the student sees the details and is directly interacting with the computer and the problem to be solved. Successful completion of the lab occurs when the underlying concepts are understood.

Here is a summary of some of the labs:

Introduction to MATLAB Explore MATLAB and learn how to plot sinusoids.

Synthesis of Sinusoidal Signals In this two week lab we synthesize waveforms composed of sums of sinusoidal signals, sample them, and then reconstruct them for listening. This lab requires the synthesis of:

sine waves at a specific frequency and
sinusoids that mimic Ramblin' Wreck.

Complicated Sinusoidal Signals In this lab, we synthesize more complicated sinusoidal waveforms composed of sums of sinusoidal signals,each with a different frequency (e.g. harmonics) . Phasors are used to generate these complicated waveforms.

Frequency Modulated (FM) Signals The lab starts with generating simple FM signals such as linear chirps and ends with synthesizing a clarinet defined by its the amplitude and frequency envelopes.

FIR Filtering of Sinusoidal Waveforms For this lab we define an FIR filter as a discrete-time system that converts an input signal x[n] into an output signal y[n] whose values are given by the formula:

We then look at time and frequency plots of various signals before and after filtering to learn which coefficients cause smoothing (e.g. averaging three points) and which cause more peaks to appear (e.g. difference between adjacent points). Then we listen to the signals to discover how their sounds relate to plots.
In a later lab we look a images before and after filtering to learn what coefficients cause blurring or sharpening of an image.

Conclusions

In creating our new sophomore-level course on discrete-time signals and systems, we were motivated initially by a desire to create an introductory electrical/computer engineering course that would be more motivating than the traditional linear circuits course. Specifically, we wanted to motivate students by showing them how mathematics and physics that they have learned in freshman courses can be combined with modern computer technology to understand the underlying principles of multimedia signals and systems. Our course makes use of traditional teaching tools such as lectures, problem sets, and written notes (soon to be published by Prentice Hall). Somewhat less traditional is our extensive use of laboratory exercises carried out using state-of-the-art computer software.

Perhaps most interesting is the fact that our course focuses on the very heart of the multimedia explosion that is currently underway. Therefore it is eminently reasonable that we begin to apply multimedia technology in the teaching of the course itself. Toward this end, we have begun to explore how CD-ROMs and World Wide Web technology can be used effectively in teaching about multimedia signals and systems. In this paper we have shown examples of some of our early activities in this emerging area of education. Our success so far makes us highly optimistic that modern information technology will revolutionize education in engineering and in many fields.

URL = Rose:Desktop Folder:FIE1995paper.cd
Last updated 21:27 on 1-Jun-1995

Tuning Forks		In class, the equation of the frequency of a tuning fork is derived. This demo shows how the mass and type of material affect the pitch of the tuning fork. Some QuickTime® movies of various tuning forks being played allow the student to see the different forks and hear the different pitches. Plots of the sound of one fork is also shown.
Reading Sinusoids		This is a MATLAB program that drills the student on how to read a sinusoid. It uses the MATLAB graphic user interface so the student who hasn't learned MATLAB (yet) won't get lost in the details of MATLAB.
Rotating Phasors		Is there a better way to learn about a rotating phasor than to watch one rotate? This uses some MATLAB generated QuickTime® movies to show phasors in action.
Reading Spectrograms		Spectrograms are a great way to see what a signal is doing. This demo introduces the spectrogram and allows the student to both see and hear sounds.
Mapping from the Z-plane to Frequency Response		The relationship between poles and zeros and the frequency response comes alive with this demo which flies the student by the mountains of a 3D pole-zero plot while they watch the unit circle being lifted from the plot and unrolled into a frequency response.

Introduction to MATLAB	Explore MATLAB and learn how to plot sinusoids.
Synthesis of Sinusoidal Signals	In this two week lab we synthesize waveforms composed of sums of sinusoidal signals, sample them, and then reconstruct them for listening. This lab requires the synthesis of: sine waves at a specific frequency and sinusoids that mimic Ramblin' Wreck.
Complicated Sinusoidal Signals	In this lab, we synthesize more complicated sinusoidal waveforms composed of sums of sinusoidal signals,each with a different frequency (e.g. harmonics) . Phasors are used to generate these complicated waveforms.
Frequency Modulated (FM) Signals	The lab starts with generating simple FM signals such as linear chirps and ends with synthesizing a clarinet defined by its the amplitude and frequency envelopes.
FIR Filtering of Sinusoidal Waveforms	For this lab we define an FIR filter as a discrete-time system that converts an input signal x[n] into an output signal y[n] whose values are given by the formula: We then look at time and frequency plots of various signals before and after filtering to learn which coefficients cause smoothing (e.g. averaging three points) and which cause more peaks to appear (e.g. difference between adjacent points). Then we listen to the signals to discover how their sounds relate to plots. In a later lab we look a images before and after filtering to learn what coefficients cause blurring or sharpening of an image.