Audio Graph Introduction

An Introduction to Representing Data With Sound


Sonification:

Sonification is a term used to describe the translation of an abstract data set into an auditory (sound) form. While there are many methods of accomplishing this task, the process discussed here is to convert the Y axis to pitch, and the X axis to time.

The graph's X axis is always represented by time in the sonified graphs on this page. The Y axis of the data point is represented by the pitch of the piano instrument. The first and second derivatives of the data are represented with a drum beat. The frequency of the beat represents the slope (so the greater the slope, the more frequent the beat.) The second derivative is represented by the pitch of the drum beat, a low pitch indicates positive curvature (concave) and a high pitch indicates negative curvature (convex); Linear data is indicted with a medium pitch.

Negative data values (or Y axis values below a set-point) can be represented by changing the voice of the instrument; say from a piano to harpsichord or other tonal quality.

Here is an example of an audio graph of the function Sin(x). In this graph, the data, first derivative, and negative auditory indicators can be heard.
Download

Error bars on data sets can be represented by a white noise indicator, where loudness indicates the size of the error bar. Another method to indicate error on sonified data would be to introduce higher-order harmonics into the instrument's voice, thus delocalizing the pitch.

Here is an example of data with increasingly large error bars:
Download

When sonifying data, it is important to reduce the auditory load to a manageable level. Listeners unfamiliar with the process may find only the pitch vs. time representation useful.

The negative (or set-point) indicator however has been found to be a very useful and a low auditory load indicator. It provides a good broad sense of the data (are the data above or below a certain level of interest) without producing auditory clutter.

The derivative information has been found to not be helpful in more complicated graphs where the focus tends to center on relative height and number of maxima and minima. Only when subtle characteristics in the slope of the data, and for carful analysis of rates of change, have the derivative markers been found most helpful. Data error bar indication is a subtle feature that is used less frequently.

An introductory set of sonified graphs of increasing complexity can be found on the Sonitypes page.

Applications:


One of the simplest applications of sonification is to represent data commonly represented in X, Y plots. The necessity for this method stems from the inability of a large number of people (due to visual imparments, technological considerations or the desire to access several data sets simultaneously) to access data represented in a pictorial form.

Method of Audio Graph Construction and Presentation:


These graphs were produced by the following method:

A data set and visual graph were constructed with the Kaleidagraph Program.
Screen shot of the Kaliedagraph 
Program

Next the data set was sonified and converted to an SLG format text file with the Data Reader program.
Screen 
shot of the Data Reader program.

Finally, the SLG file was read in by the MIDIGraphy program, checked for correct sonification, and converted to MIDI and QuickTime formats.
Screen shot of the MIDIGraphy program.

However, dynamic and real-time graphing methods are generally preferred and can be used both on the Web, such as may be heard on this demonstration page for AudioPlots which employs ActiveX controls. or with a stand-alone program as can be heard in the Accessible Graphing Calculator produced by ViewPlus Technologies.

Examples:


Below is a list of graphs which demonstrate data sonification. Each link is to a page where graphs are presented in pictorial and auditory formats.

Math Functions:

Example of a graph with negative values:

  • Download

    Physics Graphs:

    Example of data with error bars: [Note: You will hear horizontal (X-axis) tick marks to help place the horizontal data. These should not be confused with the vertical (Y-axis) indicators in the previous example.]
    Download


    If you have any questions about the research study and/or specific procedures, please contact:
    Steven Sahyun
    Physics Department
    University of Wisconsin - Whitewater
    800 W. Main St.
    Whitewater, WI 53190
    USA


    Substantial portions of this page were originally posted for the Science Access Project, Oregon State University, 1998. Funding by NSF Grant #HRC-9982456
    Steven Sahyun


    Back to Steven Sahyun's Home Page Last modified April 22, 2013.