1. Conclusion
    1. 10.
    2. Summary of Conclusions of Test Results
      1. 10.1.
      2. Triangle Pilot
      3. The Triangle pilot was useful in gaining experience in question development and initial auditory graph display analysis. From this test, the auditory graphing techniques was modified in two significant ways. First, in order to accentuate the curvature, a derivative tick mark was added as a drum beat where the tempo of the drum represented the magnitude of the graph’s slope. Secondly, the data was mapped to a chromatic scale rather than the previous linear scaling. The tonal quality of the sound was also modified, but this was a result of utilizing a MIDI to implement the sound files rather than from research findings. The testing method was also modified after analysis of the Triangle pilot. The testing was changed from a guided interview method to accessing the test via web pages.

      4. Web Pilot
      5. The web pilot was important to gauge student participation for a self guided test. Participation was not a problem as when the test was offered for token credit, more student subjects participated than were in attendance when the announcement of the test was mentioned in class. The web pilot demonstrated that the scripting method was adequate to display the auditory test and record subjects’ results. This pilot also displayed the inadequacies of the initial set of questions for a full comparative test. Thus, while the testing environment did not need to be modified, the questions used in the test were extended and reworked to provide a more complete comparison.

      6. Main Auditory Graph Test

      The Main Auditory Graph test demonstrated that student subjects with auditory graphs were able to perform at 80% the level of subjects given visual graphs. While this is a significant difference, it also demonstrates that subjects are able to use the graphs to answer many math and physics questions with very little self guided training.

      This test also demonstrated that blind subjects around the world could not only access the test, but could effectively complete and answer the questions. In addition they were able to do so at a level that exceeded the student subjects, and was not significantly different from local graduate students taking the test with auditory graphs. Their response rate was also greater than 80% the level of graduate subjects with visual graphs.

    3. Further Studies That Are Suggested By Test Results.
      1. 10.2.
      2. Assumptions and Additional Auditory Graph Techniques
      3. While the experiments performed were an adequate demonstration of an effective test with auditory graphs, there were many unanswered questions that arose. First, there were several assumptions as to the auditory representation. For example the piano tone was used for the data, a drum for the derivative, and the drum pitch were subjectively chosen by the author as a useful and convenient working model to begin with. There was no indication that these choices were necessarily the best ones to make. Therefore, future studies will want to make evaluations of these assumptions.

        As was noted by one subject, the concept of 0 needs to be well represented with the auditory graph. In the previous studies, all graphs were produced for positive y axis values, thus eliminating the necessity to work with negative values. In order to have a more general and flexible graphing method, 0 and negative values need to be well represented. All graphs in these studies have been single valued, single data sets. Methods for construction of multiple data set comparisons need development, as well as the display of multi-valued functions.

        Another important effect for further study is an analysis of training times on performance. The dramatic differences between the Web Pilot and the Main Auditory tests indicate that there is such an effect. Furthermore, there may be an effect that certain graphing techniques or indicators are only valid or useful under unique circumstances. Even if specific indicators are helpful, there is a question of whether or not they are desired by the user.

        Since these studies concentrated on very simple graphs, it is still unknown how complex can graphs be for the y axis to pitch representation to still work. As graphed data becomes more complex, there may be a preference for audification (directly representing the data values as the wave pattern, and then using that pattern to drive a sound source) or other data sonification methods.

        The auditory graphs in this study only allowed for limited control of the sound parameters. Playback rate, control on the investigation of data points by the user, the ability to listen to the sound forwards or backwards could all have an effect on graph comprehension.

        These points demonstrate the need for future research. However, is it even possible to carry out such experimentation? To that extent, a method for comparing auditory graphing was developed and pilot tested as the Auditory Preference Test.

      4. Auditory Preference Test Pilot
        1. 10.2.2.
        2. Overview
        3. In order to assess the effectiveness and desirability of various auditory graphing techniques, a test was developed that used a combination of pair-wise preference comparisons, graph identification questions, and Likert preference ratings. The preference questions were used to indicate which graphing styles subjects liked best, or thought were most useful. The graph identification questions were used to indicate which graphing styles has the highest rate of being answered correctly.

          This test was created not only to find better elements for the auditory graph displays, but also to test and evaluate an alternative method of auditory graph production. This alternate technique utilizes Microsoft’s ActiveX controls to create "live" graphs that would have the potential for greater user control, customization, and flexibility than prerecorded graphs could attain.

          The results of preference tests such as this one are able to directly guide software development incorporating auditory graphs. The Main Auditory Graph test demonstrated that basic auditory graphs could be effectively used for answering questions, but tests such as the Auditory Preference Pilot can be used discover what issues should be addressed for the best optimization of auditory graphs.

        4. Sample
        5. There were 13 subjects who participated in this study. As this was a web based test similar to the Web Pilot, one subject attempted the test from a remote location. Due to technical difficulties, the auditory graphs from the AudioPlot method were not active, thus the results for this subject were not included. There were twelve subjects that participated locally using the same computer, but at different times. These subjects were solicited primarily due to their proximity to the research location and included five advanced undergraduate physics students, two graduate science and math education students, and three employees of the toxicology department. These subjects were chosen because they had not been involved in previous auditory graph research which was an attempt to reduce bias due to familiarity with previous auditory graphing techniques.

        6. Data Collection
        7. Subjects were invited to an office with a desk computer with a web browser displaying the test’s introductory page. Due to the nature of the ActiveX components for creating some of the auditory graphs Microsoft’s Internet Explorer was used as the web browser. Subjects were told briefly what to expect from the test, such as that it used a web browser to display the test and it consisted of nine questions about auditory graphs. They were also shown the controls for adjusting the sound volume produced by a pair of speakers next to the computer. The investigator indicated that he would be in a neighboring room in case any technical difficulties arose, and left the subject to take the test.

          Data collection was then similar to the method used in the Web Pilot and the Main Auditory Graph tests in that it utilized a web browser to display graphs and information, and used PERL script programs to record answers. The test consisted of an introductory page with the Informed Consent Document and a brief description of the test. Next, subjects were presented with a page to record their names from which a scripting program appended the information to a file and assigned a code number. Subjects were then presented with a series of pages containing one or more auditory or visual graphs, a multiple choice selection field, and a text entry box for comments on their graph choice. Another scripting program appended their code and text answers to a second data file and passed the code and multiple choice answer to the next page. After completing the last question, the scripting program appended the code number and the string of multiple choice answers to a third data file.

        8. Instrument Development
        9. There were nine question pages: four consisted of pair-wise auditory graph comparisons, four involved matching an auditory graph to a visual graph (two questions were matching a visual graph to a choice of auditory graphs, and two were matching auditory graphs to a choice of visual graphs), and one page with five-point Likert ratings of 6 graph types. Each question page had a text field for subjects to provide comments and reasoning for their choices. The questions can be found in Appendix D.

          The auditory graphs were produced by two methods. The first method was by playing prerecorded MIDI sound files using a piano instrument to represent the data values as was used in the previous Web Pilot and Main Auditory Graph tests. For this method, the data was mapped to a chromatic scale. The second method for generating the auditory graphs was with the AudioPlot ActiveX control from Oregon State University's Science Access Project. The AudioPlot (AP) control generated auditory graphs on the subjects’ computer from equations specified by the web page. It allowed for various graphing parameters which were also set within the web page code

          It should be noted that another potential remote subject did not participate citing security concerns with ActiveX control modules. The choice of utilizing these controls to generate the auditory graphs on the web was based primarily on the transport of Visual Basic code written for an updated version of the TRIANGLE graphing calculator. This code was quickly modified to produce the AudioPlot control modules which could be incorporated into a web based testing environment.

          Both the MIDI and AudioPlot methods would play the auditory graphs when the subject selected a "play" button on the page. The buttons were identical so the subject had no indication of a difference between the methods to produce the graphs. The AudioPlot graphs produced a smooth, continuously varying tone with optional clicks for the derivative information. The MIDI graphs consisted more of a staccato piano note with a courser resolution. The derivative information was displayed with a drum like tone. All graphs were accessed through the use of Internet Explorer on a desktop personal computer.

        10. Data Results
        11. Table 10.1 is a summary of the multiple choice results for each question. The questions and answer choices are abbreviated for reference. The full text for the questions can be found in Appendix D.

          Table .1 Summary of Answer Choice per Question

          Answer Choice as Percentage of Total








          1. Gaussian curve, A = AP, B = MIDI, C = both, D = neither





          2. Gaussian curve with derivative A = low +, high -; B = high +, low -, C = both good, D = neither good





          3. xsin(x). A = no change at 0, B = instrument change at 0, C = both good, D = neither good





          4. xsin(x). A = AP with deriv., B = MIDI with deriv. and pitch change a 0; C = both, D = neither





          5. Match sin(x)exp(-x) to graph d, AP graphs: a=sin(x), b=cos(x), c=xsin(x), d=sin(x)exp(-x), e=cos(x)exp(-x), f=none







          6. Match cos(x)exp(-x) graph e, MIDI graph: a=sin(x), b=cos(x), c=xsin(x), d=sin(x)exp(-x), e=cos(x)exp(-x), f=none







          7. Match AP graph of cox(x) to picture a. a=cos(x), b=sin(x), c=xsin(x), d=cos(x)exp(-x), e=sin(x)exp(-x), f=none







          8. Match MIDI graph sound of sin(x) to picture b. a=cos(x), b=sin(x), c=xsin(x), d=cos(x)exp(-x), e=sin(x)exp(-x), f=none







          9. Likert preference of xsin(x) graph with different sound representations 1- 5, 1 is bad, 2 is poor, 3 is neutral, 4 is good, and 5 great.

          X avg

          std. dev

          a. MIDI



          b. AP



          c. MIDI, dx



          d. AP, dx



          e. MIDI, 0



          f. MIDI, dx, 0



          By equation 2.2, the error associated with the each of the Likert averages can be found. Using a 95% probability limit, the average of the standard deviations (s avg = 1.12 = 28%), and the sample size of twelve subjects,

          , (10.1)

          or about 16% since there was a 4 point range (5-1) in the rating scale.

        12. Analysis
        13. While the testing results would provide more consistency for a larger number of subjects, the purpose of this pilot test is primarily to discover where any difficulties in the testing process and question statements may reside. However, there are still important conclusions that be drawn from these answers . Comparing the results above to the subjects’ written comments is very informative as to the reasons for the choices and greatly aids the analysis.

          The first question compared MIDI and AudioPlot (AP) representations of a Gaussian curve. These graphs only used the y axis to pitch mapping. The results for question 1 imply that there was a preference (58 to 33%) for the MIDI graph over the AP graph, and that this was a significant difference since t = 3.09 > t critical = 2.20 (double tailed t-test, df = 11, a = 0.05, s » s = 0.28 from average standard deviations of results in question 9.). This is a somewhat surprising result as great effort went to produce a pleasing smooth sound. The commentary is very interesting as unexpected factors played a role in the choice. Subjects choosing the MIDI graph mentioned that it "seemed cleaner", and that the discontinuous sounds produced a more dramatic effect, making it easier to distinguish the maximum point on a graph. In contrast, at least one subject preferred the AP graph because it was continuous.

          Several subjects commented that their choice was at least partially based on the frequency ranges of the graphs. "The greater difference between the maximum and minimum tones made the graph easier to visualize" and resulted for a MIDI preference choice for this subject, but another chose the AP graph because "I seem to make the connection better for the higher pitches." Thus, future testing will need to be careful that the that different graphing methods display the same range in frequencies.

          These choices may reflect the difference in data mapping methods used by the two auditory plots. As has been noted in previous research by Stevens in Mansur [Man85], pitch has a logarithmic association with height. Thus there needs to be a larger difference between tones at higher values. The linear mapping used by the AP graphs thus has a perceptual flattening at higher pitches and may seem less distinct.

          Question 2 investigated the pitch mapping preference for curvature. A very brief description of what the secondary, drum tone was supposed to represent was given at the top of the page. The first graph used a low drum tone to indicate positive curvature, and a high drum tone for negative curvature. The second graph had the reverse mapping. The graphs were again of a Gaussian curve. The results were that 33% (four subjects) preferred the first graph (A), while 17% (two subjects) chose the second (B). However, 33% didn't have a preference (C), and 17% didn't like either (D). This difference can not be considered significant (t = 1.97) however.

          The comments provide the additional feedback that those choosing C or D, often did so because they found the graphs confusing, or had a difficult time distinguishing between the graphs. Also, two subjects who gave a preference for one graph type, gave the opposite graph a higher preference rating in question 9. This indicates both the necessity for better descriptions, and asking the same question for several different graph patterns for consistency.

          Question 3 investigated the preference of including a change in the graphs data sound when the y axis value was negative. This was brought about from comments received during the Main Auditory Graph test (even though all of the graphs of that study had positive values.) For this representation, the data sound of the graph of changed from a piano tone, for positive values, to a harpsichord tone for negative ones. There was only a slight, non-significant, preference for the tone change. The reasons for not preferring the change are very informative. Cited complaints were that the tone change created "too many options for the ear to play with" and "broke up the graph a little too much." Those who preferred the tone change found it very helpful. One comment was: "I liked how the pitch changed when the graph went below 0. I think it is important to change the sound when some major distinction (like the zero line) is involved." A more pleasing, less distracting tone change may greatly improve the preference for the tone change.

          Question 4 compared the graphs of for the AP and MIDI methods with derivative information. The AP graph had a score of 33% and represented positive curvature with a high pitch click, and negative curvature with a low pitch click. The MIDI graph had a score of 50% with positive curvature represented by a low pitch drum, and negative curvature by a high pitch drum. The MIDI graph also incorporated a tone change for negative values which was not included in the AP graph as it did not have a similar display option at the time. The difference was just under that for significance at a  = 0.05 (t = 2.10 > t critical = 2.20 .) This question would greatly benefit from a larger sample size.

          Of the subjects choosing the AP graph (A) and providing comments, there is an indication that improvements were still desirable: "A would be better if the drum pitch had those high harmonics for positive values instead of the negative ones." and "A sound is good to me. … Sharp pitch is better to me, but this one also needs some different sound to express the ups and downs." Of the subjects commenting on the MIDI graph (B), they often cited that their choice was because "the distinct sounds in B were much more clear than in A," and because of the "negative change and you can pick up the slope/curvature better."

          There was also a comment by one subject, who chose the neither (D) option that "both seemed rather arbitrary in relation to the graph, at least in the derivative department."

          Comparing the results of questions 5 and 6, which were graph identification questions, shows identical results between the those choosing the correctly, and in the distribution of incorrect responses. In question 5, subjects were asked to match a visually presented graph of to one of 5 AP auditory graphs (with derivative information.) In question 6, subjects were asked to match a visually presented graph of to one of 5 MIDI auditory graphs (with derivative and 0 information.) Thus, subjects seemed to be able to match a pictured graph to its auditory representation equally well with both methods.

          Comments about the AP graphs (question 5) indicated that some subjects found the choices indistinguishable. There were statements of "I started to choose E or D, but really I didn't like any of the choices" from a subject choosing None of the Above (F), and "But frankly, a-d sounded all the same" from a subject choosing the correct answer (D). One subject who chose incorrectly, noted the disparity between the choice and their reasoning: "I just like the sound of C the best, however, listening to the pitches, it almost seems like the two maximums reach the same pitch, but on the graph, the second one is lower."

          Of the subjects commenting about their choices on the MIDI graphs, those who answered incorrectly indicated that "the drums in the background created confusion as to what was going on" but that there were many close choices as the comments of "A and E sounded nearly the same" from one who chose A, and "E seemed the closest, but the derivative portion seemed wrong" from one who chose F indicate. One subject stated gave an incorrect choice D but stated that "the tempo of the drum was most clear in describing the slope of the line, as was the change in sound describing the negative values of the curve."

          For questions 7 and 8, which also involved graph identification. subjects were given an auditory graph and were asked to choose between several visual graphs or a "None of the Above" choice. Question 7 asked to match an AP graph of cos(x) and had a correct response rate of 75%, while question 8 matched a MIDI graph of sin(x) and had a correct response rate of 92%. The difference was just under the level for significance ( t = 2.10) and would also have benefited from a greater number of subjects.

          In question 7, one subject who answered incorrectly mentioned a difficulty in identifying the starting sound. Question 8 would have had a 100% correct score, but the one subject who chose F mentioned that the graph "seemed to mostly fit B, but I don't think the derivative was correct." The response rate on question 8 may have been greater than that of question 7 due to it being a fairly similar graph as the prior question.

          The last question asked subjects to rate different representations of the graph of on a Likert scale of 1 to 5, where 1 was bad, 2 was poor, 3 was neutral, 4 was good, and 5 was great. The results are given in Table 10.2, but are a bit vague due to the high standard deviations and all rankings should be viewed as essentially equivalent as the averages were all within the smallest standard deviation. ANOVA analysis of the average Likert scores from question 9 display of different graphing methods shows no significant difference between the methods at the a  = 0.05 level. (F = 0.82 < Fcritical = 2.35) All the average scores were between 3 and 4 indicating that the methods still could be greatly improved.

          Table .2 Ranking of Preferred Graph Types


          Average Rating

          Graph Type



          MIDI with 0

          2 (tie)


          MIDI plain, AP plain



          AP with derivative



          MIDI with 0 and derivative



          MIDI with derivative

          For question 9, several subjects also provided general comments on what they found helpful or annoying. These comments tended to focus on the drum beat (or clicks) indicating curvature, and the change in tone indicating negative values.

          A few selected comments demonstrate the greatest strengths and some potential problems with these auditory graphs:

          "They all represented the graph well, it just depended on if one was interested in slope and curvature."

          "I like hearing positive and negative. I like having pauses between notes instead of one constant sound. I like really hearing the slope. I don't like the soft drums because it's hard to differentiate them from the sound of the computer loading." [The computer had a somewhat noisy fan.]

        14. Conclusion for Auditory Preference Pilot

      The Auditory Preference Pilot demonstrated some useful innovations in the development, production, and comparisons of auditory graphing techniques. While the focus of this test was to provide an initial comparison of several of the assumptions used in the Main Auditory Graph test, it also provided a testing medium for a new control module to produce auditory graphs with dynamic flexibility and customization on the web.

      The results of this pilot test indicate that a variety of graphing techniques are acceptable from a users’ standpoint. Some characteristics tend to be favored by a majority, but by no means all, of the subjects , and that their preference seemed to change over the course of the test.

      Comments and preference choices about graphing techniques showed a favoritism toward graphs where the sounds were clear and distinct with a wide tone variation. However, there were also indications that by the end of the test, some of the distinct display techniques became bothersome. In question 9 one subject remarked: "I am starting to find the drum beats to be annoying." From comments such as this, it is evident that when designing commercial graphing displays there is an inherent need for user configurations of items such as pitch range, turning on and off derivative sounds, sound transformations at the zero point, and continuous or "broken" sound playback.

      Because the Auditory preference test had questions where several display items changed between the two graphs, it is difficult to make a complete determination as to which techniques are ultimately preferred. The display due to a desire to receive feedback on graph options which had not been implemented in the AudioPlot controls.

      The use of the AudioPlot controls for graph generation has many powerful advantages. Once the control is loaded on a remote computer, many complex auditory graphs can be produced with little more than embedded commands in a web page. This eliminates the need for pre-produced graphs, and creates a dynamic display where users can provide a more in depth investigation of the graph than simply passively listening. The use of Visual Basic to create the ActiveX controls can result in short development times when adding features. Disadvantages of the ActiveX control system are potential security risks for users, the potential for missing support files (.dll files) on user computers, and the limitation to a single platform and virtual limitation to a single web browser. The use of the JAVA language to create the auditory graphs is a possible candidate to remove the limitations of ActiveX for web based auditory graph production methods.

      Future studies will need to carefully construct both questions and many graphing variables in order to provide definitive answers. A longer set of questions, with repetition of graph types to provide multiple comparisons is highly desirable as subjects tend to change their views about which styles are favored as they gain experience with the graphs. Ultimately, the graphing methods should not be controlled by the ability of the software programmers to develop the interface, but by the end user to select the styles that are most evocative for the particular graph that is being listened to.

    4. Final comments
    5. The series of experiments described in this work has been an effort to demonstrate not only why there is a need for auditory graphs, especially in scientific areas such as physics, but how these graphs can be implemented and used. The use of auditory graphs benefits not only visual disabled people who have the right, and with these techniques, the ability for quick access of data displays, but also allows anyone to effectively use the displays wit very little training. With the equivalent of a short description and a few examples, subjects demonstrated the ability to perform at a level that was at least 80% of what they would have achieved with visual graphs. With more training or experience with graphs, this can easily be increased to 90% or more. It was also demonstrated that auditory graphs are not limited to displays in research laboratories with fixed environments, but can be effectively utilized throughout the country and world, and that subjects do not have to be sighted to accomplish this feat.

      Auditory graphs hold great promise as a display technique. The Auditory Preference Pilot test demonstrated some of the many areas that future research can be focused to provide for even more effective displays. Finally, here are two last quotes from subjects. The first is from the Main Auditory Graph test:

      "I think the whole idea is great and I think the drum beats to show curvature and slope are particularly functional and innovative. It is really important to develop the ability to hear negative values."

      The last is from the Auditory Preference Test demonstrating the accomplishment of the previous subjects’ request:

      "Again, I really like the negative value changing tone. It really helped to see the graph with my eyes closed."

To Table of Contents
Last:Chapter 9

Copyright 1999 Steven Sahyun