-
6.
It has been noted by many educators that "graphing is a tool used in science to display data and aid in the analysis of relationships between variables. Also, graphs are part of our daily existence with their use in all media." [Wav89, p. 373] However, several studies have uncovered areas where students have difficulty interpreting graphical information that is used in physics, mathematics, and even economics. [Bei94, Mcd87, Lei90, Coh94] Because of these difficulties, there has been considerable energy devoted to the teaching and study of graphs in used in physics.
While progress is being made in teaching graphical information, some attention to how the data is displayed is warranted. There has been discussion on the effectiveness of display methods [Tuf90], as well as much literature concerning optimal display methods. Unfortunately, almost all of these studies concern only visual display methods. There are several problems with focusing on only the visual data display aspect, most importantly is: What happens when one can not see the graph in question? Thus, it is important to explore other avenues of displaying the information contained in graphs. For this reason, auditory graphs were used as the basis of comparison.
The current research is directed towards not only demonstrating that people can understand auditory graphs, but that they can be used as an effective display method for understanding and analysis of the information they present. Previous research has focused on how people perceive graphs and how they use graphs to learn about physics. There have been several studies investigating how well people can make judgements about graphs, but none of the previous studies have demonstrated if auditory graphs can be practically implemented, and what sort of results could be drawn from students using auditory graphs.
The ability to effectively present data in an auditory format is one of the prime goals of this research. The working hypothesis for this study is that: in many cases, sound graphs can be as effective as visual graphs for data representation and for making inferences
If graph types are highly equivalent, as suggested by the studies by Flowers [Flo92, Flo93, Flo97], then there should be no difference (aside from perhaps effects due to unfamiliarity with the sound format) between a student's ability to identify and interpret information given auditory or visual graphs. Asking questions based on graphical material helps to identify if the auditory graphing method hypothesis is valid.
To test this hypothesis, it is important to see how well students are able to answer graph based questions. One method for testing is to have two groups of subjects answering questions, with each group either receiving visual graphs, or auditory graphs. Beyond identification of simple graphs, is the ability to interpret what those graphs mean. Thus, this study includes two types of questions: those that involve interpretation to identify a function, and those that require analysis of the data for interpretation of the graph.
With a test devised such that subject are divided into groups of auditory or visual graphs, a comparison of their relative performance can be made. It is conceivable that by having a combination of graph types, subject's understanding of the graph in question may be enhanced as there becomes a redundancy of information. The subject can pick which format is most helpful at a given time, or use the combination of formats to enhance the graph.
By comparing the performance of subjects ability to answer graph based questions with respect to which graph type they receive, there can be several outcomes. The first, is that if student performance is equivalent among the auditory, visual and the combination displays, then the display modalities are equivalent. They can answer and analyze questions equally well.
The possibility for equivalent performance in answering questions exists as studies by Flowers & Hauer demonstrated that there are several areas of perceptual equivalence between auditory and visual graphs. Turnage et al. [Tur96] also reported rough equivalence between auditory and visual graphs in identification of properties for simple periodic wave patterns.
A second, albeit unlikely, comparison outcome may also exist in that auditory graphs could outperform the visual counterpart. This might be the result of an increased salience from the auditory cues as was noticed in some parts of the discrimination studies by Flowers and Hauer. [Flo95, Flo97]
The more likely situation, however, is that there would be a performance difference due to greater familiarity of the visual graphs. This is understandable as students are trained to recognize and use visual graphs for many years by the time they take university level courses. Auditory graphs on the other hand, are a completely new experience, and the amount of training they receive may strongly influence their performance.
Comparable, but not equivalent, performance on discriminating differences between data sets within auditory and visual graph groupings have been shown in the aforementioned studies. [Flo97, Tur96]
If subjects completely fail in their understanding of the data presented as auditory graphs, this would call into question currently reported research. The result would be that there may be a finite limit on the practicality of auditory displays. Also, it may demonstrate that the understanding of the auditory graphs is not intuitive, and that subjects would need more intensive training. For simple data comparisons and analysis, alternate auditory methods would need to be investigated.
6.At the most fundamental level, this research is aimed at providing a method to portray graphical information to people who have vision difficulties such that they are unable to see or understand what is seen. While haptic methods to create graphical information have been used in the past, there are several difficulties including interactiveness, resolution, production, portability, and storage issues. Haptic graphs also require a significant amount of time for the person using the graph, as well as the use of an additional tutor for explanation, to understand what is presented. The auditory format can remove many of these limitations.
The basic auditory display used throughout this study, centers on mapping the y axis data value to pitch and the x axis to time. The exact relationship for the y axis pitch varies between the experiments, but always uses the association of high pitch (higher frequency values), on the order of a couple kilo Hertz, to represent high data values, and low pitch (around 200 Hz), to represent low values. This method is analogic, that is, pitch has a direct one to one mapping with the data values. In both pilot tests and the main test, all the graphs had zero or positive values, thus the lowest magnitude value had the lowest note, and the highest magnitude value had the highest note.
The association of pitch to the magnitude of the data value is common practice as it has a certain relationship to musical notation and has been widely used in research and other data sonification programs. There are different sound mapping methods, but pitch is the most common, has been applied in many cases, and appears to be intuitive for most people.
While previous studies have focused on general similarities, or the ability of subjects to identify trends or differences in comparative data sets, there have not been any studies investigating how well subjects can perform given typical analysis type questions. The current research addresses this issue by investigating how well students are able to answer physics and math questions based on graphed data.
6.-
6.3.
- Genesis of Testing Process.
- General test design.
- General data collection procedure.
- Testing Considerations.
The use of personal computers to generate sounds has been utilized in many auditory display studies. The development of the TRIANGLE program by Oregon State University's Science Access Project took advantage of this sound capability to generate a preliminary auditory graphing technique as a compliment to, and substitution of, the visual graph display.
TRIANGLE's primary purpose is to provide a workspace for reading,
writing, and manipulating mathematics for students and scientists. The calculator function permits evaluation of most standard math expressions and evaluates y versus x functions that then are displayed in a plot window. An auditory graph of the function or of data, with a number of display options, provides a blind or visually impaired user with a quick semi-quantitative overview of the graph. A moving icon on the screen provides similar information for deaf blind users. [Gar96]
The ability to generate the auditory display by the TRIANGLE program created the question of: How useful is this type of display to the intended user base? In answering this question, it was necessary to develop an unbiased testing method between auditory graphs and visual graphs in the context for which they would be used in the program, namely, to investigate properties of mathematical functions, and scientific data.
The initial investigation centered on using the TRIANGLE program as a testing medium, as it was that program that was the genesis of the research. This program displayed both visual and auditory graph formats, as well as a text region that could be used to display questions about the graphs. Hence, in the initial stages, it was a good candidate for implementation of the testing process. Later, a testing process based on the World Wide Web proved to be a more flexible alternative, and one that provided many advantages as is discussed in chapter eight.
6.3.The first stages of the testing process required several assumptions. The first was that the auditory display method implemented with the TRIANGLE program would be sufficient to the task. This was not an unreasonable assumption given that previous research utilized similar auditory mapping methods. Also, TRIANGLE had been tested by several people for use and stability.
Using the TRIANGLE program as the initial basis for the study, it was necessary to formulate an unbiased testing process that would demonstrate the effectiveness of subjects to answer and evaluate questions based on graph types. A standard causal-comparative design procedure is a pre-test, treatment, post-test method. In this type of testing process, subjects are given a pre-test to measure their initial state, some form of teaching or learning treatment, and a post-test to measure their final state. By comparing the pre and post-test scores, a judgement on the effectiveness of the treatment method can be made.
While a causal-comparative method is attractive, in the case of determining the effectiveness of auditory graphs as a substitute display method, it would not be an adequate procedure. This method would primarily demonstrate that subjects could learn to use auditory graphs, and mostly be a reflection on the type of training that they would receive. The demonstration of auditory graph use at a basic level has been explored in previous research, [Man86, Flo92] and is not the main focus of this study.
A more illuminating study can be performed when the performance between two or more groups of subjects are compared. By combining multiple groups with the pre-test, treatment, post-test method, a comparison of different treatments can be accomplished. In this manner, the pre-test is used to verify that the groups have equivalent abilities, or to give a basis for correction if the groups are found to be non-equivalent. The treatment can take many forms, but in the current study, it was the display of a graph (visual or auditory) within the questions of the post-test. The results of the post-test are then compared to judge the effectiveness of the treatment (graphing) methods.
Knowledge of the subject matter used in the questions will be important as the effect of a subject's understanding of the material will affect the overall performance. However, since subjects are randomly assigned to different groups, each method will be equally affected with regards to student performance. By comparing the performance of two groups on identical questions, any difference is thus focused on the ability of subjects to utilize the graphs, and not necessarily on the knowledge of the material in the questions. The questions act as a basis for different reasoning structures that one would like to investigate.
The level of difficulty of the graph questions should be gauged to the target population for which the graphing method is to be used. As the TRIANGLE program was designed for college level use, appropriate questions centered on introductory college level math and physics. Physics was chosen due to the large range of examples that utilize graphs, and the previous research on student difficulties with graphs in physics. The population for the study was thus drawn from subjects who had taken, or were in the process of taking college level physics courses.
There is a difficulty in gauging subject involvement when answering the questions in this study. Since subjects recruited in this study are all volunteers, and no incentive for performance level could be applied, there is no guarantee that the subjects performed at their best level when answering the questions. However, this should not affect the overall results of the study as there is a comparison between groups randomly chosen from the same population. On average, any performance issues will be the same for both groups of subjects.
6.3.The specific methods utilized in the study to collect data varied between the pilot tests and the final study and will be more fully developed in the chapters relating to each test. There were two pilot test studies to investigate testing techniques and questions, and a final, web based, study to gauge student's abilities to answer graph questions in math and physics. The general process for data collection in each study consisted of recruitment of various instructors to volunteer their classes. The student subjects were given a statement of informed consent to read and agree to, a questionnaire for demographic purposes, a pre-test to assess the equivalence between groups, and then took a main test consisting of one or more randomly assigned graph types.
The informed consent page consisted of a statement of the test procedure that was involved, the names of the principal researchers and contact numbers, and an agreement clause. This page was required by the Institutional Review Board as human subjects were involved.
The questionnaire was used to gather data such as gender, age, the number of physics, math, and any other courses relating to graphical information that subjects had taken. This page also queried whether the subject had musical training, or vision or hearing difficulties.
The pre-test consisted of five questions about two graphs. The first questions were to tell whether the subject could properly read a graph, and the last concerned interpretation of the physics described by a graph. The last question was similar to those found in the main test.
The main test was presented in different manners depending on the study. For the Triangle pilot study, the subjects were presented with multiple choice questions on a computer screen, and either listened to and/or looked at a graph that the question related to. They then answered the question, which were recorded in a written format, and the next question was presented. Assignment of the graph presentation method was random, with the subject receiving a single method (visual, auditory, or both visual and auditory) for all questions. For the Web pilot study, the subjects accessed a series of web pages that presented the graph and multiple choice question, with one question per web page. Answers were transmitted by selection of multiple choice "radio buttons" and the answer was recorded by a scripting program. For the Main Auditory Graph study, the same presentation and recording method was utilized as for the Web pilot, although the number and type of questions were modified and extended due to reliability and validity issues.
6.3.There is a possibility that a difference in performance levels between visual and auditory graph groups does not necessarily demonstrate an inability of subjects to understand and interpret the presented material. Instead, the difference could be attributable to training and familiarity effects. Since the subjects have had a common educational experience as subjects are drawn from the same standard physics course (year and level), they will have been exposed to many visual graphs, but virtually no auditory graphs as this is a new representation. Thus, it can be assumed that they have much more experience with visual graphs than with the auditory graph representation. Training about the auditory graph representation is necessary, but the amount of required training remains undetermined.
This issue of performance effects due to the familiarity of graphs and subject material can be addressed by comparing the subjects' results with those from more experienced graph users. By looking at how well a group of experts (physics graduate students) perform on the questions when given auditory graphs, it can be demonstrated whether the questions are answerable, and what would be the best expected outcome for the groups. Any issues of unfamiliarity with the testing material can be eliminated, thus focusing only on the difference in the graph styles.
There are several reasons why expert subjects are not solely used for these experiments. First is the issue of the audience that the auditory graph representation is trying to target. This method is envisioned to be used as a common tool to help students understand basic data graphs. As such, it is important to discover whether beginning students can understand these graphs with little training and experience.
Another issue is the number of subjects required for meaningful
statistical results. Given a normal population distribution, for the 95%
probability level, the approximate size of a group required for a 95%
chance that the average measurement, 
, is within the limits of m
±
L is given by:
(6.1)
where m is the measurement value and n is the group size. [Sne89, p. 52] Now, assuming a standard deviation of 20%, since there are 5 possible answers to each question, and an error limit of 5%,
(6.2)
Thus, each graph test group should have a minimum of 62 subjects.
The prime target audience of the auditory graph representation is blind users. While it would be desirable to use 62 blind first year physics students to gauge their ability to answer the questions using the auditory graphing technique and compare their results to sighted users of equivalent background, this is not possible. There are extremely few blind students meeting the conditions of ever having had physics at the college level, even on a national scale. There have been several blind people who have completed physics courses, and they were solicited for their participation. However, only a very small number (five) were able to participate. This will be discussed in more depth in chapter 9.
In any test, there is a question of whether the test is a valid measurement of the subject. By dividing the test questions into two groups of equivalent questions, the performance of subjects for half of the test can be compared to their performance in the second half. This is called a split-half testing method. If there is a high degree of correlation between the scores in the two halves, then there is a greater probability that similar questions will have similar results. This helps to validate the testing process as the study concerns the question of how well subjects will be able to reproducibly use the auditory graphs to answer questions.
One difficulty with the testing process that should be noted was the high reliance on technology used in this study. While this posed certain challenges, the technological problems affected all subjects equally in the web based tests. In the preliminary Triangle pilot, an instrument method that was not technologically dependent was utilized for initial comparative purposes.
It is possible that a better scheme for testing and producing auditory graphs can be developed. The Main Auditory Graph test was an evolution of from the processes used it the pilot tests. As was previously stated, the auditory methods used in this study were chosen to a large extent by previous research, similarity to musical representations, and prior device development.
While research into better graphing techniques is warranted, it is the issue of further studies. Some indications of possible graph questions as well as alternate auditory display methods is discussed in chapter 10.