COMPUTER OR PAPER-PENCIL: A COMPARISON OF TESTING METHODS FOR NON-STANDARDIZED ACADEMIC TESTS
Abstract
A variation in objective testing is using computers in the testing process. This study will provide a transition in the research on traditional instruction with traditional testing and that with standardized computer tests. The primary purpose of this study was to assess the effects that the administration methods of paper-pencil and computer with and without feedback would have on achievement test performance, test reliability and test item statistics in a college level classroom. The learner characteristics of cognitive style for field dependence/independence and attitude toward the role of computers in society were used as covariates. The study was conducted in an undergraduate philosophy course. The instructional method was traditional lecture-discussion format. The 32 students were randomly assigned to three groups each of which took a test under three methods of administration. The tests were instructor-constructed over particular segments of the course material. The methods used were paper-pencil, computer with and computer without feedback. The feedback method was an indication of the correctness/incorrectness of a response with the correct answer supplied for the latter. The test scores for each test were standardized independently in order to be considered equivalent instruments. Measures of cognitive style and attitude were obtained prior to any testing. Measures of attitude were also obtained after the testing was completed. The research design was a univariate repeated measures ANOVA. The hypotheses were: (1) Test performance on computer administered tests both with and without feedback will be better than test performance on paper-pencil tests. (2) The test item responses will be similar in pattern among all methods of testing. (3) The tests given by computer will be more reliable than the paper-pencil tests. (4) There will be a positive difference in test performance on computer-administered tests when cognitive style and attitude are held constant as covariates. (5) There will be a positive change in student attitudes toward the computer. The results of this study were that the tests were all reliable at .90 or higher. There were significant differences among these reliabilities with computer administered tests higher than paper-pencil and feedback higher than no feedback. There was a significant difference in response patterns among the groups for each test apart from chance. The covariates of cognitive style and attitude were found through preliminary regression measures to have only a chance relationship to test performance. Therefore, the use of covariates in the ANOVA was deleted. Analysis showed that there was no significant main effect for groups or for groups x methods interaction. There was a significant main effect for methods at p < .05. Student attitude became significantly more negative at p < .05. The primary implication of the current study is that the use of computer administered tests to college level classes for academic assessment is as reliable and as appropriate as a paper-pencil version. In the case of this study, the performance was actually better on the computer administered tests, especially the feedback method.
This paper has been withdrawn.