A Comparison Between Multiple Choices and Short Answer Questions in Reading Comprehension Tests

...

After the test, the test papers were graded by the author according to the authorized reference answers. One point was given for each correct answer. Thus, the total scores for the four passages in the paper were 20 points. It was unnecessary to worry about the marking reliability of MCQ as the keys were fixed. Test-takers would receive one point if their answer was right and a zero point if their answer was wrong. But the test takers’ answer to the short answer questions are not so easily to judge. Under this circumstance, global marking method is necessary. In this case, test takers’ answer can get one point if it can fulfill one of the following conditions: 1) the answer is exactly the same with the correct answer made by the test maker, or 2) the answer covers the core meaning or main idea of the key made by the test maker. One other thing should be made clear here. For this is a test to examine the test takers’ reading ability, the criteria employed here is not so strictly. As long as the answers the test takers offered have no difficulty in understanding, they can get one point even there are minute misspelling and grammatical mistakes. All the students in this class attended the exam and no one cheat in it. At last, the teacher collected 60 pieces of valid statistics. And then the author input the data into Excel and processes it by SPSS16.0.

3. Data Analysis and Discussion

As the test of this experiment being done, the rest work is to deal with the results of the study and the analysis of these results. The statistical analysis will be operated by the SPSS16.0 software package, and based on the results of this analysis; the author will then testify the four hypothesis of this study. A general conclusion will be drawn according to the thorough and comprehensive analysis of these data.

3.1 Concerning Research Question 1

In order to find out whether test-takers perform the same if different task types were applied to measure reading comprehension, descriptive statistics of the two task types is calculated by the author, which is demonstrated in the following table.

Descriptive Statistics

N

Minimum

Maximum

Mean

Std. Deviation

MCQ

60

4.0

10.0

6.817

1.5348

SAQ

60

2.0

10.0

6.333

1.8288

Valid N (listwise)

60

Table 2 Descriptive analysis of scores of MCQ and SAQ

Secondly, one-way ANOVA is carried out by the author. Generally speaking, ANOVA can be used to determine if the means of more than two groups (include two groups) are significantly different from each other. From Table 2, it is obvious that the means of the scores test-takers received in MCQ and SAQ respectively have minor differences, thus one-way ANOVA is conducted in order to check whether there is any significant difference or not. As the independent variable is manipulated to determine its effect in the dependent variable, in the one-way ANOVA, the scores of participants are dependent variable while the two task types serve as factors that influence the dependent variable.

Table 3 One-way ANOVA of Two Task Types

ANOVA

Sum of Squares

df

Mean Square

F

Sig.

Between Groups

7.008

2

3.504

1.228

.000

Within Groups

336.317

118

2.850

Total

343.325

120

The results of the one-way ANOVA which are demonstrated in Table 3 reveal that at the level of .05, the two groups of the experiment in the present study are significantly different. This shows that test-takers perform differently if different task types are applied to measure reading comprehension. In short, the results of one-way ANOVA demonstrate the fact that under different task types, test takers do perform differently.

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

88.3%

70%

71.7%

43.3%

80%

98.3%

48.3%

76.7%

61.7%

43.3%

Q11

Q12

Q13

Q14

Q15

Q16

Q17

Q18

Q19