Taken together, the results in Tables 1 and 2 indicate that, on average, homework scores improve, total time spent on homework assignments likely remains the same or slightly increases, and exam scores are unchanged when students are given two relative to one homework attempt.
To examine how multiple attempts alter the marginal decision facing students, Table 3 shows the difference in mean homework scores and time spent for each homework assignment between the first homework attempt of the two homework attempt section and the single homework attempt for the one homework attempt section.
Panel B of Table 3 examines the mean difference in time spent on each homework assignment, contrasting the time spent on the first attempt for the two attempts section against the only attempt of the one attempt section.
Specifically, how are students achieving a higher homework score in approximately the same amount of total time while spending less time and performing worse on the first of two attempts relative to the one homework attempt section?
To decompose the homework gains due to students having two homework attempts relative to one, we considered the following hypothetical scenario.
Next, we test if the difference between the average homework score in the two attempts section and this counterfactual average homework score exceeds zero.
The difference in means test between the average homework score in the two attempts section and the average homework score for the one attempt section given a hypothetical second attempt that involves guessing is shown in Table 4.
Hence, the homework gains attributable to the learn-by-reworking effect appear to apply to only four assignments.
In Panel B, the average homework scores for the one attempt section given a guess increases as the probability of guessing correctly increases.
Specifically, we attempt to bound the amount of grade inflation due to the guessing effect for each homework assignment and for course-wide grades.
Assuming the results in Table 4 represent the homework gains due to the learn-by-reworking effect, we recover the portion of homework gains due to the guessing effect by a difference-in-differences approach, specifically differencing the total gains in Table 1 from the learn-by-reworking gains in Table 4.
Finally, we extend the magnitude of grade inflation from a per homework basis to a course-wide basis by calculating course-wide lower and upper bounds.
In this paper we present compelling evidence that, when given the option of a multiple attempt homework, in which correct and incorrect answers are identified after the first attempt, students employ an optimizing behavior that results in higher homework grades with no additional effort.
Taking the logic of our analysis further, and while we find no evidence of differences in exam performance, to the extent that students factor homework grades into the their decisions, the grade inflation is a disincentive to study for exams because (1) with higher homework scores a lower exam score will still achieve a target grade; and, (2) higher homework scores, simply due to better guessing odds, may give a student the false perception that he/she is well prepared for an exam.
Comparison of student performance using web and paper-based homework in college-level physics.