Disaggregate and Adjust Value-added Learning Scores
A tool to disaggregate Scantron, ZipGrade, Quick Key, or Akindi pre- and post-test responses into value-added learning scores (Walstad and Wagner 2016) and adjust them for guessing (Smith and Wagner 2018).
Using sample data files (download), we can walk through the files needed to use the software. The zip file contains four data files. Exams 1 and 2 are the Scantron raw format. For the purposes of the fake files, I've made up some students with responses. Given the data is fake, the results don't make a terrible amount of sense, but we need a set of files that can be publicly shared. Students.csv is simply a list of student IDs to use in the assessment process. If you omit this file, the program will analyze the subset of students that exist in both the pre- and post-test. However, when specified, this feature allows you to analyze a subset of the students such as majors.
Finally, assessment_questions.csv contains the question numbers for each of the questions you wish to analyze. The first column is the assessment question ID. The column Exam1 specifies which number the question appeared on the first exam (or pre-test); Exam2 specifies where it appeared on the second exam. This way, the question positions on each exam don't have to match. Finally, the column 'options' specifies the number of available answers on each question, e.g. '4' on a four option multiple choice exam (this column can contain a non-integer value). The option column can also take a value specified between 0-1 in which case it assumes the specified value is a probability. When this file is unspecified, the program assumes the pre- and post-test questions are in the same order, all matched questions should be analyzed, and the number of options equals four.
The software assumes Scantron formatted exam files have the following characteristics:
The software will compare the answer key (first row) to each student's answer thereby determining if the student correctly answered the question.
ZipGrade, Quick Key, and Akindi Format
If the exam files are ZipGrade formatted, the software assumes the files have the following characteristics: