• What It Does
• Input Files
• Graphical User Interface
• Command Line Interface
• Output Files
• Video Instructions

This software has been deprecated. The software at https://www.assessmentdisaggregation.org/ does everything this software does, and more.

Disaggregate and Adjust Value-added Learning Scores

A tool to disaggregate Scantron, ZipGrade, Quick Key, or Akindi pre- and post-test responses into value-added learning scores (Walstad and Wagner 2016) and adjust them for guessing (Smith and Wagner 2018).

Input Files

Using sample data files (download), we can walk through the files needed to use the software.  The zip file contains four data files.  Exams 1 and 2 are the Scantron raw format.  For the purposes of the fake files, I've made up some students with responses. Given the data is fake, the results don't make a terrible amount of sense, but we need a set of files that can be publicly shared.  Students.csv is simply a list of student IDs to use in the assessment process.  If you omit this file, the program will analyze the subset of students that exist in both the pre- and post-test.  However, when specified, this feature allows you to analyze a subset of the students such as majors.

 

Finally, assessment_questions.csv contains the question numbers for each of the questions you wish to analyze.  The first column is the assessment question ID. The column Exam1 specifies which number the question appeared on the first exam (or pre-test); Exam2 specifies where it appeared on the second exam. This way, the question positions on each exam don't have to match.  Finally, the column 'options' specifies the number of available answers on each question, e.g. '4' on a four option multiple choice exam (this column can contain a non-integer value).  The option column can also take a value specified between 0-1 in which case it assumes the specified value is a probability.  When this file is unspecified, the program assumes the pre- and post-test questions are in the same order, all matched questions should be analyzed, and the number of options equals four.

 

Scantron Format

 

The software assumes Scantron formatted exam files have the following characteristics:

 

1. The first row is the answer key
2. The second column is the student ID
3. The third column of the first row specifies the number of questions
4. The fourth column and higher are the student answers

 

The software will compare the answer key (first row) to each student's answer thereby determining if the student correctly answered the question.

 

ZipGrade, Quick Key, and Akindi Format

 

If the exam files are ZipGrade formatted, the software assumes the files have the following characteristics:

 

1. The first row of the file specifies the column names
2. There is a column named 'id', 'student id', 'external id', or 'zipgrade id' specifying a unique ID number for each student
3. Each exam question column is named 'Q' followed by the question number (e.g. 'Q67' for question 67)
4. In each question column cell, '1' indicates they got it correct and {'0', ' ', 'a', 'b', 'c', 'd', 'e'} indicates the student got the question wrong.  Both ZipGrade and Quick Key place a '1' in any cell where the student correctly answered the question.  ZipGrade places a '0' in any cell where the student got it wrong while Quick Key puts the (incorrect) student response ({' ', 'a', 'b', 'c', 'd', 'e'}).   Akindi's columns are the same but the values differ.
5. If the column 'Grade' exists in the input file, the program will assume the 'Q' columns are in Akindi's format. Each cell containing {'a', 'b', 'c', 'd', 'e'} will be marked correct while all other values will be counted as incorrect.   Akindi places the response in each cell of the column; if the response is incorrect, the response is wrapped in parentheses.