• What It Does
• Input Files
• Graphical User Interface
• Command Line Interface
• Output Files
• Video Instructions

This software has been deprecated. The software at https://www.assessmentdisaggregation.org/ does everything this software does, and more.

Disaggregate and Adjust Value-added Learning Scores

A tool to disaggregate Scantron, ZipGrade, Quick Key, or Akindi pre- and post-test responses into value-added learning scores (Walstad and Wagner 2016) and adjust them for guessing (Smith and Wagner 2018).

Output Files

Running the software will result in five files:

 

1. Walstad_Wagner_types.csv - The disaggregated outcomes by assessment question number
2. Walstad_Wagner_types_by_student.csv - The disaggregated outcomes by student ID
3. Walstad_Wagner_types_by_student_group.csv - The disaggregated outcomes by student ID grouped by number of options (probability) on each question
4. Questions_output.csv - Outcomes by assessment question on each exam
5. Student_output.csv - Individual student performance on assessment questions on each exam

 

In all Walstad Wagner files, you will find the raw disaggregated learning types as well as columns labeled 'gamma', 'alpha', 'mu', and 'flow'.  These correspond to "corrected" measurements of the learning types when factoring in the number of students guessing.  \(\hat \gamma\) (gamma) is corrected positive learning, \(\hat \alpha\) (alpha) is corrected negative learning, \(\hat \mu\) (mu) is corrected pre-test stock knowledge (corrected retained plus corrected negative learning), and flow is the corrected pre-test/post-test delta (\(\hat \gamma - \hat \alpha\)). Formally, the following equations are used to find the corrected values:

 

$$\hat \gamma = \frac{n (\hat {\text{nl}}+\hat {\text{pl}} n+\hat {\text{rl}}-1)}{(n-1)^2} $$

$$\hat \alpha = \frac{n (\hat {\text{nl}} n+\hat {\text{pl}}+\hat {\text{rl}}-1)}{(n-1)^2}$$

$$\hat \mu = \frac{\hat {\text{nl}}+\hat {\text{rl}}-1}{n-1}+\hat {\text{nl}}+\hat {\text{rl}}$$

 

where \(\hat {\text{pl}}\) (positive learning), \(\hat {\text{rl}}\) (retained learning), \(\hat {\text{zl}}\) (zero learning), and \(\hat {\text{nl}}\) (negative learning) refer to the raw learning type values and \(n\) is the number of answer options.  It is important to use these corrected values as the raw scores can be sensitive to the percent of the class guessing.  Smith and Wagner 2018 details this adjustment.

 

Most files are self explanatory, however, the difference between the two student level Walstad Wagner files can be confusing.  In the file 'Walstad_Wagner_types_by_student_group.csv', the corrected values are calculated at the student level where each 'Options' value is the same (e.g. 4).  These results are then presented along with the number of observations per options group.  For instance, suppose there is a 10 question multiple choice paired pre- post-test where half of the questions have four options and the other half have five.  This file would produce two rows per student, one for the four option questions and one for the five option questions. 'Walstad_Wagner_types_by_student.csv' is a weighted average of the student group file.  Once the learning types are calculated per option group, a weighted average of each learning type is calculated using the observations per option group as the weights.  So, in the example above, the two rows per student would receive equal weight as there are five questions each.  This results in a single row per student.

 

Note on files: While all of the Walstad Wagner output files use matched pairs, the "Questions" output does not.  Therefore, the pre- and post-test results may not match if you have some students (listed in the id file) that took only one of the two exams.

 

Note on encoding: While very rare, it is possible to encounter empty results in the output files if the program can not read the input files.  In most cases this is due to an advanced character encoding.  If you encounter this issue, please save your input files in a more basic character encoding.