|
Summary:
| |
When I print one of the Student
Performance by Question (SPBQ) reports, each question displays different
numbers of students. Is there something wrong with the report or my
data? |
Details:
| |
We have investigated this issue
and have found out the following information. This issue stems from
the fact that some students, while being marked as having taken the same
form of the test, have different descriptor sets within the data. We
have traced this all the way down to the actual Harcourt data itself and
have confirmed that it is not a problem with SOL Tracker. The reason
for the two sets of descriptors for the same form is unknown to us.
Logically, students that take the same form of a test (during the same
testing period) should have identical descriptor sets. But this
appears not to be the case.
We have examined the data for the possibility that certain students had
their form designation inaccurately marked. If the wrong form were
marked, the questions for those students would have been inaccurately scored
and the answers given should fall into a random pattern which would result
in approximately 25% of the questions answered correctly (given 4 possible
answers for each question). But the number of questions answered
correctly is comparable to the students with the other descriptor set, so
that rules out inaccurately marked form designations. Also, if the
forms were mis-marked, then the students would logically have the same
descriptor sets within the data (just with more "incorrect" answers).
This too is not the case.
We have determined that these differing question sets fall under the same
SOL curriculum sections, so therefore the differing descriptor sets are
comparable with each other.
From a reporting standpoint, we cannot isolate the two sets, because there
is nothing in the data to differentiate them. Both sets are within the
same Core, as well as the same Form. We cannot group the data by the
number of students taking the descriptors because this too can differ.
There is typically a handful of overlapping descriptors between the two
differing sets (descriptors that appeared within both sets). For
example, if one set shows 98 students, and the other set shows 2 students,
there can be a handful of questions showing 100 students.
There is a theory that these differing question sets are different points in
time within the same testing period. Possibly, Harcourt re-keyed the
questions for a form in the middle of testing. However, if this is the
case we still have no way of differentiating between the sets from the
Harcourt data. The time of testing is only designated in the data by
season and year (i.e.; Spring 2003). Also, the data for both sets
arrives together in the same Harcourt file. |
|
Last Modified: |
April 07, 2004 |
|
Versions Applicable to: |
4.0 |
|
|
