Illustrating the Pain

This simple figure illustrates the barrier between teachers and (a) assessment training and expertise, (b) access to higher quality test items, and (c) self-awareness about actual assessment expertise. The barrier is multi-faceted and includes availability, opportunity, time, and money. Our edstartup idea provides clear and innovative solutions to each of these facets.

OpenAssessment.org – Our EdStartup Idea

The Idea
We* want to create a collaboratively developed database of openly licensed assessment items and measurement training. All of the items in the database will be written by trained educators and other subject-matter experts from various content areas and at various educational levels. The items will be aligned to educational standards (where applicable) and made available to educators in multiple formats, including PDF download for free, use within an internal delivery and analytics system, and use within external delivery systems. The openly licensed measurement training will also be freely available to all users.

The Problem
Teachers lack the time, resources, and expertise needed to create high quality classroom assessments. And even in the rare cases where time, resources, and expertise are not limited, typical classroom assessments still lack the number of responses necessary for the kinds of measurement analyses that could help teachers improve their classroom assessments in a systematic way.

The Solution
A collaboratively developed database of openly licensed assessment items will allow teachers to improve the quality of their classroom assessments by (a) providing them with increased access to higher quality test items, and (b) providing them with access to measurement training and support. All users will have free basic access to all items in the database because the items will be openly licensed. In addition, the collaborative approach to item development will decrease the individual time needed to create high quality assessments by capitalizing on the inputs of a community of subject-matter experts. Also, item writing training and psychometric support will increase teachers’ practical measurement skills. Finally, items used within the internal delivery and analytics system will have large enough response numbers to allow for more sophisticated psychometric analyses and item improvement.

The Why
High quality assessment of student learning is important for making sound pedagogical and administrative decisions. However, many important educational decisions are based on assessment results that aren’t highly valid or reliable. This is especially true at the classroom level. But this is not because many teachers don’t want to improve their assessments. Most teachers couldn’t do much about this problem even if they wanted to. They not only lack the funds and training, but they simply don’t have enough item responses to conduct many psychometric analyses. Indeed, many teachers lack the time, software, training, and secure server space to even maintain a database of items and their psychometric properties. By providing teachers with increased access to this kind of database and to practical measurement training we can improve the quality of assessment in our educational institutions.

*Current intellectual contributors to this idea include myself, David Wiley, T. Jared Robinson and Dan Allen.