Validating Our Assumptions

Today I chatted face-to-face with eight teachers from various schools and subject areas (as well as with a couple of people within a school of education) to verify that the assumptions we have made about teachers’ pain are accurate.

Through my interviews, I was able to confirm each of following seven basic assumptions:

1. Teachers lack assessment training.

2. Teacher training programs do not have curricular space (or priority) for assessment training.

3. Assessment development is time consuming and often tedious.

4. Teachers rely on test banks and other sources of assessment items that have unknown quality and alignment.

5. Many teachers aren’t doing much to solve this problem. However, some teachers have started collaborative assessment development activities at the department, school or district levels. This means that some teachers, at least, are looking for ways to increase the test development efficiency and quality.

6. Very few teachers obtain assessment training through professional development activities. Generally, teachers who do obtain some training are those who have been asked to help in the development of large-scale standardized tests. This kind of training has limited applicability at the classroom level.

7. Teachers have limited access to high quality test items, but many teachers desire more access to such items.

In addition, my conversations with these teachers (and other education experts) revealed at least one other critical reality:¬†teachers often don’t know what they don’t know about assessment.

It turns out that most teachers feel that they have sufficient capability in assessment development and evaluation. However, a few teachers that I talked to who had received minimal training (and one currently enrolled in a measurement Ph.D. program who has received significant training) were emphatic that their assessment knowledge prior to training was exceedingly limited but that they weren’t aware of the fact. What this means is that it will be critical to find the best way to help teachers realize the benefit that could be gained by using our product. One teacher (who is training to become a principal) recommended focusing our marketing toward administrators who, if bought-in, could persuade their teachers to use it. A second way to increase use would be to offer some sort of credit (college or licensure) for those who complete the training.

Finally, most of the teachers I talked with gave very positive feedback about the idea itself, including saying things like “I would use your database” and “Your training and database would be so helpful to me and other teachers I know.”

Photo Credit: http://www.innovationtools.com

2 thoughts on “Validating Our Assumptions

  1. So many gave positive feedback, that’s great. What’s often more instructive is the few that didn’t. What did they say? What kind of complaints did they have? What did you learn from them that you didn’t already know? – you didn’t already know everything. =)

  2. I learned quite a few things that I didn’t know beforehand. First, I realized that buy-in from administrators is probably going to be an important aspect of getting teachers to use the database. Second, I learned that teachers often don’t know what they don’t know. I went in with the assumption that teachers realize they lack assessment expertise. But what I learned is that teachers often feel like they’ve got a good handle on how to write/evaluate test items – even though most will readily admit to never receiving any training in this particular area. Also not every teacher was completely positive. One teacher mentioned that this database would be very useful if he were just starting out, but now that he has four long years of experience behind him, he feels like he’s got it all figured out. Helping teachers realize the benefit they could get from using our database might be a challenge in some cases. Finally, I assumed that teachers got test items from two main sources: textbooks and themselves. Through the interviews with teachers, I learned that some districts host their own databases of test items and that many teachers share items with one another (within a school or district). Differentiating our database from others out there will be important.

Leave a Reply

Your email address will not be published. Required fields are marked *