Texas Legislature to OER Community: Where are you?

Last week I had the chance to chat with Rep. Scott Hochberg, a Texas legislator from Houston who has been heavily involved in getting OER policy passed in his state. He’s had remarkable success with bills like H.B 2488. This bill, in particular, has allowed the state board of education, which creates the approved instructional materials list for Texas, to include OER on that list. This is only POTENTIALLY impactful policy because, while school districts have authority to use OER, there has not been a lot of motivation to use it or go looking for it. It’s inefficient to have 1000 school districts looking for and approving OER.

As Rep. Hochberg explained it, the problem is that the OER community (especially producers of OER directed at K-12 audiences) do not seem too interested in getting involved in processes where they have to do something to get materials approved. In fact, traditional publishers have continued to have great success in Texas, despite all policy doors being open for OER. One of the reasons traditional model continues to dominate is because there are strong economic reasons for traditional materials publishers to jump through all the hoops. For instance, there is a big profit motive for Pearson to submit its textbooks. The same motivations are not there for OER producers. Thus, if you’re an administrator and you have Pearson knocking on your door and showing you stuff (which, by the way, is already approved by the state board), then OER becomes less appealing – even though it’s cheaper and possibly of equal or better quality.

In the end, Rep. Hochberg expressed his frustrations with the OER community not doing more to “get in the faces” of potential OER adopters at the K-12 level in Texas. There is so much potential in Texas right now. At least one policymaker is crying out for the OER community to take advantage of it!

Intro to OER – NRMERA 2012 – Park City, UT

I’ve posted the slide deck from the OER Workshop I gave at the Northern Rocky Mountain Education Association meeting in Park City, UT last Friday.

Check it out on Slideshare.

The first part of the deck is a remix/revision of some of David’s earlier slides. For those who’ve already seen David’s excellent intro to OER, skip to slide 37 for slides on OER policy, implementation, business models, initiatives, and research.

OpenAssessment.org Pain Test

The purpose of this post is to conduct the first part of a pain test on our edstartup idea. The general goal of a pain test is to better understand the specific pain that potential end-users of our idea are experiencing and thus refine our idea to more directly alleviate this pain. In this post, we answer six key questions that get at the underlying assumptions we have about the problem we are trying to solve.

1. What causes the problem?

There are several causes to the problem we are trying to solve.
First, teachers generally lack training and expertise in the assessment theory and practice. Teachers do not usually receive this training because assessment and measurement are often not the highest priorities for teacher preparation programs that focus more on pedagogy, curriculum development, and classroom management. Simply put, there just isn’t enough curricular room in pre-service training programs for courses that cover assessment and measurement in any useful depth or breadth. Teachers in higher education, on the other hand, rarely receive even a basic training in the teaching profession – let alone training in assessment.
Second, assessment development is time-consuming. Teachers who desire the highest quality assessments for their students will attempt to write tests themselves in order to align their assessments as much as possible with the curriculum in their classroom. The time spent writing assessments can be an incredible time sink. What’s more, without measurement expertise, these tests are often of suspect quality in terms of validity and reliability.

Third, assessment adoption is either very expensive or results in tests that are of unknown quality. Perhaps most often, teachers will adopt test items from other sources – like textbooks and test banks. Test banks containing high quality items are expensive (ETS spends upwards of $200 per item to develop their highest quality assessments – like the SAT and GRE – for example). In reality, most test banks are written by people with very little measurement expertise, and they are always written to broad, general standards (if at all). Rarely are items aligned specifically to any one set of state (let alone district or school or teacher) standards. As an example, one of our colleagues recalls with fondness his days as a graduate student when his advisor would pay his students $1 per item to help compile a test bank for a new textbook he had written. One can only guess how much attention was given to each item by those starving graduate students.

2. Think about the people with the problem. What are they currently doing, or willing to do, to solve it?

Teachers with this problem are generally not doing much to solve it, even if they want to. For starters, there are very few options for in-service or pre-service teachers to obtain training in measurement and assessment. Thus, we suspect that most teachers do not see this is a practically solvable problem or simply view low-quality test banks as one kind of solution. Despite this, some forward-thinking teachers have begun collaborating with colleagues within their school, district or institution on assessment development or item adoption initiatives.

3. What are all of the current solutions to the problem?

Current solutions to the problem of low assessment expertise among teachers are generally quite expensive. There are some assessment training workshops that teachers (or schools) can pay to attend, but these have deep costs in terms of fees, travel, and time.  We don’t know of any current solutions to the problem of inaccessibility to higher quality test items. The higher the quality, the more expensive the item. In all cases we know of, these costs are always passed on to the item user.

4. Why aren’t the current solutions good enough?

The expense of the current solutions is prohibitive, especially in the current economic climate. In fact, as the demand for high quality classroom assessment goes up, the money available to spend on training and materials for the creation of such assessment is going down.

5. How long has it been a problem?

Classroom teachers have always lacked training in assessment and measurement. However, the demand for high quality assessment is much more recent. Ever since the first education reform policies hit the ground in the 1960s (in the U.S. at least), the focus on measurable progress and assessment has increased with each subsequent policy revision. In addition, access to high quality test items has been prohibitively expensive for as long as such items have been available.

6. How easily could something change to make the problem go away?

This problem could be fairly easily solved with the use of open educational resources, a little bit of money, and current technologies – especially the collaborative potential of the Internet. You can read more about our proposed solution to this problem on a previous post.

With our general assumptions now explicated, our next step in the pain test is to conduct a little market research. Talking directly with teachers (and faculty in education programs) will help us verify these assumptions and make revisions to our ideas. Most importantly, this field test will help us know if the pain we’re trying to alleviate is as real and sharp as we assume.

OpenAssessment.org – Our EdStartup Idea

The Idea
We* want to create a collaboratively developed database of openly licensed assessment items and measurement training. All of the items in the database will be written by trained educators and other subject-matter experts from various content areas and at various educational levels. The items will be aligned to educational standards (where applicable) and made available to educators in multiple formats, including PDF download for free, use within an internal delivery and analytics system, and use within external delivery systems. The openly licensed measurement training will also be freely available to all users.

The Problem
Teachers lack the time, resources, and expertise needed to create high quality classroom assessments. And even in the rare cases where time, resources, and expertise are not limited, typical classroom assessments still lack the number of responses necessary for the kinds of measurement analyses that could help teachers improve their classroom assessments in a systematic way.

The Solution
A collaboratively developed database of openly licensed assessment items will allow teachers to improve the quality of their classroom assessments by (a) providing them with increased access to higher quality test items, and (b) providing them with access to measurement training and support. All users will have free basic access to all items in the database because the items will be openly licensed. In addition, the collaborative approach to item development will decrease the individual time needed to create high quality assessments by capitalizing on the inputs of a community of subject-matter experts. Also, item writing training and psychometric support will increase teachers’ practical measurement skills. Finally, items used within the internal delivery and analytics system will have large enough response numbers to allow for more sophisticated psychometric analyses and item improvement.

The Why
High quality assessment of student learning is important for making sound pedagogical and administrative decisions. However, many important educational decisions are based on assessment results that aren’t highly valid or reliable. This is especially true at the classroom level. But this is not because many teachers don’t want to improve their assessments. Most teachers couldn’t do much about this problem even if they wanted to. They not only lack the funds and training, but they simply don’t have enough item responses to conduct many psychometric analyses. Indeed, many teachers lack the time, software, training, and secure server space to even maintain a database of items and their psychometric properties. By providing teachers with increased access to this kind of database and to practical measurement training we can improve the quality of assessment in our educational institutions.

*Current intellectual contributors to this idea include myself, David Wiley, T. Jared Robinson and Dan Allen.

The Birth of OER Policy

In our vigor to advocate for open educational resources and policies that support their development, it would be wise to keep in mind some fundamental things about policy and the policymaking process itself.

1. OER policies should solve more problems than they create. Indeed, policy is “a set of answers to questions, a series of solutions to problems” (Cooper et al., 2004, p. 63)

2. The problems solved by OER and its related policies should be clearly defined and explained. “If a policy issue is not well-defined, it will not be perceived as important, making it difficult to attract enough attention to reach the policy agenda” (Fowler, 2004, p. 167).

3. Policies related to OER should come with solutions attached. “Even if policymakers want to address a problem [such as the rising costs of education], absent a proposed solution or plan of action, the problem cannot be addressed…[Some even] argue that public officials will not take a problem seriously unless there is a proposed course of action attached to it” (Cooper et al., p. 65).

4. The policymaking process is NEVER objective or ideologically neutral. “Before a policy choice can be made, a problem in society must have been accepted as part of the agenda of the policymaking system–that is, as a portion of the range of problems deemed amenable to public action and worthy of the attention of policymakers” (Peters, 1986, p. 39). What’s more, “cultural values, interest group advocacy, scientific information, and professional advice all help to shape the content of problem definition” and the policymaking process itself (Rochefort & Cobb, 1994, p. 4).

5. New OER policies should be clearly tied to older, related policies. “The more a new issue can be made to look like an old issue, the more likely it is to be placed on an agenda” (Peters, 1986, p. 47).

6. Advocates of OER policy should understand and use the power of the media (new and old) to shape policy. “Media influence on the process and content of policies varies according to type of issue, stage of process, time frame, and political and media systems. Under certain circumstances, the media can be a crucial player. It depends on what is covered, how often, and how it is framed” (Paletz, 1998, p. 234).

7. The language of policy can be as important as the policy itself. “Social” rhetoric and symbolism should be used in advocating for OER policies. “To heighten participation, issues may be connected to sweeping social themes, such as justice, democracy, and liberty” (Rochefort & Cobb, 1994, p. 5). And, “the more closely a particular problem can be linked to certain important national [or international] symbols, the greater its probability of being placed on the agenda” (Peters, 1986, p. 48).

8. OER policy reform should be influenced, if not led, by people with expertise in education. “What is significant about much of the interest group activity [in educational policymaking], and what distinguishes it from previous activism, is the prominence of non-educators leading the reform efforts” (Cooper et al. 2004, p. 76).

The Cryptotheory of Open Education

In a recent paper published in ETR&D, Yanchar and Gabittas (2011) discussed the space between eclecticism and traditional orthodoxy in educational design. They argued that while eclecticism seems to offer practical advantages by allowing researchers and designers to freely pick and choose among manifest features of various theories (choosing only that which seems practical), it “ignores or discourages critical reflection regarding implicit assumptions and values” of those theories.  The authors argued that all educational designers use “conceptual design sense” which “entails an [educational] designer’s assumptions and values – often unarticulated and unexamined – about the diverse aspects of the enterprise of instructional design.” Their point was that everyone engaged in designing instruction or content is operating under certain guiding principles and ideas about concepts like knowledge, mind, agency, learning, technology, and assessment. Those (it appears rare) individuals who operate within a traditional orthodoxy generally subscribe wholesale to the assumptions and values of this orthodoxy. But the much more common eclectic usually operates within “a kind of hidden framework” informed by various traditional theories, assumptions and values. And these theories, assumptions and values “to some significant degree guide important aspects of the [educational] design process.” Yanchar and Gabittas call this hidden framework a “cryptotheory.” As evidence of the existence of cryptotheories, Yanchar et al. (2010) found that “designers reported using theoretical principles eclectically in their work, but were often vague on which ones they uses (or their names), how they used them, and what guided their choices and applications.”

To manage these issues of design sense in eclecticism, Yanchar and Gabittas (2011) proposed an alternative approach to educational design, called critical flexibility. “Critical flexibility,” they argued “calls for a critical stance towards one’s own design sense and an awareness of alternative views that may facilitate the gradual development of one’s practice.” In a word, they offered a means of becoming aware of one’s own cryptotheory.

So what does all of this have to do with open education? It seems to me (though I haven’t done a specific study on this…yet) that the people involved in designing MOOCs, open courseware, open digital courses, and other open educational resources would be inclined to an eclectic approach – an approach to educational design that draws on many theories and uses those that seem most practical. Open education is all about practicality. However, Yanchar and Gabittas have got me thinking. How much better could these OER be if their designers were more critically aware of the cryptotheories guiding their work? In the same way, how much better would research on the effect of open educational initiatives be if the researchers and the funders were more self aware of the cryptotheories under which they operate? I’ve run into this personally where it was learned after the fact (meaning: after the check had been cut) that the funder and the researcher were operating under very different assumptions about educational outcomes and even about what constitutes learning.

What’s your cryptotheory?

Why I Advocate For Open Education

I asked my extremely intelligent wife to read my previous post before I published it to the world. She said she thought it was well-written (thanks, hun), but also said it sounded like a simple regurgitation of a bunch of things I’ve learned about open education and OER. She explained that based on the title, she was sort of expecting something along the lines of an opinion piece that explains why I think open education is important and why I spend so much time doing what I do. My wife was spot on. I did not intend to opine about the virtues of openness in that post. Instead, my intention was to provide a cogent technical argument for why and how open education, and OER specifically, can increase access to education. But the editorial conversation with my sweetheart got me thinking, and in between my pushing the publish button and her giving birth to our fourth child (good job, dear, you were awesome!), I decided to face the critical questions head-on: Why do I think open education is important and what drives me to study OER?

The short answer to both of these questions can be found in a single quote from an at-risk community college student: “I have no expendable income. Without this free OER textbook, I would not be able to take this course.” This student was considered “at-risk” of dropping out of college for primarily economic reasons.

Of course, there is a longer answer to why I advocate for open education.

First of all, I realize that education costs money. Teachers, in order to devote their lives to teaching, have to make a living. Research that generates new knowledge cannot be conducted in a vacuum (well, some types of chemical and astrophysical research require a vacuum, but that’s beyond the point). Technologies to assist in the contextualization and sense-making of new knowledge are not cheap to develop or to implement. The very important, but often overlooked, curators of knowledge (like librarians) cannot be expected to do their critical work pro bono. Valid assessments that show that sense has been made do not write themselves (trust me on this one; sometimes I moonlight as a psychometrician.)

In fact, I don’t know of many people who are trying to make an argument for completely free education. Sure, there are excellent opportunities now for individuals to gain access to content (and even pedagogical sense-making) at little cost to them personally. But, as argued above, the generation, distribution, and meaning-making of content all cost money at some time and in some way.

The real advantage of open education and OER, especially, is that they allow for a redistribution of the costs of education toward better ways to improve student success. For instance, money saved on textbooks through the adoption of open educational resources (like the nearly $25 million in estimated savings in Utah this year) could be used to hire more (and better) teachers, make the teaching profession more competitive, or promote the reality that teachers are, in fact, professionals (as well as help teachers improve their profession through advanced PD). As an aside, some may argue that perception of teacher professionalism is not an important educational factor. A recent report from OECD, however, would counter this by arguing that all of the top performing countries on the PISA have high public regard for teachers as professionals. The United States, on the other hand, does not (just ask any teacher you might know).

These are just a few examples of how OER could positively impact education. Other possibilities include redistribution of funds toward more educational research, improving teacher education programs, developing and implementing new educational technologies, and so forth. I haven’t included any direct student benefits of OER here, but they are very important as well and will certainly be discussed in future posts.

The main point is this: Education is not free. Those who want a credential should probably always be expected to pay something for it. But why should they continue to spend money on components of the educational enterprise that no longer cost much at all, like the copying and distribution of educational content? The at-risk community college student I quoted above was willing to invest in a formal, credentialed education. However, the significant – and now mostly unnecessary – costs of educational content threatened to become a barrier to her success.

Another point is this: Some education can be (essentially) free. Individuals who are not interested in, or cannot afford the base price, for a credential, now have access to massive amounts of educational content. These people can access the inherent benefits of knowledge without working toward a formal degree. And millions are doing so on a daily basis, thanks to generous and forward-thinking people who realize that they don’t need to keep the content (or even the sense-making) secure, just the credentialing process. MIT’s OpenCourseWare is a model example of this type of openness in education. And there are many others.

So there you have it. I advocate for open education because I see how it can break down potential barriers to student success and create opportunities for learning at every level of society in nearly every society on the planet.

I would love to know why you advocate for open education as well. Or, if you see holes in my argument, please engage and tell me where I’ve missed the boat…or even the train.

Why Open Education: The Technical Argument

Education is about sharing. Teachers share knowledge, skills, feedback, criticism and encouragement with students. Students share questions, understanding, and answers with teachers. Thus, sharing is at the very center of education; it is the soul of the educational endeavor. In other words, “If there is no sharing, there is no education.

Sharing itself is a fascinating concept. Most of the things teachers share with students are somewhat magical in that they can “be given without being given away.” When a teacher imparts her knowledge to a student, the teacher does not lose possession of her knowledge. In fact, some would argue that knowledge can even be deepened through the very act of sharing it with others. Yet, some things teachers share with students, like textbooks, must be physically given away. If I have a textbook and then I give it to you, I lose possession of the book and you assume it.

In recent educational history, textbooks have become the central modicum of the knowledge sharing process. As valuable as these resources can be to students (and to teachers), the rising costs in textbooks have begun to seriously impact the overall cost of education at all levels. Researchers and others are just now exploring the true impact of these costs, especially in terms of how they affect student persistence and completion rates in college and how they affect K-12 school budgets.

With the advent of the digital age, a breakthrough in the knowledge sharing process has occurred. For the first time in human history, knowledge and the physical expressions of knowledge (textbooks and other educational resources) can both be “given without being given away.” For example, the Internet allows for all of the content in a newspaper to be distributed to millions of subscribers without a single physical transfer of paper. In the physical world, the cost of copy and delivery per object is quite high. For example, the cost to copy one 250-page book by hand is roughly $1,000. The cost to copy the same book by print-on-demand services is about $4.50. But the cost to copy the book digitally is only $0.00084! And this isn’t all. The cost to distribute the same 250-page book by mail is about $5.20. The cost to distribute it online: $0.00072.

There is more to the educational sharing process than just copying and distributing knowledge, however. Once in possession of physical expressions of knowledge, the real craft of teaching is in sense-making and meaning-making. Teachers do this by reconnecting new knowledge with prior knowledge and/or relating it to past experience. In a word, teachers are master editors of content. Because of the high cost of editing physical expressions of knowledge like textbooks, teachers accomplish their sense-making through verbal communication or digital summaries. But in this brave new world of digital content, editing physical expressions of knowledge has also become essentially free.

With the removal of the economic constraints on copying, distributing, and editing content, we are able to share (and educate) like never before. Except we can’t, because copyright laws prohibit and otherwise restrict the copying, distributing, and editing of content. In essence, what the Internet enables, copyright forbids.

To solve this problem and truly open up education in the digital world, some very innovative movers and shakers envisioned a way to use existing copyright law to enforce sharing. Inspired by the open source movement in software development, in 1999 David Wiley and others started thinking of educators as publishers and created the first open publication license – a copyright license designed to legally certify that a particular piece of digital (or even analog) content could be reused, redistributed, remixed, or revised. David et al. were not, however, legal experts, and the validity of their license was questionable. Thankfully, a few years after the initial inception of open educational licenses, Larry Lessig (a certified legal genius) and Creative Commons developed a legally valid way to “minimize legal, technical, and social barriers to [the] sharing and reuse of educational materials.” Creative Commons licenses allow individuals and institutions to develop educational materials that can be legally copied, distributed, and edited.

The open in “open education” and “open educational resources” is the free permission to reuse, redistribute, remix, and revise educational materials (now commonly called “the 4 Rs”). Openness is the means whereby we can overcome the restrictions of copyright law and use digital content to its full educative potential. What the Internet enables, OER now allows. And what the Internet enables and OER allows is sharing and educating at an unprecedented scale.

This post presents an introduction and a technical argument for open education. It can also be found on a permanent page here. Future posts will address the political, financial, serendipity, quality, innovation, and moral arguments of openness in education.

Note: Many of the ideas, as well as the framework, for this post came from a single presentation and myriad personal conversations with David Wiley.