Learning Outcomes: Academia’s Addiction to a Highly Problematic Model
Backwards course design is appealing because on one level it makes sense. The steps take the following form:
Step 1) figure out what’s important for students to know and do at the end of the course,
Step 2) write learning outcomes, (LOs)
Step 3) design the course so content points toward student fulfillment of learning outcomes.
This is often called backwards course design. Nothing is particularly wrong with backwards course design except that the weakness is in the application, in which case the good design intentions get perverted in every manner imaginable. Structurally, a course ends up being a series of steps, events, lessons, and activities that are prescribed by the learning outcomes. A, then B and then C and so on are covered, and by the end of the course, learners are expected to have attained the outcomes, whether they have or not. Students who do not fulfill the outcomes are penalized. Content is narrowed. Inquiry is diminished.
Learning doesn’t work this way. The how, the design of the course, replaces the facilitation of student learning.
“The concern is partly whether the faith in Outcomes Assessment is built on wishful thinking.” (Powell, 2011)
At one point, I was sold on the idea of backwards course planning, and while the overall outcomes can be useful, as I worked to design curriculum and then saw what occurred in practice, I became a little and then a lot skeptical. I now consider backwards course design a limited model that fits an administrative, or even a professorial mindset, that is ultimately a technocratic model to use the phrase of Icelandic philosopher and educator Atli Hardarson. I intend to explore his view but so that we are speaking the same language, I take Hardarson’s definition of technocratic to mean is that the course is designed by individuals on the basis of their expertise and knowledge base. This is made clear when it is seen that course design arises from a predetermined a priori frame into which students are situated and within which they conform rather than to have a more open framework that allows for meaning creation via different routes and in differing forms.
Generally, Hardarson takes aim at a model of teaching and learning as an instrumental means to terminal endpoints that students are supposed to meet. It is a prevalent view, one so widely presupposed both in curriculum design and in teaching to learning outcomes, that I suspect if asked, many professors wouldn’t be able to describe an alternative.
Academic leadership is able to cite numerous positive outcomes of LOs, some curricular based, some assessment based, some efficiency based, some economically based. Nicholas Addison (Addison 2014) has put together a comprehensive list and I’ll draw from that here. These perceived benefits include the idea that courses meet standards, the design is thought to link assessment and learning, clarity and parity is presupposed, and teacher-proof scripts may be a goal (as though teacher knowledge bases and viewpoints about their domain of expertise must be hidden). Courses become transportable, intra- and inter-institutionally, as well as from the physical classroom to a virtual learning space. Academic leaders can argue that their learning outcomes are guarantees the institution meets quality assessment standards of curriculum by governments or other organizations. They also believe the LOs allow them to identify and manage outmoded or unjust practices, and that they can account for and measure the impact of teachers.
As we will see, LOs are frequently touted as student centered and course design relying upon LOs are thought to enhance/empower student centeredness and to encourage all the good pedagogical goals, deep approaches to learning, intrinsic motivation, ownership, and evaluation metacognition. In fact however, a good deal of research suggests the exact opposite. We see a scenario where the creation of LO’s by experts or the academic leadership are privileged and students are thought to progress to attainment by a narrow range of ways and means, often against their own interests and identities.
Considerations and critiques of curriculum design are not new. Hardarson traces a lineage of curriculum design from Franklin Bobbit in 1918 to Ralph W. Tyler, to Benjamin Bloom, yes think Bloom’s Taxonomy, to and to more contemporary related applications of early ideas including the Bologna Process. While the history is not absolutely uniform, the thematic advocated by Tyler that the aims of education form a technocratic model.
The technocratic model of curriculum design, according to Hardarson and which has been highly influential “assumes the aims of education can and should be:
1. Causally brought about by administering educational experiences;
2. Specified as objectives that can be attained, reached, or completed;
3. Changes in students that are described in advance.”
Hardarson writes regarding the aim that education can causally be brought about by administering educational experiences, that “From the premise that every rational action has an aim, it does not follow that it brings about or contributes causally to some end or purpose that is distinct from the action.” The point is it’s not clear cut and there are alternatives. He uses an example by Peters (1973, p. 127) in which equality is an educational aim. In the example there are two ways this can be done. One can teach equality or have assignments around the topic ‘acting with equality’ yet this is very different than equality as a norm built into practices so that students experience and appreciate its value in an ongoing real-life manner, a practice that simultaneously exemplifies and contributes to the aim of acting with equality. In the first version administering educational experiences dovetails nicely with LO’s. In the second LO’s are more problematic as we will see.
With respect to education as the progress and attainment of specified learning outcomes (Hardarson uses the word “aims”) it is clear that some learning outcomes can be reached, going for a walk for example. Hardarson calls these closed aims. Other aims are open, more akin to ideals, such as staying healthy. Learning outcomes typically relate to closed aims. In a sense, this is why those who design learning outcomes advise not using words like “understand” because the verb does not describe one specific thing a student will be able to do upon the completion of the course nor is their progress of understanding or the outcome of understanding measurable. Thus, it is considered too vague to be used for learning outcomes. Course designers prefer carefully articulated closed aims.
A related problem arises when closed and open aims are mixed, For example a course objective might state, “The student will write three short stories of high quality.” Completing the three stories is a closed objective (versus subjective) aim and it is likely all students can meet this. The “high quality” is open, subjective, prey to interpretation and bias, and similar in vagueness to the verb “understand.” On one side of the coin, simply listing closed aims focuses, in the view of Hardarson, on the trivial. On the other, Hardarson prefers open aims in what would be a constructivist paradigm. The attempt by teachers to provide further clarity is found in the use of rubrics, which often are a more descriptive version of the learning outcomes that embody the same problematic issues. At any rate, in the example it is easy to imagine a scenario where attainment of the closed aim (three stories) would be easy and attainment of the open aim (high quality)could be nearly impossible for a wide range of reasons, from semantics to clarity to teacher bias to conflicting presuppositions about form and content.
Equally significant is the way in which LOs can create a classroom climate in which students’ differences, knowledge bases, learning rates, cognitive abilities, identities, and so forth are diminished or ignored. Parity, which academic leadership believes is occurring as a direcdt result of the LO design, becomes much like the cartoon that shows a variety of animals, a monkey, a penguin, an elephant, a goldfish, a seal, and a dog lined up before a teacher who says, “For a fair selection everybody has to take the same exam: Please climb that tree.” The technocratic model then becomes a behaviorist paradigm, in which certain normative behaviors are privileged. Hardarson says that it is “wrong to assume that educational aims can generally be completed or reached.” (p. 70)
As for describing student changes in advance, the technocratic model seems confident in such a prediction given that we have a continuum running from “principles of design” to “principles of reform.” (Hardarson, p. 67). Switching knowledge domains often highlights the strengths and weaknesses of an argument and so, imagine a therapist stating that they will describe their clients’ changes in advance, and I use plural purposely to indicate that a classroom is never one student.
Consequently, Hardarson describes two sets of course design characteristics. First there is the technocratic, defined by causation, closed aims, and top down planning. The technocratic design favors prescriptivism and a culture of control that Kevin Williams (Williams 2007) says leaves little room for spontaneity and creative responses to unforeseen questions and opportunities. According to Hardarson, thinking through a top down, technocratic lens can give unrealistic expectations of what can be accomplished. More fundamentally we “lack the intellectual repertoire needed to criticise educational aims and modify them to meet changing circumstances.” He continues, “If the focus is exclusively on goods that are caused by educative activities then opportunities to make learning an experience that is rewarding in itself will be missed.”
Second there is the non-technocratic course design in which curriculum may be designed so that the learning and general aims arise from the nature of the activities and course itself, they are subsumed, or absorbed, or exemplified in a way that becomes lived experience. The exact outcomes for each student in an open aim model may vary as they engage with the curriculum and make it meaningful to their own work. The structure then is bottom up, or to rephrase, a form of student centered inquiry in which the student formulates questions and makes discoveries as they create meaning and develop connections. But we now get to ask the question, is this good?
The answer is an unequivocal yes according to a review of inquiry-based learning literature, with a priority on the past ten years, was undertaken by Sharon Friesen and David Scott (Friesen & Scott, 2013) with respect to a shift away from education focused on the needs of an industrial past to a myriad of 21st century challenges. They suggest a shift has been prominent yet the model in use is one of presenting authoritative facts. They see this particularly in the teaching of science and history, but I’d suggest it’s the same in all disciplines. And they note that assumptions about a technocratic method based on students learning about a topic rather than taking part in the process of creating knowledge has “become so deeply ingrained in how we think about education that ongoing attempts at educational reform often fail to question the efficacy of organizing learning around elementis and aboutis.” Here the authors draw upon words by Perkins (2009) who says students view learning in two ways, via elementis: in which complexity is learned gradually, a sort of use A and B to get to C and so on; or via aboutis, in which students learn about a topic rather than how to do it. For studentst then, knowledge becomes a collection of facts to be conveyed and learned. The main point here is that both elementis and aboutis deserve to be questioned, especially in this day and age. Neither allow students to be, to be in, and to wholly engage as much as possible within the limitations of a course or formal learning institution.
A good deal of research backs up Friesen and Scott’s work, and one can reference the article on line to read more. Alternatives, based upon compelling evidence, include inquiry-based learning, authentic pedagogy and assessments, authentic intellectual work, and interactive instruction. Methods would involve project, problem, challenge-based learning, and inquiry-based learning. In conclusion they say there is a diverse and wide body of research that suggests inquiry-based learning positively impacts students’ ability to grasp core concepts and procedures.
The fact that Friesen and Scott provided this review to the Alberta Ministry is good given the tensions between humanities agendas and capitalist agendas in the province. A similar optimistic envisioning is found in Inspiring Education: A Dialogue with Albertans as focusing on engaged thinkers, ethical citizens, and entrepreneurial spirit (Alberta Education, 2010). The importance of lifelong learning and the availability of and access to post-secondary education is a fundamental goal. And finally, the Alberta Post-Secondary Learning Act rightly recognizes Alberta’s nationally and internationally known post-secondary education, the rights afforded to citizens to access it, and the benefits the education brings as people reach their full potential and contribute to society. I admire these documents and the work their authors put into them however I simply suggest that on the ground there remains the presupposition that education fits a ramped up capitalist agenda and the media jargon in Alberta often positions secondary education as job training, as it does in most provinces and states. But that aside, Friesen and Scott’s literature review is valuable.
Link their findings with best pedagogical practices and frame it with an open, big picture foundation, such as a through-line question and a solid curriculum design plan emerges.
It should be apparent that open-ended engagement with a topic allows for students to discover differing pathways to knowledge and meaning. It also means that teachers must give up a good deal of their power and control in the course. Any particular student may formulate a route of inquiry that opposes the beliefs of the teacher, and for those with a technocratic mindset, this might be a difficult challenge. Students may not all reach the same endpoint. They may not all “succeed” whatever that means here, to the same level but this in itself is no indicator of learning.
For example, many students who write up half-decent papers or give interesting presentations are only setting up straw man arguments to shoot down what is already common knowledge because in doing so they know they’ll get a good grade. They refuse to get into the slough of despond caused by challenges on all sides that make the issue at best a murky mess and one of ongoing and unresolved discourse. On the other hand, students who often take this approach, who in attempting to sort out some truth, find there is none to be had. And so by offering various positions as equally valid or murky they are penalized, when in fact they may be closer to the reality of the state of knowledge and the student thus gains deeper awarenesses as opposed to normative “successful” completion.
Another model is provided by the document Challenge Based Learning (Challenge Based Institute) with a tri-part framework of Engage, Investigate, Act. To engage is to “move from an abstract idea to a concrete and actionable challenge.” Students explore a big idea through “essential questioning” that turn into challenge, “a call to action to learn deeply about the subject.” The investigative stage builds from the challenge. Students are asked to contextual learning experiences, and to “conduct rigorous, content and concept-based research.” These arise by generating question that “include everything that needs to be learned to develop an informed solution to the challenge.” Prioritization and categorization occurs, in the sort of light version of grounded theory. Resources (literature) and activities (experiments) take place in a search for possible answers to questions. Finally, within a time limit, a synthesis of the work is asked for to demonstrate that learners have addressed the questions. In the third stage, students act by developing evidence-based solutions, I would scare-quote “solutions” keeping the output more provisional than conclusive as in the best of scenarios they kick the student back to asking more questions. The outputs are a finality and a new beginning, again reflecting the nature of knowledge. Embedded within the process is reflexivity, a feedback loop, and to restate, the framework can be aligned with best pedagogical practices.
We can hope that the best practices of inquiry based learning enter into all classrooms and I am always optimistic. However, Peters points out (1973, p. 123), the technocratic view “haunts all our thinking about the promotion of what is valuable.” Learning outcomes are plotted on various forms of grids or swot-like analyses, courses exist without any consideration of what student learning actually requires in terms of best course design and classroom practice. Thus, courses, progression, and programs remain in silos. Bloom’s Taxonomy and inquiry based learning remain locked into great in pedagogy courses without further application.
In the UK formal learning institutions of higher education have had to adopt learning outcomes under the aegis of the Quality Assurance Agency which in a knowledge economy are generally focused on transferrable skills. But this view is everywhere. As Nicholas Addison says in his article Doubting Learning Outcomes in Higher Education, “LOs have undoubtedly won the day.”
Addison has compiled an extensive look at literature outlining differing positions on LO’s. The list of deficits he found in the literature is considerable; further each deficit is often the tip of a deep weaknesses. I present the deficits at length below because in my view we won’t question learning outcomes unless we clearly see these deficits. (See the original article for citations of the deficits).
Deficits regarding inclusivity specify the inhibition of personal interests and emergent needs, a neglect of pedagogy as political, a failure to consider multi-contexts and difference, an encouragement of standardization, homogeneity, and micro management, and the closing down of critical, esoteric discourses.
Deficits around accountability specify the commodification of learning, a promotion the fiction of parity, a production of a technocratic culture, a promotion of surveillance and performativity (on the part of the students and the teacher), an undermining of academic independence and autonomy, a lack of learning for learning’s sake, a negation of teachers’ agency, a reduction of the teacher to a designer/assessor in which in some instances teachers become interchangeable.
Deficits with respect to curriculum design specify a dependance on language transmission that requires interpretation and negotiation, context specific material that often may be confusing to all but the author, a production of vague or detailed information, information overload, a failure to recognize learning other than that intended, a reduction of assessment to the easily measurable, a belittling of holistic approaches, a dismissal of tacit understanding, an undermining of situated, self-determined communities of learning, a reduction of discipline specificity and cultural inherence as a resource.
Deficits with learning specify the disabling of learning-centered, co-constructive methods, an impeding of responsive teaching (teachable moments), the perpetuating of false dichotomies, a rejection of the teacher as model or public intellectual, the production of cynical instrumental attitudes, the encouragement of teaching to the test, a rewarding of strategic approaches, the breeding of a culture of cynicism, the homogenization of learning identities.
Deficits with respect to Art and Design in specific include a militating against negotiated, emergent, and divergent processes, a discouraging of explorative and experimental teaching, the ignoring of domain specific learning, an unsuitability for complex, indeterminate process such as imagination, creativity, and risk taking, a neglect of Bloom’s affective, psychomotor skills and practical knowledge, a limiting of teachers’ opportunities for reflection-in-action.
This is a list that should cause anyone who cares about education to take a big step away from LO’s. But as Addison reflects, “Given their benevolent aims, the inclusive rhetoric of Learning Outcomes has made it difficult for critics to counter their pernicious effects. As a consequence, the resulting technocratic trap is well attested but ignored. Driven by assessment and management concerns over accountability and efficiency, LO’s deny the complexity of learning/teaching by rejecting its contingent, emergent and unknowable qualities.”
References:
Addison, N. (2014). Doubting Learning Outcomes in Higher Education Contexts: from Performativity towards Emergence and Negotiation. International Journal of Art & Design Education, 33(3), p. 313–325.
Friesen, S., & Scott, D. (2013). Inquiry-Based Learning: A Review of the Research Literature. A paper prepared for the Alberta Ministry of Education.
Hardarson, A. (2017). Aims of Education: How to Resist the Temptation of Technocratic Models. Journal of Philosophy of Education, Vol. 51 (1), p. 59–72.
Perkins, D. (2009). Making learning whole: How seven principles of teaching can transform education. San Francisco, CA: Jossey-Bass.
Peters, R. (1973). Authority, Responsibility and Assessing Educational Objectives: Applying the New Taxonomy. Thousand Oaks, CA: Corwin Press.
(Powell, J. (2011). Outcome assessment: conceptual and other problems. AAUP Journal of Academic Freedom, Vol. 2. Retrieved online from https://www.aaup.org/JAF2/outcomes-assessment-conceptual-and-other-problems#.XPSe6y_MyRd
Williams, K. (2007). Education and the Voice of Michael Oakeshott. Exeter, UK: Imprint Academic.
Challenge Based Institute. (n.d.). Challenge Based Learning. Retrieved online from: https://www.challengebasedlearning.org/framework/
