Embarking on a grand experiment in conceptual teaching

I pride myself on being a conceptual teacher: although you will likely learn a lot of interesting facts in my classes, the focus of my teaching is on ideas and concepts. I want my students to understand how things work and why things exist, and both of these pursuits are fundamentally conceptual in nature. Almost all of my classes already emphasize a conceptual approach. In my introductory Great Adventures in Evolution course, I ask students to extract important scientific concepts from stories about the work and discoveries of well-known scientists. My Ecology and Evolution of Sex courses both challenge students to represent scientific concepts through a creative work. And in all of my courses I try to create visuals, discussions, and activities that are highly conceptual.

But how effectively do I foster conceptual understanding in my students? And am I focusing on the right concepts in my classes? These two questions are a bit hard to answer, because they require some assessment.

Assessment comes in many forms: we are constantly assessing things we do in a lot of informal ways, and occasionally we create formal assessments. Scientists take formal assessment a step further by trying to remove sources of bias and/or uncertainty from the assessment process and analysis of the data assessment produces.

Do I scientifically assess how well I foster conceptual understanding in my students? Clearly the answer is “no”, although this does not mean that I do not make formal assessments and or have a collection of data to work with. Every test or quiz my students have taken, every assignment my students have completed, and every day’s discussion in class provides a lot of informal feedback on how well I am teaching. I ask my students a question or send them off on a task, and how well they are able to meet my challenges gives me some idea of how well I am teaching. Perhaps the very best assessments I have are the major projects that my students produce, some of which effectively convey understanding and some of which effectively convey confusion.

But I would be the last to say that I have or even could have looked at these assessments scientifically. Most teachers work on intuition, eliminating what seems not to work well and trying to add new features on inspiration, and that pretty much describes what I do. Even if I were to start using a more scientific approach to assessing how well I teach key concepts, what concepts would I focus on? For most of my classes I have a little bit better answer to this question, especially because I have used concept mapping to deconstruct many of my lessons and even whole sections of some courses. But what I do not have for any of my courses is a concrete inventory of the concepts I seek to teach. Important concepts sort of melt their way into the curriculum, often because they are featured in a reading that I think is valuable to students. Occasionally I realize that students are missing a concept and actively add this concept into my lessons. But I never actually look at the overall “concept load” of any of my classes, and this can be a problem: in my ambition and enthusiasm I tend to just keep piling concepts and content into my courses until they become bloated and overloaded. Part of teaching effectively is deciding what absolutely must be taught and what can be left aside, and an inventory of concepts taught would force me to reckon explicitly with the breadth, depth, and overall burden of what I am trying to teach.

So as you might guess from that long-winded introduction, this semester I am trying to more explicitly and systematically plan and assess my conceptual teaching. I have chosen as my target The Evolution of Cooperation, a course that I simultaneously love and detest. My love for this course stems from its topical focus: what I care most about in both my research and teaching is to better understand and explain the conditions that foster cooperative behavior. My hate for this course stems from its history: although I have pumped a lot of time and energy into the course, it has gotten mixed reviews from students, and I have failed to settle in on a “method” for teaching the course that works. I have tried all manner of research and creative projects as well as different homework assignments and class formats, none of which have felt coherent or effective.

One of the struggles of this class is that it aspires to be an upper-level course. I should explain the context of this aspiration, because where I teach is a bit unusual. At Pratt, there are no majors in science. Non-major undergraduate students are required to take two courses within the Math & Science Department. For most majors, students can choose these courses a la carte, and rarely are there prerequisites for any course. This may seem odd, but it is almost necessary, because students have so little room in their schedule that requiring a prerequisite for a course is very likely to result in under-enrollment. Our students also come with tremendously variable backgrounds in science: some have taken three or four science Advanced Placement exams in high school, and some just squeaked by with whatever level of science learning met the minimal requirements. To deal with this diversity we offer courses at a variety of levels. A 100-level course is not the course you start with as it would be at other colleges or universities, but rather our most basic course for students whose general background in and motivation for science is least strong. The Evolution of Cooperation is a 400-level course — the highest course designation for undergraduates — so it is designed to address the needs of students with the strongest background and interesting in science. If students don’t pay attention to the course number and just sign up for the course because it fit into their schedule or sounded interesting, the course will be mismatched to the students it is supposed to serve.

But even if students pay close attention to the course numbers and I get the right students enrolled in this class, it still faces a number of challenges. The first is that I cannot assume that even the most ambitious of high school biology students to have a strong understanding of evolution or ecology. Both of these subjects have — at least historically — been treated as mild diversions from the major focus of high school biology, which is predominantly organismal. A lot of high schools offer electives in AP Environmental Science, and biology curricula do seem to be slowly embracing Dobzhansky’s “nothing in biology makes sense except in the light of evolution” idea. But it is still too early to assume that my students have been given the basic understandings of ecology and evolution that are required to understand how a more complex phenomena such as cooperation would evolve, so I need to teach ecological and evolutionary concepts along the way and as necessary.

And there is a final factor that needs to be considered about this course: it cannot require too much workload from my students. I guess this is true for all courses, but it is particularly true for general education courses at Pratt. Our students are so heavily loaded by their majors courses — especially those with a studio component — that they simply cannot dedicate too much time to other courses. I have learned this the hard way: the last two times that I have offered this course, I over-shot on the amount of work required and good students went bad on me as a result. For all my courses, I need to maximize the educational-payoff-to-work-input ratio, but particularly so in this course.

So here is what I can assume going into this Fall’s round of The Evolution of Cooperation  (I am teaching two sections of fifteen students each):

1. Many if not all of my students will at least be aware that they have signed up for a more demanding science course;
2. Most of those students will be motivated towards the subject, having chosen it over less specific and perhaps less demanding alternatives;
3. Most of my students will lack the fundamental background in ecology and evolution required to understand how cooperation evolves; and
4. All of my students will be faced with severe restrictions on how many hours they can spend on my course.

Reading this list, it is clear that I face a challenge, and that is why I am so excited to really assess the quality of my conceptual teaching this semester. Over the next month I will be:

• Completely reviewing what I have done in the past, including readings, homework, lectures, in-class activities and assignments, and exams;
• Inventorying all the concepts that I have been attempting to teach;
• Reorganizing these concepts to create the leanest, most effective framework for my course;
• Reassessing my readings, homework, lectures, and in-class activities and assignments to maximize their delivery of conceptual understanding;
• Designing formal assessments to provide clear feedback on whether students actually master the concepts that I am attempting to teach.

My hope is that I can create a dynamic feedback loop between what students experience as my teaching and how students perform on my assessments. I know this is obviously what assessment is always supposed to do, but I would submit that most of the time this is not what happens. Assessment is generally informal: I want to formalize assessment by explicitly designing my assessments to provide me with feedback on what I want to know about my teaching.

What kinds of assessments do I plan to use? Ah, this is where things get a bit uncomfortable. Right now, what I have conceived of is pretty conventional: quizzes and exams. My reason for this is simple: while I could use other forms of assessments (such as in-class activities or discussions), I do not want to waste valuable teaching time on assessment. I want the longer-form activities we do in class to be dedicated to learning, not assessment. So the easiest way to do assessment is through conventional testing. My assessment approach is going to be three-pronged: students will complete reading response questions (RRQ’s) after completing their readings and before arriving at class, the will complete post-class questions (PCQ’s) immediately after class, and they will complete two exams (a midterm and a final). What I hope will be different about my exams is their design: questions will be created with the goal of assessing whether or not particular coursework was effective. I am hoping through this assessment process to (eventually) learn what the best practices are for teaching students about how cooperation evolves. With so few students, I am definitely going to run into all of the statistical issues of small sample size, but I have to work with what I have.

One of the reasons that I am so excited about this project is that eventually I would like to write a textbook on The Evolution of Cooperation. Although many behavioral ecology and a few evolution textbooks contain sections on the topic, no one has provided an accessible synthesis of all that is known about how cooperation evolves. By learning the best ways to teach my students, I hope to write a better text, which would become part of my expanding Online Cooperative Resource.

I will be posting throughout the semester to provide a little window into this “grand experiment”.

The work described in this post is part of the larger Conceptual Teaching Assessment Project.