Education and a Civil Society: Teaching Evidence-Based Decision Making


Back to table of contents
Eamonn Callan, Tina Grotzer, Jerome Kagan, Richard E. Nisbett, David N. Perkins, and Lee S. Shulman
Teaching Evidence-Based Decision Making in K-16 Education

Lee S. Shulman

I write the introduction to this collection of papers at the conclusion of the historic 2008 presidential election. This election is germane to this collection because it highlights the responsibility of citizens to weigh the arguments and proposals of competing candidates and to support those whose proposals are most reasonable and whose reasoning is most compelling. But voting citizens are also evaluating the characters, beliefs, personalities, and life histories of candidates. Should these judgments and evaluations be pursued through critical analyses of data and arguments, through the exercise of intuitive empathic “gut” feelings, or through some combination of both?

Is it a worthwhile goal for every citizen in a democracy to exercise good judgment? If such a goal is worthwhile, can good judgment be taught? Can the wherewithal to think critically, analytically, and soundly be learned by students, whether in the precollegiate years or in colleges and universities? Can these ways of thinking and reasoning be taught in formal educational settings? Even if they can, are the most important determinants of good judgment to be located in developed cognitive capacities or in the development of good character?

The founders of democratic societies disdained the claim that critical thinking and good judgment were hereditary gifts that were possessed only by a small, elite segment of the society. If such talents are not innate and not limited to a select aristocracy of women and men—if, in fact, these modes of thought and action are teachable—should they be treated as “basic skills” comparable to reading, writing, calculation, and—in our present era—fundamental technological literacy? Because evidence-based decision making is unlikely to be learned in the family or the neighborhood, should schools not be responsible for teaching this skill from kindergarten through graduate school? The question then becomes whether teaching evidence-based decision making really matters, whether it is possible, and how it might be accomplished.

An alternate view would subordinate evidence-based decision making to the development of desirable values, beliefs, attitudes, or habits. Other virtues may indeed be far more important than critical reasoning—such as kindness, empathy, love, loyalty, fairness, fidelity, beauty, or faith. We may err if we place too much emphasis on the academic values of reason and not enough on those in the moral, humane, religious, and aesthetic spheres. Which is better: to have leaders of questionable virtue reasoning clearly and relying on carefully collected and weighed evidence; or to rely on fundamentally decent, sensitive, and just men and women who will use evidence imperfectly but nevertheless in a humane and considerate manner?

This volume includes five essays commissioned by the American Academy of Arts and Sciences to respond to these questions. The respondents include three psychologists: Jerome Kagan, Richard Nisbett, and David Perkins; a science educator, Tina Grotzer; and a political and moral philosopher, Eamonn Callan. They offer quite different perspectives on these common questions, perspectives sufficiently distinctive that finding common grounds for comparison is frequently difficult.

Three of the authors—Nisbett, Perkins, and Grotzer—gather the growing evidence that critical judgment, probabilistic reasoning, and clear thinking can be taught, whether in general or specifically in the teaching of particular disciplines, such as the sciences. They further argue that once learned, such modes of thinking can, with limits, be transferred to other domains and are therefore likely to lead to more general capacities to think critically and effectively. Kagan scoffs at such claims for generic critical-thinking abilities, insisting that good judgment is much more likely to be domain, discipline, and context specific. He further expresses doubt that cognitive capacities alone are sufficient. Callan shares a modicum of Kagan’s skepticism. He reflects on how difficult it has been for him, a seasoned philosopher and experienced scholar, to learn to think across traditional disciplinary boundaries. He suggests Americans should learn to reason about and embrace questions of values and moral choice rather than learn to weigh likelihoods and the warrant of empirical evidence.

Concern for whether higher education is indeed “adding value” to the abilities of undergraduate students has now reached the state and national policy arena. U.S. Secretary of Education Margaret Spellings appointed a commission—under the chairmanship of Charles Miller, who had led the Texas higher education commission—to examine the quality of U.S. higher education. One of the central items on the commission agenda was whether colleges and universities were holding themselves accountable for improving the quality of knowledge and thought demonstrated by their students. While the value added in specific professional programs, such as law, engineering, and dentistry, might be quite obvious, what is added by undergraduate liberal education?

Although most of the critics of the Spellings Commission attacked the specter that multiple-choice tests might be employed to answer that question, the most frequently cited example of such an assessment was not a multiple choice test but an open-ended essay exam, the Collegiate Learning Assessment. This assessment of critical thinking and evidence evaluation has been designed by some of the nation’s premier scholars of assessment and cognition and seeks to measure the gains in general critical thinking and reasoning abilities achieved by college students over the course of their undergraduate experience.

Americans may tend to orient toward general processes of thinking and problem solving rather than to deep substantive understanding within specific disciplinary domains. This, at least, appears to be the perception overseas. Several years ago, a group of colleagues and I met at Rhodes House at Oxford University to discuss the first draft of our book Educating Lawyers (Sullivan et al. 2007). At a critical point in the two days of discussion, one of the British colleagues asserted—with substantial concurrence from his fellows—that the trouble with U.S. legal education is that its goal is to teach students “to think like lawyers,” to engage in the analytical reasoning associated with the processes, strategies, and dispositions of the law rather than with the concepts, principles, and facts of the law—the “black letter law”—that are the core of legal understanding and competence.

My colleagues and I at the Carnegie Foundation have for the past decade been studying education in a number of professions—law, clergy, engineering, medicine, nursing, scholarship (the Ph.D.), and school teaching. We are persuaded that education for the learned professions is not a matter of intellectual development alone, though that is a central challenge of every professional school. The candidates in these fields must also develop high levels of technical skill; they must learn to do, perform, and act. Even though technical accomplishments are often disdained by the academy (note the lower prestige of “clinical appointments” in academic schools of medicine, nursing, or education), they are the attributes that distinguish the most accomplished professionals. Yet even technical excellence is insufficient. The well-educated professional must develop an internalized sense of responsibility, an integrity that is inherent in their identities and embodied in their thought, skills, and capacities for responsible judgment and action. In the Carnegie studies we refer to these three kinds of learning as habits of mind, habits of practice, and habits of the heart. I am persuaded that this threefold representation would also map nicely onto the learning needed by citizens: to think and act with integrity is no small feat.

The readers of these essays will find them challenging in their disagreements, but need not make an either/or choice among the several perspectives presented here. They are not mutually exclusive because all, in their way, promote one of the most powerful ideas in contemporary thought, that of ‘practical judgment.’ When we say that we aim to help students develop the capacities to reason critically and act responsibly, we may be saying that we wish them to learn to engage well in the process of practical reason. Whether choosing a car to buy, a candidate for whom to vote, a medical treatment to elect, or an engineering design to pursue, individuals reason practically as they mull their alternatives. They do seek and evaluate empirical evidence in connection with their possible choices. How much fuel does a hybrid car really save, and how much lower is the amount of pollution it pours into the air? What is more important, a candidate’s positions on Iraq, abortion, evolution, prayer in schools, or healthcare? Do I prefer a leader with the “right” policies but questionable character to one who appears honest and sincere but disturbingly uninformed about important issues?

These are the kinds of questions that call for the exercise of practical reason, a form of thought that draws concurrently from theory and practice, from values and experience, and from critical thinking and human empathy. None of these attributes is likely to be thought of no value and thus able to be ignored. Our schools, however, are unlikely to take on all of them as goals of the educational process. The goal of education is not to render practical arguments more theoretical; nor is it to diminish the role of values in practical reason. Indeed, all three sources—theoretical knowledge, practical know-how and experience, and deeply held values and identity—have legitimate places in practical arguments. An educated person, argue philosophers Thomas Green (1971) and Gary Fenstermacher (1986), is someone who has transformed the premises of her or his practical arguments from being less objectively reasonable to being more objectively reasonable. That is, to the extent that they employ probabilistic reasoning or interpret data from various sources, those judgments and interpretations conform more accurately to well-understood principles and are less susceptible to biases and distortions. To the extent that values, cultural or religious norms, or matters of personal preference or taste are at work, they have been rendered more explicit, conscious, intentional, and reflective.

In his essay for this volume, Jerome Kagan reflects the interactions among these positions by arguing:

We are more likely to solve our current problem, however, if teachers accept the responsibility of guaranteeing that all adolescents, regardless of class or ethnicity, can read and comprehend the science section of newspapers, solve basic mathematical problems, detect the logical coherence in nontechnical verbal arguments or narratives, and insist that all acts of maliciousness, deception, and unregulated self-aggrandizement are morally unacceptable.

Whether choosing between a Prius and a Hummer, an Obama or a McCain, installing solar panels or planting taller trees, a well-educated person has learned to combine their values, experience, understandings, and evidence in a thoughtful and responsible manner. Thus do habits of mind, practice, and heart all play a significant role in the lives of citizens.

Our writers offer a rich array of perspectives on the potential for the educational process to inform and enrich this complex set of processes. I commend their offerings to the readers of this volume: you will find them stimulating, disturbing, and enlightening in turn.


Fenstermacher, G. 1986. Philosophy of research on teaching: Three aspects. In Handbook of Research on Teaching, 3rd ed., ed. M.C. Wittrock, 37–49. New York: Macmillan.

Green, T.F. 1971. The Activities of Teaching. New York: McGraw-Hill.

Sullivan, W.M., Colby, A., Wegner, J.W., Bond, L., and Shulman, L.S. 2007. Educating Lawyers: Preparation for the Profession of Law. San Francisco: Jossey-Bass.