The Back Page

Direct Instruction rocks: Or does it?

by Richard Hake
David Klahr¹, in his provocative Back Page essay in the December APS News “Inquiry Science rocks: Or does it?” makes three points in his introduction:

1. “the relative effectiveness of different types of instructional ‘approaches’ is not always investigated with the same rigor that permeates all strong scientific disciplines–clear definitions, well-defined empirical procedures, and data-driven conclusions”;
2. “for many aspects of science instruction, ‘discovery learning’ is often a less effective way to teach than a direct, didactic, and explicit type of instruction”; and
3. some in the physics education community may regard point (2) as “a foolhardy heresy, while for others it may be a dark secret that they have been reluctant to share with their colleagues.”
   

I have previously stressed² the crucial importance of operational definitions in educational research and strongly agree with Klahr’s first point. And considering the work of Chen & Klahr³ and Klahr & Nigam⁴, I would agree with point (2) IF “discovery learning” is defined as by Klahr & Nigam⁴ as including near zero teacher guidance. However, as explained below, I would amend point (3) to read: “few physicists who read Klahr¹ carefully will regard point (2) as heresy, or a dark secret to be kept from their colleagues.”

In support of point (2) Klahr describes a 1999 experiment by Chen & Klahr³ which appears to demonstrate that for the instruction of seven-to-10-year-old students in the “control-of-variables strategy” (CVS), a direct-instruction-like pedagogy called “Training–Probe” produced better learning outcomes than a discovery learning–like method called “No Training–No Probe.” The details of both methods are fully described by Chen & Klahr in their article.

In a later 2004 report, Klahr & Nigam⁴ made what was later acknowledged by Klahr & Li⁵ to have been a mistake by calling the “Training–Probe” method “Direct Instruction” (DI) and the “No Training–No Probe” method “Discovery Learning.” This switch to loaded language led to a maelstrom of media misinterpretation (referenced by Klahr & Li⁵) in which it was often erroneously implied that Chen & Klahr³ had shown that “direct instruction” in all its various forms was superior to “discovery learning” in all its various forms, in much the same way that Klahr’s¹ Back Page essay could mislead some to think that “direct instruction” in all its various forms is superior to “inquiry science” in all its various forms.

Klahr & Li⁵ wrote: “In hindsight, we may have muddied the interpretation of our findings by incorporating popular terminology like ‘direct instruction’ and ‘discovery learning’ into articles and public presentations of [Klahr & Nigam⁴]. Only when we tuned in to the recent political debate in California² about the permissible amounts of ‘hands-on science’ vs. ‘direct instruction’ .... did we become fully aware of how easy it is for someone to pick up a terminology, and imbue it with whatever meaning suits the purpose of an argument.”

In his Back Page essay Klahr¹ attempts to better convey the meaning of Chen & Klahr’s³ “Training–Probe” and “No Training–No Probe” methods as follows:

The “Training–Probe” method is equated to “Type A” instruction in Klahr’s¹ Table 1: hands on materials; teacher designed experiment; probe questions, explanations, and summary by teacher; no student execution of experiment or observation of outcomes.

The “No Training–No Probe” is equated to “Type C” in Klahr’s¹ Table 1: hands on materials; student designed experiment; no probe questions, explanations, or summary by teacher; student execution of experiment and observation of outcomes.

But Klahr¹ correctly points out that the above descriptions must be supplemented by details if the methods are to be replicated, stating: “In our full scientific report...of course, each of the cell entries in the table was augmented by a detailed ‘script’ for how that component of the instruction was actually implemented, so that it could be replicated in other labs.”

Considering that the Chen & Klahr³ experiment concerns: (a) the process skill CVS, (b) seven-to–10-year-old students, and (c) the above descriptions of “Training–Probe” (TP) and “No Training–No Probe” (NTNP) methods, I think there is no reason for some in the physics education community to regard the apparent superiority of the TP over the NTNP in that study as surprising or grounds for heresy, as Klahr¹ suggests in point (3) of his introduction.

Consistent with the above, as far as I know, physicists who read Klahr’s Back Page essay carefully are not preparing to burn Klahr at the stake as a heretic. And–I think –for good reason: the apparent superiority of the direct-instruction–like TP over discovery-learning-like NTNP in the study of Chen & Klahr³ has almost nothing to do to do with the demonstration by physics education researchers^6-11 that discovery-learning- like “interactive engagement” (IE) courses are superior to direct-instruction-like traditional (T) courses for promoting conceptual understanding of Newtonian mechanics in introductory physics courses–by about two standard deviations⁸ in average normalized gain <g>.

For example, Fig. 1 shows a histogram⁸ of the average normalized pre-to-post-test gain
<g> = (<%post> - <%pre>) / (100%–<%pre>)..(1)
       = <%G>/max possible <%G>.....................(2)
achieved by “Interactive Engagement” (IE) and “Traditional” (T) courses. In Eqs. 1 & 2 the angle brackets indicate class averages on concept inventories [Mechanics Diagnostic⁶ (MD) or Force Concept Inventory⁷ (FCI)–see the Wikipedia entry on “Concept Inventories” at http://bit.ly/dARkDY] of conceptual understanding of Newtonian Mechanics for introductory physics courses.

It should be noted that: (a) a high positive correlation coefficient r = + 0.91 was found⁸ for post-test scores on the conceptual FCI test and the problem-solving Mechanics Baseline (MB) test⁹; and (b) references to 25 research reports of average normalized gains for IE and T courses consistent with ref. 8 are listed on page 12 of ref. 11.

This Fig.1 histogram showing the apparent superiority of discovery-learning-like IE courses over direct-instruction-like T courses is to be compared with the Fig. 1 histogram of Klahr, showing the–at first sight polar opposite–apparent superiority of direct-instruction-like “Type A” pedagogy over discovery-learning-like “Type C” pedagogy.

Here IE and T courses are operationally defined⁸ as follows:

IE courses are those designed at least in part to promote conceptual understanding through the active engagement of students in heads-on (always) and hands-on (usually) activities that yield immediate feedback through discussion with peers and/or instructors ......................... (A)

An elaboration of “A” has recently been given by Meltzer & Thornton¹⁰.

T courses are defined as those reported by instructors to make little or no use of IE methods, relying primarily on passive-student lectures, recipe laboratories, and algorithmic problem examinations.................................... (B)

As indicated above, Klahr¹ cautions that it’s necessary to provide a detailed “script” for how each instructional component of a method was actually implemented, so that it can be replicated in other labs. The “scripts” for IE courses are provided in ref. 8 since “A” is used as a proxy for the forty-eight IE courses (N = 4458) which are fully described in the companion paper “Interactive-engagement methods in introductory mechanics courses.” I think the T course “scripts” are so invariant and so well known to physicists and to students who have taken traditional introductory physics courses, that the abbreviated description “B” is adequate.

In conclusion:
A. Klahr’s1 Fig. 1 histogram and the research of Chen & Klahr³ and Klahr & Nigam4 suggest that if one’s goal is the enhancement of a process skill such as the “Control of Variables Strategy” (CVS) among elementary-school students then (s)he should probably consider utilizing Klahr’s direct-instruction-like “Type A” pedagogy rather than discovery learning-like Type-C method with near zero teacher guidance.

B. The present Fig. 1 histogram taken from ref. 8, its corroboration by others listed in ref. 11, and the high positive correlation of post-test conceptual FCI and problem-solving MB tests, suggest that if one’s goal is the enhancement of conceptual understanding and problem-solving ability among high-school or undergraduate students then (s)he should probably consider utilizing discovery-learning-like “Interactive Engagement” pedagogy rather than direct-instruction-like “Traditional” pedagogy.

Richard Hake is an Emeritus Professor of Physics, Indiana University, now living in Woodland Hills, CA. He is a Fellow of APS.

1. Klahr, D. 2012. Inquiry Science rocks: Or does it? APS News 21(11), 8, December; online at <http://bit.ly/WmqHMj>.
2. Hake, R.R. 2004. Direct Science Instruction Suffers a Setback in California - Or Does It? AAPT Announcer 34(2), 177; online as a 240 kB pdf at <http://bit.ly/aWsazm>. A 132 kB pdf version of the slides shown at the meeting is at <http://bit.ly/d40ksO>.
3. Chen, Z. & D. Klahr. 1999. All Other Things being Equal: Children’s Acquisition of the Control of Variables Strategy, Child Development, 70, __ 1098 - 1120; online as a 950 kB pdf at <http://bit.ly/VzI2A8>.
4. Klahr, D. & M. Nigam. 2004. The equivalence of learning paths in early science instruction: effects of direct instruction and discovery learning. Psychological Science, 15, 661-667; online as a 1.1 MB pdf at <http://bit.ly/UmU6E2>.
5. Klahr, D. & J. Li. 2005. Cognitive Research and Elementary Science Instruction: From the Laboratory, to the Classroom, and Back, Journal of Science Education and Technology 14(2), 217-238; online as a 549 kB pdf at <http://bit.ly/UrF0ik>.
6. Halloun, I. & D. Hestenes. 1985. The initial knowledge state of college physics, Am. J. Phys. 53(11), 1043-1055; online at <http://bit.ly/b1488v>, scroll down to “Evaluation Instruments.”
7. Hestenes, D., M. Wells, & G. Swackhamer. 1992. Force Concept Inventory, Phys. Teach. 30(3): 141- 158; online as a 100 kB pdf at <http://bit.ly/foWmEb> [but without the test itself]. For the 1995 revision by Halloun, Hake, Mosca, and Hestenes see <http://bit.ly/b1488v> (scroll down to “Evaluation Instruments”).
8. Hake, R.R. 1998. Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses, Am. J. Phys. 66, 64-74; online as an 84 kB pdf at <http://bit.ly/9484DG>. See also the crucial companion paper “Interactive-engagement methods in introductory mechanics courses” at <http://bit.ly/aH2JQN>.
9. Hestenes, D. & M. Wells. 1992. A Mechanics Baseline Test, Phys. Teach. 30:159-166l online (but without the test itself) at <http://bit.ly/b1488v>, (scroll down to “Evaluation Instruments”).
10. Meltzer, D. & R. Thornton. 2012. Resource Letter ALIP-1: Active-Learning Instruction in Physics, Am. J. Phys. 80(6), 478-496; online to subscribers at <http://bit.ly/MHQg6Q>. An abstract and outline are online at <http://bit.ly/O35gtB>.
11. Hake, R.R. 2008. Design-Based Research in Physics Education Research: A Review, in Handbook of Design Research Methods in Education <http://bit.ly/dkLabI>. A pre-publication version of that chapter is online as a 1.1 MB pdf at <http://bit.ly/9kORMZ>.