Twelve findings on why explanations fail, each with the bad version and the good version written out. Collected because the same faults kept turning up in a lecture theatre and in these guides.
One paragraph. Read it, then stop, and notice what happens in your head.
The arrangement matters more than most people assume. Begin with whatever will take longest, because the rest can be fitted around it while you wait. If there is too much, two rounds beats forcing it all in at once, since crowding is the usual cause of failure. Keep anything that behaves differently well away from the rest, since mixing them is a mistake people make exactly once. Most of the work is done for you, but nothing will judge on your behalf, so the decisions have to be made before you start. When it finishes, do not leave things standing, or you will have to begin again. Everything then goes back where it came from, and before long the whole business repeats.
Every sentence is grammatical. You knew every word. You cannot hold any of it, and in an hour you would have nothing.
It is about doing a load of washing.
Begin with whatever takes longest: the first load goes in while you sort the rest. Crowding is the usual cause of failure: an overstuffed drum washes nothing. Anything that behaves differently: the red shirt. Nothing will judge on your behalf: the machine does not know what is delicate. Do not leave things standing: wet laundry goes sour.
Nothing changed except the order in which you received three words. That is the whole of this page, demonstrated on you and never described. Study 1 below is the experiment it comes from.
The findings are below. These are what they add up to, and they are worth having before the evidence for them.
State the question before the answer, and the answer before the evidence. Every section opens with the thing it is trying to settle, then settles it in one sentence, then descends into why. A listener who does not know what is being answered cannot file anything they hear.
Give the frame before the content, never after. One paragraph, more general than what follows, hooked to something they already know. Arriving at the point eventually does not work: the experiment in study 1 shows that the frame given at the end performs no better than no frame at all.
One new term per sentence, defined in the sentence it first appears in. Never two new terms that define each other. The cost is not additive: see study 3.
Name the two or three gateway ideas and give each a session of its own. Some ideas block everything downstream until they are crossed. Spending twenty minutes on one is cheaper than losing the next four weeks.
Signpost out loud. “Three reasons, here is the first.” It costs four words and tells the listener how many boxes to open and when one is closing.
Make them produce it, more than once, spaced out. Meeting a term once does almost nothing. Being made to say it back a week later does almost everything. See study 7.
This is the half that gets skipped, and without it the six moves above decay, because nothing tells you which one you dropped. Each of these samples the whole room, which is the fault that matters: the people who answer questions are selected for already understanding.
The muddiest point. Last two minutes, everyone writes on a slip: what was the single thing that made least sense today? Anonymous, no names, collected on the way out. It costs two minutes, and the question is built so the only possible answer is a failure.
Make everyone commit before anybody speaks. One hard conceptual question, everybody votes at once, and you look at the spread before a single person explains. One confident student can no longer hide the room. Then: turn to your neighbour, argue, vote again. See study 8.
Wait longer. Measured average pause after a question is about one second, which only the person who already knew can use. Three to five seconds changes both how many people answer and how long their answers are. See study 9.
Audit the vocabulary once a term. List every technical word you used and ask people to tick the ones they could not define. Ten minutes buys you what you cannot feel in thirteen weeks, for the reason in study 4.
Explain it to one confused person and watch where they stop. Low volume, highest fidelity. Everything on this page came out of doing exactly that, repeatedly, and writing down where it broke.
It looks for a confirming instance and stops at the first one. Silence confirms nothing, and one person saying yes confirms that one person. The falsifying version asks for the failure directly: what made least sense, which cannot be answered with agreement. There is also a reason the silent ones stay silent, called pluralistic ignorance: each person privately thinks they are the only one lost, and takes the visible confidence of everyone else as evidence.
Each one is shut. The heading says what is inside. Open them in any order. Numbers 1, 3 and 4 do the most work.
What they did. Subjects read a short passage of ordinary English with no concrete nouns in it, all vague referents and generic verbs. One group read it cold. One group was told the title first. One group was told the title afterwards. Everyone then rated how comprehensible it was and tried to recall it.
What they found. Cold, people rated it near the floor and recalled almost nothing. Title first, comprehension and recall roughly tripled. And the third group, given the title at the end, did no better than the group that never got a title.
That third condition is the one worth keeping, because it kills the natural defence of a wandering lecture, which is that the point does arrive eventually. A frame that arrives after the material cannot reach back and file it. By then the material is gone.
Why it works. A schema is a structure you already have that new facts hang on. Washing, in the demonstration above. With one, each sentence gets a hook and is stored as part of something. Without one, each sentence is stored as an isolated string and decays within seconds.
The claim of the reading was that the scientific method comes out of the Greek study of nature. In the lecture that sentence appears about ten minutes in, as a subordinate clause, after Homer being an oral poet, a supervisor's red pen, Herodotus being an armchair historian, a map of the Mediterranean, the population of Athens, the School of Athens painting, slavery in Athens, and whether Socrates existed.
Every one of those is a brick of the argument. Not one of them had anywhere to go when it arrived.
Here is the question for today, and it is a question about you.
When you run an experiment, you think you are doing something modern. Controlled conditions, measurement, repeat and check. The reading says none of that is modern, and that the shape of it was fixed twenty-four centuries ago by people with no laboratories at all.
The argument runs through two words. The Greeks had physis, which we translate as nature, meaning the way a thing grows and behaves when left alone. And they had technē, meaning skill, craft, making: what a person does to force a thing into a shape it would never take by itself. Our word physics comes from the first and technology from the second.
The claim of the reading is that the split between those two words is where science comes from, and that you still work inside it. When you say an experiment shows what something really does, that is physis. When you worry the apparatus is interfering, that is technē contaminating physis. You inherited that worry and nobody told you.
Everything else today is the two people who built the split: Plato, who said the real thing is not the one in front of you, and Aristotle, who went and cut one open.
Homer can come next, and now he has somewhere to sit.
Bransford, J. D., and Johnson, M. K. (1972). Contextual prerequisites for understanding. Journal of Verbal Learning and Verbal Behavior, 11, 717 to 726.
The term first, because it misleads. An advance organizer is not a person and not an outline. It is a short piece of writing or speech, given in advance, pitched at a higher level of generality than the material it introduces, whose job is to connect the new material to something already in the listener's head.
What he did. Two groups read the same passage on the metallurgy of steel. Before it, one group read 500 words of organizer, framing the general ideas that the passage would make specific. The control group read 500 words of historical background about the same subject. Same length, same reading time, same topic, so attention and effort are held constant.
What he found. The organizer group retained substantially more, and the effect was larger for people who knew less about the subject beforehand.
The part people get wrong. An outline is not an organizer. “Today: positivism, Kuhn, paradigms, incommensurability” is a list of labels you cannot yet read. An organizer is an idea, usually one you already hold, that the new material will modify.
So today we're going to talk about Kuhn and scientific revolutions. And this marks a break with what is called positivism, which describes a positive paradigm in the period of the Enlightenment. The Enlightenment is Francis Bacon's era and underpins the major principles of the natural science. This approach is based on non-contextual, formal and standard research that literally separates distinct variables. Schadewaldt is more critical of this, and Lynn White acknowledges the role of Christianity. And both these people are living through World War II, which is a massive amount of technology put to extreme harm.
Nine proper nouns and four abstractions inside two minutes, and no question anywhere that any of them answers. This is a list of labels, which is the thing Ausubel showed does not work.
You already believe something about science and you have never been asked to look at it. You believe it accumulates. Each generation adds to the last, mistakes get corrected, the pile gets bigger and truer. That belief has a name, and almost everybody held it, including every author we have read so far.
In 1962 Thomas Kuhn wrote a book saying it is wrong. His claim is that science stands still for long stretches and then lurches, and that after a lurch the new science is not the old science plus more. It answers different questions, so there is no scoreboard both can be measured on.
If he is right, then “science progresses toward the truth” is a sentence we are not entitled to use. That is what this term is about and today is the argument for it.
Everything I say for the next hour is one of two things: the view Kuhn attacked, or a piece of the attack. I will tell you which, every time.
The belief that science accumulates is something the listener already has. That is the hook. The rest of the hour now modifies something, where before it was building on nothing.
Ausubel, D. P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267 to 272.
The mechanism, which has a name worth keeping: element interactivity. Learning four unrelated facts is four separate small loads, and you can do that. Learning four things where each one only makes sense once you have the other three means you must hold all four at once, and what you pay for is the number of relations between them, and never the number of them. Four items have six relations. Six items have fifteen.
Three kinds of load, since the distinction decides what to cut. Intrinsic load is the difficulty of the material, which you cannot reduce without teaching something else. Extraneous load is difficulty added by how it was presented, which is free to remove and is almost always where the problem is. Germane load is the effort that goes into actually building the schema. Bad explanation spends the budget on extraneous and leaves nothing for germane.
The finding this produced. Novices learn more from studying worked examples than from solving problems, because solving consumes the whole budget on search. And it reverses: once somebody is expert, worked examples start to hurt, which is called the expertise reversal effect, and it is one reason a lecturer's own preference is a bad guide to what a beginner needs.
This marks a break with what is called positivism, which underpins the major principles of the natural science. This approach is based on non-contextual, formal and standard research that takes and literally separates distinct variables.
To parse this you have to hold positivism, non-contextual, formal and separating variables at once, and each is being explained by the others. There is no order in which they can be taken, which is why the sentence slides off, where a hard sentence would merely be hard.
Positivism. One idea: the only real knowledge is what you can observe and measure. Everything else, purposes, essences, God, is not merely unproven, it is meaningless, because no observation could settle it either way.
That is the whole of it. Hold just that.
Now a consequence they drew. If knowledge is observation, the person observing should not matter. Two competent people pointing instruments at the same thing ought to get the same reading. So good method is method that takes the person out.
And that is the claim Kuhn goes after.
Four steps, one new idea each, every step landing on the one before. Same information, and the load now adds up where before it multiplied.
Sweller, J. (1988 onward). Cognitive load theory. Element interactivity: Sweller and Chandler (1994). Expertise reversal: Kalyuga, Ayres, Chandler and Sweller (2003).
What she did. Pairs. One person picked a song everybody knows and tapped the rhythm on a table. Before the listener guessed, the tapper predicted how often a listener would get it.
What she found. Tappers predicted around 50 per cent. The actual rate was 2.5 per cent, roughly one in forty.
Why that is the right model for teaching. The tapper is not hearing taps. They hear the whole song, the words, the instruments, and they cannot switch it off. The listener hears knocking. The tapper's error is not arrogance or laziness. It is that the experience of knowing has overwritten the experience of not knowing, and there is nothing left to consult.
Two results that make it worse. Hinds (1999) found experts underestimate how long a novice needs at a task, and keep underestimating after being told about the bias and asked to correct for it. Nathan and Petrosino (2003) found experienced algebra teachers systematically mis-rank which problems students find hard, which is the expert blind spot: subject expertise actively degrades your judgment of difficulty.
So the practical consequence is not “try to remember what it was like”, because that has been tested and it fails. It is to build something external: a list of terms, a diagnostic, an anonymous slip of paper. Something that tells you, because you cannot feel it.
A term is used in passing with no gloss, because to the speaker it is no longer a term, it is a word. In two lectures these all went by undefined: ad hoc, teleology, developmentalist, metaphysics, positivism, natural philosophy, paradigm.
The listener's experience is not confusion about the word. The word eats the sentence it was in, then the next sentence, because they are still working on the first one.
Ad hoc is Latin, “for this”. An ad hoc move is one invented purely to rescue a theory from one particular objection, with no independent reason to believe it.
Here is one. Your theory says all swans are white. Somebody shows you a black swan. You say: that is not a real swan. You have saved the theory and learned nothing, because you invented the rule “real swans are white” for the sole purpose of surviving that one bird.
So when somebody says “that is ad hoc”, the accusation is: you patched it, you did not find anything out. It is one of the standard insults in this subject and you will meet it often.
Name, plain meaning, one concrete case, and what the word does in an argument. The term is now usable, where before it had merely been heard.
Newton, E. (1990). The rocky road from actions to intentions. Stanford doctoral dissertation. Term from Camerer, Loewenstein and Weber (1989). Also Hinds (1999), Nathan and Petrosino (2003).
The claim. Meyer and Land argued that within any subject there are a small number of concepts with four properties at once. They are transformative: once you have one, you see the whole subject differently. They are irreversible: you cannot go back to not having it, which is where the curse of knowledge comes from. They are integrative: having one makes previously unrelated things connect. And they are troublesome: counter-intuitive, or in conflict with what you already believe, which is why they take time, where a definition would do for anything else.
The practical consequence is that effort should be distributed unevenly. A syllabus that gives each week equal weight is wrong, because two of those weeks contain a gate and the rest are a list.
Diagnosing one. It is a gate if people can repeat the definition and still get every application wrong, and if their errors are all the same error. That pattern means they have the words and not the idea.
“Socially constructed” used as if it were ordinary English, somewhere in week 2, with a one-line gloss. It is then used every week after that, in a different sense each time, and nobody notices because the phrase stayed constant.
The failure is invisible from the front of the room. People nod, because they recognise the phrase.
We are going to spend the whole hour on one phrase, because the rest of the term runs on it and it is three words hiding seven different claims.
Weakest end. A twenty dollar note is worth twenty dollars because everybody acts as though it is. Stop acting that way and it is paper. Nobody disputes this, and the note is completely real: forging one still gets you arrested.
Strongest end. There is no fact about whether atoms exist apart from scientists having agreed to say so.
Those two have almost nothing in common, and here is the test that separates them. Could the thing have existed if humans never had? Money: no. Atoms: yes, for thirteen billion years.
Now: five more positions sit between those two, and almost every argument you will ever see about this is two people standing at opposite ends shouting.
An hour spent here is repaid every week afterwards, because from now on the question is always which one of the seven.
Meyer, J. H. F., and Land, R. (2003). Threshold concepts and troublesome knowledge. ETL Project occasional report 4.
The idea. Lave and Wenger studied apprenticeships: tailors, midwives, quartermasters. Learning there is not transmission of facts. A newcomer is given real but peripheral work, and moves inward as competence grows. The phrase for it is legitimate peripheral participation: legitimate because the work is real, peripheral because it is at the edge, participation because you are in it and never watching it.
Why it explains a bad seminar. The format is built for people who already share the vocabulary and the moves. When two of them argue, it is lively and productive and it is not an explanation of anything: it is two insiders testing each other's models, which both already have. A newcomer watching is not at the periphery of that practice. They are outside the room it happens in.
The fix is not to stop doing it. Argument is how the subject works and watching it matters. The fix is to name the move, which converts a performance into a demonstration and gives the newcomer an entry that costs no prior knowledge.
A student raises a point about Copernicus and the church. A fast exchange follows, three turns, ending in agreement. It is the liveliest ninety seconds of the hour.
Nothing was said about what kind of move was made, so the only people who gained were the ones who could already have made it. Everyone else learned that other people understand this.
Notice what he just did, because it is the move this whole subject runs on.
He did not say I was wrong. He took my premise, kept it, and then found one case where it gives an answer we both reject. That is a counterexample, and it is how claims get killed in philosophy.
You do not need to know more than me to do that. You need one case. Everybody in this room can do it by Wednesday.
The exchange stops being evidence that other people are clever and becomes a thing that can be copied.
Lave, J., and Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation. Cambridge University Press.
The two effects, which are separate. Spacing: the same total study time split across days beats it massed into one session, by a lot. Testing, also called retrieval practice: being made to pull something out of memory strengthens it far more than putting it in again by re-reading.
The result that surprises people. Roediger and Karpicke (2006) had students learn passages, then either re-study or take a test. Tested five minutes later, the re-studiers did better. Tested a week later, the retrieval group did substantially better and the re-study group had collapsed. And when asked to predict, students expected the opposite, which is why nobody adopts this voluntarily: the method that feels worse is the one that works.
What this says about a course design. A term in which a word is said once in week 2, never required again, and tested in week 13, has no mechanism for learning it. The problem is structural, and no amount of student effort reaches it.
Three exams, no essays, no problem sets, no quizzes. A term appears in a lecture, appears in a reading, and is next required under exam conditions eleven weeks later. In between, nothing in the course asks anybody to produce it.
And the defence, that looking words up is the student's job, fails on this finding before it fails on fairness: looking a word up once is a single encoding event, and single encoding events decay.
A term is named, explained, and used again the following week without re-explanation. It then appears in a problem set, where the student has to produce it from memory to get anywhere. It appears again three weeks later inside a harder problem.
Four spaced retrievals, two of them effortful, and none of it labelled as technique. That is the entire difference, and it is a property of the assessment design and not of the lecturer's warmth.
Cheapest version for a course with no problem sets: open each lecture with two minutes of “write down what these three words from last week mean”, uncollected. It costs two minutes and converts a dead term into a retrieved one.
Ebbinghaus (1885) for spacing. Roediger, H. L., and Karpicke, J. D. (2006). Test-enhanced learning. Psychological Science, 17, 249 to 255.
The story, which is the useful part. Eric Mazur taught introductory physics at Harvard with strong student evaluations and students who scored well on exams. Then he gave them the Force Concept Inventory, a set of conceptual multiple-choice questions written in plain language with no calculation in them.
The result. Students who could solve the equations could not answer the concept questions. They had learned to manipulate symbols while keeping the intuitions they walked in with. His own account is that a student asked him whether to answer “according to what you taught us, or how I really think about it”, and that this was the moment it became undeniable.
The point for anybody explaining anything. He was not a bad lecturer. He was a good one, and his blind spot was total, and the only thing that revealed it was an instrument designed to find a failure. Evaluations could not: they measure whether people enjoyed it. Exam results could not: they were measuring the thing he had accidentally trained.
What he built instead, called peer instruction. Pose one hard conceptual question. Everybody commits to an answer at once, by hand or clicker. Look at the distribution before anybody explains anything. If it splits, say so and tell people to turn to a neighbour and argue. Vote again. The second vote usually moves sharply toward correct, because a student who has just understood something is far closer to the confusion than the lecturer is.
Does that make sense to everybody? ... Yes? Good, moving on.
Three failures stacked. The question can only be answered by agreement. Nobody is required to commit, so silence reads as assent. And the one person who answers is selected for having understood, so the sample is guaranteed to be unrepresentative.
Hands down, eyes on me. Here is a claim: the old primatology got the animals wrong. Is that a sentence a relativist can say? A: yes. B: no. C: only in a weakened sense.
Everybody picks. Now, hands: A... B... C...
Right, that is roughly a third each, which means this is the thing to spend the next ten minutes on. Turn to the person beside you, say which you chose and why, and try to change their mind. Two minutes.
He now knows the state of the room in forty seconds, with a number, and no confident student could have hidden it.
Mazur, E. (1997). Peer Instruction: A User's Manual. Force Concept Inventory: Hestenes, Wells and Swackhamer (1992).
What she did. Mary Budd Rowe recorded classrooms and timed two gaps: the pause after a teacher's question before somebody speaks, and the pause after a student answers before the teacher responds.
What she found. Both averaged around one second. When teachers were trained to extend the first to three seconds or more, the changes were large and went in several directions at once: answers got longer, more students volunteered, unprompted contributions rose, students started responding to each other, failures to respond dropped, and teachers themselves asked fewer and better questions.
Why one second selects the wrong sample. Retrieving a half-formed idea and turning it into a sentence takes longer than retrieving a fluent one. At one second you have run a filter that admits only people with the answer pre-built, which is exactly the group whose understanding you do not need to measure.
Why it is hard. Three seconds of silence feels, from the front, like failure. It feels like nobody knows and the lecture has stalled. That feeling is the whole obstacle, and the fix is to count.
So why does Kuhn think that matters? ... anyone? ... Right, well, the reason is that paradigms are incommensurable, which means...
About a second and a half, then the lecturer answers their own question. Nothing was learned about the room, and the room has learned that questions here are rhetorical and need not be attempted.
So why does Kuhn think that matters? Do not answer yet. Take ten seconds and write one sentence.
[ten seconds of silence, which is uncomfortable and is the point]
Right. Read me what you wrote, even if you hate it.
Writing first removes the risk of speaking, extends the pause past any awkwardness, and gives everybody a thing to read out, so the people answering are no longer only the fast ones.
Rowe, M. B. (1972, 1986). Wait-time and rewards as instructional variables. Journal of Teacher Education, 37, 43 to 50.
The argument. Shulman noticed that research on teaching had split into two piles, and the thing a good teacher has fell between them. One pile studied subject knowledge. The other studied generic skill: classroom management, questioning technique. Neither captured what a good teacher of this specific topic has, which he called pedagogical content knowledge.
What it consists of, concretely. The misconceptions students reliably bring to this topic. The representations and analogies that work, and the ones that mislead later. Which parts look easy and are hard. The order that functions, which is almost never the order of the logic. And what makes this particular idea difficult, as opposed to difficult in general.
Why this matters for your question about good teachers. It is topic-specific and it is accumulated, which means it cannot be inherited from being clever, and it cannot be transferred from one topic to another. A world expert has none of it on first contact. Somebody who has taught the same hour thirty times and paid attention has a great deal.
And why watching good teachers does not give it to you. You see the output and not the decisions. You cannot see which order they rejected, which example they dropped last year, or which misconception they are steering around in a sentence that looks ordinary. Worse, watching a smooth explanation produces a fluency illusion: it feels like understanding, and understanding the content feels like being able to deliver it. Watching Feynman does not make you Feynman, and it can convince you it did.
Asked to explain the Copernican revolution, a lecturer who knows the history well says: “Copernicus was able to demonstrate, given all the tools everyone else was using, that in order for the planets to move in circular motion it needed to be the reverse. How? I can't explain more than that, I'm not that good at astronomy.”
That is a real sentence from a real lecture. The subject knowledge is there. The representation that would make it land has never been built, so the key example of the key week arrives as an admission.
Here is the problem Copernicus was solving, and you can see it from a car.
Watch Mars across a year and most of the time it moves one way across the stars. Then it stops, goes backwards for a few weeks, stops again, and carries on. That is real, anybody can see it, and it is the single most annoying fact in pre-modern astronomy.
If the earth is the centre and everything orbits it on circles, nothing should ever reverse. So Ptolemy added small circles riding on the big circles, and it worked, and by the 1500s you needed dozens of them.
Copernicus: put the sun at the centre instead. Now the earth is also moving, and faster than Mars, because it is closer in. So once a year we overtake it. And when you overtake a slower car, it appears to slide backwards against the hills behind it, even though it never reversed.
That is it. Same observation, and the annoying fact comes out automatically, where before it had to be patched.
No extra astronomy is needed to say that. What is needed is having worked out, in advance, which picture makes it obvious, which is precisely the knowledge Shulman named.
Shulman, L. S. (1986). Those who understand: knowledge growth in teaching. Educational Researcher, 15, 4 to 14. Fluency illusion: Rozenblit and Keil (2002) on the illusion of explanatory depth.
The rule. Minto, the first woman professional in McKinsey's consulting staff, wrote it for business writing and it generalises. Three parts. One: every document or talk answers a single question, and the answer goes first. Two: ideas at any level are summaries of the ideas grouped beneath them. Three: ideas in a group are the same kind of thing and are in a deliberate order.
The reasoning behind it, which is the part that matters. A reader can only hold a new fact if there is already somewhere to put it. Deliver bottom up and the reader has to hold every fact unplaced, waiting to find out what they were for, and the holding capacity runs out long before the point arrives.
Why writers do it backwards by default. Because discovery order and delivery order are opposites. You worked it out by gathering evidence, noticing a pattern, forming a conclusion. It feels natural and fair to walk somebody through the same path. It is the worst possible order for them, because they lack the thing that made the path navigable: knowing where it ends.
The test. Cover everything except the first sentence of a section. If it does not say what the section concludes, the section is upside down.
In the 1970s women started entering primatology, and they saw things the men had not seen. Not better eyesight. Something else.
Three sentences that withhold. Saw what? The reader stops and asks, which is what happened. Every noun phrase here is a promissory note: “things the men had not seen” names nothing.
Before the 1970s every textbook said the same thing: male primates are promiscuous, females are coy and stick to one male. It looked obvious because of an arithmetic. A female can be pregnant with one infant at a time, so mating with five males gets her the same number of offspring as mating with one, while costing her time and risk. Pure cost, no gain.
Then women entered primatology and watched the female animals. Female langurs leave their own troop to mate with males from another one. Across most troop-living species, females mate with several males each.
Sarah Blaffer Hrdy found what the female gets, and it is in a column nobody had been adding up. A male langur that takes over a troop kills the infants of the previous male. A male who might be the father does not. So a female who mated with four males has four males with a reason not to kill her infant.
The old account counted conceptions. Hrdy counted survival. Same behaviour, and on the survival column it is insurance.
Every sentence names the thing it is about. Nothing is held in suspense, and the reader never has to carry an unnamed object.
Minto, B. (1973, revised). The Pyramid Principle: Logic in Writing and Thinking. The classical name for the same concern is dispositio, arrangement, one of the five canons of rhetoric.
The logic first, because it is the reason the technique works. Popper's point about theories applies exactly here. “Does everybody follow?” can only be confirmed: silence confirms it, a nod confirms it, one person saying yes confirms it. A question that cannot come back negative is not collecting information. The falsifying version asks for the breakage directly: what was the single thing that made least sense today? That cannot be answered with agreement.
The two statistical faults in the usual method. First, the sample is selected on the outcome: people who answer are people who understood, so the more of them speak the better the room sounds and the less the silence means. Second, pluralistic ignorance: each lost person privately concludes they are the only one, using everybody else's visible confidence as their evidence, so the lost systematically do not report.
The technique. Angelo and Cross collected classroom assessment techniques that take minutes and need no equipment. The two that matter most are the minute paper, which asks what the main point was, and the muddiest point, which asks what was least clear. Anonymous, on a slip, last two minutes, collected at the door.
Why anonymity is load-bearing. Naming turns the slip into a public admission of not understanding, which is the exact thing pluralistic ignorance already suppresses.
And what to do with them. Open the next session with the three most common ones. That does two jobs: it fixes the confusion, and it proves to the room that saying you are lost produces a result, which is the only thing that keeps the slips coming.
Any questions? No? Great.
Unfalsifiable, volunteer-sampled, and it reports the opposite of the truth: the more silent the room, the more confident the lecturer becomes.
Two minutes, everybody, and this is not marked and has no name on it.
Slip of paper. Two lines. One: in your own words, what was the main point today? Two: what is the one thing that made least sense?
If you write “all of it” on the second line that is a useful answer and I would rather have it than a blank. Leave them on the desk by the door.
Everybody is sampled. The second question can only be answered with a failure. And permission to write “all of it” removes the last reason to say nothing.
Angelo, T. A., and Cross, K. P. (1993). Classroom Assessment Techniques, 2nd ed. Jossey-Bass.
This page exists because the same faults kept appearing in two places at once: in a lecture theatre, and in the reading guides on this site. The lecture transcripts supplied most of the bad examples. The guides supplied the rest, including the one in study 11.
The through line is one thing. Every finding here says some version of: the listener needs the top of the structure before the pieces, and the speaker cannot feel this, because knowing something destroys the memory of not knowing it. Which means the only reliable correction is external. Ask somebody where they stopped, and believe them.