How do you know if you've learned something?

It's an important question. Consider the implications. If we had an answer, we could quantify the effectiveness of school. We could judge the quality of books, articles, and blog posts. And we could find new ways for startups to create value.

Merriam-Webster defines learning as "gaining knowledge or understanding of or skill in by study, instruction, or experience."

Unfortunately, this definition is non-actionable. As such, I'd like to propose a new working definition

You've learned something if you can answer a question that you could not have answered previously.

Before we proceed, a couple of caveats. First, this model only works for cognitive learning. That means things like muscle memory are out. Second, this model treats actions as implicit questions. For example, if we learn how to drive a nail then we've answered the implicit question "How do I use a hammer?"

I've challenged several people to find an example that breaks my model. So far no one has succeeded. Even if there is some corner case that breaks the model, I think it's pretty safe to say that it holds for 99 and 44/100% of cases.

The implications here are huge. I'll start with books and blogging, and then I'll tackle school and entrepreneurship.

Good (non-fiction) books answer questions. Often, but not always, the questions answered are right in the title. Glancing over at my bookshelf, I see the following

• Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A's, Praise, and Other Bribes. The question here is easy to spot: What is the trouble with gold stars, incentive plans, etc.?

• War Is a Force That Gives Us Meaning. Again, there is a clear implicit question: How does war give us meaning and what are the implications for the individual and society?

• What Is Calculus About? Self-explanatory.

As a rule of thumb, the clearer the question the better the book. Which perhaps explains why most textbooks are so poor. Next to me is a copy of a textbook called Marketing. This answers no question. Or if it does, it's a trivial and ill-formed question like, "What the deal with marketing, yo???"

So if you write a blog that answers no questions, does that mean you're stupid and your blog sucks? Not necessarily.

In The Age of the Essay, Paul Graham writes that

An essay you publish ought to tell the reader something he didn't already know. But what you tell him doesn't matter, so long as it's interesting. I'm sometimes accused of meandering. In defend-a-position writing that would be a flaw. [...] But that's not what you're trying to do in an essay. An essay is supposed to be a search for truth. It would be suspicious if it didn't meander.

Certainly true. But I suspect that what makes the best of Paul's essays so good is that they do answer interesting questions. And while they often meander, the meanderings add interest precisely because they introduce further interesting questions.

But enough about writing. What about school, yo??

So you do your homework. But have you learned?
And you go to lecture. But have you learned?
And you get your `A'. But have you learned?

Can I answer a question I could not have answered previously?

But if the goal isn't to learn, no need to ask yourself these questions. Or any questions for that matter.

(And if you think about it, I think you'll see that working harder doesn't mean learning more.)

12:01:30 AM Helen: i bet readers will ask at some point, the dichotomy between good and bad learning

Well, my theory is that information only becomes useful or non-useful because of the questions it answers. Like say, for example, you learn what Britney Spears ate for breakfast. Certainly you've learned a new fact. But perhaps there are more useful facts (or ideas or theories) we could have learned. How do we know?

The problem is there isn't really any way to compare two facts directly. After all, they're facts. The only way one can say that one fact is more important than another is because it answers a more important question. So if learning what Britney Spears ate for breakfast can be said to be trivial, it's only because the only question we can answer there is, "What did Britney Spears eat for breakfast?"

If facts, theories, and ideas are the atoms of human thought, then questions are the glue that binds them together. Information with no intuitive model connecting it is useless or gets forgotten or both. That's why you read a book like All Marketers are Liars and it's seared into your brain. But then you open a textbook called Communication and you're asleep by page three.

Instead of grouping facts and theories by the questions they answer, textbooks group facts by how `similar' they seem. This actively thwarts the sensemaking layer of human cognition, preventing us from transforming information into understanding. If you've ever read through a textbook that seemed like a list of unconnected facts, it probably was.

Which is a problem, since being well educated is a function of the questions you can answer, not the facts that you know.

There is this common misconception that we've entered the information age, implying that information is somehow intrinsically valuable. Nothing could be further from the truth. The driving factor of the last half-century hasn't been information, but information-technology. What is IT? Information-technology applies technology to information to answer questions that wouldn't be answerable by looking at raw data alone.

For a full theory of applying technology to information to answer questions, check out Bret Victor's essay Magic Ink. It is one of the more insightful articles on software design ever penned. The basic premise here is that web entrepreneurs should exploit their medium to answer the user's questions.

Incidentally, Paul Graham takes this one step by advising entrepreneurs to treat the business idea as a question.

The initial idea is just a starting point-- not a blueprint, but a question. [...] Treating a startup idea as a question changes what you're looking for. If an idea is a blueprint, it has to be right. But if it's a question, it can be wrong, so long as it's wrong in a way that leads to more ideas.

One last point before we're done. So who gets to decide if your question is answered?

To borrow some logic from Seth Godin at Gel 2006, if you think your question has been answered, it's been answered. Other people will try to convince you that they think they've answered your question. I DON'T CARE if you think you've answered my question. If I don't think you've answered my question, you haven't answered my question.

Comments

[Harry Chong]

Very interesting.

[Mark]

This sounds just like standard learning theory, i.e. learning is best done 'task-oriented'. When writing learning objectives, never use goals like "to understand" or "to learn" - instead say "how to do X".

Maybe I'm missing the point of the post?

[Jenni]

I enjoyed this article. I thought many of the points were consistent with learning theories as well as written in fun/humorous/layman terms. My question, however, is how do you judge whether a question is important or not? In my classroom, it is often that students get derailed from the original lesson plan as they explore questions that are meaningful to them... i.e., important to their lives. How do you compare the importance of one question to another? How would this translate to standardization techniques used in public schools? Based on these ideas, how would I address student's inquiry of "why is this important to know?"