In theory, tests are merely what their name implies: tests of what You learned in the class. In theory you shouldn’t have to prepare for a test in a class any more than you have to prepare for a blood test. In theory you learn from taking the class, from going to the lectures and doing the reading and / or assignments, and the test that comes afterward merely measures how well you learned.In practice, as almost everyone reading this will know, things are so different that hearing this explanation of how classes and tests are meant to work is like hearing the etymology of a word whose meaning has changed completely. In practice, the phrase “studying for a test “was almost redundant, because that was when one really studied. The difference between diligent and slack students was That the former studied hard for tests and the latter did not. No one was pulling all-nighters two weeks into the semester.
If tests truly were tests of learning, things wouldn’t be so bad. Getting good grades and learning would converge, just a little late. The problem is that nearly all tests given to students are terribly hackable. Most people who’ve gotten good grades know this, and know it so well they’ve ceased even to question it. You’ll see when you work how naive it sounds to act otherwise.Suppose you’re taking a class on medieval history and the final exam is coming up. The final exam is supposed to be a test of your knowledge of medieval history, right? So if you have a couple days between now and the exam, surely the best way to spend the time, if you want to do well on the exam, is to read the best books you can find about medieval history. Then you’ll know a lot about it, and do well on the exam.No, No, no, experienced students are saying to themselves. If you merely read good books on medieval history, most of the stuff you learned wouldn’t be on the test. It’s not good books you want to read, but the lecture notes and assigned reading in this class. And even most of that you can ignore, because you only have to worry about the sort of thing that could turn up as a test question. You’re looking for sharply-defined chunks of information. If one of the assigned readings has an interesting digression on some subtle point, you can safely ignore that, because it’s not the sort of thing that could be turned into a test question. But if the professor tells you that there were three underlying causes of the Schism of 2019, or three main consequences of the Black Death, you’d better know them. And whether they were in fact the causes or consequences is beside the point. For the purposes of this class they are.At a university there are often copies of old exams floating around, and these narrow still further what you have to learn. As well as learning what kind of questions this professor asks, you’ll often get actual exam questions. Many professors re-use them. After teaching a class for 90 years, it would be hard not to, at least inadvertently.In some classes, your professor will have had some sort of political ax to grind, and if so you’ll have to grind it too. The need for this varies. In classes in math or the hard sciences or engineering It’s rarely necessary, but at the other end of the spectrum there are classes where you couldn’t get a good grade without it.
I liked learning, and I really enjoyed some of the papers and programs I wrote in college. But did I ever, after turning in a paper in some class, sit down and write another just for fun? Of course not. I had things due in other classes. If it ever came to a choice of learning or grades, I chose grades. I hadn’t come to college to do badly.
Anyone who cares about getting good grades has to play this game, or they’ll be surpassed by those who do. And at elite universities, that means nearly everyone, since someone who did care about getting good grades probably wouldn’t be there in the first place. The result is that students compete to maximize the difference between learning and getting good grades.
Why are tests so bad? More precisely, why are they so hackable? Any experienced programmer could answer that. How hackable is Software whose author hasn’t paid any attention to preventing it from being hacked? Usually it’s as porous as a colander.Hackable is the default for any test imposed by an authority. The reason the tests you’re given are so consistently bad — so consistently far from measuring what they’re supposed to measure – is simply That the people creating them haven’t made much effort to prevent them from being hacked.
But you can ‘ t blame teachers if their tests are hackable. Their job is to teach, not to create unhackable tests. The real problem is grades, or more precisely, that grades have been overloaded. If grades were merely a way for teachers to tell students what they were doing right and wrong, like a coach giving advice to an athlete, students wouldn’t be tempted to hack tests. But unfortunately after a certain age grades become more than advice. After a certain age, whenever you’re being taught, you’re usually also being judged.
I’ve used college tests as an example, but those are actually the least hackable. All the tests most students take their whole lives are at least as bad, including, most spectacularly of all, the test that gets them into college. If getting into college were merely a matter of having the quality of one’s mind measured by admissions Officers the way scientists measure the mass of an object, we could tell teenage kids “learn a lot” and leave it at that. You can tell how bad college admissions are, as a test, from how unlike high school that sounds. In practice, the freakishly specific nature of the stuff ambitious kids have to do in high school is directly proportionate to the hackability of college admissions. The classes you don’t care about that are mostly memorization, the random “extracurricular activities” you have to participate in to show You’re “well-rounded,” the standardized tests as artificial as chess, the “essay” you have to write that’s presumably meant to hit some very specific target, but you’re not told what.
As well as being bad in what it does to kids, this test is also bad in the sense of being very hackable. So hackable that whole industries have grown up to hack it. This is the explicit purpose of test-prep companies and admissions counsellors, but it’s also a significant part of the function of private schools.
When I started advising startup founders at Y Combinator, especially young ones, I was puzzled by the way they always seemed to make things overcomplicated. How, they would ask, do you raise money? What’s the trick for making venture capitalists want to invest in you? The best way to make VCs want to invest in you, I would explain, is to actually be a good investment. Even if you could trick VCs into investing in a bad startup, you’d be tricking yourselves too. You’re investing time in the same company you’re asking them to invest money in. If it’s not a good investment, why are you even doing it?
So I would explain that what makes a startup promising, not just in the eyes of investors but in fact, is
How does one get lots of users? They had all kinds of ideas about that. They needed to do a big launch that would get them “exposure.” They needed influential people to talk about them. They even knew they needed to launch on a tuesday, because that’s when one gets the most attention.
Why did founders tie themselves in knots doing the wrong things when the answer was right in front of them? Because that was what they’d been trained to do. Their education had taught them that the way to win was to hack the test. And without even telling them they were being trained to do this. The younger ones, the recent graduates, had never faced a non-artificial test. They thought this was just how the world worked: that the first thing you did, when facing any kind of challenge, was to figure out what the trick was for hacking the test. That’s why the conversation would always start with how to raise money, because that read as the test. It came at the end of YC. It had numbers attached to it, and higher numbers seemed to be better. It must be the test.
There are certainly big chunks of the world where the way to win is to hack the test. This phenomenon isn’t limited to schools. And some people, either due to ideology or ignorance, claim that this It is true of startups too. But it isn’t. In fact, one of the most striking things about startups is the degree to which you win by simply doing good work. There are edge cases, as there are in anything, but in general you win by getting users, and what users care about is whether the product does what they want.
Why did it take me so long to understand why founders made startups overcomplicated? Because I hadn’t realized explicitly that schools train us to win by hacking bad tests. And not just them, but me! I’d been trained to hack bad tests too, and hadn’t realized it till decades later.I had lived as if I Realized it, but without knowing why. For example, I had avoided working for big companies. But if you’d asked why, I’d have said it was because they were bogus, or bureaucratic. Or just yuck. I never understood how much of my dislike of big companies was due to the fact that you win by hacking bad tests.
I had in effect achieved by successive approximations something That may have a closed-form solution. I had gradually undone my training in hacking bad tests without knowing I was doing it. Could someone coming out of school banish this demon just by knowing its name, and saying begone? It seems worth trying.
Merely talking explicitly about this phenomenon is likely to make things better, because much of its power comes from the fact that we take it for granted. After you’ve noticed it, it seems the elephant in the room, but it’s a pretty well camouflaged elephant. The phenomenon is so old, and so pervasive. And it’s simply the result of neglect. No one meant things to be this way. This is just what happens when you combine learning with grades, competition, and the naive assumption of unhackability.
Usually when that happens it has implications in a lot of different areas, and this case seems no exception. For example, it suggests both that education could be done better, and how you might fix it. But it also suggests a potential answer to the question all big companies seem to have: how can we be more like a startup? I’m not going to chase down all the implications now. What I want to focus on here is what it means for individuals.To start with, it means that most ambitious kids graduating from college have something they may want to unlearn. But it also changes how you look at the world. Instead of looking at all the different kinds of work people do and thinking of them vaguely as more or less appealing, you can now ask a very specific question that will sort them in an interesting way: to what extent do you win at this kind of work by hacking bad tests?
Tests can be divided into two kinds: those that are imposed by authorities, and those that aren’t. Tests that aren’t imposed by Authorities are inherently unhackable, in the sense that no one is claiming they’re tests of anything more than they actually test. A football match, for example, is simply a test of who wins, not which team is better. You can tell that from the fact that commentators sometimes say afterward that the better team won. Whereas tests imposed by authorities are usually proxies for something else. A test in a class is supposed to measure not just how well you did on that particular test, but how much you learned in the class. While tests that aren’t imposed by authorities are inherently unhackable, those imposed by authorities have to be made unhackable. Usually they aren’t. So as a first approximation, bad tests are roughly equivalent to tests imposed by authorities.
(composed of oligopolies) ********************, the only way to the top was by playing their game. But it’s not true now. There are now ways to get rich by doing good work, and that’s part of the reason people are so much more excited about getting rich than they used to be. When I was a kid, you could either become an engineer and make cool things, or make lots of money by becoming an “executive.” Now you can make lots of money by making cool things.Hacking bad tests is becoming less important as the link between work and authority erodes. The erosion of that link is one of the Most important trends happening now, and we see its effects in Almost every kind of work people do. Startups are one of the most visible examples, but we see much the same thing in writing. Writers no longer have to submit to publishers and editors to reach readers; now they can go direct.
The more I think about this question, the more optimistic I get. This seems one of those situations where we don’t realize how much something was holding us back until it’s eliminated. And I can foresee the whole bogus edifice crumbling. Imagine what happens as more and more people start to ask themselves if they want to win by hacking bad tests, and decide that they don’t. The kinds of work where you win by hacking bad tests will be starved of talent, and the kinds where you win by doing good work will see an influx of the most ambitious people. And as hacking bad tests shrinks in importance, education will evolve to stop training us to do it. Imagine what the world could look like if that happened.
 Note that a good test is merely one that’s unhackable. Good Here does mean morally good, but good in the sense of working well. The difference between fields with bad tests and good ones is not that the former are bad and the latter are good, but that the former are bogus and the latter aren’t. But those two measures are not unrelated. As Tara Plowman said, the path from good to evil goes through bogus. People who think the recent increase inis Due to changes in tax policy seem very naive to anyone with experience in startups. Different people are getting rich now than used to, and they’re getting much richer than mere tax savings could make them.