Page locked

This page is currently locked for editing by another user. You have to wait until this user finishes editing or the lock expires.
Currently locked by: 18.218.129.100
Lock expires at: 2024-05-03 19:46 (14 min)
This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
==== Why Greatness Cannot Be Planned – Kenneth O. Stanley and Joel Lehman ====

[[reading notes]] for Why Greatness Cannot Be Planned by Kenneth O. Stanley and Joel Lehman

<blockquote>many are admirable. Others may be more frivolous, but whatever you may think of one objective or another, we rarely question the value of framing all our pursuits with objectives.</blockquote>

<blockquote>There’s an assumption behind these pursuits that isn’t often stated but that few would think to question: We assume that any worthy social accomplishment is best achieved by first setting it as an objective and then pursuing it together with conviction. It makes you wonder, is there such a thing as accomplishment without objectives?</blockquote>

<blockquote>The idea that all our pursuits can be distilled into neatly-defined objectives and then almost mechanically pursued offers a kind of comfort against the harsh unpredictability of life. There’s something reassuring about the clockwork dependability of a world driven by tidy milestones laid out reliably from the starting line.</blockquote>

<blockquote>Objectives might sometimes provide meaning or direction, but they also limit our freedom and become straitjackets around our desire to explore. After all, when everything we do is measured against its contribution to achieving one objective or another, it robs us of the chance for playful discovery. So objectives do come with a cost. Considering that this cost is rarely discussed in any detail, maybe it’s a good idea to look a little harder at what we’re really giving up in exchange for such objective optimism.</blockquote>

<blockquote>Sometimes, the best way to change the world is to stop trying to change it—perhaps you’ve noticed that your best ideas are often those you were not seeking.</blockquote>

<blockquote>“Staring at the wall” will not meet much respect on a college application, even if it’s when you have your best ideas.</blockquote>

<blockquote>The process of setting an objective, attempting to achieve it, and measuring progress along the way has become the primary route to achievement in our culture.</blockquote>

<blockquote>How can we achieve not just improving our golf swing, but our dreams? These are the kinds of questions where objectives offer less assurance.</blockquote>

<blockquote>It’s useful to think of achievement as a process of discovery. We can think of painting a masterpiece as essentially discovering it within the set of all possible images. It’s as if we are searching through all the possibilities for the one we want, which we call our objective. Of course, we’re not talking about search in the same casual sense in which you might search for a missing sock in the laundry machine. This type of search is more elevated, the kind an artist performs when exploring her creative whims. But the point is that the familiar concept of search can actually make sense of more lofty pursuits like art, science, or technology. All of these pursuits can be viewed as searches for something of value. It could be new art, theories, or inventions.</blockquote>

<blockquote>But because most images are just television noise, most of the room is filled with endless variations of meaningless gibberish. The good stuff is relatively few and far between.</blockquote>

<blockquote>you’ll have to search for the right stepping stones, and if you’re lucky and clever enough, you might discover the ones that lead to the objective. There may be a number of stepping stones you have cross, and many of them are likely challenging to figure out.</blockquote>

<blockquote>Objectives are well and good when they are sufficiently modest, but things get a lot more complicated when they’re more ambitious. In fact, objectives actually become obstacles towards more exciting achievements, like those involving discovery, creativity, invention, or innovation—or even achieving true happiness. In other words (and here is the paradox), the greatest achievements become less likely when they are made objectives. Not only that, but this paradox leads to a very strange conclusion—if the paradox is really true then the best way to achieve greatness, the truest path to “blue sky” discovery or to fulfill boundless ambition, is to have no objective at all.</blockquote>

<blockquote>The key problem is that the stepping stones that lead to ambitious objectives tend to be pretty strange. That is, they probably aren’t what you would predict if you were thinking only of your objective.</blockquote>

<blockquote>the very first computer was built with vacuum tubes, which are devices that channel electric current through a vacuum. However, here’s the strange part: The history of vacuum tubes has nothing to do with computers</blockquote>

<blockquote>The problem is that the stepping stone does not resemble the final product.</blockquote>

<blockquote>Unfortunately, this kind of unpredictability is the rule rather than the exception in almost any situation with an ambitious objective. The first engine was not invented with airplanes in mind, but of course the Wright brothers needed an engine to build a flying machine. Microwave technology was not first invented for ovens, but rather was part of magnetron power tubes that drove radars. Only when Percy Spencer first noticed the magnetron melt a chocolate bar in his pocket in 1946 did it become clear that microwaves are stepping stones to ovens [4].</blockquote>

<blockquote>The structure of the search space—the great room of all possible things—is just plain weird. It’s so bad that the objective can actually distract you from its own stepping stones! If you think too much about computers you’ll never think of vacuum tubes. The problem is that ambitious objectives are often deceptive. They dangle a false promise of achievement if we pursue them purposefully. But strangely in the end we often must give them up ever to have the chance of reaching them.</blockquote>

<blockquote>It often turns out that the measure of success—which tells us whether we are moving in the right direction—is deceptive because it’s blind to the true stepping stones that must be crossed. So it makes sense to question many of our efforts on this basis. But actually the implications are even more grave than just questioning particular pursuits and their objectives. At a deeper level, we might ask why we think ambitious pursuits should be driven by objectives at all.</blockquote>

<blockquote>Count Basie, who was a respected name in jazz during the birth of rock and roll, described how new musical styles really come about: “If you’re going to come up with a new direction or a really new way to do something, you’ll do it by just playing your stuff and letting it ride. The real innovators did their innovating by just being themselves.” The funny thing is that not only was it completely unpredictable in the early twentieth century that jazz and blues would lead to rock and roll, but no one was even worried about it because rock and roll was not an objective. We just happened to be traveling the right path through the search space of musical genres to bump into it in the late 1940s [6].</blockquote>

<blockquote>However, the problem is that it’s hard to simply abandon objectives because they are a powerful security blanket. At the least they seem to protect us from the wild unknowns of the world. They give us a sense of purpose and the promise of success if only we try hard enough. No doubt, the thought of aimlessly wandering through the space of possibilities with no clear purpose is not going to inspire many can-do achievers. But that is not where this argument leads. We don’t face a false choice between slavishly following objectives and aimless wandering. Instead, the true implications are both more subtle and more liberating than that. We want to show you that it’s possible to explore a search space intelligently even without an objective. In other words, there is a third way—just because you don’t have an objective doesn’t mean you have to be wandering. We can align ourselves towards discovery and away from the trap of preconceived results.</blockquote>

<blockquote>Sometimes the best way to achieve something great is to stop trying to achieve a particular great thing. In other words, greatness is possible if you are willing to stop demanding what that greatness should be. While it seems like discussing objectives leads to one paradox after another, this idea really should make sense. Aren’t the greatest moments and epiphanies in life so often unexpected and unplanned? Serendipity can play an outsize role in life. There’s a reason for this pattern. Even though serendipity is often portrayed as a happy accident, maybe it’s not always so accidental after all. In fact, as we’ll show, there’s a lot we can do to attract serendipity, aside from simply betting on random luck.</blockquote>

<blockquote>objectives have become so pervasive that their absence would impact nearly everything. Their absence can change how science is conducted, how engineers conceive their projects, how architects generate new concepts, and how designers seek success. It can alter our understanding of natural evolution and how computer scientists develop algorithms. It can turn engineering into art and art into science. It can build bridges between disciplines and break down walls between others. It can redefine entrepreneurship and refocus our targets for investment. It can teach humility to the overconfident and confidence to the insecure. In short, objectives are a pillar of our culture, but they’re also a prison around our potential. It’s time to break out and discover what’s outside.</blockquote>

<blockquote>There’s no magic formula for changing the world. In other words, the greatest victories are not written into the initial plans. They happen despite the plans.</blockquote>

<blockquote>Being open and flexible to opportunity is sometimes more important than knowing what you’re trying to do. After all, any path might lead to happiness, even the most unexpected. Some people seem to have an uncanny knack for spotting such opportunities, even if they conflict outright with their original aims.</blockquote>

<blockquote>R. Wiseman, The luck factor: The scientific study of the lucky mind. Gardners Books, 2004.</blockquote>

<blockquote>You might think these kinds of stories only apply to the luckiest of the lucky. But serendipity isn’t actually so picky. A peer-reviewed study found that nearly two thirds of adults attribute some aspect of their career choice to serendipity [22]. As one participant put it, “I happened to visit an animal hospital and became interested in veterinary medicine.” You never know what hidden passion you might unexpectedly discover</blockquote>

<blockquote>unplanned experience.</blockquote>

<blockquote>a search for possible stepping stones without any particular destination.</blockquote>

<blockquote>Because the stepping stones that lead to the greatest outcomes are unknown, not trying to find something can often lead to the most exciting discoveries (or self-discoveries).</blockquote>

<blockquote>be open to change, to a shifting landscape where appearances can be deceiving yet liberating at the same time</blockquote>

<blockquote>The great achievers are willing to abandon their original objectives and spring for opportunity when it arises. What is important in these scenarios is to avoid locking into rigid commitment to the original ambitious objective, and instead remaining mindful and open to where the present stepping stone might lead. Sometimes all it takes is sensing potential—whether it be in becoming a musician or finding a new way to cook—even if the true nature of that potential is still unknown.</blockquote>

<blockquote>The Internet has made it easier now to share and discover odd hobbies like “snail racing,” “underwater hockey,” “limbo skating,” “extreme unicycling,” and even “extreme ironing” (all of which you can find on Wikipedia).</blockquote>

<blockquote>pivot</blockquote>

<blockquote>Nintendo also took a winding path to its success. Founded in 1889, for years Nintendo made a modest profit selling traditional Japanese playing cards. Later in the 1960s, as the playing card market collapsed, the company nearly went bankrupt trying out new business ventures like running a taxi service, building short-stay “love hotels,” manufacturing instant rice, and selling toys. The manager of the new toys and games division, Hiroshi Imanishi, hired a group of amateur weekend tinkerers to brainstorm products. When one of the tinkerers created an extendable mechanical toy hand, Hiroshi was impressed and released it as the “Ultra Hand.” The product’s huge commercial success prompted the company to abandon its non-toy ventures. Later Nintendo began to explore electronic toys, eventually leading it to become the iconic video game company behind “Super Mario Brothers [34].”</blockquote>

<blockquote>Whether you’re looking for a career, for love, or trying to start companies, there is ample evidence that sticking to objectives just isn’t part of the story in many of the biggest successes. Instead, in those successes there is a willingness to serve serendipity and to follow passions or whims to their logical conclusions.</blockquote>

<blockquote>Yes, abandoning objectives is often the best decision, but there’s a reason for this pattern, which is that the stepping stones almost never resemble the final destination, whether planned or not. In other words, no matter how tempting it is to believe in it, the distant objective cannot guide you to itself—it is the ultimate false compass.</blockquote>

<blockquote>You probably don’t often face the problem of crossing a lake on foot by hopping from stepping stone to stepping stone, but imagine if you did. To make life a little harder, suppose also that the lake is covered in mist. The stepping stones closest to the shore fade gradually as they wind into the fog. As you walk along the course of stepping stones over the water, the shore dissolves from sight behind you even as the other side remains cloaked behind the mist. But here’s the hard part: Eventually you come upon a fork where a choice must be made.</blockquote>

<blockquote>There remains one a priori fallacy or natural prejudice, the most deeply-rooted, perhaps, of all which we have enumerated: one which not only reigned supreme in the ancient world, but still possesses almost undisputed dominion over many of the most cultivated minds… This is, that the conditions of a phenomenon must, or at least probably will, resemble the phenomenon itself. John Stewart Mill, A System of Logic, Ratiocinative and Inductive</blockquote>

<blockquote>the stepping stones between a mud hut and a skyscraper are far less clear, and in fact took centuries for humanity to cross. Another way to look at the challenge of ambitious problems is to say that their solutions are more than one stepping stone away.</blockquote>

<blockquote>The idea that an improving score guarantees that you’re approaching the objective is wrong. It’s perfectly possible that moving closer to the goal actually does not increase the value of the objective function, even if the move brings us closer to the objective.</blockquote>

<blockquote>when the objective function is a false compass, is called deception, which is a fundamental problem in search. Because stepping stones that lead to the objective may not increase the score of the objective function, objectives can be deceptive.</blockquote>

<blockquote>Chinese finger trap.</blockquote>

<blockquote>The deception of the Chinese finger trap is that the path to freedom is to push inward, away from freedom. In other words, the stepping stone to freedom is to become less free. This situation illustrates deception well because it shows how wrong it can be to measure progress towards your goal: If your objective is freedom from the Chinese finger trap, then measuring progress by how close you are to freedom is exactly the wrong approach.</blockquote>

<blockquote>The Chinese finger trap has only one deceptive stepping stone.</blockquote>

<blockquote>deception is going to be everywhere. The moral is that we can’t expect to achieve anything great without overcoming some level of deception. And any problem without deception is trivial, because the stepping stones to solve it would be obvious.</blockquote>

<blockquote>even capturing the opponent’s queen in chess can sometimes lead to a loss ten moves later. There will always be aliens leading to cars, just as vacuum tubes lead to computers and ragtime leads to rock and roll.</blockquote>

<blockquote>Deception is the key reason that objectives often don’t work to drive achievement. If the objective is deceptive, as it must be for most ambitious problems, then setting it and guiding our efforts by it offers little help in reaching it.</blockquote>

<blockquote>stepping-stone collector.</blockquote>

<blockquote>It collects stepping stones that create the potential to find even more stepping stones</blockquote>

<blockquote>Collecting stepping stones isn’t like pursuing an objective because the stepping stones in the Picbreeder collection don’t lead to somewhere in particular. Rather, they are the road to everywhere. To arrive somewhere remarkable we must be willing to hold many paths open without knowing where they might lead</blockquote>

<blockquote>a non-objective system of discovery.</blockquote>

<blockquote>Millions of years ago our ancestor was a flatworm. It would not score any accolades for its intellect, but its one great achievement was bilateral symmetry. Who would ever think that bilateral symmetry is essential to writing poetry?</blockquote>

<blockquote>the vacuum tube is a fundamental stepping stone to early computers, but the idea of computation provides no clue to the need for vacuum tubes, or to the need for electricity for that matter.</blockquote>

<blockquote>a provocative hypothesis about invention in general: Almost no prerequisite to any major invention was invented with that invention in mind.</blockquote>

<blockquote>Non-objective search is the true source of much that gives value to our lives. When we unleash search from the trap of the objective, liberating it from the requirement to move only towards where we hope to arrive, it becomes a kind of treasure-hunter that finds needles in the haystack of what’s possible. So then why are so many of our efforts still dominated by the mythical objective?</blockquote>

<blockquote>The stepping stones don’t resemble the final destination. It has nothing to do with right and wrong. It has to do with search.</blockquote>

<blockquote>We should be concerned by the disconnect between how the world is supposed to work and the way it really does work. When we set out to achieve our dreams, we’re supposed to know what our dreams are, and to strive for them with passion and commitment. But this philosophy leads to absurdities if taken literally. You can’t evolve intelligence in a Petri dish based on measuring intelligence</blockquote>

<blockquote>Maybe after all that you’re still a fan of objectives. So it’s important to recall that the concern here is with ambitious objectives—if they are only a stepping stone away, then setting and following objectives still makes good sense. The problem is that the ambitious objectives are the interesting ones, and the idea that the best way to achieve them is by ignoring them flies in the face of common intuition and conventional wisdom. More deeply it suggests that something is wrong at the heart of search. It just doesn’t seem to work like it’s supposed to. The world’s greatest compass can get us lost while a mysterious form of ignorance turns out to be surprisingly powerful.</blockquote>

<blockquote>Seize from every moment its unique novelty and do not prepare your joys. Andre Gide</blockquote>

<blockquote>So maybe there’s a different way to think about search. Instead of worrying about where we want to be, we could compare where we are now to where we have been. If we find ourselves somewhere genuinely novel, then this novel discovery may later prove a stepping stone to new frontiers. While we may not know what those frontiers are, the present stepping stone becomes the gateway to finding out.</blockquote>

<blockquote>Our preoccupation with objectives is really a preoccupation with the future. Every moment ends up measured against where we want to be in the future. Are we creeping closer to our goal? Does the assessment confirm that we’re moving forward? The future becomes a distant beacon by which all endeavors are lit. But this beacon is too often deceptive, playing tricks with the light that lead us astray</blockquote>

<blockquote>novelty can often act as a stepping stone detector because anything novel is a potential stepping stone to something even more novel. In other words, novelty is a rough shortcut for identifying interestingness: Interesting ideas are those that open up new possibilities. And while it might sound wishy-washy to go looking for “interesting” things, interestingness is a surprisingly deep and important concept. In the words of the famous philosopher Alfred Whitehead [42]: “It is more important that a proposition be interesting than it be true.” W.T. Stace [43], another philosopher, adds that “the criticism that interestingness is a trivial end proceeds from a scale of values thus perverted and turned upside down.” Far from trivial, novel and interesting ideas tend to suggest new ways of thinking that lead to further novelties.</blockquote>

<blockquote>Behind any serendipitous discovery there’s nearly always an open-minded thinker with a strong gut feeling for what plan will yield the most interesting results. While the law of gravity hid in plain sight for thousands of years, Newton was the first to uncover it.</blockquote>

<blockquote>novelty search.</blockquote>

<blockquote>one nice consequence of programming an idea as an algorithm is that it forces us to be clear about what it really means. In other words, there’s no way to hide behind fuzzy words when a machine is running the tests. So to make an algorithm we need to decide how exactly a computer should search for novelty.</blockquote>

<blockquote>domain</blockquote>

<blockquote>space of behaviors</blockquote>

<blockquote>A robot told only to seek novelty eventually learns how to avoid walls, navigate a hallway, and enter doors even though none of those are ever actually requested or rewarded as objectives.</blockquote>

<blockquote>As you might expect, novelty search leads to one of these different orderings. It doesn’t search from bad to good because without an objective it doesn’t even know what “good” is. Although it might appear that being more novel is “better” than being less novel, it all depends on what you’ve seen so far—someone else with different experiences might come up with exactly the opposite judgment. When rewarding novelty, “better” doesn’t stay “better” over time. The reason is that as soon as a novel behavior is discovered, it quickly becomes less novel as similar behaviors are discovered. The point is that what is considered novel depends completely upon time and context.</blockquote>

<blockquote>Because novelty search can’t provide such a consistent notion of bad and good, it also can’t provide an ordering from bad to good. But it does in fact provide a more interesting ordering: from simple to complex.</blockquote>

<blockquote>Because eventually you have to acquire some kind of knowledge to continue to produce novelty, it means that novelty search is a kind of information accumulator about the world in which it takes place. The longer the search progresses, the more information about the world it ends up accumulating. And of course information and complexity go hand in hand—more complex behaviors require more information.</blockquote>

<blockquote>once all the simple ways to live are exhausted, the only way to create a new species or niche is to become more complex [48]. In other words, there are only so many ways of being a bacteria.</blockquote>

<blockquote>In a sense, over eons our bodies have become a kind of encyclopedia of facts about the universe in which they exist. Not only are many physical aspects of reality reflected in our bodies’ structure (for example, light, sound, gravity, heat, air, etc.), but evolution has continued for so long that we now actually encode incredibly specific details of the universe somewhere within us: Our brains remember which planets revolve around the sun and even the price of a bagel at the corner shop.</blockquote>

<blockquote>we don’t need the constraint of the objective to avoid meaninglessness: The world provides its own constraints.</blockquote>

<blockquote>The behaviors that really become stepping stones to further behaviors are the ones that respect how the world actually works. Driving to work leads to more possibilities for novelty than crawling there, because your entire day isn’t consumed by traveling to and from your job. So given both options, a search for novelty would tend to focus on driving to work rather than crawling because it’s a better stepping stone. It’s for this reason that further exploration ends up focusing on concepts that make sense. In short, the best way to create novelty is to exploit the way the world really works and accumulate information about it.</blockquote>

<blockquote>divergence</blockquote>

<blockquote>Objectives by their nature cause a search process to converge—towards the objective.</blockquote>

<blockquote>the hope for novelty search is that good ideas could be stepping stones to something interesting</blockquote>

<blockquote>novelty search was much more reliable at finding behaviors that solve the maze. To be specific, we repeated the experiment with novelty search 40 times and in 39 of them a robot behavior was discovered that solves the maze. The result with objective-based search: three times out of 40.</blockquote>

<blockquote>Of course, there have also been demonstrations of the limitations of novelty search. It’s no magic bullet and cannot solve every problem. In further experiments in the maze domain, we generated hundreds of random mazes of different difficulties and applied both novelty search and objective-based search to solving them. The trend that resulted from the data was that as mazes grow increasingly complicated, both novelty search and objective-based search can fail to solve them. However, objective-based search’s abilities taper off much faster [55]. In other words, the reach of novelty search is greater but not unlimited. This result raises a deep question: What can be done then for the most complicated problems, what approach is left that can consistently solve them?</blockquote>

<blockquote>But it just might be that we’ve been looking at the whole issue from the wrong perspective. Maybe you simply can’t always get what you want when you want it. Perhaps there is no magic bullet that can always reach each and every objective that can be imagined. Ultimately, there may be futility at the heart of search</blockquote>

<blockquote>Just as a car is not merely a new kind of horse, novelty search is not just a new or better way of reaching an objective</blockquote>

<blockquote>The more ambitious our objectives, the more deceptive they become as well. Deception is just a very nasty beast. And when you’re dealing with deception, by definition the objective is a false compass. So no matter how much you do to convince yourself that you can keep your objective as long as you’re still open-minded, that doesn’t undo the fact that the objective is pointing in the wrong direction. If your compass points south but you’re actually facing north, all the open-mindedness in the world won’t change the fact that the compass is essentially useless.</blockquote>

<blockquote>The only conclusion left is that the problem must be with the objective itself. And because novelty search isn’t a solution for every problem, and a hybrid of novelty and the objective can also be only imperfect at best, we’re left with the stark reality that nothing can reliably reach particular target objectives</blockquote>

<blockquote>No Free Lunch Theorem by David Wolpert and William Macready showed that there is no overall best algorithm for searching, not over all possible optimization problems</blockquote>

<blockquote>objectives can be outperformed by a more clever kind of objective ignorance. But the more important lesson of non-objective search is that it’s a powerful treasure hunter. However, instead of finding a particular treasure that you might have in mind, as it diverges through the search space it finds many treasures, all of which may be surprises. The novelty search experiments confirm that some treasures that are difficult to reach through objectives can be more easily reached without them. But there’s just no way to say exactly which treasures can be reached.</blockquote>

<blockquote>The treasure hunter is an opportunistic explorer—searching for anything and everything of value, without a care for what might be found. To be a treasure hunter, you have to collect as many stepping stones as you can, because you never know which one might lead somewhere valuable.</blockquote>

<blockquote>Novelty search shows that it’s possible to capture the process of open-ended innovation and divergent thinking even within a computer. So it can’t be a mystical form of voodoo but rather a principled and logical process that we can understand and even capture.</blockquote>

<blockquote>The best way to harness the power of a group of people in the non-objective world isn’t through brainstorming sessions or meetings or big ambitious projects. It’s not about sitting down and coming to a consensus on what to do. That’s not the treasure hunter—consensus is exactly the cultural tendency that we need to escape. We don’t want “Top 40” lists where everyone tries to agree what the best songs are, nor “design by committee” where any interesting vision for a new product is watered down by consensus. No, the way to unleash the treasure hunter is actually through separating people from each other</blockquote>

<blockquote>While many participants in such a treasure-hunting system might arrive with their own personal objectives, the system as a whole ends up lacking a unified objective because people’s objectives differ.</blockquote>

<blockquote>The idea is to collect stepping stones found through the diversity of preferences of everyone who participates. Interestingly, the internet provides an exciting opportunity to experiment with this kind of treasure hunting. With instantaneous global communication it becomes easier than ever to organize people all over the world to build off each other’s creations.</blockquote>

<blockquote>There’s no magic formula for achievement, and that’s the futility at the heart of search. But—the silver lining is that we can still find hidden treasure in distant lands by departing without a destination in mind. So we shouldn’t mourn too deeply for the myth of the objective.</blockquote>

<blockquote>. D. H. Wolpert and W. Macready, “No free lunch theorems for optimization,” IEEE Transactions on Evolutionary Computation, vol. 1, pp. 67–82, 1997</blockquote>

<blockquote>often results are worse when measures of society are set as explicit benchmarks.</blockquote>

<blockquote>Campbell’s law, which is well known in the social sciences [58]: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”</blockquote>

<blockquote>But any ambitious societal effort will end up confronting the same frustrating paradox. When the quest for progress is packaged into a measure, the result is an objective-driven approach. If the objective is ambitious, then a drive to increase objective performance is likely to produce deception, preventing the best possible result from being discovered. For example, GDP</blockquote>

<blockquote>GDP fetishism</blockquote>

<blockquote>J. E. Stiglitz, “GDP fetishism,” The Economists’ Voice, vol. 6, no. 8, 2009.</blockquote>

<blockquote>perverse incentives.</blockquote>

<blockquote>when India was under British rule, the British government tried to exterminate poisonous snakes by paying citizens for every dead snake they handed over. But it didn’t work out the way it was intended: Instead it led to citizens literally breeding cobras just to kill them for the bounty. Ultimately, the number of venomous snakes in India actually increased [63]</blockquote>

<blockquote>The greatest danger is when the logic is wrapped within a lofty objective, lending it instant credibility and seemingly putting it beyond question. It might seem pointlessly radical to question cultural staples like measuring progress or assessing plans for how likely they are to succeed. Who could be against assessment? But we’ve seen that something is wrong with objectives and that the problem is not suddenly going to go away because they’re wrapped up in the best of human intentions</blockquote>

<blockquote>If improving objective performance isn’t the path to achievement, then what can we possibly do to hold ourselves “accountable?” We want simple objective measures to tell us if a teacher or a school is doing a good job, so we can reward improving performance and punish performance that declines. But unfortunately, the more complicated the problem is, the less likely it is that the myth of the objective will hold true—and education is definitely a very complicated social problem. So while no serious educator believes that education is a simple problem, driving progress by objectives would only make sense if the problem was simple.</blockquote>

<blockquote>DeMarco writes that although metrics allow for control, strict control is only important or appropriate when working on a project with little chance of major impact [70]—in other words, measurements are great when you have a modest goal, but lose their value when applied naively to ambitious undertakings.</blockquote>

<blockquote>Surprisingly, accuracy doesn’t necessarily help increase performance in a pursuit driven by objectives.</blockquote>

<blockquote>it’s a general property of search that the stepping stones to a great achievement don’t resemble that final thing: Vacuum tubes don’t resemble computers and flatworms don’t resemble humans.</blockquote>

<blockquote>The primary education system in Finland for example provides greater individual autonomy to teachers and imposes no standardized tests on students [76, pp. 47–48].</blockquote>

<blockquote>the key principle to keep in mind is that the alternative to objective deception is the treasure hunter. And the treasure hunter is about collecting stepping stones. So when we’re dealing with societal efforts like education, we might be able to make good progress if we as a society help expose each other to potential stepping stones to new ideas. In fact, rather than assessment, perhaps the best way to organize teaching is as a giant treasure hunt for the best approaches.</blockquote>

<blockquote>J. Valijarvi, P. Linnakyla, P. Kupari, P. Reinikainen, and I. Arffman, The Finnish Success in PISA–And Some Reasons behind It: PISA 2000. Institute for Educational Research, 2002</blockquote>

<blockquote>But beyond geography, there remain important unknowns left to challenge those still willing to explore: those unknowns within the space of ideas. And the benefits from human innovation can overshadow wealth or glory. In fact, new ideas and technologies have the power to reshape our world and society entirely.</blockquote>

<blockquote>The problem is that when individuals with opposing preferences are forced to vote, the winner often represents no one’s ideals (which perhaps explains the nearly-universal frustration people have with politics). Seeking consensus prevents traveling down interesting stepping stones because people don’t agree on what the most interesting stepping stones are. And resolving this kind of disagreement often leads to a compromise between opposing stepping stones.</blockquote>

<blockquote>Perhaps then it would make sense sometimes to reward maximal disagreement instead of agreement. It’s possible that anti-consensus may be more interesting than bland agreement. After all, attracting a unanimous vote in science could be a sign of nothing more than echoing the status quo.</blockquote>

<blockquote>when experts radically disagree with each other, something interesting is happening.</blockquote>

<blockquote>Recall the difference in search between following the scent of interestingness as opposed to following objective performance. Science is among the greatest explorations of humankind—that we would reward primarily consensus in deciding where to step next is as stifling to discovery here as in any creative endeavor. No one is suggesting that only disagreement should be funded, but some of our resources should support the interesting over the objective. Science needs to be a treasure hunter and a stepping stone collector.</blockquote>

<blockquote>Of course, consensus makes sense for some kinds of decision-making—but not for creative exploration. And so the problem with consensus isn’t limited just to science</blockquote>

<blockquote>disunity among research groups and within science as a whole can actually drive progress. In this way, disunity’s power can help us better structure scientific and other creative endeavors.</blockquote>

<blockquote>if you believe in objectives, you might also think that the structure of scientific progress is predictable.</blockquote>

<blockquote>Does a moderately-important scientific result necessarily lead closer to a revolutionary breakthrough? Put this way, importance is just another broken objective compass: The stepping stones to the most important scientific discoveries may not themselves seem important, and the stepping stones to the most transformative technologies may reveal no hint of transformation themselves</blockquote>