Luck in Science

Genius is one percent inspiration, ninety-nine percent perspiration.
— Thomas Edison

Chance plays a leading role in science, but to gain the glittering prizes it is not sufficient to be in the right place at the right time.
— Frank Close “Neutrino“, 2010

Science has a much more decorative word for “luck” — “serendipity“.

So, considering my posts of Games & Luck, how much luck do you think is involved in science?  Do you have any famous quotes I can add?


Filed under Philosophy & Religion, Science

12 responses to “Luck in Science

  1. “Chance favors the prepared mind” – Pasteur

    I think there’s a lot of luck in science; but how much varies a great deal depending on various factors.

    I considered switching from artificial intelligence to molecular biology when I was a grad student. One of the reasons I decided not to was that there is too much luck involved. Some of the Grand Old Men of biology, I decided, were pompous fools who had happened to be in the right place at the right time when they were 25.

    Conversely, you can be brilliant and hard-working and never get anywhere. For instance, my sister spent several years trying to get the jelly out of fishes’ ears. She had a killer experiment lined up that would have opened a new area of neurophysiology, except that she couldn’t get the chemical to go into their ears because the goop blocked it. Eventually she gave up. Luckily she had other pots on the stove and got good (if less earth-shaking) results in ferrets. I decided I didn’t want to go that route, however.

    For me, part of the test of whether a scientist is really smart or just lucky is whether s/he has succeeded in several different unrelated areas. My prototype while I was deciding not to become a biologist was Sydney Brenner. (I see from his WP page ( that he’s gone on to get a Nobel Prize (richly deserved, which not all are IMNSHO) and has written an autobiography—I should probably check it out.)

    In most areas of science, at any given time there are a set of famous open problems, and most of the most competent people spend most of their time racing to solve those. Because there is not all *that* much difference in problem-solving ability among the best scientists, who gets the result first is largely a matter of luck.

    This is a huge waste of resources. There’s massive duplication of effort during these races. They also are largely winner-take-all in terms of academic credit assignment.

    I think this dynamic is due to the standard alpha-male competition pattern. Highly competent scientists tend to have a mixture of alpha-male and geek characteristics. They’ve got enough of the testosterone thing going that they want to *win*. This is, in my opinion, maladaptive for both the individual scientist and for society.

    My approach (in science and then later in business) was always to find problems that would be very valuable to have a solution to, but which the alpha males were not fighting over. The world is totally full of these—there are way more worthwhile problems than alpha males. The great thing about such problems is that the alpha males aren’t interested in them, because they aren’t recognized zones of competition. In other words, they aren’t interesting to the individual alpha because they aren’t interesting to the other alphas, who collectively define which problems constitute races.

    So, these are often easy to solve, because no one competent has bothered to look at them.

    “Easy to solve” boils down to “little luck involved”—it’s nearly deterministic that you’ll succeed.

    I wish young scientists were taught more about how to choose problems (instead of the whole focus being on how to solve them). But the young scientists are selected and taught by the winning alpha males, so this doesn’t happen. As a result, we have enormously more people who know how to solve well-defined scientific problems than people who can turn vague uncertainties into clearly-defined problems, and extremely few people who are good at intuiting which vague uncertainties are worth trying to formulate as a crisp problems that demand solutions.

  2. Hey David !

    Fascinating stories — thank you for sharing the personal stories. I am sympathetic with your insights. I agree that “luck” is present in much more that we want to imagine — science included. But this is not a despairing observation, as I wrote in my post “Life is a Game“.

    Like you, I have seen retro-credit grabbing as pervasive in everyone.

  3. given how most of the things we use now were discovered by accident (microwave, cornflakes, post-in notes, super-glue etc) i would totally agree that there is a large amount of luck involved in science. but one must be open to the possibility to learn from mistakes.

  4. Fantastic comment, David. It reminds me of this classic talk by Richard Hamming, who worked with Claude Shannon, Fermi, Feynman, Teller, etc.

    He offers great advice on choosing your area of research, and begins in part like this:

    Let me start not logically, but psychologically. I find that the major objection is that people think great science is done by luck. It’s all a matter of luck. Well, consider Einstein. Note how many different things he did that were good. Was it all luck? Wasn’t it a little too repetitive? Consider Shannon. He didn’t do just information theory. Several years before, he did some other good things and some which are still locked up in the security of cryptography. He did many good things.

    You see again and again, that it is more than one thing from a good person. Once in a while a person does only one thing in his whole life, and we’ll talk about that later, but a lot of times there is repetition. I claim that luck will not cover everything. And I will cite Pasteur who said, “Luck favors the prepared mind.” And I think that says it the way I believe it. There is indeed an element of luck, and no, there isn’t. The prepared mind sooner or later finds something important and does it. So yes, it is luck. The particular thing you do is luck, but that you do something is not.

    For example, when I came to Bell Labs, I shared an office for a while with Shannon. At the same time he was doing information theory, I was doing coding theory. It is suspicious that the two of us did it at the same place and at the same time – it was in the atmosphere. And you can say, “Yes, it was luck.” On the other hand you can say, “But why of all the people in Bell Labs then were those the two who did it?” Yes, it is partly luck, and partly it is the prepared mind; but `partly’ is the other thing I’m going to talk about. So, although I’ll come back several more times to luck, I want to dispose of this matter of luck as being the sole criterion whether you do great work or not. I claim you have some, but not total, control over it. And I will quote, finally, Newton on the matter. Newton said, “If others would think as hard as I did, then they would get similar results.

  5. Joshua, thank you very much for that pointer! I once read a summary of Hamming’s talk somewhere, but not the whole thing. It’s really great. I’d like to expand every one of his paragraphs into a full paper.

    I happens that I wrote something similar at about the same time (How to Do Research at the MIT AI Laboratory). I thought advice about “how to do research” was highly necessary, and I was able to find very little. I wasn’t satisfied with the paper, but there was so little available on the subject that it went viral. I got hundreds of emails from young researchers in all sorts of different fields (math, chemistry, even Eng. Lit.) saying that it had helped them a lot.

    So I thought seriously about taking a year or two to write a full book on the subject; but didn’t. I retrospect, it would probably have been much more useful than what I did do in the two years after my PhD.

    Since then, other people have written much better things, so maybe the effort would have been duplicative anyway!

  6. crl

    David: What of female scientists?!

    There is luck (or, rather, chance. let’s be scientific here!) in all things, science included, but without skill, intelligence, creativity, and a lot hard work, one isn’t going to get anywhere. On top of luck, we have to worry about the politics of who gets credit (think Rosalind Franklin) versus who actually gets work done; humans are humans, and scientists certainly aren’t immune to standard office drama.

  7. @David –

    Oooh, good advice in there. Some of the sections could definitely be whole chapters. The tone and style remind me a lot of “Hitchhiker’s Guide to the Internet”, which came out around the same time; and I assume this was part of the same sort of FAQ culture that was so nice back then. I feel like we’ve gained a lot since then, but also lost a lot of the spirit of curation and helpfulness that was more common then.

    In addition to the good advice bits, and a bit of a walk down memory lane about life before Powerpoint and Gopher, I got a chuckle out of some of the tips:

    Knowing EE is also useful when it comes time to build a custom
    chip or debug the power supply on your Lisp machine.

    I have to ask, was this ironic even in 1988, or is it only ironic in hindsight?

    Don’t write destructive criticism like “garbage” on a paper. This contributes..
    nothing to the author. Take the time to provide constructive suggestions.

    This is only so funny because it is perennially necessary advice, and people still use the adjective “garbage” in critiques.

  8. @crl – I love your blog title!

    I agree – skill, intelligence, creativity, and tons of hard work are crucial. A few scientists do just get lucky, but the ones I admire are the ones who keep doing good work in different fields, so chance is statistically implausible as an explanation.

    Yes, credit assignment in science is a snake pit. When I said that I didn’t respect all famous biologists, Crick was one name I had in mind.

    One reason I left academia for entrepreneurship is that in “the real world” there’s an objective measure of success: either you make money or you don’t. Too much of success in science is politics – and of course the squishy fields are much worse.

    In science as elsewhere, zero-sum winner-take-all games do not seem to favor women. That might be in part because women have enough sense not to participate in them; but there is doubtless also deliberate exclusion involved. The dynamics of alpha male competition seem to be the same in science as elsewhere, and present women with the same set of unattractive choices. (What is your experience of this? You’d probably be able to say much better than me.)

    I found Evelyn Fox Keller’s books on women’s distinctive ways of doing science inspiring. Somewhere she introduces the idea of “wet data”, a counterpart to “hard data” – with deliberate sexual symbolism. Wet data are ambiguous, inchoate, mysteriously fascinating. It’s wet data, not hard data, that draw you into the questions that really matter. (I wrote a riff on this titled “Going down on the phenomenon,” in which I compared the style of science I favor with cunnilingus. Perhaps fortunately, that doesn’t seem to have made it onto the web.)

  9. @JS Allen – Hmm, I’m enough of a dinosaur that I’m not sure why it’s ironic even in hindsight! Is it because modern hardware is so reliable that computer scientists never have to fix it? (Not that anyone could fix a modern computer – they are effectively disposable…)

  10. Hmm, I’m enough of a dinosaur that I’m not sure why it’s ironic even in hindsight! Is it because modern hardware is so reliable that computer scientists never have to fix it? (Not that anyone could fix a modern computer – they are effectively disposable…)

    I had to use Lisp for my work around 1992, but didn’t have an actual dedicated Lisp machine. I had heard of Lisp machines, but now find the concept to be a little bit ironic in hindsight. The more ironic part is the idea of debugging a power supply, since that component of all things should be pluggable and general-purpose. When I read that sentence, it almost sounded like, “You will need leadership skills to motivate the highly-trained oompah-loompahs who run on the treadmills to power the Lisp machine”.

    Maybe it’s because I had a dream last week about “debugging a power supply” that I was predisposed to giggle a bit about that. In my dream, someone had just replaced the power mechanism on some household fixture that had previously been powered through a small wind turbine on the roof. They had taken out the wind turbine and replaced it with a large battery pack that drove a small engine to turn the crank, where the wind turbine previously drove the crank. They had called me in to ask if I could help them change the gear ratios to get more power. I had to explain that they were using a motor to drive a generator, and the power was going to be consistent with the battery pack size, and why not just connect the battery pack directly to the fixture?🙂

  11. Great dream!

    I can’t remember for sure, but I don’t think the text was meant to be ironic in that way. Lisp Machines were like race cars: hand-built, very fast, incredibly expensive, each one unique, on the hairy edge of feasibility, and temperamental as hell. The hardware would crash every few hours for sure. Power supplies in those days were not generic; they were engineered specifically for each device. I don’t specifically remember, but it’s plausible that the Lisp Machine’s was notoriously flaky.

    I fear this may be wandering off Sabio’s topic…

  12. crl

    My experience? I’m just a student. Most of my knowledge of the scientific community comes from my father, a geologist. At least I have an idea of what I’m getting myself into!

    The general march in science (and most other fields) is towards greater equality, so, most likely, in the future, there will be more women producing hard and wet data alike. What this will do to the general scientific climate I have no clue, but, since woman are known to play dominance games as well, I doubt it will solve all “alpha-male” type problems.

Please share your opinions!

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s