How to come up with a great idea
“How do I come up with a great idea?” I remember the question, and that it was a crisp, fall afternoon in New York City. I was sitting in my sparse office on the 12th floor of the glass and steel Northwest Corner Building at Columbia University.
In the course I was teaching, I asked students to propose a set of experiments that might answer an important unsolved scientific question. Most students settled quickly upon a research topic, but some students struggled to identify a question to study. This particular student was having trouble. “I don’t know how to come up with a new idea or make an important discovery,” he explained. He pressed on. “Is there a way you can teach me to do that?”
As I sat and stared out the window at the miniature figures scurrying across the campus far below, I realized that I had no clear idea of how to answer. I was skilled at helping students refine an existing idea, and design experiments to test their idea, but I had never had to explain the process of creating an exciting idea in the first place. No one had ever asked me this question before, and no one had taught me to come up with a great idea. I realized I had no obvious way to answer his question. I couldn’t explain how scientists come up with great ideas and theories. I could explain hypothesis-based research — generate a hypothesis, design an experiment to test it, use the right controls, analyze the data, and see if they support or reject a hypothesis. But this didn’t explain how to actually craft an exciting new hypothesis or idea to begin with. That was a tougher problem.
As I pondered this question in the subsequent years, I realized that most people, even active researchers, often have no clear idea about the process that leads to their best ideas. Each year in my course at Columbia, I now ask the students to come up with a great idea and a plan to test it. I meet with students and discuss their idea-generation process, and I’ve learned a lot about how different people generate ideas, and which creative processes work better than others.
Counterintuitively, trying to solve a big problem often doesn’t help you generate a great idea — the big problem is too insurmountable — a sheer rock face without a foothold.
Often the great ideas come from the bottom up. A student is interested in a topic and starts reading papers, and the more they read, the more interested they become in the details of the topic. Then the key insight comes when a student learns about an intriguing observation related to their topic for which the mechanism is not known. Suddenly, the student is grappling with an exciting mystery, and they try to think of hypotheses for what the mechanism might be, and how they would test it. Now they are starting on the path towards a great idea.
I’m partial to this approach to idea generation, because I’ve seen it work not only in the classroom, but in the lab. However, in the lab, we have the opportunity to generate our own intriguing observations, rather than just reading about them in other people’s papers.
When I was starting my own laboratory in 1999, my first goal was to assemble a collection of thousands of drugs that had been approved for use in humans. I began the first day in my new laboratory reading through every page of the 3-inchthick catalog of Sigma-Aldrich, the main chemical supplier for research labs. I highlighted in bright yellow every drug that had been approved for use in humans by the US Food and Drug Administration, as well as any chemical that had been reported to cause biological activity in human cells. Then, I called Sigma-Aldrich and told them I wanted to order 2,000 chemicals. The person on the other end of the line had a hard time believing me, telling me it wouldn’t be possible to sell that many chemicals at once.
After a number of conversations with managers up the hierarchy, I eventually struck a deal in which Sigma-Aldrich would ship me all 2,000 chemicals so that I could test them in my new lab. The bottles started arriving day after day, case after case, some in giant white plastic containers, some in tiny brown glass bottles. With the help of a lab assistant and an undergraduate student, I scooped out small amounts of each chemical and prepared them for testing.
By testing the 2,000 drugs in a variety of experiments, my students and I discovered that some drugs were effective in cell models of diseases for which they hadn’t been approved or explored. These results suggested that there might be a different route to finding new medicines and treatments: repurposing existing drugs for new diseases. Other researchers also began looking for such repurposing opportunities with existing drugs — these efforts expanded and continue to this day. Each time we find a new, unexpected effect of a drug in a new disease model, we have an intriguing observation that might be the basis for an impactful proposal.
The bottom line. My advice to those trying to come up with an exciting new idea for a research project is to first read a tremendous amount about the topic you are interested in, and find a striking observation for which the mechanism is not known. Then think creatively about a possible mechanism, and find existing evidence in unrelated fields to support the possible mechanism. This same process can be applied to other areas of life outside research proposals — you can generate ideas for what makes a successful company, product, or work of art. If you have an exciting observation, and an intriguing hypothesis about the mechanism, you have just come up with a great idea.
