90% of feedback is crap: how to find the next big startup idea

_Editor's Note: This article was written in 2013. Flatiron Health was acquired by Roche for $1.9 billion in 2018._ When you ask successful entrepreneurs about how to discover the next

great startup idea, many suggest solving a problem you face on a regular basis — to build something that ‘scratches your own itch.’ Nat Turner, co-founder of Flatiron Health, has a different

approach. He's built a unique methodology to systematically find the next great thing without being what he calls “a visionary founder.” It’s already worked once before — with his

first company, Invite Media, which sold for a reported $80 million to Google in 2010. Now he’s back at it again with his second venture-backed company, Flatiron Health. He's recently

raised his first round of $8 million, led by First Round (that’s us) and Google Ventures. TURNER’S ENTIRE PROCESS CAN BE BOILED DOWN INTO A HANDFUL OF KEY STEPS: * Begin with a general space

of interest and look for any seed of an idea * Network like crazy in the industry and take introductions extremely seriously * Create a deck and start pitching anyone smart and relevant you

can find on the specific idea – and be sure to take meticulous notes and follow up * Tweak the deck between every meeting and hack together a demo and start pitching (pre-sell software that

doesn’t exist yet but will) * Find a trusted group of advisors to discuss key learnings from the field and get feedback * Continue to iterate as hard and fast as possible on the feedback

you’ve received * Hope you’re smart and good enough to make it happen BUILDING A KNOWLEDGE NETWORK In 2007, Turner interned at VideoEgg (now Say Media) during his sophomore year in college.

During a sales meeting, an executive from a potential advertiser said, “We want to advertise on VideoEgg, but we don’t have a big production company to build a video ad.” That night, he and

his co-founder, Zach Weinberg, sat down and began to think of ways they could help companies create video ads quickly and affordably. But instead of just researching the idea, or talking

casually with others in the online advertising space, they immediately began to pitch — which is a core tenant of Turner’s approach to finding great ideas. Turner recounts the early days of

Invite Media, “We pitched everyone. We pitched potential advisors, investors, potential customers, friends, everyone. I can remember hundreds of meetings we had.” Importantly, these weren’t

pitches to get investment — they were pitches to get feedback. After beginning to pitch the idea, Turner and Weinberg and their two other co-founders built a demo as fast as possible. They

realized that it's almost always better to bring something tangible to show people — because people have an easier time reacting to a product than an abstract concept. After endless

meetings of pitching a video ad creation product, the feedback they received began to form a pattern that they could act on. Every time Turner and Weinberg work on an idea, they’re always

looking for a few key characteristics: * Can this be a big business? Does the opportunity to build a billion dollar company exist in this space? * Is this sustainable? Can this idea become

sticky and last for a while? * How scalable is this? How well does this business scale and how does this grow nonlinearly? “Within four months of our initial video ad creation idea, we

realized that it was going to be really hard, based on talking with people, to get enough ad inventory and to scale the demand. Places like YouTube were just getting big and controlling

their inventory. It would also be hard to work with small businesses — who would be the customers that would have issues building video ads. It was just a slog,” Turner noted. After this

first failed attempt, the pair began searching for their next germ of an idea. Around this time, Facebook was just launching their developer platform — and that piqued their interest. Nat

and Zach built a Facebook app related to advertising and pitched it to more people. Again, they took the approach of building and pitching as hard and fast as possible. One of their pitches

was to Brian O'Kelley, now Co-Founder and CEO at AppNexus and formerly CTO of Right Media. A few minutes into the pitch, O’Kelley dismantled their idea and explained why it wouldn’t

work. Their plan rested on being able to modify Facebook’s ad platform. O’Kelley told them that Facebook would never let them do what they wanted to do. But, he planted a seed for a new idea

that worked better. O’Kelley believed that “there was this big opportunity with ad exchanges in display advertising where there's going to be more than one, and ad networks have their

own inventory, publishers have their inventory, agencies might have their inventory. There's all this inventory, and someone needs to make sense of it all." After that meeting,

Turner and Weinberg had a new product to pitch: a universal buying platform for display advertising. Again, they built a demo and started pitching anyone who would listen to them. PITCH

& ITERATE The challenge with this approach of intensive pitching and listening is you get a tremendous amount of feedback. The market will tell you lots of things: some right, and some

wrong. It’s your job as an entrepreneur to separate the signal from the noise. > Being a good entrepreneur requires having a hundred conflicting data > points in your head and being 

able to still make a decision. That’s the gift. Turner explains, “If you're going to be successful as an entrepreneur, the biggest thing is being able to take in conflicting feedback

from all these people, many of whom are jaded or have bad habits, and some who are spot on. You have to be able to sit there and distill that information into something valuable. The hard

part isn’t coming up with ideas; it's distilling all the information you have, 90% of which will be crap, and finally figuring out what is the good 10%, recognizing that the good 10%

may change rapidly depending on the industry.” Turner and Weinberg look intently for the valuable 10%, build a demo, pitch it and then repeat the process of iteration as fast as possible.

They tweak the demo and the slides even during the scant hours between meetings. Every time Turner takes a new meeting, he updates the deck with fresh feedback from the previous one. His

current company, Flatiron Health, is on deck version number 30 (with working software now!). They began the process of building Flatiron the same way as they built Invite: by picking a

generally interesting area, in this case health. Turner elaborates, “Our first idea was a second opinion site, and the second idea was a clinical trial matching tool, and the third idea was

a business intelligence tool. It's changed a hundred times in the last hundred days.” NOT ALL INTROS ARE EQUAL One of the foundational elements of pitching ideas consistently is finding

the right people to pitch, and then connecting with them in a meaningful way that aligns their incentives with yours. Turner focuses on finding the most valuable people to pitch (which he

tracks in Trello and Asana), and then does the hard work of researching and hustling to get a high-quality introduction. During the actual meeting either Turner and/or Weinberg records

meticulous notes in their Moleskine notebooks and then transfers them into Evernote (Turner finds having a computer out to be too distracting in meetings for the participants). After the

meeting, they have a postmortem to analyze the pitch. They pull the most valuable information from it, and start work on all the action items from the meeting. In addition to the

post-meeting download, Turner is maniacal about following up. During the meeting, every action item is denoted with a star to make sure nothing falls through the cracks. “Within 24 hours you

have to respond to that person that you set up the meeting with, thanking them for their time and being very diligent about making it as easy as possible for them to provide value such as

introductions. It’s amazing how many people just don’t follow through with this simple step.” When someone has offered to make an introduction for him, Turner makes the follow up simple by

sending a clean email that’s easily forwarded to their target attached with a one pager if appropriate. He also uses Boomerang to make sure that he gets a reminder if he doesn’t hear back.

“If you’re diligent about follow up,” says Turner, “you’ll be more effective than 90% of the entrepreneurs out there.” CIRCLE OF TRUST In the process of meeting with hundreds of people,

Turner and Weinberg utilize a trusted group of advisors. They’re not the typical “figurehead” advisors whose names are thrown into a pitch deck to puff it up. Rather, these advisors serve as

a sounding board throughout this idea refinement process. In the case of Invite Media, Brian O’Kelley, David Brussin (Founder and CEO at Monetate), Josh Kopelman and several others all

performed that role in the very early days. Every other week, Turner would connect with this group of confidantes informally to share new information from the market, and to get feedback on

big strategic questions that were exposed. They compensate this circle of advisors with equity (if the person was not already an investor) to align their incentives with the new company. The

big strategic questions are tracked in a Google doc throughout the company building process. Turner and Weinberg also track any product features that come out of their conversations as well

as any open questions in a Google doc so it’s easily referenceable at all times. DOMAIN EXPERTISE IS OVERVALUED Turner and Weinberg’s last two companies were both in spaces that they knew

little about at the beginning of the process. All too often you hear from investors about the importance of “domain expertise,” but Turner doesn’t really buy it. He’s found that while a bit

of experience — roughly under a couple of years — can be helpful, most of the time it’s actually a detriment. Turner says, “If you've been a doctor for example, one end of the spectrum,

I think you're pretty much going to be a questionable entrepreneur because you're so used to the constraints of the industry, how the whole system works and the incentives, that

you can't see through the mess. You also spent 5+ years getting an overdose of a specific skillset, as opposed to the diverse skillset you need to start a company. There are obviously

exceptions, but it’s more of an uphill battle.” Turner’s method of investing massive amounts of time into learning about a space by pitching knowledgeable advisors, creating lightweight but

polished demos, and continually improving the process is a way of gathering useful domain expertise without accumulating the biases of working in an industry for decades. You can instead

draw upon collectives of hundreds of years of cumulative experience by going out and pitching, and pitching, and pitching to get the best quality knowledge available.