The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses

Part 2 begins with Chapter 5, “Leap.” This chapter begins with an anecdote from Facebook’s early years. Mark Zuckerberg, Dustin Moskovitz, and Chris Hughes of Facebook came to Silicon Valley in 2004; by summer they had raised $500,000 in venture capital and, less than a year later, they had raised $12.7 million more (79). Ries uses Facebook as an example of how a company can raise enormous amounts of money despite their product’s actual usage being small. The first impressive thing about Facebook was its ability to validate its value hypothesis to its investors. The second thing was the rapid growth rate, which validated its growth hypothesis. Facebook spent no money on customer acquisition, yet still maintained high engagement. This anecdote leads into a discussion on the role of strategy and the importance of asking the right questions in a startup’s early days.

A key concept in the Lean Startup model is that “strategy is based on assumptions” (81). The goal of an early startup is to test these assumptions as early as possible. First, the entrepreneur needs to build an organization that can test the initial assumptions of the product and the customers. Second, the startup needs to start experimentation without losing sight of vision. A leap of faith is the entrepreneur’s belief and action in an assumption, which can lead to success or failure. Ries writes that leaps-of-faith function as arguments and are made in the form of analogies. The problem with analogies, however, is that they obscure the truth about the assumption. They are used to persuade investors, employees, and partners. Instead, Ries recommends that entrepreneurs make analogies that identify what empirical testing is needed first. The analogy can only be made based on the assumption that customers really want to embrace a new product or technology.

Ries then introduces the notion of analogs and antilogs. Both are used to plot strategy, rather than create assumptions based on leaps of faith. Analogs find current products that answer the startup’s assumptions, like the Apple iPod developed after the Sony Walkman to address the question of whether people would listen to music in public spaces through headphones. An antilog addresses the assumed concerns of the customers in relation to the product’s capabilities (like Napster surpassing traditional CD-ROM and cassette tape hardware because it assumed customers would not want to pay for downloadable music). Both were leaps of faith made on the empirical data that proved the assumption true.

The discussion then shifts to “the value creation hypothesis” and “the growth hypothesis” (85) as the first steps in understanding a product’s potential success or failure. Products are either “value-creating” or “value-destroying.” A value-destroying initiative would be one that raises money from investors for the startup while not actually developing a useful product. Although the company technically continues to grow, the growth is not actual customers paying for products. This kind of startup is doomed to fail.

A basic strategic design that Toyota Production System follows relies on understanding a customer base. This core principle, genchi gembutsu, is a crucial keyword for lean manufacturing. In English, this phrase translates to “go and see for yourself” (86), suggesting that entrepreneurs make decisions based on first-hand knowledge. When Toyota decided to design a new model of minivan for the North American market, the chief engineer, Yuji Yokoya, decided to travel more than 53,000 miles across the US, Canada, and Mexico while “talking to and observing real customers” (87). Based on these first-hand observations, Yokoya discovered that North American adult buyers were less important than the children passengers. With this insight, Yokoya designed the Sienna minivan to include “internal comfort features” (87) for long-distance family road trips. This example of sustaining innovation led to a 60% increase in Sienna minivan sales from 2003 to 2004.

The concept of genchi gembutsu tests the assumptions of an entrepreneur in the real-world with real customers. First, an entrepreneur should confirm that their product designs are not misled by projected numbers, theoretical market research, or unvalidated assumptions. Early contact with a customer base is crucial. It helps build a customer archetype, a “brief document that seeks to humanize the proposed target customer” (89). This document is a guide for product development and maintains the prioritizations of crucial decisions. Yet, even this document is a hypothetical one and startups should not think that once this data is collected that the experimentation ends. This would stall a startup’s progress and innovation. Alternatively, some startups suffer from analysis paralysis, a syndrome that occurs when entrepreneurs refine their plans incessantly without implementing any action based on those plans.

Chapter 6, “Test,” introduces the minimum viable product (MVP). The minimum viable product is a cheap, low-quality design that can be launched for immediate customer feedback. The MVP allows entrepreneurs to kickstart the Build-Measure-Learn feedback loop as quickly as possible. Ries uses the example of Groupon, who released an MVP in the form of a WordPress blog that posted everyday about a single sale. Customers would then email the address in the blog post and receive a PDF with a coupon. Although primitive, this product worked, and it attracted customers. Groupon eventually became the fastest company in history to achieve $1 billion in sales (93).

The MVP begins the process of learning so that necessary adjustments can be made as soon as possible. It is not a prototype or a concept, but a real product to be released and sold to gather customer feedback. These products are not meant to be perfect. In fact, the MVP is designed to figure out the early adopters of an innovation movement. Early adopters differ from the mass market in that they are attracted to products outside of mainstream availability and capability. They want to be on the frontline of cutting-edge innovation. Early adopters provide valuable feedback on what the product is missing because they care more about being the first to use a new technology, formula, service, or design. The MVP should be simple, not polished or even high-quality. All additional work and extra features beyond the requirement to start learning about what works and what doesn’t is a waste.

Ries provides different examples of MVPs. One is the concierge MVP. The concierge MVP is a personalized service released in extremely small batches (sometimes just one product) that is designed to test assumptions in the company’s growth model. This type of MVP can save significant a company time and resources: “In fact, a common outcome of a concierge MVP is to invalidate the company’s proposed growth model” (102). For example, Food on the Table (FotT) is a startup that creates meal plans, recipes, and grocery lists for customers based on their favorite foods. The company’s flagship service tracks the best deals from local grocery stores and devises a shopping list for the individual or family. The concierge MVP of this startup began with one customer. The CEO and VP talked to grocery stores and designed the user platform while trying to find a single customer who would buy their service. Once they found that customer, they carefully prepared all their services by hand and adjusted the recipe and grocery lists based on the customer’s life, habits, and obligations. Although inefficient, the team learned something new about their product each week from just a single customer. In return, the customer paid for the service and received in-person visits and personal treatment. Eventually, only after the team became too busy with the success of their MVP did they implement automation into the process. By focusing their efforts on a single product and customer, the founders avoided creating waste and continued to learn how to make their product better. FotT went nationwide shortly after this experiment.

Another type of MVP uses Wizard of Oz testing. Wizard of Oz testing is when a product pretends to be automated but uses real employees behind the scenes. Google did this when they designed Aardvark, a subjective question search engine that would specifically answer users’ questions based on user experience, taste, and assessment, instead of just factual information. Google launched a series of small products to test, all of which were run by real employees answering the questions. Google gathered information about their customers through this technique until they were able to design an automated version that met the expectations of the customers.

Chapter 6 then addresses the issue of quality. Although the ultimate goal of a startup is to design a high-quality product, validated learning and lean manufacturing uses low-quality MVPs to gather empirical data about customer needs. With this information, the company can then design higher and higher quality products. Ries’s point is to reduce waste by making small, inexpensive (both in money and time) changes to product design. Ries advises on a simple rule: “remove any feature, process, or effort that does not contribute directly to the learning you seek” (110).

Some obstacles that occur when building an MVP are patenting, competition, and branding. Learning domestic and international patent law can help with the first issue. Worrying that someone will steal the startup’s idea should not be a major concern because the biggest hurdle of a startup is getting known in the first place. To beat competition once the startup gets traction, the goal is to learn faster and smarter. If an MVP fails, then the startup has just gathered valuable data for adjustments. The MVP is a tool for persistent commitment to learning, despite failure, and a strategic approach to innovation in startups.

Chapter 7 introduces innovation accounting. A startup must always assess its current status and devise experiments that will move it closer to its envisioned business plan. The myth of perseverance tricks entrepreneurs into thinking that they can succeed despite the causes of failure being apparent in the metrics. Accounting can be the key to success for startups, albeit not in the traditional sense. The unpredictability of a startup requires a kind of accounting “geared specifically to disruptive innovation” (116).

Innovation accounting takes the leap-of-faith assumptions and transforms them into a “quantitative financial model” (116). One example is the contrast between a manufacturer and a marketplace company. The manufacturing company’s rate of growth depends upon the profit gained from each customer, the cost of gaining new customers, and the repeated purchase rate from these customers. Conversely, a marketplace company needs a different growth model. The success of the latter depends on what attracts customers to the destination where they can contract business. Customers want a marketplace with lots of competition so that they can get the best deals. The network effect of the marketplace company relies on the retention rate of new buyers and sellers. Although both the former and latter companies have different growth models, they both supply a framework for how accountability plays into innovation accounting.

Chapter 7 defines innovation accounting in three steps: 1) use an MVP to establish real data of the company’s status, or baseline, 2) refine the engineering process so that it aligns closer and closer to the startup’s vision, and 3) pivot or persevere (118).

To establish a baseline for the startup, entrepreneurs should create an MVP to test the assumptions and establish baseline metrics. Another strategy is for a startup to build separate MVPs and test a variety of different metrics to determine the baseline. After the startup determines a baseline, they need to “tune the engine” (119). Every planned innovation—marketing, new product development, initiatives—needs to drive the growth model. Validated learning provides feedback on the effectiveness of this tuning process. If a new product changes customer behavior in a positive (and profitable) way, it should be implemented in the baseline model. If the innovation doesn’t have positive effects, it should be recorded and then removed from the strategy, so the same mistakes are not repeated. The results gathered from validated learning tell the startup whether to persevere or pivot.

The chapter uses another example from IMVU to capture how innovation accounting works. After a period of stagnation at IMVU, Ries and his team started to track their metrics more frequently and use product development to tighten the feedback loop. For instance, they allocated $5 a day for advertisements on the Google AdWorks system. This amount of money brought in about 100 clicks per day. Every day, the team was able to measure the product’s performance in terms of customer engagement. Each day, the team would modify the advertisements, changing messages for first-time customers, or changing how customers were initiated into the network. These inexpensive and random trials allowed IMVU to experiment and modify their product to better match customer appeal.

The team used this data to conduct a cohort analysis. A cohort analysis looks at “the performance of each group of customers that comes into contact with the product independently” (123). This method allows entrepreneurs to view customer engagement through a quantitative lens. The efficacy of product improvements, the new customer retention rate, and the new customers willing to pay money for the product can be tracked. Thanks to the cohort analysis, IMVU realized that despite all the modifications in product development tailored to the customer feedback, the number of new customers who paid for the product remained the same throughout the experimentation period. This “failure” prompted Ries to ask: “Why aren’t customers responding to our product ‘improvements’?” (125). This question became a crucial pivot point in the company’s history: move “away from an IM add-on used with existing friends and toward a stand-alone network one can use to make new friends” (125). The innovation accounting framework addresses when a company stagnates (i.e., needs to pivot) and when it needs to stay the course (i.e., needs to persevere) with quantitative data. However, the chapter also warns against “vanity metrics,” like gross number of customers, which appear to indicate growth but may only be appealing because they satisfy investors. Vanity metrics are opposed to actionable metrics, which track the rate of growth, profitability, and customer acquisition and retention across different quantitative methods. Vanity metrics look at one or two positive areas of improvement, ignoring the red flags in others.

An example of a company using actionable instead of vanity metrics is Grockit. Grockit, founded by Farbood Nivi, is a peer-to-peer learning platform for people who can’t afford an expensive online class or private tutor. Nivi used agile development to create his product. Agile development uses a series of sprints, or one-month iteration cycles, which create user-stories. These user-stories would write about a new feature from the point of view of the customer, as opposed to writing a specification for that new feature (132). The user-story helped developers focus on the customer’s experience using the product. Each sprint allowed the team to learn which features customers liked and disliked without putting in the work to integrate those new features. Eventually, the Grockit team learned that their innovations, however, were being directed by vanity metrics: the total number of customers and the total number of questions answered. This caused the team to waste unnecessary effort on product development. So, while still using the principles of agile development and innovation accounting, Grockit started to track different metrics. They changed to cohort metrics from the traditional cause-and-effect metrics. The team started to split their customer sample into users of the new version and users of the old version. This method of split testing allowed Grockit to figure out what customers wanted and didn’t want, while still maintaining their quick pace of development.

Agile development and innovation accounting are similar to the lean manufacturing principle of kanban, or capacity constraint (138). For Grockit, this meant cataloging each user-story in one of four buckets: backlog, in progress, built, and validated. Only when a user-story becomes validated can it be removed from the buckets. No bucket can contain more than three stories at a time, making the team prioritize the completion of their tasks. So, instead of measuring success by the number of user-stories produced, the team pivoted and started measuring the number of user-stories that were validated. This was validated learning in action.

The Grockit demonstration followed the three As of metrics for startups: “actionable, accessible, and auditable” (143). Actionable refers to a cause-and-effect metric. This determines what specific actions produce clear results. Accessibility refers to the level of availability and understanding that each member of the company has with the metrics. Accessibility means the access to reports and evaluations for all members of a team. Auditable ensures that the data employees are using is credible. This metric guarantees that the actionable and accessible metrics are being corroborated with the master data of a certain experiment in product development. Chapter 7 ends with the myth of perseverance. Startups need optimism to keep afloat, but without real data to match that optimism, the company will spin its gears into failure.

Chapter 8 hones in on the two strategies of growth introduced in the previous chapter. The decision of whether to pivot or persevere is crucial to the success of a product’s development. A pivot is defined as “a structured course correction designed to test a new fundamental hypothesis about the product, strategy, and engine of growth” (149). An entrepreneur who decides to pivot needs to acknowledge both human creativity and scientific rigor. Getting stuck on either of these without addressing the other is a recipe for failure. Although the Lean Startup method uses scientific approaches in product development, it necessarily incorporates the human element in its strategic practice.

One example is CEO David Binetti who pivoted his company, Votizen (an online voter registration platform), using innovation accounting. Binetti’s initial company assumed for leaps of faith about customer behavior: 1) registration, 2) activation, 3) retention, 4) referral (151). Binetti launched an MVP to gather feedback on these assumptions. Registration and activation scored 5% and 17%, respectively, in the cohort analysis. Retention and referral, however, recorded numbers so low that data couldn’t be collected. Binetti then ran a split testing experiment for optimization of the product. The numbers recorded skyrocketed to 17% registration, 90% activation, 5% retention, and 4% referral. However, after another experiment with split testing, the numbers remained the same. At this point, Binetti needed to decide whether to pivot or persevere.

Despite the optimizations improving the metrics, these initial experiments were not generating sustainability for the business model. Binetti decided to do a zoom-in pivot, a method of adjustment that focuses on one feature of a product. According to customer feedback, customers enjoyed using the registration technology, but they didn’t see value in the social networking aspect of the platform. For this reason, Binetti changed Votizen to @2gov, a “social lobbying platform” (155). With this change to the civic social network, customers would be able to contact their elected representatives via existing social media networks. This became the new MVP to experiment with customers. @2gov witnessed improvements across all metrics and displayed a sustainable value through new customer retention. Binetti continued to conduct pivots: a customer segmented pivot where he changed audience focus and a platform pivot where he enabled people to become customers with just a credit card. All these pivots involved manageable yet novel MVPs to validate the learning from each experiment.

The next scenario is the possibility of a startup losing funds. Rather than measure how much money a startup has according to its financial holdings, startups should measure their time in the number of pivots they can accomplish with that money. The Lean Startup method advises entrepreneurs on how to “achieve the same amount of validated learning at a lower cost or in a shorter time” (161). Entrepreneurs face three fears when needing to pivot. First, they allow vanity metrics to cloud their judgement. Second, they fail to establish a clear hypothesis for product development and customer behavior. Third, they fear failure. A successful entrepreneur who leads a successful startup needs to have the courage to pivot, despite the possibility of failure.

Startups should have regular “pivot or persevere” (164) meetings with both the product development team and the business leadership. The product development should draft a report on the results of product optimization experiments over time. The business leadership team should bring accounts of their conversations with new, current, and past customers. Once both teams have shared their data, they can decide whether to persevere with the current strategies or pivot.

Chapter 8 then outlines 10 different pivot strategies: A pivot is not just a change, but instead is “designed to test a new fundamental hypothesis about the product, business model, and engine of growth” (173). The zoom-in pivot develops a single feature of a product into the whole product itself. A zoom-out pivot identifies that a single feature is not able to support the whole product and decides that more features are needed. A customer segment pivot occurs when a product hypothesis is confirmed, but not for the anticipated/target customer. A customer need pivot emerges from interviews with a customer base that reveal a genuine want, desire, or need from the buyers. The product then changes to satisfy these new insights that are directly marketed to the customer base. A platform pivot changes the application or platform of a product in light of new data on customer registration, activation, retention and referral. A business architecture pivot changes a company from a high margin, low volume approach to a low margin, high volume approach, or vice versa. This means that a company can shift from business-to-business to consumer products, i.e., from long expensive sales cycles to mass market. A value capture pivot refers to when a company discovers that the value of the business, product, or marketing strategies needs to be adjusted for a new product hypothesis. The engine of growth pivot changes the strategy of the company for faster and more profitable growth. This can be viral, sticky, or paid growth (see Chapter 10). The engine of growth pivot also changes the way value can be captured. The channel pivot alters the means of delivery for a product to its customers. This pivot realizes that the same product can be delivered through more efficient and cheaper avenues without changing the product’s quality or experience for the customer. A technology pivot happens when a company discovers that a new or different technology can deliver the product in a more efficient way.

Chapter 8 ends by reminding readers that to pivot means to make a strategic hypothesis (176). Based on validated learning, innovation accounting, and the engines of growth, a pivot or persevere approach can be quantitively determined for greater possibilities of entrepreneurial success.