startupmap

[startupmap]

How To Survive a Ground-Up Rewrite Without Losing Your Sanity

aka: Screw you Joel Spolsky, We're Rewriting It From Scratch!

This is a guest post by Dan Milstein (@danmil), co-founder of Hut 8 Labs.

Disclosure: Joel Spolsky is a friend and I'm an investor in his company, Stack Exchange(which powers the awesome Stack Overflow) -Dharmesh

So, you know Joel Spolsky's essay Things You Should Never Do, Part I? In which he urgently recommends that, no matter what, please god listen to me, don't rewrite your product from scratch? And lists a bunch of dramatic failures when companies have tried to do so?

First off, he's totally right. Developers tend to spectacularly underestimate the effort involved in such a rewrite (more on that below), and spectacularly overestimate the value generated (more on that below, as well).

But sometimes, on certain rare occasions, you're going to be justified in rewriting a major part of your product (you'll notice I've shifted to saying you're merely rewriting a part, instead of the whole product. Please do that. If you really are committed to just rewriting the entire thing from scratch, I don't know what to tell you).

If you're considering launching a major rewrite, or find yourself as the tech lead on such a project in flight, or are merely toiling in the trenches of such a project, hoping against hope that it will someday end... this post is for you.

Hello, My Name is Dan, and I've Done Some Rewrites

A few years back, I joined a rapidly growing startup named HubSpot, where I ended up working for a good solid while (which was a marvelous experience, btw -- you should all have Yoav Shapira as a boss at some point). In my first year there, I was one of the tech leads on a small team that rewrote the Marketing Analytics system (one of the key features of the HubSpot product), totally from scratch. We rewrote the back end (moving from storing raw hit data in SQLServer to processing hits with Hadoop and storing aggregate reports in MySQL); we rewrote the front end (moving from C#/ASP.Net to Java/Tomcat); we got into the guts of a dozen applications which had come to rely on that store of every-hit-ever, and found a way to make them work with the data that was now available. (Note: HubSpot is now primarily powered by MySQL/Hadoop/HBase. Check out the HubSpot dev blog).

It took a loooong time. Much, much longer than we expected.

But it generated a ton of value for HubSpot. Very Important People were, ultimately, very happy about that project. After it wrapped up, 'Analytics 2.0', as it was known, somehow went from 'that project that was dragging on forever', to 'that major rewrite that worked out really well'.

Then, after the Analytics Rewrite wrapped up, in my role as 5 Whys Facilitator, I led the post-mortem on another ambitious rewrite which hadn't fared quite so well. I'll call it The Unhappy Rewrite.

From all that, some fairly clear lessons emerged.

First, I'm going to talk about why these projects are so tricky. Then I'll pass on some of those hard-won lessons on how to survive.

Prepare Yourself For This Project To Never Fucking End

The first, absolutely critical thing to understand about launching a major rewrite is that it's going to take insanely longer than you expect. Even when you try to discount for the usual developer optimism. Here's why:

Migrating the data sucks beyond all belief

I'm assuming your existing system has a bunch of valuable data locked up in it (if it doesn't, congrats, but I just never, ever run into this situation). You think, we're going to set up a new db structure (or move it all to some NoSQL store, or whatever), and we'll, I dunno, write some scripts to copy the data over, no problem.

Problem 1: there's this endless series of weird crap encoded in the data in surprising ways. E.g. "The use_conf field is 1 if we should use the auto-generated configs... but only if the spec_version field is greater than 3. Oh, and for a few months, there was this bug, and use_conf was left blank. It's almost always safe to assume it should be 1 when it's blank. Except for customers who bought the Express product, then we should treat it as 2". You have to migrate all your data over, checksum the living hell out of it, display it back to your users, and then figure out why it's not what they expect. You end up poring over commit histories, email exchanges with developers who have long since left the company, and line after line of cryptic legacy code. (In prep for writing this, when I mentioned this problem to developers, every single time they cut me off to eagerly explain some specific, awful experience they've had on this front -- it's really that bad)

Problem 2: But, wait, it gets worse: because you have a lot of data, it often takes days to migrate it all. So, as you struggle to figure out each of the above weird, persnickety issues with converting the data over, you end up waiting for days to see if your fixes work. And then to find the next issue and start over again. I have vivid, painful memories of watching my friend Stephen (a prototypical Smart Young Engineer), who was a tech lead on the Unhappy Rewrite, working, like, hour 70 of an 80 hour week, babysitting a slow-moving data export/import as it failed over and over and over again. I really can't communicate how long this takes.

It's brutally hard to reduce scope

With a greenfield (non-rewrite) project, there is always (always) a severe reduction in scope as you get closer to launch. You start off, expecting to do A, B, C & D, but when you launch, you do part of A. But, often, people are thrilled. (And, crucially, they forget that they had once considered all the other imagined features as absolutely necessary)

With a rewrite, that fails. People are really unhappy if you tell them: hey, we rewrote your favorite part of the product, the code is a lot cleaner now, but we took away half the functionality.

You'll end up spending this awful series of months implementing all these odd edge cases that you didn't realize even existed. And backfilling support for features that you've been told no one uses any more, but you find out at the last minute some Important Person or Customer does. And, and, and...

There turn out to be these other system that use "your" data

You always think: oh, yeah, there are these four screens, I see how to serve those from the new system. But then it turns out that a half-dozen cron jobs read data directly from "your" db. And there's an initialization step for new customers where something is stored in that db and read back later. And some other screen makes a side call to obtain a count of your data. Etc, etc. Basically, you try turning off the old system briefly, and a flurry of bug reports show up on your desk, for features written a long time ago, by people who have left the company, but which customers still depend on. This takes forever all over again to fix.

Okay, I'm Sufficiently Scared Now, What Should I Do?

You you have to totally own the business value.

First off, before you start, you must define the business value of this rewrite. I mean, you should always understand the big picture value of what you do (see: Rands Test). But with rewrites, it's often the tech lead, or the developers in general, who are pushing for the rewrite -- and then it's absolutely critical that you understand the value. Because you're going to discover unexpected problems, and have to make compromises, and the whole thing is going to drag on forever. And if, at the end of all that, the Important People who sign your checks don't see much value, it's not going to be a happy day for you.

One thing: be very, very careful if the primary business value is some (possibly disguised) version of "The new system will be much easier for developers to work on." I'm not saying that's not a nice bit of value, but if that's your only or main value... you're going to be trying to explain to your CEO in six months why nothing seems to have gotten done in development in the last half year.

The key to fixing the "developers will cry less" thing is to identify, specifically, what the current, crappy system is holding you back from doing. E.g. are you not able to pass a security audit? Does the website routinely fall over in a way that customers notice? Is there some sexy new feature you just can't add because the system is too hard to work with? Identifying that kind of specific problem both means you're talking about something observable by the rest of the business, and also that you're in a position to make smart tradeoffs when things blow up (as they will).

As an example, for our big Analytics rewrite, the developers involved sat down with Dan Dunn, the (truly excellent) product guy on our team, and worked out a list of business-visible wins we hoped to achieve. In rough priority order, those were:

Cut cost of storing each hit by an order of magnitude

Create new reports that weren't possible in the old system

Serve all reports faster

Serve near-real-time (instead of cached daily) reports

And you should know: that first one loomed really, really large. HubSpot was growing very quickly, and storing all that hit data as individual rows in SQLServer had all sorts of extra costs. The experts on Windows ops were constantly trying to get new SQLServer clusters set up ahead of demand (which was risky and complex and ended up touching a lot of the rest of the codebase). Sales people were told to not sell to prospects with really high traffic, because if they installed our tracking code, it might knock over those key databases (and that restriction injected friction into the sales process). Etc, etc.

Solving the "no more hits in SQLServer" problem is the Hard kind for a rewrite -- you only get the value when every single trace of the old system is gone. The other ones, lower down the list, those you'd see some value as individual reports were moved over. That's a crucial distinction to understand. If at all possible, you want to make sure that you're not only solving that kind of Hard Problem -- find some wins on the way.

For the Unhappy Rewrite, the biz value wasn't perfectly clear. And, thus, as often happens in that case, everyone assumed that, in the bright, shiny world of the New System, all their own personal pet peeves would be addressed. The new system would be faster! It would scale better! The front end would be beautiful and clever and new! It would bring our customers coffee in bed and read them the paper.

As the developers involved slogged through all the unexpected issues which arose, and had to keep pushing out their release date, they gradually realized how disappointed everyone was going to be when they saw the actual results (because all the awesome, dreamed-of stuff had gotten thrown overboard to try to get the damn thing out the door). This a crappy, crappy place to be -- stressed because people are hounding you to get something long-overdue finished, and equally stressed because you know that thing is a mess.

Okay, so how do you avoid getting trapped in this particular hell?

Worship at the Altar of Incrementalism

Over my career, I've come to place a really strong value on figuring out how to break big changes into small, safe, value-generating pieces. It's a sort of meta-design -- designing the process of gradual, safe change.

Kent Beck calls this Succession, and describes it as:

"Design changes are usually most efficiently implemented as a series of safe steps. Succession is the art of taking a single conceptual change, breaking it into safe steps, and then finding an order for those steps that optimizes safety, feedback, and efficiency."

I love that he calls it an "art" -- that feels exactly right to me. It doesn't happen by accident. You have to consciously work at it, talk out alternatives with your team, get some sort of product owner or manager involved to make sure the early value you're surfacing matters to customers. It's a creative act.

And now, let me say, in an angry Old Testament prophet voice: Beware the false incrementalism!

False incrementalism is breaking a large change up into a set of small steps, but where none of those steps generate any value on their own. E.g. you first write an entire new back end (but don't hook it up to anything), and then write an entire new front end (but don't launch it, because the back end doesn't have the legacy data yet), and then migrate all the legacy data. It's only after all of those steps are finished that you have anything of any value at all.

Fortunately, there's a very simple test to determine if you're falling prey to the False Incrementalism: if after each increment, an Important Person were to ask your team to drop the project right at that moment, would the business have seen some value? That is the gold standard.

Going back to my running example: our existing analytics system supported a few thousand customers, and served something like a half dozen key reports. We made an early decision to: a) rewrite all the existing reports before writing new ones, and b) rewrite each report completely, push it through to production, migrate any existing data for that report, and switch all our customers over. And only then move on to the next report.

Here's how that completely saved us: 3 months into a rewrite which we had estimated would take 3-5 months, we had completely converted a single report. Because we had focused on getting all the way through to production, and on migrating all the old data, we had been forced to face up to how complex the overall process was going to be. We sat down, and produced a new estimate: it would take more like 8 months to finish everything up, and get fully off SQLServer.

At this point, Dan Dunn, who is a Truly Excellent Product Guy because he is unafraid to face a hard tradeoff, said, "I'd like to shift our priorities -- I want to build the Sexy New Reports now, and not wait until we're fully off SQLServer." We said, "Even if it makes the overall rewrite take longer, and we won't get off SQLServer this year, and we'll have to build that one new cluster we were hoping to avoid having to set up?" And he said "Yes." And we said, "Okay, then."

That's the kind of choice you want to offer the rest of your larger team. An economic tradeoff where they can chose between options of what they see when. You really, really don't want to say: we don't have anything yet, we're not sure when we will, your only choices are to keep waiting, or to cancel this project and kiss your sunk costs goodbye.

Side note: Dan made 100% the right call (see: Excellent). The Sexy New Reports were a huge, runaway hit. Getting them out sooner than later made a big economic impact on the business. Which was good, because the project dragged on past the one year mark before we could finally kill off SQLServer and fully retire the old system.

For you product dev flow geeks out there, one interesting piece of value we generated early was simply a better understanding of how long the project was going to take. I believe that is what Beck means by "feedback". It's real value to the business. If we hadn't pushed a single report all the way through, we would likely have had, 3-4 months in, a whole bunch of data (for all reports) in some partially built new system, and no better understanding of the full challenge of cutting even one report over. You can see the value the feedback gave us--it let Dan make a much better economic choice. I will make my once-per-blog-post pitch that you should go read Donald Reinertsen's Principles of Product Development Flow to learn more about how reducing uncertainty generates value for a business.

For the Unhappy Rewrite, they didn't work out a careful plan for this kind of incremental delivery. Some Totally Awesome Things would happen/be possible when they finished. But they kept on not finishing, and not finishing, and then discovering more ways that the various pieces they were building didn't quite fit together. In the Post-Mortem, someone summarized it as: "We somehow turned this into a Waterfall project, without ever meaning to."

But, I Have to Cut Over All at Once, Because the Data is Always Changing

One of the reasons people bail on incrementalism is that they realize that, to make it work, there's going to be an extended period where every update to a piece of data has to go to both systems (old and new). And that's going to be a major pain in the ass to engineer. People will think (and even say out loud), "We can't do that, it'll add a month to the project to insert a dual-write layer. It wil slow us down too much."

Here's what I'm going to say: always insert that dual-write layer. Always. It's a minor, generally somewhat fixed cost that buys you an incredible amount of insurance. It allows you, as we did above, to gradually switch over from one system to another. It allows you to back out at any time if you discover major problems with the way the data was migrated (which you will, over and over again). It means your migration of data can take a week, and that's not a problem, because you don't have to freeze writes to both systems during that time. And, as a bonus, it surfaces a bunch of those weird situations where "other" systems are writing directly to your old database.

Again, I'll quote Kent Beck, writing about how they do this at Facebook:

"We frequently migrate large amounts of data from one data store to another, to improve performance or reliability. These migrations are an example of succession, because there is no safe way to wave a wand and migrate the data in an instant. The succession we use is:

Convert data fetching and mutating to a DataType, an abstraction that hides where the data is stored.

Modify the DataType to begin writing the data to the new store as well as the old store.

Bulk migrate existing data.

Modify the DataType to read from both stores, checking that the same data is fetched and logging any differences.

When the results match closely enough, return data from the new store and eliminate the old store.

You could theoretically do this faster as a single step, but it would never work. There is just too much hidden coupling in our system. Something would go wrong with one of the steps, leading to a potentially disastrous situation of lost or corrupted data."

Abandoning the Project Should Always Be on the Table

If a 3-month rewrite is economically rational, but a 13-month one is a giant loss, you'll generate a lot value by realizing which of those two you're actually facing. Unfortunately, the longer you solider on, the harder it is for people to avoid the Fallacy of Sunk Costs. The solution: if you have any uncertainty about how long it's going to take, sequence your work to reduce that uncertainty right away, and give people some "finished" thing that will let them walk away. One month in, you can still say: we've decided to only rewrite the front end. Or: we're just going to insert an API layer for now. Or, even: this turned out to be a bad idea, we're walking away. Six months in, with no end in sight, that's incredibly hard to do (even if it's still the right choice, economically).

Some Specific Tactics

Shrink Ray FTW

This is an excellent idea, courtesy of Kellan Elliot-McCrea, CTO of Etsy. He describes it as follows:

"We have a pattern we call shrink ray. It's a graph of how much the old system is still in place. Most of these run as cron jobs that grep the codebase for a key signature. Sometimes usage is from wire monitoring of a component. Sometimes there are leaderboards. There is always a party when it goes to zero. A big party.

Gives a good sense of progress and scope, especially as the project is rolling, and a good historical record of how long this shit takes. '''

I've just started using Shrink Ray on a rewrite I'm tackling right now, and I will say: it's fairly awesome. Not only does it give you the wins above, but, it also forces you to have an early discussion about what you are shrinking, and who in the business cares. If you make the right graph, Important People will be excited to see it moving down. This is crazy valuable.

Engineer The Living Hell Out Of Your Migration Scripts

It's very easy to think of the code that moves data from the old system to the new as a collection of one-off scripts. You write them quickly, don't comment them too carefully, don't write unit tests, etc. All of which are generally valid tradeoffs for code which you're only going to run once.

But, see above, you're going to run your migrations over and over to get them right. Plus, you're converting and summing up and copying over data, so you really, really want some unit tests to find any errors you can early on (because "data" is, to a first approximation, "a bunch of opaque numbers which don't mean anything to you, but which people will be super pissed off about if they're wrong"). And this thing is going to happen, where someone will accidentally hit ctrl-c, and kill your 36 hour migration at hour 34. Thus, taking the extra time to make the entire process strongly idempotent will pay off over and over (by strongly idempotent, I mean, e.g. you can restart after a failed partial run and it will pick up most of the existing work).

Basically, treat your migration code as a first class citizen. It will save you a lot of time in the long run.

If Your Data Doesn't Look Weird, You're Not Looking Hard Enough

What's best is if you can get yourself to think about the problem of building confidence in your data as a real, exciting engineering challenge. Put one of your very best devs to work attacking both the old and the new data, writing tools to analyze it all, discover interesting invariants and checksums.

A good rule of thumb for migrating and checksumming data: until you've found a half-dozen bizarre inconsistencies in the old data, you're not done. For the Analytics Rewrite, we created a page on our internal wiki called "Data Infelicities". It got to be really, really long.

With Great Incrementalism Comes Great Power

I want to wrap up by flipping this all around -- if you learn to approach your rewrites with this kind of ferocious, incremental discipline, you can tackle incredibly hard problems without fear. Which is a tremendous capability to offer your business. You can gradually rewrite that unbelievably horky system that the whole company depends on. You can move huge chunks of data to new data stores. You can pick up messy, half-functional open source projects and gradually build new products around them.

It's a great feeling.

---

What's your take?  Care to share any lessons learned from an epic rewrite?

Like this article? You can now find more popular articles as part of the LinkedIn Influencers program.

Source

http://onstartups.com/tabid/3339/bid/97052/Screw-You-Joel-Spolsky-We-re-Rewriting-It-From-Scratch.aspx

[startupmap]

From Selling Scoops Of Ice Cream To Founding ZeroCater

Editor’s note: Arram Sabeti is CEO of ZeroCater. Follow him on Twitter @arram.

When you throw a rock into the water, it will speed on the fastest course to the bottom of the water. This is how it is when Siddhartha has a goal, [...] This is what fools call magic and of which they think it would be effected by means of the daemons. Nothing is effected by daemons, there are no daemons. Everyone can perform magic, everyone can reach his goals, if he is able to think, if he is able to wait, if he is able to fast – Siddhartha by Herman Hesse

Five years ago I moved to the Bay Area because I wanted to start a company. I came here armed with that single goal and the education of a dozen Paul Graham essays. To me, determination has an almost magical quality. I’d always felt that with enough of it I could do absolutely anything.

People say startups are risky, but for me, working at a job my entire life seemed like the worst possible outcome, so starting a company was the least risky thing I could do. I think that’s what people mean when they talk about founders being a little crazy. I realize this is an unusual personality trait.

To fund my adventure, I sold my car and bummed a ride to the Bay Area. I spent two days on the Berkeley public library computers looking for a place to live. I convinced a retired dentist to give me the keys to a cheap studio in El Cerrito before he’d seen any money or even gotten my signature on a lease. Looking for my apprenticeship, I started applying to startup jobs.

It turned out that 21-year-olds with no degree or marketable skills weren’t in great demand. As luck would have it, a paper I picked up told me it was the worst summer for jobs in a decade. Eventually my cash ran so low that I walked to downtown Berkeley and went into every business with an open door asking for job applications. Even there the response was abysmal. A week later I had exactly one offer. It was from Ben & Jerry’s, which I accepted.

A few days later I saw a job opening on Hacker News for a Community Manager at Justin.tv. I remembered reading about the founders when they sold their last company on eBay, so I excitedly emailed them and soon got an invitation to interview.

I spent the train ride across the Bay figuring out how I’d stand out from other applicants. When I got there I pulled out my notebook and made sure to interview them as rigorously as they were interviewing me. I even said “Just to be fair, I should mention I’m going to quit in a year to start my own company.”

Over three interviews, the candidates were whittled down from sixteen, to six, to me and one other. At the end of the third interview, Justin Kan asked, “how much salary do you need?” I told him ”I’m only in it for the experience; I only need enough to live. 

I got the call that night. “It was close, but we want someone who’d be around for more than a year.” I thought I’d shot myself in the foot until he added, “but we want to create a new position for you.” I gave my notice to Ben & Jerry’s.

The job was a grab-bag of unwanted tasks. I tested new releases of the site, managed job applicants, and even ordered lunch for the team. The work was tedious, but getting hired at Justin.tv was the best thing that could have happened. I got to work next to the most capable people I’d ever met, and I learned how startups really work and met dozens of people who’d provide invaluable help and advice later. Two of the founders would end up as investors and advisors in my company.

One of the best things about the startup community is the incredible pay-it-forward attitude. It’s almost like the bond between war veterans. The individual circumstances may vary, but every founder can empathize with the struggle.

After a year and a half at Justin.tv, I still didn’t have a startup idea, so I gave myself a deadline: I was quitting my job in six months, so I better have something paying for rent and ramen by then. I added a countdown timer to my desktop and changed my work password to “SixMonthsQuit,” something I’d be typing eight times a day. (You clear your cookies often doing site testing.)

THE PROMISE OF LUNCHTIME STRESS

I had lots of conversations with friends about potential ideas. Around that time Justin mentioned that a friend at another company had asked for the list of restaurants I was using for lunches. Maybe I could help them? At the time, ordering lunch was, surprisingly, the most annoying, stressful part of my job. My restaurant list had taken a year of trial and error to build. I got a meeting with their office manager, and she seemed incredibly relieved to hand off the responsibility. Managing company meals turned out to be a hair-on-fire problem. I started asking for introductions to office managers from everyone I knew and one customer became two, then three. I quit my job three months after setting the deadline.

I remember being deathly embarrassed that my “tech” startup consisted of 0 lines of code, an inbox, and an enormous spreadsheet. Eventually, I got over my embarrassment and learned that the first rule of being a salesman is being unashamed of selling. If someone at a party mentioned that their company ordered food, I pulled out my phone and got their email address right there. When I received emails from sales people, I’d flip the script: had their company considered having food delivered? Could I talk to whoever would make that decision? I’ll listen to your pitch if you listen to mine.

Pitching became a reflex. When I was buying a phone, a Verizon employee saw my card and asked what I did. Not even expecting a sale, I started pitching by reflex, “ZeroCater makes it simple for companies to feed their people. You can just tell us, for example: We have 74 people, two vegetarians, and 1 vegan, and want lunch every Monday, Wednesday, and Friday. We create custom orders across a curated list of restaurants, food trucks, and caterers. Awesome food just shows up and you never have to think about it again.” When I got home, I found that all the Verizon stores in San Francisco had signed up to feed their employees during the new iPhone launch.

The beginning was the hardest, when only a few customers stood between me and not making rent, but a year went by and the company grew. The spreadsheet I used for scheduling grew to five hundred columns and the processes I had in place started to break. Billing grew to take 20 hours a week. I started looking for a technical cofounder to build software. Unfortunately, most of the developers I knew already worked at promising startups.

Eventually, I convinced a developer friend to join me. We applied to Y Combinator and got an interview. Luckily, Paul Graham loved the idea. “You help companies with their number one problem: recruiting!” He then turned to Jessica Livingston and asked, “How do you like managing catering for Y Combinator events?” “I hate it!” Despite this, we were still too nervous to get any work done until we got the acceptance call that night.

Near the end of Y Combinator, an investor heard about us through someone we were trying to recruit and asked for a meeting. We spoke over coffee and he emailed us the next day: “Good news- we discussed ZeroCater as a team and we’d like to move ahead with a term sheet. [...] We can do a convertible note or a “bridge” equity round (assuming you’re not ready for Series A), and we’d also like to know if you’re willing to go up to $1mm.”

The view had been breathtaking but it was at that moment that my roller coaster reached the top of the tracks. The next day my technical cofounder told me he was quitting.

It had been a stressful three months and we were often leaving the office past midnight. I had been talking about building a billion dollar business, and I think that’s when he realized he didn’t love the company enough to give it 100% of the next 4-8 years of his life.

There are two things you don’t want to be when you’re trying to raise money as a tech company. You don’t want to be a single founder, because everyone knows startups are too much work for one person, and you don’t want to be non-technical, because everyone knows that a business guy with an idea has no inherent value.

I was both. It didn’t matter. I was going to build a company.

Demo Day was approaching, where I’d pitch my company to a room full of investors.

After I’d tried several rewrites, two days before demo day, Paul Graham suggested the ending of my presentation: “more and more companies are feeding their employees, and we want to ride that wave, a wave of food.” Looking for a way to wake investors up from the 44 other presentations they’d be sitting through, I found an artist who could create a painting overnight that would illustrate the “wave of food.”

My presentation ended with “We want to ride that wave,” where I’d pause, flip to the painting, and say “a wave of food.” This got a lot of laughs.

I must have practiced that presentation over 150 times, right up to the moment I went onstage, much of it in the Y Combinator parking lot, where a friend came up to me and said “Arram, you know there’s such a thing as over practicing.”

There’s a great quote from Teller (of Penn & Teller): “Sometimes magic is just someone spending more time on something than anyone else might reasonably expect.”

I presented, and ZeroCater was voted #1 at Alumni Demo Day.

A few investors balked at the idea of investing in a single, non-technical founder but fortunately most were enthusiastic enough that I raised $1.5 million – more money than I’d planned.

Over the next two years we built the team and kept on growing. We now feed over 350 companies each month, including Sony, ZipCar, Yelp, and eBay and have served well over a million meals.

Team ZeroCater on a boat.

Paul Graham has written that one of the surprising things they learned after starting Y Combinator is that determination is the most important factor in predicting success, even more important than intelligence.

I think of Bill Walsh, coaching the 49ers, with the worst record in NFL history, trying and failing to break an eight-game losing streak against the Miami Dolphins. I think of him sobbing in the dark on the flight back to San Francisco, while his assistant coaches shielded him from being seen by any of his players, and nearly submitting his resignation before changing his mind and going on to win three Super Bowls.

I think of Buckminster Fuller, broke, seeing his daughter die of polio, and on the verge of suicide, before deciding to embark on “an experiment, to find what a single individual could contribute to changing the world and benefiting all humanity.”

I think of Tony Hsieh selling LinkExchange to Microsoft for $265M, eventually investing in, then joining a struggling shoe company called Zappos. I think of him investing more and more of his own money until he went broke and sold his apartment at a loss just to make payroll before being acquired by Amazon for nearly $1B.

I think of Elon Musk, on his third failed rocket launch, after putting his entire fortune into his three companies, having to borrow money to pay rent, with just enough cash for one more launch before successfully reaching space and closing a deal with NASA.

Musk describes it best: “It’s like chewing glass and staring into the abyss.” In the worst times, the pain is both physical and mental. There’s a part of his 60 Minutes interview I watch at least once a month, where Musk is asked if he considered quitting after his third failed launch. He replies in a quiet, understated voice: “Never. I don’t ever give up. I’d have to be dead or completely incapacitated.”

There’s a reason Musk is on the very short list of people who’ve founded three separate companies worth over a billion dollars, and that moment captures it perfectly.

You determine the greatness of your accomplishments by the amount of pain you’re willing to pay down. The more ambitious you are, the more glass you’ll have to chew. Everyone has their grind, even people doing what they love.

Anyone, given health and a reasonable amount of intelligence, can accomplish goals on a global scale. I really don’t think there’s an upper limit. That’s what my five-year experiment has confirmed for me: You can do anything with enough grit.

I’ll leave you with a quote from Marc Randolph, the Founder & CEO of Netflix:

I keep saying how brutally hard this is. Each time you crest the rise in front of you, it just makes it clear the size of the even larger hill that looms beyond it. It goes on for a long time. I pissed blood for years keeping Netflix alive while we figured that shit out – as did every other successful entrepreneur in the valley.

Source

  http://techcrunch.com/2013/04/06/how-i-started-zerocater/?query=4664

[startupmap]

Why Startups Should Choose Canada Over Silicon Valley

Recently, there’s been a lot made about the advantages for starting technology companies in Canada, as well as the optimism for the Canadian Startup Visa.

But to me, it’s always felt slightly disingenuine and self-promotional. Canadians trying desperately to convince themselves that they’ve arrived on the tech startup map. In a global market, would anyone with experience, connections, and residence in Silicon Valley, actually choose to start a company in Canada?

Yes. I did.

I’m a Canadian citizen who sold my last startup Attassa in 2010 to Silicon Valley’s Yousendit. I also led mobile Product at Zecter, a Y-Combinator company that was later sold to Motorola/Google.

In the last five years in the Bay area, I’ve built a healthy Rolodex of Silicon valley connections. But in January 2013, I moved back and incorporated my new company Zenlike.me in Canada; and not because of SR&ED, not because of IRAP, not because of soaring engineering costs in Silicon Valley, not even because of the US’s luddite immigration policy which makes it difficult for foreigners to start companies in the US.

I did it largely because of the incestuous bubble of false positives that pervades Silicon Valley.

The valley is an amazing place, and in a lot of ways, I’d say there’s no better place to start a company. If it weren’t for the fact that I have a strong network there, allowing me to leverage what the valley is amazing at (capital, marketing, business development), I might not feel as strongly about starting in Canada.

But for all the positives of the Bay Area, there’s one downside that few talk about which can kill startups: false positives. False positives lead to premature scaling. And premature scaling leads to startup’s death.

It’s well known that startups, new products, and taking risks are all deeply ingrained in the Bay Area culture. But what’s not talked about is the downside to this.

Ideas often succeed there—but nowhere else. In the Bay Area, investors, friends, and early adopters are so embracing and supportive of new ideas that startups get funded, apps get downloaded, and ideas get thumbs up, even if they won’t scale.

All too often these startups hit a wall when they try to scale, after having spent way too long on a vision that doesn’t scale. I love Zaarly, believe in the team, and vision, but I can’t help but wonder if they would have changed directions earlier had they been based out of Vancouver or Edmonton, where they could have better validated their business, instead of being based out of San Francisco, and raising money on (literally) day one from Ashton Kutcher.

It’s not all upside for Canadian startups. People are less risk averse so recruiting is hard.  But it’s hard anywhere. I’d argue it’s harder in the Bay area, where instead of competing with RBC, Atco, and Telus, you’re competing against Pinterest, Facebook, and Twitter.

Fundraising is possibly a bit harder in Canada.  But Canadian firms I’ve interacted with—namely iNovia,BDC, Version One, and Real Ventures—are great. Furthermore, a 2010 change in tax laws has essentially removed all the barriers for foreign investment in Canada.

SEE ALSO: Canada Officially Makes Call to Foreign Startups for New Startup Visa Program

The real challenge, in my opinion, is to constantly be striving for a massive vision in an environment where few peers are doing the same.  In the valley, you’re crazy if you’re not shooting for the moon. In Canada, you’re crazy if you are.  It’s easy to settle for status quo in Canada.

At the end of the day, my advice to Canadian startups is to stay in Canada, but build connections and spend real time in the valley focusing on business development and marketing. And finally to surround yourself with other local startups shooting for the moon.

Source

http://www.techvibes.com/blog/choosing-canada-over-silicon-valley-2013-04-08

[startupmap]

Pricing Rules You Need to Know

Customers are irrational about prices. How can you make the most of their weirdness? This roundup of pricing rules of thumb based on research can help.

Martin Diebel/Getty

Pricing is notoriously hard. Simply asking your customers what they would pay sounds sensible, but often doesn't work. People frequently don't know what things are worth, and their stated preferences frequently don't match their real-world actions. But simply taking your best guess and slapping a price tag on something usually means you're leaving money on the table.

So what's an entrepreneur with a product to price to do?

One answer: turn to ConversionXL for advice. The site recently rounded up a wealth of tips on pricing all grounded in science to help business owners decide what to charge for their products.

The list reveals, first, that people are strange. Given that money is obviously important to our quality of life; you'd think people would generally make sensible decisions around what they choose to buy and how much they pay for it. Nope, notes ConversionXL, "people are weird and irrational," but there are plenty of handy rules of thumb to help you deal with customers' odd behavior, the site notes. Here's a sample.

Use a Decoy

You'd think the key factor in deciding on whether to buy something is whether we want the iteam and it's worth the price, but in reality we often decide on purchases by comparing them to other options. Change those options and you'll change customers' preferences.

"When people were offered to choose a trip to Paris (option A) vs. a trip to Rome (option B), they had a hard time choosing. Both places were great, it was hard to compare them," explains the post, but when, "they were offered three choices instead of two: trip to Paris with free breakfast (option A), trip to Paris without breakfast (option A-), trip to Rome with free breakfast (option B). Now overwhelming majority chose option A, trip to Paris with free breakfast. The rationale is that it is easier to compare the two options for Paris than it is to compare Paris and Rome."

How to put this to use? Plant a decoy option that's clearly inferior: "If you add a slightly worse option that is similar to A (call it A-), then it’s easy to see that A is better than A-, hence many people choose that."

Show Them the Deal

Everyone likes a sale sticker. But they like a sale sticker that also lists the original price even more. "Researches found that sale price markers (with the old price mentioned) were more powerful than mere prices ending with the number nine," says the post. So a sign saying an item is currently $40 but was original $48 outsells a sign simply stating an item is on sale for $39. Even better, according to researchers, is a sign that lists the original price and has a current price ending in nine.

These are just a few of the principles explored in the long post, so if you found them interesting, check out the remainder here. Or, for more insights on pricing, learn the ABCs of pricing here on Inc.com.

Source

http://www.inc.com/jessica-stillman/pricing-is-hard-science-can-help.html

[startupmap]

What an Accelerator Program is Really Looking For

Some teams impress us by coming on strong. Some build their case slowly. But all the teams we invest in have certain things in common.

Six brand-spanking-new companies with very energetic and passionate founders started week one of our 12-week accelerator program on March 11. It’s a very exciting time for all of us in the Triangle Startup Factory, including our fabulous mentors, who volunteer their time to work directly with the founders. To celebrate our investment in their companies and to introduce them to the community, we hold an open house. It is one of my favorite events. Kind of like a debutante ball. Everyone smiles.

At our open house I was asked why these companies, out of the 150 that applied, made it through. It is a question we get all the time. There are really two different answers, reflecting two kinds of relationships.

Most of our founders are strangers to us until they make their application. Then, in a matter of weeks, we run them through our process and methodology, some of which I’ve written about previously. There are three essential steps:

Screen for individual and team entrepreneurial DNA

Screen for idea and market

Screen for our ability to accelerate their growth

Each of the three steps serves as a gate with pass/fail implications. We are getting better and better at identifying key characteristics at each gate, and so we are getting happier and happier with the entrepreneurs and concepts we commit to.

The other way of getting into TSF is by developing what we call a drip, drip, drip relationship. Sometimes I really like this slower, more methodical way of getting to know a company, its people, and its product.

Last fall, a group from Washington, D.C. drove down to Durham to attend one of our applicant open house events. They followed up a few months later to schedule some time during our open office hours. Some emails went back and forth. Some time after that, they submitted a formal application.

Their product is a digital comment card, focused for now on restaurants. What, other than the fact that the market reaction to the product has been extraordinary, made us want to get into business with them?

They asked great questions and listened to a few nuggets of advice,

They made some product adjustments over that time,

They acquired a minimum number of early customers and tested their application over a couple thousand users. (Yes, a couple thousand.)

In this time period they revealed to us that they are smart, passionate, willing to adjust their model, and able to create and react to data--all of the things we want in a Triangle Startup Factor team.

Source

http://www.inc.com/chris-heivly/what-an-accelerator-is-really-looking-for.html

[startupmap]

College Startups: The 'New Master's Degree'

As a student at University of Indiana’s Kelley School of Business, Derek Pacqué lost his coat at a bar, got angry, and came up with a business plan. He borrowed and saved $500 to purchase racks and hangers to start a coat check business at local hangouts.

CoatChex does not require patrons to keep tickets, which often get lost. Instead, someone at a kiosk photographs clients’ faces and coats with an iPad or smartphone and then uses their phone number and photos for secure pick-up. A paltry original investment eventually had Pacqué negotiating with—and turning down—a $200,000 offer from entrepreneur Mark Cuban on ABC’s Shark Tank for a 33 percent stake in the business. In the last two months, CoatChex earned $100,000.

“You go to school to get a job or an education,” says Pacqué, who graduated in 2011. “I went to college because I wanted to create my own career, to create something of value.”

Pacqué is among a new breed of undergraduate business students. Professors and classmates say they hunger to be their own bosses. More undergraduate business students than ever before are launching startups right after graduation—or sometimes while still at school, say administrators. A query to the top 20 undergraduate business schools asking for contacts with promising startups launched by students, or by very recent graduates, resulted in at least 100 responses.

The popularity of entrepreneurship courses is skyrocketing. For example, at New York University’sStern School of Business, the number of undergraduate entrepreneurship courses has increased from two in 2003-2004, with 62 students enrolled, to seven courses, with 281 students enrolled, in 2012-2013. The school also introduced its academic track in entrepreneurship in 2012.

No doubt the recession has driven undergraduates to embrace entrepreneurship. Charles Sinn, president and co-founder of Guys and Dollies, a storage business for college students in Indiana, says he wasn’t sure whether he should work to grow his startup or get a traditional job when he was about to graduate from the Kelley School in 2009.

“I saw classmates in investment banking who made a lot of money for a year; then they lost their jobs and had no resources,” Sinn says. “It helped me decide to stick with Guys and Dollies moving forward.” Today the company offers summer storage and moving help to more than 1,000 students at Indiana and Purdue universities; it is “income producing,” he says.

A desire to make a difference in the world is also a contributing factor, says Scott Stimpfel, assistant dean of student engagement and innovation at Stern. In fact, many of his students are pursuing social entrepreneurship, or at least getting exposed to it, as early as high school, he says.

Jeffrey White, who graduated from Boston College’sCarroll School of Management in 2012, launched Rightside Shirts, a nonprofit T-shirt company. Kids in public schools in the Boston area create T-shirt designs; Rightside, of which White is chief creativity officer, chooses one design from each school, puts it on shirts, and sells them to the community to fund art programs in the schools. White is not able to support himself with Rightside Shirts alone, and he waits tables twice a week to pay the bills. But the company has taken in $10,000 in revenue so far, he says.

Technology has made entrepreneurship more convenient, especially for a generation that has never lived without the Internet and its offerings. “We’re living in a time when it has never been so easy or cheap to start a business,” says White. “The barriers are so low, thanks to social media. If you have a good idea, you can gain momentum.”

The chances that a new company can withstand the test of time are daunting. The U.S. Bureau of Labor Statistics says only half of all new businesses survive for five years, and only a third make it to the 10-year mark—a fact about which many undergrads seem unaware. Many an undergraduate business student wants to be the next Mark Zuckerberg. Ted Zoller, director of entrepreneurship at UNC’s Kenan-Flagler Business School calls it the “Zuck Distortion.”

“He’s created a perception that anyone with a good idea can take off after school and become a billionaire,” says Zoller. “I wish that were true.” While Zoller encourages his students to follow their dreams and give their ideas a decent shot, he tempers their expectations.

Source

http://www.businessweek.com/articles/2013-03-26/college-startups-the-new-masters-degree

[startupmap]

Why So Many Jobs Are Crappy

I've been thinking about it. Work, being a core part of life, is meant to be interesting, engaging, and meaningful. Otherwise, why are we wasting our time on this planet? Yet, for many, work is not living up to its name. Work of the good kind is less and less on offer in the jobs being created. I've been reflecting on possible reasons why, and decided it's really simple. The problem is not the jobs. It's us. Most humans are simply not the kind of people a boss would want to hire.

Take yourselves as a case in point. I'm guessing you’re the kind of people who'd prefer to feel needed rather than expendable. Well, that kind of attitude won't do. Bosses want to keep your wages down, and that would be harder to do if you were given opportunities to make yourselves invaluable and near on irreplaceable. Bosses need to keep their options open in case some of you get ideas about better pay and conditions, or just generally become 'difficult' or, dare I say, 'bolshie'.

You know it's true. A boss needs to be able to dump you at the drop of a hat. Maybe it's to boost profits. Maybe it's to cut costs. Or maybe it's just because it feels good.

And a boss needs to be able to dump you without it having detrimental effects. There must be ready replacements, eager to crank it up, the moment you're out the door. And if morale suffers, because the buddies you left behind miss you, the boss will want to send them packing too, and bring in a fresh batch of wage-slaves.

Quite simply, there is little place for satisfying roles, the kind where you get to learn more and more interesting stuff over time. The only good on-the-job learning is no learning at all. Or, if you must learn, thirty minutes tops to master a dead-end role.

Although the point seems obvious, I don't believe it has been treated with the kind of gravity that only the economics blogosphere is fully equipped to deliver. It's high time the situation was spelled out in painstaking – okay, not painstaking – analytical terms. Then we can all lower our expectations and knuckle down to a lifetime of short-lived McJobs and frequent sackings. If we're lucky.

One way to characterize a job is by the learning that occurs in it. This can be described by a learning curve:

In the diagram, u(t) stands for the unit labor cost that a worker – let's say you – achieves at a given point in time, after you have built up an amount, t, of experience. It is how much you cost the employer per unit of output produced, at a given moment in time. We can call this the 'instantaneous unit labor cost', or sometimes just 'efficiency' for short. If learning occurs on the job, you get more and more efficient, and your unit labor cost falls over time. The curve is drawn assuming a particular wage level. If the wage increases, the curve will shift up.

On first being hired, you were green, and cost the bosses m + c per unit of output. Eventually, through learning on the job, you will get this down almost to m, which is your 'potential efficiency', given current pay and conditions. Some jobs will allow more learning than others, which will be reflected in the amount c, which is the 'scope for learning'.

Here is one possible algebraic representation of the learning curve:

The second term is the one that captures the learning process. When you just start the job, t = 0, and the second term equals c. Just as shown in the diagram, you cost the bosses m + c per unit of output. The longer you stay in the job, the larger t gets, and the closer the second term gets to zero, which it approaches asymptotically. Your unit labor cost converges on m, as indicated in the diagram.

The rate at which you reduce your unit labor cost from m + c down to m depends on how fast learning can take place on the job. The 'rate of learning' is represented by λ. When &#955 is large, the learning curve will be very steep initially, and almost all the learning will occur just after being hired. When &#955 takes intermediate values, the learning process is steadier and longer lasting, reflected in a more gradual curvature in u(t).

McJobs are those where learning is either nonexistent or extremely rapid but short-lived. If there is no learning, c = &#955 = 0 and the learning curve would just be a horizontal straight line showing a constant unit labor cost of m. Rapid but short-lived learning would mean the learning curve slopes down almost vertically until m is nearly reached, then stays almost flat after that. It would probably take most of us a half shift to master flipping burgers, but after that we'd have it down pat. Bosses love McJobs. They make us readily replaceable.

Satisfying jobs – let's call them 'good jobs' – will generally be ones where learning occurs at a steady pace more or less indefinitely, probably as part of a defined career path. Bosses would prefer not to offer these, and will always be looking for ways to deskill roles that, for now at least, need to allow workers greater autonomy, ingenuity, and scope for on-the-job learning.

Once you gain experience in a good job, you will soon become much more efficient in the role than an inexperienced replacement would be. This might remain true even if you happen to win a pay rise, work less hours, or start operating at a more leisurely – let's say human – pace. Any of these things would shift your learning curve up, because you would now have a higher unit labor cost at any given level of experience. Even so, you might still be more efficient than a prospective replacement.

In fact, let's say you do win a pay rise, plus a longer lunch break. Thanks to your experience on the job, you have realized that you can afford to be more bolshie. The boss knows that you can't be replaced without some cost, at least in the short term.

Your new situation is illustrated below. You have switched from your original learning curve, now called u2, to a higher learning curve, u1, after an amount τ of experience.

Incidentally, that greek letter tau is meant to look the same as the greek letter tau in the diagram. I have no idea why it doesn't match. Probably some bolshie worker at Microsoft is the culprit. Typical.

Anyway, you are sitting at point A and the boss is not happy, since before your improvements in pay and conditions, you were on a point almost directly below it on curve u2. Extensions to the boss's country house are in jeopardy thanks to your militant demands for higher pay and a full half-hour for lunch.

As it happens, your immediate boss is sufficiently annoyed to call in his bosses. They're having a liquid lunch for the express purpose of dealing with the situation you've placed them in. They need to decide whether to kick your sorry butt out of the joint and find a replacement, somebody who will accept a lower wage and be subservient, like you used to be before you went all bolshie. The bosses want someone who is willing to be positioned on learning curve u2. Unfortunately, such a worker would have no experience, so would be back at point E at time zero.

Algebraically, the two curves look like this:

The diagram shows that m2 < m1. This means that a replacement worker's potential efficiency would be superior to yours. It's also possible that a replacement worker would learn a bit faster, now that you are insisting on operating at less than sweatshop pace. This is reflected in the equation for u2, where the rate of learning is vλ rather than λ, and v is assumed to be greater than 1.

Right at this moment, with the bosses plotting your downfall, you happen to have a lower instantaneous unit labor cost (at point A) than a replacement would have on first starting the job (at point E). But the bosses wonder whether, given enough time, a replacement would save the firm money and get the country house extensions back on track.

How patient are the bosses? In the diagram, they have a 'time horizon' of h. If a replacement would cost less over time period h than you would, the bosses will gleefully liberate themselves from your services. Given that amount of time, the replacement's experience on the job would increase from 0 to h. You've had a 'head start' of τ, so you would be able to build up your experience from τ to τ + h.

The bosses can see that the replacement would get down to point F after time h on the job, which represents a lower unit labor cost than you would offer them at point B. However, for much of the period, you would outperform the replacement. What matters to the bosses is the total accumulated costs over the entire period. In the diagram, if area ABCD is larger than area EFGH, you will be on your bike. Otherwise, you'll be safe, even though the bosses are itching to see the back of you.

It is self-evident that you are more likely to be fired the bigger the gaps in potential efficiency and scope for learning (the m's and c's), the larger the difference in learning rates (determined by v), and the longer the time horizon, h. On the other hand, the more experience, τ, you already have under your belt, the harder it will be for the bosses to get rid of you.

More interesting, though, is the influence of learning speed on the decision of the bosses. As indicated at the outset, bosses don't like jobs that provide an opportunity for indefinite learning at an intermediate rate.

We can see this by looking more closely at the areas ABCD and EFGH. Call the first area C1. It represents your accumulated unit labor costs from time τ to τ + h. Similarly, the second area can be called C2, and refers to the corresponding measure for a replacement worker employed from time zero to h.

Algebraically,

Let's define the 'firing payoff', Z, as:

This measures the cost savings the bosses stand to reap if you are fired and replaced by somebody else. If Z > 0, the firing payoff is positive and the bosses will replace you. Otherwise, you win out, even though you've become intolerably bolshie!

The behavior of the firing payoff, Z, as the rate of learning, λ, is varied is always the same. For very low and high rates of learning, it is more likely that firing will pay off for the bosses. But for intermediate rates of learning, there is a greater chance that you will survive. What qualifies as 'intermediate' values for the rate of learning varies with the choices for other parameters, but the pattern is always the same.

For λ < 0.6779, Z > 0, and you are fired. For λ > 4.3889, again Z > 0, and you are hanging out in soup kitchens. But there is a sweet spot. You get to stay in the job with your better pay and conditions in between those two cutoff points.

Fortunately, the bosses weren't the only ones able to zero in on the behavior of the firing payoff. Unbeknownst to them, you had already made the appropriate calculations before formulating your most recent demands for a pay rise and longer lunch break. You had the bosses right where you wanted them. They were screwed from the outset, and didn't even realize it. No wonder they would prefer to eradicate all the good jobs and leave us all fighting for scraps in McDonalds.

Extensions

I tried to keep the model as simple as possible without losing the key relationship between the firing payoff and the rate of learning. Various extensions are possible, but they only make the situation worse for the bosses when it comes to good jobs.

One obvious extension, though not very interesting, is simply to introduce a discount rate. If the bosses value cost reductions in the present more highly than cost reductions in the future, it is appropriate to include a positive discount rate. This doesn't alter anything much other than to make being bolshie even more likely to pay off.

Another extension is to take into account that the traits of the replacement worker are not completely known. The bosses will worry that a replacement will become bolshie, just like you did, before your removal has a chance to pay dividends. As with a positive discount rate, incomplete information concerning the traits of a replacement worker stack the odds more in favor of the incumbent.

The incompleteness of information adds an additional wrinkle, which is that now even large gaps between potential efficiencies will not necessarily hurt you as the incumbent. The reason is that there will now be a chance that the replacement will simply approach the same unit labor cost as you. Basically, for a big gap in potential efficiencies to matter, the probability of the replacement becoming bolshie can't be too high.

So, next time we hear people complaining about the crappiness of their jobs, we can sooth them with the knowledge that it is better for the bosses that we all remain unchallenged and expendable.

Background Reading

The idea for this post comes, in a roundabout way, from a 1981 paper by A. M. Spence entitled, 'The learning curve and competition', which was published in the Bell Journal of Economics. In the paper, Spence argues that early entrants into an industry will be more likely ultimately to dominate the market when production is subject to a learning curve.

The idea is also linked, again in a roundabout way, to a 1994 paper by Flora Gill entitled, 'Inequality and the Wheel of Fortune: Systemic Causes of Economic Deprivation', and published in Australian Economic Papers.

A different application of the argument presented here would be to consider the circumstances under which instances of bad luck in competition for employment could have long-lasting effects on the earnings of individual workers. Workers who luckily gain appointment into good jobs, perhaps initially contrary to merit, will get the opportunity to learn on the job and lock in their advantage, whereas unlucky workers consigned to bad jobs, again perhaps contrary to merit, might have a tough time reversing their fortunes.

I like the present application, though, in which capitalist deskilling emerges partly as a response to the effects on worker discipline that are potentially created by on-the-job learning.

Source

http://heteconomist.com/why-so-many-jobs-are-crappy/

[startupmap]

Your Startup’s 10 Most Important Metrics

With the analytics tools today, it's easy to measure hundreds if not thousands of different metrics for your business. Cutting through all the chaff to determine the most important or insightful metrics can be quite a challenge.

Below are the ten metrics I've found to be most useful in board meetings. They answer the questions of how should a startup founder might measuring the business at the highest level. You should have many more metrics than these, but I've highlighted the ones that I recommend presenting to your board and reviewing each week.

Metrics Format

Clear data leads to productive conversations. To best understand a data point and its implications, we have to put it in context.

I've found dividing top level data in three slides, one for company priority (Distribution, Engagement, Revenue) helps to set the right context. Within the slide, a table that shows the metric and compares it to last month, then explicitly calculates the monthly change, the trailing six month average and finally compares the metric to the goal best communicates the state of that metric. 

The TSM Average column is the Trailing Six Month Average. It is calculated in this way:

(ending_value/starting_value)^(1/num_periods-1)-1.

In most businesses, a monthly growth percent is too volatile to be meaningful. However a TSM Average smooths out the monthly average into a rolling average. Comparing the monthly to the TSM, we can get a sense of whether the monthly growth is accelerating or decelerating and how it compares to the goal you set each quarter. In this example, the total user growth was slower this month than in the past six month, but activity is way up. The next question, the one board members and founders should ask, is why?

Metrics/Question Format

Now that we have the base format of the metrics, let's talk about which metrics matter. Each metric is followed by the question it's designed to answer. Pick the ones that are relevant to your business.

Distribution

New users added last month by channel/TSM growth rate: How are well are we growing the user base? Which user sources are the best?

Total user base/TSM growth rate: How important is our monthly growth compared to our total user base?

Cost of customer acquisition, lifetime value, pay back period: Can we grow faster through paid acquisition? Are we acquiring customers profitably? How much can we afford to spend on new customers? How is this changing over time?

Engagement

Active users (can defined in several different ways depending on your product) by channel/TSM growth rate: Are we getting better at giving our customers what they want/need? Which channels of users are most effective in finding us the right kind of user?

% of users using top 3 key features in a given month: Are our product initiatives the right ones?

Revenue

Revenue / TSM Revenue growth: Are we growing our revenue?

Conversion to paid rate in that month/by cohort: How many users converted to paid? Are we improving our ability to convert customers to paid?

Avg spend per paying customer of a managed account vs solo account: What is the impact of the account management team?

Churn rate/ TSM Churn rate: How well do we retain our customers?

Burn rate: When are we profitable? When do we run out of cash? When do we need to raise?

These are the metrics that have been most valuable/insightful for me working with our companies. Let me know if there are other metrics you use to measure your business that might be useful to add to this list on this Branch.

Source

http://tomtunguz.com/your-startups-10-most-important-metrics/

[startupmap]

An idea for non-technical co-founders: try a service-first business

Over the past several months I have been having a lot of conversations with non-technical folks who want to start, or are starting a company. I’m drawn to these conversations for the difference in perspective. I’m technical and I admittedly tend to have a build-first mindset – build out neat ideas first, ask questions later. But I’ve noticed an interesting pattern:

Non-technical folks often also have a build-first mindset. They want to build neat stuff and then go out and get users despite their skillset being on the ‘hustling’ side of the spectrum.

They have an idea, maybe some sketches or wireframes, but are held up by the lack of technical person able to build their vision.They just need a programmer. In this situation, the programmer will sadly think why do I need this person?

So I have an idea: don’t find a technical co-founder (yet) and don’t learn to code. Use your current skills and try your hand at a service-first business.

***

I graduated from YCombinator a few months ago ( S12 Easel ). One of the most interesting parts of experience was being very close to my fellow batchmates. I was able to see in real-time and almost first-hand all the little struggles and successes. I got to see a huge variety of different skill-type distributions – some companies with all tech founders, some with mostly non-tech founders – attack their problem spaces in many different ways.

As a technical guy, watching the companies who were solving non-technical problems was particularly eye opening. I developed an extreme appreciation and respect for these hustling skills. To give an overview, their approach was generally this: build almost nothing at first (MVP!), pound the pavement and get customers to pay right away for a service they do manually, then build things to make the manual pains go away as they get more customers. It even worked for one technical team – an enterprise API. When called, the API would just email the founders, they would do the work manually, then asynchronously return the result.

This approach has wormed its way into my brain and I now think about startups in one of two categories: tech-first and service-first.

Definition

Tech-first products are those that need to be built to show they work. Google is a tech first product, as are games, productivity apps, etc. The point with tech-first products is that you can’t get users for a new product until it is built because it depends so much on how the technical piece works. You just can’t fake a new search engine or a file syncing utility. Marc Andreessen defines this as technology or product risk. Mitigating this product risk requires a technical person.

Service-first products depend on a service or community at their core. Groupon, Airbnb, Exec, and TaskRabbit are obvious examples. These companies needed almost no technology at first – they inherently have very low technology and product risk. They just hustled their way into town. Maybe your idea is a marketplace that connects widget makers to people who buy widgets? At first, relationships can be built with widget makers and buyers can be found without tech. The tech is applied later to help scale. Service-first products might be a subset of Ben Parr’s traction startups.

Explore only service-first products

In almost all cases of the non-technical folks I’ve spoken with, their ideas consist of tech-first ideas. Don’t do it!

Use your strengths! And find an idea that lets you use them immediately.

Explore only ideas that require no more tech than maybe a wordpress blog or simple website. Don’t think about the tech, think about how to make a service or community work in a tiny way. Provide your service 100% manually until you just can’t do it without more people or tech.

As a non-technical founder, you are likely good at building and maintaining relationships. You are probably good at selling things. You might be great at framing, so people buy into your vision or get excited about your ideas. These are amazing, difficult-to-acquire skills. They are skills you don’t get to use right away when building, say, a photo sharing app. But you can use them right nowwhen building, say, a new kind of marketplace.

If you are able to show that your service or community is working in a tiny way without having built much, you will have orders of magnitude more ammo when trying to find a technical co-founder. Likely with investors as well.

In practice, when I was attending meetups, I did not meet a single non-technical person who had done any of this leg work. I would have taken them and their idea very seriously if they had.

Finding ideas

How do you go about finding these ideas? It seems many of them are marketplaces. Do you know of some latent supply somewhere? Do you have a novel way of rounding it up? Can you connect it to buyers? Can you take something that’s generally an expensive luxury item to the everyman (think stylist, personal assistant, etc)?

Do you have a difficult to acquire skill? For example, businesses in all verticals are terrible at marketing, maybe you can help them? Maybe in only a specific vertical?

What kind of people do you know? A lot of lawyers? Engineers? Construction workers? Maybe they need something you can provide?

I have been lucky enough to watch a number of my YC batchmates do an amazing job building service-first businesses. I cannot write about all of them, so I have chosen just two and I will explain through their their example.

Vayable

Vayable connects tourists with locals who will provide experiences. Think: AirBnB for travel activities. Vayable’s Jamie Wong wanted to go anywhere in the world and be able to have someone local show her all the interesting nooks.

For months, Vayable’s tech piece consisted of only a wordpress blog. She started locally in San Francisco and her first step was finding tour guides. All of the work was in pounding the local pavement. She hit up friends, friends of friends, and travel bloggers to be guides in SF. She even hosted meetups to find interesting people.

With some tour guides on hand, she set out to find tourists. Her initial batch of tourists came from a genius move: she hosted Airbnb guests, figured out what they wanted to see, and set up tours for them.

For example, one of the guests wanted to go on a startup tour of SF. Jamie knew startup people and had recently been introduced to three guys who used to work at as professional tour guides. They agreed to the startup tour. Then she hustled to find more folks for the tour: she made flyers, went to startup meetups, and even got some of her local friends in the mix.

Jamie found a latent supply. Turns out people know their cities really well, and are really eager to show them off – especially for some spare cash. And tourists love to be off the beaten path, experiencing local flavor and getting a personal touch. She was able to find these things out and get people to pay her real money with no tech. Just a blog with a bit of info, a couple pictures of the guides, a few video testimonials, and a whole lot of hustling.

Tastemaker

Tastemaker is a service that connects interior decorators with homeowners. They’re bringing interior decorating services – a luxury item – to homeowners on a budget.

Tastemaker actually started out working on a slightly different product – a sort of shoppable pinterest for home furnishings. Their current product – a marketplace for decorating services – came out of a simple task: understanding how people bought items for their homes. After digging deeper into the problem space and talking to a lot of target customers, they found that people had a lot options for doing the shopping, but really needed help designing their space.

So Tastemaker started talking to interior decorators, and it turned out that many of them were interested in taking on smaller jobs online to fill the gaps in their schedule, but didn’t know how to efficiently find customers. This was an epiphany. Tastemaker figured they could become the matchmakers, facilitate the interaction, and bring decorators work.

They had the supply-side of the marketplace, but most homeowners currently don’t buy interior decorating services, so the demand-side of the marketplace needed to be proved.

They asked a subset of the decorators from their initial conversations to be the ‘inventory’. Three decorators agreed. At this point Tastemaker’s two technical co-founders started building a web app but had a lot of unanswered questions about how it should work. Instead of waiting for them to finish it, the two non-technical co-founders went out to find customers. There was no website, so they did everything with pen, paper, and email.

The first customers were found by mining their personal networks. Just asking around until they found someone who wanted a room decorated, then talking to them about their needs and connecting them to a decorator who might be a good aesthetic fit.

The initial jobs went really well. They made money and found out that an interior decorator marketplace was viable and could be a real business. And just like Vayable, they did it with no tech up front. Tastemaker found people with excess supply, connected them to buyers, handled the interaction, and did everything manually. In the process they learned how the customer/decorator interaction should work and this helped them develop a web app that really met both the customers’ and decorators’ needs.

Conclusion

I want you – the non-technical co-founder – to go after ideas where your skills are the star, where you can make a ton of progress without anyone technical. I absolutely believe the tech in my example companies is important to their business. But it is not the core; it is not the star. Without the service component, the tech is meaningless. The tech is applied as the business grows to remove manual pains and scale.

Your skills are extremely valuable. Vayable, Tastemaker, and numerous others have shown that it is possible to get a company off the ground by building next to nothing. So I urge you to run with what you got.

Source

http://benogle.com/2013/03/25/an-idea-for-non-technical-founders-service-first-business.html

[startupmap]

How Etsy Attracted 500 Percent More Female Engineers

Want cognitively diverse teams? It’s not as simple as hiring more female technologists. When Etsy stopped poaching and started training junior women to be rockstars, more senior engineers--men and women--saw the company’s progressive policies and started calling.

In Silicon Valley today it's not possible to hire more women simply by recruiting them. Good engineers today have their pick of jobs, and good female engineers are being stalked like the last antelope on the African veldt.

For Etsy, diversifying wasn’t just about good citizenship--it was vital to the product. Eighty percent of Etsy customers are female, but the company itself used to be known in startup circles as engineer-centric and something of a dude-fest: As of January 2011, the company only had three female engineers out of 47. Despairing, management gave up searching for senior female engineers and set about training junior prospects. Today, Etsy’s engineering team is 20 ladies to 90 guys, or 500% more women than two years ago.

Hiring female engineers is a chicken-and-egg problem

In 2011 the company decided to make recruiting women a stated core value for the year, but by December, they had only added one female engineer out of the 40 technologists they hired, driving the gender balance down 35%. "This is over a year when we were saying it's really important, we're working really hard on this," said Etsy CTO Kellan Elliott-McCrea, in a talk at First Round Capital’s annual CTO Summit. "Something wasn't working. This was deeply broken."

Etsy CTO Kellan Elliott-McCrea: 'Something wasn't working. This was deeply broken.'

Women hold just one quarter of engineering and computer-related jobs, and at many of the supposedly most innovative companies, the ratio is much much lower. There's lots of embryonic efforts to get girls into coding and entrepreneurship, like Change the Ratio,Rails Girls, and Black Girls Code, but many are too far up the pipeline to help a company that's hiring now.

Then there's a chicken-and-egg problem. How are you going to pull in senior women candidates when they look at your staff and see all the women in support roles? How do you make a female engineer feel welcome when almost every single coder is a dude?

"Great women engineers are not only NOT looking for work,” says Elliott-McCrea, but also, they're wary of being burned by the culture. If all they see is men, “there's a decent chance, based on their experience, that your workplace is going to suck."

Less headhunters, more hacker school

So instead of shopping for senior engineering talent to poach, the company risked an investment in training junior women with an eye toward hiring them. When the company changed its focus, it grew from just four female engineers to 20 in a single year, 2012.

The key, says Elliott-McCrea, was partnering with other companies to fund a training program that would attract candidates ready to learn. Etsy, together with 37Signals and Yammer, kicked in for $7,000 per student in grants to cover women's living expenses for a Hacker School session held at Etsy's offices in the summer of 2012. (For the uninitiated, Hacker School is a three-month intensive free coding training program in New York that trades on its culture of mutual respect.) Over 600 women applied, which Hacker School narrowed down to 23 attendees, or nearly half of the session for that semester.

Education researchers have shown both genders feel most comfortable when there's a balance. And the evidence was in students’ reactions. "I never realized the impact of being the only woman in the room until I wasn't," blogged Martha Kelly, one of the participants, about her first day at Hacker School. "Can you imagine STEM [Science, Technology, Engineering, and Math] being a diverse and supportive community building amazing things together? I can :)"

What senior women engineers want to see

At the end of the summer, Etsy hired Kelly and seven of her classmates from Hacker School, five of them women. While the company is taking a risk on some less experienced hires, it is already paying off elsewhere. After word spread in the engineering community about Etsy’s Hacker School grants, they attracted some very high-level candidates, men and women "whose names you would know," who weren't explicitly job hunting, but loved their initiative, leading to two senior-level hires and three more who are in talks.

As for Hacker School, they just announced a new round of grants for women--Dropbox, GitHub and PhotoShelter are joining Etsy to sponsor the 2013 class.

Source

  http://www.fastcolabs.com/3005681/how-hack-broken-gender-dynamics-workplace

[startupmap]

The Startup Trap

you have joined a new startup.

you are a multi-talented mega-being.

you can work 60, 70, 80 hours per week to get the job done.

you are a top-notch coder and designer.

you won't fall into the traps that others have fallen into.

you will make sure that this time will be different.

you are so good that the rules don't apply to you.

you are fucked.

The Start-Up Trap.

It's a sad story that we've seen over and over again. A young programmer begins with all the best intensions, learns all the right disciplines, develops all the right skills, and then falls prey to The Start-Up Trap. The Start-Up Trap is the idea that your situation is different -- that everything you've learned about how to do software well somehow doesn't apply to thisparticular job. You think it will apply later, once you've succeeded. But not now. Not yet. Not while you are in a race to succeed.

The Start-Up trap is the thought that the start-up phase is different; and that while you are in that phase success depends upon breaking the rules. This is stupid. The start-up phase is not different. The start-up phase is simply the first of many phases, and it sets the tone for all those other phases. Come back to that company in five years and (if they've managed to survive) they'll still have the same attitude towards the rules that they had in the first phase -- except, perhaps, for the overtime. (giggle).

Here's a little tip: The disciplines that lead to successful software are always valid, no matter what phase the company is in. It is laughable to think that good disciplines are less important during the start-up phase. The truth is that, during the start-up phase, those disciplines are just as critical as they are at any other time.

Of course one of the disciplines I'm talking about is TDD. Anybody who thinks they can go faster by not writing tests is smoking some pretty serious shit. Oh, I know you are a warrior-god. I know you can write the code perfectly every time. I know that the deadline looms and you just don't have time to write tests. -- I'm sorry for your impending failures. I'm sorry that you're going slow and just don't know it yet. And I'm very sorry that when you finally brute-force your way to some modicum of success that you will credit your bad behavior, and recommend it to others. God help us all, because you won't.

Ask yourself this: How does the accounting officer of a start-up behave? This person is responsible for managing the money of the investors. Do you think that accountant has deadlines? Do you think he's under pressure to deliver projections, forecasts, cash-flow reports, etc? Do you think his bosses tolerate schedule slips in his duties? I'll tell you now that the guy managing the investors' money is under a hell of a lot more pressure than any software developer is.

So how does this accountant behave? Does he double check his work? Does he practice double-entry bookkeeping? Does he follow all his rules and disciplines? Or are the rules different because he's in the start-up phase?

What if it was your company, and your money. What would you think of a start-up accountant who didn't check his sums; who neglected the debit side of the books and trusted the health and future of your company to the single unchecked sums of the credit side?

You'd fire his ass! You'd fire it so fast that the rest of his worthless carcass would be left outside the door wondering where his ass went!

Is your code somehow less important than that account's spreadsheets? Are errors in the code somehow more tolerable than errors in those spreadsheets? Can errors in the code take the company down and ruin it's reputation with it's customers, and investors? You know the answer to these questions. And you know this: If accountants can find a way to practice their disciplines in a start-up; so can you.

Is neglecting TDD going to help you go fast? To quote Captain Sulu when the Klingon power moon of Praxis exploded and a young Lieutenant asked whether they should notify Star-Fleet: "Are you kidding?" ARE YOU KIDDING?

NO, you aren't going to go fast. You're going to go slow. And the reasons are simple, and you already know them. You're going to go slow because you won't be able to refactor. The code will rot -- quickly. It will get harder and harder to manage. And you will slow down.

You won't notice it at first because it still feels fast. You are working hard and spending 60, 70, 80 hours per week on the code. The sheer effort you are applying is enormous; and that feels fast.

But effort and speed are not related. It is easy to expend a tremendous amount of effort and make no progress at all. Hell, it's easy to expend gargantuan effort and make negative progress. Effort equates neither to speed nor direction.

As time passes your estimates will grow. You'll find it harder and harder to add new features. You will find more and more bugs accumulating. You'll start to parse the bugs into critical and acceptable (as if any bug is acceptable!) You'll create modules that are so fragile you won't trust yourself, or anyone else, to modify them; so you'll work around them. You'll build a festering pile of code that, with every passing week, requires more and more effort just to keep running. Forward progress will slow and falter. It may even reverse as each release becomes buggier and buggier, and less and less stable. Catastrophes will become more and more common as errors, that should never have happened, create corruptions and damage that take huge traunches of time to repair.

You know the story. You know this is where others have wound up. If you are old enough, you have probably wound up there once or twice yourself. And yet that Start-Up Trap still sings it's siren song and lures you into destructive, slow, catastrophic behaviors.

If you want to go fast. If you want the best chance of making all your deadlines. If you want the best chance of success. Then I can give you no better advice than this: Follow your disciplines! Write your tests. Refactor your code. Keep things simple and clean. Do Not Rush! You hold the life-blood of your start-up in your hands. Don't be careless with it!

Remember: The only way to go fast, is to go well.

Source

http://blog.8thlight.com/uncle-bob/2013/03/05/TheStartUpTrap.html

[startupmap]

How To Hire Me a Programmer

Tech companies seem to be having trouble finding good technical talent. Maybe I can help. For you, the hiring person who is having trouble finding programming talent, I’ve created this brief hiring guide. It is based on my experience over the years both as a prospective employee and as a person doing the hiring.

Recruiters Who Can’t or Won’t Recruit

I probably ignore 95% of recruiter emails. Even for the 5% I consider, I’m still astounded at how the attitude is essentially “come apply for this job”. This is such an obvious non-starter that I’m surprised it persists.

Example: Not long ago I received an email from an in-house recruiter for a “hot” startup, saying essentially “Hey, we’re hiring bro, you interested?”. When I replied asking for specifics about the position I was referred to their website. Not even a link, just a suggestion to “go to our website and look at our jobs page”.

Seriously?

If I’ve asked you a question, that is something salespeople call a BUYING SIGNAL. As in, I’m buying into the idea that you might be a cool company to work for. On top of that, since I was approached by this recruiter (not the other way around), I’m already (again in sales terms) a QUALIFIED LEAD. This particular recruiter clearly gets no coffee. Why? Say it with me: COFFEE IS FOR CLOSERS.

Don’t even get me started with the outside recruiters who just spam people.

Resume Walls I am expected to meekly submit my resume into The Wall and hope and pray for a response. Could you be less humane? I might expect this from some megacorp, but I’ve seen this for a 10-person startup. That’s insane. I’m a person. You’re a person. So be personable! Interact!

My “favorite” resume wall was for a company whose CTO publicly ranted about enterprise software vendors doing the exact same thing to him when he wanted to buy a product. I guess he didn’t see the connection.

Micro Positions “Node.js V8 Korean-language backend optimization engineer”

Ok, I made that one up. But I’ve seen close cousins.

Sometimes you need a very specific skill set NOW to solve a serious problem. That is totally normal. What isn’t normal is pretending that you need that level of specificity in every position, all of the time. What happens when your company stops using Javascript on the backend? Am I fired? Or what if Karen quits? Can I pitch in and help, or should I just mind my own goddamned business?

You should be hiring people, not units of labor. I don’t go quite so far as Heinlein on this, but there is a point of diminishing returns on specialization.

Under-Described Positions I can’t tell you how many “Ruby on Rails developer” positions I’ve seen. This says exactly zero about what the actual job is. What sorts of problems are you looking to solve? Is there more programming going on than just tinkering in the Rails stack? Would my C/C++ background be of any use? Would my distributed programming background be of any use? My interest in graphs and set theory? Are you going to make me cut up PSDs? I’m left with questions and no motivation to seek answers.

Assuming I Am Only The Last Thing I Was I’ve been a programmer for 16 years. I am an experienced developer with a range of skills, both technical and non-technical. I am not a “Rails dev” or a “.Net engineer” just because those were the last positions I held.

I like to solve problems with software. And sometimes hardware. And sometimes by just sitting and thinking and chatting. I’m a person. My name is Matt. Nice to meet you.

Asking For “Rock Stars” or Variants Thereof I honestly thought this one was dead. But I just saw a rockstar-style posting a few minutes ago.

I can’t say for sure if this draws in the douchebags or not, but I know from personal experience and conversation with other devs that this chases away the modest and those experienced enough to know they don’t want to work with alpha-brogrammers. It also completely excludes folks who suffer from Impostor Syndrome, who might otherwise make loyal and productive employees for many years to come.

The Cookie-Cutter Technical Interview You know what I’m talking about. There are many variants, but they all boil down to something along the lines of:

Ask people to write code on paper or a whiteboard

Ask someone to code a fibonacci function

Ask any question that starts with the phrase “tell me about a time when”.

PLEASE DO

Be Interesting The number one thing you can do to improve hiring is to actually, genuinely be an interesting place to work. I’m not talking about ping pong tables and beer in the fridge (both of which are nice). I’m talking about the kind of things your company does. Solve interesting, challenging problems. Or solve boring problems for famous and/or incredibly attractive people. Find some way to make it interesting. And it absolutely does NOT have to be “fun”. If what you do is interesting enough, smart people will find a way for it to be fun.

Care Genuine passion is contagious and powerful. Fake passion is dangerous: At best you look like an ass. At worst you attract gullible idiots to work for you.

If you don’t have genuine passion for where you work and you do the hiring, you’re committing some kind of moral fraud. You might want to see to that.

Acknowledge Applicants Acknowledge me when I show some interest in your company.

Example: I submitted a resume to a resume wall a few years back. I was well-qualified for the position and I wrote a custom cover letter as requested. I heard nothing back. At all. I was sad. But Lo! Six months later I got an email out of the blue from the company asking if I was still interested. Mental whiplash. By that point, no, I wasn’t interested in the company anymore. I had hurt feelings. Yes, it’s irrational. But it still happens. Alienating people isn’t cool.

Get to Know Programmers Pop quiz for hiring managers: Name all of the programmers you know personally. Don’t count the ones you already work with!

How many names did you come up with?

The real meat and potatoes of hiring people is getting out into the community and meeting people. This is a graph traversal problem; they call it “networking” for a reason. Unfortunately the term “networking” seems to have become synonymous with dull mixers and luncheons where people generally fail to establish meaningful connections with other people.

You don’t have to be a social wizard to tackle this problem. I once had this boss, Andrew. The first time Andrew would meet anyone – almost anyone at all, really – he’d ask them “Do you code?”. It was awkward but fairly effective. 9 times out of 10 the answer was “no”, but it opened up a conversation. The network effect kicked in. People knew that we were hiring programmers. We got some referrals. Did it solve everything? No. But it helped.

Just like a single person dating, you’ve got to get out and meet people. And in that same way, it’s not about meeting Mr./Ms. Right, it’s more about meeting people who may eventually connect you to Mr./Ms. Right down the road.

If you’re serious about building a quality team, you’re in it for the long term. Act like it.

Let Me Code, Dammit The tools for a technical interview are simple: An intelligent interviewer and a computer. Everything else (whiteboard, conference room) is optional.

The best interview experience I ever had involved pairing for a half hour with every single person on the team. It made for a long interview but it answered questions that the employer and I wouldn’t have known to ask otherwise.

Source

http://www.youell.com/matt/writing/?p=898

[startupmap]

23 Tips on How to A/B Test Like a Badass

You know you’ve hit the big time when you end up in a feature article in Wired. That’s what happened when Brian Christian wrote about the A/B test — the technology behind some of the biggest e-commerce online.

A/B testing is hitting the mainstream because it is so effective. And with so many tools available it has become very easy and very inexpensive to run.

But talk to most marketers and they only have a basic understanding of the art of A/B testing. They haven’t quite figured out how to take it to the next level.

That’s what this article is all about: helping you become a badass A/B tester with the following 23 tips I’ve picked up in my experience in the field.

Let’s go.

1. Test Your Software

Not all A/B testing software is the same. For example, Neil Patel said that he would run A/B tests where there were swings as high as 40 percent. That’s a significant difference that might lead you to believe you are on to a winner. But Neil discovered that the revenue wasn’t changing based on the increased conversion, so he suspected that there was something wrong with the software.

How do you tell if software is accurate? Run an A/A test.

In other words, test the control against the control. If you see a significant difference in results, then you can be sure that your software is inaccurate.

2. Minimize Friction

Friction is any element on your landing page that slows down conversion. It’s the staple for good A/B testing—the thing that you must look for immediately. Here’s how we do it at Treehouse:

What should you look for? Here are classic elements that cause friction:

Form Fields – Forms should be simple to fill out. The more information you request the slower the process. Make sure you can justify ever piece of information you are asking for.

Process Steps – The more steps the slower the process. Like forms, every step should justify its existence. If you can eliminate it, eliminate it.

Page Length – A long page may not cause friction since you provide all of the essential information below the fold. Even then you should always test page length.

Any element of friction will pull down your conversion rates. Tweak the obvious and then A/B test. But as you’ll see you should even test anything that becomes conventional wisdom. I’ll explain in a few minutes.

3. Information-Entering Anxiety

While this is hard to test for, you can usually spot where this is a problem by a pattern in abandonment at a certain step in the process.

Anxiety is created when people aren’t sure if they are going to be rewarded for all of their work. If they suspect that the process steps aren’t useful, they may bail. You see this happen a lot on pages that request a lot of information for an exchange of low value (email newsletter, for example). Surveys also cause anxiety. The longer the survey the more drop out you will have.

However, if people feel like they are going down the right path, it doesn’t matter how many steps there are in the process. As long as they are following the right scent they will be happy to click away and fill out forms.

4. Clarity Trumps Persuasion

This might upset some sales copywriters, but persuasion isn’t what always works the best. Sometimes it’s nothing more than being insanely clear. This means that you can avoid a lot of the language of influence by answering these three questions:

Where am I? Make it clear where your visitor is in the sales process.

What can I do here? Make it clear what you want the visitor to do. One page, one goal.

Why should I do it? The value proposition should be obvious. Your visitor shouldn’t have to guess what he should do—neither should he guess why. Make it clear.

5. Interview for Insights

An A/B badass knows that one of his first jobs is to sit down with the client and the ideal customer and listen to what they have to say. It’s not all about software.

This is the best way to collect facts and feedback. It’s also hard to go wrong when you observe behavior and see how customers walk through your landing page. After that, pull them aside and listen to customers to why they did what they did. That action will lead to some deep insights.

6. Learn from Pricing Takeaways

Nailing down an acceptable price is probably one of the hardest things to figure out. This is why you need to build hypothesis with real data—and then test those hypothesis with an A/B test (don’t forget to run an A/A test to calibrate your software).

But when testing price you have to understand that it’s not all about the numerals. Customers care more about the value that you provide behind that price—so tinker with your sales copy, the bullets and ask yourself if you are showing great value.

It’s more important to base price on what your customer thinks it is worth and not how much it cost to create and operate.

7. Learn from Higher Prices

Never assume that lower prices will raise sales. Nor never assume that raising prices will drop sales. You may have your prices too low for the value you are providing so customers walk away thinking that you are hyping up a poor product. If you raise prices to match the value, then you might see sales jump with higher prices.

Work in small increments and batches. For example, start with a 2 percent increase in price against the control, document change and then try another 2 percent on top of that. Small incremental changes will keep your customers from feeling like you are overcharging.

8. Test Social Features

I see a lot of social share buttons on product pages. I’m not sure why they are there—did the company test to see if they raise or suppress sales?

An interesting study done by Empirica Research found that like and tweet buttons suppressed shares on a Clearasil skin product page by up to 25 percent. Some products are just too sensitive to think about sharing, so that’s a huge decrease in sales from just adding a like or tweet button.

Appropriately Amazon puts social buttons after the order:

And you can bet they tested that.

9. Lower Your AdWords Position

Common sense might tell you that the higher your AdWords position the more traffic you will drive to your landing page. You’ll hear me say this more than once, but all A/B testing badasses know that you should always test common sense.

Besides, a difference between position one and two, or two and three could only be a half a percentage point—but a huge divide might exist between the bid prices for those positions. In other words, you might get a half a percent less traffic for a lower position but your cost is more than 50 percent --and if you calculate your ROI you’ll realize which one is more effective. Moral of the story: test all of your assumptions.

10. Test All Best Practices

Best practices arise because enough tests have been performed that have returned similar results. Some of the more common A/B testing dogma you’ll hear is that headline copy should match your text ad copy or endorsements will raise your trust levels for the landing page.

While those standards are true in most cases you should always analyze your elements. Sure, use best practices when you don’t have the time. When you have the time, however, or when exceptional results are critical, take the time to test all elements—even if you think you know the answer.

11. Employ Small Tests When Under the Gun

When you don’t have time to be comprehensive you need to isolate effective changes with small tests. For instance, using software like Silverback can allow you to record a person using an ecommerce site and get useful data within five minutes. I’ve seen short test like that results in cart conversion increases from up to 13 percent. Think about it: 5 minutes work for 13 percent increase. That happens when you solve real customer problems.

12. Don’t Ignore the Conversion Trinity

A/B testing badasses know that they can do some serious CRO improvement when they enhance one or all of the elements in what are called the Conversion Trinity:

Relevance: Is your landing page relevant to the visitor’s wants or needs? Are you maintain the scent? Are you being consistent?

Value: Is your value proposition answering all of the meaningful questions that the visitor is asking? Will your solution provide them with the right benefits?

Call to action: Is it clear what the visitor needs to do next? Can they perform that action with confidence—or are they confused?

This is all about customer fit.

13. Test the Message

While making micro site changes can improve conversion you shouldn’t ignore macro issues. For example, testing the difference between “Join Now” versus “Buy Now” is a good idea; you can also see dramatic changes in conversion by testing the pages entire message.

Is the message from the headline to the first sentence to the call-to-action consistent? Do the designs and images reinforce the message?

Sometimes it’s healthy when A/B testing to step back from the trees and look at the forest. You might be surprised by the insights that you find.

14. Design with On Page and One Goal

Every A/B testing badass will tell you that the best conversion advice you can ever give is to focus on one goal per page. And that doesn’t take any complicated analytic software to figure out. It should be plain with a quick glance.

15. Test Appeals in Your Headline

Knowing your customer is commandment number one when it comes to testing. And knowing what appeals to them is essential. But often when you ask them they won’t be able to tell you—that is until they actually see it.

What do I mean by appeal? I mean benefits that resonate with them. Often products provide numerous solutions to a client—but only the biggest and best should be presented in the headline. The problem is that we often think we know what benefit is the biggest and best—and we are often wrong.

You can bet that Optimizely tested headline appeals. Here’s their current favorite:

Make a list of every single benefit your product provides. Then test those in the headline. Stick with the winner.

16. Test Insanely Small Elements

AJ Kohn tells a story of how back in 2007 that he tested the difference between using initials in the URL of a Google AdWords test versus not using initials: www.YourDomain.com versus www.yourdomain.com.

The results surprised him: the URL with the capitals beat out the non-initial URL by 53 percent. That’s huge.

Now Google doesn’t allow that sort of customization but it goes to show that big improvements in conversion can be achieved in seemingly small changes.

17. Violate the Sacred

Want to see some unexpected and big changes? Then test things that you know should not be tested.

Let’s say you have a tightly optimized page. Most people would dust their hands of testing thinking they’ve nailed it. But you might be surprised at how much more conversion you can squeeze out of a page by taking a list of the best practices—things that should never be violated—and violate them.

18. Don’t Be Perfect

Often conversion experts think they have to produce perfect landing pages. But that’s not the goal of A/B testing. Conversion is built on making things better than they were before—not perfect.

For example I’ve seen tests where clients have had ugly landing pages and by simply creating a new page with a new design and a clear call-to-action—not a perfect page by any stretch of the imagination—but a page that was just better than the old one, and seen profit jump by 75 percent.

The trick is not to go back to that page and test some more. In other words, create a minimum viable landing page that is better than the old one—and start testing from there.

19. Look for Cost-Effective Ways to Test

Slick A/B testers aren’t all about using the latest software technology or employing sophisticated methods to testing. It’s often just finding out the improvements through paper prototyping or running a 5-second test of a wireframe.

You don’t need a polished product to test nor do you need to always use complex technology. Never discount tests that are cheap or simple. Better yet, perform these cheap and simple tests, then validate the results in the web browser.

20. Let a Client Contradict A/B Testing

Successful A/B testing is a balance between proven optimization techniques and information you get from clients. It’s really all about testing assumptions—your client’s and yours.

I remember one instance where there was a doubt that trimming excessive content would improve the landing page conversion rate. While the client admitted in not testing to prove this assumption, they shared plenty of anecdotal evidence that said their approach was working.

We came into test using over conversion techniques and testing usability against excess content. To our surprise the client was right—excess content pushed the conversion rate higher. Our next step was to figure why that was the case.

21. Wait Until the Test Is Over

If there is any one single lesson to learn from A/B testing it has to be this: it’s hard to predict results. Even when using our proven assumptions we are often surprised by the results.

One time I was working on a nine-month project where we churned out over 200 pieces of content. I think I got pretty good at predicting what headlines would work—I was right probably about half the time, meaning it was no better than a coin flip. Part of the problem was that I was going off early information—I wasn’t waiting until more results came in.

22. Never Trap a User

This falls in the category of best practices, but it’s so important I wanted to isolate it in its own section. Common sense with landing page conversion would say to remove all escape routes on that page. This would mean limit the links and eliminate navigation.

You can bet Copyblogger Media tested navigation versus no navigation. See which one won?

Unfortunately I’ve seen into many tests that when you block off all exits people simply abandon the page. When visitors feel trapped they get nervous and bail. Again, this is something that you should test and not simply affirm because it is a best practice.

23. Always Be Testing

Finally, A/B testing badasses won’t accept the status quo. They’ll challenge every fundamental by testing it because they know that just because something has worked over 60 percent of the time of 70 percent of the sites doesn’t mean that it will work on their site. They treat every site or page as a unique and test the best standards.

And even when they’ve established controls they will eventually get around to re-testing those controls. They know that every single control must justify itself. There are no free lunches and no sacred cow.

Your Turn

Do you have any unusual tips that can make someone a A/B testing badass? If so, please share them in the comments.

Source

http://searchenginewatch.com/article/2223888/23-Tips-on-How-to-AB-Test-Like-a-Badass

[startupmap]

How to Launch Your Startup Idea for Less than $5K

“Creativity is 99% perspiration and 1% inspiration” – Thomas Edison

In the corporate world, there’s a standard ladder to climb to get to the top. In the world of entrepreneurship, there is a similar ladder to climb. Entrepreneurs should start with small ideas and learn how to execute those ideas. Learn how to gain traction, learn from your mistakes, and learn how to make something out of nothing. And eventually the ideas you pursue get bigger and bigger.

I’m a firm believer in execution and follow the Thomas Edison quote pretty closely. The secret behind launching your startup idea is to always move the ball forward on your ideas through execution. Every journey for an entrepreneur is completely different but I’d like to share the process that eventually led to the launch of Skillshare.

1. Exploration & Execution. This is a time for the entrepreneur to creatively explore his/her personal interests, ideas (small or big), and consume as much of the world as possible. For me, this involved traveling around the world, reading as many articles/books as possible, meeting as many interesting people I could fit into a day, and executing ideas.

I also knew that I wasn’t ready to be an entrepreneur so I gained startup experience by working at Behance and Hot Potato. On the side, I launched a ton of small ideas including The Feast Conference, By/Association, World Series of Good, TBD, and Lovely Day. By executing all these smaller ideas, the filter for what I wanted to work on got higher and higher.

The idea for Skillshare didn’t happen overnight. It took 5+ years of climbing the ladder of ideas and immersing myself in a lot of different experiences. There is no rush in understanding yourself and your passions. Keep in mind that most entrepreneurs get stuck in this stage because they never execute anything. The more you execute, the more your learn about yourself and your passions. Your goal at this stage is to find a problem you are truly passionate about solving.

2. Brainstorming & Validation. While I was exploring, I wrote down a list of startup ideas, which quickly became a list of over 100+. From there, I narrowed it down to a few ideas and flushed them out extensively. I brainstormed ideas, sketched wireframes, and did everything else I could to understand the opportunity with Malcolm Ong (Skillshare Co-Founder).

We fell in love with one idea around democratizing education and turning cities into huge campuses, which eventually became Skillshare. Rather than spending nights and weekends flushing out the idea even more, we let the idea sit and marinate for a pretty long time. I personally spent a lot of time validating the idea as I didn’t want to fall in love with it too easily. In other words, I did all the research I could to convince myself this was worth my time, which is the true goal at this stage.

3. Feedback & Commitment. Once I convinced myself this was an idea I’d like to pursue, I asked a dozen really smart people I knew what they thought about the idea with a small twist. Rather than asking them if they liked it, I asked them why the idea wouldn’t work, why it would fail, and why I shouldn’t work on it. Malcolm did the same thing and we eventually had a list of 20-30 huge holes in our idea.

We hit the drawing board again and came up with solutions to plug all the holes. We went back out to a dozen different people and asked the same questions. We repeated this until almost every hole was plugged in our startup idea. We eventually had rebuttals for rebuttals and felt very confident that we should commit to working on this for the next 5+ years of our lives.

Your goal at this stage is to “go all in” on your idea and put your stake in the ground. This was the hardest part of the process for me but once you put your chips in the middle, it’s the best feeling in the world. Remember that you’re running a marathon, not a 5K.

4. MVP (Minimum Viable Product). Malcolm and I agreed that we wouldn’t write one line of backend code unless we knew this was something that people wanted. We did this by setting two really simple milestones: a) 1,000+ email addresses for our alpha page and b) selling out all of our initial classes ( 3 per month for 3 months). We didn’t want to spend too much money on the idea so we set an overall budget of $5K.

I bought the URL and hired Ed Nacional to design the brand identity around Skillshare. From there, we put up the “alpha” version of Skillshare, which took less than a day as it was HTML/CSS. We used Campaign Monitor to capture email addresses and Eventbrite to power all of the ticket purchases. I taught the first Skillshare class around poker, which you can still view on Eventbrite. We needed some marketing juice so I wrote a controversial article on “Why College is Overrated” for GOOD Magazine and we put up a Kickstarter project around the same topic.

We ended up getting over 5K+ email addresses from folks that signed up to be notified about Skillshare. All of the initial classes sold out (huge validation) and our Kickstarter project got fully funded. If you follow the lean startup methodology, you’ll know that most startups don’t fail because of the technology, they fail because they are solving a problem that no one cares about.  Our goal at this stage was to see if anyone on this planet wanted Skillshare to exist. We passed that initial test, kept hitting our next milestones, closed a seed round led by Founder Collective, and launched a full beta site in early April.

This was a very long process for me. It didn’t happen overnight. If Biology 101 is the weed-out classes for doctors in college, launching your startup idea weeds out the majority of new and aspiring entrepreneurs. I failed over and over, but as long as you make the best decisions and take calculated risks, you can successfully launch your idea and make the world a better place!

Source

http://www.mikekarnj.com/blog/2011/08/01/how-to-launch-your-startup-idea-for-less-than-5k/

[startupmap]

Moving The Ball Forward

Every action taken (or not taken) either pushes your idea forward or holds it back. Sometimes we ignore the intangible elements of productivity, namely self-awareness, discipline, and confidence. You must manage yourself as you manage ideas. If you stop and think about it, your ability to make ideas happen is dependent on you and…the ball.

Are you moving the ball forward?

With everything you say and do, you either move the ball forward or backwards. Excess chatter in meetings, repetitive questions, and inconsequential concerns only distract you and others from action – thus moving the ball backwards. Aspire to content-make rather than commentate. Moving the ball forward is adding value that enriches and expedites the outcome.

Is the ball even moving?

Insecurity and apathy can inhibit progress. In the process of pursuing ideas, you need to keep the ball in motion. Remember, idle balls sitting still on the floor are dangerous: you can trip on them. Catch yourself when the components of an active project are still. Dwelling over inactive projects can destroy motivation.

Is your eye on the ball?

Typical meetings are full of digressions and points made beyond the scope of the meeting. Sometimes interesting stuff takes the place of relevant stuff. If you can stay focused on the purpose and the points that gain traction, then you can help foster productive discussion.

Who has the ball?

The ball is either in your court or someone else’s. The problem is that, after many meetings, the ball is left in neutral territory. No-man’s land. Are you supposed to call the person to further discuss the problem or are they supposed to come back to you with possible solutions? Always know where the ball is, and never ever leave it in your court. If we all committed to never let the ball rest in our respective court, then the ball would ALWAYS be moving.

It is hard to manage the many forces required to push ideas forward. A ball is round, it rolls, and it is the key to productivity – so keep an eye out for it!

Source

http://99u.com/articles/5598/moving-the-ball-forward

[startupmap]

Developing new startup ideas

If you want to start a company and are working on new ideas, here’s how I’ve always done it and how I recommend you do it.  Be the opposite of secretive.  

Create a Google spreadsheet where you list every idea you can think, even really half-baked ones.  Include ideas you hear about (make sure you keep track of who had which idea so you can credit them/include them later).

Then take the spreadsheet and show it to every smart person you can get a meeting with and walk through each idea.  Talk to VCs, entrepreneurs, potential customers, and people working at big companies in relevant industries. You’ll be surprised how much you’ll learn.  The odds that someone will hear an idea and go start a competitor are close to zero.  The odds you’ll learn which ideas are good and bad and how to improve them are very high.

Every conversation will contain some signal and some noise. Separating the two is tricky. Here are some broad rules of thumb I’ve developed for how to filter feedback based to the profession of the person giving it to you.

1) Employees at relevant big companies. 

These people are great at providing facts (“Google has 100 people working on that problem”) but their judgment about the quality of startup ideas is generally bad. They tend to have goggles on that makes them think every good idea in their industry is already being built within their company.  For example, every security industry person I talked to thought SiteAdvisor was a bad idea.  (If it wasn’t, they think, someone at McAfee or Symantec company would have already built it!)

2) VCs. 

VCs are good at telling you about similar companies in the past and present and critiquing your idea in an “MBA-like” way:  will it scale? what are the economics? what is the best marketing strategy?  I would listen to them on these topics but pretty much ignore whether they think your idea is good or bad.

3) Potential customers.  

If your product is B2B, remember you’ll be selling to that person 2-3 years from now and by then the world and their priorities will likely have radically changed.  If your product is B2C, it’s interesting to hear how regular consumers think about your product but often they really need to use it fully built and in the proper context to really judge it.

4) Entrepreneurs. 

This is the one group I listen to without a filter.

Even though I have no intention of starting a new company for a long time (if ever), I still keep my idea spreadsheet and update it periodically.  Some of the ideas I wrote down a few years ago are now companies started by other people (some successful, some not).  A few I had the chance to invest in. It’s interesting to compare my notes and ratings of each idea with how those companies have actually performed. I also keep a list of “on the beach” ideas in case I have time in between startups. These are mostly non-profit ideas.  I don’t know if I’ll ever get to those but they are particularly fun to think about.

Source

http://cdixon.org/2010/03/14/developing-new-startup-ideas/

[startupmap]

One of the most important lean startup techniques is called the minimum viable product. Its power is matched only by the amount of confusion that it causes, because it's actually quite hard to do. It certainly took me many years to make sense of it. I was delighted to be asked to give a brief talk about the MVP at the inaugural meetup of thelean startup circle here in San Francisco. Below you'll find the video of my remarks as well as the full slides embedded below. But I wanted to say a few words first. First, a definition: the minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort. Some caveats right off the bat. MVP, despite the name, is not about creating minimal products. If your goal is simply to scratch a clear itch or build something for a quick flip, you really don't need the MVP. In fact, MVP is quite annoying, because it imposes extra overhead. We have to manage to learn something from our first product iteration. In a lot of cases, this requires a lot of energy invested in talking to customers or metrics and analytics. Second, the definition's use of the words maximum and minimum means it is decidedly not formulaic. It requires judgment to figure out, for any given context, what MVP makes sense. As I talked about in a previous interview, IMVU's original MVP took us six months to bring to market. That was a pretty big improvement over a previous company, where we spent almost five years before launching. Yet in another situations we spent two weeks building a particular feature that absolutely nobody wanted. In retrospect, two weeks was way too long. We could have found out that nobody wanted the product a lot sooner. At a minimum, a simple AdWords smoke test would have revealed how utterly bad the concept was.

Source

http://www.startuplessonslearned.com/2009/08/minimum-viable-product-guide.html