How to build a great engineering team from the ground up

When building a house, if you want to develop one that will last, you will start with the foundation and make sure that's done right. After the foundation, you build the walls, then the roof, and then finally spend a long time inside and outside, adding the finishing touches. You will see throughout this article that building up Software Engineering teams is not too different in that regard.

So the first rule of building anything well is to understand what you are trying to build fully. We need to recognize that hiring engineers, and building a great engineering organization, will be unlike many of the other hires we make in our company. In the book "What You Do Is Who You Are", Ben Horowitz explains:

“If a customer asks, “Do you have feature X?” a good engineer will answer yes or no. A good salesperson will almost never answer that way. She will ask herself, “Why are they asking about that feature? Which competitor has that feature? Hmm, then they must be in the account trying to take my deal. I need more information.” So she’ll reply with something like, “Why do you think feature X is important?”

Having their questions answered with questions drives engineers insane...

In a well-run organization, engineers get compensated more for how good the product is than for how much money it ends up bringing in, because there are often serious market risks that are outside the engineer’s control. Great engineers love to build things and often code on side projects as a hobby. So creating a comfortable environment that encourages round-the-clock programming is vital. Hallmarks of engineering cultures often include casual dress, late morning arrival times, and late or very late evening departure times.

...

Salespeople represent the company to the outside world, so they need to dress accordingly and show up early, when their customers punch in. Great sales cultures are competitive, aggressive, and highly compensated—but only for results.”

We learn from Ben that the way you hire, the incentives you set up, the culture you build-up will be different for your engineering group than all the other parts of your company. From experience, I can tell you that this isn't just true at the startup stage; this is true for the best organizations in the world too.

The foundation is the key.

The first few hires of any engineering organization or team will intentionally or unintentionally set the bar for the entire thing. The first few hires need to be as amazing and as senior as you can find and afford.

"The best properties are rarely for sale.
The best employees are rarely job hunting.
The best clients are rarely shopping.
The best option is usually off the market. Most people think this means you can't have it. What it really means is you have to go find it.." -James Clear.

That you get good people early, this part is critical. Reed Hastings, the founder and CEO of Netflix and a software engineer himself, explains exactly why in his book "No Rules, Rules":

“At 6:30 a.m. nine trainee programmers were led into a room with dozens of computers. Each of them was handed a manila envelope explaining a series of coding and debugging tasks they would need to complete to their best ability in the next 120 minutes.

Millions of keystrokes have since been devoted to discussing the results on the internet.

The researchers expected to find that the best of the nine programmers would outperform his average counterpart by a factor of two or three. But of the group of nine, all of whom were at least adequate programmers, the best far outperformed the worst. The best guy was twenty times faster at coding, twenty-five times faster at debugging, and ten times faster at program execution than the programmer with the lowest marks.

The fact that one of these programmers would so dramatically outperform another has caused ripples across the software industry ever since, as managers grapple with how some programmers can be worth so much more than their perfectly adequate colleagues.

With a fixed amount of money for salaries and a project I needed to complete, I had a choice. I could hire ten to twenty-five average engineers or I could hire one “rock-star” and pay significantly more than what I’d pay the others, if necessary. Since then I have come to see that the best programmer doesn’t add ten times the value.

She adds more like a hundred times.”

Engineers will build things that others can build on top of or do the opposite and build things that others need to spend days, months, or years unwinding.

Early engineers will be responsible for the initial best practices, the architecture, the code reviews; they will be responsible for helping you hire other engineers and will be in the interview loops. Good people will screen for other good people. Although the hiring practices in engineering can be hit or miss in identifying great talent, great engineers have good intuition. If they have the right level of experience, they will do a much better job at hiring. A few percentage points better in hiring in an industry that is hit or miss makes a huge difference in building up talent density. The difference between a sound engineer and a bad one can be massive when it comes to code, but the list of other vital things goes on and on, and it's not just the code.

If we make a mistake, and it can happen to anyone, of hiring a lousy engineer at this early stage, we need to correct it right away. I cannot stress how important this is. As my mentor once put it, "the issue with hiring a bad engineer is not that they won't do their work, that's a small risk, but they will do damage and create tons of work for the rest of the team." Just like a house, if that foundation of the engineering team is rotten, you may have to break it down and start over.

We now know that we have to have a solid foundation and that the first few hires should be exceptional. We also know that they should be great even if we have to overpay and even if, as Reid told us, we only end up being able to hire one or two good ones vs. many mediocre ones. I wrote an article about how we can recruit and retain great engineers that spells out some of this in more detail for anyone interested.

Run fast and stay focused.

The early stage of your team with a few amazing engineers on it will be pound for pound the most productive of any of the other stages. A small engineering team building things from the ground up can run faster in this day and age than at any other point in our industry's history. In fact, after this early stage, we will spend nearly all of our time thinking of ways to try and keep this sort of velocity going in our organization.

If we got the foundation right, we could now begin to build the walls of this house, and we can do it fast.

The CTO of Amazon, Werner Vogels, says that the secret to Amazon's success over all other eCommerce players was speed. He says that "everyone in the industry was afraid to take on technical debt and was focused on reduced duplication at the traditional enterprises. Amazon bucked this trend and built a distributed system that had technical debt and duplication, but they were also adding features at a pace no one else could keep up with."

This part about speed might all sound counter to what I just told you in part one; it's not. Great professional and senior software engineers that understand the tradeoffs they are making, and do so consciously, can move fast. They can take on the debt and still get the business value we desperately need at these early stages of a team or company. It is critical to keep this early group focused on building the product and not just technology.

A great way to keep the team focused on the product is to always work from the customer backward at this stage. Start with the product briefs, the design document as our teams did, or the customer press release as Amazon does and iterate on those documents. Then as Werner Vogels says, "Thou shalt not build more than that."

Most great engineers know that the code we write at these organizations has marginal value, but the expertise and domain knowledge we build up is priceless. It is why once a system is out there, a great engineer can think of ten ways to rewrite it and improve it. The only way to rewrite and improve a system, though, is to survive long enough as a company to get the chance to do that.

A small, nimble engineering team with amazing talent density can do all sorts of damage, especially when paired up with good product and business intuition. Of course, if our product is working out well and customers start using it, we won't be able to stay small forever, so next, we need to tackle scaling our team.

Culture begins to get formed at this early stage. Some people believe in letting the culture emerge, while others want to be more intentional about it. Regardless of how you chose to do it by this stage, after your first few great engineers, the pillars that make up the team's culture need to be solidified and embedded into the team. In my teams, in the early days, a few of those pillars were borrowed from the larger company: Trust, Fairness, Transparency, and to that, we added Perseverance and Value Minded as a group.

Scale and the diminishing returns.

If your team and company made it this far then, it is time to start introducing rigor. Now is the time for rigor because no amount of rigor will help you early on if you don't have customers for your product. Building on top of that initial group is now the key; they have the knowledge and have become experts. Besides writing the code, they have become domain experts. That initial group of great people knows where the bodies are buried too. They can begin to look back and direct people to pay back the technical debt.

Thinking on the house analogy, this is where we build the roof of the house and reinforce anything else we've built on top of our solid foundation and walls.

Now you are ready to break it up into multiple teams by more granular domains. We break it up so we can move fast again. We can continue to innovate while making things scale and ensuring we aren't breaking our promises to our customers.

At this stage, we can begin to ease up on the types of hires we make and start hiring more junior developers or taking on interns. Good processes that force rigor will enable us to take on great people that might have a little less experience and build them up. We never want to lower our bar; less experience is not the same as laziness or incompetence. We always want the bar to remain high and the talent density to be of the highest caliber in an engineering organization or team, in every stage.

To add rigor, we need process, but we want to be careful because the more processes we add, the more we can inadvertently slow things down. The first two things we will add are a Design Review and a Production Readiness Review. Those documents that our teams should have been writing all along now will get presented to all of engineering. In one of my previous articles about the design review, I wrote:

"I will argue that the design review process has been instrumental to us building a world-class engineering organization. As you'll read in this post, I attribute much of our uptime, cross-pollination of technical ideas, and the reliability of our systems to these two engineering-wide meetings. They help catch edge cases and really push our systems in the right direction."

We also need a way to review production incidents. If we want rigor, there is no better way to get it than to start writing Root Cause Analysis documents for every incident and start sharing those with the whole team. I wrote another post on how to do this well, "After the Bug, the RCA," but a quick weekly or daily meeting with the whole team to review the root causes of incidents will lead to permanent fixes and mistakes that never get repeated. The incident reviews will also force the teams to start spelling out SLAs in places they might have missed.
We can include many other processes to create more rigor, such as Error Budgets, but we need to move thoughtfully and retain our teams' velocity. Around this stage, our new team, now turning into an engineering organization, will begin to harden its principles. A while back, I spelled out some of this in "Principles for High Growth Products." These principles will drive our architecture as we scale and be the glue we fall back on to keep making great decisions.

Build it right, it will be far more impactful and durable.

The process I outlined above to build up a great engineering team might seem more difficult than just slapping things together. It is also true that the core parts of a house are the hardest to get right, but it makes the home far more durable. An engineering team built this way will also be the most durable, engaged, productive, innovative, and impactful type of engineering organization you could build up.

Machiavelli said in the Prince of Hiero of Syracuse, who comes to power by his abilities rather than good fortune:

"This man being, as I said, made head of the army by the Syracusans, immediately recognised the uselessness of that mercenary militia …. and thenceforward made war with his own arms and not those of others."

"He abolished the old militia, raised a new one, abandoned his old friendships and formed new ones; and as he had thus friends and soldiers of his own, he was able on this foundation to build securely, so that while he had great trouble in acquiring his position he had little in maintaining it."

I am incredibly proud of the teams I have been fortunate to build up throughout my career. I am especially proud of those early hires who then turned into leaders who came up the ranks to build additional teams with me using this framework. I am proud because the teams have had some of the highest engagement scores across multiple organizations. It is one of the most diverse groups of people I have ever worked with. On top of all of that, the teams have proven to be incredibly durable, even living long after the Jet.com acquisition.