A Conversation with Allen Kleiner
We were very fortunate to have a candid and authentic conversation with growth engineering expert Allen Kleiner. In our conversation, Allen shares his experiences as a Growth Engineer at Pinterest and Growth Engineering Manager at Segment. Having worked in growth engineering during hyper-growth periods at both B2C and B2B companies, Allen has a tremendous amount of experience and a wealth of knowledge that he shares with us here!
Here are just a few key takeaways from our conversation:
- Growth engineering is about unlocking new avenues of business growth through experimentation, rapid iteration, constantly trying new ideas, and by continuously learning how to best serve users.
- Growth engineers have the opportunity to work on highly-complex engineering projects, including large-scale, distributed systems. However, they also need breadth—the ability and skills to jump into any part of the product’s codebase.
- New user onboarding is a common and challenging project growth engineers work on. Defining levels of product engagement, instrumenting the application, leveraging data for in-app behavioral targeting, automating coordinated out-of-app experiences (messaging, notifications, etc), and of course, experimentation are all integral to improving activation.
- Two of the biggest differences between growth engineering at B2B SaaS and B2C companies are experimentation and internal partnerships. At B2B SaaS, lower data volumes means experiments take a long time to reach statistical significance, so you have to develop good practices around ‘directional correctness’. On the partnerships side, growth engineering teams at SaaS B2B companies form tight partnerships with their colleagues in Sales, Marketing, and Customer Success.
- Leveraging 3rd-party tools and technologies can increase velocity but you have to be careful that they don’t increase complexity too much. No-code solutions, where engineers have little to no understanding and/or control of the product, don’t just increase complexity—they also reduce visibility and increase brittleness.
Today with Switchboard, we have Allen joining us for a conversation about his experience in growth as an engineer. Thank you for joining us, Allen!
Thanks for having me guys!
To start out, I would love to learn a bit more about your journey, how you found your way into growth engineering work and what drew you into it?
I started off my career at Pinterest, where I was super lucky to work on a bunch of different teams. I started in web infrastructure and then I progressed towards product work. I joined this core user interface, like a central UI team, that owned a lot of the main user flows throughout the Pinterest products.
On that team, I collaborated with a team on growth. An engineer on their team left and they had this big backlog of pretty fun work that I was pretty jazzed about so I started moonlighting a little bit working for them, picking up some stuff off their backlog.
Then I eventually transferred over to their team where I spent a little under a year working. For me, that was super exciting work because I had never done growth before and this was a growthy product team, so they had a lot of really great growth experimentation work and they had a lot of really great verticalized product work.
The team was geared around making Pinterest more social, which I thought was really a cool and very greenfield idea and strategy for the team. I spent a lot of my time leading a lot of efforts for that team and just really fell in love with growth—as an ideology almost; but also the work itself, it was really empowering. A couple of years later I ended up at Segment, leading the growth teams there, where I really, really enjoyed the work of leading activation and monetization efforts. I eventually spun up a team as an engineering manager to take that work on. So that’s a little bit about my journey.
That draw—the work being fun—what about it was so engaging for you?
Some of it is a little bit cultural. At Pinterest and at Segment there was this ideology around bottoms-up growth. For traditional, verticalized product teams that meant it was the product and engineering manager’s job to synthesize a bunch of inputs, work with the team to figure out backlog items, and then go from there.
But a lot of that comes from customer feedback. It comes from research. There are concrete inputs. What was really fun about growth, and the bottoms-up method of doing growth engineering was that the engineers themselves are empowered to think of ideas, put the product manager hat on, do opportunity sizing, go execute on the experiment or product idea that you had, measure its success, and then make the call. I found that to be super liberating.
I’ve always had a love for product engineering and to get ownership is a really empowering thing for a lot of engineers who really do care about building great user-facing products. For me that was such a big, big draw and a very addicting feeling. You get this full sense of ownership. You get to release features out in the wild, stack up wins, and you can draw a direct line to your own personal impact on what you ship. That’s why growth engineering is a really exciting form of engineering.
What is Growth Engineering
From what we’ve heard from the community, there’s not really a single definition of what growth engineering is—it can look quite different at different companies. How do you think about it?
Growth engineering is iterative in nature. You have to have an appetite to iterate, an appetite to learn, and you have to be okay with failure.
Generally speaking, growth engineering is heavily focused on experimentation, rapid iteration, trying new things, and seeing what sticks. That can be a really unique mental shift towards building—where traditionally, engineers take requirements, put out an engineering design doc, ship it, and then are on to the next thing.
Growth engineering is much more focused on unlocking new avenues of business growth by influencing different levers of the business. Whether that’s users, top-line metrics, or conversion throughout the funnel.
Then, how can we iterate on each part of that? How can we iterate on user acquisition? How can we iterate on activating these users? How can we iterate on upselling or retaining users?
Compared to a lot of product engineering teams, growth oftentimes takes a more broad, cross-cutting approach. It also may be focused on a more nebulous thing. Take engagement at Pinterest for example, there’s a team entirely focused on user engagement that just finds whatever the highest impact is, so long as it’s under the guise of engagement, which is really exciting.
Growth Engineering at B2B vs B2C
A lot of people are talking about growth, but it tends to skew heavily towards consumer companies or around growth product practices. You’ve seen both sides at Pinterest and Segment, so I’m curious what differences you’ve between the two and what’s actually similar between each side.
In general, what’s very different about a traditional B2B SaaS growth engineer compared to even B2B2C where there’s a heavy self-serve user component, is a lot of your focus on engineering in B2B SaaS is actually how to empower your sales team or how to empower your marketing team to hit their goals.
Ultimately, your job as a growth engineer is to ensure that users get the most value out of your product. And that is a commonality between consumer growth and B2B SaaS.
The partnerships look very different. For B2B SaaS, you’re working with sales to figure out what are the levers or product experiences that you think differentiate your product when you have a sales conversation. And if they say there’s a particular feature set or there’s a particular user flow, then you, as a growth engineer, your goal is to guide users through that flow. Your goal is to ensure that the sales team is equipped with a full 360-degree view of what this customer’s history is, what are the things they might be interested in, and ensure that that book of business closes, which is made a lot more helpful when the sales team has that context.
On the marketing front, your goal is to ensure that not only are you steering folks to find what it is that they’re looking for on your marketing site, but you also ensure that folks are seeing the right pieces of content, guiding them throughout the discovery process for your product.
Similarly, your goal is to convert those people into users who sign up for your product or eventually folks who, they’re called hand-raisers in B2B SaaS, actually want to talk to sales and get a demo and start that sales process.
The final piece, which is interesting, is there’s a lot of talk around diversification of product bets. What I mean by that is your goal as a growth engineer should be taking small quick wins and medium-size bets that you’ve done some due diligence on de-risking, and then take a few big swings at large product bets for the sake of learning and discovering more.
A lot of that small tweaking—if you have a large enough user population, you can tweak button text and see lift in your experiment metrics—in B2B SaaS is potentially harder to do, just given the inherent nature of the product that you’re building and who you’re building for. So it tends to skew more towards those medium, semi-de-risked bets and then some larger big swings, in terms of building things for discovery or new unlocks of surface area.
What Growth Engineers Actually Do
Self-Serve and Product-Led Growth
We’ve seen a lot of companies that are trying to figure out how to build more successful product-led motions or move to a place where they’re putting the self-service product in front of people much more directly than they had in the past.
When the product becomes a channel to reach users it becomes critically important and every function in the business wants access to it. That can create conflict because there are different parts of the business vying for users’ attention. It’s hard to keep them all aligned. Have you seen that?
Every team has a user activity that they think is important or that they want to track. Marketing may have MQLS or marketing qualified leads. Sales may have sales-qualified accounts. What is that minimum threshold of activity within your product for this product-led, self-service growth motion?
Incentive alignment, especially in B2B SaaS, is actually really hard to do. Because ultimately marketing is goaled in a certain way, sales in another particular way, and you’re being pulled in two different directions. Depending on the size and scope of the team, a lot of B2B SaaS product orgs end up having a user acquisition team entirely focused on partnering with marketing. That way you can enable this team to avoid the sales thrash. Your goal is to entirely work with marketing to ensure that we get an appropriate amount of hand-raisers. We get an appropriate amount of signups. All of it is well attributed.
Similarly for sales, you build work more traditionally with existing accounts but the goal is to upsell them. We look at product feature usage and other leading indicators to identify more mature customers. The sales team can then figure out tactics to close that book of business.
In early days of PLG there was a sort of push and pull, with marketing site activity as well as in-product activity. But at the end of the day, good B2B SaaS growth engineers are able to give both the full picture of user activity—here’s all this marketing site activity, here’s the moment that they signed up for our product, and here’s all this in-product activity. That way when the deal closes, marketing can take some attribution for that. Similarly for sales, they say this came from our product, so it’s doing its job.
It can be hard to get statistical significance on a B2B product everywhere, especially outside of your core signup and onboarding funnel. You mentioned one of the differences between the types of bets that you might make at a consumer scale company and at a B2B company with lower user volumes are the small bets, or how you think about experimenting to make certain changes.
At Segment, how did you and the team approach experimentation from a growth engineering perspective? Were there things that you built or things that you put in place to be able to support that?
Yeah. We built a really lightweight experimentation, user segmentation tool. It bucketed a user in terms of an A or B group. What that did was fire off a track event that said what the experiment name was and what group the user was in. We had a consistent way of bucketing users so we didn’t reallocate a user from one group to the other.
For analyzing experiments we had a bunch of query templates that allowed you to punch in the experiment name, what timeframe or look back window you wanted, or what version the experiment was and it would do the split for you.
With regards to your question around statistical significance, you’re right. We would have to run experiments at Segment for eight or nine months in order to get statistical significance. That’s just debilitating for a growth team that inherently wants to move quickly and learn.
I really like this term ‘directional correctness’, which just means the experiment is trending in roughly the right direction. You do some back of the napkin math to figure out: “If things trended this way, assuming normal distribution of population, with this amount of users, here’s roughly what we would see in this metric. Here’s the potential downside in other metrics.”
We would essentially make the call on that ‘directional correctness’ rather than statistical significance. That allowed us to just wait a month or 2, instead of 8 or 9 months, and make a call and go from there.
Did that also change the types of things that merited an experiment, based on either what you expected the lift to be or where in the product you were actually making that change?
Yeah, totally. In general, if we thought that it was a better user experience and the metrics weren’t screaming “Oh no!” and the top line, money critical flows were more or less flat, or even if there was a slight drop off, we were fine. So long as we felt it was an enhanced user experience that also let us unlock an additional follow-up stream of work.
We weren’t just experimenting with CTAs, we were building more involved experiences but we still had to make sure the juice was worth the squeeze. We had a good grasp of what parts of the product were more trafficked than others and that gave us a north star to prioritize against.
What about the graduation of successful bets or an experiment? Where you’ve moved quickly and you’ve built something and you’ve proven a model out. What does that look like? Who owns that? Can you talk about what you’ve seen work there?
That is a big part, graduating experiments or bets. It can look like a whole new team dedicated and staffed to growing that metric even further or growing that line of business even further. It’s unlocking a new focus area for the team and pivoting what a team previously thought it knew into what was presently learned.
It helps with prioritization. When you figure out what to work on, you often ask yourself what has worked well in the past. What have we tried? What was the book of experiments that we ran before? That helps you iterate and push things forward from there.
For large enough experiments, it absolutely means unlocking a whole new way of thinking about a particular problem set. It unlocks potentially new teams and growing an org to tackle those business problems separately and to more deeply understand them. More often than not, it unlocks a new way of thinking. A new direction for the team that helps them be laser-focused and even more successful than they previously were.
Onboarding & Activation
We’ve seen a lot of teams looking at activation as their biggest lever to drive growth. The teams that we’re working with are trying to figure out how to tailor their onboarding and put that into action in the product.
Is that something you’ve looked at? Can you talk a little bit about what those bigger bets on the onboarding and activation front looked like for you?
That was something we actually struggled with quite a bit. We wanted dynamic audience definitions and it was really hard to get those. We had a way to build audience segmentation on Segment’s platform but it was really hard to get it back into our product. It was really easy to push it to external tools but to re-ingest it was always a challenge. We ended up with, unfortunately, pretty primitive user definitions. We had an onboarding questionnaire where we would ask what industry you are in, your job title, etc.
That was something where we wanted to experiment more with. We had the definition for really high-intent, highly-engaged users and were able to monitor those north star metrics. But what does a medium-engaged user look like? We never had a great proxy for that. We had access to our own production database to run queries and expose API endpoints against but that wasn’t, unfortunately, as sophisticated as “Gimme all the times that a user has come back to this particular page and clicked on this CTA”—that would indicate an insanely high-intent user action and we had fairly primitive methods of doing that, unfortunately.
If there were any proxies of user activity that weren’t event-oriented, but rather object-oriented, they’ve created a bunch of a certain resource for example, that was the main lever that we had. We could use that as a proxy to offer activation-type experiences. “Hey, you seem like you’re really enjoying the product. Would you like to talk to sales?”
In terms of the really sophisticated, “all users who did this but not this in the past seven days”, that’s awesome. Something that we would’ve loved a more sophisticated solution for.
What else was hard for you and the team when actually building this? Were there other challenges on the engineering front that you’d run into or things that were frustrating that you saw yourself or others on the team struggle with?
A big part was out-of-app coordination. That was tricky. We had an existing notification email service that was primarily transactional emails. If somebody deleted a resource or changed permissions on a resource, notify everybody else in the org, or those who were permitted to know, what just happened. It ended up becoming a grab bag though of fire and forget type emails as well—“Hey, so and so invited you to their team. You want to collaborate on it?”—and we could pass metadata there.
The issue was that we didn’t have visibility. For example, customer.io was a tool that was pretty widely used internally at Segment and we didn’t have great visibility into what emails were going out from customer.io. What was the proportion of internal, like email service related emails, versus third party emails?
Moving Fast and Taking Names
Hi Allen, I’m Joe, CTO and Co-Founder of Switchboard.
A really interesting aspect of your background is that you’ve worked on design systems and in growth. Design systems can be challenging to iterate and evolve rapidly and growth is at the other extreme. Iteration and speed and the pace of learning are crucial.
How did you navigate those two worlds when they came to a head and what lessons can those teams learn from one another?
I’ll give you an example from when I was on the core central UI team at Pinterest. We housed the design systems team as part of the larger team structure. When growth teams would put up code changes implementing their experiment and it was on a surface that we owned, which were some of the most highly traffic surfaces in the product, we would take a look at screenshots in the pull requests and be like okay, this is interesting. A lot of times we found ourselves meeting with growth engineers one on one to better understand what it was that they were trying to do.
Growth is very obsessed with moving metrics. A really quick line of thinking to move metrics is to just build something that catches a user’s eye. If we want people to save more or to do more of an action, a really common growth tactic is to stick a tool tip that’s really big and flashy. The inherent user behavior is, “What is this thing? I want to get rid of it.”
We found a lot of times the growth team would go down this line of implementation and you would end up with quite a few different UXs. Because we had that full, central context, we could tell other teams that growth actually tried this exact or similar experiment and what we recommended to them.
With design systems your goal is to ensure UX consistency. Growth challenges a lot of the design system and tries to mold it and break it in particular ways that are actually really impactful for evolving the design system and pushing it forward in ways that are interesting and make product building more diverse at the company. Ensuring UX consistency is actually a massive growth win too because users are met with a similar experience every single time. User expectation are preserved rather than broken, which oftentimes leads to negative user behaviors.
It’s this healthy tension. I found that there was a mutual benefit to growth teams really pushing what the current design system can do. Instead of building a one-off, working with the design systems team to incorporate different variants or different styles of a particular user component or components and other teams, later on, could reuse them. Everybody wins in that case.
Life as a Growth Engineer
In terms of what makes a good growth engineer, how big of a factor is the ability to jump into very different parts of the product and the codebase and productively move things forward? From a technical background/aptitude/way of thinking about the world standpoint, does that shape who is successful in the role?
Absolutely. I think it’s easy to get caught up in the nitty-gritty or how a full end-to-end system is built. Obviously, it’s great to understand the big picture and how different things fit together and where you need to go to make your change and have a really strong impact.
Product engineering teams tend to obsess and figure out how to evolve the systems that they currently own. Growth engineers need to quickly context switch and figure out how to be productive near immediately, in any given system in a particular engineering organization.
The goal is to not miss the forest for the trees and that is absolutely a skill that growth engineers need—to be thinking about the problem that they’re trying to solve, not thinking about the technical systems, or how to evolve them better. They need to stay really focused on quickly picking up what they need to know and knowing enough to be dangerous with it.
You talked about the unique aspect of growth engineering being immersed in product thinking and product work and almost acting as their own PMs in many cases.
Does that shape differences in what career progression within growth looks like? Where do growth engineers need to spike? How is that different from traditional models of product engineers progressing throughout their careers?
Certainly there are senior growth engineers who go down the very deeply technical path where maybe they’re building really sophisticated machine learning models based on user signals and intent and piping that through different services that other teams own. Absolutely that sort of technical depth of engineer exists.
There’s also the staff+ engineers and, in short, you think about them either in terms of technical depth or technical breadth. If you think about the whole product development life cycle, the breadth piece for growth engineers happens way further up from implementation than it does for a traditional staff+ product engineer, somebody working on infra or something like that.
For those who are incredibly product minded and want to have a really great impact on the business. Their growth is central to the business doing well.
You have to understand user behavior. You have to understand the whole context of what’s been tried before, what doesn’t work, why doesn’t it work. One of the running jokes is also just understanding a lot of how the world works. For example, we would see dips in certain countries on particular days by 20% or 40% of user traffic. So your encyclopedia of worldwide holidays grows quite a bit—just understanding those different pieces.
With staff+ engineers, your goal is to cross-pollinate ideas throughout the org to ensure that barriers are broken down. Growth engineering, as I alluded to earlier, is inherently doing that already. You’re iterating on other product teams’ surface areas, you’re sharing ideas that you’ve learned from talking to a particular team, you’re helping understand and shape other teams’ thinking and how their team operates. Being a bridge to the rest of the product engineering org—that’s really powerful.
As growth engineers get more senior, they develop a much deeper understanding of the business. They develop really rich, strategic thinking and are able to make a ton of impact because they’re really precise with how they tackle experimentation. They’re really methodical about how they do that and share that with the org. That way they’re not keeping that knowledge to themselves.
They’re informing other growth teams, other product engineering teams about what works, what doesn’t, and how the business moves forward—in addition to having their own outsized business impact as well.
I do tend to see really, really fantastic growth engineers become almost like advisors in the org. They go talk to different teams and understand how they’re experimenting with services. What’s working, what’s not. Offering strategic guidance, how to pivot, how to execute this experiment differently, understanding your user targeting better.
That’s super impactful, right? Because at that point you’re having impact beyond just your team, but in the entire org beyond growth as well.
Perception vs Reality
A recurring theme that’s come up is the perception of growth engineering. They move really quickly and they need to make some pretty hard trade-offs with respect to quality versus speed. They’re focused on the holistic journey and need to touch large swaths of the product and areas that other teams own. All of that can create pretty significant conflict and tension which you were just touching on. Do you think growth engineering has a stigma?
I don’t know if it is a stigma. I do think growth engineering shakes things up in an org. Conflict isn’t the right word but it’s diametrically opposed to how product engineering stays within their boundaries of the flows that they build. I do think it’s healthy in some ways. Because like you said, growth has to make these trade-offs between moving really quickly and learning a lot.
It’s good because growth is a pacesetter for other product engineering teams in the company. They see growth engineers moving really quickly, having tons of impact, stacking up wins. From a morale standpoint, that’s really important. Engineers want to see things ship. If you’re working on a team that’s moving slowly and growth is in your periphery, “Hey, what can we do to move things a little bit faster?” But growth engineers can look at product teams and understand the prioritization and due diligence that they do before releasing features. So, there’s a healthy medium at some point.
Growth engineers touch large swaths of the product. I think this cross-pollination of ideas is really important, especially for engineering orgs above a certain size.
Like I said, in product engineering teams it’s easy to stay within the boundaries of your team’s surface area. If you see a growth engineer talking to other teams, they could be knowledge-sharing what they’ve learned from another product engineering team with your team. All of a sudden that chasm has been crossed a little bit. I think that’s a really valuable and important skill or trait of growth engineers that gets overlooked quite a bit.
I couldn’t agree more. Teams operating in silos is the absolute worst. I love having the team altogether in one big monorepo so that they can jump around freely. It’s lovely.
What are the misconceptions you’ve seen come up with product engineering teams about growth work that you’ve had to combat?
Growth sometimes gets a bad rap for not being technical enough. It’s “Oh, growth engineering, you’re changing button colors. You’re adding a tooltip here.” Actually, growth is really, really challenging. And every engineer that I’ve talked to who’s made the transition from product engineering into growth has shared this exact sentiment.
If you think about the product engineering timeline or the different steps that it takes to release a feature out in the wild, there’s a lot of due diligence that happens way before implementation ever starts. It’s a lot of digging through metrics, understanding user behavior, making sense of what’s going on. It’s partnering with research to understand anecdotally what our customers are really liking. It’s vetting with design a bunch of different paths that a particular feature can go down. Then, it’s ensuring the success or understanding after something is implemented. Why is this thing not working? Why am I seeing drops in metrics that are unexpected?
Also, the pace of iterations is really, really fast. Growth teams are highly incentivized to keep turning out more and more experiments and unlocking new threads of learning.
There are really insane technical challenges in growth teams. A lot of times, multi-tier distributed systems for processing event data, experimentation platforms, SEO experiments at scale, and notification systems—these are built in growth. These things are really hard to build and oftentimes require deep knowledge of distributed systems, concurrency, etc.
Speaking from experience, I love building user products. For me, thinking about all of the technical and scalable implications of what I’m building are secondary. What I actually want to do is get things in the hands of users, let them try it out, play with it. Then worry about scalability later.
In general, product engineers aren’t used to thinking about all of the stuff that happens before a feature gets decided that it’s going to be worked on. A lot of times product engineering teams aren’t just handed features but, more or less, it’s relatively decided for them (and they can shape it a little after the fact).
Growth engineers have to practice this really uncomfortable art of being their own product manager in a way. You’re doing two jobs at once. That’s really, really tough and I don’t think a lot of people truly understand that aspect of building your own products or taking full end-to-end ownership of what that looks like.
You can find that at small startups. But growth engineering teams exist at scale and doing that at scale is really challenging. It’s a different way of thinking.
I think that’s probably the misconception that I often fight. Growth engineering is freaking hard, man. If you want to figure out what is the most impactful thing to work on? By all means I’ll let you go run crazy with it. It is really, really tough. Owning a lot of the job aspects of being your own PM or product manager, so to speak, is really challenging as well.
Real-world Challenges in Growth
If you think back, either to Pinterest or Segment, on some of the hardest challenges you and the engineering team solved within growth, what comes to mind?
At Pinterest it was a cool user mentality shift that we unlocked. There’s a little bit of context that I have to set. Pinterest inherently is a very individualistic product. A lot of what Pinterest prided itself on at the time was, while other social networks were primarily users interfacing with friends and getting this imposter syndrome seeing what other friends are doing—everybody’s trying to post their best life, Pinterest was much more idealistic and a lot more self-centered in a really positive way.
One of the big things that we unlocked in terms of a product/user comprehension standpoint was this notion that there are moments where Pinterest is actually ripe for collaboration or collaborating on the platform. Breaking this notion that Pinterest is strictly individualistic, for example, was one of the really interesting things. It wasn’t challenging from a technical standpoint but challenging from a product and growth standpoint—proving that this is a really impactful growth lever.
Not only is it organic but it’s not reliant on SEO or SEO content ranking. It’s not reliant on typical growth hacks, so to speak—“Oh, invite more friends to the platform, etc.” We thought about really rich use cases for collaboration. For example, if you’re planning your wedding on the platform, you might want to invite someone who’s responsible for the wedding dress, food vendors, beverage vendors, or the venue just to get an understanding of what your dream wedding could look like. So we built a lot of tooling around that—to up-vote certain pieces of content or to collaborate more richly on a particular pin, the base product unit for the company.
That was really interesting. Taking a step back—why do I think that was so challenging?
Growth is about not just you building the business. It’s about learning what could be possible for the business. What haven’t we tried yet that can be a big unlock for the business?
We were a more growth-y product team. We made really big product investments into those collaborative surfaces. We defined what they were, did a ton of research on them, and now it’s moving along as this new standalone growth channel, which is really exciting.
From a technical standpoint, it wasn’t necessarily any different than building product features from scratch. It was about this mentality and creativity around proving that this is a very healthy and exciting growth channel for the business, which I think is always a win. If you can get more users coming back to the platform to use a particular subset of features, everybody wins in that case.
Tools: Build vs. Buy
What was the technical architecture/codebase like at Segment and what parts of it were growth engineers contributing to and playing in?
We had a single-page react application, which was just our app front end. That was the authed product. Our marketing site was just static pages that were generated from a Next.js bundle. Each of those had its own design system. We had a dedicated, component library for the marketing site. We had a dedicated component library for the app. The app’s architecture then extended back. We had a GraphQL server that talked to a bunch of different services and we had what I call the microservices renaissance.
Segment, at the time, went really deep down the rabbit hole of microservices and built services dedicated sometimes almost just too small in scope in terms of the service boundaries. There’s a bunch of folks at Segment who wrote some really great blog posts about going down that rabbit hole and then walking it back to a pseudo-microservice architecture.
Where it landed when I was there was a somewhat monolithic service that housed all of the Segment-specific resources as a node express server with a bunch of custom goodies to make developing in it really fun and type-safe (API endpoints specifically).
We had a few other Go based services that were in our data plane. We had an authorization service that also housed a lot of our invite logic, etc. It was pretty intense with lots of different roles, and lots of different hierarchies of permissions. We had a dedicated email service as well.
To your point earlier, our changes oftentimes required touching 5, 6, 7 different things. Setting up experiment logic in our app front end, building new GraphQL endpoints, new queries and resolvers. Then build the corresponding API endpoints for our control plane service. Then building the resources in the service that the control plane talked to, that the monolithic service talked to, if it didn’t already house the existing data models.
There was a lot! Our team moved really quickly—oftentimes, you would see a stack of 6 diffs adding a field to an existing model and then piping that all the way down to the front end.
It was just the nature of the game. Segment cared a lot about the engineering team. Cared a lot about engineers being super, super productive. So what sounds like, “Oh man, 6 diffs just to add a simple field change on a model!” wasn’t all that bad. You do it in an hour or two and move on. But it definitely required a lot of knowledge about where service boundaries were and what was appropriate to throw in a particular service or not.
What about tools or other desires that those teams have to actually get more direct about reaching users in the product? Like being able to message users in the product or show them experiences—I know a lot of companies will bring tools in to do that.
As a growth engineer, you sit at that intersection point where you’re responsible both for building the experiences that the product team wants to use to onboard and activate users, but also having to deal with integrating and testing these third-party tools.
There’s a current challenge right now in the market with these tools. There are quite a few of these pop-ups, ‘offers’ is what I’ve heard them called, where you offer a discounted upgrade path for really high intent users.
A lot of these tools though are no-code and engineers don’t have full visibility into what these tools are doing and how they’re being implemented in the product. That’s really challenging because customer success managers, or folks focused on in-product growth, will bring a no-code, out-of-the-box tool that isn’t integrated tightly with the code. Engineers also implement some of these things themselves. You end up with a lot of conflicts and clashes—potentially really messy.
Tools that allow users, especially growth engineers, to bridge that gap where those who are focused on product growth can configure what these experiments might look like, but the actual implementation and instrumentation exist in code. That’s a really powerful concept and something that every growth engineer needs in their toolbox.
Were there other places where those build vs buy questions came up? How did you navigate those or make those types of decisions?
Definitely in terms of experimentation tooling. That was the main one. We had a couple of others around sales tooling, like outreach and how we would send off triggered emails or email campaigns.
We were lucky at Segment to have a customer success team dedicated entirely to self-service, which was really powerful. If there was anything that they could do ‘out of app’ that would save us from having to build something ourselves, then we could just focus on product experiences that would be complemented by a great email or drip campaign.
We were really focused on building and we had a reasonable infrastructure already in place, at least in terms of logging. We were lucky to use Segment for Segment. We used our own Personas product (it’s now called Engage) as our audience and cohorting tool. For anything that we didn’t already have immediately at our disposal, we would typically lean on our Customer Success counterparts because they were really good at navigating vendors and they would consult us from an engineering standpoint. We tended to lean on them to ensure that we weren’t busy building internal tools when we could just pull something off the shelf.
Thank you, Allen! One final question before we wrap: what’s something surprising about you that most people don’t know? It doesn’t need to be growth engineering related!
Oh man. I think what’s surprising… I was in a collegiate acapella group. I don’t sing. I beatbox, which I thought was a strange skill that I picked up randomly.
Also, I’m a diehard Houston sports fan. All professional Teams from Houston—I fully support. It’s strange because I was born in Houston, then was raised in DC, went to college in Illinois, and then came out to California where I’ve been since. So I’m this strange mutt of geographies that always takes people aback.
Those are awesome and I want to hear you beatbox at some point! Let me close this off by saying thank you again, Allen. We really appreciate you joining us today!
Of course. Pleasure being here!