I recently had a conversation with a tech futurist who predicted that AI would take over most software development within five years. From a capability standpoint, he’s probably right. But whether companies can actually leverage that capability is a different question entirely.

The companies that will win aren’t the ones with the best AI tools or the most aggressive adoption strategies. They’ll be the ones with the best tech systems: the underlying architecture, infrastructure, testing, and evelopment processes that make it possible to move fast at scale.

This isn’t a new insight. Amazon figured this out 20 years ago in the pre-AI world. But AI is about to make it mandatory instead of optional.

Subscribe now

The Amazon Lesson

Amazon’s dominance didn’t come from better retail strategy or more inventory. It came from disciplined investment in technical systems that let them move faster than anyone else.

They mandated service-oriented architecture when it was painful and expensive. They built deployment systems that let teams ship independently. They created infrastructure that could scale. They invested in these things not because they had immediate ROI, but because they knew these systems would become force multipliers.

The payoff took years, but when it came, it was massive. While competitors were still coordinating deployments and dealing with monolithic architecture, Amazon could experiment rapidly, launch new services, and scale effortlessly. AWS itself only became possible because of that systems investment.

The key insight: Amazon treated systems as strategic infrastructure, not technical overhead.

Most companies haven’t made that investment because they haven’t had to. If your bottleneck is building features, you optimize for feature velocity. Clean architecture and robust infrastructure are things you’ll “get to later.” This has been a rational choice for decades.

AI is about to make it irrational.

The Systems Divide

Good tech systems have always been valuable. But AI fundamentally changes the ROI calculation. This is an inflection point.

With human developers, your velocity is roughly linear with headcount. Better systems help, but you can also just hire more people. The constraint is people, and people are expensive but available.

With AI, the constraint is different. AI can generate enormous amounts of code, but only if it has the context and structure to do so safely. The multiplier effect of AI on a well-structured system is dramatically higher than on a messy one.

Think about what AI needs to be effective: clear architectural boundaries, explicit contracts between components, comprehensive testing that verifies correctness, deployment systems that make shipping safe, and codebases that are understandable without months of institutional knowledge.

These are exactly the characteristics of mature technical systems.

Companies with good systems can use AI to multiply their engineering capacity. They can move faster, experiment more, and deliver more with the same or smaller teams. AI becomes a genuine force multiplier.

Companies with messy systems get marginal gains. AI helps with code generation, but the bottlenecks (understanding, coordinating, verifying) remain. The fundamental constraint isn’t addressed.

This creates a compounding advantage. Companies that invested in systems can leverage AI to pull further ahead. Companies that didn’t invest are stuck with the same limitations they had before, while their competitors accelerate.

What Changes in Practice

Let me be concrete about what this advantage looks like.

A company with decomposable architecture can use AI to build new services rapidly because each service has clear boundaries and contracts. A company with tangled dependencies needs humans to carefully coordinate every change.

A company with comprehensive testing can trust AI-generated code because the tests verify correctness. A company with sparse or flaky tests needs extensive manual verification for every change.

A company with robust CI/CD can ship continuously because the deployment process is reliable and automated. A company with brittle deployments still needs careful human oversight and coordination.

In every case, the difference isn’t about the AI capabilities, it’s about the systems that enable AI to be useful.

The gap between these companies will widen rapidly. The compounding works both directions. Companies moving faster can invest more in both features and system improvements. Companies moving slower fall further behind because they’re consumed with just maintaining what they have. It’s the fundamental difference between virtuous and vicious cycles.

The Window Is Closing

Here’s what I expect to happen over the next five years:

The companies that invest in tech systems now will pull away from everyone else. Not gradually, dramatically. They’ll be able to leverage AI in ways that companies with messy systems simply cannot.

This will become obvious to the market. Initially, some companies will have a mysterious advantage in shipping velocity and product quality (we are already starting to see this). Within a few years, it will be clear that the advantage comes from systems investment.

At that point, it becomes table stakes. Every company will need to invest in these systems to compete. But the companies that invested early will have a massive lead. They’ll have years of compound advantage: better products, more customers, more resources to invest in staying ahead.

The companies that wait will find themselves in a difficult position. They’ll need to invest in catching up while simultaneously competing with companies that are already leveraging AI effectively. It’s not impossible, but it’s expensive and slow.

This is fundamentally a question of strategic timing. The capability of AI is improving rapidly. The companies that prepare their systems now will be ready to leverage those capabilities as they arrive. The companies that wait will be scrambling to catch up while their competitors are already pulling away.

What This Means for Leaders

If you’re a CEO or board member, this is a strategic investment question. Technical systems have traditionally been seen as cost centers or operational necessities. That framing is about to be wrong.

Systems investment is about to become a source of competitive advantage on par with product strategy, market positioning, or sales execution. The companies that treat it that way will win. The companies that continue to see it as overhead will struggle.

This means some uncomfortable conversations. Engineering leaders need budget and time to invest in architecture, testing, and infrastructure. These investments don’t produce immediate features. They produce the capacity to move faster in the future.

For CTOs and VPs of Engineering, this is your moment to make the case. The business case for systems investment has historically been difficult: it’s hard to prove ROI on refactoring or testing infrastructure. AI changes that equation. The companies that invest in systems will be able to leverage AI as a force multiplier. The ones that don’t will get marginal gains at best.

The evidence will be clear over the next few years. The question is whether you invest now while there’s still time to build an advantage, or later when you’re trying to catch up.

The Real Opportunity

I want to be clear about what I’m arguing. This isn’t about fixing broken engineering teams or paying down technical debt. It’s about recognizing that the rules of competition are changing.

For the past few decades, software velocity has been primarily about headcount and talent. The companies that could hire and retain the best engineers had an advantage. That’s still true, but it’s about to become insufficient.

The next competitive advantage is systems. The companies that invest in mature architecture, robust testing, reliable infrastructure, and clear development processes will be able to leverage AI in ways that dramatically multiply their engineering capacity.

This is a genuine strategic opportunity. Not every company can be first to market. Not every company can out-hire their competitors. But most companies can choose to invest in technical systems. The companies that make that choice now will be the ones that dominate their markets in five years.

The companies that don’t will find themselves perpetually behind, trying to compete with organizations that can ship faster, experiment more, and adapt more quickly. It won’t be impossible to catch up, but it will be expensive and painful.

Amazon figured this out 20 years ago, and it made them dominant. The companies that figure it out now, before AI capabilities fully mature, will have the same kind of compounding advantage.

The window to be early is closing. But it’s still open.

It's apparent that many of the companies that have tried to adopt AI in the past few years, haven't had the success they were hoping for. There's often initial promise, followed by a Wylie-Coyote-style fall off the AI cliff.

Coming from 20+ years working with AI/ML systems, one important reason for this is clear: Adopting AI is never as simple as introducing a new tool. It's a journey that touches the whole organization, from needing to clarify strategic goals, to understanding and updating processes, to building a platform/infrastructure layer to support the technology.

This piece is about that journey: what it means for companies, what questions to ask yourself and your colleagues, and what to watch out for to make your path as smooth as possible.

First, Clarify your Goals

Especially using AI and automation for efficiency (either internally or in a SaaS product), you need to carefully think through the fundamentals - what pain do people have, how are you solving that pain, and, do you need AI to solve it?

Why? Because AI tools are inherently non-deterministic, meaning that they look for patterns and apply them. They all "hallucinate" when the patterns they have uncovered don't apply correctly 100% of the time (which they never do). That means these systems are more expensive and complex to use for business operations than standard, deterministic software. If you apply the slight randomization introduced by ML/AI systems to your (or your customers’) critical business processes, it will be a disaster if you don't know what you're trying to accomplish.

One example: At a machine learning company I worked at, we had a system that was applying algorithms to essentially sift through very large data sets and propose actions to our customers, some of which could be fully automated in workflows (including by maintaining state and chaining automations together).

It wasn't working very well. The error rate was high, the proposed actions were often not correct, and overall, it needed enough hand-holding that it wasn't realizing the promise of streamlining operations and reducing manual effort.

To fix this, we first ripped out the machine learning to make sure we had the business process right for our customers before trying to make it "smart." During that process, we realized that we hadn’t fully understood the goals and current processes, which was making it very hard to use ML effectively. Once we completed this process and added ML back in strategically, the whole system was much more effective because we were using the right tools for the job, and, we knew what exactly the job was.

Here are some questions to ask yourself and your colleagues:

• Are you sure you know what problem(s) you’re trying to solve?

• Are there multiple ways to solve the problem? If so, which might be simplest?

• Does it need AI, or are there other, deterministic, ways to approach the problem?

• How sensitive is this problem from a business perspective? Can you manage the risk?

Second, Take an AI-First Perspective

Using AI requires a different way of thinking about processes than more standard approaches do. This is because of the non-determinism (discussed above), and the fact that often when we are adopting AI for business processes, we intend for it to operate in a largely automated fashion, which means accuracy at scale is key.

There have been a lot of reports of companies giving employees generative AI tools hoping for efficiency gains, just to get complaints about overwork and difficulty getting benefit from the tools instead. The reason behind this is clear: it's because in order to use AI effectively, the process has to change. It's not the case that employees can just use AI and automatically reduce their workload.

This means you need to fully understand the processes you're automating, determine where AI will have the most impact, and then decide how you will automate overall; where you will use AI, how you will control quality, where you will have humans in the loop, etc.

Also, AI needs data. If you want to run business processes with AI, you need to make sure the system can get the data it needs, it can keep state if needed, the outputs end up in the right places, and that the appropriate quality controls are in place, based on business sensitivity. Understanding these details will help you determine what kind of technical solution you need and how robust it has to be.

I've gone through the full "AI-ification" of large-scale business processes end-to-end twice (multi-year projects) and consulted on several other such projects. In both the E2E cases, we had to map out the steps in all processes to make flows more explicit and include details about data and where humans fit in. Then there was tweaking to really get things working. This can take a lot longer than you think (many months in some cases).

The end products were much more streamlined with better throughput, but it took time to get it right. If you just give AI to your team and tell them to go for it, results will definitely vary based on the people you have using it. If you want consistency and real team-wide efficiency, that takes process design.

Questions to ask:

• Do we know how we would (or do) solve it manually?

• Do we know (exactly) what data we need and where it is?

• Do we need to create outputs that will be used in other inputs, or that will need to be inspected by humans?

• Do we know what controls (compliance, risk, etc.) we currently have in place? Will this be sufficient for automation and AI?

Third, Understand the Technology Options

There are lots of different ways and degrees to use AI. And a lot of them require at least some level of "platform." By platform here, I mean the technology used to move data around, build and run models, drive automation, manage quality, and enable human-AI interfaces. Essentially, the infrastructure making the AI-ificiation of the business processes possible.

There are a number of ways to approach this. First, before making technology decisions there are some questions to ask:

• Is this process business critical? If so, you need a solution with better scalability and robustness.

• How important is accuracy? Does it need to be 100% correct, or is 80-90% okay? If it needs to be above, say, 85% on average, you also need to put rules and quality controls into place.

• How many people will be involved and at what stages of the process? You have to explicitly plan for people in the system, which affects technology decisions.

• How much and how often does the process change? Is it very static, or does it require flexibility? The more flexibility you need, the more "platformy" your solution needs to be.

• How fast and how frequent does it need to be? Does it run every day? Every second? How long can you afford to wait for results? The faster it needs to be, the more "platformy" you need to be.

Once you understand this, you can start thinking through the options. There is essentially a spectrum, from simple human prompt usage, to Airtable or similar tools that loosely chain together tools and people, to more automated and robust AI orchestration platforms.

There's a lot to learn here, and it deserves it's own piece, but the main point is that you have to understand what you're trying to do before you can choose an approach. And the available approaches vary a lot in terms of implementation time, cost and capabilities. Basically, the more fast, robust, business-critical, automated, and frequent a process needs to be, the more you need a robust platform-based solution.

In any case, this is usually a crawl-walk-jog-run adoption path, as opposed to just going all-in on a platform from day one.

Finally, Plan for the Crawl-Walk-Jog-Run of AI Adoption

Most successful AI adoptions I've seen follow a pretty predictable pattern. Companies that try to skip steps usually end up back at the beginning, just with less budget and more skeptical stakeholders.

Crawl: Individual Usage and Experimentation This is where most companies start - and where many get stuck. People use ChatGPT or Claude for individual tasks. Maybe you set up some team subscriptions. You run a few hackathons. Some people love it, others don't see the point. Usage is inconsistent and results vary wildly based on who's using the tools and how skilled they are at prompting.

This stage is valuable for building familiarity and identifying where AI might help, but it's not where you'll see real business impact. The key here is to capture what you learn and start identifying patterns in where AI is actually helpful versus where it's just novelty.

Walk: Process Integration and Tooling This is where you start building AI into specific business processes. You might create custom prompts for common tasks, build simple workflows that combine AI with human review, or integrate AI tools directly into your existing software stack.

At PURE, we built a VS Code plugins that help with code review, commit messages, and unit testing. And we adjusted our development processes to use these tools in a consistent way across the organization. We made adoption into a project, as opposed to just making the tools available.

The goal here is to reduce friction and increase consistency. Instead of everyone figuring out their own approaches, you're creating shared tools and processes that work the same way every time.

Jog: Systematic Process Redesign This is where you start actually changing how work gets done, not just adding AI on top of existing processes. You map out your (or your customer’s) workflows, identify where AI can take over certain steps entirely, and design quality control mechanisms to catch errors.

This stage requires real investment - in time, people, and often technology infrastructure. You're not just using AI, you're rebuilding processes around it. But this is also where you start seeing real efficiency gains and business impact.

Run: AI-Native Operations This is the end state that most companies are aiming for - where AI is deeply integrated into how the business operates. Processes are designed from the ground up to leverage AI capabilities. Quality control is automated. Human oversight is strategic rather than tactical.

Few companies have actually reached this stage yet, partly because the technology is still evolving, and partly because it requires fundamental changes to how organizations work.

What Success Looks Like

Successful AI adoption doesn't look like replacing humans with robots. It looks like humans and AI working together in ways that leverage the strengths of both.

The companies I've seen succeed focus on:

• Clear, measurable goals for what they want AI to accomplish

• Deep understanding of their current processes before trying to change them

• Systematic approaches to technology decisions and quality control

• Patience with the learning curve - both technical and organizational

The AI adoption cliff is real, but it's avoidable. The key is treating AI adoption as an organizational capability you're building, not a technology you're deploying. Take the time to do it right, and you'll get results that actually matter to your business.

Subscribe now

This transformation is likely to accelerate the need for platform products.

What Makes a Platform Product? The development of user-focused platforms falls along a continuum, from point solutions to full use-case-agnostic platforms like AWS. Somewhere in the middle fall platform products - multi-use-case SaaS products that enable multiple use cases within a domain, in a highly interconnected way with considerable customizability.

Three key characteristics define platform products:

1. Multiple connected use cases within a domain

2. Strong data flows between these use cases

3. Modular, swappable components that preserve business workflows and provide flexibility

The Historical Challenge Building platform products has traditionally been hard. It requires significant upfront investment in architecture and often means slower time to market. Most startups can't afford to spend years building a platform backend before launching their first product. This has led many companies to start with point solutions and try to evolve into platforms later, often with mixed results. Regardless of the difficulty of transitioning towards platform architecture over time, it has typically not made sense to focus too much on platform capabilities early in a product’s life because of the cost.

The AI Shift

AI, and especially AI agents, will make this investment make more sense.

Why? Because AI is changing how we think about software architecture. Instead of hardcoding business logic into individual applications, AI can handle complex workflows across multiple systems. Take Microsoft's example of Python in Excel - it's not just about adding AI to spreadsheets. It's about Excel becoming part of a larger system where AI agents can plan, analyze, and execute across different tools.

This shift has three major implications:

1. Business logic becomes fluid. Instead of being locked into specific applications, rules and workflows can be managed by AI across systems. Also, user expectations are already moving towards fluidity. Adding new capabilities seems easier than ever from a user-perspective, so the expectation that all software should work that way is increasing.

2. Data needs to flow. AI needs access to data across different use cases to be effective. The old model of data silos in individual applications doesn't work.

3. Tools become interchangeable. When the intelligence lives in the AI layer, individual tools can be swapped out more easily while maintaining the overall business workflow. Also, the rapid pace of technology change requires businesses to stay nimble with respect to the tools and approaches they use.

Why This Leads to Platform Products

This AI-driven shift naturally pushes software products toward platform architectures. You now need to chain together different AI systems, layer in traditional machine learning and workflow automation, and manage quality to avoid embarrassing hallucinations and errors.

Consider what happens when AI starts managing more of your business processes:

First, you need connected data. If you're using AI to help with property management, for example, it needs to understand maintenance requests, accounting data, and tenant communications together - not as separate silos, but all integrated and available everywhere. This forces you to think about data flows and interconnectivity upfront.

Second, workflows become more fluid. AI can handle complex, multi-step processes that cross traditional software boundaries. A single user request might touch your accounting system, maintenance tracking, and communication tools. Your architecture needs to support this flexibility.

Third, tools need to be modular. As AI gets better at handling business logic, you'll want to be able to upgrade or swap out individual components without disrupting the overall workflow. Maybe you want to try a new machine learning model for maintenance predictions, or upgrade your communication tools. A platform architecture makes this possible.

The end result? Products that wouldn’t have originally been on the platform spectrum at all start to look a lot like the platform products we defined earlier - multiple connected use cases, strong data flows, and modular components. But instead of this being a nice-to-have, AI makes it almost a requirement.

What This Means for Practitioners

If you're building software products today, this shift has important implications for how you approach development. The standard advice of "start focused and add platform capabilities later" will need to be updated.

Start Platform Thinking Early You don't need to build a full platform from day one, but you do need to think like a platform product from the start. This means:

• Design your data architecture thoughtfully. How will different types of data connect? What schemas will support multiple use cases?

• Build good API hygiene early. Use APIs internally even if you're not exposing them yet.

• Think modular. Design components that can be upgraded or swapped and connected to each other in different ways without breaking the whole system.

Balance Speed and Architecture Most startups still need to get to market quickly. The trick is knowing what to invest in early versus what can wait:

Do Early:

• Data modeling that supports multiple use cases

• Basic API structure

• Clear component boundaries

Can Wait:

• Full platform infrastructure

• Multiple use case support

• Advanced customization capabilities

Learn from Others' Journeys Companies that have tried to retrofit platform capabilities later often struggle. It's much harder to restructure data and workflows once they're embedded in your architecture. At the same time, companies that spend years building perfect platforms often never make it to market.

The sweet spot? Build for your initial use case, but architect with platform principles in mind. This approach costs a bit more up front but saves massive headaches later - especially as you start adding AI capabilities.

The Future is Platform-Shaped As AI continues to reshape how we build software, the line between applications will blur. Success will depend less on individual features and more on how well your product supports AI-driven workflows across use cases. Starting with platform thinking isn't just good architecture - it's becoming a strategic necessity.

An acquaintance asked recently how I get a handle on product and build out the roadmap when I join a new company (he’s a CTO moving into a CPTO role). Here's my process, based on experience at B2B and enterprise scale-ups/start-ups:

1. Plan for a plan

2. Team interviews

3. Customer interviews

4. North star goals and key pillars

5. Roadmap

6. Start the drumbeat

Plan for a Plan

The first thing to do when starting as head of product, assuming there isn't an immediate emergency to deal with and assuming you have a team in place, is to lay out how you will do the investigations that will result in a roadmap. I find that stakeholders feel better when they understand what's coming and what the milestones are.

First, spend time with key stakeholders to understand goals, challenges and expectations. Then, lay out goals and set a tentative timeline so that stakeholders have something to orient towards in your periodic update meetings (which you should be having). The plan can change as you go along (as you will see in the example below), but just having something to track to helps a lot.

Here’s an example of an initial plan-for-a-plan timeline for an internal product:

In this particular case, after getting up to speed on the ongoing development and plan (which was very early-stage), I came up with a hypothesis (“straw man”) about what the go-forward plan might be based on what I had been told before joining. Then I spent what ended up being a couple of weeks doing in depth interviews with the team and internal customers of this proposed product to test and update my understanding and hypotheses.

Over time, I learned that the project had been under-scoped and I took a step back with the overall plan and re-did this timeline to add a deeper research process (it ended up being about 6 weeks longer than originally planned). It’s fine to make adjustments to the plan as you go - the main point is giving visibility to stakeholders so that they know what is happening and what you are learning.

This slide becomes the artifact that you review with stakeholders at weekly or bi-weekly status meetings to help everyone stay aligned about where you are in the process.

Team Interviews

Joining a new team is a great opportunity to listen, learn, and set the foundation for any potential adjustments in the future. Listening is key here. You want to understand:

• What are people working on?

• Why they are working on it?

• What they think the company goals are (for tech and in general)?

• How they think the work of the team moves those goals forward?

• What processes are being followed in the team?

• How decisions get made?

• What friction is there in the planning and development processes?

• How people feel about what they’re doing and how they’re doing it?

• etc…

These interviews should span people from across the team - not just product. You need to understand the team and the ongoing work holistically. Assuming you have an engineering counterpart, this is also a great opportunity to start building trust in that key relationship by listening, asking good questions, understanding any challenges, and agreeing on how your individual and team interactions will work.

Also, I like to create artifacts as I do these interviews. If someone explains a process, I diagram it. If they tell me about a feature, I update the documentation if needed. This is a form of active listening that gives the team a sense that you are adding value from the get-go, and it does actually also add value.

Customer interviews

For external customer interviews, assuming you have some, I usually collaborate with sales and/or customer success to find appropriate customers. Do your research beforehand to understand the product, how it is used (using Hotjar or similar if available), the differentiators, how customer relationships work at your company, how delivery has been (do customers get what they are promised on time), etc. Also, it’s a good opportunity to start building relationships with the revenue teams and show how you engage with customers and the sales process.

For customers, this can be an opportunity to provide feedback and suggestions for the roadmap. Most customers appreciate meeting the person leading product and like to be involved in making the product they use better.

Be aware that, depending on how things have been going, you might meet annoyed people (for example, I once worked at a company where customers had been promised certain features for 2 years that were not even on the roadmap). Usually, there is a reason the company is hiring a new head of product, and you may find out more about why as you go through this process. Regardless, this is a good opportunity to listen and to come back to customers with a plan. It is obvious, but always under-promise and over-deliver to customers. If you can’t fix a problem and/or you don’t have timelines that you can stand behind, don’t promise. Be honest and knowledgeable (because you did your homework), and focus on building a good relationships. These initial interviews are probably with important and/or friendly customers, so they may be potential members for a customer council or people you will want feedback from in the future.

In terms of questions to ask, it will depend, but I like to understand why the customer chose this product, what they use it for, what other products they considered, how their experience has been, what feature requests they might have, what they know about the roadmap, what they like most/least, and what they’re looking forward to. It’s critical to understand what is top-of-mind about your company to customers and what they expect, both now and from past conversations with the team.

Subscribe now

North Star and Pillars

A key part of my process for aligning stakeholders are teams is having a north star goal (usually for the year, but can be less if the project is newer) with execution pillars. Essentially team OKRs. I usually make one slide that captures this and then show it all the time.

Here’s an example for an early-stage machine learning cybersecurity product. In this case, our hypothesis was that getting to 25 customers was related to improving our adversary detection and the cycle time for our proof-of-value process for potential customers.

We put together the goals and then set up a system (with specific KPIs) to measure our progress. At our weekly team meeting for that 6-month period, we tracked these goals and ensured everyone on the team was working on something related to one of them. By the end of the period, we got to 22 of the desired 25 new customers.

This approach makes it a lot easier to get to the full roadmap (because everyone is already aligned at a high-level), and it is much better for alignment because people don’t have to keep all the details in their heads.

Roadmap

Once you’ve gotten agreement on what the team is targeting and how it relates to company goals, you’re ready to make the roadmap. There is tons of content on how to do this, so I won’t go into much detail here. I typically use Janna Bastow’s Now-Next-Later roadmap approach and avoid timelines if possible. If timelines are necessary (which they often still are), I bucket items into themes/pillars and stack rank - with a very clear focus the overall goals. I find that keeping attention on the north star goals and pillars, and ensuring all stakeholders essentially have these memorized, makes it possible to be more flexible at the feature level. When stakeholders know what goals are being achieved, they don’t worry as much about the exact set of features needed to make that happen.

Note, if the team is not already in place, I usually sketch out the goals and pillars and then a very high-level sketch of the features needed to achieve those goals (instead of a full roadmap). I use this to make a staffing plan and start hiring. A roadmap without a team is kind of pointless.

Starting the Drumbeat

Finally, once you have buy-in on the goals and roadmap, you’re ready to get the drumbeat going. In my view, this is the most critical step in actually turning the goals into reality. My target here is to ensure that 90+% of the team and all the key decision makers are able to answer questions about what the customer story is, what the goals are, how the goals will be achieved (at a high level), and, for the technical team, how the thing they are specifically working on maps to the goals for the team and company. I’ve found that if you can achieve this milestone, the right things just start to happen naturally because people are genuinely aligned.

To do this, I use KPI dashboards, posters, and repeated goal slides (one slide they see all the time for months - I usually show it at the beginning of all scrums and team meetings). Posters can focus on the team goals, personas, even operating principals (like the one at the top) - whatever needs to be top of mind to ensure the goals get achieved.

The main point is alignment. We are all doing this thing together and we’re 100% focused on making it happen. It goes without saying that an important part of your role as head of product is to manage up and ensure that the alignment holds at the management level, and that the team actually has enough time to realize the goal. If you keep things super clear and transparent, and you choose a company with solid/experienced management, this is usually possible.

The 90 Day Mark

That’s it. By the 90 day mark, I try to have this drumbeat going with the existing team, or be underway with hiring and building out a new team if needed. Sometimes, it takes longer, but if stakeholders are clear about the process and where you are on the journey, it usually works out fine. I would love to hear what process you use and whether you have alternative approaches to any of the steps here.

Subscribe now

As AI adoption increases, there seems to have been an uptick in product managers focused on machine learning science work. There doesn’t seem to be much content available on this subject, so wanted to put down some thoughts. I’ve been the product manager for three different ML science teams over the years. The first time I served in this role, there was a lot of trial and error, some of which could certainly be avoided.

ML/Science Teams are Different

The first big thing to know is that being a PM for a science team is very different than working with a feature team.

• Their timelines can be very long and more open-ended. Some projects can last over a year without many agile-style deliverables.

• Not every project works out. Sometimes there’s research with a hope to finding a particular approach or making a breakthrough, and it just doesn’t pan out. This is expected, but makes it different from feature teams, where finding that something is not doable at all usually doesn’t happen.

• There’s an important translation layer across teams. The value of science exploration isn’t always immediately obvious, though the impact can be extremely high. This means that more communication legwork is needed to ensure that stakeholders across the organization understand the potential benefits and how they will be realized. Also, additional development work is often needed to productionize science work once the results have been achieved, so detailed involvement with the relevant feature teams is also important.

The upshot of all this is that the science PM plays a key role in ensuring the considerable value of science work is realized, and that their work doesn’t disappear into an expensive black hole.

How to Get Started

Get the Lay of the Land

If you're new to product managing science teams, here's where to start:

First, talk to your organization’s leaders, and understand what "valuable" looks like to them. In my experience, value is typically measured in two ways:

• New products or features: Did the research result in a sellable new feature?

• Improved quality: Did the work enhance the overall product quality or make it "smarter"?

Then, engage deeply with the science team. Learn what they're working on and why. You don’t need to be technical to do this, but you do need to ask good questions, be an active listener, and demonstrate the value of a non-technical perspective. For example, if someone explains an architecture to you, come back with a diagram of it. This will allow you to check your understanding, and you will have produced a valuable communication artifact for the rest of the team. Similarly, if someone explains something to you, produce documentation that benefits the team and the rest of the company. Make yourself indispensable to the team and their stakeholders.

Once you understand what is going on in the team, categorize the work into foundational research versus product-focused efforts. Understand, in a deep way, how the team feels their work will benefit the organization and what needs to happen in order to realize that value. For foundational work, this path might be long and the value less clear. For product-focused work, the path should be straighter. For every project, ask a lot of questions and try to really understand the value and how you can champion it across the company.

Once you know what all the projects are, you can start working through the roadmap…

Build a Research Roadmap

Once you have this information, you can start building the roadmap. Make it outcome-focused with clear milestones, rather than strictly time-boxed. Ask yourself: What's the desired outcome of each project? What process are they following? What are the critical decision points? What needs to happen for the work to land in the product or benefit the platform/team in some way?

For each project, create a research plan. This helps everyone understand the project's trajectory and goals. A good research plan includes:

• A clear statement of the research goal

• Breakdown of milestones (decision points, not deliverables)

For example, in a project aimed at improving unstructured data parsing, your plan might look like this:

1. Goal: Improve parsing of a special kind of unstructured data into tables

2. Milestone 1: Investigate parsing options - we will do experiments A, B, C, and D and then reassess.

3. Decision Point: If we can't find an approach meeting our criteria after these 4 experiments, reassess and possibly abandon. If successful, move to next phase.

4. Milestone 2: Develop streaming table population method for the parsed data... and so on.

Doing it this way will allow you to report on the overall plan and on the outcome of each experiments during your stakeholder update meetings. It will make the work more concrete and allow everyone involved to understand progress.

When balancing foundational and product-focused work, I aim for about 70% product-focused. It adds value more quickly and clearly. However, this can vary. For teams working with very new technologies, the percentage of foundational work might be higher.

For foundational research, it's crucial to map out clear milestones and benefits, and be very clear under what circumstances the project would be cut off, even if this isn’t easy to do. This prevents stakeholders from feeling like they're funding a black hole and helps the project from getting cut prematurely. Be prepared to explain the potential long-term value of this work and why it’s worth a more open-ended investigation.

Help Ensure Science Work Adds Value

One of the trickiest aspects of the science PM role is managing interfaces between the science team and the rest of the company. You need to maximize the chances of the outputs being useful. This might involve:

• Coordinating early with other teams on how to incorporate the research into production software.

• Educating others about the science team's work and its importance.

• Telling great stories about the benefits and how they will transform the product or company to get stakeholders excited.

For example, I once worked with a science team that struggled to get their work into production. They'd complete research, pass it to feature teams, and nothing would happen. They were confused about this, because they knew their work was valuable and could have a meaningful impact to the bottom line. Talking with the rest of the team, it was clear that the understanding of what they were doing and how it would benefit was not widespread. Additionally, the code coming out of the science team was not production ready, it was in a different programming language, and it wasn’t clear enough how to productionize it. To address this, I first clarified the research objectives and provided more context to leadership and teams about what was being done. Then I worked with both sides to smooth the integration process by helping the science team write their code in a way that would be easier to move into the platform, and by helping the feature team plan the migration and build time into the schedule to get features into the platform. This reduced friction and accelerated the path to value.

If a project doesn't pan out as expected, communicate it promptly (you should be having bi-weekly or monthly check-in meetings with key stakeholders). As soon as you hit a failed milestone, inform stakeholders and either revise the research path or conclude the project. If you have set clear decision points and have been providing regular updates, this shouldn't come as a surprise.

Have Fun

Science PM work can be incredibly rewarding. You're not just shipping features; you're nurturing new ideas and bridging the gap between cutting-edge research and practical product applications. It's challenging, but when research successfully translates into product improvements or new features, it's extremely satisfying.

So, if you're stepping into this role, focus on clear communication, understand the value proposition of your research, and enjoy the process. Product

managing a science team offers a unique set of challenges, but it's also one of the most interesting jobs in product management.

We have been working on using AI in our software processes for about 3 months now at PURE. I wanted to give a more complete update on what we've learned, and where we are so far.

The TL;DR - Our goal is to empower our team with AI, and ultimately reimagine our software development lifecycle (SDLC) in a more AI-first way (if the results warrant).

So far, we have built a set of (publicly available) VS Code plugins that support coding, refactoring, unit tests, code reviewing and pull requests - it’s particularly good at testing. We’ve gotten a lot of initial benefit from this approach and are currently rolling it out across the team.

We’ve focused on breaking down the SDLC to make our AI-use more modular and help the team understand how best to apply it. This seems to be an effective approach so far. Our next step is improving measurement/metrics so we can talk about results more concretely.

What we've done - the details:

To start, we wanted to make sure everyone on the team had an opportunity to try out ChatGPT and Claude (or other tools) officially. About half of our team had already used these tools individually as part of their development process, but a number of people hadn’t (or had only used Copilot). We got team subscriptions to both tools and gave all the developers Claude to start. The product team requested both Claude and ChatGPT so that they could compare.

We then ran two projects in parallel: (1) investigating how to optimize our SDLC and (2) running a Hack Week to get initial outputs

Optimizing our SDLC part 1: Framing & Prototypes

Based on experience with human crowdsourcing, we started from the hypothesis that we should break tasks down as much as possible. This makes it easier to get good results, and, importantly, to quality control work as it goes along.

This led us to map our our SDLC and brainstorm each step and what we might be able to do to automate. We also released a couple of initial tools as a proof-of-concept, which are publicly available here: Mo Commit and Mo Mo Code Reviewer.

Hack Week

Then, after everyone had the tools for a couple of weeks, we ran a hack week where people could come up with their own projects either using AI for development or adding AI features into our platform. Here are some of the projects that resulted:

AI for Software

• AI-driven unit testing tool

• AI-driven test refactoring tool

• AI-driven component refactoring

• prompt library for product specs and tickets

AI for Product Features

• instant localization of our UI into multiple languages (Spanish, French, Russian)

• natural-language custom report builder from our Postgres database

• recommendation engine to match residents to properties

• internal chat bot to answer property manager questions

Optimizing our SDLC, Part 2: Mo Coding Assistant

Finally, we looked at everything we had learned during month 1 of this project and decided to bring it all together. This has resulted in our Mo Coding Assistant - VS Code plug-ins for all the major parts of our development processes. This work was done by Felipe Mantilla and Michael Rodriguez.

Overall, this tool works by providing specialized prompts to Claude, with code context, to support specific pieces of the SDLC. Here’s more detail about the key features:

• 🤖 AI-Powered Commit Messages: Generate meaningful commit messages automatically based on your code changes.

  

• 🔧 Intelligent Code Refactoring: Improve your code quality with AI-suggested refactoring.

  

• 🧪 Comprehensive Unit Testing Support: Create, update, run, and validate unit tests with ease.

  

• 📚 Context-Aware Assistance: Add files to the AI's context for more accurate and relevant responses.

  

• 🎨 Interactive Webview Interface: Access all features through a user-friendly sidebar interface.

  

Applications for Product

At the same time, we also built out a v1 prompt library for product that handles spec creation and ticketing. This work was done by Kelly Kane. We were able to write a mostly complete spec that needed some editing and turn it in to an epic full of Jira tickets that were able to be refined by the team. In our next version, we are planning to revise the prompts and directly plug them into Jira.

What we’ve learned:

Here are some key takeaways from our experiments so far:

1. Breaking down the SDLC helped us be faster and more targeted. We knew exactly what problems we wanted to solve most and where we thought we could get value. Testing and refactoring have been pain points for us, so we started there and were able to make a lot of progress.

2. Centralizing tools and bringing in more context helps adoption. This is obvious, but it helped the team a lot when we brought everything into our dev ops world and provided the tools and training to use them. Everyone is using the tool now, whereas before, adoption of AI in general was variable in our team.

3. Human-AI interaction can improve code quality. For example, we set up Mo Code to write harder unit tests that focus on things that it thinks should work. This has, unexpectedly, actually led us to improve our features and catch errors and issues we might otherwise have missed.

What’s next:

We’re currently working on measuring the impact of our coding assistant on team velocity. My next post about this project cover this topic in depth.

Subscribe now

Try Mo Coding Assistant