Product Management Plays a Central Role in helping Define the Product Strategy

How do you define the role of product management in Compute and AI innovations?

Product management plays a central role in helping define the short-term and long-term product strategy. We typically formulate about a 16-to-18-month roadmap, because the space is so dynamically changing, we try to keep very limited focus so that we can make changes as and how we need to react to new emerging paradigms, shifts in customer requirements, how the ecosystem is evolving, new usage models that are emerging and facilitating more and more IT and on prem infrastructure to effectively move to the cloud. So, that becomes the basis of how we start doing product management. That leads to a longer-term roadmap that we build around.

The product management organization is also responsible for defining what the products are; that’s where there's an extensive amount of engagement with customers in helping with that definition process. We are very closely connected at the hip with our engineering partners so that within lockstep, we're making the right trade-offs. That level of transparency and engagement is super critical to be a functional organization at the scale at which we operate.

We are also responsible for all the outbound go to market motions, including pricing, the go to market strategy, launch of the product, managing the business once it's launched, and also being the first line of defence for customer issues, as in how they come up. Holistically, we are the custodians of the business and product management plays a central role as part of that.

What are the emerging trends in Compute and AI that you believe will shape the industry in the next 3-5 years?

Even though AWS and cloud computing have grown significantly over the past decade or so, we still genuinely believe that only a fraction of on-prem IT has effectively moved to the cloud. So, a lot of opportunities still exist in different forms. There's enterprise class, workloads like SAP HANA databases, or the foundational workloads that we're going after and trends like generative AI, which caught us by the rate at which these took off were eye opening to begin with.

So continually monitoring some of those trends, working with customers and looking for more and more opportunities to help incentivize on-prem IT to move to the cloud that's how we generally approach this.

Compute and AI markets are fast-moving. How do you stay ahead of the curve in terms of customer needs and technological advancements?

We do a few different things. One is, we're very tightly coupled with customers. Everything that we do at AWS is directly connected to the customer working backwards process where we engage with customers, trying to understand exactly what problems they are dealing with and all of the products that we build are directly related to supporting that.

The second thing is we have significant investments in technology. That's where some of our innovations in custom silicon come out and through our extensive engagement in understanding how these workloads are in the cloud, we know how we can optimize these so we start looking at optimizing across different layers of the stack. To be more specific, we started this long journey of what we call as a nitro platform. It's our cloud platform where we offload some of the overhead associated with virtualized infrastructure. With Nitro, we offload everything, which means all of the available compute resources are made directly available to the customer, which means for every dollar that a customer spends, they get more, more performance. There's a natural advantage in terms of price performance, that has become the foundational investment block for us. Nitro has allowed us to innovate at a tremendous scale through which we're able to bring in more products quickly and bring our own custom silicon. We can actually look at optimizing every layer of the cloud platform stack all the way down to the silicon, it effectively means lowering the cost for us to build and operate these products. When we can optimize the cost, those cost savings are naturally passed on to customers in terms of lower prices and better price performance, and that is one of the main central themes that is seen emerging in a lot of what we do today on the core compute side.

The third thing that we also do is, while we invest in our own technology stack, we're very tightly coupled with all the major providers like Intel and AMD, and we co-innovate with them. We have some long-term strategic relationships with these vendors where, through those mechanisms, we can build custom solutions that are only available on AWS. So, if Intel releases a new CPU, we are the only cloud provider that has a customized version of it, and that customization will allow us to get better performance.

How do you balance the need for innovation with the practicalities of development timelines and budgets?

Time to market is super important. That's exactly the focus we have in mind and that's where the product strategy, the roadmap, planning process, what do we want to deliver and by when all of that is formulated upfront, and we have a general sense of what is possible.

We have some very strong engineering organizations that deal with a lot of these challenges that are spontaneous and new. These are new innovations and new discoveries that are happening as we speak, so, we have formulated mechanisms through which we can work around some of these technical challenges, make the right trade-off. We never compromise what we need to deliver to customers just to get a product out. We don't have this concept of an MVP. We have what we call as an MLP, a most lovable product. MVP is more internally focused versus an MLP, which is the minimum product that a customer would need for them to love what we're building. So that mindset percolates in the way we approach product definition and execution.

Compute and AI often require cross-functional collaboration. How do you ensure alignment between engineering, data science, and business teams?

All of our teams are very closely connected at the hip. One of the things that we strive for is full transparency between product and engineering. We have engineering leaders working very close with the product definition and product management teams. They get a firsthand view of customer requirements as they're coming in, emerging trends as they're developing. One of the most important things to be successful in this space is the ability to be nimble and to be able to react to changing dynamics and you can only achieve that if all of these different organizations, like engineering, product, supply chain, finance organizations, they all have to be in lockstep so that we can quickly react to changing shifts in the ecosystem and we emphasize that extensively in every part of our product decision process.

What challenges have you faced while scaling AI models into production systems, and how did you overcome them?

We launched a new Trainium2 (Trn2) Ultra Server that allows us to configure four Trn2 servers in a coherent domain using a custom neuron link solution that allows us to take model sizes with parameters of up to billions and trillions that can now scale on the ultraserver. When we build these products, we're working in very close cohorts with customers to have them test out and help us both optimize the hardware stack as well as the software stack.

A good example of that is our engagement with Anthropic. Anthropic is going to be building massive clusters, using Trn2 Ultra servers, where they're going to get about five times the amount of exaflops that they can get versus anything else that they've built in the past. So, with some of these progressive customers who are building some of these latest state-of-the-art foundation models, large language models, transformer diffusions, we work in lockstep to see how we can continue tuning the hardware stack as well as the software stack.

How do you identify and validate customer pain points in a technical domain like Compute and AI?

That's central to everything that we do. We have a very active customer engagement process. One of the top priorities for everyone in the team on a day-to-day basis is remaining engaged with customers, and that can be on several different levels. We also engage with customers opportunistically when we build new platforms and they're ready to ship. If our engineering schedules allow, we will onboard customers, allow them to do previews, get feedback and make sure that anything that's discovered as part of that process is fully resolved before we launch.

We do selective public previews too, where we will announce public previews where customers can sign up, come on board and test. We have a very vibrant customer engagement process where, on an ongoing basis, we actually host customers on-site. AWS's ecosystem is broad, there are compute, databases, analytics and AI, and it's hard for each service to send their rep to meet a single customer. So, we actually host customers on site, and we have these well-coordinated sessions where each service team go and meets with them in a very coordinated way. We'll have very detailed plans on exactly what they want to discuss, what their pain points are, become a full holistic discussion across everything that they're running on AWS, as well as all the other innovations that are coming down the pipeline that they're exposed to.

How do you balance customer customization requests with the need to maintain a scalable product?

It comes down to providing full transparency to customers and understanding exactly what their pain points are and then using the roadmap process to help plan and prioritize exactly what it is that we need to solve and how to solve those. It's a balance between time to market and actually meeting a customer to solve whatever the problem that is, and that can be through a new product, making changes to older products, architecting different solutions. We look at look at it holistically, not just a point in time product problem. We have this emergent view on what is working well and what is not. What we also try to do is to be advocates for multiple customers because if one customer has encountered a problem and we've helped them solve it, it's very likely that somebody else is going to encounter that. We have mechanisms through which we allow customers themselves to share some of their findings and their resolutions through forums that we enable and we provide. So, it's an ongoing process.

What KPIs do you track for Compute and AI products, and how do you use this data to make informed decisions?

We look at our usage adoption and financial goals. We look at what different workloads and usage models are moving to the cloud, and we adjust depending on the situation, to make sure that we remain absolutely competitive in this space.

How do you foresee the convergence of AI with edge computing, cloud computing, or other emerging technologies?

As far as AI ML is concerned, we are still in the very early days. There's a lot of investment that's going on in the space, and then eventually what's going to come out of that is how can they make use of all the investments that they're making in the space, be it in terms of improving the business efficiency or driving newer monetization models. Every step of the way, the supporting infrastructure and service model that is needed to help facilitate that is something that we are aggressively supporting on the AWS side, all the way from the infrastructure layer to our bedrock layer, where we have multiple different foundation models available to the developer layer. So that's how we plan to remain vigilant and ahead of the curve.

-- As Narrated to Shipra Sinha