Everything's a Product with Sarah Morgan

Matty: [00:00:00] One of my quirks is I think everything’s a DevOps problem. It’s time for Arrested DevOps, the podcast that helps you achieve understanding, develop good practices, and operate your team and organization for maximum DevOps awesomeness. I’m Matt Stratton. We have a fabulous show for you today talking about some things maybe not always we would think about when it comes to DevOps. But before we get into that, Let’s have a word from our sponsors. Collecting compliance evidence shouldn’t involve spreadsheets and scavenger hunts. With automated controls and over 75 integrations, Drata automates the process without needing to be an expert. Drata supports 14 frameworks including SOC 2, GDPR, HIPAA, and ISO 27001. Companies like Notion and LemonAid have shared how Drata simplifies audits through automated evidence collection. Don’t let audits slow down your dev cycle. Request a demo today and get 10% off along with waived implementation fees at drata.com/partner/arresteddevops. Let’s talk about one of the most exciting events in the DevOps community, DevOps World 2023. If you’re someone who’s passionate about learning, networking, and staying up to date on the latest trends then attending DevOps World is an absolute must. So what can you expect from DevOps World? The list is endless. First off, get ready to hear from some of the most inspiring and innovative speakers in the industry. The sessions will cover everything from AI automation, cloud-native architecture, security and risk management, to continuous delivery. And the best part is that DevOps World Tour 2023 is coming to 5 cities across the globe, New York City area, Chicago, Silicon Valley, Singapore, and London. Find a city near you and register today at arrestedevops.com/devopsworld. Feeling like you have too many alerts, overwhelmed by vulnerabilities, and at the end of the day, not deploying apps as quickly as you would like? Sysdig hears you. Security in the cloud can be overwhelming and security posture is suffering. You need a way to prioritize what matters so that you can move faster. Shift left is the right operating principle, but you must shift left the right way. Sysdig roots everything it does in runtime insights. By knowing what is running in production, you can prevent, detect, and respond to threats and do it at cloud speed. To learn more about Sysdig, visit sysdig.com/arresteddevops. I’m curious, before I start this, we’ve been giving that same Arrested DevOps intro about achieve understanding, develop good practices, and operate your team, etc., for now going on probably I think we shifted to that about 6 or 7 years ago. So listeners, let us know if you think we should change it. And if so, what should we change it to? But this is not the meta podcast. We do that at the end of the year. This is a regular podcast. And because of that, I have a guest. So joining me today is Sarah Morgan. Sarah, welcome to Arrested DevOps.

Sarah: [00:03:06] Hi, thanks so much for having me.

Matty: Could you give our listeners just a quick little intro about who you are, what you do, and what you think is cool?

Sarah: Yeah, absolutely. So, I am the Senior Product Manager at Telemetry Hub, which is a Scout APM observability product. I’m working on both our APM and our new observability platform. For me, it’s been super cool because my career dates back to the early 2000s when we didn’t have these kinds of tools. So, it’s been pretty awesome to see where it’s gone and where it’s going.

Matty: One of the things we talked about when we chatted the other day before having the show was thinking about What are the, like, product ownership, product management, whatever we want to call it, kind of a product way of thinking, how that can apply maybe to some of the non-producty normal things we think about, right? If we’re thinking about software that we’re delivering, that seems very natural. But then thinking about applying this to different parts of tech, different parts of an organization.

Sarah: Yeah, I think, you know, I’ve been in product now for about 10 years, and I came from an engineering background. And one of the things I think that a lot of product people kind of don’t think about is like going beyond the business case and the features to things like reliability and stability and, you know, that kind of user experience that’s more than just the feature set. And for me, it’s really made me think about kind of ownership, you know, because it seems to have gotten more, a bit more siloed as things have gotten more complex, you know, where everybody kind of owns their little piece of the puzzle, but there’s not one great centralized location where you can be like, okay, I get it. I see the big picture. I understand how everything’s working together. So, that’s something I’ve been thinking about a lot as we have kind of, you know, dived into this new observability platform that we’ve been building.

Matty: [00:04:57] It’s funny too, because I remember at PagerDuty, we had a product owner for the SRE team, and I’ve actually tried to find any blog posts or talks or anything that she had given, and there wasn’t really anything, because a lot of people kind of think about that when you want to apply it to a place like that, Some stuff seems to come really naturally and some things less. So maybe again, as someone who comes from a product-oriented background, if you sort of were needing to give like, not the 101, but just sort of the broad thinking about like, how do you reason about things differently when you think about them as a product than you might in a traditional like IT service, like an SRE team or a TechOps team or something like that, or even a database team?

Sarah: Yeah, I think in my perspective, and this, you know, this may differ from person to person, but I think that I think a lot more about sort of that last line. So, that kind of, you know, where the, where the user hits your product. So, where that experience lies. And I think a lot when I was doing things like supporting a database, that was what I thought about. I thought about the health of that SQL Server cluster, and not a lot else, because that was what was important to me. And, you know, if there was something happening upstream or downstream, downstream, I wasn’t quite as concerned. And now as a product manager, I really have to care about all of it, whether or not I understand all of it, because it all impacts that end user. So the more information I can get, you know, the more comfortable I feel kind of communicating and supporting and, you know, being an advocate for my tools and the things that my team’s building.

Matty: [00:06:33] When we’re thinking about, like, kind of product-oriented outcomes as well, right? You know, and I think that’s The thing, and this is kind of taking those, if you think about traditional product management, you know, think about Marty Cagan stuff and you’re like, okay, how do we, how does that apply to me who, you know, provision systems for our developers and whatnot? And I have a talk about this called Everything’s a Product. And one of the things I said in there, I said, you know, you’re not going to put an NPS score on your Jenkins pipeline, you know, but But kind of are you, right? Like, I mean, it’s, it’s thinking about your users and getting in feedback loops, right? And that’s DevOps is feedback loops.

Sarah: Yeah, it really is. I mean, I think you almost want to be thinking about, like you said, like an NPS score, but some sort of health rating of every piece of your system. You know, it’s not just about like, is this something that is secure and, you know, healthy, but is it doing what we need it to do to support the people that are going to be paying our bills?

Matty: [00:07:36] And what are some of the ways that we do this? I mean, I joked about NPS scores, right? But I guess that’s one of the interesting things about kind of treating something as a product when it’s internal. You have customers or users that maybe don’t have an option, but they sort of do. And where does— how does that feed into things? At least how you’ve approached it.

Sarah: Yeah, I think that’s interesting. I’ve been almost strictly in B2B software. So I have had a lot— a lot of my users have no choice. I dabbled in consumer software for a bit, which is a whole other world. But one of the things that you really have to consider is, you know, are you, are you helping people get things done as efficiently as possible, or are you making their lives more difficult? And I think that’s true for your users when you’re thinking of it from a product perspective, but also your internal stakeholders. You know, you want your, your, your colleagues to be able to do their jobs efficiently. You don’t want to build things that are so you know, so hard to understand or so, you know, lacking access that they can’t even begin to sort of make sense of what’s going on. And for me, that’s been the big thing. Like, as I’ve moved further away from technical roles, it’s made me very nervous because I’m like, oh man, like, now I don’t necessarily know what’s going on and I don’t know the right questions to ask necessarily. I don’t, you know, I haven’t been exposed to all the technology. So being able to have some kind of transparency and communication across those lines makes me feel a lot more confident in my work for sure.

Matty: [00:09:16] Where are some of the things you’ve seen that make that challenging though, right? When it’s kind of going against type, right? From, we don’t think about a team like an SRE team or a team like that thinking in that way. You know, how, where’s that? Maybe some of the stumbles you might’ve seen or seeing people do.

Sarah: Yeah, it’s kind of interesting. I worked for a log me in company as the product manager for the DevOps team, which was really interesting. They hadn’t had that role before, and it was new, you know, DevOps was sort of new for me at that time. And I think a lot of the stumbling blocks were really about kind of communication, but also understanding of what all of the different teams were doing. You know, I think people are working on what they know they need to be working on, and they don’t think about the fact that somebody else may care. You know, like, I think product managers are often very guilty of this. They, like, assume that the rest of the team that they’re working with doesn’t need to know what the use cases are, what the customers are going to be doing. And I think that’s a huge mistake. And it’s something that I, you know, kind of philosophically try really hard to avoid. Like, I want everybody who’s touching something that a customer is going to see to understand why we’re doing it. And, you know, what the pitfalls might be, and, you know, what usage might look like, what, you know, kind of like load might look like, that sort of thing. And I think a lot of people don’t do that, because they’re just like, if the feature is right, that’s all I worry about. And then if stuff goes wrong, it’s ops’ problem. And I just think that is, it’s really a communication issue, right? And I think a lot of PMs assume they won’t be able to understand the more technical stuff. But that’s not true. You know, again, like, I I may not be able to get in there super deep, but I can understand enough to be able to kind of like give useful information to my ops teams and that kind of thing. And if you have all of that walled off, you know, I think it just doesn’t— the end result just isn’t as good as it could be.

Matty: [00:11:08] Yeah. I’d love to hear more about when you were in that role. Right. So like kind of what, what kinds of things you did as the product owner for the DevOps team. First of all, like, I’d love to know, like, what was the remit of that team? That will help because, as you know, you ask 7 people what that team does, you’ll get 9 different answers. But in your mind and your understanding, what was sort of the scope of that team? Like, what did they do that you were the product owner for? And like, how did you go about that?

Sarah: Yeah, it was, I mean, it was such a huge, like, sea change from any product work I had done before. So the team that I was working on and the product that we worked on was a messaging gateway for an IoT platform. So working with just huge, huge amounts of data and a lot of different customers. And the DevOps team that I was working on was primarily responsible for managing all of the development environments, all of the AWS infrastructure, managing all, like, federation access control, you know, security, disaster recovery, all of that. And it was a fairly small team. We were co-located in Boston and Budapest, and I think we were, like, 6 people, and I made it 7. So a lot of what I did was, you know, I kind of got in there and watched for about a month. And then I was like, okay, like some of this is very repetitive. We could productize it, you know, we can automate some of this. And I tried to treat it as a product. You know, you look for things that you can automate, things that maybe don’t need to scale and you don’t have to worry about it. But honestly, a lot of what I did was run interference between my team and people who were used to just going straight to their favorite engineer and being like, Oh, like, I, you know, need, I need additional access to whatever. And I was like, nope, there’s a process. Like, we’re gonna stop doing that. But it was, you know, it was fascinating. And for me, it helped me bridge the gap between like my professional services teams, my marketing teams a lot better, because I could understand a bit more of what was happening.

Matty: [00:13:04] From kind of a product management perspective, you know, so you’re doing a lot of interference running, a lot of that, but what, what were the, the ways that you sort of applied those feedback pieces, I guess, you know, kind of that might be different than just a normal, like, okay, we have tickets, we have a thing that we do.

Sarah: Yeah, a lot of what I did, again, was just kind of trying to look at it like, like a traditional product, except for that my customers were the engineers that were on my team, like the software developers on my team. So that meant, you know, kind of looking to see what their pain points were, what did they have to do that was slowing them down or creating stumbling blocks and that sort of thing and building out a bit more of a back than I think you traditionally see on the operations side of things. I think you do, like, you see a lot of, like, pipeline-style ticketing where I’m just going to work on, you know, the thing that is the oldest or the thing that people are screaming about. But actually building, like, a strategic roadmap for, like, this is where we want to go with our platform, I think is just a bit of a different way of thinking. But I, I like it because I like to be able to focus on something. And I think my team liked it because they felt like they had more of a direction. They weren’t just all working in different, you know, on different things that were unrelated and that they didn’t have a lot of input into. We could come up with like a strategy collectively and then execute on it and then, you know, look back on it. What did we learn? You know, take the next thing and do it better, which I just think— I think it applies to so many things. I try to do it at home with my family and they’re like, enough.

Matty: [00:14:37] I know.

Sarah: Yeah, right.

Matty: That was— I— we, we did this exercise in my leadership team. We did this like how to work with me kind of deck thing, and one of the things was about you know, like your quirks. And I said one of my quirks is I think everything’s a DevOps problem. Everything’s a cross-functional breaking down silos problem. And it, it, it, it is. It always ends up in the same way. Like, we should do— I’m curious, you talked about a roadmap, and it was funny because I’ll put a link in the show notes to this, this talk I gave a couple times, but I have a slide in there that says this is why roadmaps are bad, and if you have one, you should feel bad. And I don’t remember enough about like what my point was about that, but I think it has a lot to do with commitment and sort of like things being flexible. And it’s definitely, I think, more about public roadmaps. And this is interesting because this has come up in other conversations. So this is kind of— I’d love to see kind of how you think about this because one of the problems with the roadmap is it can be interpreted as a promise, right? How do you sort of balance that? Again, having that good transparency about this is what we’re planning to do, but also understanding that like we are in a— we have a world of agility and adjustment. I’m asking because I want to know how to do it. And I’m assuming you do look at the pressure. Like, how do you balance that thing with your customers or with your exterior people to say, like, I want to give you as much as possible, but please don’t, like, screw me on this. Right.

Sarah: [00:15:52] I mean, honestly, that’s like the bane of every product manager’s existence. And then the pain is felt all the way down through the entire company, I feel like. And I handle it in a very sneaky way that I hope nobody that I work with is listening to.

Matty: It’s like, I don’t tell anybody anything.

Sarah: So my approach to this sort of thing typically is to not call anything a roadmap beyond me. So like I have things that I call a roadmap and I share that with my engineers and we kind of shuffle things around in there and that is where dates happen. And then outside of that, I try to keep it super vague. You know, I’ll have, I’ll share documents that are called things like H1 priorities. And, you know, I just continue to be like, this can change. This is not set in stone, especially where I am now. We’re, you know, a really fast-moving startup, and we’re reacting to things that are happening in the industry. I refuse to nail things into a stone tablet and say, this is what we’re going to execute on no matter what. You know, maybe I’ve, you know, I’ve worked at giant companies where they can do that, they have that luxury. But, you know, at this stage that I’m in now, I would never, and I would never like hold my team beholden to those things either. You know, I just think that, you know, I know that marketing needs it, right? They need to be able to plan things that they’re going to communicate with. Sales always wants to be selling one step ahead of where the product team is, but I always kind of ask them to keep it vague. You know, if we’re building a very specific feature, I’ll be like, tell your customers that we’re working on new collaboration tools, and we don’t get into the specifics. We’re working on alerting integrations. I’m not going to tell you if it’s PagerDuty and a webhook. I’m just going to tell you that it’s new alerting integrations. And then the closer you get, the more kind of detail you can share. So, transparency is key, but like, the granularity of it, I guess, is where I try to kind of like pull back a bit and get a little bit more and more granular as I start to nail down an actual release date.

Matty: [00:17:54] In the talk where that comes in is that the lead-in is saying, hey, at least half your ideas are going to fail, and talk about the Zune and Google Wave and all sorts of other fun stuff. And I said, that’s why roadmaps are bad. You should have one, you should feel bad. And it’s because we can’t predict the future. It’s interesting when you, when you talked about, you know, everyone wanting to be ahead and sales wanting to sell ahead. I was very fortunate that I got to have— I got to Marty Cagan. And again, I’ll put a link. Marty Cagan is very— if you’re in the product management spaces, well-known person and written a bunch of books and stuff and relatively sharp. And for all I know, it’s not relevant anymore. I don’t know.

Sarah: No, I think he’s still the guy.

Matty: Okay. That’s cool. So this was years ago. So when I worked at apartments.com and I was just running tech ops there, but they had Marty Cagan came in and spent 2 days with us talking about our, from a product perspective. And it was one of those cool things where I’m like, this has nothing directly to do with what I do, but I, because I’m in management, got to sit in the room. And this also dates the reference when I talk about when he was working at LivingSocial, which raise your hand if you’re listening to this and even remember what LivingSocial was. But one of the challenges is he talked about, again, they were doing experiments and then they would say, okay, we sort of ran this experiment on the product and now we are getting rid of it. And he said, and the sales comes and says, you can’t do that. I’ve already sold that. And what I think this connects to is that And this goes back to like, you can’t just be agile in one part of your company. Like when you’re thinking about this work, it changes the way you do everything because the way you sell is different when you have this type of a mindset, the way you market it, the way you communicate and connections between marketing. And I sit in a DevRel team that sits in marketing and we deal with this a lot. And it’s not a failure. I’m not saying anything bad about my colleagues or my colleagues in product, but what if we aren’t well connected? Into what’s happening, what we end up with the, okay, cool, we’re launching this thing. Can y’all write like 7 blogs about it in the next 2 weeks? And you’re like, well, no, we planned all our stuff. And so this goes back to then thinking about it in a different way, which is how do we have somebody embedded that’s the connection point into that product team? Again, back to my thing where everything’s a DevOps problem, right? I’m like, we have a cross-functional team that has a connection. And now you can’t do that with sales because you need your salespeople out there like getting POs. They can’t, But I guess getting into that idea of you want to have— we want to have this culture of experimentation, but no, not like that, right? And it goes— I saw a talk at GOTO Chicago from Andrew Clay Shafer, and he said nothing’s changed over many years. Everybody wants the same thing. They want more reliability, more stability, more velocity, listed all the things without changing anything. And I think that’s the hard part if you’re going to try to take on something like this is it is counter, especially in a non-traditional team that would use that, all the systems around how that team works are contrary to that, right? They are— you are, you know, especially in an operations team or an SRE team, you are interrupt-driven. As much as you try to not be, that, that can happen. You can have incidents, you can have these things. That’s not necessarily as much, you know, so the way you manage it is a little bit different. And like you said, you know, If you’re a feature team, you’re working towards the future and you’re kind of in your own place. So, it’s all of your inputs to there and your customers are, your users are probably a little bit better defined. On the other hand, again, you have to fight harder for your users. And this is one of the reasons I’m thinking this is really relevant to the conversation right now is in this idea of platform engineering, right? That now we’ve rebranded everything to PlatEng. And if you’re saying, okay, If I am providing a platform, because fundamentally, this is my take, and we did an episode you can go look at. I think if you go to restdevops.com/platformengineering, you’ll see the episode we just released with Daniel Bryant talking about what platform engineering is, maybe. But to me, it’s fundamentally providing Heroku inside your company, like in a lot of ways, right? That’s what we’re trying to do is say, hey, give a developer platform, give a place that we have this, you know, service-oriented thing. Well, so on one hand, if I’m Heroku, let’s say, you know, when I cared about selling Heroku before Salesforce bought it and they didn’t care, but if say I am something like a Heroku, a public, you know, like a SaaS, if I don’t get customers, I don’t exist, right? Like I have to have my revenue, I have to do my thing, and my customers have alternatives, right? They can say, okay, I’m not going to do Heroku, I’m going to do Engineered. Wow, boy, did I really date myself with that one or something, right? Or even different, I’m not going to use AWS, I’m going to use Azure, I’m going to go to this thing. Your internal platform, on one hand, kind of doesn’t feel like you have that problem because you’re like, well, you have to use it because that’s what we use here at JPMorgan Chase or at Target or whatever. But do you? Because that just means people— if people aren’t going to consume your platform, they’re going to go around it, right? And/or they’re going to not consume it the right way, right? Which is— and I’ve thought about this in terms of When people are building platforms, it’s all— I don’t want to say it’s all, this is unfair, but generally we think about it from the compute and the orchestration layer. And we’re like, cool, we get our Kubernetes, we have this Kubernetes cluster and all this stuff around this and blah, blah, blah, blah. And you’re like, well, what about event streaming? What about data pipeline? What about messaging? What about this? Oh, well, but I’m a Kubernetes guy. I don’t know that stuff. And you’re like, but that’s part of the platform. So the thing is, if you don’t, provide that, your customers are still going to figure out a way to get it because they’re going to still need it. So either you’re going to end up with them not adopting your platform at all, which will be a failure on your side, or they’re going to kind of half-ass adopt your platform and they’re going to pick up that other stuff somewhere else. And now you have to deal with a mishmash of things. So like, I think when you think about Pledge, it’s like you have to be— you have to take it from such a product place Because you have to provide the features that your users need, because otherwise they’ll find them somewhere else, right? It’s shadow IT all over again. It’s the same thing. People are going to go, fine, whatever, I’m not going to use your thing. I’m going to go use Heroku or whatever, and I will get an exception. And now, so, and I’m curious, like, how many of these teams are thinking about it truly as a product, because I think they look at it from the— can look at it from the perspective of This is a service that I provide internally. Like the same thing, you know, thank goodness most of us in ops area, we don’t have to run email anymore, but there was how many decades we spent being Exchange admins or whatever. And you didn’t have a choice. That was fine. Like you want to use email at this company, you got to use the one we have. There is no sign up for your own, but that’s not the, you know, so if we think that way, we’re like, well, no, it’s the CIO said this is what we do. You kind of have to think about it in that way.

Sarah: [00:24:41] Yeah. And then you like go through your SOC audit and you find out how many like rogue platforms people are using and then it keeps you up all night wondering if any of them are giant security leaks. But it is, you know, I think like it’s one of your points there too, like not only do you need to think about it as a product, but you need to think about like the use cases because if you provide something that technically fits the bill, but you’re not thinking about how people are going to use it, they’re going to use it in a bad way, that’s going to hurt something, right? It’s going to cause you pain in the long run. And they will also be unhappy. I’ve seen at a previous company, we had a backend service that did not handle a bulk request. And one of our frontend engineers just wrote something that iterated over like, you know, a couple thousand rows. And it worked as long as there was like only one person using the platform at a time. And as soon as more than one person was working, everything fell over. And that was kind of, like, highlighting that, like, technically, they built the right thing, but somebody didn’t do what somebody wanted. So, they worked around it in a way that broke everything. And nobody ever anticipated that. And it took forever to fix, because nobody understood what was happening, you know. And I think it’s, those are the communication gaps. That’s when you need to think about, like, who are my stakeholders here? What might they do? That I don’t expect them to do? What expectations do they have that I’m unaware of? You know, it’s, it’s a— people hold so much in their head that they think is common sense because it is in their domain that, you know, you hear that so many times. Well, obviously we should have done XYZ. And it’s like, it wasn’t obvious if we didn’t do it. So again, I think it all comes down to communication and also like not selling proof of concepts and MVPs as like concrete features that you can never move back from. That is— MVPs are not MVPs anymore, you know.

Matty: [00:26:36] So that was another fun one that I, I feel like— and, you know, listeners of the show know that I’m pedantic except when I’m not, but like, I feel like being pedantic when it actually— when it matters because it changes how you think about it. So I think if you ask the majority of people what MVP means, they will tell you it means minimum viable product, and Kagan will tell you it means minimum viable prototype.

Sarah: Yep.

Matty: So even at that point, your MVP is still not even a fully fleshed out prototype. It’s not even, it’s not a not fully fleshed out product. It’s the beginnings of the beginnings of a thing. And, but we tend to think that MVP means version 1, right? Like it’s our first iteration of it. And I’m going to do this MVP, you know, and it’s, it’s really there just to help you learn something.

Sarah: And you can’t do it fast, which is the point of it, if it’s supposed to be production ready. And I think that’s true, you know, kind of across the board, you want to experiment quickly, it doesn’t need to scale, you don’t need to worry about stability, like we’re testing this out to see if it is useful or not, we don’t expect all of our enterprise customers to onboard it tomorrow. But that’s how we treat it. And then you can never back away from it. Like, I can think of so many things that I’ve had to support that was an MVP, you know, that got built and then there was no version 2 and it’s still out there, you know, being a piece of crap. Everybody hates it, but, you know, we moved on to the next shiny thing and we should have sunset it or made it better, but we didn’t either.

Matty: [00:28:09] It’s the equivalent of how many, again, the old ops person in me sees the how many times there was a critical production system that was running under somebody’s desk. That was like a— oh my God, compact computer, you know, that was under there. It says, you can’t unplug that because that’s running— that, that machine under my feet there is running billing for the entire company, you know. Or I had— yeah, I remember, you know, times where there would be a server in the data center, it’s like nobody knows what it does. You’re like, well, we’re just gonna turn it off. And well, or be like, we turn off and see what happens sometimes if anybody comes running. My absolute favorite story of this was when I worked for Bank One, then Chase, I went to one of our data centers here in downtown Chicago and was talking to the computer operator. He was sort of looking and he pointed at this one machine and he was like, do you know what one of those is? Or no, he said, have you ever seen one of those? And I go, no. He goes, nobody has. He’s like, nobody even knows what this is, but it’s just here. And it’s funny, but it’s true because you build the thing to help solve the problem and then it’s there. Now the problem has been solved and we move along and it’s, it’s sort of, it’s all tech debt, right? And some tech debt is fine, but you know, it, it, it, it requires interest. You talk too about like asking these questions and stuff. And I think it’s like thinking about discovery, right? Like, you know, you, you, you know, it’s, it’s how do you have a way because you can’t just ask someone what they want. Because they don’t necessarily— people know what they want. I think that’s unfair to say, like, well, nobody knows what they want, but they don’t necessarily communicate it to you in the way that translates into the thing that you can build. So, I think the skill of product folks is being able to say, how do I ask these questions? How do I suss out the things either from direct questions or indirect ways, proxies of seeing, did this work the way that you expected it to and not? And then working on, working on those.

Sarah: [00:30:09] Yeah, exactly. And I think a lot of it is like repeating it back to them to confirm what they said, because sometimes when they hear it, it’s like, oh no, but it’s going to be for like hundreds of thousands of users, not just, you know, this team or— and then the next thing we expect it to be able to do in a year from now is whatever. And, you know, you, you don’t pigeonhole yourself into, well, we built it to do one very precise thing and it’s never going to be able to grow from there without without like a major refactor. You know, that’s the stuff that you just, you just have to have conversations, you know, like a PRD or a ticket isn’t going to get all of that in there. So it can be async, but it doesn’t, you know, it’s not going to be like, I have written something down and given it to you. And now you have everything you need to be successful.

Matty: I’ve always found it, there’s like 2 places that I’ve seen product folks like excel or be slash be interested in that, that a lot of times people wouldn’t think of. But we think about this like, especially at like community conferences, not like re:Invent or, you know, big, like the booth costs $30,000, actually $30,000 would be cheap at re:Invent, but that would, that would get you a spot on the sidewalk. But, you know, you know, a DevOps Days or a KCD or whatever, it’s like some of the best people to work that table are product people. Product people love it. Like I learned this at PagerDuty, like my friends in product, they would do that. And they were like, this is amazing. Cause I just spent this entire day talking to the user group, right? Talking to the people and like hearing what they have to say, like in this, this big conversation. The other one, and I remember I was telling you about this when we were talking before, was when I was at PagerDuty at first, this predated me, but in earlier days, the setup was for incident, we use the incident command system. And so the incident commander and the rotation for incident commander was engineering management. So like, It would rotate through there. And then, and there were several folks, you know, a bunch of people in the IC rotation. And, you know, one day this woman named Lilia, who was a product owner, said, you know, this seems interesting to me. She’s like, I would like to learn how to be an incident commander and participate in that. And everyone was kind of like, well, can you? I mean, not, not, you know, it was kind of thing. You’re like, you’re not an engineer. You’re not a software developer. Like, you don’t know our technical systems. And she was like, well, from everything I understand about incident command, that’s a feature, not a bug, right? And so she got the training, you know, the internal training or whatever, and she became— she was PagerDuty’s first non-engineer. Now I should say non-engineering. I mean, Lily actually is an engineer, but like, you know, her role wasn’t engineering, incident commander. And when I was leaving PD, I, at that point, there were no engineers in incident command. Like it was sort of a thing where it’s helpful because one of the reasons is You, as Ron Swanson of Parks and Rec would say, never half-ass 2 jobs, whole-ass one job. So you can’t be the incident commander and also be the resolver and actually be fixing the problem. So if you are not a software engineer on the product, you can’t, like you literally will never have that problem because you’re not there. So it’s helpful, but there’s an interesting thing because I’ve talked to other companies where this has happened and there’s a really fun kind of echo effect almost that happens when you do this is how product folks think about reliability stories starts to change. And it’s not like the glib thing is, oh, well, because you had to carry the pager, so now you give a shit. And it’s not— it’s a little bit of that, but I think it’s also because the, it’s the, the fidelity of the communication and understanding is much better because if you’re the product owner managing the backlog and the prioritization and all this stuff, all the reliability stuff is coming to you secondhand in the best way that your on-call folks, when it occurs to them, can think about communicating it to you and what it means. Whereas if you’re in the firefight, you’re like, oh, I am hyper-aware of this now because I actually see what’s happening. And one of the things that I found is that operational folks and Just engineering, we are not very good at communicating, turning that stuff into business value. And the other thing is that, and I’ll tell a story about that in a second, but the other reason I find product owners make really good incident commanders is the skills are very similar. Because what do you need to be a good incident commander? Be good at prioritizing, communicating, delegating. These are all things product folks tend to be pretty good at. I think back to though, like about how you can communicate. So we had an issue at one point where we had this web service and it was, One of the sysadmins on my team who was the ops person on this project was very frustrated because he kept trying to, in his mind, he kept trying to escalate to the product owner that we had to fix this issue because it was— what was happening is it was throwing a lot of errors that weren’t errors, like to the point of, I mean, it was like 25,000 a minute, like, and our logs were flooded, right? And the big problem, it was just not doing a great job as an ops person to sort of say like, this is why this matters, because he was just leading with the symptom. Which is it’s doing this. And our product owner is like, well, okay, I’ve got this long list of shit from the business that we have to do and whatever. And this is annoying you in your logs. Why do I care? And he was having a hard time explaining. Well, the reason we care is because it’s not because our logs are full and we have to rotate them. It’s because if something actually goes wrong, we won’t know because it’s normalization of deviance, which is how you explain that. And it wasn’t done. And actually the way that we solved this slightly was they put all those fun, like, dashboards up in the office, and I got to control the dashboard, and I put the dashboard of the error rate of that service up there, and that got fixed in about 2 days. Because then it was like, wait a minute, why is this having all these reds? That’s not the best way to do that. The best way to do that is understand how to translate your operational concern into a business outcome or something, and that’s a skill that we could all get better at.

Sarah: [00:36:03] That’s a really good point, and it reminds me of something that I actually had to do Again, working on the DevOps team as a PM, you have to sell it all the way up the chain occasionally, especially if there’s money and time involved. And, you know, we recognized the potential for a lot of hurt if we did not have a good failover solution, right? But my GM was like, absolutely not. We’re not paying for it. We’re not investing in it, whatever. So, my approach was to work with my team to come up with like basically the worst possible scenario. So that they could see what that potential outage could be like if we didn’t spend the time and the money on having a good DR strategy. And then when they see that, it’s like, oh, now I get it. Like, now we’re talking dollars, we’re talking engineering time, and that’s dollars. And that’s what makes sense to people kind of outside of our teams, I think. Like, I tend to not think in dollars, I think in time. And I think in things like, you know, satisfaction. But if I have to sell something up the chain, that’s the first thing I try to do is be like, how much can I say that this would cost?

Matty: [00:37:13] I think there’s a, there’s a nuance to what you said that I think we want to reiterate. It’s not because the only thing that other people in the organization understand is money. It’s that money is a lingua franca. Everybody understands money. So that’s the— it’s the sort of easy way to speak the same language. Because otherwise you have to go and speak their language. Now, with some parts of the business, that language is the dollars 100%, and that’s the, the be-all and end-all of it. But even in places where it’s not, it’s like, that’s the thing, we do understand the bottom line. We do understand revenue, lost revenue, cost, OpEx. Maybe we don’t use those words for it. Actually, I just realized not everybody is going to know that. But I was like, people listen, like, no, Manny, I don’t know what that is. I’m really pleased about the fact that I don’t. But how do you— and, and It also reminds me, I think these are all sidecar skills that is what sets you apart as an engineer, right? As a thing like this, it’s like always look at what’s your plus, your yes and, right? Like, okay, I am the best React engineer, but I also understand how to write a business case. Like, that’s what’s it. And, and I think about, I had, you know, some, some developers and stuff that I used to work with at this place. And I remember they wanted to do a thing, whatever it was, I don’t even remember, it was something they wanted to build. And their CTO said, Cool. Write me up the business case for it. And they’re like, yeah, I don’t want to do that. And then they gave up. And I was like, and she wasn’t like, right. She was like, I want to say yes, like, but I need you to justify it, right? Because you can’t just come to me and say, Pat, I want this. And you go, okay, right? You, this is not the way the world works. This is what I explained to my kids. I, you know, like, sell me on it. I, you know, and it’s funny. This happens a lot as a manager. We run this a lot. Where you’re like, I’m trying so hard to say yes to you. I just need you to do this. And it’s not because it’s baloney process. It’s because I want you to think it through and you going through the exercise will funnel a lot of stuff out of it, but I’m trying to say yes. And then you’re like, I don’t want to do that. You’re like, I really want to say yes to you. And I say to my kids all the time, I’m like, dude, I’m trying to say yes to you, but I need you to do this.

Sarah: [00:39:25] It’s like, you know, trying to find that product market fit. And it’s all about validation. And if you can’t validate something, it’s not worth your time. And sometimes you have to kind of revisit and tweak a little bit your approach. And that, that does come with that kind of validation, like you’re saying, if you can’t, if you can’t justify it, it’s just like, I have a lot of ideas that I think are really cool. But if I shopped them around, I would discover that it’s probably just me and a very small group of my friends that thinks it’s and not worth building.

Matty: Well, or even, even just going through the exercise of doing it is almost your own rubber duck, right? You sit there and say, well, now that I had to write this out, I went, oh, oh, now that I had to think this through further than the initial idea, I actually— maybe not even to say you don’t want to do it, but you’re going to adjust it. You’re going to say, oh, now only think about it this way. So there’s, there’s reasons, but it’s also just from a pure career perspective. Again, like any of those little, like— and I’m not calling them— they’re not soft skills, they’re, they’re just additional Yeah, skills and abilities, it’s what sets people apart, right? You know, because, because it is part of the, you know, I mean, you think you can make the arguments either way about Amazon and their writing culture and stuff like that, but that’s— you want to be successful at Amazon, you have to learn how to be good at writing because you’re going to write a lot.

Sarah: [00:40:46] And I think you can learn it too. I think that’s what some people don’t realize is, you know, especially when I came out of undergrad, I didn’t want to have to work with customers. I didn’t want to have to work with sales. You know, all I wanted to do was to kind of be left alone to build what I was building. But over time, I’ve really had to work on my interpersonal skills, my interviewing skills, my listening skills, my public speaking skills. None of it came naturally to me at all. It’s all been learned. And I think a lot of people don’t give themselves enough credit for the fact that they can improve on these things. And it’s a matter of just practice.

Matty: Nobody’s born with any of this. I always think back to my senior year in high school, and I was in forensics, a speech team, and I had my partner, and we were co-captains of the team. And this is our— the senior banquet. It’s like we’re reveling in awards and everything. And one of our coaches came up to us, and she turns to my friend Jason, and she says, Jason, you have gone the furthest with the least amount of talent of anybody I’ve ever met. And it was— in her mind, it was an absolute compliment, which was He works so hard. And then she turned to me and said, you know, Stratton, like, if you ever worked at something, you might accomplish a thing. And her point was that I was the inverse of that, which was I was coasting and whatever. And again, it was that— it wasn’t even a backhanded compliment, right? Like, she— none of it was supposed to be offensive in her mind. She thought she was saying it. But that’s the thing, when you look at people who have these skills, they don’t just appear. Like, some things you might have a natural tendency towards it. But even then, even people who are like, you think about public speaking, right? Nobody started with being amazing at it. Now you might be less scared of it, but there are things that maybe what happened more likely is that some of us just started doing this kind of thing younger. So we got to practice more. And this is like my daughter who is now this year giving her second conference talk as an 11-year-old. I told her when she gave her talk last year, I said, you know, you realize now you are never going to be allowed to come up to me and say, no, I can’t do this presentation at school. I’m like, nope, nope, you’ve already— yeah, you’re, you’re in trouble now. I mean, there’s some people enjoy it more or don’t, but we all get— and, and also, and this, this goes into a whole other rant that I have about— now we’re talking about like public speaking and conference speaking— is how many people are like, oh, I don’t need to practice, I’m better off to come. Like, no, you’re not, I guarantee, because this is Me on the record for the 15th time on the record saying the best speakers at tech conferences are barely adequate public speakers. And I include myself. I’m saying this is fine. Like, so the people that we consider that are like, my God, you’re an amazing speaker, are adequate, right? Like, we all could do so much better at this. And it’s a skill and it’s a thing that requires practice, but the writing, Requires practice, right? So that’s the other thing too, is like practice. Like the more you do this stuff, the easier it gets. And it unlocks your ability to do a lot more things you want to do. Not, you know, so.

Sarah: [00:43:48] Absolutely. I think too, the thing that people do kind of on the flip side is underestimate other people’s capacity for learning new things. I know that I’ve been guilty of assuming that my sales team is either not interested or doesn’t have the aptitude to be able to understand more technical details of a product. And it’s done them a disservice. I know one of the best salespeople I’ve worked with really tried to learn and understand the product and what was happening under the hood. And for me, it’s the same thing. Like, as a product manager, some people don’t give me the benefit of the doubt that I can understand some of what’s going on, you know, in our infrastructure and our tech. And I want to know that stuff, you know, I don’t have to know it, you know, to the detail that some people do, but I want to be able to understand it. So if something goes wrong, I kind of get it and I can talk to my team about it, my management, you know, my customers. And I think that that’s something that you learn over time, just to never write anybody off or assume that they either don’t care or can’t understand it. Because if they care even a little bit, they’ll make an effort. And even just the conversation of them trying to understand is good for you to have.

Matty: [00:45:01] There’s 2 pieces with that that I’m violently nodding my head about. Like, and so one, one came in again, it was thinking about this having a product person, you know, be an incident commander is because the incident commander, if you, if you want to take the— see how well this analogy works, like you want to be able to troubleshoot what’s wrong with your car in terms of does it need, like, do I need more oil? Does it need gas? Has it got a flat tire? That’s what the incident commander understands the system well enough to know what part of the system might be broken, but it’s not the person that says, I know how to rebuild the engine. Or I know how to replace the carburetor. That’s my SME that’s on here. But I have an understanding of the system well enough to know who are the people. Oh, wait, you know what? I should be calling AAA now, or I need to take this to this place, or I need to bring in this. And I think the other place, this is just a rant I have about DevRel. I work in DevRel, so it’s allowed for me to complain about it, is folks in developer relations Generally, all generalizations are suspect, but we’re an area that believes very strongly that we’re very empathetic and everything unless you work in sales and then we hate you. We like to make really snarky comments about you and call you a sales droid and say you couldn’t understand anything. My experience is similar to yours and maybe I feel this way because I was a sales eng for a while, but still, I will tell you, there are sales, there are account executives and there are sales engineers and solution architects, SAs or whatever they’re called in your company, in your company that know way more about your product than you do, Mr. and Ms. Engineer, right? Because they have to, because that’s— they wouldn’t be able to do their job if they didn’t. And yes, there are plenty of toxic people in sales, just like everywhere else. There’s plenty of people that don’t know jack and they’re just trying to relationship sell or whatever. But I think about— I mean, there’s, you know, they know it and they have to and they want to. And, and Same thing, your product folks want to know this and they want to know where it is. And your marketing people need to understand it because they can’t tell the story. Everybody has to understand, you know, everybody in your company wants to understand this shit, right? And they are actually pretty capable of doing so. The difference is they are not necessarily interested in the how do I rebuild the carburetor part of that, nor do they need to be because you know what, you can rebuild the carburetor, but the entire role of the company is not to rebuild carburetors.

Sarah: [00:47:25] You know, so I think too, like, you know, being able to add some of that context, you know, thinking about sort of that triaging, you know, as a product owner, as a product manager, I can escalate that issue to the right team with helpful information so they’re not starting from scratch. That’s something that I have always tried to do because it helps my understanding, but it also cuts down on the resolution time. You know, if I’m not just like kicking along Oh, this guy said things are slow, but this is exactly what he was doing. This is what time it was. This is the area, you know, that he was working in, in the platform. All of that is going to be a more helpful starting point for anybody who is doing the fixing. But it’s useful for me to have an idea of what that architecture looks like so that I know what information’s relevant, right? Because otherwise, it’s just like funneling noise and complaints from one place to another, which is frustrating to work in.

Matty: Wow, this has been a lot to, to take on. I mean that in a good way. No, no, no, this is, this is, this is one of my, one of my favorite topics. I guess as we’re kind of wrapping up, if you were to give one thing to start doing and one thing to stop doing for people that wanted to think about their services or their, their teams in a product-oriented way, what would those be? So one thing to start doing and one thing to stop doing.

Sarah: [00:48:47] I would say start Thinking about who your stakeholders are, so that you can think about, you know, what you’re building as a product. You need to know who your users are and what they care about. And I think the thing to stop doing, to touch on something that you mentioned earlier, is stop building things without validating them. You know, don’t waste your time if there is not a reason to be doing something. Fine if it’s experimentation, but don’t invest huge amounts of effort on something that seems cool, or it’s a new cool technology, or it’s something that your friend’s doing somewhere else and they like it. Experiment, test it out, but don’t make that what your main focus is.

Matty: Fantastic. I’m going to take that to heart. So, everybody on my team that’s listening to this show, you can check, because my team could be listening to me like, oh, you’re full of crap, Matty. You didn’t do any of that, or you didn’t stop doing any of that. But if you head over to arresteddevops.com/everythingisaproduct for this episode’s show notes, if we have any interesting links, we’ll drop them in there. I’ll put in a link to that talk I’ve given before and any fun resources we find. If you go to arresteddevops.com/itunes and leave us a review in the iTunes Store, that helps other people find the podcast. I know it hasn’t been called iTunes in years. I refuse to change the URL, similar to how I keep calling it the Sears Tower and not the Willis Tower. But yes, Apple Podcasts. You can also find us on Spotify, iHeartRadio, Audible, and everywhere that fine and less fine podcasts are sold and distributed. Sarah, thank you for joining me today. This has been a really fun conversation. Where people find you either on the internet or I don’t know if you’re at events or conferences or—

Sarah: [00:50:33] Let’s see, up next I will be speaking at KCDC at the end of the month, which will be a lot of fun. So if you’re in Kansas City, come say hi. And then we have a pretty good YouTube channel. It’s just Telemetry Hub is the channel name and lots of kind of interesting conversations there as well.

Matty: As always, this has been Arrested DevOps. And remember, there is always DevOps in the banana stand.