Pete: [00:00:00] I feel like Zoomers embrace the sysadmin concept, which is we have to shorten everything that we do, right? If charisma was an open source app or a product, it would be called Riz. Riz.io.
Matty: It’s time for Arrested DevOps, the podcast that helps you achieve understanding, develop good practices, and operate your team and organization for maximum DevOps awesomeness. I’m Matty Stratton. We are going to have maybe possibly the first, as I know of, of the rogue sessions of Arrested DevOps, where we’re going to talk about a topic we’ve talked about recently, but maybe with a different perspective. But before we get into all of the rogue fun, a word from our sponsors. Collecting compliance evidence shouldn’t involve spreadsheets and scavenger hunts. With automated controls and over 75 integrations, Drata automates the process without needing to be an expert. Drata supports 14 frameworks including SOC 2, GDPR, HIPAA, and ISO 27001. Companies like Notion and LemonAid have shared how Drata simplifies audits through automated evidence collection. Don’t let audits slow down your dev cycle. Request a demo today and get 10% off along with waived implementation fees at drata.com/partner/arresteddevops. Feeling like you have too many alerts, overwhelmed by vulnerabilities, and at the end of the day, not deploying apps as quickly as you would like? Sysdig hears you. Security in the cloud can be overwhelming and security posture is suffering. You need a way to prioritize what matters so that you can move faster. Shift left is the right operating principle, but you must shift left the right way. Sysdig roots everything it does in runtime insights. By knowing what is running in production, you can prevent, detect, and respond to threats, and do it at cloud speed. To learn more about Sysdig, visit sysdig.com/arresteddevops. So depending upon when you’re listening to this, either the other day or many, many weeks ago, we had a great episode of the show called— on the topic of platform engineering. Daniel Bryant joined us. If you go to arresteddevops.com/platformengineering, you can listen to that episode if you haven’t so far. But joining me today, as we’re going to maybe take a wander down this topic from maybe a different opinion, I don’t know, we’ll see, is longtime friend of the show and previous guest, Pete Cheslak. Pete, welcome back to Rested DevOps.
Pete: [00:02:38] Hello, friend of the pod, as, as the kids say.
Matty: Do they? Is that a thing?
Pete: Maybe like 5 years ago.
Matty: Okay, yeah, we don’t know what— probably funny. We don’t know what any current kids actually say other than—
Pete: not even a little bit.
Matty: It’s very cringe. It might be mid, I don’t know. Bring your rizz to the show.
Pete: Wow, that’s a new one.
Matty: Oh, oh yeah, I learned that one recently. I thought it meant like relationship or something, but apparently rizz is, is Zoomer for charisma. Oh, so it’s like, let’s bring our rizz to the topic of platform engineering. When we were talking about this earlier, Pete’s like, well, if I was better prepared and knew this we were going to talk about, I would have listened to that episode with Daniel Bryant, which he didn’t. So you will come into this conversation unadulterated. So what’s up with platform engineering? Give me your theory on this. Like, what’s this about?
Pete: DevOps with better marketing. That’s, I don’t know, that’s probably the hottest take. You know, so I’ll start off by saying that, like, I don’t know where I first started hearing the term. I mean, obviously, it was probably on Twitter or the, you know, blue hell site, as many people refer to it as. You know, probably saw some people talking about it. And I’m not sure if it really came from, like, did I see it from Gartner first? You know, did I see it from people throwing shade at Gartner, or from companies that are like platform engineering companies, or have some sort of tool that they want to sell you that is platform engineering, right? But I definitely saw it, and people started talking about it. And my first kind of immediate visceral reaction was one that is very unoriginal, and one that many people have said to me as well, is that hasn’t that what you Isn’t that what you’ve been doing this whole time? Right? Like, you’ve been building a platform for your team. And I know, Matty, you’ve talked about this too, of like the classic James Governor tweet of everyone is trying to rebuild Heroku poorly. Like, we’ve spent a decade rebuilding Heroku poorly, you know, kind of like the one of the earliest or one of the, you know, most loved, I guess, platform, platform as a service.
Matty: [00:04:42] So Yeah, and maybe going a little bit, like you said, with ill-remembered history of things, but even thinking about the idea of PaaS, it’s kind of funny because I feel like when I first started thinking about this, like, where it comes to mind is, you know, pre-Heroku is kind of Azure, right? So a little bit of when I think about when Azure was first coming up, you know, after, you know, AWS was there and then Microsoft, we’re gonna do this Azure thing. The model of Azure originally had nothing to do with traditional— what we would think of traditional cloud IaaS or whatever. Like, if you kind of look at the first Azure stuff, and again, true cloud historians might say, well, no, Matty, technically the first Azure service was blah blah blah, but really it was this web service and worker service approach. So what you would do is, again, it was, it was PaaS. And the problem was it was a little bit ahead of its time. And this is, this is my take on Azure marketing and go-to- market and stuff. But having someone who was a big Microsoft customer at the time and then went and worked for an Azure partner consultancy, 10th Magnitude, which is where, you know, Sasha Rosenbaum and I met and worked together. And you kind of— the way that I see it is Microsoft was like, we’re going to do this thing, which is basically PaaS. And then they went, oh, nobody’s doing this right now. Oh shit, we better have an IaaS. Which is why for the longest time, the IaaS implementation or offering from Microsoft wasn’t great because the infrastructure behind it was not built for IaaS. So you would run into this weird stuff. And as someone who did consulting—
Pete: [00:06:26] Ran into it quite often, I’m sure.
Matty: You would hit these things where like disk performance wasn’t very good, but it was because like the whole disk story inside Azure was not built for the idea that people had VMs. It was like optimized for the web worker services and the web services and all this stuff. And then they went, we need to sort of bolt Hyper-V on top of this and then offer it as a thing. And it took them some time to get that. Whereas you look in some ways, AWS is the opposite, right? Like AWS, I mean, yes, I know that EC2 wasn’t the first thing, but any of the non-IaaS offerings like various workers and stuff in AWS, they came after. So like, so their IaaS was a lot more mature because it was built for purpose more. So what I’m getting at is, This is to me, when I first started thinking about PaaS, it was in the context of even Azure or like Engine Yard. Do you remember Engine Yard? Oh yeah. Right. Like that was to me, I, for the longest time, years ago, I kept conflating Heroku and Engine Yard in my head. Like people would talk about Heroku. I’m like, oh, I’ve been doing that for a while. I’m like, oh no, no, that was Engine Yard. So yeah. Right. This idea. But if you look at the core of all of it, it’s, it’s how do we have some way to abstract like to get the software running somewhere faster and not have to worry about the bits and bobs that run it. That’s PaaS, right?
Pete: [00:07:48] Right.
Matty: And whether it’s internal to you or it’s— you’re consuming it like one used to consume Heroku when one consumed Heroku when it was something that wasn’t trying to be driven into the ground. We just want to be able to say like, I’ve got stuff and I want to get that I’ve got business logic, I’ve got application, and I need to get that deployed and running as quick as possible. And that’s the— Heroku is the git put, right? All this stuff predate— is like GitOps, like in a way, right? It’s like, I deploy this, it runs. I’ll tell you one of the— and actually, I think now it’s officially gone because I didn’t pay my Heroku bill, but the chess bot ran on it. That’s exactly how— and even to the point of, I didn’t have the code for the chess bot bot anywhere except in Heroku. Like, because I never even put it in GitHub because it didn’t matter. And then I would need to change it and I would be like, how do I actually get a copy of this out of— it was so— but my point is, I was so dependent on— like, that was how clear it was. It was just like, all right, just push this shit up here and it just goes and it works. And so, I mean, that—
Pete: [00:08:52] I feel like that was the most exclusive announcement that has ever happened on Arrested DevOps, is the fact that You’ve now just outed yourself as the owner and maintainer of Pete Chess Bot. It’s finally out there. I’m honestly, I was surprised with all the Twitter API changes that Pete Chess Bot was still working. And then you had said at one point, now it’s probably because it flipped over to the paid tier.
Matty: Yeah, I don’t know. Well, it really, what it boils down to, yeah, well, the Heroku part flipped over and then I was getting enough like nasty grams from Heroku saying you owe us money. And I just kept ignoring them because I didn’t care. And then I think I got an email the other day that just said, We’ve shut all your shit down. So too bad.
Pete: Yeah, you’re definitely right though. The, the, that simplicity of, of, of just pushing the thing up there obviously drove a lot of the, a lot of people to push, you know, their apps to Heroku in an easy way. Obviously the, the kind of devil’s in the details there because we kind of want that experience but are also like creating more complex apps. I suppose it was easy when you just had a Rails monolith that really had everything all wrapped in and get pushed to Heroku and call it a day. But, you know, the market kind of shifted, right? And then everyone started building lots of these services, microservices, and etc., to kind of run their apps, right? Generally, kind of like, it was easy when it was all just many little monoliths around, right, versus the many microservices we have.
Matty: [00:10:15] So we’ll take it away a little bit from the third-party PaaS or commercial PaaS, you know, sort of true PaaS, right? Like outside. And we say, okay, if you want to have your internal PaaS, I think the reason for doing it is really clear, right? You’re trying to sit there and say, we have this infrastructure and we sort of try, I guess maybe part of the reason that we ended up down this path is trying to shoehorn infrastructure into agile delivery. Just, I guess we finally figured out that just doesn’t work, right? Again, we sort of go down memory lane of how did we get to DevOps? Like I’ve said before, I always think of DevOps as sort of the, If you took ITIL and Agile and shoved them together and said, let’s take the things that matter out of both of them. Because historically, if you look at ITIL, it was a whole lot of, okay, this is how we think about our service desk and our infrastructure and our services and all this. And then it’s like, what about software development? They’re like, oh, they, I don’t know, they do that somehow. Right. And then we look at a lot of the ways Agile was implemented. It was like, let’s all be cross-functional and bring in test and requirements and software developers. And then those of us on the infra side were like, how do we fit in? They’re like, don’t you have ITIL? Don’t you have a thing? And I remember like going through an Agile transformation and the whole entire idea, the way that the Agile coach said is every time I brought up infrastructure, they’re like, but that’s a service that gets consumed and people ask for that from you. Like you don’t need to be in this conversation. And the fact that that’s what Agile coaches were saying in 2010 is why the DevOps movement had to happen. And so, we kind of did that, and we can argue, I have a whole other show about, did we actually do that? I don’t know that we did. I think we’re back to where we were, but better, and maybe that’s PlatEng. But part of the reason that it’s really hard to say, be completely cross-functional, is that’s not how companies are built. So, when we think about our infrastructure team, and again, I’m putting ourselves in the Wayback Machine to 12 years ago to 2010, let’s say. I think that was when the story I’m telling happened-ish, right? Like when we were at Apartments.com going through our agile transformation. One of the problems that I ran into when— because I pushed real hard and said, no, no, no, you need to have sys engineers in all these squads, right? You— these agile teams have to have embedded DevOps.
Pete: [00:12:35] Very much a term, right?
Matty: Yeah, we didn’t call it that because DevOps was you know, had happened, but nobody told me yet. I still had a year before I even heard of it. But the problem I ran into was for software engineers, we had enough of them that they could be on one team. I didn’t have enough people, so my sysadmins were on multiple teams. I sat down there and I looked, and this could become a whole rant about Agile ceremony and how much time gets spent in meetings in traditional Agile. If you looked at the amount of time that my the people on my team had to do. They had no time to do any actual work because to attend all of the ceremony for the 3 squads that each of them were on, they had no time other than to basically go to these things, which meant they didn’t go to them, which meant that it all fell apart. And so maybe the idea is if you’re offering something that you cannot be fully dedicated to that particular feature team, then you have to create an abstraction layer because then it doesn’t matter. And I think that’s what we’ve been doing all this time. And that’s the evolution into PlatEng in theory.
Pete: [00:13:39] Yeah.
Matty: Yeah.
Pete: I guess, I guess my, my like visceral negative reaction towards the term platform engineering, just all of a sudden, even though honestly it’s a great term, right? Like who wouldn’t want to be like director of platform engineering because it’s a way better title than director of DevOps. You know, that no one really wanted that title, but you know, I kind of look back in my career and 3 times now in that last decade, right? As this cloud and DevOps transformation happened, I’ve essentially built this platform, like tried to solve this problem, which is, it’s kind of a numbers problem. It’s to your point, there are going to be 100 developers and like 3 systems people, right? DevOps, whatever, release engineer, whatever you want to call them. But like, there’s gonna be people writing code and there’s gonna be a lot of them. And there’s gonna be a much smaller number of the people that need to basically deploy or operate the code, right? And so at some point, there’s this new bottleneck that gets created. And the way that at least I’ve solved it is by self-serve platforms. And whether that is kind of the original route, I solved this with Chef and EC2, where it was like, hey, here’s like a knife command to spin up a server, developer, if you need a thing. Or later on, it was like we were in the bare metal world kind of weirdly after, but that was Dyn, a DNS company that had to have hardware. But then we moved into like provision hardware via Chef, right? Run a Chef command and then get you a server, right? And now you have a thing that your app can go to. To then in one of my last companies, probably the last time I did operations for real at ThreatStack, we essentially kind of followed that same model, which was to the development team, here is the framework for your your application. It should kind of look like this. And if you fill these things out and make it look like this, then all you have to do is commit that code and it will go where it has to go and you don’t have to think about it. And we didn’t use Kubernetes, we didn’t use Docker containers, we didn’t use any of those things. We just, we try to keep it really simple. Granted, we were startups, so you have to, otherwise you’ll die. But, and it worked, it worked really well to get the developers basically to just get me out of the picture, right? If at that time it was me that was like having to like do all the systems and like our 20-something developers, right? So like I’m the bottleneck. It’s like there’s only one way out of this thing and it’s to hopefully get them to, you know, to consume the services that I would build, whether it’s the provision servers, the observability platforms I built or whatever. The one thing that I’ve actually said to a few other friends in this space, you know, kind of colleagues in this world is I think the thing that many companies are missing is that they don’t have a product. There is no product manager for the ops team, right? And I think that’s what is the problem, right? There’s no— they’re a product team. The ops team is, right? My products were, you know, release engineering and observability and whatever else. I had customers. The developers were my customers. I had stakeholders. I was essentially the de facto product manager, which is why I ended up trying to do product management work because I felt like a product manager running successful platform projects in the past. So yeah, I think that’s, that’s my hottest take. That’s probably not actually super hot. I think a lot of people would agree with it is that, you know, companies need— and I know that this exists.
Matty: [00:16:58] I’ve talked to bigger companies that have these big teams and it’s a thing, which I think we should talk about a little bit more here because it’s so important. I— a couple of things that kind of came out of that. You’re exactly right. Like if you are going to offer a platform. It’s a product and you have to treat it as such. And I gave this talk many years ago now called Everything’s a Product, and it’s applying— says, how do you apply product management practices to technology services? Most notably as the talk that includes Emily Freeman’s mother on a slide because of reasons I don’t remember, but it was a whole thing. And it was funny, like there’s little things in there like, you know, do you do an MPS survey on your Jenkins pipeline, but sort of, right? I remember we had someone at PagerDuty who was the product owner for SRE. And what’s frustrating about this is nobody talks about it, probably because those are some of the busiest people you’ll ever meet. And they don’t have time to give talks and write blog posts and stuff. Because I was trying to find— I remember even when I was working on this talk that I gave back in 2018 or 2019, whenever it was, I was trying to find anything published that Marguerite at PagerDuty had talked about. So let’s say you are part of this platform team, this PlatEng team, you know, you’re providing, you know, whatever Artemises or whatever nonsense, you name it, that is your internal platform at bigbank.com. Well, now in some ways you might be like, well, we don’t have to worry about things product owners have to worry about because I don’t have to worry about adoption because people don’t have a choice. ’Cause our CIO said we are all using Artemis, right? So versus like, I have to worry about getting all this feedback and stuff, ’cause if I don’t build the right thing, no one’s gonna give us any money and the company goes out of business. Sometimes, and I believe the, what I just said is not anything that anybody consciously thinks of, but that’s probably if push came to shove, they would be like, well, wait a minute, why would I worry about, we don’t have to worry about that because this has been decided at the architect level that this is how we do things inside the bank. Okay, but is it? Because by the way, that attitude is what got us shadow IT how many years ago, right? So if you are— what I’m getting at, if you are that plat team and you don’t build what people need, a couple things are going to happen, right? One is maybe people will use it, but they’re going to complain a lot or whatever, and you probably get some of that, but they will find a way to get what they need, and then you will fail. You still have an adoption requirement. Because you’re in, you’re at some point, maybe as the individual contributor who is the Kubernetes expert inside of there that’s contributing to that, you’re not on the hook for it, but that initiative is. And the reason this matters is because when you talk about a platform, it’s very holistic. And I feel like a lot of the conversations that I see around platform engineering are incredibly Kubernetes-focused. They are, we build this thing and it handles the container runtime. And then you’re like, well, what about the data?
Pete: [00:19:58] I don’t know.
Matty: People do that somewhere else. No, guess what? That’s part of the platform. And if you don’t offer that batteries included thing, you’re going to run into one of two things is going to happen. Either they’re not going to use your thing at all, which is bad for you, or they’re going to, they’re going to go do it the way they want to do it. And now your ass has to integrate with 15 different types of ways that someone does Kafka inside your organization. Wouldn’t it be nice if instead of that, you could do, okay, this is our observability solution, it’s already baked into it, and we gave you what you needed, and we can make sure it’s passing our security requirement, it’s doing all the things we need. I think platform too much to too many people means the runtime. Yeah.
Pete: That outcome is the inevitable outcome for anyone that’s been a part of building software before. It’s the inevitable outcome of not having someone who owns basically the product, right? You have people who are like, here’s the solution. We’re going to use it. Why are we using this as the solution? Because it’s what everyone’s using. Right. I don’t see how it’s any different. People say now, like, yeah, we’re going to use Kubernetes to solve this one. How is that any different from being like, no one gets fired buying IBM? It’s the exact same response. If you don’t understand those requirements of what your teams need, to get their job done. And those teams have product managers, and those product managers report to the people who run the company. So they have to deliver whatever those people want, and they’ll go around you if need be. I think this also, this whole not having a product manager as part of the platform DevOps teams, whatever we want to call them, this is something that I’ve— again, not just me, everyone’s talking about it, but it’s something that I’ve been talking about online recently and writing even some blogs around. This idea that, you know, we’re entering into or we’re waking up to the DevOps hangover, which is we’ve had 10 to 15 years of just excess where we could go and do whatever we really wanted to in that world. We could go and consume as much Datadog as we wanted to. We can provision as many AWS systems as we need. We can go build 10 iterations of Kubernetes. You know, no one, no one really cares because, you know, growth was the only thing that we had to do. And now we’re waking up to it realizing like, yeah, well, we actually can’t do all of that stuff anymore. Or, you know, why is our Datadog bill, you know, $50 million or whatever? You know? And so now we’re all waking up to realize that like, well, hey, hold on a second. There’s, there’s a monetary cost to running Kubernetes, right? Has someone planned for that? Have you scoped out what that cost is going to be? And not just the cost of inherent waste in having Kubernetes and not being optimized, but just like in the time cost as well. And also like half your team just got laid off, by the way, because, you know, tech is a weird place right now. You know, so I think there’s these things are all kind of like tying it together into that one little nugget, right? Of like, I guess we should have just had product managers the whole time.
Matty: [00:23:02] There, it’s that easy. The cynic in me says, if you even have someone in that role who’s probably one of the first person to get laid off. Is going to be that. They’re gonna be like, wait a minute, why does it matter? Why do we have a product owner over our platform of this thing? Doesn’t it just work? Yeah. So that’s probably—
Pete: well, yeah, I know. And that’s a, that’s a real challenge too.
Matty: I—
Pete: it’s also what kind of person might you hire for that too? Like, you, you, you know, there’s— I’ve, I’ve worked as a product manager for reasons unbeknownst to me, and then I went back and did it again because I, I’m clearly a masochist who just likes the pain of being a product owner. But it’s the least—
Matty: it’s the—
Pete: it’s like product management is like the least defined role I’ve ever seen in my life. It’s like depending on where you’re at, you could be doing more customer-facing things, just getting their feedback. You know, smaller, smaller companies, you might be doing a lot of different stuff. Larger companies, you might just own one very specific feature. What I was actually most surprised by is hearing product managers talk about their roles at just various companies. Depending on the size, some like Google, the product managers are like little mini CEOs, right? They run the whole thing like a business unit. And it’s an interesting concept if you think about it, that a business only operates if like a capitalistic business only operates if like you can have more revenue than costs. And so if the product manager is kind of running it like a little mini business, then they’re going to basically ensure that those 2 things stay in the right, you know, direction. But without it, what are you gonna do? You’re just like, oh, they asked me for more Datadog. So here you go. Get, you know, get you a Datadog, you know, go from there.
Matty: [00:24:42] It’s that simple. Just get you a Datadog.
Pete: Yeah. What can it be? What can a Datadog cost, Michael? $10, $15?
Matty: I think we sort of already went down this path a little bit. What’s the cynical look at this, right? That’s kind of our whole, whole take on it is that you’re just wrapping up the same same mistakes again, right? Or same problems. And it’s, it’s like the, The Simpsons thing. It’s like there’s 3 ways, right? The right way, the wrong way, and the max power way. And they’re like, what’s the max power way? It’s the wrong way but faster, you know? I think that’s the thing we need to look at is how do we not have this be the wrong way but faster, right? Like, how are we not just going down the same thing? We just, we’re just moving the silos around, which can be okay. I think we, we started this conversation with Sometimes those silos are important. And I think there’s another tangent of this. I’ve been avoiding the Blue Hell site because it’s making me upset. And actually what I’ve decided, what’s funny is what makes me more upset about it lately is less about like every time I switch over to Twitter off of something else, it’s not even so much of like all the Elon stuff, which is a lot of it, but also having distanced myself from it because of that. I’m like, I love tech and I love talking about tech, but it’s really frustrating. As soon as I go to Twitter, I’m like, oh my God, everyone’s just fighting again about whatever. And it’s— and, and, and this is not a dig at like the people involved in this conversation about to say. They’re all people that I really respect and love. But even this, even this particular topic, I went, I just have to— I can’t even look at this thing. And it was, you know, there was a thread going with Kelsey Hightower and Adam Jacob jumped into it and a bunch of people and about shift left. And Kelsey’s thing was, all we’re doing is moving all the work to the left and we should never— and everyone’s like, no, no, no, it’s security’s always been blah, blah, blah and all this stuff. And I’m still like, all of in my mind, I’m not saying Kelsey is right or wrong. I would not say, but I’m still like, yes, that’s what I’ve been complaining about. And everybody in that thread is still missing my point, which was we’re just thinking about it earlier. Like the shift left is not move, is not dump everything on the developers. And the problem is when people say that with the security is everyone’s job, then it’s easy to retort to that like Kelsey did, which is, I’m sorry, this is how it’s been in software for my entire career. Yes, we all think about security. I’m like, great, but how are we having the conversation earlier? How are we taking our domain experts earlier? I’ve lost my train of thought about why I went on this subject rant, but it’s sort of like, I guess, in the platenged thing about how do we look at things more holistically without it turning into this, like, because that’s what DevOps was supposed to be and we screwed that up. So whatever, whatever happened happened. And we still— oh, and that was sort of my thing. Sorry, this is how I got to what I said. Silos are okay because the idea of all of this work is not to say everybody knows everything and everybody can do everything. We’re still like, you still have domain experts, but how do we get them to work together better? And I think that’s the same thing if we’re not careful, like in the PlatEng, you still go like, okay, well, what we’ve done is I just like— and this is my question that I was sort of talking to Daniel about a little bit— was how much of this is the PlatEng is just the SRE team, which actually was just the rebranded ops team, and with the cloud ops team, which used to be the sysadmin team. And it’s like, I had a coworker who was like, I have sat at the same desk doing the exact same job for the last 12 years, and I’ve worked for 5 different companies, right? You know what I mean? Like, this was in the, the heyday of outsourcing. And but the same thing, right? How many of these teams are just a continued evolution of what was the sysadmin team 15 years ago? And but the work hasn’t fundamentally changed, we’re just rebranding. And this goes back to if you were not the one on the episode, I would be quoting you and saying, this is why I’ve made my peace with, you know, fine, whatever, rebrand yourself the DevOps team, go get some more money, as Pete would say.
Pete: [00:28:41] But I was just about to say, I don’t know, how much does platform engineering pay? If it’s 24% more than an SRE, then guess what? I’m going to be a platform engineer.
Matty: So that’s good for the individuals, but does that help the organization? That’s what I’m saying. Like, I’m thinking about what does— Does this help us if we’re still doing, I mean, how many people sitting in a PlatEng team today are doing fundamentally the same work they’ve done for the last 10 years in a way, but, but now it’s on Kubernetes instead of on EC2, instead of on vSphere, instead of on Solaris. Right. You know what I mean? Like, but you’re still doing tickets for people. You’re still doing requests. Like, are you really treating it like a platform? Again?
Pete: Yeah. It’s, it, it, It’s the step that we’re still missing. And I think only people, only some people kind of figured this out where this requirements gathering step is like talking. If like, what’s the DevOps thing that we screwed up on is that we might’ve broken down some silos, but like people aren’t talking to each other, right? We haven’t solved the people part of it because I think if teams or people or a person, right? That person who might be like kind of like a product manager, who is a type of person who would actually have stakeholders in a bunch of different areas and report back across the business and could kind of see across the business to understand what the requirements actually are. And maybe this is your director of platform engineering would do something like that. I think that’s how we end up to make this actually be a better thing, right? Which is, it’s not like build a platform, get you some Kubernetes and everything will be fine. It’s just like, Before you do any of those things, before you get your first YAML, whatever Kubernetes thing going, it’s like, can you talk to the other members of your organization to just understand what is the outcome you’re trying to get to? Right. But it’s like, we’re so fascinated with solving the problem. Like we see, you know, we see the hammer. And so we pick it up and just start bashing everything.
Matty: [00:30:44] Right.
Pete: We see the Kubernetes and we’re like, no, no, like it’ll work. And, and we just run with it versus just maybe not doing that and, and talking to someone.
Matty: Crazy. It’s almost like this is all about communication.
Pete: It’s, it’s like a people problem, right? It’s always a people problem. Humans.
Matty: So I guess one of the things, and I will admit I should know more about this before I talk about that, but I’m a dude with a podcast. Why ever would I know what I was talking about is everybody’s excited about Backstage. That’s a thing, right? And I feel like in the, the conversation— but the other thing to remember is Backstage is not a platform. It is a tool for creating your portal to your— which would be to your platform and to your things. So I think that’s an important thing for people to remember is, again, you’re not just gonna Rub some Backstage on it, now you have a platform. Because again, your platform is a product which is made up of lots of things. It’s very esoteric. I might even venture to say is a socio-technical system. And again, it’s all the socio part of the platform is I think what can be missed because it’s easy to focus on the technical, on the, oh, okay, well, we’ll do, you know, we’ll rub some Tanzu on it, we’ll rub some whatever on it. It’s like, okay, but how are you working? What’s all the things around that? You can, and I want to be clear, like, this is the kind of thing that would be hard for someone who’s working on one of those things. If you’re not, if you don’t have that, you’re like, I could, you could be listening to this show and be like, I am an individual contributor on the platform engineering team at this large retailer. And I agree with everything you’re saying, but that’s not how we’re doing work here. And guess what? I’m not someone who can fix that.
Pete: [00:32:29] Right.
Matty: Right. So that’s tough because again, these are the problem I think you run into with any time you touch platforms. Is it requires buy-in and effort and work from many, many people that are at the chop wood, carry water level, but they require decisions and purchase and vision that is across the whole organization. So it’s kind of like, how do you connect those two? And it’s like one of the reasons that when I was at Red Hat, like selling OpenShift is hard. And the reason it’s hard is because you won’t implement it without buy-in from your chop wood, carry water, right? Like you have to build that thing. The problem is it’s so expensive as a buy that you can’t just like have a team of 2 people decide they want to OpenShift some shit and whatever. You might be listening and tell me, no, no, no, Matty, there’s a way you could do that. But reality-wise, That’s not, that’s not how deals get done. So you have to like, it has to be a little bit of a mandate because you’re buying half a million dollars, a million dollars, $2 million of open share because it’s, it’s this big buy. So it has to go all the way up to the C-suite almost, right? Or to the very senior level. And then how do you connect those 2, right? And then you get screwed in the frozen middle as usual.
Pete: [00:33:53] This is like my favorite story from a company years ago that did DNS for the people, and then they don’t do DNS anymore. But when I went into that company, they had servers and data centers. They had some automation. They had actually all the automation. They were using all the things because teams were very siloed, right? And so it’s like one team would solve it one way and moved on, and one team would solve it a different way. And so my hope was to bring this platform, which was like API-provisioned bare metal infrastructure, And even the term platform got abused internally. It was like platform V2. It’s like a trigger word for me now. V2 is always bad. I hate anything that’s labeled V2, but everything just kept on getting dumped on. Oh, well, platform V2 fixes that. And platform V2 fixes that to the point that I just had to stop everyone and, and just say, here is a 10-page like product requirements document. Here’s a 10-page document that details the what and the why and what this is and what this isn’t and all the details and the budget and everything else around And also the project is now called Honey Badger because if anyone says platform again, I’m gonna like lose my mind. But the, the, the endgame around that is we, like, I had to basically build this like almost grassroots campaign in the business as different teams all saw the same problem. We all knew there was a problem and we were all trying to solve it in ways that worked for us, but no one was trying to solve it holistically for the whole business. And so that’s what I was trying to do is talking to every team to find out, Well, they like this thing because they have this very specific requirement. Okay, cool. Like let’s track that one. And this other team likes this other tool because of this other very specific requirement. You know, can we essentially distill this down to its like core principles of like, what’s the absolute most minimum thing we need? And hilariously it was provision a server somewhere in the world with a default operating system and give it a Chef role, right? That was kind of what we distilled it all down to. It’s like, let’s just start there. Right? We don’t actually have to go and solve all these one-off edge cases. We can just start there. But it only worked because you had cross-team buy-in, not just in the different teams who were going to be your product users, the customers, but also the budget required to direct an engineering function to go build this and the capital cost of buying servers and all the things needed to do it. Right? It’s far— and not that Oh, well, you’re saying like, oh, it’d be easier if you just do Kubernetes. But like, I don’t think so. Right. I think like, I think it might feel easier to like spin up a Kubernetes stack on EC2 and, or use EKS or something and say, yeah, we did it. But like, there’s a lot more that you have to figure out before you can really be successful.
Matty: [00:36:41] So I just had a thought, like, this is all the hard stuff of, of, of, of making this happen. Right. It’s like, how do you, you, a lot of things you’re talking about, I was thinking, So this is my little— again, not knowing who could attend, but anyway, there’s this talk coming up at DevOps Days Chicago in a couple months that was supposed to be a talk we had last year, but unfortunately the speaker Joshua Zimmerman, he had COVID the morning of the conference and couldn’t do it. But it’s called Never Tell Me the Odds: Navigating Organizational Politics. And it is 100% about how do you— it’s like, how do you identify the structures inside your organization, learn how decisions get made in your organization? How can you affect all this change? So I would highly recommend if you aren’t already, try planning to attend DevOps Days Chicago in Chicago, August 9th through 10th. Arrested DevOps listeners, if you would like 25% off your DevOps Days Chicago registration, which you can go to at devopsdays.org/chicago, use the code PeteIsCool, one word, PeteIsCool. But I also have 2 free tickets, which will just go to the first 2 people to register with the code ADO-FREE. So ADO, like Arrested DevOps, dash free. If you go to devopsdays.org/chicago, go to the registration, the first 2 people will get a free ticket. Free ticket does not include travel and hotel and stuff. You have to get yourself there on your own, but at least you will save you $299 of registration or use the code PeteIsCool to get yourself 25% off your registration so you can hear Josh Zimmerman talk about it. All right, enough shilling about that. Speaking of shilling, Yeah. So Pete has a little project he’s been working on for a while that, like all good projects, started off as like basically some form of a shitpost that now has consumed probably 50% of your life. Do you want to tell us about it?
Pete: [00:38:30] Yes, I, I do. So I’ve had this idea in my mind for quite a while in that— and again, not unique at all, but apparently when I think of it, it becomes unique globally— that I’m honestly fascinated with how we pronounce tech words that we often read and rarely, rarely say. We read them, we write them, but we rarely actually say these things. And this goes way back to my first job as a sysadmin 20-something years ago and a lot of the words that I learned from there. And so I had this idea for a while that it’d be fun to record friends of mine pronouncing these words in kind of a game show style format. Well, you know, life happens and company changes happens and You know, it’s still kind of in the back of my mind that this would be a lot of fun. And I was in a Slack organization where some former colleagues of mine started making fun of me about how I pronounce a specific load balancer spelled H-A-P-R-O-X-Y. Some people might say HAProxy. I had to struggle there because I say HAProxy. I don’t know why I say HAProxy. Then when people were like, do you know you’re saying it wrong? And this was after the whole company learned how to say haproxy as haproxy. They all went off to other companies. I just like, if, if I was a troll, it would have been my most effective troll ever.
Matty: [00:39:43] I like how you say if you were. If I were.
Pete: So then, then people started asking me about that. They’re like, well, what, why do you say that? And I had to like, I don’t know if this is the right term, like retcon like the origin story of how I did it. And I basically was like, well, You know, it’s like, haha, it’s a proxy. And then people would be like, oh, that’s funny. And they wouldn’t ask me any more questions, thank goodness, because they would find out I was completely full of shit. So anyway, that’s how it started. And the company I work at now is a company called AppMap. And I do— I mean, it’s a 10-person company. We do— I do a lot of things for them. And one of the things I’m always looking for is like, you know, fun, interesting marketing-related things, which is like not really like related to like runtime code analysis like we do for our business, but just in like, hey, like, let’s have fun, right? And so they were very kind enough to think that this was a good idea that I should go and record these things and spend my time doing it. And so that’s what I did. And so there is a series now, I’m actually been posting these weekly, I would say is the iterations every Friday, I kind of spam them across all the social media sites. And it’s essentially how do you say, and this first iteration that’s out there right now is what I would call the supercuts based on actually Matty Stratton’s great advice that, you know, there are 20 words and I’ve split them out. Maybe 18 or so have very different pronunciations. And, and yeah, they’re like 30 seconds long and you basically hear people pronounce different tech words in wildly interesting ways. And so some of the ones we’ve done recently, let’s see, I’m actually going to my like little release order. So we’ve done favorites like SQL or sequel or squirrel was my favorite pronunciation of that one. Another one which I was actually— I added due to a coworker had mentioned it was, I say epoch, E-P-O-C-H, or epoch was very common as well based on like, you know, computer time counting. Fisk, F-S-C-K, was probably one of my favorite ones because It was incredibly varied. And then when I looked up the Wikipedia site, there were, I don’t know, 20 different alternative pronunciations to which then I recorded myself saying them all, which I thought was pretty funny. But then another one too that really came out of, you know, when you, again, when you start talking to people about, hey, what are some words that you don’t know how they’re pronounced? Or, you know, that you only write and you don’t say. There is a technology from Auth0 or a standard or a framework. I don’t exactly know what they refer to it as, called a JSON Web Token. It has to do with authentication-related things. And, and, uh, it’s, it’s written out everywhere as JWT. You can actually go to jwt.io and you can learn more about what these JSON Web Tokens are. What’s hilarious is in the docs, it is, it says JWT pronounced jot, like J-O-T. And, and I was like, this is fake news. And I, I’m gonna bet that I’m going to record 30 people, which is how many— 31 people is how many people I interviewed, and no one will say Jot. And that’s going to be my great glorious thing. And then at some point people started saying it and I was like, how did you even know? And they’re like, yeah, well, we did an implementation of that. And that’s why.
Matty: [00:42:59] That seems to me though, I’m just going to take a step back, like bad marketing if that’s the case. Because if I only hear that, I will never be able to look it up because I’m going to look for J-O-T. Like if someone’s giving a talk and is like, or especially worse, they’re on a podcast and they’re like, oh yeah, well then we use JWT to do that. And I’ll be like, well, that sounds cool. I’m going to go try to find it. I am never ever in the wildest dreams going to type in JWT.
Pete: So, well, I’ve realized too is that many of these words are actually pronounced the way they are, or at least I’m going to pretend they are, as actually a way to help someone else who’s listening to you be able to type them out. And my favorite examples of these are like the etc directory, Etsy, you know, the Etsy directory. And actually that’s kind of split. Some people say Etsy, some people say ETC. And even like other ones that I actually, I didn’t do as part of this one were be like the library directory, like on a Linux or Unix system, you have /lib, L-I-B. Most people would say /lib because if you said /library, right, someone would actually type that out. Or, you know, so it’s, it’s funny because that’s what I just thought of.
Matty: [00:44:05] I would always say lib. I would never, I’ve never heard anybody say lib. Doesn’t mean it’s Isn’t that wild?
Pete: But you know what, because it’s, it’s the library—
Matty: like, there’s some— been some sort of memeing going around or threads about like words that you’ve been pronouncing incorrectly because you only ever saw them written in books, right? You know, you’re like, I’ve never heard it, but I read a lot when growing up, and so I always knew this word, and then I encountered it 20 years ago in spoken language and we’re like, that’s how you say that? You know, and that’s probably true with a lot of this stuff because You see it in documentation or in code and you’re not talking about it with mouth words so you don’t know, right?
Pete: Yeah. And honestly, when I had this idea in my head, what I didn’t want is to create some series where we’re gonna make fun of how people pronounce stuff because that’s stupid. And I say it, all pronunciations are valid. Any pronunciation you have is valid for this and that is totally fine. So that was a big part of it. And yeah, a lot of these things I learned because I picked up a book to learn Linux or, you know, but even in more modern times, you know, you’ll learn because like your coworker will send you a Slack DM of like, yeah, run this command. And all you’re doing is ever reading it. And so there’s, there’s no one there to tell you. I think what’s funny is that there are tools out there that claim here is how they’re pronounced and they put it on their website, which is cute because people are going to pronounce things however they want to, you know, without your pronunciation, you know? So I, you know, I didn’t want it to be like you’re, you’re doing it wrong.
Matty: [00:45:36] So I really want to, I want to say despite that, The genesis of this entire project was someone telling you that you were wrong, right? Which is not the right way to do things.
Pete: Yeah, exactly right. You know, people telling me I was wrong and is, and I’m like, well, I don’t want to make fun of other people for it. Yeah, exactly. I think that’s, you know, but because there’s also the other aspects too, is that just in, in accents, the way people accent in different languages, English, people who are English as a second language or not as a primary language will have different pronunciations for different words as well. There’s a lot of people not native English speakers who are technologists as well. And I almost love to find a bunch more people globally, internationally for this one to kind of see, you know, how they might pronounce some of these things, you know. And so, yeah, it’s been honestly, it’s been a fun project. As Maddy, you kind of mentioned, I’m drowning in content, which for many companies is a dream, except I’m trying to keep up a cadence of posting these things out there on a weekly iteration. And so that’s basically what I’ve been doing is editing and posting these. So yeah, like I said, it’s been a lot of fun. You can find them on my Twitter @PeteChesslock or my LinkedIn. I post them there. I’ve honestly been testing out all of the social medias. So just because I’ve never had to do such a, like a wide-scale kind of video sending out things. So I’ve put them onto the AppMap YouTube page. I was playing around with YouTube Shorts, and, and I actually, I, I went, I found, uh, I found a Zoomer to help me figure out how to make TikTok work, and I’ve posted them to my, my, my Pete Cheslock TikTok.
Matty: [00:47:12] We’ll put a couple links in the show notes to maybe where to find those as well, if you— because maybe the way that you would pronounce Pete Cheslock is different than the way it’s spelled, and you would go to the wrong way. I don’t, I don’t know how that’s possible, but it could happen. You’re like, trust me, I know.
Pete: Whew. I’ve, I’ve heard, I’ve heard Shazlak pronounced some, some unique ways. So you would think it’s, it’s, it’s impossible to screw up.
Matty: But I will tell you this, I don’t know about pronouncing, but spelling.
Pete: Yeah.
Matty: I, I’ve told my kids this before because Stratton is pretty easy to spell, so to speak.
Pete: Right?
Matty: Like generally it’s pretty much phonetic. Yes. There’s a double T and stuff like that.
Pete: Yeah.
Matty: But my name, Matt Stratton, is pretty close to Joseph. So I, I, I remember my senior year in high school, I, there at the VFW, they have a thing called Voice of Democracy. This is not important to why I’m telling you the detail of why I was, my name was in this program, but I had won this thing at the VFW Hall. And so we went to a, a ceremony. And so there was a program and my name was in it 3 places spelled differently and incorrectly every time in every place. And I was like, first of all, I didn’t know that there were 3 different ways to, my favorite was Strahen. It was, I think someone obviously reading the handwriting thought the double T was an H, but it was S-T-R-A-H-E-N. And I was like, and then I was, I was, yeah. The other one, and this is just, I, this is an autocorrect thing that happens, but being Matty, I cannot tell you how much I get that is Marty. But I don’t think, because every now and again I think people think they read that and my name really is Marty. But I also, I’m pretty sure 90% of the time it’s an autocorrect, you know, that someone was tweeting it and then it goes to Marty or whatever.
Pete: [00:48:49] But I have many flaws. One of my top 10 greatest flaws is my inability to pronounce largely anything. I don’t know if that’s just because I’m from the Midwest. Or what, you know, we really struggle with pronunciation. And then I moved to Boston and like we just throw ours around wherever. Like you just— there’s no way to need to know how to pronounce things. But for this, this, you know, kind of game show style I did was very much like, hey, and welcome our guest, you know, Maddie Stratton, you know. And there’s a lot of folks that joined. I’m very, very bad at pronouncing names. There’s nothing I love more than when people have like a transliteration of their name because like I just seeing the letter combinations together like that just, you know, I can’t, can’t make the mouth words happen. Uh, and so that was a fun experience of the unrecorded aspect of those videos was me spending a minute or two at the beginning to be like, all right, I’m gonna say your name a bunch of times and you tell me which one I got right.
Matty: Well, you know what? So again, if people are listening, because I think about this too and I don’t always think about it as well. So some of this is— there’s, I would say, I think that’s— I really love the way you approach that, which was to say, I’m going to say this a couple different ways, tell me. But I would even go so far as to say, maybe I would reframe it to say, like, I’m going to say, which one do you prefer? What sounds best to you? You know, or something like that, because it takes away from the right. But I do, I do think, and this is a little bit of a tangent, but like, as we think about inclusivity, a lot of times people will say, like, uh, you’ll also— I guess all I’m going to write out is that if you’re going to do a thing like that, especially if there’s multiple people in the conversation, Ask everybody, even if someone’s name is one where you’re like, I know that, I know that. You just don’t assume. And I actually feel like it’s always— unless you are like inherently friendly with that person and you’ve said their name a bunch of times, it never hurts to say, tell me how you would like your name pronounced. The other reason— there’s a reason why— oh, I remember we did this on DevOps Party Games. We would go around and say, how do you want us to refer to you? And the reason was because that was a way to get people— among other things, first of all, because some people are like, just call me Pete. Some people— I mean, in that case, what it was, that gave the opportunity to have someone say their name and we could hear how to say it. But it wasn’t like, oh, I don’t know how to say your name, but I know how to say Joe’s name. It’s just like, everybody do that. I have a quick question and then we’ll kind of— as we kind of wrap up. So when you talked about not being able to pronounce things, that even into the course of tech, but what’s a word that you stumble over still to this day? Like, do you have one? Like, I have one for me that I cannot get. And it’s crazy. And I avoid saying the word, like I reframe how I— words I use so I don’t have to say it if I can.
Pete: [00:51:25] But so I’m even gonna screw up saying that.
Matty: That’s what I want. That’s what I want to happen here, actually.
Pete: Yeah. Yeah. Well, so I’ll tell you why this still comes up quite a bit. So, you know, I’ve got, I’ve got kids who are, who are generally still pretty little. A nighttime routine I’ve done with them now for their entire lives is we read book. And as, as they were younger, we’d read Goodnight Moon, which is good stuff. As they got older, then they moved into chapter books, and that’s kind of where I’m at right now. And so reading to my, my oldest, who’s in middle school still, she loves it, just kind of chills out while I read. I have to do all the work of reading the book and reading out loud to her. I, I get caught up on just so many words, but the ones that still get me to this day are through, throw, through, th, you know, like any of the like th, r-o-u-g-h, through, Though, like, I just, I, I hit that and it’s just like I make it through the first half of the word and then like my brain just stops. I’m like, wait, wait, wait a second. And I get caught up on there. It does not help also that my daughter reads, she’s into like fantasy books and it’s just, I, there’ll be a word that comes up and when I read fantasy as a kid and I got stuck on a word, I would just be like in my head, I would just be like, and I wouldn’t even attempt to pronounce it and I would just move on. I can’t, when you read it out loud you have to say the word. Then I try to just say, oh, whatever, and I move on to keep going. But then it shows up repeatedly and I’m like, oh god, this is killing me.
Matty: [00:52:51] I was gonna say, first of all, my word that I stumble over and cannot deal with is arugula. Did I say that right finally? Arugula. Oh wow, that was hard.
Pete: I never do it.
Matty: I instead just say rocket like they do in England because it’s easier. That same type of lettuce is called rocket. In the UK, and then nobody here knows what I’m talking about. It’s just helped by the fact that I don’t like it, so I rarely will have to say it. I was just saying the last thing when you talk about like reading the books and how you would read them differently. So I’m a big fan, I love audiobooks. I mostly listen to audiobooks of books that I’ve already read because I listen to them to fall asleep. So it’s important to say that these are books I’ve read, and a lot of times books that I’ve read many, many times, grew up on and such. And so what gets me is character names, especially in something like fantasy or where their names are not Sally or Fatima or something like that. They’re Falar or Ibadaj or whatever. And then like, I’ll be listening, and especially in a series that like I’ve known, and I know this guy, and then you’ll like hear it and you’re like, who are you talking about? The other one that gets you is like a narrator who decides the voice of the character is very different than you had in your head. And I’ll tell you, which, that’s one thing, but the one that really messed me up is, so there’s a whole series by Anne McCaffrey I love, The Dragonriders of Pern, and there’s tons of books in this series. And I’ve listened to all the audiobooks. And the narrator, like, the same person doesn’t read all of them, which makes sense because there’s dozens upon dozens over how much time. And so I listened to a whole bunch of them, and then I went to one where the narrator switched, and the one character who the previous narrator had always spoken of him like this. It was very regal and blah, blah, blah. Suddenly he got a drawl and I was like, wait, what? Like, this is— first of all, that character should never have had a drawl in the first place. Like, that was— the other implementation was a lot closer to what I would imagine. But then I was like, this is just— that was a choice, man, is what I’m saying there. So yeah.
Pete: [00:54:49] Well, on that point, I mean, I’ve watched a lot of shows with my daughter after we have read a series. And she’ll just look at me whenever they say one of the words. She’ll just look at me. And I mean, like middle schooler, she’s just got all the attitude that she’s happily giving to me. And she just says, wow, you weren’t even close on that one.
Matty: You know, like, nope, that’s how you aren’t even close on that one is how I felt watching a couple of Pete’s videos that I participated in afterwards. I went, oh, okay. Yes, I totally did not know. So With that, uh, you weren’t even close on that one. Hopefully we were a little close on some of the things we talked about. If you go to arresteddevops.com/devops-with-better-marketing for this episode’s show notes, we’ve got probably some links to things. You can always go and find us if you go to arresteddevops.com/itunes, leave us a review in the iTunes store. That can help other people find the podcast. So I’ve been told, yes, I know. You know what? I’m not even going to say my joke about how it’s not called iTunes anymore because I have said this in every episode for multiple years. You all are tired of hearing about it. So, you know, if you’re, you know, if you’re a new listener, listen to an older episode to hear my stupid joke. But in the meantime, you can also find, if you’re new listening to Arrested DevOps, you want to find other places to find us to listen in other ways, we’re also available on Spotify, iHeartRadio, Audible, all sorts of places like that. Pete, thank you so much for joining me today. This was super fun. It’s always a good time to have you on the show. And is an excuse to just, you know, talk to people I like to talk to.
Pete: [00:56:22] I know, I appreciate you having me on here and for being able to shill my How Do You Say series. So, and if there’s, you know, as if you do follow me on Twitter, or you can, you know, there’s, there’s talk of a season 2, and, and I’ve got a form that I should probably pin on my Twitter. Maybe I’ll put a link somewhere that if you have words that you don’t know the pronunciation of or think would be fun in some sort of episodic series and/or want to join for one of these, I’ll go grab the Google Form and you can feel free to fill that out. And if I decide after a little while of not editing, you know, 30-something videos that I want to do this again, then I’ll shoot you an email and we’ll do it again.
Matty: Sounds fantastic. This has been Arrested DevOps, and remember, there is always DevOps in the banana stand.
