how to eff up devops

Matty: [00:00:07] Welcome to Arrested DevOps, episode 14, How to Fuck Up DevOps. I’m your co-host, Matt Stratton, @MattStratton on the Twitters.

Trevor: And I’m your co-host, Trevor Hess, @TrevorGHess on Twitter.

Matty: Arrested DevOps is brought to you by TenthMagnitude, a cloud services company that figures if you’re listening to this podcast, You’re pretty cool. You can find out about joining their cloud services team at 10thmagnitude.com.

Trevor: Joining us this week, we’ve got Pete Cheslock, Nathen Harvey, and Randi Harper. Welcome guys.

Matty: And this episode is sponsored by PagerDuty. PagerDuty eliminates the noise, chaos, and manual processes across the entire incident lifecycle to decrease resolution time. PagerDuty is trusted by companies like Etsy, Nike, and GitHub. To sign up for a free 30-day trial, visit arresteddevops.com/pagerduty.

Trevor: This podcast is also brought to you by Datadog, a monitoring tool that helps bridge the gap between operations and dev teams. Datadog brings together system metrics, changes, alerts, and events from over 70 common infrastructure tools such as Chef, Docker, and AWS so that the dev and ops teams share their key data and alerts in a single place and collaborate on issues in real time. Datadog is available for a 14-day trial at arresteddevops.com/datadog14. That’s Datadog and the number 14.

Matty: [00:01:28] So now that we’ve paid the bills— thanks, sponsors— we’re going to introduce our panel. Uh, we’ve decided that a good way for our guests to introduce themselves is to tell us what exactly the word DevOps means to them in a couple sentences, no dissertations. So, uh, Pete Cheslock, how about you introduce yourself and Tell us what means the DevOps.

Pete: Hey everyone, yeah, really happy to be here. My name is Pete Cheslock, @PeteCheslock on the Twitters. I’m a rabble-rouser of DevOps, and yeah, what does DevOps mean to me? I kind of take the historical view of DevOps, the classic Adam Jacobs cultural and professional movement. So I think it’s, you know, a little bit tools, a little bit culture, kind of meet in the middle, shake it up, and boom.

Trevor: Awesome. Hey, Nate, what do you think? Nathen.

Nathen: So first, hi, I’m Nathen Harvey. I work with Matt over at Chef. I was super lucky because after he got a job there, he convinced them to hire me as well, which was awesome. But so I think DevOps is really— to me, DevOps is really all about everyone sort of working together towards a common goal, and that goal typically is a delightful customer experience. So what do we need to do in terms of development, operations, business, marketing, sales, finance— how do we all come together and make our customers super happy? That’s what DevOps is to me.

Matty: [00:02:51] I dig it. And finally joining us is Randi Harper. Randi, tell us what means the DevOps.

Randi: Hey guys, I’m Randi. I work for KicksEye, also known as FreeBSD Girl. What does DevOps mean? Well, I took a break from engineering for about 4 years, and this is after being an engineer for about 11 years. So when I came back, There was this title called DevOps, and I had no idea what the hell it was, but it sounded like what I’d been doing for the 10 years prior. So as best I can tell, DevOps is sysadmins who actually know what the hell they’re doing. It’s the people who aren’t afraid to get in there and use strace and use developer tools and actually work with developers to, you know, figure out infrastructure and problems and work together to solve these.

Pete: Can I retweet that? Because that was perfect.

Randi: Did my stuff just go up?

Matty: Oh, we will.

Pete: We will.

Trevor: It’ll be on the Twitters. So our topic for the evening is how to fuck up DevOps. We talk about, people talk about, everyone talks about how to do DevOps right. Tonight we’re going to talk about how to do it wrong.

Matty: [00:03:55] So why don’t we—

Trevor: oh, Matt just has to stick to the script word for word.

Matty: I wrote it!

Trevor: Matt doesn’t want any of the ad-libbing. No ad-libbing from Matt. Let’s start off with the, with the kind of getting the waters a little muddy. What are some common misconceptions about what DevOps is so that I try to stick to Matt’s script and then garble my words?

Nathen: I think a great misconception is taking the role of sysadmin and putting DevOps engineer onto that role and saying that, hey, if you’re a sysadmin, you’re now a DevOps.

Pete: Yeah, you know, and we’ve all totally been there. As someone who might have chosen to be a DevOps at one point, I know, I know, Randi, you’re also part of this club. Sometimes we can’t choose our titles, but I will say this, which is I’ve kind of stopped fighting this fight because if people want to use DevOps as a way— and Randi, I think you nailed it— it’s operators using developer tools, writing code, It’s like this, it’s a qualifier for kind of a senior-level experienced operator. Like if people want to use a term to classify that versus like, what was the old terms? Like operations engineer 4. I mean, if we want to have a new term for that, I think that’s okay. You know, I’ve kind of, I’ve kind of realized that, you know, I can fight that battle only so far.

Randi: [00:05:19] You know, even here in my organization, we have a bunch of different terms for it. Like on my business card, it says like senior senior DevOps engineer, but I’ve heard myself be referred to as an embedded engineer by people on my teams. All these terms are kind of interchangeable. There’s a lot of knee-jerk reaction around the word DevOps, but I kind of liked it. I saw it and I was like, yeah, that’s mine. And the pay scale was pretty nice for it too, as opposed to like, you know, your— the stuff before that.

Pete: Actually, oh sorry, I was just gonna say I had someone I worked with, uh, years, years and years ago when I first heard the DevOps term. So this would have been like 2010, basically said— and this is the best description I had of DevOps at the time— it was like, well, I do dev work and I do work, so therefore I’m DevOpsing, right? And it was— these were people who you couldn’t say they were ops and you couldn’t say they were dev because they were so good in both areas. But part of me still kind of sticks true to that one, which is like people who are really good at ops and people who are really good at dev, and it’s like this senior-level world where it just— it does kind of work. Roll with it, I guess.

Nathen: Yeah, I did find— I did find that when I found the lies on the internet and stayed up late at night to correct them, And I didn’t say ever.

Matty: [00:06:23] Someone on the internet is wrong.

Nathen: Yes. So when people call themselves DevOps engineers, like you, Pete, I’ve kind of capitulated, like, fine, internet, you win. You can call yourself a DevOps engineer. You can be a DevOps team. That’s, it’s okay. But for me, what it really comes down to is DevOps is more than just sysadmins with a new name. It’s exactly as you’ve talked about, Randi and Pete. It’s like, it’s the sysadmin who has a broader view. Than just, is the disk full? Are the CPUs performing well? Are we starting to swap? And if so, then what the fuck did the developers do? Because it’s their fault, right? Like, so I think that that is a good distinction also. And kind of going back to my definition, right? What we care about is not the individual bits, it’s that customer experience or the full system’s thinking almost, if you will. And you can’t think about the system by thinking specifically about one small piece of it.

Matty: To me, the problem I have with DevOps engineer, and this is, I, you know, know more from my geo, like in the Chicago area, this is a pretty big problem right now. And you guys might be able to speak from your locations, but it’s totally this, like you said, stamping sysadmin with a DevOps engineer job. We’re not hiring sysadmins in Chicago anymore. There are no sysadmin jobs. So if you want to be a sysadmin, too bad. But boy, are we hiring a lot of DevOps engineers. And these DevOps engineers have job descriptions like, install patches on Windows servers and perform backups. And it’s, you know, it’s ridiculous. And I think that there’s a couple problems with that. I mean, one is that it misses the point, right? Which is you’re saying, okay, I’m just going to throw a different label on it. But it’s also kind of counterproductive for those people who are doing that hiring, because I think the idea is like, oh, well, we’re going to catch people like in our little recruiter net, right? Because people are looking for DevOps and they want to do that. And the people who really want to do the DevOps are not going to want those jobs. And you’re not going to hire those people. And then what you do run into, even now, as someone who’s been hiring— well, not now, because I don’t get to hire people at Chef, but previously— there are people that I would see, they would have like DevOps experience, so to speak, on their resume. And then I look at them like, you guys, yeah, that was your title, but you don’t— you know, I’d interview them and be like, so explain to me what continuous delivery is. And they’re like, I don’t know. You know, it’s like, but I’m good at patching, and our group was called DevOps, and it’s, it’s diluting. But I don’t know, again, I don’t want the term to die because if nothing else, I would have to get new license plates. But I would like us to kind of maybe take it back a little bit, right? You know, at least help guide it, right?

Pete: [00:08:58] So, so I’ve been, as some people may know, I’ve been out of work now for like a month and a half, which has been pretty nice, I have to say, taking a little fun employment time. But I’ve definitely been meeting a lot of companies to talk about some of their challenges and The one thing I’ve definitely noticed is a lot of companies have the same problems, and their problems are, we’re trying to do continuous deployment, continuous delivery, we’re trying to improve our operations, we’re trying to make developers and operations work better, all this business value. There’s all these things that they say. It’s very repetitive in kind of a funny way, but they always want to throw kind of like the DevOps label on it, and in some cases, I’ve even had places say, I want the DevOps. Like, can you bring me the DevOps? You know, and the issue that I’ve seen and things I’ve seen in the past have been, you know, companies I’ve worked for or even places, you know, other friends of mine have worked for where they say, come and give us some DevOps because we want all this change. And then you tell them like, well, you need to change the following things. And they’re like, whoa, whoa, whoa, we don’t actually want to change anything. We just want the DevOps, you know? And so that’s where a concern I have a lot where, you know, there’s some bit of, and I love what you were saying, Matt, about kind of taking the word back because there’s like the conflating DevOps with change, but it’s like, well, only if you actually want to make some change will you, you know, get that change that you really desperately need.

Nathen: [00:10:19] So I do think you raise another good point though, Pete, around a good way to mess up DevOps. And that is to bring in a DevOps expert to change your organization and to bring DevOps into your organization. I think that, no offense, I know you’re a DevOps thought leader, Pete. But just to hire Pete Cheslock and expect that now we’ve got the DevOps, like, I think that’s a really, a really bad way, or a really good way to fail.

Trevor: Isn’t that just like bringing in an Agile coach to teach your team how to do the Agile and get everything going Agile and being, you know, more Agile?

Pete: Oh, we’re going to Scrum. We’re going to Scrum like crazy. We’re going to Scrum all over the place. We’re going to have to Scrum it out of that DevOps.

Randi: So I was brought into this organization and we’re kind of switching over to the DevOps mentality. Like before there was a centralized ops group. Group, and our developers would interact as little as possible with the centralized ops group. And now we have DevOps/Embedded Engineers or whatever you want to call it. And it’s this huge cultural change. And I kind of like the term DevOps because it’s kind of a bridge for developers. Like, they see this new title and they don’t see you as just being this ops person where they throw stuff over the wall. I know we use this metaphor a lot. And it’s really hard to kind of drill the point home that they don’t have to throw stuff over the wall. You know, bring me into the meetings. Make me part of the infrastructure changes. Talk about what we’re doing. And it’s just, it’s a really hard thing to do. And I like the DevOps title because, you know, it’s kind of letting them know, hey, dev, here for you.

Pete: [00:11:46] Yeah, that’s actually a great point. And that’s the place where I love, you know, hearing the term DevOps, where it’s like, you know what, no, we’re, we’re a different thing. And I think you’re seeing a lot more conversations in the enterprise about that, where people are saying, we’re going to have to have, like, we have these huge teams that are, like, ingrained in our organization, like, we’re going to bring in this new team. Interestingly enough, those teams really look a lot like release engineering, but You know what? That’s okay. We’ll call them DevOps. They’ll just do release engineering.

Matty: I think one of the things that, and you guys have probably seen me tweet about this because I feel like I’m super brilliant for finally figuring this out, is the problem of, of when people hear the word of not thinking it’s a portmanteau. I don’t even know if I’m saying that right. What do you call them?

Trevor: Portmanteau.

Pete: Yeah.

Matty: Not thinking it’s one of those, but thinking it’s an abbreviation for development operations as opposed to being dev plus ops. And I think, so I’ve sort of been starting to tell it that way. I’ve been saying those words. I’ve been saying DevOps is Dev plus Ops to kind of drive that home, because that’s where you start to run into that idea that it’s just, it’s this release engineering thing. And yeah, Pete portmanteaued it. It also comes from the idea that yes, it was a suitcase that’s 2 pieces together.

Pete: [00:12:56] I was just like, I have to Google this because I have no idea what it is.

Matty: It’s like, uh, Bennifer. Bennifer is a portmanteau, you know, or, yeah. Okay. I can’t speak French.

Randi: Yeah, we fancy up in here.

Matty: So, so I have a question. So we talk about like, there’s this idea of code smell and code smell is for those of us who aren’t developers or haven’t heard the term before, is really, to me, my understanding is sort of like things that kind of, that you can sense, make you kind of look at and you’re like, this is making me think there’s a deeper underlying problem. So, I’m curious about what would be DevOps smell? What would be something if you were in an organization or seeing one that was doing DevOps and to the outside world that kind of looks like they’re doing DevOps, like we talked about, what might be some things that make you think that something’s rotten in the state of Denmark there?

Nathen: I think one thing that I would call to mind, or that comes to mind immediately, is when you hear the developers start saying, oh, well, that’s DevOps’ problem. That’s DevOps’ fault. Like finding blame in the DevOps team or shoving off that work saying, that’s the DevOps work, that’s not my job.

Matty: [00:14:09] I think another thing to me would be having a DevOps project that’s about implementing tools. To sit there and say, okay, we’re going to do this thing and we’re going to get a bunch of DevOps tools in place. I think that’s usually something that sounds really good to the organization because they’re like, oh, we’re doing DevOps. But then it becomes about tooling. And so it’s kind of, you know, Nathen has said before, correct me if I get it wrong, is that the only DevOps tool is someone with the title Director of DevOps.

Pete: Hey now.

Matty: But it’s something that I’ve seen a lot of, is like people keep talking about DevOps tools, and the problem that I have with it is that It’s not completely inaccurate. They’re tools that when you’re doing DevOps, they’re good to have, but it kind of brings with it this idea that the tool gives you DevOps.

Pete: So I think that’s a problem. That’s actually a really, I think that’s a really good point is, so I did this talk at DevOps Days Pittsburgh that was basically like how to, how to fail at DevOps with one weird trick. And I’m giving a longer version of that talk at Agile 2014, which is including actually what exactly you’re saying there. It’s like the marketing aspect of DevOps, which is, you know, A lot of companies out there are trying to sell you DevOps, and there’s the classic line of, like, you can sell DevOps, but it’s not something you can buy type of thing. It’s this world where companies want to sell it to you, and you have this tool and whatnot, but if you implement a tool on top of the broken culture, you’re going to have 2 problems, essentially: a broken culture and a broken tool. I think that’s the thing that people kind of overlook is, oh, we’ve implemented continuous deployment. But we haven’t actually instituted trust in our organization or anything like that.

Nathen: [00:15:58] Yeah, I also think that, like, to say, we’re going to implement this tool so that we can get DevOps, you’ve clearly missed the point, right? I think that if you implement any tool and, like, implementing the tool is the end goal, then you’re doing it wrong. You have to really stop, step back, and think about, like, what is it that we’re trying to achieve? And if continuous delivery is the thing that you’re trying to achieve, like, there are cultural changes and there are tooling changes, and you cannot do one without the other, and you have to sort of lay out what those objectives are in the first place so that you can make the appropriate changes to get you from where you are today to where you’re headed.

Pete: You know, another thing that I’ve seen, I don’t know, probably the last couple years or so, is, you know, this concept of it’s a developer or operations problem, right? And so we talk about DevOps and it’s like, oh well, you know, the dev team’s not talking to the ops team. It’s like, well, that’s not really the problem here. Whenever people talk about DevOps but they ignore the fact that there’s no— like, where’s the product team in this discussion? Where is QA? Where is the security team? Where are all these other teams? Are they working together? You know, I’ve seen DevOps initiatives succeed because the executives at the company say that we want trust in the organization and we’re going to give trust to these people. And I’ve seen it fail because executives say, Well, we want the DevOps, but we don’t actually want you, like, to actually trust you to make change. So, you know, there’s an aspect of, you know, getting not just— and also like sales too. I mean, sales and marketing, why are they selling features that don’t exist? You know, why are they not getting embedded into these different teams? So I think from a code smell perspective, one thing I would say is when people talk about DevOps and they’re only talking about developers and operations, and they ignore the whole rest of the company, I think it’s a whole company change. And that’s why a lot in more recent DevOps days you find a lot of people who are not developers and operators kind of showing up because it’s really expanding to a much larger group.

Matty: [00:17:56] That’s a really good point, Pete, and I’ve kind of thought that when it comes to Agile. I know that being in companies where they’re going through an Agile transformation, that that was always one of the big things is that it’s organizational, right? It’s the whole thing. And just, you made the point with sales, right? If you’re transforming your organization to Agile, your sales team has to learn that you might have things that disappear because they sucked, right? Or they didn’t deliver. So you can’t sell everything as soon as you see it, or you might have these changes that have to happen. You have to figure out how do I do Agile with sales? How do I do Agile with legal? Like all the parts, right? And so, like you said, it’s not just DevOps, it’s DevOps, QA, prod, product, security, biz dev, whatever, you know, but that’s just too long to say. So.

Trevor: It seems to me like the usage of the term DevOps is inversely proportional to a company’s understanding of DevOps.

Matty: The more you say DevOps, the less DevOps you’re doing.

Pete: Yes, we’re gonna call that Trevor’s Law right there. I’m gonna go to Wikipedia, it’s gonna be a thing.

Trevor: [00:19:00] I’ll take it. So is DevOps really just about culture then?

Randi: I see it that way. I mean, keep in mind I am fairly new to this whole concept of DevOps. I have been working this job for about 3 months, and before that not in engineering for a while. But it’s kind of— I see it as being exactly what I was doing before, but with a different culture behind it. Before, it was hard to get company buy-in. It was hard to get developers to see that, you know, hey, I’m submitting this patch to you because I found a problem with your code. No, it’s not my job, but this is a thing that I understand. And oh, by the way, I’m not just going to add more memory to the server. Let’s actually talk about, you know, the serious memory leak that you have and going in with all of their infrastructure problems and all of their issues and getting them to listen. And yeah, it’s still a problem in some organizations, I’m sure. Personally, I haven’t had it here, so other people would have to chime in on that part.

Trevor: So I think—

Pete: sorry, I was just gonna say, I think, I think a large part of it is definitely culture. I think you have to start there in a lot of cases. There’s definitely some tools to it as well, but I think at this stage of the game, if we were talking about 2009, 2010, then I think more often you’d hear people say, well, the tools have to change. The problem is, it’s 2014, and if you’re not using config management, then you’re probably doing something wrong. Like, if you’re home-growing your config management when you should just be using what’s already out there, you know, that’s probably an issue. Now granted, if you have something that you built 10 years ago and it works great for you, I think that’s fine. But it’s not like you have to change the tool you’re using if the tool can work for you. But You know, a lot of the companies who are kind of doing DevOps, you know, air quotes, you know, a lot of these companies, they’ve all been formed within the last 5 years or so, 5, 6 years, with a lot of the maturity of these new tools. So more of the conversations I think have been both tool and culture focused. I think the interesting conversation that’s happening is definitely with larger companies. So companies that get to 5,000, 10,000, 20,000 people. How does a 5,000-person company do DevOps essentially? Like, how do they bring that culture to their org? You know, I think that’s the more interesting discussion that, you know, it’ll be fun to listen to over the next few years in the community.

Nathen: [00:21:10] So yeah, and I’d like— I hate to do this, but I’d like to call bullshit. I don’t think it’s just culture, and I also don’t think that it’s 2014, so of course everyone does config management. I think that the two are so intertwined, the culture and the tools, that it’s nearly impossible to pull them out. And it certainly does depend on the size and age of your company. Or your team. If you, if you’re a brand new startup starting today, like, yes, probably DevOps is the thing that you’re going to do from today forward. If you’re a company that’s been around for 10 years or longer or is of a certain size, like, let’s talk about what does it mean to have a DevOps culture. One of the things that it means that we’ve already talked about is that you have trust in your team and you trust your teammates to do things properly. Well, if you think about that, just that one thing, trusting the individuals on your team, and think about that in terms of a tool, and the tool that I’ll mention is version control systems. Right? So first, there are still too many people out there in the world— it’s hard to believe, but it’s true— that a version control system means copying a file to .bak, right? Or to .timestamp, right? That is not a version control system. So, like, those people or teams that do that, they have to implement some tool that shows trust. There are teams that are still on centralized version control systems, not to pick on any particular one, but its initials are Subversion. Where, you know, it’s not about trusting your team, it’s about having that central authority. Whereas if you look at a distributed version control system, again, not to mention any names, but its initials are Git, like, you’re trusting everyone on your team to have a copy of the repository, to have the full history. And I think that just looking at something as simple as a version version control system is a great way to look at how DevOps culture and tools are intertwined, how you cannot actually separate them. You can’t say that I trust all of my individuals yet have a centralized version control system. I think that that doesn’t work. And to say that people, like, of course everyone’s using config management, like, of course that’s bullshit, Pete. But at a certain size, whether that size is the age of your company or the size of your team, yeah, absolutely, everyone is using configuration management. But come on, like, that’s not a universally true statement.

Randi: [00:23:29] Okay, there’s also a difference between using configuration management and using configuration management. Like, for example, say having Chef or Puppet or any of them installed and not actually having much in there, but yeah, it’s going out to your servers. Ask me why I’m talking about that right now. So, something that people— that you guys keep talking about is DevOps as tools. And maybe it’s because I’m following the right people on Twitter, but I’ve never heard this before. I always hear of DevOps as culture. Like everybody I’m following talks about DevOps as being a culture. And maybe it’s just because like they’re enlightened, so they know about the tools, but is this actually a thing?

Matty: I think part of it, and I’ll, I’ll say like where I see it and Randi, I’d say this is the difference, right? You’re doing real work. Everybody else on this, we’re fricking consultants or vendors. So unemployed. I mean, let’s be honest, right?

Randi: So we’re hiring, by the way.

Matty: Yeah. So I think that’s sort of the thing is, so like Nathen is going out there and he’s talking to companies, you know, who are, I don’t know, bringing him in to talk about crap, right? And before I was at Chef, I was at 10th Magnitude and I was a consultant and I was, I’ll be honest, I was a DevOps consultant. That’s what I did. But it was one of the challenges with that is, so the companies want that, right? They want, the customers want you to come in and bring them the DevOps and you can do that if it’s about tools. You can’t do that if it’s about what the company is. So back to your point about saying like, are people wanting this? I could totally see. And it’s totally true that when I, before I was consulting in the space, when I was a practitioner, right. But I didn’t see this that way. I was like, oh, it just seems all logical. And yeah, everybody’s saying the same thing. But then every customer I talked to, not every, that’s not fair, but many customers I talked to in both parts of my job, you know, in this job and the one before. It’s a total tools conversation. So I think it’s a lot. It’s unfortunate. And I remember, just to sort of paraphrase, Jez Humble was speaking at some event here in Chicago about a month ago or so, and he said something to the effect of that the bar in our industry is so low right now. He’s like, on one hand, it’s great because we’ll always have jobs as consultants in this space, but it’s also, as much as you can preach it, it’s not being heard. And that’s the challenging thing because this cultural shift, it has to happen from the top. It can’t come grassroots. You cannot make a cultural change happen in a grassroots way in an organization. And so, as long as the people who are making those decisions are still seeing it as a tools problem, it’s never going to get any better.

Trevor: [00:26:04] Okay, so that answers the question I was going to ask you. So basically, to sum up what you just said, doing just culture or just tools is another way of screwing up DevOps.

Pete: Boom, right there.

Matty: Yeah, I concur.

Pete: Yeah, I think this would be a— this would be maybe a DevOps smell, but just doing one or the other— you definitely have to do both because you have to do both with specific purpose. You can’t just say like, we’re going to use Chef and we’re going to change the culture. It’s like, well, you know, why are you using Chef? Is that solving a specific problem you have? You know, if you already have CFEngine, then it may not solve a problem unless there’s a specific reason why you want to make that change. When I was working at Dyn, and I started, they were using a few different configuration management systems. I ended up putting them on Chef because, at the time, I had built teams around Chef. So, if my job was building a team around config management, I don’t know how to build a team on CFEngine. I’ll be honest. I could probably find some people like that that could do it, but I’ve been in the Chef community long enough that I could find those people. So, why pick Chef? I kind of laid it all out in a document. You know, there was a lot of work that had been done in Chef. It just wasn’t out in production because CFEngine was in production. So it was this classic time of operations had pushed one solution, devs had pushed another solution, but because they weren’t shared, they weren’t talking the same language, there was no way that both were ever going to kind of make it out there. So, you know, and then that’s where I would say that’s where the culture would come in. You know, how do you get these 2 teams or multiple teams to agree that you want to standardize on one thing? And can you get them to agree to standardize on what a lot of people would say would be the right thing. And the right thing is not Chef or Puppet or whatever. The right thing is the thing that delivers value to your customers. I think that’s the biggest value.

Nathen: [00:27:50] Yeah, change for change’s sake is stupid, right? We have to sit back and say, like, what are we trying to change? How do we make our customers happy? How do we improve our business? And if those are the questions that you’re asking, and they probably should be, right, it’s probably safe to assume that there’s something wrong with the current way you do things, or else your customers would be super happy, or you would be just crushing it in your business, right? So you’re likely undertaking change to some end. And so to understand what that end is, and that end needs to be understood by everyone across the entire value chain, right? The sysadmin has to understand why we’re undertaking this change also. And to assume that the tools that you have in place and the culture you have in place are enough I think can be a false assumption. And, you know, your old tools, your old processes, they aren’t delivering what you’re after. So you’re going to have to change them and they’re going to change together.

Matty: So we have a question from the internet. So Fred Moyer wants to know, how do I convince coworkers it is DevOps and not DevOps? Is a pitchfork a usable DevOps tool? So So, what might be some advice to someone who’s trying to affect that change in how people that they work with are thinking about DevOps?

Pete: [00:29:08] I guess I can put a couple of thoughts in this one in that I think a lot of times, to start with, it probably is just Dev and Ops. I mean, as much as I’m going to cringe because I think it involves many more people, if anywhere, you have to start somewhere. And so, if somewhere is taking predominantly 2 teams that never used to communicate well and getting them to work better together, then that’s a step in the right direction. I think the hardest thing is, and I would love to have anyone’s comments on this one, I think that personally that you can’t change culture. I think that going into a company with your task to change the culture is kind of a fool’s errand. The culture is so ingrained in many companies that the only way to actually change culture is to basically change everyone in the company or hire enough new people that change the culture that way. It’s like this majority rules type of thing. So I wish I could say like, oh, here’s what you exactly do, it’s super easy to solve it, you know, to change the culture in your company. I think it’s a hard problem, and the only way I could say is to get there in the right direction is if you can spend your time building something that shows value. So like at Dyn, one thing I did early on was they had CFEngine, they had a couple other tools, and I showed them how they could take Chef and have this really nice workflow integrated into a continuous integration pipeline. But that was something they didn’t have with CFEngine. So I built out the initial, like, spec for it, but people who were much smarter than me built out the kind of final solution. So I always kind of stick by my thought process of if you can kind of build something cool and show it to people with a lot of enthusiasm, you’ll probably get them to be convinced to come over to your side versus standing on the sidelines saying, hey, you over there, Do the DevOps. It’s a good thing. You know, it’s kind of, you know, put up or shut up is one thought I would always have.

Matty: [00:30:57] Yeah, I think it’s super valuable to show, but I also think if you, especially if you want to work, like his question was around like, how do you show the DevOps, right? The Dev+Ops versus the Dev SpaceOps. So it’s show, but invite, I think. So those other parts and say, hey, here’s a thing that we’re starting to use kind of in our little part of the world. But I’m inviting you to be a part of it, not to come to some brown bag so I can show off this thing that I did, but more to say, like, hey, we’re starting to use this, so how can we make it even more valuable by having you participate? You know, what’s your thoughts? Like, how can we make it cooler? And also, it’s cooler by you being part of it.

Randi: So, so I kind of just had a light bulb go off while you guys were talking about this. I mean, basically what we’re talking about here is communication and being good at communicating between teams and speaking all these different languages. Whether it be trying to communicate a new culture to management, QA, your developers, your operations. And I realized something. Most of the really good DevOps people I know, whether you like the title or not, are public speakers. Or they do podcasts. Or they’re kind of outgoing. And they go out there and they talk to people about what they’re doing. And I think that’s a really good skill to have when you’re trying to push this culture change. I think you can push a culture change. I think you can be the driving force behind it. But you have to have that charisma. Charisma, well, for some people. Charisma or just, you know, the iron fist of being able to talk to people and they’re going to listen.

Matty: [00:32:27] You have to buy your own bullshit. You know, I mean, you have to really believe in it, I think.

Nathen: Well, yeah, and I think to your point, Randi, like a couple of ways that I’ve seen DevOps get implemented successfully at clients of Chef and other organizations, right? Sometimes development and operations sit in different parts of the building. So one of the best ways to build trust is to make them sit together or to allow them to sit together, have them eat lunch together, go out for beers after work together. Like, if I get to know you as a person, I’m much more likely to trust you than as that person that’s on the other floor.

Matty: Another—

Trevor: tell me my stuff broke their stuff, right?

Matty: Right.

Nathen: Another thing that I find works really well and that we’ve seen implemented at a number of our customers is that you Pull together a cross-functional team and lock them in a room and give them a project and say, don’t come out until that project is done. You, the cross-functional team, we might even call you a DevOps team, as much as that, like, pains me. In a large organization, right, sometimes the right way to break down silos is to build another silo and put a group of people together in a room and say, all right, you are now the DevOps team. Here’s your project that you should do. Work together until you solve this problem. And then, after you’ve done that, let’s go back or have you go back into your individual silos or back into your regular group, and let’s start taking some of the lessons that you learned back to your home silo, if you will, and start sort of breeding that cultural change across the organization. So I think that that’s another successful way to do it. So put the people together in the same room. Really reflects the idea of trusting one another and getting to know one another. And then to your point, Randi, like, you don’t want the bastard operator from hell who refuses to talk to anyone to be like the first person that you put on the DevOps team. You need to pick someone from your team that’s interested and able to talk to— not even like public speaking necessarily, but I can go and talk to the developers and have a meaningful conversation with them. I can go and talk to sales or to marketing or to management and have a good conversation with them. So, someone that can, you know, express some empathy, carry on a conversation. I think that these are some good traits of people that make good DevOps engineers.

Trevor: [00:34:47] I know a lot of developers who are completely introverted, whose biggest fear is talking to anybody but another developer. How do we help people get past that?

Pete: Force yourself. Yeah, I mean, I think there’s one part is this is where those pesky managers come into play, and this is, you know, being a former pesky, you know, what is it, spiky-headed manager.

Matty: Pointy-haired boss.

Pete: Yeah, sorry, pointy-haired boss. God, I’m so terrible. I don’t even know.

Nathen: You rabble-rouse.

Pete: Yeah, I think you go in and you really try to instigate stuff. So one thing I’ve done in the past, I think, you know, both Nathen and Randi, you both kind of mentioned this one, is that You know, there is a concept of either a team or a group of people that have those types of things you’re trying to do as a focus. I think, Randi, you said you were referred to as an embedded engineer. You know, that’s something that actually I had heard of but also implemented at a few places where, you know, I would take people, I would take engineers, people who were passionate about something. It didn’t really matter what they were passionate about, but that they wanted to learn, and that was the biggest thing. If you wanted to learn, then you were automatically part of that group. And then we would take you and train you up on something new like configuration management. Maybe that was the thing we were focusing on. And then we would put you into these teams who desperately needed some help in those areas. And basically you were like patient zero for something new. You would go in there with your subject matter expertise and talk about things that were interesting. And so me, in my role, I would basically do like evangelizing and training, and I would meet with different groups, and maybe I would would mock out different ways that you could use the tools that we’re playing around with. But then those people who spent every hour of their day on this stuff, you know, they would build something much better and greater, and they would do it in these individual groups, and then they would infect all the people in those teams, and it would just kind of expand and grow from there. So, you know, Nathen, I know when you mentioned kind of like a DevOps team, I think as much as it pains me, you’re right. It’s, it’s not, not pains me to say that you’re right, but as much as it pains me to say that that a DevOps team is needed. I think, if anything, what you call the team is irrelevant. You could call it the Unicorn Rainbow Team for all that matters. It doesn’t matter. But, you have a specific team that is focusing on this new initiative, and they’re getting embedded and involved in all of the other groups. They’re bringing this excitement, which a lot of times companies maybe don’t have anymore. There are people just kind of down in the muck. There’s a lot to be said for people who can just go in and talk about interesting things and show something new and get people excited again.

Matty: [00:37:25] I think as long as you make those teams— and Nathen, I liked your thing where you said sometimes you break down a silo by building a new one, but I think they have to be temporary. They have to be bridges, right? Because otherwise, you end up with this thing where now it’s just, okay, that’s just the DevOps team does that, and it’s a whole other problem. I think when you look almost, those teams can help be almost evangelists around there and, and help guide and, and bring people along. Because sometimes it’s just a matter of, like you said, getting excited about it, but then also it can be really, really intimidating to take this stuff on. You know, there’s a lot, right? When you want to start learning about this, it’s like, where do I even start? And so having some direction certainly helps. Well, now I know Randi is going to have to leave, so I’m going to shortcut. I just want to see, Randi, if you have any kind of final wrap-ups of advice to our listeners before you bail about how to— what would be the one way you could totally fuck up DevOps?

Randi: The one way you could totally fuck up DevOps. Oh God, I’ve seen so many ways recently. So, I’m going to talk a little bit about tools here. Don’t use one tool for everything. And maybe this isn’t necessarily related to DevOps. I don’t care. It’s something that’s really on my mind. Because it’s been pissing me off so fucking much. Don’t use a Jenkins build job to execute scripts on servers. Don’t use it to restart services. This is a bad thing. If one more person asks me for this, I’m going to punch them in the face.

Nathen: [00:38:54] Like, it’s—

Pete: This just in: Jenkins is not orchestration.

Randi: It is not.

Matty: I knew of an organization that will remain nameless that uses Jenkins as a job scheduler for all of their build jobs, but their batch jobs for their product. Because hey, Jenkins does that stuff, so why not?

Randi: Yeah. Just because it can do it doesn’t mean it should do it. Please stop.

Pete: It’s probably fine.

Randi: That’s why you need DevOps to tell you you’re doing it wrong.

Matty: Well, thanks, Randi. Thanks for joining us. It was awesome.

Trevor: Yeah, thank you very much.

Randi: Yeah, thanks for having me. And one last thing, Kicksi is hiring. We need another DevOps on my team. Please, for the love of God, I need another person.

Pete: Send us the link, we’ll put it in the show notes.

Randi: Alright, cool. I’ll talk to you guys later. Thank you so much for having me.

Trevor: Take care.

Matty: Yeah, bye-bye. So I have a thought, and, and I don’t know how far this will go because it’s something that I kind of thought of the other night while having a couple drinks. But so I’m gonna throw it out this way and see, see where we go with it. But can the opinions of a tool help drive the culture? And I guess before I explain it, maybe I’ll just put that question out that way and see how you guys would reply to that.

Pete: [00:40:10] So, okay, I can start on this one. So people have biases, right? And people are biased in a lot of different ways. And I think that’s the reason why companies are not able to affect change in their organization is because of these, you know, different organizational biases. I mean, I’ve worked in places where And I’m not gonna sit here and say I’m like a Ruby expert. I write terrible Ruby. I can hack it together to make it do some things, but I’ve been in places where, oh, you do Ruby? You’re such a hipster. And I’m like, isn’t Ruby like 20 years old at this point? You know, people will hear a language and they’ll immediately kind of respond to it, and, you know, we get down those different places. And I think that’s one of the biggest problems in our industry right now is people who look at tools or look at— kind of look down their nose on other people for tool usage and things like that. So yeah, you know, it’s like I’ll hear people say like, oh, you know, you shouldn’t use Chef, that’s crap, you should use Puppet. It’s like, well, like, why? Like, why? Like, oh, because one’s— I don’t even know, like, there’s— they’re so close, like, almost like I don’t even get it. But, you know, but there, I think at the end of the day, I always try to, you know, push back on those types of conversations with people, like, when we get down those places of talking about those specifics, it’s like, well, why? Like, why is X better than Y? Can you give me some reasons? Like, here’s a reason that we want to choose X, and it has nothing to do with, you know, the type of language or the type of tool.

Matty: [00:41:35] But, you know, so that’s totally not what I meant. However, that was an awesome point. I didn’t even think about it in that way. Although it does always go towards the biases, so No, but then now I look at my question, I’m like, oh my God. So I meant the opinion belonging to the tool, as in a tool’s opinion, as in if you have an opinionated tool, like if you think about Rails being an opinionated framework, right, that kind of drives you in a certain direction. So I started thinking about this, about how, you know, tools are easy and changing people is hard. We can kind of parrot that all day long, and we all agree with that, I would imagine, at this point, right? So, and we’ve sort of spent the last hour kind of talking about how it’s so important to change how we think about things But boy, is that really hard, and you can’t change a whole culture of a company except by firing everybody, you know, or whatever. And so I’m just sort of ruminating on this, and this might be something just to kind of marinate on and see where it goes over time, is that, is it possible as these tools, these DevOps tools become more embedded and more popular, they bring with them a fair amount of power, right? And so the creators of these tools have the capability and maybe responsibility of expressing their opinion through the tool. The example I gave when I was explaining this to somebody recently was kind of thinking about how you train a bonsai tree, right? So maybe the tool, you first bring the tool in because it’s going to do this thing, but by using the tool enough, you kind of wake up one day and you’re doing DevOps because that’s the only way that you can use the tool effectively. I don’t know.

Trevor: [00:43:10] I would agree with you, Matt. Because, yes, I’ll use Chef as an example. Chef, you know, we’ve talked about, you and I even, just how you were so somewhat siloed in the ops space. I was somewhat siloed in the dev space, but as we talk about configuring a server using code, how we kind of have to answer each other’s questions and come to a shared understanding of how that tool works by using that tool. So I gain ops knowledge, you gain code knowledge, all because we’re using Chef. So I think, yes, it can help drive a culture of communication because you have to talk to each other.

Matty: Kind of going back to Nathen’s point— go ahead.

Nathen: Yeah, I mean, I was going to say, I also think that that’s kind of a reflection of what I was saying earlier about how SVN versus Git, or, you know, how those 2 different tools actually drive different patterns of behavior within your organization. And then just to sort of stick with the configuration management, right, if we were going to talk about Chef or Puppet, I will tell you that one of the things that I learned very early on when I was brand new to configuration management, like the way that I worked with configuration management, or more specifically infrastructure as code, was I would write up the code that was required to provision a machine I would provision a machine, and that would work, and that would be great. Then, sometime in the future from that point, a change in my policy would be required. What I would do is I would go back to my codebase, and I would make that change in my codebase, and then I would test, provision a new machine. Does the new policy change actually get implemented? It does.

Randi: [00:44:50] Great.

Nathen: Now, what I need to do is roll out that policy change across my production infrastructure. But I’m not actually running my infrastructure as code client on a regular basis. I’m only running it on an ad hoc basis. So I wonder what has actually changed since the last time I ran that client to enforce my policy and now where I have this policy change. And if I run this client, what is going to be the impact across my infrastructure? And so the way that I address that concern and that uncomfortable sort of queasy feeling that if I run this, I’m gonna bring down production, is I would manually make that change in my production environment on all of my machines in my production environment. And guess what? That was fucking stupid. And it was my fault. It was— it had nothing to do with the tool. It had to do with me trusting the tool and me trusting the process. And I think that as I, as I, and more specifically my team, became more familiar and more comfortable with the tooling, like, that drove us to a completely different place where we started to use the tool appropriately, and we started to just have the client enforcing policy on a continual basis, which is what it’s meant to do. And I— so I think that that absolutely caused some cultural and process changes on our team. We matured as a team that was managing our infrastructure as code. We matured the way that we were working with tools and so forth. I don’t know if that little story helps illustrate your point, but there it is nonetheless.

Matty: [00:46:27] No, absolutely. I’ve seen that same thing with other people in that tool space where it’s a matter of learning to trust the tool. Then by trusting the tool, I guess that helps you trust people.

Pete: Yeah, actually, you brought up a good point, which is the usage of the tool and trusting the tool. I’ve seen places where implementing new tools, like, there are some developers who just don’t trust the tool. They’ll say, hey, Chef changed this file, and it broke everything. It’s like, well, it changed the file because you wrote code to say to change the file. So, I’m not sure what you mean by it broke everything. In a lot of those cases, there’s always that mistrust of, you installed something that broke it, not this code or this other thing. You know, so a lot of times it’s that, you know, education, it’s, you know, talking with people, it’s the communication aspect of, well, let’s get to the fundamental reason of like what the problem is here. It’s like, it’s, you know, you did this, here’s what the logs say, you know, I run it in debug because I’m thinking this is gonna happen, you know, this is what it did, you know, don’t distrust the tool because it’s new. Let’s talk about what you’re actually trying to do with the tool. So, and a lot of that is just pure training, education side of things.

Nathen: [00:47:39] Yeah, and I think you see that across a number of different frameworks. Like, look at every Java developer who moved to Rails, and the Rails that they write looks like Java. Look at every sysadmin who moved to Chef or Puppet, and the code that they write in Chef or Puppet is completely procedural code. It’s not declarative, it’s not utilizing the tools and the framework the way that they were meant to be. And over time, you learn to do those things properly the way that the tool wants, and that does cause a change within you and within your team.

Matty: Awesome. So we have one last question before we get into our checkouts, and this goes back actually a little bit to something Randi said earlier where she said, I don’t see this when you people talk— you guys are talking about, you know, people that wanted the tool to drive everything. And, you know, she’s like, maybe I’m just following the right people on Twitter. So my question is, how do we avoid the echo chamber, right? So I mean, I can sit and I can follow Pete and Nathen and, and, and on Twitter, and we’re all going to basically have this mutual admiration society all day long, right? And be like, wow, Pete, that was super smart, you know, favorite, you know, and on and on. And none of us are getting new ideas necessarily. That’s unfair. I get new ideas from you guys all the time, but we already know we agree with each other. So number one, that’s a question I have. And then it also, the second part of this goes to things like conferences, and this is a thinking specifically about something like DevOps Days. So, you know, plug, shameless plug. So I’m helping plan the first DevOps Days Chicago. Which will be October 7th and 8th. Go to devopsdays.org to see about it. And CFPs will be next week. But one of the things is, when we’re trying to think about how do we get the right people to come to DevOps Days, and it’s not like the right people like the cool people, because in fact, frankly, I don’t really care if the cool people come. I want the people that didn’t drink the Kool-Aid yet. I want the people who haven’t gotten religion in this town to attend. So how do we— so that’s 2 things, and I think it goes back to the echo chamber. How do we not have, as much fun as it is, to have a conference or Twitter or whatever and we all agree with each other, I don’t think it helps our industry and I don’t think it helps our community. So how can we make that better?

Pete: [00:49:39] Yeah, I mean, I love DevOps Days. It’s a great unconference. You know, you get a little bit of the talks and a little bit of the open spaces, which I think is great, but it is a massive echo chamber. I mean, the people that are there have heard of it, DevOps, and are probably doing some parts of that kind of cultural change. You know, there’s a lot of conferences out there. I think I’m actually most excited for my Agile 2014, you know, discussion and talk and stuff because that’s going to be a much different group in which I’ve been in before. And I think it’s going to be, you know, it’s going to be definitely a big shift for me to talk about that. Yeah, I mean, there’s a lot of local meetups that I’ve been to as well that haven’t really kind of hit the kind of DevOps world as well, so I think it’s more on all of us to try to get away from just the echo chamber. Matt, you said it perfectly. It’s really easy for all of us just to sit around and say, oh, yeah, DevOps, like, do the automation and stuff. But, to get out of that world and to experience something else, that’s where I keep going back to the enterprise because I think that’s really where the next level— pretty much startups won’t call it DevOps because it’s just common sense. But enterprises, they are the places that really need that big shift. I think there’ll be a whole world of enterprise-level stuff that really needs that focus, that DevOps change in their environment and their organization.

Nathen: [00:51:08] Yeah, so for me, I get what you’re saying about DevOps being an echo chamber— sorry, DevOps Days specifically being an echo chamber. I think that there are a couple of things that help fight that. First is you’re seeing DevOps Days pop up in new cities where they haven’t been before. So for example, the Pittsburgh DevOps Days, I think there were a lot of people that were interested to know what is this DevOps thing. So I think that that’s one way that you can sort of help escape the echo chamber is to make sure that you’re going to new cities.

Matty: Don’t—

Nathen: like DevOps Days Silicon Valley, it’s gonna, you know, we just had the 4th year of that. So yeah, sure, there’s some echo chamber there. But I think honestly, from my perspective, for every DevOps event that I go to, whether it’s the 4th year of Silicon Valley DevOps Days or it’s a local DevOps meetup, the thing that I find that makes me really sad is that there are always a strong bent towards the operations side, towards the sysadmin side. So my, like, the one little trick that I think you can do to help escape the echo chamber is bring someone from the development team along with you. To that DevOps Days event, whether that’s a meetup or a DevOps Days conference or any other conference, like bring a developer to ChefConf, whatever that happens to be. So you personally are working with someone that’s outside of operations. Bring that person with you to that event. I think the other great thing to do that happens a lot, right, one of the things that DevOps kind of looks towards is other industries and research that’s done in other industries. So we bring a lot in from like Lean Manufacturing. Go and read those books and listen to those podcasts that have nothing to do with technology, but have to do with implementing change in your organization, have to do with more efficient processes to run your business. You know, expose yourself to those things. Go to those types of conferences where you’re out of your comfort zone.

Matty: [00:53:08] Awesome. So we’re really towards the end here, so let’s do our checkouts real quick. So Pete, what do you got for us to check out?

Pete: I feel like I’m gonna get ganged up on by all the Chef people, but in my fun employment time, I felt like since I’ve spent so many years playing around with Chef, I should play around with something else. And so I’ve actually been playing around with Ansible, and I gotta say, it’s pretty awesome. You know, if I was getting started for something quick and simple, I probably would throw some Ansible together. So big props to the Ansible team for building a really awesome tool with great documentation. So that’s my checkout. I think you should definitely play around with some Ansible. It’s, you know, it’s a lot of fun if you got some, you know, simple tools. I think the only issue I have with it is that I’d be curious how well you can really be expressive in, you know, YAML or whatever. So, but until then, you know, it’s fun for now. And I’ve made Ansible’s name, and I’ve pretty much alienated every person at Shep Inc. Sorry, Nathen. I love you too.

Matty: [00:54:13] We love everybody. Nathen, what is your checkout? What is it, CFEngine?

Nathen: CFEngine is pretty awesome. They do have the science. Mark Burgess is a brilliant, brilliant man. You should go read all of his books, or both of his books, as it were. Those aren’t my checkouts for today. My first checkout, actually, it was inspired by you, Matt. I think a Facebook post that you made over the weekend. Remap your Caps Lock key as your Control key. Make your pinky happy. You will be surprised at how— like, you talk about the tooling, can it change people? Can it change your culture? Like, the happiness that you will have 2 weeks after you change your Caps Lock key into your Control key cannot be expressed or cannot be overstated. ’Cause wow. So my other checkout, I do have another checkout, is the new Supermarket. So today at Chef, we relaunched our community site, and the community site at community.opscode.com is now also known as supermarket.getchef.com. And this is the new way for folks to share cookbooks with other folks in the Chef community. So I highly encourage you to check that out, re-sign your CLA or reaffirm your contributor license agreement, as it were, on the supermarket, and you’ll be super happy that you did. And also tune into the Food Fight Show later this week because we’re going to talk about supermarket. Those are my checkouts.

Matty: [00:55:44] Awesome.

Trevor: So Randi had said she didn’t have any checkouts prepared, but she said a don’t checkout, and she said don’t check out Jenkins. So as for me, I read a— keeping in the spirit of mentioning something Game of Thrones in like almost every episode by somebody— somebody on the internets created a program for Windows that when the machine starts up, every character input to the computer outputs Hodor. Any input to the computer becomes Hodor as text, which there’s a whole write-up about actually making that happen in Windows, which is kind of interesting. We’ll post a link to that. And then secondly, I finally get to live my life on the bridge of the Enterprise. I can now talk to my phone whenever I want. Anybody who’s watching will see that my phone screen is off, but I can say, okay Google, and it’ll wake up, and I can ask it a question. So I can say, okay Google, what time is it?

Randi: [00:56:45] The time is 8:03 PM.

Trevor: And I get to live on the Enterprise, and it’s awesome.

Matty: You don’t need friends now, Trevor. You have your phone. So my checkout, I’m gonna make it real quick. It’s something I just kind of got excited about, and it is the Sunrise calendar app for iOS and OS X. I actually had started using Sunrise quite a while ago when they first started, and they were basically just like a daily email reminder of your Google Calendar. And I thought that was kind of all they were, and I still used it, like I still get it all day. And then I read— I was on, I don’t know, Masha Crunch or whatever the crap the other day. I’m like, wow, they have this iOS app and a Mac app and they’re totally free. And they are really nice calendar apps that have a lot of integrations to all the usual stuff, you know, your, your Google calendars, your Exchange calendars, your Facebooks, but it’s just a really nice experience. So I will put a link in the show notes, but you can get it at calendar.sunrise.am. So, reminder as usual, we have a newsletter. It comes out when I remember. You can sign up for it at arresteddevops.com/bananastand. It is the best way to know about upcoming podcast episodes and cool news with DevOps. We do not spam you and we won’t sell your email address to spammers. We promise.

Trevor: [00:58:05] And thanks to our sponsors PagerDuty and Datadog, and also a special thanks to Mandy Moore at @therubyrep on Twitter. You’ve been doing an amazing job cleaning up the audio for the episodes. You put Matt to shame.

Matty: Yep. And thanks to Pete Cheslock, Nathen Harvey, and Randi Harper for joining us tonight. We really appreciate, uh, all you folks joining and be sure to check us out on the Webertubes at arresteddevops.com and follow us at Arrested DevOps on Twitter. Go to facebook.com/arresteddevops. I don’t know, whatever other ways you can find us on the internet. We do have a Google+ page too, but I don’t know what it’s for. So, as usual, I am Matt, @MattStratton on Twitter.

Trevor: And I’m Trevor, @TrevorGHess on Twitter.

Matty: We are Arrested DevOps, and remember, there’s always DevOps in the banana stand.