Veronica: [00:00:00] You lose data, you grieve a little, and then you put your head down and create it again.
Matty: For Arrested DevOps, the podcast that helps you achieve understanding, develop good practices, and optimize your team or organization for maximum DevOps awesomeness. We are coming to you live from DevOps Days Chicago. I’m your co-host, Matt Stratton, and with me is Trevor Hess. Before we get started talking to our great panel, we have a word from our sponsors. DevOps shows that delivery automation is important. Our work is changing software, and software is useful after it’s delivered. So, how do we develop our delivery? Is it scattered across dozens of repos, or could we use code? Is it a loose collection of YAML and Bash, Or can we unit test our delivery too? Do we even need all those pipelines? There is a better way. When you’re tired of patching up pipelines, when you’re serious about safe delivery of code, check out Atomist at atomist.com. Arrested DevOps is brought to you by PagerDuty. In an always-on world, teams trust PagerDuty to help them deliver a perfect digital experience to their customers, every time. With PagerDuty, teams spend less time reacting to incidents and more time building for the future. From digital disruptors to Fortune 500 companies, over 12,000 businesses rely on PagerDuty to identify issues and opportunities in real time and bring together the right people to fix problems faster and prevent them from happening again. PagerDuty, solutions before problems.
Trevor: [00:01:43] The worst thing about the Arrested DevOps podcast is when it ends. You’re left wondering what to do next. What are you going to listen to on your commute home? How do you occupy your time when walking the dog? What are you going to listen to during the quarterly all-hands meeting? But fear not, dear listener, there is a solution. You need to subscribe to Software Defined Talk right now. It’s a weekly podcast that recaps all the news in cloud computing, DevOps, and enterprise software. The hosts, Cote, Matt Ray, and Brandon Wichard, will keep you up to date on all things cloud while offering tips on how to optimize your Costco haul and how to PowerPoint. It’s a fun, free-flowing conversation that will keep you entertained and informed. What are you waiting for? Subscribe to the podcast today by visiting softwaredefinedtalk.com or by searching for Software Defined Talk in your favorite podcast app.
Matty: So again, we are here at DevOps Days Chicago, live, live, the 6th time we’ve had this event, and we’ve got a great panel. Of different speakers who have spoken at the event and had different experiences at DevOps Days. So, first, starting to my left, just maybe introduce yourself for our audience.
Jessie: [00:02:56] Hi, I’m Jessie Frazelle, currently self-employed.
Veronica: Veronica Hanus, I’m a contractor in Brooklyn, New York.
Jeff: Jeff Smith, I’m Director of Production Operations at Centro, official black guy for DevOps Days.
Matty: It’s true, it’s official.
Veronica: Everyone pauses and they’re like, oh, he said that.
Matty: Fair. Um, what I’d like to do is maybe start, since you all, uh, spoke here at DevOps Days today, maybe for people who are listening to the show before the talks are online, maybe give a kind of a little overview of what you talked about, uh, in your presentation, and then we’ll kind of, uh, get some feedback and talk about it.
Jessie: Uh, so I talked about open source firmware and roots of trust and why those are important.
Veronica: So I talked about— I gave an Ignite talk and I talked about, um, very briefly about developer use of comments and how attitudes for those comments affect growing developers and may affect their documentation styles later.
Jeff: [00:03:57] And I talked about ethics and technology and computing and why it’s difficult and hard.
Trevor: That’s fantastic. If I’m not mistaken, when we were on the stage at the Holiday Inn last year for DevOps Days Chicago, you had had an open space on ethics in computing.
Jeff: I can’t remember if I actually did that or not, like if I kicked it off, but I was definitely present. And it was definitely something that informed this talk because a bunch of us had conversations afterwards and said like, this is a conversation that needs to continue. And we had similar results today in that open space, a conversation that needs to continue to happen and grow.
Matty: So one thing I’d like to kind of have a little bit of a chat about, ’cause we have different experiences with DevOps Days and with DevOps Days Chicago in general. So, kind of looking down this, so Jeff, this is the 5th time you’ve been speaking at and been part of DevOps Days Chicago. Jesse, I know this is your first time at DevOps Days Chicago, but you’ve been to several other DevOps Days. Veronica, this is your first DevOps Days ever. So, maybe kind of starting with Veronica, what are some of your impressions you’ve had, some of your thoughts about the DevOps Days experience so far?
Veronica: [00:05:02] Sure thing. So, coming, I’m coming from New York City and this is my first DevOps Days, so I’m very much like, what to expect? How does it work? And how does public transit work when you’re not in New York City anyway? And all those little things that you worry about when you’re traveling. And one thing that’s been really phenomenal is how much support the team has given. I tweeted earlier shortly after I was accepted. They literally asked the speakers would they like us special song when they go onto the stage. And I was like, I love this group more than anything else in the world because not only are they like, we’re going to help you figure out transportation, we’re going to make this part easy and that part easy, and you have all the support, and you’re going to eat the best food you’ve ever eaten at a conference, but in addition to all of that, we’re going to make you feel really amazing when you walk onto the stage. How about that? So that is kind of the first impression I have. Being here so far, nothing has surprised me. Nothing has gone against that first impression. So hurrah, hurrah organizers. I see all the moving parts moving extremely smoothly.
Matty: [00:06:14] Jeff, what have you seen over the years that you’ve been here? How does the event feel different? Does it feel the same? What do you see?
Jeff: In a lot of ways it feels the same. In a lot of ways it’s different. You definitely see the event grow. Right? And you see the process get smoother, audio, video. Funny I’m saying that now, but audio, video has gotten a lot smoother. But the community just continues to grow. It’s absolutely one of my favorite times of the year. And even if I’m not speaking, I would definitely be attending DevOps Days Chicago. It really is sort of, for me, it’s like the bar at which I judge all of the other conferences because it’s always such a tremendous experience. You know, the participants are great, the open spaces are fantastic. And there’s just an energy here that, for me, is unmatched at a lot of conferences.
Matty: And, you know, Jesse, you’ve been to some other DevOpsDays, and I know this is your first time here in Chicago. What’s— like, what did you— from what you participated in so far and giving your talk, kind of what are some of your initial feels?
Jessie: [00:07:15] Yeah, I would say that it’s organized really well, not to suck up. But yeah, also just like the varied audience. Backgrounds, so people who come from different fields, because I was talking to someone who does data science and then one that’s a technical writer. And it’s nice when conferences can draw different groups of people versus just having group thought or whatever.
Trevor: Now that we’re only about halfway through the event at this point, but what have been some of your favorite conversations that we’ve had so far here this couple days?
Veronica: So I think one thing that’s been really exciting to me, a lot of the talks that I’ve heard have been really neat.
Matty: So my background is not not DevOps.
Veronica: I’m coming, giving a talk that is interesting to a lot of folks, but it’s one of those varied conversations. And so hearing talks where people are describing situations that occur within DevOps, but within the context of something a human would experience. We had talks that involved things going viral online and people getting tickets because like signs are difficult and what’s that even anyway? And so one of my favorite conversations has been like when we were in open spaces and talking about what makes an interesting, you know, what’s like one hook you could have if you’re working on your own talk submission is, you know, talking about ways you can bring in, you know, that, you know, the technical with like the the lived experience portions.
Matty: [00:08:52] I think there was a really interesting parallel between Jeff’s talk and Jesse’s talk in different ways though. So Jesse, like she said, talked about why open source firmware was important. And I always like to think of this talk as the one that will scare the shit out of you when you realize how terrible it all is. And I think where like kind of the thought around those ethics also comes in, right, is that it’s like just because it seemed like a good idea to put a web server in there, right? ’Cause what could go wrong is how can the, you know, always think about how can these things be misused, right? And I’m interested to kind of see like where you see those parallels.
Jeff: Yeah, it’s definitely something that happens, right? And when you see technology sort of misapplied, and we sort of do it ourselves in the operation space, in the engineering space where we might choose a technology for the wrong purposes because we wanna play around with it or we think it’s interesting, not thinking about all of the downstream implications of it, right? Like, I think of the Zoom breach recently where they thought it was a great idea, like, oh, we’re gonna install a local web server that’s gonna automatically reinstall the software. And somewhere, I’m sure someone wasn’t thinking of that as being malicious. They were thinking, this is gonna make it so much easier for me to be able to accomplish X. So, it becomes difficult to think about all of the different paths that technology can take us down and the sort of consequences that come from it. So, I think the only thing that we can really do around that is to have more conversations around it, to bring in more viewpoints, more discussions. You know, I remember when Google, you know, had that snafu when they were classifying apes as Black people, and I was like, there wasn’t a Black guy in that room, right? Because that would’ve got caught. So it’s about, you know, sort of opening up the conversation, broadening the perspectives, and, you know, trying to think one step beyond what am I trying to get done right now and think about the long-term implications.
Trevor: [00:10:49] Well, Matt even started with the question of what could go wrong with putting a web server in the sublayers of the system? But why don’t we ask the question, what could go right?
Matty: Yeah, or actually do ask the question, what could go wrong? ’Cause that was sort of the thing, I was saying it rhetorically, but yeah, actually say what could go wrong.
Trevor: Seriously ask what could go wrong, not just facetiously ask what could go wrong.
Jeff: It’s funny, I was looking at some permissions for someone in AWS and I look at their, their policy and the SID for the IAM profile is, what could go wrong? Allow star star.
Matty: Well, at least that’s like being self-aware, right?
Jessie: I think like a lot of it is also, a lot of employees are very process-oriented. So if it’s not a part of your process to like think or even ask that question, what could go wrong? You just do A, B, C, D, and then you finish your task. And sometimes I’m just like, at what point were you like thinking? Or, you know, like wondering what could actually go wrong.
Matty: [00:11:52] What do you think kind of drives that, you know, ticking the boxes, going through the motions kind of thing? Because I think again, we’re not really thinking— most people are wanting to do good work, you know, but there’s something within the organization, there’s something that’s caused this to be the, I’m just sort of taking, you know, going through. The motions.
Veronica: I think sometimes that’s a matter of like psychological safety within our organization, because I’ve certainly been in positions where it would be extremely difficult to push process and to say, why don’t we X? And there have also been situations where people have been very open to that. And the, the, the amount of like— we talk about innovation a lot in tech, and like the amount of innovation you can do in an environment where it’s like okay to say, why is it like this? And to take more time and to do research is, you know, they’re night and day. And also I think like what people feel comfortable asking for and doing, it really like guides how much, you know, puts us in a position where it might feel best to check those boxes and to be like, well, there might be, mm, blah, blah, blah.
Trevor: [00:13:05] Well, I mean, if we tie it back to what Jeff was saying this morning about potentially the idea of some sort of ethical body around technology. I think one of the reasons why we make those split-second decisions to do something potentially unsafe is because we have some pressure saying we need to get something accomplished or something finished. But we don’t often have the moral pressure to challenge just getting something done, right? You may do something that, you know, like setting all stars on an access policy, Right? You may need to do that just to get your task done, but you don’t have the authority maybe or the moral authority to say this is wrong and it’s going to cause us a big problem.
Jeff: And sometimes it’s also like organizational standards and culture because, you know, you think of a world where those things aren’t allowed. I remember 10 years ago I was at a job and the idea of putting a password in a config file wasn’t that crazy. But then you go to a job and someone’s like, what’s wrong with you? Why would you do that? Suddenly you’re like, oh, that’s not acceptable here. And eventually that just keeps moving up and up on the pipeline. It gets more and more sophisticated. But, you know, I’m curious, does— I would just imagine, like, does anyone at Google, you know, submit a new function without unit testing, right? They probably get laughed out the door, I hope.
Jessie: [00:14:25] Probably not. Yeah, it’s definitely culture. Like, if your culture is, like, make money even if it comes from, you know, the expense of exploiting your own customers and no one ever gets ramifications for doing that, then that’s like the culture that you set.
Veronica: Back when I was in a research setting, I was in a research center where we weren’t putting anything on the cloud yet because we don’t trust the cloud. And especially as a large government research center, we definitely don’t. And so things would be stored and hopefully duplicated. But if not, if something crashed, we lost data. And that was There was definitely, you lose data, you grieve a little, and then you put your head down and create it again. And it’s like that was part of almost an expected process. Like, oh, that happened. It happens sometimes. And now when I think about that, it just sounds bizarre to me that we put ourselves in that situation.
Matty: I want to go back to Veronica when you were talking about a lot of this coming from people, whether or not being in a place of psychological safety. And so we like to speak somewhat in practicalities on the show every now and again. So this might be a good time, especially given kind of the balance of this panel. So Jeff, you’re a management-type person. I mean that in a nice way. But you’re a leader. You lead teams. And so first, I want to think from the perspective of if someone saying, I want to build more psychological safety on the team that I lead. What are some things you do or some things that you do— hopefully you’re doing things. Yeah, I believe you are.
Jeff: [00:16:07] Vulnerability. I think that’s probably the biggest thing you can do is show vulnerability, not being afraid to say, I don’t know. One of the big things that we have on my team is, you know, I don’t know is an okay answer. It’s just not the end of the conversation, right? You have to figure things out. But giving people that sort of room and that space to be able to learn, to be able to explore. Another thing I like to do is push education as part of your job. There’s no reason, I mean, you’re already on call. Why should you have to learn this new technology for my company’s benefit on your own time? Spend time at work learning this thing because it is part of your job. I don’t expect you to just know it. So if you’ve gotta take, you know, an hour or 2 hours a week to just go sit in the kitchen and read, that’s fine. I wanna make sure that you feel comfortable doing that. So, you know, between those things, I think people begin to get comfortable. I think when people also want to dissent on an opinion, creating that safe space so that we can have a dialogue and talk about it, because that’s probably one of the biggest areas where people bottle up, where they don’t feel comfortable disagreeing, so they kind of go along with something that they know is wrong or that they feel strongly against. Creating an atmosphere where people can disagree in a constructive manner and know that at the end of the day We’re still a team and we’re going to be able to figure this thing out. I think that’s strong. I think it’s a testament when the teams disagree, but at the end, we come to a resolution and everyone is still all in.
Veronica: [00:17:34] One thing that I’ve noticed is I’ve been in work environments that were very helpful in this regard and could be very unhelpful as well. And the major difference was having that time to be able to learn and When things did go wrong, you know, in my research life, one time we had like a piece of machinery that had a problem that no one realized for a month. And there was a lot of like backtracking that needed to happen. But we went through it in like what I think we would call a like blameless way. And it was an educational experience for everybody. Like we said, okay. And we all knew how that machine worked inside and out at the end. Why? Because we had something go very wrong, cost us time, and we had to trace back. And those 2 pieces, being able to sit in the kitchen and get better at your job while at your job instead of having it be something that’s burning you out in the evenings, and having a situation where when something goes wrong, it’s less about the person who made the commit and more about learning about the process better, because that’s obviously something the team needs to do, have been measurable in both fields that I’ve worked in, actually.
Trevor: [00:18:56] So I think—
Matty: What are maybe some, again, thinking, we kind of talked about things that leadership can do. Maybe what are, sometimes we’re in a position where we’re not able to we’re not leading the team, we’re an individual contributor, and we might be in a position where we don’t feel that level of psychological safety. What might be some things that we could do to help influence leadership, to help influence the team based on your experiences, hopefully positive ones somewhere, or not?
Jessie: In most cases where I was in situations like that, I tend to have a lot of candor with leadership, and that can go one of two ways. Depending on who the person is. So I myself, like, if someone came to me and did what I do to other people, I would be like super happy to know like exactly what they felt. But some people do not like knowing exactly how people feel. So yeah, it really depends.
Veronica: I just wanna work with you.
Jeff: [00:19:58] I think it’s also important to think about leadership as a role, not necessarily as a position, because I’ve been on teams where I wasn’t the manager, but I was definitely the leader of the team. I don’t remember what author said it, but someone said, if you walk into a group of people, you can instantly pick out who the leader is regardless of their position. So, I think it’s important to recognize that if you are the leader, the emotional leader of the team, that there is a bit of psychological safety cover you can provide for the rest of the team. And you can be that person to engage with your manager and say like, hey, listen, XYZ isn’t working out for us. Maybe we can try something different. Maybe it’s you that solicits the extra opinions, right, when there’s a conversation in the everything has sort of fallen flat, you kind of know that like, all right, I know Max is stewing about this thing because he hates Node.js. So why don’t I, you know, give Max an opportunity to sort of talk about it? So just sort of creating those environments where one, it’s clear that you’ve got that person’s back, but then also set them up to be able to, you know, respectfully disagree.
Matty: [00:21:00] I think as well, One of the, like, when you think of one of the things and one of the definitions around teams that have high level of psychological safety, as an example, is there’s an equal amount of conversation. Everybody speaks the equal amount, right? And that’s easier said than done. And if you’re not in that, again, if you’re not the manager, I think about, to like, back here a couple years ago, we had Anwen Simmons gave a great talk about lending privilege. So if you are in that team and you have privilege, a responsibility to help drive that, right? To help drive the— you know what, it’s real easy for me to talk a lot. Surprise, surprise to everybody who’s been at the event all day, right? But think about ways that I can use that ability that I feel safe doing that all day long to help people who are not necessarily in that position right now.
Jeff: I think that’s a huge point because, you know, when I first started in technology, I was really taking any job that anyone would give me. And as I became more experienced and became more confident, now I feel confident, you know, putting stuff on the line and saying like, you know what, no, this is a really bad idea and I don’t mind calling you out on that. So you’re right, lending that privilege to be able to, you know, perhaps shield someone and say like, hey, you know, Max has got a really good point and we need to talk more about this. You know, that can be huge for someone because, you know, if I get laid off or fired tomorrow, I feel confident that I’ll be able to get another job. Someone else may not be that confident in their position, and being able to lend that privilege is, is huge for them because it gives them an opportunity to, you know, speak up.
Jessie: [00:22:37] I, I completely agree. Like, starting out my career, um, I, I wasn’t necessarily the person who could, like, be kind of aggressive and get away with it. Um, but now I have, like, so found myself in such a position that, yeah, like, I will do that whenever needs be, because especially having known the other side of it and knowing when you can’t speak up, I always try to at least say something.
Veronica: One thing that I wanted to say, kind of jumping back a little bit, is when you’re approaching, when you’re taking the position where you’re like, hey, I’m going to approach my manager and say, hey, this isn’t quite working, one thing that I found really helpful is the whole praise in private, What is it? No, praise in public, reprimand in private, not the other way.
Matty: I see you’ve worked for some of the people I’ve worked for.
Veronica: So you got to get that right. But going to that person and recognizing that they may be feeling their own vulnerabilities as well. We all have different paths into leadership and comforts in that role. And that person may be with their face on trying to figure things out and feeling just as vulnerable as you all are. So taking the time when that time is available and trying to make that a smooth conversation can help a lot. Some of the worst workplaces I’d been in would involve people just coming up, ambushing you in the hallway kind of situations. And that can be really stressful and kind of ruin your your whole prospect on what your day is and like make you want to hide in the bathroom. And we don’t need that.
Trevor: [00:24:20] And when it comes to lending privilege too, sometimes it’s not— doesn’t need to be in the moment, like in front of that person either. Like bringing people up in conversation, reminding people that the idea that came that you’re all moving forward with was somebody’s idea and talking about them in that light that lets them shine in the best way possible is another way and a very important way to lend that privilege.
Matty: So as we’re starting to get towards wrapping up, I wanna go back to your talks and to the things that you talked about today. And if you had one key message from the presentation you gave, what would be that thing you want people to take away?
Jeff: That we need to take a bit of responsibility for the things that we’re creating. We need to own some of that, some of those consequences so that these aren’t things that are just happening in a vacuum. So when there are bad actions happening, we can’t just pass the buck on that. It’s up to us to step up and say like, hey, something’s wrong. And maybe we’re not currently as an industry in a position where people feel safe doing that, but, you know, it’s on us to figure out how to make that safe for people to do it. Because as technology gets more advanced, the stakes are gonna get higher and higher.
Veronica: [00:25:36] Comments are a form of documentation, and documentation is good. And we should make sure that we’re supporting people in doing the best documentation for them.
Jessie: I would say the people who work on various layers of software should talk to each other more, and in talking to each other, also actively listen, and then kind of try to come to a solution on the problems between the interfaces.
Matty: Fantastic. And one last thing I’d like to do, and then before we kind of wrap up, because I know we’ve got a panel who’s doing all sorts of interesting and different things, and we wanna give you an opportunity to plug a thing you’re doing, whether it’s your podcast, whether it’s— I’m not gonna tell you what to plug. So even though I just did. So we’ll let you start, Jeff.
Jeff: God, I’m in the process of writing a book, and maybe one day I’ll finish it. I don’t know. It’s a tough slog. But right now, I’m working on a book tentatively called Real World DevOps. And the idea is to give it from the perspective of an individual contributor and what they can do in sort of imperfect organizations. I think there’s a lot of great work out there for leaders that wanna do sort of a top-down approach, but there’s a lot of people out there that are struggling in environments where there’s a lot of stuff that they can do, And they just really need sort of a field guide on how to go about accomplishing that. So that’s sort of the target audience for my book, due out 2020 sometime.
Matty: [00:27:07] Plus or minus. Yeah.
Trevor: Hey, Matt, how’s that book you’re writing coming?
Matty: Which one?
Veronica: So hold on to your hats. This is my first time announcing this publicly. I met 2 fabulous other first-time speakers when I went to my— and spoke for the first time this last— this February. And the 3 of us are starting the Quick Developer Guides YouTube channel. And we’re going to have a series of 10-minute videos on everything from how do you get started using version control to, so you’re thinking of quitting your job and doing this I’m going to be a dev thing. How are you going to budget for that? So a whole bunch of different skills that people will want to consider. If they’re getting started in this whole, this whole world.
Matty: So yet again, some new project is announced for the first time on Arrested DevOps. The, uh, Michael Ducey and Ross Clanton’s podcast was announced first here. I believe, Jesse, you and Andrew started your podcast, talked about it on ADO. So like, you know, breaking news. But yeah, what, what new rad thing do you want to plug and promote? Or old rad thing, whatever.
Jessie: [00:28:24] Well, I’m currently unemployed, so I guess I’ll plug some books. Uh, Soul of a New Machine and Ben Horowitz’s The Hard Thing About Hard Things. Very good.
Veronica: And if you’re job searching, you should tell us what kind of role you’re looking for.
Jessie: I am not actively looking for a job.
Jeff: All right, I was gonna say, I have a hard time believing she’s looking.
Matty: I will say it’s like every time, like wherever I’m working and something happens and Jesse might be available, like, let’s get Fred. I’m like, yeah, no. I mean, not that I don’t want to work with him, but do you know who you’re talking about? Right? Yeah. She doesn’t want to come.
Jeff: She’s not looking. She’s choosing.
Matty: Right, exactly. Yeah. We would know. So, yeah. So this is great. I’m glad to have another wrap up another Arrested DevOps podcast here at DevOps Days Chicago. I want to thank our awesome panel for joining us. Really appreciate it. And also, as an organizer, really appreciate all of you being part of our program. I’m really excited. So, thank you. Let’s give it up for our panel. It’s a live studio audience.
Veronica: [00:29:24] Applause.
Matty: Hey, when we recorded this in Minneapolis, I actually had like applause signs. So, at least I didn’t do that to you. So, yeah, you can check us out on the internet at arrestadevops.com. You can find us on iTunes at arrestadevops.com. Leave us a review in the iTunes Store. That’s a thing you can do. We are now on iHeartRadio apparently. So that’s a thing if you like iHeartRadio and we’re on Spotify. So we’re all the places you can find us and listen in your heart’s content. And with that being said, as always, I am your co-host Matt, @MattStratton on Twitter.
Trevor: And I am Trevor, @TrevorGHess on Twitter.
Matty: We are Arrested DevOps and remember, there’s always DevOps in the banana stand.
