Fostering a Quality Assurance Culture for Marketing TeamsFostering a Quality Assurance Culture for Marketing Teams

Fostering a Quality Assurance Culture for Marketing Teams Transcript:

Announcer: You’re listening to The Kula Ring, a podcast made for manufacturing marketers. Here are Carman Pirie and Jeff White. 

Jeff White: Welcome to The Kula Ring, a podcast for manufacturing marketers brought to you by Kula Partners. My name is Jeff White and joining me today is Carman Pirie. Carman, how you doing, sir? 

Carman Pirie:  I’m doing well, Jeff. Great to be on another podcast with you. 

Jeff White: Indeed. 

Carman Pirie:  Yeah. It’s always a fun half hour. 

Jeff White: It is, indeed. 

Carman Pirie:  And welcome to everyone listening. 

Jeff White: And you know, I’m really stoked on our guest today, because we’re gonna be talking about something that isn’t necessarily considered by every… Manufacturers have, depending on their scale, and how international they are, they may have a large marketing department. There could be numerous people working there. But they’re not necessarily thinking about things the way that our guest does. You know, even if they’re running a big kind of internal agency. 

Carman Pirie:  Yeah. Most people don’t even think about moustaches the way our guest does. 

Jeff White: This is true. 

Carman Pirie:  So, yeah. I mean, let’s just get it going. 

Jeff White: Yeah. I mean, you know, it’s unfortunate that not everybody gets to see that we get to see this luxurious moustache daily. 

Carman Pirie:  And it’s not a video podcast, sadly. 

Jeff White: No, it’s not. It’s not. 

Carman Pirie:  But the guest that we’re going to introduce to you is Kula’s Director of Operations, head of quality assurance, and by far and away the best moustache in our organization. 

Jeff White: Probably best dressed, too. 

Carman Pirie:  Well, yeah. 

Jeff White: It’s a whole package. 

Carman Pirie:  Yeah. That’s fair. That’s fair. Maybe we should just not do the show and just put out some model photos or something of Wade. 

Jeff White: Yeah. Or you know, record selection. Also-

Carman Pirie:  Maybe it’s all about a culture play here rather than work. 

Jeff White: As it so often is. 

Carman Pirie:  Anyway. I don’t know where we’re going now. 

Jeff White: I don’t either. 

Carman Pirie:  But that’s the… Welcome to the nature of a podcast introduction, where you try to be just a little off topic before you narrow in. 

Jeff White: Get on topic. 

Carman Pirie:  Yeah. 

Jeff White: Yeah. And this is what agency life is like. 

Carman Pirie:  Something like that. 

Jeff White: It’s a cast of characters. 

Carman Pirie:  Well, let’s get on with it. Wade Prue, right? 

Jeff White: Indeed. 

Carman Pirie:  Isn’t that his name?

Jeff White: Wade Prue, Kula’s Director of Operations and all things quality. Welcome to The Kula Ring, Wade. 

Wade Prue: Thank you. Thank you for having me. It’s a fun, different thing to do today. 

Jeff White: Well, it was until that introduction went completely off the rails. 

Carman Pirie:  Yeah. 

Wade Prue: Yeah. No, certainly-

Carman Pirie:  All right. Today… Well, look. Before we dive into a conversation about what we might mean when we say a culture of quality within the digital delivery of manufacturing marketing, why don’t we just maybe start with learning a little bit more about Wade from the horse’s mouth here that doesn’t have to do maybe with the moustache? Wade, tell us a bit about yourself. 

Wade Prue: For sure. I’ve been working in IT for my entire life for the most part. I guess where it really kind of got quality driven as far as my career path goes is kind of the early 2010s. I’d started at Blackberry doing more kind of enterprise support leaning stuff and then kind of leaning more towards quality at the end of my tenure there. Then transitioned off to work in doing oil and gas, which was very, very quality driven, so looking at kind of interaction between software and then hardware running on oil and gas rigs until I found myself employed with you happy fellows in I guess 2016, now. 

Jeff White: Wow. It’s been five years. 

Carman Pirie:  Yeah. Time flies. Yeah. Yeah. Well, there you go. And it is lovely to have you on the show. I’m excited about today’s topic. I think it’s a really… So, I guess let’s jump into it. This notion of I guess one thing I… We don’t want to beat up on our listeners too much or our listeners’ organizations too much, so bear with us I guess, folks, just a little bit. But one thing that we would say is that manufacturers have an outstanding amount of focus on product quality that is certainly… In some ways, it almost goes without saying. And in a world where the marketing department is often spending more on technology than the IT department, what our hope is for today is that today’s guest can take us through ways that we can bring that quality focus and emphasis to the world of marketing and marketing’s IT endeavors, and really, if you will, level up to the level of quality that the organization might be seeing elsewhere. 

Jeff White: Absolutely. And you know, it really is the case that with so many marketing initiatives coming to life digitally these days, it really requires having that lens of quality assurance on every part of the deliverable. You know, whether you’re putting out a campaign, or reviewing how your analytics and KPIs are set up, or launching a new digital platform, website, eCommerce store, what have you, quality assurance is something that is well known in the software world but not necessarily as well thought about or as frequently thought about in the marketing world. Wouldn’t you say, Wade? 

Wade Prue: For sure, and I think you kind of picked something out there that kind of jumped out to me when I was thinking about this ahead of the podcast today, is that you can have all of those very specific elements that someone can look at the analytics around a campaign, or look at how an asset behaves, and there’s often so much more to be garnered by how all those pieces can come together and how they’re represented to the customer, especially when it’s gonna be different people kind of working on those various elements before that campaign’s launched. And then ideally, after that campaign’s launched, as well. It’s easy to look at it as like a collection of individual parts that don’t have kind of a cohesive structure kind of driving them the same way it’s delivered to the client. 

So, having that ability to kind of step back and see it from the client’s point of view, how they’re all integrated together, often brings kind of a whole new light to it both when launching a new campaign or monitoring it as it’s already out in the wild. 

Jeff White: So, what sorts of… You have the opportunity to see an awful lot of digital platforms, both the ones that we build and ones that are brought to us by new clients and things like that. What sorts of things are you seeing and where do you think most organizations need the most help? 

Wade Prue: I find it’s in that back half of what I mentioned, where they’ve got an asset or a campaign that they really point to as their bread and butter, be it a calculator or a PDF that they really hold up as kind of the definition of what they do or what their kind of top of funnel offer is, and you go digging into that and there’s elements about it that just don’t hold up because of the progression of time, where something doesn’t hold up technically the same way that it did, or that people have not been looking at it from like a say performance point of view. If it’s let’s say a page has been sitting in the wild and traffic just isn’t hitting it the same way, it may have not made it back to that organization that there’s a particular problem with that as far as kind of how it ranks just in terms of performance, broken links, like very particular little things that just no one’s gonna notice unless they go back and really dive into it. 

Carman Pirie:  So, you say some of it’s performance based and kind of just the passage of time, and so it sounds like what you’re saying is it’s just really about understanding that QA isn’t just something that happens pre-launch of a new initiative. It’s thinking more on an ongoing, be it… Is it, are we thinking like a regularly programmed and identified QA structure, where every quarter or year or what have you there’s an initiative to check existing assets? I guess how would you suggest addressing this challenge? 

Wade Prue: I think there’s two really good ways of approaching it. If you’ve got an asset or a campaign that you’re actively promoting or you’re holding it up as your kind of foundational offer for a given client, or for your organization rather, having those scheduled checks certainly doesn’t hurt. As we both touched on, because expectations around performance, expectations around how a given browser renders a page can differ over time so much, if you’re either depending on that asset or campaign to derive leads from or if you’re pointing a pay campaign or a PPC campaign at it, you really do need to expect it to kind of keep doing what it’s doing over time, so having those scheduled checks certainly doesn’t hurt. 

Carman Pirie:  And are we thinking  quarterly? Is this an annual thing? What’s an appropriate passage of time for something that’s that mission critical? 

Wade Prue: I think it would really come down to kind of lead volume and the value of those leads but depending on how much you can kind of tighten up those checks and the amount of time that you’re going to invest into a check like that, ideally like once a quarter, really. But beyond that, it doesn’t necessarily need to be a scheduled thing. There’s a lot of value in having kind of UIs on a problem, too. It’s certainly something that comes up in a QA context is that I can kind of sit down and test the same rote thing again and again and see no problems with it, and you bring a fresh set of eyes to it, be it a marketer at Kula Partners or a different QA member, and they can sit down, run roughly the same test outline, and see wildly different things because of the experience they bring to it. 

Jeff White: I don’t imagine most of our listeners have ever been through a QA process themselves or formulated the kinds of tests that you’re used to doing. What are you doing when you’re testing something for quality, and performance, and accessibility? What are the steps that you’re taking and what are you looking at and how… What sort of methodology are you utilizing in order to get or to bring things up to the level that you’re hoping that they’ll be? 

Wade Prue: I think it’s always gonna depend on what it is you’re testing and why you’re testing it and who you’re testing it for. And then obviously a time crunch as well is a huge factor. But ideally, before you kind of sit down and decide how you’re gonna approach it, it’s trying to figure out, well, how much time do I have and what are the risks associated with this? And once you’ve got a handle on that, you can kind of focus more on what are the requirements of what this is supposed to do? Be it functional requirements, so if it’s like a form, does it submit properly? Or non-functional requirements, like performance. 

And really trying to get kind of a little bit from each category into a given test outline, in particular. And then if we’re talking about something that’s the sum of a whole, let’s say it’s a marketing campaign like I touched on before with a conversion path and asset, analytics tracking around it, as well, it’s trying to take a step back at the end, once you’ve tested all those individual pieces, to make sure that you’ve kind of got that top level, final check, smoke test of everything before that goes live. 

And then to Carman’s question earlier, to possibly reuse a test like that once a campaign does go live, to act as documentation both for what was done in the past and then how to come back and kind of assess that in the future. 

Jeff White: I think there’s an interesting, nuanced point that people who’ve worked in development would understand, but not necessarily everybody, although everybody has experienced software bugs or hardware issues and things like that, where something doesn’t quite work properly. But all software has bugs in it. Even the most fully tested piece of code is going to have problems when it encounters a certain situation that you didn’t necessarily think about when it was being developed. So, I think there’s an interesting… You almost have to let go of this idea of perfection and realize that there are categories of testing that need to be done, and they go from capturing the biggest possible issues that you might see, and then knowing that you’re not necessarily going to have the time or the budget to fix absolutely everything, and some things are going to have to be left for either another time or they’re less critical to the functionality or the performance of the campaign. 

Wade Prue: I wasn’t gonna jump into the minutiae of that, but that really hits it on the head, and it touches on what I struggled with kind of early in QA, is the notion of kind of priority versus risk versus need. Be it your own internal needs from something or the client’s needs from a given kind of solution that you’re developing for them. 

Most test strategies, as far as kind of like a test theory and test certification programs that exist in the world, will talk about exactly what you mentioned, is that you can’t get rid of every bug in a piece of software. It’s about risk mitigation and that’s it. It’s your job as a tester to sit down, assess the risk, and mitigate as much of that risk as possible. That being said, it’s often hard to kind of… especially when you’re new to a role, or new to a product, or new to a tech stack, to determine where that risk lies or where the priority is. So, it’s kind of also so important internally in an organization for people to feel empowered to kind of speak up when they see something that doesn’t pass that sniff test, be it whether or not they’re a QA tester, they’re a marketer, or kind of just poking their head in on a campaign. 

Putting your hand up and saying, “Well, I noticed this. Is this right?” It’s that outside lens that may have not been shone on a given deliverable that can really change how it’s approached and change how testing that type of thing or similar things down the road is approached, as well. 

Carman Pirie:  I think there’s something here when we think about… You know, we started this by saying a lot of marketing organizations might not have established a culture of quality assurance around their digital marketing efforts in the way that we might hope or imagine. It’s just a completely different skillset versus… Provided the marketer’s been around for a while, I’m gonna… I’ve got enough grey hair at 45 years now or so that I can… I still used to run a print shop, and I remember doing print ads as a marketer client side, and print ads can be perfect. I mean, the ad’s not perfect, admittedly. I suppose you could always shoot holes in the creative director, something like that. But you know what I mean-

Jeff White: Technically. 

Carman Pirie:  We’re talking about Pantone specs. You would go do a press check. Is the Pantone spec, is it hitting right on this brochure that I’m doing? So, perfection was the goal. This notion of, “Guys, you know what? You need to be okay with the fact that this product brochure is just never gonna be perfect. There’s always gonna be a bug in it.” People are like, “What are you talking about?” 

Jeff White: What? Oh my God! 

Carman Pirie:  So, it does seem like we’re… To get to where we’re encouraging people to go, it’s gonna require not just new skills, but maybe a different mind shift, a different mindset. I think that’s a really… I think that’s oftentimes the harder challenge, isn’t it? 

Wade Prue: I think that actually ties back to kind of the risk mitigation aspect of what we touched on before, is that if you’re kind of throwing out like a one-time email to your customers, that’s a very different thing than say like a sales calculator tool that you’re trying to put in front of them, where that’s gonna have so many shifting conditions around it. That’s the same ones we touched on before. And then your expectations as a business and the numbers if that’s going to turn out, so if there’s elements X, Y, and Z that go into the calculator, and a given number that comes back, surely as a business those are gonna change over the years and you kind of have to put time, budget, and effort aside to kind of maintain that asset going forward. It’s a very different thing to put that out into the wild than it is to throw a print ad out there or throw an email out there and kind of hope for the best. 

Announcer: Are your digital marketing efforts bringing in too many junk leads? Stop wasting time and distracting your sales team. Account-based marketing can help give your marketing strategy the laser focus on the qualified buyers that you need to increase your pipeline velocity, close more deals, and grow your business faster. We’ve created a sample manufacturing ABM plan to help you get started. Download the sample manufacturing ABM plan at bit.ly/sampleABM. That’s B-I-T.ly/sampleABM. 

Jeff White: Man. I mean, who among us hasn’t sent an email incorrectly and hopefully it wasn’t to tens of thousands of people, but you know-

Carman Pirie:  We have interviewed another Kula team member on this podcast who will remain nameless who was responsible for sending a failed Christmas greeting email way back in the day, and I don’t think they’ve ever fully recovered from that moment. 

Jeff White: Fully recovered. No. Yeah. You just brought… I just got waves of chills and all of that, like I feel like I’m like 24 hours post second dose of Pfizer vaccine at this point. 

Carman Pirie:  Yeah. Guys, I gotta acknowledge that I’m reasonably easy to get along with until probably something like that happens, and then probably the first 10 minutes of things that come out of my mouth should just be ignored by people because it’s probably going to sound a lot more harsh than is intended. 

Jeff White: Yeah. 

Carman Pirie:  So, I think that’s part of the scarring that’s left on that. That was good fun. 

Wade Prue: I had totally forgotten about that. And that same sense of dread just washed over me all over again thinking about that fail. Love it. 

Carman Pirie:  Yeah, so you know. 

Jeff White: But everybody makes mistakes. 

Carman Pirie:  Yeah. 

Jeff White: And that’s why we have QA. 

Carman Pirie:  Exactly. 

Jeff White: So that they’re minimized and hopefully… And in all honesty, yes, it can be embarrassing to send an email that’s not finished, or incorrect, or to the wrong people, or whatever, but it’s generally not fatal. It could be, but you know, it generally isn’t. But you know, a major tool that is being put out there with a significant budget that is meant to drive leads, that is business critical, these are not things that you can… You can’t be lackadaisical about the quality side of those, and it would be the same thing as calibrating a machine or whatever on the factory floor. 

You know, those are things that are widely understood within manufacturing organizations, but-

Carman Pirie:  That that calibration needs to be revisited, and that quality output needs to be monitored, and continuously checked, and I guess that’s exactly what Wade’s advocating for here. 

Wade Prue: That’s exactly it. And it’s to I guess dig into a little more too, it’s not just that it’s assets coming to us, that we’re coming upon them and going, “Okay. Well, this was like this, and we need to fix kind of problems X, Y, and Z with it,” either. It’s asking those questions before you develop something like that and making sure that you do have that time for this, you do have these kind of questions rattling around in your head as you design something like that, that you expect to have that kind of longevity. And I can think of multiple clients where we would work with, where we talked through a problem like that, and you really start the wheels turning in their head about how they’re going to manage something like that once it’s out in the wild and kind of what the upkeep expectations are going to be. 

Carman Pirie:  I really like the idea of combining this a little bit with the thinking from a previous guest on The Kula Ring, who took us through this notion of marketing products versus marketing projects, and in some way, that might be an interesting frame to think about. If you’re a marketer and you’re undertaking a new initiative, ask yourself, is what I’m doing here, is it a project or is it a product? Am I creating something that needs ongoing iteration, improvement, testing, evolution? Or am I creating something that is from the start, we understand it to be a one-and-done? 

And I think if marketers ask themselves that question, they would probably step back and look at the totality of what they do in 2021, 2022, versus what they may have been doing in 2012, and they would say, “Wow, there’s a lot more products here than there used to be.” There’s a lot more stuff that I do that’s product now and there’s probably a lot less that’s just a project. And for everything that’s a product, it needs to have a QA plan associated with it that is… whose longevity matches the expected longevity of the product. 

Jeff White: And be prepared to make that investment. 

Carman Pirie:  Yeah. 

Jeff White: Yeah. 

Carman Pirie:  And it doesn’t necessarily mean that it’s a crazy investment. We don’t want to suggest that it is. But it’s a commitment. Yeah. I think Wade, am I… Have I drank the Kool-Aid? 

Wade Prue: I think that’s exactly it. I think some of the most interesting conversations I’ve had with clients since I started at Kula Partners are centered around requirement generation for those sorts of problems, is that they have a need to develop a software solution, a calculator, and asset, and then you start drilling into those questions. And like I said, you see the wheels turning and then building out the requirements for that both for the deliverable and the long term is absolutely key. So, everyone’s expectations are the same going into it. It would be just terrible to kind of take any kind of asset like that, throw it out into the world, and then have it be a surprise to the organization after the fact that it does require that care and attention for whatever reason once it’s out there. 

It is that long-term plan, like you said, Carman. 

Jeff White: Well, and I think too, there’s an element of… Because QA technically is often done towards the end of an engagement, before something goes out the door. But it’s actually thought about from the very beginning because the strategy that you’re implementing and the requirements that you’re gathering, and the diagnosis that you’re doing on the existing platforms, and all of that, are all helping to inform how you’re going to think about how this new thing is created, whether it’s a digital campaign, or a website, or a calculator, or tool, or something like that. The QA doesn’t just get added on at the end. Okay, can you just go and check this and make sure it’s okay? No, it’s like it’s from the very beginning. You’re thinking about, “Okay, what is this thing going to need to do and how am I going to ensure that it accomplishes those goals?” 

And you know, it really isn’t a last stop before going out the door. The quality process needs to start at the very start. 

Wade Prue: I completely agree. I feel like maybe you quietly ducked out at lunch and read a test foundations manual-

Jeff White: I did not. 

Wade Prue: Because it’s so core to kind of test foundational thinking, making sure that it’s not a, “Oh, and just one last check before we go out the door,” like you said. It has to be ingrained from the start. It has to be ingrained when you’re deriving requirements, writing stories about what the outcomes are supposed to be, and making sure that QAs eyes are on it all the way along. And I found like even in my experience at Kula Partners that that’s where things really started to kind of click for me and shift, is that I came on early and it was panicking about every little thing before it went out the door, and then as soon as I could kind of see a project from start to finish and kind of assess, well, here the customer needs are kind of pushing this project along, and then kind of get my hooks into where those testing touch points are throughout the project, both on the dev side and the marketing side, the things really started to kind of click and I could kind of put aside the stuff that really didn’t matter and focus on the specifics when it needed it and the sum of the whole when that was more important. 

Carman Pirie:  You brought this up a number of times, this notion of testing the individual pieces versus the whole. It’s been a consistent thread through this conversation. I guess is that… I mean, do you see a lot of problems there, that people, even folks who have embraced the notion of QA, they want to test what they’re putting out there, they want to do this right, do you see that even in those instances sometimes people can fall into the trap of only testing the pieces and not testing the whole or thinking of the whole? And then I’m curious, because part of me, if that’s true, the cynic in me would say, “If every piece of it’s been tested and works, then the whole thing should work.” 

But I get a sense you’re telling me that that’s wrong. 

Wade Prue: Yeah. No, and I certainly didn’t mean to harp on it at all, but it’s-

Carman Pirie:  I didn’t suggest you were harping on it. 

Jeff White: Bringing focus. 

Carman Pirie:  It’s a consistent theme, Wade. 

Wade Prue: Sure. I think it’s a few different things. If I’m looking at it in the nicest light, I think a lot of it comes from people not wanting to kind of overstep their bounds as far as domain, so when I think about marketers that I’ve worked with, they go in and they’ll do their checks of copy, and some checks of links for instance in a given asset, and then it’s then on kind of developers to kind of bring their spin to it, or a designer to bring their spin to it. And they may not want to step outside of their kind of contribution to a given asset, and I found that just about in every case when I’d sit down and talk with them and talk through kind of, “Well, how is this going to work with the other pieces,” that they were very interested in how those other pieces worked. 

I had a conversation as recent as last week where I was getting into how I was approaching something when it came across my desk, and that was after kind of all the various domains at Kula had touched it, and got kind of a very positive response when I said, “Okay. Well, great. If you want to quickly look at how it is all working together,” like you said, Carman, “This is how you do it.” And then went away, and came back, and that was very informative. I had no idea those worked like that. I’d stepped outside of my comfort zone. And it sounds like a made-up story as I’m saying it out loud, but like I was surprised to get that feedback back, that that was something that  they were both interested in and that it brought value to kind of what they contributed to that particular campaign. 

Jeff White: There’s also something, going back to what you said at the beginning of the show, Carman, around how marketing now generally has a greater spend in information technology than IT does in a lot of organizations, and that usually also means numerous disparate platforms getting used in every campaign. You know, it would be rare that there would be less than three different pieces of technology coming together to deliver even the simplest of marketing deliverables these days. Things have to tie back to the CRM. They’ve gotta show up properly in analytics to make sure that we’re tracking the right things. The web landing page has to function correctly. And then there’s the ads that are driving traffic to it. I mean, that’s four, and that’s a really basic thing for just talking about a basic PPC campaign. 

So, this idea that you can only really look at your own little piece of it and be comfortable that the whole thing is working well is just… It’s just a fallacy, because there’s too many moving parts these days for that to ever be sufficient to truly be comfortable that what you’re putting out  there works. 

Wade Prue: For sure. And like I’d said, it’s not always that people don’t want to. It’s really just kind of encouraging people to feel empowered to kind of take that step outside or step back and start poking at something from a different perspective. I’ll touch on it because you jumped on kind of testing foundational concepts a couple of times, but a kind of self-explanatory testing concept is the idea of exploratory testing, is coming to something and just poking at it. Time boxing it to a set period of time and saying that you’re trying to learn X, Y and Z about something that you’re trying to test. And then just write down your findings. 

And I’ve worked in companies where that was an exercise that was taken on cross-functionally on a regular basis, and the amount of information that was derived from doing something like that was very interesting, but then the empowerment that came from it, as well. You’d see people who wouldn’t normally speak up about stuff like that putting their hand up more often and going, “Well, this doesn’t smell right to me. What is the problem with this? Is this something we can fix?” And then you can then take that back and kind of make that that priority risk assessment and then triage it appropriately. 

Carman Pirie:  That’s really interesting. That’s-

Jeff White: Practical? 

Carman Pirie:  A very practical approach that I think a team could adopt tomorrow and be able to get multiple contributions to a culture of quality started. I think that’s-

Jeff White: And some real value. 

Carman Pirie:  Yeah. Wade, this has been a fantastic conversation. Thank you for sharing your expertise and experience with our listeners. I am certain that they will have enjoyed it as much as I have. 

Wade Prue: Pleasure talking about my day to day. It’s always interesting for I guess me to take a step back from my own job and kind of focus at the higher organization level. 

Jeff White: It’s a rather meta day. 

Wade Prue: It is. Yeah. It’s bizarre. 

Jeff White: We’ll make sure we get some QA on this podcast before it goes out. 

Wade Prue: Awesome. Thanks, guys. 

Jeff White: Thanks a lot. Cheers. 

Announcer: Thanks for listening to The Kula Ring, with Carman Pirie and Jeff White. Don’t miss a single manufacturing marketing insight. Subscribe now at kulapartners.com/thekularing. That’s K-U-L-Apartners.com/thekularing.