The big deliverability episode with Brad Gurley of MessageGears and Nick Schafer of Mailgun

00:00:05

Eric Trinidad: Welcome to Emails Not Dead. My name is Eric and this is Jonathan.

00:00:08

Jonathan Torres: Hello.

00:00:08

Eric Trinidad: Hello. We're just a couple of geeks here to talk to you about email and we have a very special extra large episode today here on Emails Not Dead. We have Brad Gurley, director of deliverability at MessageGears, joining us today. And we have our own in-house deliverability daddy, senior manager of deliverability and compliance, Nick Schafer. How are you, sirs? Both of yall?

00:00:32

Brad Gurley: Doing well, thank you.

00:00:34

Nick Schafer: I'm doing good. I was waiting to see if Brad would say I'm doing good first.

00:00:37

Eric Trinidad: Okay. You're a gentleman. I appreciate that Nick. Well, guys, it's our big deliverability episode. You know, we talk about it in bits and pieces here and there. We like to, like, focus in on it on most of our episodes as well, when we do get the chance to, of course. So we thought we'd have a couple of experts in the field just kind of other than Jonathan and myself, of course, talk about it a little bit more in depth. So how you feel about it? What's deliverability like to you guys? Like, has anything changed since the last time we spoke?

00:01:07

Brad Gurley: Oh, gosh. What hasn't changed, I think in deliverability is usually the question that we're asking. You know, I mean, we've seen a lot of upheaval in the public opinions about a lot of things. And deliverability specifically. I mean, tracking has been a huge concern over the past couple of years, and that has its own sort of impact on deliverability and kind of how we measure things, measure engagement. We've seen a lot of impact from, you know, our friends at Spamhaus. You know, they've been quite a hot topic in the deliverability space over the past few months. So we've seen a lot of activity from them, a lot of discussion around that. Yeah, I mean, those are a couple of things that just kind of come to mind. Nick, what about you?

00:01:45

Nick Schafer: I'll echo the Spamhaus comment. These days I'm focusing a lot on anti-abuse and protecting the Mailgun platform, so I'm looking for patterns and trends. And, you know, whenever we see an increase in listings from, you know, Spamhaus or other blocklist operators, it gets our attention. So we'd like to do some investigation. Keeping up to date with the new things that are going on is really important. Spamhaus', obviously, you know, changing things which any blocklist operator or anti-abuse company has to do, they have to maintain their systems to evolve because the bad actors out there are always evolving as well.

00:02:21

Eric Trinidad: Brad, you know, Jonathan, I met you at a point of our lives and understand you have voluptuous pipes. You can sing with the best of them. Other than singing, how long have you been in the email game?

00:02:35

Brad Gurley: Well, I have been in email for roughly 20 years now, I think. Well, we're coming up on 20 years, so been doing it for a little while. I've been in the deliverability space for most of that time. I've been with a few different ESPs, I guess five different total. So I've seen kind of the gamut from SMB, you know, kind of mom and pop shops all the way up to current environment, which is kind of super enterprise senders. So I have seen a lot of change and a lot of things come and go and deliverability during that time for sure.

00:03:05

Eric Trinidad: We talked about some of the changes there with Spamhaus and things evolving, you know, with some of these changes and the ever-evolving space. Is there like a provider out there that you can kind of see them making huge leaps?

00:03:18

Brad Gurley: Really haven't seen anybody have as much impact just from a blocklist perspective as Spamhaus and really kind of make notable changes. And you know, it may be that Spamhaus really has become the most prominent blocklist out there. I mean they're really are the one thats sort of is most trusted people really rely on that data that you don't see from other small ones not to disparage or kind of, you know, talk smack about any other block lists out there. But yeah, I mean, it's, you know, Spamhaus really is kind of the authority when it comes to blocklist. So when they make a change, it's something that can tend to have ripples across the industry. We've certainly seen some other providers make some adjustments, tighten things down here and there. But usually the one that we hear, you know, everybody starts talking about when they make a change is going to be that Spamhaus.

00:04:01

Nick Schafer: With regards to Spamhaus I actually listened to the podcast you guys did with Matt Stith recently, and you know, he made a good point on that. And this is something I think people fail to remember is that they're just providing the data. It's up to the people that are receiving mail and what happens. And since they are trusted, that's why you always hear about Spamhaus. There's hundreds of blocklist operators out there that do the same thing, but they don't have the trust that Spamhaus does. So it's an important kind of piece to remember that Spamhaus isn't doing the blocking, they're just providing the data. But it is trusted.

00:04:35

Jonathan Torres: It feels like millions sometimes, Nick, that there's a blocklist out there and I kind of like when we were having like that discussion of like what Spamhaus is doing and it really overall when it comes to how platforms are behaving right, like what are we doing? And I think it goes a little bit hand in hand a lot of times with what the platform, what the ESP is doing, what you know, where you're sending from, what data you're using, what datasets your using, how they're reacting to that data, and then a little bit of the sender of themselves. Right? We have to kind of pay attention as the sender. What am I doing with this data? How am I consuming it? What am I seeing that might be different? Or what do I need to pay attention to in a different way to make sure that I'm complying with all the rest of that stuff? And I know that was a big part of our conversation and I think we need to start revisiting that because I know the first thing is I had two notes that I took in our meetings leading up to this, but like one was, you can see the code. It really comes down to both you guys. I feel like it's The Matrix. You just kind of paying attention to everything that's going on, looking stuff out and seeing where things need to be adjusted. But then also it literally is just the code itself because there's so many different codes for message delivery and not everything lines up all the time. And I think that's where I wanted to start the conversation. What does all that code mean? You know, like we want to pass it through blockers. We want to be more on the neo end of things where, you know, you just you don't even see the code anymore. You just know and understand what it is telling you. So I think the first thing was the division of like hard bounce, soft bounce. What does it mean? Because I know those are words and people put different definitions to those words. So it can mean a bunch of different things to many different people. So I just wanna start breaking that down first.

00:06:06

Brad Gurley: You know, I always joke that the biggest way to start an argument in a roomful of deliverability folks is to ask them what hard and soft bounces are because that is something that, you know, just as you said, you know, it means different things to everyone. Everyone kind of handles those terms a little differently. So depending on, you know, what provider is out there, they certainly do have different definitions and interpretations for those. But generally, when we look at what we call a hard bounce, again, you know, everybody's a little different. But the general consensus is that sort of a permanent bounce. So you attempt to deliver to, you know, brad@ gmail and you get a bounce back from Gmail that says, you know, this doesn't exist or, you know, can't deliver or whatever, 500 error code, you know, generally that's going to let us know, okay, stop trying to deliver to this. Soft bounces are a little bit more nebulous, but generally, you're looking at it's kind of a 400 error code and something to the effect of can't deliver now, try again later. So it's more of a transient or temporary or, you know, a few different terms that people use for it. But that's really more of that type of, you know, deferral or throttling. All of that sort of falls under what we consider to be soft balances. I think in most circles.

00:07:14

Nick Schafer: I'll kind of follow up on that. To your point, Brad, like it's dependent on, I guess, ESP. It could be different here at Mailgun than it is somewhere else or with you. But yeah, generally speaking, you know, you see those five X errors or four X errors, that's going to be the delineation permanent being the or. See there. There we go. Permanent is what I refer to hard bounces in general. It's just it's going to be different depending on who you're talking to. And, you know, conversations or arguments may follow. But looking at those five X areas versus four X is going to be the general line between, you know, hard bounces soft bounces.

00:07:53

Jonathan Torres: And I think that's also one of those things where, I mean, the way that I was taught, you know, by people here within the company or just kind of like as I've gotten out into the world, is that like even then, like the word bounce, it's one of those things where it's like, maybe that isn't the right word in the way that we should be using it. Because like, bounce is always to me indicated, this doesn't exist, it's bounce because it's not there. And then it's like the delineation then between temporary failures and permanent failures. And then what does then that break down too, because there's so many of those that that you start going into things. And this is the fun part because then you start getting into, well, what error codes should I be looking at? Even if it's a 400 that says, try again later, should I actually try again later? Or if it doesn't say try again later. It's just the 400 technically means the same thing, right? It means I can't receive now, but maybe I can receive later? And then you get something that's a hey mailbox full, you know. So does that mean try again in 5 minutes or try again the next time I'm sending out an email and then just kind of breaking that down. What are you guys using as far as like your thoughts in your thought process on when to make that delineation?

00:08:53

Brad Gurley: There are definitely a few factors that come into play there. I know one thing that that I've been sort of referring to and actually I'm planning to update is we did at MessageGears back in 2020, literally, I think it came out like the week the pandemic blew up. So great timing there. But we had a really kind of detailed belt research project that we did. So we went and looked at, you know, different types of balances and how likely they were to engage after that type of bounce. So a lot of our bounce handling has been informed by that. Like we saw, for example, you know, mailbox full balances were 80% likely to open a message within the next year or something like that. So that gave us an insight that, okay, maybe we should be looking at these mailbox full bounces as not necessarily immediately bad, but what came into play there and what comes into play, I think for a lot of ESPs is that you always have to have a threshold. So it's never just, oh, well, we saw that, you know, in a year these people are 80% likely to engage. So we should just leave them all on the list forever and never take them off, you know, So there's not really that sunset policy. So we've really tried to balance that with, you know, looking at the frequency that you're sending, looking at what types of bounces you're getting, again, whether it's a 500 or 400. Matters to some degree. But specifically, what is the messages being returned? Is it mailbox full? Is it, does not exist? Is it you know, etc. for things like does not exist generally that's a one and done. You know, if you tell me the address doesn't exist, it's gone. Where we get into, you know, more of the debate, the discussion, lively, whatever you want to call it, you know, that's where you're talking about those, you know, temporary deferrals or, you know, mailbox forwards like, well, you know, in an era where everybody has gigs of space in their mailbox, if their mailbox is full, does that really mean that they're even using that mailbox? So we've done a lot of that. We're trying to do some additional research now that there have been a lot of changes in the industry just around the pandemic, around the thing Spamhaus is doing trumps the things some others are doing just to see if that holds true. We're still seeing that same kind of pattern. So yeah, it's kind of a round the way answer to that, but hopefully they get some insight for sure.

00:10:51

Jonathan Torres: Yeah, definitely. I guess that thought process, I guess exactly what I, what I was thinking about is like what? You know, we don't need a direct answer. I don't need to know what everybody's doing. But, you know, like, what are we thinking about? Like when it comes to that kind of stuff? Because it does make a difference and it does make an impact whenever you start accumulating those. And then where do I start to start deciding what that line is?

00:11:10

Nick Schafer: You know, whenever you're doing this investigation and research into different failures, whether they're permanent and hard bounces or, you know, soft bounces. As it gets really confusing to me, whenever you see things done by one mailbox provider that is done differently by another mailbox provider. So it gets challenging from time to time. That's why I bounce, you know, a classification about handling how your ISP is doing it is extremely important. Brad mentioned Mailbox too full a second ago. That one is really interesting to me because, you know, generally you're going to see that mailbox too full error, accompanied with, you know, four x related codes, but there are some providers that will return five X's there. So should you stop sending to them, add them to suppression list or request the sender to remove those from your list? There's just a lot of different ways to think about this kind of stuff, and it's really kind of interesting.

00:12:01

Eric Trinidad: You know, Gmail with over quota or sometimes over space quota or over dash quota or, you know.

00:12:09

Brad Gurley: Yeah, yeah. Great point. It's like, yeah, you'll get the same message back from the same provider but in four different formats. Yeah.

00:12:17

Jonathan Torres: I love that. And there's always like the guide on how we're supposed to be doing that and what you know, ISP's and ESPs are supposed to be doing with the stuff and then you start getting into my threats that don't ever really hold any weight, but when somebody starts asking about that kind of stuff, I'm like, Don't make me put an RFC article in the Slack thread. I will, I will totally do that. But when you look at RFC and we break it down by the guide, right? It's supposed to be a certain way. We've been told that this is the guide of how it's supposed to be done. And then you look at it and like things just don't match up. What are the struggles with that? Like, I know that this is kind of like a little bit of a one of those like nebulous questions again and just kind of just wanted to see what you guys perspective are. How closely are you looking at that? How close are you trying to follow it compared to what everybody else is doing? Because I know that varies a lot even from, you know, ESP to ISP and so on and so forth.

00:13:03

Nick Schafer: I could take a stab at this one first, Brad, if you'd like, because I'm actually I was thinking about this before the podcast and, you know, RFC is great if everyone followed it. That's the problem is yes, it's a great guideline. You know, here you go. This is what you should do. But not everyone does that. And, you know, I went digging just to kind of look for some examples. And, you know, one of them stuck out to me. It was a user unknown response back from a receiving mail server. And everyone's going to say, well, that's a hard bounce. That's easy, right? Generally, those are accompanied by 5xx. This one was a 450. So i'm like, RFC definitely does not say anything about this. This is not what you should be doing. So that's why it's important. You know, again, when it comes to the ESP, how is the ESP classifying those messages? And in this case, you know, I'm erring on the side of caution. And if they're telling me this user doesn't exist, I'm going to hard bounce that message. So, yeah, RFC is something that is great if everyone kind of implemented it on their, you know, infrastructure the same way, that's just not the case, unfortunately.

00:14:11

Brad Gurley: And to that point, Nick, you know, there is the RFC, which is, you know, sort of the recommended best practice because, you know, we'd like it to be, you know, mandated, but we know that it's not. So there's that. And then there's just sort of the what we call in some of the industry groups like Best Common Practice is like, what are most people doing? You know, what seems to be working the best? And, you know, one of those best common practices is just as Nick said, making sure that you're looking not just at 5xx or 4xx, or that you're looking just at the text of the bounce message, but that you're looking at both together. Because a lot of times if you look at just one or just the other on an island, it doesn't give you that context that you're getting, you know, user unknown to your point, you know, okay, get rid of them, but it's a 400. And vice versa. We can see sometimes a 500 will come back with something that clearly shouldn't be a permanent error. So or at least we don't think it should anyway. So the other piece of that is kind of the way that different ESPs handle that to our point. You know, my ESP, my employer, we are more of a sort of data agnostic platform. So our customers database is the system of record. So we do, you know, kind of some basic suppressions on our side. But a lot of what we do is kind of educating our customers and saying, here's what you should be doing based on your data. You know, based on what we're seeing for you, you should be suppressing after X, whereas, you know, there are lots of other ESPs and providers out there that will say, you know, automatically if we see four soft bounces in a row that we're going to suppress or maybe they give you some flexibility there. So and again, you know, I use soft balance there as a, you know, get a generic term, but it's. So what does it mean? Yeah, but yeah, exactly. So you'll see that happen across different ESPs, not just the way they look at the data and interpret it, but then what are those common practices? And that's why I think one of the issues that came up over this past year was, well, we're not really suppressing people if they're not coming back with, you know, a clear, you know, don't try this address again. What if it comes back as, you know, blocked for spam? How long do you continue to try and, you know, you try to resolve the issue, of course, if you're seeing a spam block. But then how long do you keep trying to deliver that mail after you either think you resolve the issue or maybe you don't know if you resolve the issue?

00:16:18

Jonathan Torres: It's definitely one of those things that I know the landscape is confusing, and I always think it's funny because like RFC does definitely do you know, what do the numbers mean? What are the type of things that happen? But everything after that is just so ISP specific, like they can put whatever they want in the text of that. It's one of those things that just annoys the mess out of me because it's like cool over quota. That means the exact same thing as mailbox full, you know, to a bunch of different people and then everybody implements in a different way. And then it makes it difficult because you have to be paying attention to so many things. I know the way that, you know, the Mailgun platform works. It's one of the most familiar with and kind of as an ESP and the type of stuff that's been built out on like definitions on, you know, you see the code. Okay, cool. The code is the code. But what are they saying about it? And then what is that and the breaking down too, and then trying to break down over time, learn, implement change because it's the Wild West out there when it comes to those things. Anybody can do whatever they want, whatever they feel like doing in typing in that day for what an error is going to mean and what they're going to return for it, so be it. So it just makes it a little more difficult. I'm not going to say it's impossible. It just makes it more difficult because you have to pay attention to what's going on. You have to pay attention into what you're doing, what you're seeing, what the returns are. And, you know, to Brad's point, like just yeah, you got to notice, if you're not paying attention to it, you're doing it wrong. So you just got to pay a little bit of attention and it helps, you know, it gets a lot clear for sure.

00:17:35

Eric Trinidad: In speaking with clients, you know, and customers, you know, just over the years, like everybody just you know, we have some folks that, you know, absolutely try to just, you know, stick to the speed limit and, you know, follow to the letter like, okay, this is the bounce. I need to get it out here. But others are just trying to just kind of like move along with traffic. Okay. I know I'm kind of ignoring that, but what are other people doing? How long can I ignore this before I need to take action? So, you know, everybody's a little bit of different flavor on that.

00:17:59

Brad Gurley: And I think that lends itself to the discussion that we've had a number of times is, you know, is deliverability an art or a science? You know, how much interpretation is there, how much hard data is there? And I think the term that we coined on one of the previous engagements I was on was the dark arts of deliverability, we're sort of the dark arts of email.

00:18:16

Jonathan Torres: I like it. I mean, I'm even going to go as far as say, like I want to be more like Full Metal Alchemist, you know, kind of exploring some things. Maybe we shouldn't be exploring and trying to get something to happen. Yeah, that shouldn't be happening. I don't know. We'll see. We'll travel down that rabbit hole some other time. And with that, whenever we start looking at that kind of stuff and we start, you know, creating that division line, like how for me and this is something that I know I struggle with and I've kind of gone and gone back and forth with the way that I want to consider it, and that's when is too much cutting too much. Whenever I cross that line, What am I doing? You know, am I hurting myself by doing too many of these. Can I get to way too many, I guess is the right question there. Like like what do you guys see? Like, what would you guys consider? Like, the more I cut in, the more I prune mean that I get better deliverability or is just is there a point of like diminishing returns where it really doesn't matter anymore?

00:19:07

Brad Gurley: For me, it's been a bit of an evolution. You know, earlier on in my career in deliverability, I was a little more aggressive with some of those situations where, you know, there was not a lot of leeway to be had there. It's like, okay, there's a hard and fast rule where you have to cut X of your list or you have to completely wipe your list out and start over. And as the industry has changed and I've I've kind of grown in the industry, I've seen that there's a lot better balance that you can do between business needs and making sure that you're complying with email consent, you know, all the requirements that are on the other end. So there's absolutely a balance there. How do you know if you haven't done enough Spamhaus will tell you. How do you know if you've done too much? That's where, you know, it becomes a little more of a science or art, if you will. We try to balance that looking at incremental increases there. So if we see that, okay, you know, we're hitting traps and we're listed some. Where we're going to start with kind of the most risky, if you will, type addresses, will look at anybody who hasn't engaged or anybody who may have unclear permission status or maybe there is some question of how long have they been on the list without you know, without actually purchasing or engaging with the site. So we try to start in an incremental basis and say, okay, let's get rid of the most risky recipients first and then sort of follow up from there. So if we are dealing with a, you know, we're going to pick on Spamhaus because they're they're kind of the most popular. But if we do have a Spamhaus issue, we're going to go to try to address the things that we think are most glaring and then reach out to Spamhaus, try to get some feedback. Okay. Are you still seeing the issues? And if so, then we continue to increment from there. So we're not just doing one big hack and slash job on the list and kind of getting rid of all these people at once. Yeah.

00:20:46

Nick Schafer: So I was going to say it's kind of situational right? It's dependent on your standing as a sender. Like if you're doing well, you're not seeing any issues. You could probably be a little more flexible if you're seeing issues with, you know, blocklist operators or specific mailbox providers, you should probably take a look at, you know, the things that you're doing and maybe be a little more aggressive in pruning your list or, you know, whenever you're trying to decide which recipients you want to remove. Just be a little more aggressive in those decisions. Things are going well. You have more flexibility. It's not just you should do this or you should do that. I think it definitely kind of changes over time.

00:21:24

Eric Trinidad: Getting ahead of that, you know, being proactive with some of that part of the deliverability topics that we discussed, you know, getting, you know, sunset policies in place before you start having issues. You know, it's always good just, you know, before to get a handle on things, you know, just to see kind of where things are going. Go ahead, JT. I had a thought, and now it's gone.

00:21:41

Jonathan Torres: I'm sorry. It was my fault. It was my fault. The topics that we've talked about over the length of the podcast and, you know, kind of like leading into the season, I always tried to like, link it back to things that we've talked about because it all is so connected. But whenever you start looking at your reputation and you're moving through things like data, one, you got to keep that data. You got to have that data on hand because if you don't or haven't looked at it, you haven't collected it, then it's not going to be useful to you going forward because let's say you are in a bad spot and you have to prune, you have to cut. You have to be very critical of all those things that are happening. When you start getting errors and you start seeing bad results and you slowly go on improving or you do improve, then you need that data back to say, Hey, like I prune this person because, hey, the mailbox was full and I got rid of a bunch of those because like that seemed to be, you know, an easy win just to get, you know, remove some volume. The people that weren't engaging aren't doing stuff. So maybe I can use those to slowly start building up again. Right. Reach out people that maybe didn't get my stuff because, you know, if they had mailbox full, maybe there was another temporary failure always getting at an ISP that, you know, I just remove them and you know, hey, like lets do a win back later on? But you got to have the data from before. And I love it .Like I love hearing it because I mean, from from on both like Brad and Nick said, it's, you know, you want to cut, you want to do things. You want to definitely improve. Like if you see bad results, you probably need to do more. But once you've done those things and start getting to a better spot, it doesn't mean those people are gone forever or that your list is now, you know, so low or that's it. There's always opportunity to bring back more. Not that you want to just bring back anybody overall, you know, you just can't start sending to your full list again. But like use that data and use those data points to bring in the right people, the right audience, because it happens, man. Like, I know I've seen it so many times where they drop people like they prune out and then, you know, they start bringing people back and then they start getting great engagement from those people that they pruned out before. But it's because they have better reputation overall now. And those people that maybe weren't seeing those messages that weren't getting the info like now they are and yeah, they do want to participate and be a part of it. They just weren't seeing it before. So it's a little wrap on that portion of it.

00:23:44

Brad Gurley: Just to to illustrate your point a little, actually, we've been working with a pretty well-known retailer this year, and they were having some issues with one of the big providers earlier in the year. They had to do some pretty drastic cuts to their list. And they've been able, you know, throughout the year to reinstate so many of those recipients because they improved their reputation, they improved their standing with the provider, and they've now, you know, kind of gotten back to almost where they were before they started having the issues. From a reach standpoint, they're obviously adding new subscribers on top of that. So, yeah, just to give an example that, you know, we're definitely seeing that happen quite a bit.

00:24:18

Nick Schafer: I love that point that you guys both just made right there. And it brings me back to my technical account manager days whenever I had the clients that Eric you're speaking of just a second ago, that was one thing they were always kind of pushing back against. I don't want to ruin my list or I don't want to reduce it by half. But that was one of the points is like, this is all based on the reputation of you as a sender. If we can get your reputation back to where it needs to be in the mailbox, providers want to let your traffic through to their users. You can, you know, go back into those lists, get them back going. Now, something that was hard for senders to realize until, of course, you know, they had hard data and, you know, the results to prove.

00:24:57

Jonathan Torres: That hard data is hard to come by. It's like what are those things where it's like everybody can show you know, either, you know, particular use cases or case studies that have been done and on industry. And they're always want to know like, what is my industry done before? And you show them a few things, but it's like until you see it with your own list, with what you're doing yourself, like it's just so it's definitely like difficult to prove. So I mean, that's just my plea for everybody. Like, please believe us when we're saying this stuff. Like if you have a bunch of people here, different experience, like coming from a bunch of different spots and like, we see the same results because like, we know what we're talking about and we've seen it happen over and over again, it's like the proof is there. It's just a matter of getting there, allowing that to happen and allowing those changes to do what they need to do. And I know everybody expects like changes overnight or from one send to another, you know, even a month. You know, it's sometimes not enough time to give it a chance to work and it does work. So I'm just gonna throw that out there. Thats just my little plea for the day. When it comes down to it I know what brought along this topic, and I know we've picked on them a lot already, but the Spamhaus info listings. So whenever we're like trying to put this all together, like what's the best way, I guess, to consider whenever we are already in trouble with Spamhaus? Like where do we start with this?

00:26:06

Brad Gurley: I listen to your podcast with Spamhaus before because I was trying to find those answers and you guys didn't give them to me, so I was a little disappointed.

00:26:13

Jonathan Torres: Sorry, sorry about that.

00:26:14

Eric Trinidad: Didn't we give out Matt's email? I'm pretty sure we did.

00:26:17

Jonathan Torres: But it was his personal phone number, was it not?

00:26:20

Eric Trinidad: Yeah.

00:26:22

Brad Gurley: Yeah, for sure though I kid. But there are certainly some things to keep in mind when you're working with any blocklist, but particularly with Spamhaus and particularly with some of the changes that they made last year to maybe clean up some areas that they thought that they can use some improvement in the way that they were filtering mail and or again, they're not filtering the mail themselves, but the way that they were tracking data and listing IPs and senders, there. One of the biggest things for us was not looking just at bounces because we've talked so much about bounces here, but the side of it that we haven't talked about is, you know, has anything actually gotten through successfully to that recipient? And that's one of the areas that we were a little bit surprised when we looked at some of the data that, you know, we saw bounces, but they were maybe very generic bounces or ones that indicated that, hey, maybe we should continue sending. It was, it didn't seem like it was like, hey, this address doesn't exist or it's bad or it's it was maybe something a little more esoteric, if you will, something that we assumed was going to be transient, kind of disappear after a short time. But what we saw is that they would continue seeing those bounces, sometimes even different types of bounces. They weren't even always the same fonts, but they never got anything successfully delivered. So that was a key metric. That seems really simple when you think about it. It's like, Oh, if it didn't bounce or if it bounced and it didn't deliver. But you don't really put that together when you looking at your data, you just look at bounces a lot of times and you don't look at, okay, well this is bounce ten times in the last six months, but it has zero successful deliveries. And that's really what came to be as important, if not more important as we looked at the data.

00:27:57

Nick Schafer: Yeah, it's really important to actually look at what the error messages are saying. If you're looking at the data from just like an overview level, sure, you're going to see bounces, You're going to see that that number increase, but it doesn't tell the whole story. You have to, you know, get in there, dig into the data and dig into the error messages and error codes and really kind of find those patterns. You know, I've done similar things. It may sound like we're picking on Spamhaus a little bit, but it's actually, I think in my opinion, a respect thing. We all respect them and we know they mean business and you don't want to end up there. So that's why we do some of that stuff. They clearly tell us in E-books that they've shared or just in conversations that some of us have had is pay attention to those bounces, like read them like they're literally saying read error messages. And so whenever I hear something like that, I'm going to go do that, do the investigation. And to Brad's point, there's a lot of interesting things that you can find. It's like, okay, we never delivered to this address ever. Yeah, that bounce message may not be clear in saying like, this is user unknown or something like that, but at what point do you say, All right, we shouldn't be sending to this address anymore? So you've got to take a step back and look at it from that view.

00:29:10

Eric Trinidad: Yeah, let's stop calling them bounces or failures. Let's just look at non-delivery events.

00:29:17

Nick Schafer: I love it.

00:29:19

Brad Gurley: Yeah, I like it. I like it.

00:29:20

Eric Trinidad: All right, everybody, Good game. That's it. That's my one.

00:29:24

Brad Gurley: We've fixed, we've solved it.

00:29:26

Eric Trinidad: Yeah.

00:29:29

Jonathan Torres: Oh, man, that's good. Yeah. Yeah, I like that. You know, paying attention to it. Paying attention, paying attention to the data. And I know, like, man, it's one of those things that like reinforces again, just to kind of wrap up into other things that we've talked about sunset policies and just you got to pay attention to that stuff. And when you have the data to prove it, like great, but that's even better.

00:29:46

Nick Schafer: Sunset policies are so much more difficult now though, with things like MPP, like can you trust the engagement? Like, that's why this was actually something from, you know, going back to previous podcast which by now I hope everyone listening to this one goes back and listens to all of them. Something that Matt had even mentioned. He was saying something with regarding to sunset policies. Oh, this is it. So looking at the engagement, you know, you can see engagement from possibly some traps because that's the way a blocklist operators work these days. They need to look at content. He made a point where it was like you need to look at engagement outside of just the email, like, what are these users doing? Are they making purchases with your brand? Are they doing something on the website? I think that's where sometimes marketers fail to go look at and they're just looking at like the email engagement. You got to look at it from a holistic view.

00:30:36

Brad Gurley: Yeah, and I think to that point, you know, bring it back to what I just mentioned as well is that we did notice that, you know, there were situations where we saw zero deliveries for, you know, let's say we've never delivered to this dress, but it's recorded engagement. And that's what the sender was looking at. That's what they were actually using to filter, but they weren't looking at did we actually even successfully deliver the message. So that's one really important thing to keep in mind when you're doing those sunsetting policies, when you're doing those engagement tracking, you're including the engagement, but you're also including, you know, a successful delivery or it didn't bounce because like if you have a bounce and an open well in this case, generally the bounce is more prevalent, you know, and that's what you really want to look at there. You want to make sure that the mail is getting delivered and not just that something is being recorded, we know with mail, privacy protection, with different proxies that, you know, opens can happen by automated fashion. They can also happen to Nick's point, you know, intentionally that, you know, blocklist operators may need to see the content. So just because there's an open there doesn't mean that it's not a bad address.

00:31:35

Eric Trinidad: Moral of the story. Look at all the data, not just what's in front of you, but what are your customers doing, What are they engaging with? What are they donating or purchasing? Are they going to the brick and mortar stores? At the end of the day, you know, if they're open, you know, if they have them, they're available. But yeah, yeah. Thank you, guys. I really do appreciate I know Jonathan, Thomas and I. I'll speak for you as well Thomas, really appreciate you all coming out and joining us today. Brad, if they want to look for some information or hit you up in real life, where can they do so.

00:32:13

Brad Gurley: Absoloutley, well I am still on Twitter for now anyway, @deliverycounts. Also, as you mentioned, I work for MessageGears, so I do some posts on the MessageGears blog as well. I do have a website deliverycounts.com. Also, if you want to go check that out, but feel free, reach out. Also email me brad@deliverycounts.com. If you have questions or concerns or anything to chat about.

00:32:35

Eric Trinidad: Right. Nick for you as well, sir, if anybody has questions or concerns or if they want to look at anything you posted recently.

00:32:41

Nick Schafer: I am still on Twitter as well. I think it's @NickDSchaefer. You got to remember that d. It stands for my middle name, but you can also find me on LinkedIn and send me a message there. I'm always looking for new connections. I think everyone is these days.

00:32:55

Eric Trinidad: Let's make those connections. Speaking of connections, Thomas, where can anybody else find our information if they're looking to listen to the previous podcast that we mentioned today?

00:33:04

Thomas Knierien: Well, today Eric we've got a brand new car, no, im just kidding.

00:33:06

Eric Trinidad: New Elante!

00:33:11

Thomas Knierien: Come on down. You can actually listen to our podcast at Mailgun.com/resources/podcast. And you can also find the previous episode where we're talking about with our buddy Matt Stith over at Spamhaus that was about two episodes ago on the season. So be able to check it out. We're also getting ready to wrap up the season, so it's coming to an end yall, we hope you guys enjoyed all of our guests. Yeah, we'll see you soon.

00:33:38

Eric Trinidad: Right on, thanks again, everyone. Have a great one.