Weekly product update: improved notifications for email limits

Email notifications now alert you before your limit is reached

If you are on a Mailgun plan that has an explicit sending limit, we’ve always sent you an email notification once your email limit has been reached. However, that doesn’t give you much time to plan. So starting this week, we are also sending notifications once we see that you are getting close to your limit. Specifically,

• Customers on our free plan receive notifications once they’ve reached 75% of their daily limit

• All other customers on limited plans (like Heroku or EngineYard) receive notifications at 75% and 95% of their monthly limit

It’s a small improvement, but we hope that it helps you plan better for world-domination.

If you’re interested in how we built notifications for reliablity and scale, this next part’s for you

Sending a notification once a customer has reached 75% of their sending limit seems easy, and conceptually it is. What was challenging however was turing this into a reliable, scalable systems that functions well for thousands of customers sending millions of emails a day. When we first looked at the problem we took the obvious approach: add a field to our MongoDB-based customer database and increment the counter for each email sent.

But when we realized the load that this would put on our MongoDB (which doesn’t function well at such a high number of writes/per/second), we knew we needed to take a different approach.

An actor-based system decouples processes for better performance

Instead of complicating our main database which we like to write to only when absolutely necessary, we implemented an actor-based system which decouples most events from our main database. By decoupling events into different specialized systems, Mailgun is much more efficient, easier to code and more resilient. There are many reasons for this, but one is that different processes can run on different machines meaning that if there is a problem with one process, it doesn’t affect another. How does this work?

Mailgun is basically just a series of events. Delivers. Bounces. Opens. Clicks. When any of these events happen, we record it. Then, different processes can be set up to do something with those events. For instance, when a “delivered” event happens, we have a process that can notify a customer by a web hook .

Or, in the case of email limit notifications, it works like this:

1. Mailgun delivers and receives emails which generates events.

2. Each of these events is sent to Redis, a key-value store that is great for building event queues.

3. We then have a separate process–we call it Watchdog– that only listens to these events and calculates things like whether or not the customer has reached their plan’s email limit.

4. We then have another process that actually sends out the message or in cases where the customer has exceeded their limit, disables the account.

Each process does a single thing very well. There are few moving parts, and its easy to scale. Profit!

Well, that’s it for this week. We hope you like the new event notifications and learned a little about the way we develop at Mailgun.

Happy emailing,

The Mailgunners