Still Not Tracking Exceptions in Rails

What is “Exception Notification”?

If you’re not a developer, just the phrase “Exception Notification” probably makes your eyes glaze over. But if your business depends on a web application, it’s worth it to muscle past the jargon, because exceptions are generally the kinds of errors that hit your users with an error page before anyone on the development team even knows there’s a problem.

Imagine two scenarios: first, users are trying to place orders on a website the day after a new feature is released. Some users get an error message and can’t place their orders; eventually, a couple users send emails to customer support to complain, and customer support notifies the development team. Now consider a second scenario: the development team receives a detailed message about the error that users are seeing, including the specific section of code that’s breaking. The development team repairs the problem and releases a patch just as the first user is getting around to contacting customer support, so the support team responds to the user that the problem is already repaired and asks them to try again. That second scenario is possible because of exception notifications.

“Exception” is a technical term for errors that break an application. They’re often the result of either a bug in the code or some kind of unexpected user input. Exceptions can be “handled,” so that the system knows what alternative path to go down if the application breaks in a certain way. But it’s impossible to safeguard against all possible errors, so even in a strong codebase, occasionally an unhandled exception throws a wrench in the works.

In a website or web application, an unhandled exception usually cause the user to see some version of an error page, whether it be the default Rails error page...

...or something a little more interesting:

Showing the user an error page is a necessary evil. But you certainly don’t want your users to be seeing errors all over the place without the developers knowing about it. A vigilant team can often repair exceptions before users even have a chance to report the complaint, or at least have a head start on repairs.

Gem vs. Paid Services

That brings us to one of our Rails Survey questions.

Rails has an ExceptionNotifier plugin that lets your web application send emails to the development team when there’s an unhandled exception. What our survey results show is that use of ExceptionNotifier has been steadily dropping off since our first survey in 2009.

The obvious reason for this decline is the rise of paid exception notification services such as Airbrake and BugSnag. Why pay for a notification service when you can use a freely available gem like Exception Notifier? The paid services improve the signal to noise ratio by providing more nuanced control over who receives notifications and under what conditions. They allow exception notifications to be snoozed or assigned to a particular person to resolve. They also simplify integration with other tools, by handling tasks like automatically creating bug tickets for new exceptions.

Nothing? Really?

What’s remarkable is the declining but still significant percentage of respondents (down to 16% this year, from a high of 31% in 2009) who don’t use any exception notification at all on their projects.

Why turn a deaf ear to an important part of your application’s health?

Some developers might be depending on exception logs (where the error messages are stored in a database or file) instead of active exception notification. As long as the team is consistently paying attention, this approach can work, although it still has the pitfall of requiring constant attention.

A more dangerous reason to neglect exception notification is that in new or unstable applications, listening to exception notifications can be a noisy proposition, and the sheer volume can make it difficult to triage and resolve problems. Especially for developers who lack the budget for a paid notification service to help filter out low-risk errors, it may be tempting to turn down the volume entirely so that the team is swamped by notifications. This is tempting since many of those exceptions may not be critical.

It’s that phrase “may not be” that’s dangerous. Any application that’s throwing a lot of exceptions is at risk, and it’s better to clean things up now than to wait for a real crisis to hit.

Putting a piece of tape over the “check engine light” in your car isn’t going to do anyone any good.

Takeaways

If you’re a developer and you don’t have an intentional strategy for handling exceptions — and obviously, you’re out there! — it’s time to get in gear. Whether it’s a free gem that you dial in to your needs, or a paid service that makes things easier, the Rails ecosystem has a workable solution for you.

If you’re a stakeholder, consider taking a few minutes to talk to your development team about their strategy for handling exceptions and other problems. Get curious about the process. You might find that there’s room for improvement... or you might find that your team has been solving problems before anyone noticed them, and you can delight your developers by recognizing the hard work they do that’s usually invisible.