Django transactional emails made easy

Vinay Pai — Fri, 22 Dec 2017 22:42:27 +0000

Transactional emails are an important way to keep users engaged when you're building a community site like Facteroid. It's importation to pay attention to details to keep users clicking on them and coming back.

Unfortunately, development work on email is a bit of a chore. You have to either clog your inbox with test emails to test every little change or at best configure another awkward solution to open them in a browser. The restrictive HTML requirements are painful to work with, especially the need to use inline styles.

I built some simple infrastructure to make email development workflow so much easier. I've built Facteroid in Django, but the basic ideas should apply to any platform.

The Basic Setup

Sending a formatted email means generating three strings: the subject, the HTML body, and the plain text body. The most obvious way to do this is to out each part in its own template (say welcome_email_subject.txt, welcome_email_plain.txt, welcome_email_html.txt) and using render_to_string. This can get a bit annoying to manage because you have three separate files to edit and keep in sync.

Why not leverage blocks? The django-render-block package is perfect for this. It gives you a render_block_to_string function that works just like render_to_string, but gives you the contents of a specific block. All you have to do now is define a single template with separate blocks for the three parts.

Let's encapsulate that in a class

class Email:
    template = "emails/base.html"
    from_email = settings.DEFAULT_FROM_EMAIL

    def __init__(self, ctx, to_email):
        self.to_email = to_email
        self.subject = render_block_to_string(self.template, 'subject', ctx)
        self.plain = render_block_to_string(self.template, 'plain', ctx)
        self.body = render_block_to_string(self.template, 'html', ctx)

    def send(self):
        send_mail(self.subject, self.plain, self.from_email, 
                [self.to_email],  html_message=self.html)

{% block subject %}Welcome to Facteroid{% endblock %}

{% block plain %}Hi,
This is an example email.

Thanks,
- The Facteroid Team
{% endblock %}

{% block html %}<html><body>
<p>Hi,</p>
<p>This is an example email.</p>
<p>Thanks,</p>
<p>The Facteroid Team</p>
</body></html>
{% endblock %}

All we have to do now is create a template that extends base.html, and override template in a child class of Email and we're on our way. Of course, the HTML and body for this is likely to be a lot more involved than that, with a lot of boilerplace. In fact, the standard way to structure HTML emails involves two nested tables with all the content going into a single cell of the innermost table. It would be pretty wasteful to have to keep repeating that. Of course, we can leverage template inheritance nicely to avoid that.

{% block subject %}{% endblock %}

{% block plain %}{% endblock %}

{% block html %}
<html>
  <head>
  </head>
  <body style="font-family: sans-serif; background: #f5f5f5;">
    <table cellspacing="0" cellpadding="0" border="0" width="100%">
     <tr>
       <td width="100%" align="center">
         <table cellspacing="0" cellpadding="0" border="0" width="480">
           <tr>
             <td style="background-color: #444444; padding: 18px;"><img alt="Facteroid" src="{{BASE_URL}}{% static 'img/header-logo.png' %}"></td>
           </tr>
           <tr>
             <td style="background-color: white; padding:18px 18px; font-family: Arial, Helvetica, sans-serif; font-size: 14px; line-height: 21px; color: #444444" width="480" >
               {% block html_main %}
               {% endblock %}
             </td>
           </tr>
           <tr>
             <td style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; line-height: 18px; color: #999999; padding: 12px 18px">
               <p>
                 You received this email because you signed up for an Facteroid account with this email. If you do not want to to hear from us, please reply to let us know.
               </p>
             </td>
           </tr>
         </table>
       </td>
     </tr>
   </table>
  </body>
</html>
{% endblock %}

You can still inherit from base.html, but you don't need to override all of html most of the time, you can just override the html_main block and write a much simpler document.

While we're DRYing things up, why not generate the plain body from the HTML one? We can always specify one manually for the most important emails, but the generated body should be good enough in most cases. HTML2Text to the rescue.

class Email:
    template = "emails/base.html"
    from_email = settings.DEFAULT_FROM_EMAIL

    def __init__(self, ctx, to_email):
        self.to_email = to_email
        self.subject = render_block_to_string(self.template, 'subject', ctx)
        self.plain = render_block_to_string(self.template, 'plain', ctx)
        self.body = render_block_to_string(self.template, 'html', ctx)

        if self.plain == "":
            h = HTML2Text()
            h.ignore_images = True
            h.ignore_emphasis = True
            h.ignore_tables = True
            self.plain = h.handle(self.html)

    def send(self):
        send_mail(self.subject, self.plain, self.from_email, 
                [self.to_email],  html_message=self.html)

It's pretty common for an email to take a User object and present some information from related objects. You also want to make it easy for users to unsubscribe from specific types of emails. Those are handled with boolean fields in the user Profile. Let's encapsulate that.

class UserEmail(Email):
    unsubscribe_field = None

    def __init__(self, ctx, user):
        if self.unsubscribe_field is None:
            raise ProgrammingError("Derived class must set unsubscribe_field")

        self.user = user

        ctx = { 
            'user': user, 
            'unsubscribe_link': user.profile.unsubscribe_link(self.unsubscribe_field)
            **ctx 
        }

        super().__init__(ctx, user.email)

    def send(self):
        if getattr(self.user.profile, self.unsubscribe_field):
            super().send()


class NotificationEmail(UserEmail):
    template = 'emails/notification_email.html'
    unsubscribe_field = 'notification_emails'

    def __init__(self, user):
        ctx = { 'notifications': user.notifications.filter(seen=False) }
        super().__init__(ctx, user)

Viewing emails in the browser

Now it's pretty easy to write a view to preview emails in a browser with very little effort.

# user_emails/views.py

@staff_member_required
def preview_email(request):
    email_types = {
        'email_confirmation': ConfirmationEmail,
        'notification': NotificationEmail,
        'welcome': WelcomeEmail,
    }
    email_type = request.GET.get('type')
    if email_type not in email_types:
        return HttpResponseBadRequest("Invalid email type")

    if 'user_id' in request.GET:
        user = get_object_or_404(User, request.GET['user_id'])
    else:
        user = request.user

    email = email_types[email_type](user)

    if request.GET.get('plain'):
        text = "Subject: %s\n\n%s" % (email.subject, email.plain)
        return HttpResponse(text, content_type='text/plain')
    else:
        # Insert a table with metadata like Subject, To etc. to top of body
        extra = render_to_string('user_email/metadata.html', {'email': email})
        soup = BeautifulSoup(email.html)
        soup.body.insert(0, BeautifulSoup(extra))

        return HttpResponse(soup.encode())

Now I can easily preview emails in the browser. I also use django-live-reload in development, so I can edit template files and code and see the effect instantly in the browser window without having to reload the page.

Keeping CSS Sane

Another thing that makes developing HTML emails painful is the need to use inline styles. Unlike normal webpages, you can't just have a style block and rely on user agents to render them properly. You really do have to put style="..." attributes on every element you want to style, which makes the simplest thing like "make all my links light blue and remove the underline" rather painful.

I made that easier with a little custom template tag that reads styles from an external file, and spits out a style attribute. I could have just defined styles in a dictionary, but with a little bit of help from cssutils can keep it in a .CSS file which makes it play nicely with code editors so I get the correct syntax highlighting, autocomplete etc.

# user_email/templatetags/email_tags.py

_styles = None


@register.simple_tag()
def style(names):
    global _styles
    if _styles is None or settings.DEBUG:
        _load_styles()

    style = ';'.join(_styles.get(name, '') for name in names.split())
    return mark_safe('style="%s"' % style)


def _load_styles():
    global _styles
    _styles = {}

    sheet = cssutils.parseFile(finders.find('user_email/email_styles.css'))
    for rule in sheet.cssRules:
        for selector in rule.selectorList:
            _styles[selector.selectorText] = rule.style.cssText

Now in my HTML files, all I need to do is this:

{% extends 'email/base.html' %}
{% load email_tags %}
{% block html_main %}
<p>Hello {{user.profile.display_name}},</p>
<p>Thanks for signing up! Here's an <a href="{{BASE_URL}}/about-us" {% style 'a' %}>article</a> to help you get started. </p>
<p {% style '.center' %}><a {% style='.primary-cta' %} href="{{BASE_URL}}/get-started">GET STARTED</a></p>
{% endblock %}

Conclusion

I stripped out some of the code from this example to make it simpler, but I have all kinds of additional functionality that can be easily added with this base.

The Email base class parses the email and automatically prepends the base URL to any relative links.
The send function records a log of sent emails in the database and calls an after_send method if it exists. The after_send function in some of the email classes do housekeeping tasks like record which notifications have already been sent to the user.
My view function is a bit more complex so I can preview emails that take more than just a user (but it builds on the same idea).

I'd welcome any comments, suggestions, or questions!

Explaining Net Neutrality to Non-Technical People

Vinay Pai — Tue, 12 Dec 2017 17:20:32 +0000

Net neutrality is in the news, but I think there's a lot of confusion about what net neutrality is in the first place. I decided to take a stab at making an analogy to help non-technical people understand the general idea of what it is and isn't without getting into the weeds about ISPs, traffic shaping, QoS, zero rating and such. I'd love to hear people's thoughts and suggestions.

The internet service provider operates a toll bridge. Under net neutrality rules, they would be free to set their own prices for crossing the bridge, they can offer an ‘unlimited’ bridge crossing plan that allows subscribers to cross the bridge as many times as they want for a fixed monthly fee. They can charge higher tolls during busy hours.

They can charge heavier vehicles higher fees to compensate for the higher cost of providing service (i.e. more wear and tear caused by heavier vehicles). For example, they can have a weigh station and set the fees for crossing the bridge based on the weight of the vehicle.