Wednesday, October 19, 2011

Google Apps Split Delivery for Email - Have your cake and eat it too

Split delivery for e-mail is when you have a single piece of mail but want a copy of it sent to multiple destinations. There are a couple of reasons you would want to do this:

- You are getting ready to migrate to Google Apps but don't want to go all in yet with your current e-mail server. This is understandable since you want to test out Google Apps first to see if it will work.

- You like Google Apps but don't want to pay the yearly fee. You rather just stay under the limit of the free accounts but still want to have e-mail accounts on your domain (i.e. @example.com).

- You have other special circumstances where you want a copy of all the mail that comes in and have it delivered to some other server.


This post is actually more toward the 2nd point. 90% of my e-mails are on Google Apps but there is a remaining 10% that I rather not have a Google Apps account. However I still want them to have e-mail via some webmail client.


To get this working I am using Ubuntu with Postfix running the primary mail server for example.com. I set the MX records for example.com to this:
10 mail.example.com
20 ALT1.ASPMX.L.GOOGLE.COM

Pretty straight forward so far. The trick is that you need to get Postfix to forward a copy of all Google Apps mail to their mail servers. To do this I use Postfix's Before-Queue Content Filter: http://www.postfix.org/SMTPD_PROXY_README.html.

This allows me to create a SMTP server that Postfix will delivery a copy of the mail to Google Apps. In the file /etc/postfix/master.cf I put the following at the end:

# =============================================================
# service type private unpriv chroot wakeup maxproc command
# (yes) (yes) (yes) (never) (100)
# =============================================================
#
# Before-filter SMTP server. Receive mail from the network and
# pass it to the content filter on localhost port 10025.
#
smtp inet n - n - 20 smtpd
-o smtpd_proxy_filter=127.0.0.1:10025
-o smtpd_client_connection_count_limit=10


This makes Postfix delivery a copy of the incoming mail to the SMTP server at 127.0.0.1:10025.

Okay but now you are asking where do I get an SMTP server to do the processing? It so happens Python comes with a smtp server library. I wrote a script that basicallyhttp://www.blogger.com/img/blank.gif inherits the SMTPServer (called CustomSMTPServer). It also implements the one command that is expected by Postfix (EHLO) because Postfix actually speaks ESMTP. I did this by subclassing the smtpd.SMTPChannel class. One caveat is that I had to use the _SMTPChannel__variablename syntax because some variables like fqdn and greeting were made private by the SMTPChannel class. So with Python you use the special syntax of prepending the class name to access it. This is generally bad practice but in this case it was all I had.

You can download the script here:
http://dl.dropbox.com/u/2177278/pymailforwarder.py


Simply run the script to start a basic SMTP server listening on port 10025 and localhost. The script simply accepts a piece of mail and forwards it to Google App's mail server.

So what does this let you do? In my case this lets me run Roundcube, Horde, or SquirrelMail for users that don't need a Google Apps e-mail account. For those that do I simply create that user on Google Apps.

Things to watch out for with this type of deployment:

- You have to edit /etc/postfix/main.cf and add the value
local_recipient_maps =
(Yes that is a blank or equals nothing). This makes Postfix accept the mail even though the recipient is not in the list. This can be bad. Postfix says this in the documentation:

With this setting, the Postfix SMTP server will not reject mail with "User unknown in local recipient table". Don't do this on systems that receive mail directly from the Internet. With today's worms and viruses, Postfix will become a backscatter source: it accepts mail for non-existent recipients and then tries to return that mail as "undeliverable" to the often forged sender address.

To get around this you should really have an alias map file. For temporary testing though this setting will work wonders.

- The Python SMTP relay should not be exposed to the internet. It listens on localhost but ideally it would be nice to modify the script to accept authentication of some sort.

2 comments:

cornergraf said...

Thank you for this article.

I am in a situation where I need a setup similar to this, but I have a few questions.

1) I am assuming that your 2 MX server entry is just for redundancy, as it will only receive emails if the first one is not available. Is this correct?

2) I am assuming that the ASPMX.GOOGLE.COM address resolve to googles SMTP server. In that case, why are you setting postfix up to send emails to a local SMPT server which forwards it to google? Why do you not set it to forward to google directly?

3) I am not sure I understand the concept of the "before-filter SMTP server". How do you ensure local deliver and deliver to google at once?

4) In my setup, my SMPT server is the mail server for multiple domains. I only want emails for a specific domain to be forwarded to a google apps account. Do you know how this could be done?

Looking forward to your reply.

Matthew said...

Delivery and Delivering ability are the two sides of a coin that is internet marketing. Dissimilar in meaning and function, but they both directly depend on each other. If you focus only on delivery and not its ability, you cannot meet your goal of the email marketing campaign. A huge imbalance was noticed between them in the year 2011. In this year, only 81% of all emails was received by consumers' in-box. According to the report which was presented by The Global Email Ability of Delivery Benchmark, 7% was of all emails was SPAM, where 12% was missing. If the Inbox Delivery rate of the e-mails is the same, but profits are getting higher, its credit goes to the fluctuations of deliver ability, maybe reduced or improved.

Email Delivery