Jun 01, 2002
Introduction
One of the many reasons for writing your own software is that you discover how to do simple tasks that would otherwise require complicated, sometimes expensive, and usually restrictive software. Sending email is a case in point. Sending email is one of the simplest things you can do. And yet the basic mechanics of it continue to perplex and confound.
Now of course anyone can send email to an individual or even a list of people using a common email client. It is possible even to use common email clients to send fancy HTML-based email. But beyond this, your capacity is limited. Often, HTML-based email won't work on the receiver's email client. And, of course, nobody has solved the problem of how to send email with long URLs without splitting them.
If you run a website, the problem is even more perplexing. Your website cannot access your email client. Sending HTML-based email is not straightforward. Usually people sending email from a web server opt for a simple and ugly solution, resulting in badly designed mailing lists. But sending email automatically is simple and straightforward and should not keep web designers up at night.
The purpose of this article, therefore, is two-fold. On the one hand, for developers, it provides some quick and easy tools to add email to a website. And on the other hand, for site managers and staff, it provides enough information so you can tell your developers that it can be done after all. Even if you have no need for this technology, you may be interested in reading about the concepts behind sending email.
That said, I will restrict my comments in this article to sending email using Linux or Unix based systems. Of course, if you are running a production website you should be running on one of these operating systems in any case. But if you have opted for the Microsoft solution then you have by default opted for the expensive and restrictive software route: I can't help you.
The code samples in this article are all in a programming language called Perl [1]. While many developers use other languages, Perl has the advantage of being free and widely available. It is included with most Linux and Unix distributions. Even if you use other software to generate your website. Perl can still be used to send your email.
Sending Mail With Sendmail
Every Linux or Unix operating system should provide access to a program called sendmail. Some systems will use improved programs such as Postfix but these programs retain the sendmail interface, so the instructions below will still work. If you are a website administrator, Postfix is recommended because it caches sendmail requests, thereby minimizing their impact on other programs.
The first thing you need to do is to define what to send to sendmail. The data consists of two parts: the header, which contains things like the address the send the mail to, and the content, which is the email message itself. Sendmail accepts a series of parameters in the header, each in the following format:
Parameter name : parameter value
The full colon separates the name of the parameter from the value, while the line feed character ( ) designates the end of the value. The entire list of parameters is concluded with a double line feed character ( ).
The parameters are exactly what you would expect: to, from, and subject (other parameters exist, but these are the minimum required to send an email. In Perl, you can define them like this:
| $mailstring = | "To: stephen@downes.ca
" . "From: Stephen.Downes@nrc.ca " . "Subject:text " . "This is the body of my email message."; |
When you send input to sendmail, the program will, as the name implies, send the input as email to the specified destination. Sending input to sendmail is like printing output to a printer or to a disk file, but instead of printing to a device, you are printing to a program. Typically, you would need to open the program, print to it, and then close the program. In Perl, we do it like this (the comments on the right hand side are comments, not part of the program):
| $mailprog = "/usr/bin/sendmail"; | # Location of sendmail; may vary on your system |
| open (MAIL,"|$Mailprog -t") | # Opens the email program |
| or print "Can't find email program $Mailprog"; | # Gives you an error if this fails |
| print MAIL $mailstring; | # Print the data (see above) to sendmail |
| close MAIL; | # Close sendmail |
When you print the data contained in $mailstring to sendmail, sendmail will send your data as an email message to the recipient specified by the To: parameter.
Email Spoofing
What you have seen so far explains a common web phenomenon, email spoofing. In the description above, notice that you can tell sendmail who the email is from (in the From: field). Sendmail does not perform any honesty tests: it believes you are whomever was identified in this field. This allows certain unscrupulous spammers and virus senders to spoof n email address, that is, to send a fake email address through the From: field.
Because sendmail can accept email addresses in various formats, the fact that an email message has been spoofed may be disguised. For example, if you defined the From: field as follows:
"From: Stephen Downes
most email clients will display only the name 'Stephen Downes' and leave the email address obscured. This will fool most people and has been the basis for more than one prank on the web. In order to verify that an email has really been sent by someone, you must look at the extended headers using your email client.
Miscellaneous Tips
When you write your own email software, you get the weirdest errors. Here are four items that have come up a lot over the years:
First, Perl sometimes rejects email addresses. This is because the @ character used in an email address is considered a special character. If Perl rejects your email addresses, place a backslash in front of the @ character to indicate that it is not to be interpreted as a special character. Like this: stephen@downes.ca
Second, sendmail interprets a single period on a line by itself as a command to stop processing immediately. Not only does this shut down sendmail, it shuts down the associated Perl program. For this reason, you insert the following line into your program:
$mailstring =~ s/\n.\n/\n\n/mig;
This will remove any such dots.
Third, many email clients are now rejecting email that contains Javascript. This is because of the (generally false) apprehension that Javascript can be used to send viruses. For this reason I recommend that you not include any Javascript in your email at all.
Finally, fourth, many email clients impose a line length limit on email (the most common is 1024 characters). For the most part this line length limit remains invisible because most email clients impose line lengths on sent mail (this is why your URLs get clipped). Sendmail does not impose line lengths on your outgoing email, which gets you around the URL clipping problem, but it leaves you open to having your email rejected because the lines are too long.
Sending HTML Email
In one sense, sending HTML email is deliciously simply: simply insert HTML coding in the body of your message. For example, the following will produce nice HTML formatting:
| $mailstring = | "To: stephen@downes.ca
" . "From: Stephen.Downes@nrc.ca " . "Subject:text " . " Thisis the body of my email message."; |
Because you have inserted the header h1 command, the word 'This' will be displayed in header format, just as it would be in a web page. Any HTML command may be sent as part of the body of the email.
The real trick is in making it display correctly at the receiving end.
Some email clients will recognize the presence of HTML formatting in the email message and display it correctly. Other clients, however, will not. They need to be told that the message text is in HTML format. This is done by placing a mime-type declaration into the email header. Like this:
| $mailstring = | "To: stephen@downes.ca
" . "From: Stephen.Downes@nrc.ca " . "MIME-Version: 1.0 " . "Content-Type: text/html; charset=us-ascii " . "From: Stephen.Downes@nrc.ca " . "Subject:text " . " Thisis the body of my email message."; |
Now you have placed information in the header that will tell the email client to treat this message as an HTML file. The vast majority of email clients will respond appropriately, displaying your email message as a neatly formatted HTML page.
It is important to note that some email clients will not display your HTML properly. These clients will either display partial HTML (older versions of Pegasus, common in network environments, will do this, for example) or will refuse to display anything. For this reason, you should either provide an alternative text based version of your email or, at the very least, ensure that your email uses a light coloured background to make sure the text will show up even if your table colours don't.
Additionally, many readers use web mail to read their email. Most web mail clients, such as HotMail and Yahoo Mail, will remove any HTML headers, printing only content between the
tags of your HTML. This has the effect of removing any Javascript or Cascading Style Sheet (CSS) information you may have placed in the header. For this reason, if you must use Javascript or CSS, you should place it in the body of your message.Web Bugs
You may have heard about the use of web bugs to track a reader's email (if not, have a look at the Web Bug FAQ - http://www.eff.org/Privacy/Marketing/web_bug.html ). Web bugs are made possible by the use of HTML in emails.
The offending email is created in HTML as described above. Since the email is in HTML, it can include images. The location of the image is defined just as in any HTML file, using an image tag. When you read your email, the image is downloaded from a website and displayed within your email, just as it would be if you were browsing a web page.
Contrary to what you might read elsewhere, not all images in your email are web bugs. What makes a web bug a web bug is that the location of the image is not an image file (such as a .gif or a .jpg) but is instead a CGI file, a script that will perform a function of some sort before sending the image to your email client.
There are many ways to make web bugs effective, but the easiest it to place some unique parameter (such as your email address) into the URL defining the image location. When the CGI receives this request, it stores the value of the parameter (your email address) in a database file and returns the image. In this way, the email sender can know that you actually read the email message instead of simply deleting it unread.
The only way to really avoid web bugs is to read your email offline (and who wants to do that?) or to delete email unread (which is why advertisers take such pains to do things like email spoofing).
Conclusion
In this article I described some basic techniques for sending email. By understanding how email is sent, you can be aware of some of the common tips and tricks used by web site developers when they send you email. You can also see how easy it is to avoid some of the common email pitfalls.
There are many ways to accomplish the techniques described in this article; I have merely adopted the simplest and most direct technique in order to illustrate the concept. The basic principles outlined here form the core for what can be a powerful set of tools, tools that can add significant functionality to your website. Whether you want to send a nice note when someone registers or notify a reader if someone has responded to a discussion list post, you can use these programming techniques to make that happen.
Next month I will describe some wider applications of this tool, including how to send web pages and how to provide text-only alternatives to your email.
Notes
- Perl purists will point out that it is much more common to use the CGI.pm library to send email using Perl, as described at http://www.perlfect.com/articles/sendmail.shtml - true, but doing it this way works just fine, eliminates the need to discuss Perl modules and makes explaining the email easier.
