Previous Issue | Search NetBITS | NetBITS Home Page | Next Issue
If you have an email address, you're no doubt the unwilling recipient of floods of unsolicited commercial email. This week, our focus on dealing with spam email continues with a look at how Internet service providers can take measures to stem the unwelcome tide at the server level. We also discuss options for setting up a small-office ISDN connection and find time to explain how you can tick with the precision of an atomic clock using the Network Time Protocol.
Contents:
Copyright 1997 TidBITS Electronic Publishing. All rights reserved. To subscribe to our weekly list, email <netbits-on@netbits.net>. Thanks to our sponsors for their financial support of NetBITS.
Filter Feeders -- Following last week's Damn That Spam! article in NetBITS-003, a number of people wrote in with their systems for dealing with spam via filters. I still believe that filters are inherently doomed to fail because it's too easy to forge messages. There are some filters that work better than others; nonetheless, I always recommend caution.
Several readers wrote in to say that they filter out all messages that don't contain their email address in either the To or Cc lines to a special spam mailbox. Since your address won't appear in the To or Cc lines of messages from most mailing lists, make sure to filter all mailing list postings to another mailbox before filtering for possible spam. This technique wouldn't work for me (because it's not uncommon that I'll be legitimately blind carbon copied on a message), but it might be worth trying if no one ever sends you mail by putting your address in the Bcc line.
Jason Whong <jason@ambrosiasw.com> reports that a specific program that many spammers use has a bug that you can exploit in a filter. This program's forged headers use the timestamp of "-0600 (EST)". However, Jason notes that Eastern Standard Time is usually marked "-0500 (EST)" and Eastern Daylight Time is "-0400 (EDT)". Thus, filtering messages with the bad timestamp will catch either spam from this particular program or mail from incorrectly configured mail servers. I tried a search on my several hundred megabyte archive of stored mail and found that this technique worked well - it turned up only spam.
I received contradictory notes on the utility of changing your email address. One reader said that he was spammed immediately after changing, whereas another said that changing addresses had eliminated spam entirely. I suspect that changing your address and using it solely for individual mail (not Usenet postings or for listing on Web pages) would eliminate most spam to that address. Of course, changing your address can be a major pain and expense, so even in this case, you're paying for spam.
Some folks intentionally screw up their return address so automated address gathering programs can't use it. They then include information in each message on how to reply (usually by deleting or changing a character in the address). Although this method does help reduce spam, it's just another example of how spam shifts costs away from the spammers. In this case, the costs shift to those who must do extra work to reply to your messages. This technique also causes numerous headaches when you try to subscribe to mailing lists using the incorrect address. [ACE]
Left the Cake out in the Rain -- After using the metaphor of a seven-layer cake to describe TCP in FAQtoids 003, I had my face pushed into the creamy filling by several readers. TCP itself fits just about on a single layer of the International Organization on Standardization (ISO) seven-layer Open Systems Interconnection (OSI, just to be confusing) networking model. (The group's short name, ISO, is derived from the Greek word for "equal" - ISO is not an acronym.) The ISO standard is more of an ideal than a practical expression; the Internet itself only uses a four-layer model, which is generically called TCP/IP networking. The four-layer model describes the function of each layer; different protocols - exact methods of programmed interaction - exist for each layer. (Strangely, it's either four or five layers, depending on how you look at the last part of the process.) According to both models, these protocols should only need to talk to the layer above and below the one the protocol occupies, but reality is a little messier.
<http://www.iso.ch/infoe/intro.html>
<http://www.cis.ohio-state.edu/htbin/rfc/rfc1122.html>
The application layer describes how machines interact to accomplish a task, such as transferring files using File Transfer Protocol (FTP) or delivering mail using Simple Mail Transfer Protocol (SMTP). The application layer hands off to the transport layer, which is either TCP or UDP on the Internet. The transport layer protocols break up the application messages into pieces and dole those pieces out to the Internet layer. The Internet layer puts TCP and UDP packets in envelopes with the IP (Internet Protocol) addresses on them and then hands them off to the link layer. The link layer and physical layer are combined in the defining document, although they're different elements. The link layer knows how to put packets on the conveyor belt that takes them on their way; the physical layer is the conveyor belt. In the case of Ethernet, the functions are combined: the Internet layer passes off packets to the Ethernet driver, which both makes them ready and handles delivery on a LAN. With PPP, however, the PPP software on the host machine (whether a router or a modem connected directly to a computer) acts as the link layer while the physical device (the router or modem) moves the data over the wire.
If you'd like to read more about TCP/IP, try Troubleshooting TCP/IP - Analyzing the Protocols of the Internet by Mark A. Miller (ISBN 1-55851-450-3). OSI is well described in Andrew S. Tanenbaum's Computer Networks (ISBN 0-13-349945-6). Both are highly recommended by the TCP/IP newsgroup FAQ. [GF]
<http://www.amazon.com/exec/obidos/ISBN=1558514503/tidbitselectro00A/>
<http://www.amazon.com/exec/obidos/ISBN=0133499456/tidbitselectro00A/>
<http://www.cis.ohio-state.edu/hypertext/faq/usenet/internet/tcp-ip/resource-list/faq.html>
Those Who Repeat the Past -- We sent you back in time, instead of forward in FAQtoids 003. We said that the offset in mail headers from the zone formerly known as Greenwich Mean Time (GMT) is added to local time to get GMT or Universal Time (UT). The correct way to say it is that the offset is added to GMT to get local time. If you get mail that reads "05:00:00 -0700 (PDT)" in one of the headers, it means "add negative 0700 to the current GMT and you'll get 05:00:00 Pacific Daylight Time." At 5 A.M. PDT, it's noon GMT; adding negative 7 to noon gets you 5 A.M., the local time when the mail was sent. And don't get us started about springing forward or falling back. [GF]
by Glenn Fleishman <glenn@netbits.net>
Suppose you belong to a simple lakefront beach club, where you and some neighbors jointly contribute dues to maintain the beach and docks. Now suppose that, several times a day, a rapidly moving caravan of several thousand vehicles zoomed across the beach. "Hey!" they might shout as they zoom by. "We're not using your beach; we're just passing through!"
This is what unsolicited commercial email (UCE, or spam) does to the Internet service providers (ISPs) that form our Internet beach clubs. Spam threatens the enjoyment of our pockets (and packets) of serenity by subverting resources paid for by ISP subscribers.
Such subversion comes in two forms: spam sent directly to you, where you waste time and effort downloading and deleting it; and spam that passes through your ISP's mail server on its way to some other poor sucker. (The mail server is technically called a Mail Transfer Agent, or MTA, whether it's Unix sendmail or a QuickMail SMTP gateway.)
This second problem is less discussed. A spammer can send outgoing spam through any mail server they choose - unless the mail server is secured against this, and relatively few are. This hijacking of a mail server clogs it up and invites retribution from people receiving the spam who incorrectly identify the innocent mail server that delivered it as the source of the offending mail.
It's possible for server administrators to prevent relaying and dramatically reduce spam delivery, although they can't close the door entirely.
In NetBITS-003, Adam discussed how individuals can deal with spam, including deleting it, reporting it, and supporting legislation against it. This article instead concentrates on solutions at the mail server side, which can apply equally to small offices or big ISPs. You can suggest these alternatives to your ISP if you're getting too much spam - if they implement any or all of them, expect service to improve and spam to decrease.
What Mail Is All About -- Here's a quick lesson in how email works. When you send mail, a short set of transactions takes place between your mail client (like Eudora or Netscape Communicator) and an SMTP (Simple Mail Transfer Protocol) server, which accepts the mail and later delivers it to the recipient's SMTP server.
The SMTP transaction consists of three basic commands which tell the server who sent the mail and who should receive it, and then the body of the message. The MAIL FROM command has the return address which starts the process; the RCPT TO header has the recipient's address. A DATA header follows, after which the contents of the mail message are sent, ending with a single period on a line by itself. The mail headers you're probably familiar with from reading the tops of your messages - like Received:, Reply To:, and X-Sender: - are actually inserted by the mail server into the body of the email message. They don't have anything to do with the process of sending or receiving mail.
A simple transaction looks like this (the server responses are in square brackets):
[220 spaghettini.shlomo.com ESMTP Sendmail 1.0/1.0; Mon, 13 Oct 1997 10:06:19 -0700 (PDT)] HELO nebula.frogstar.boombox.com [250 spaghettini.shlomo.com Hello nebula.frogstar.boombox.com [192.0.0.1], pleased to meet you] MAIL FROM:<fredericks@hollywood.com> [250 <fredericks@hollywood.com>... Sender ok] RCPT TO:<glenn@fragmentarycomments.com> [250 <glenn@fragmentarycomments.com>... Recipient ok] DATA [354 Enter mail, end with "." on a line by itself] X-Hello: Hi there! Comments: I am not an authenticated user. Hope you eat your weight in ice cream today. . [250 KAA19936 Message accepted for delivery] QUIT
The reason it's so easy to spam people is that neither the MAIL FROM nor the RCPT TO headers are confirmed by a mail server beyond verifying (and even then only rarely) that the domains themselves exist. There's no login procedure for accounts that want to send mail. And, because the mail headers are just text the server inserts, they can be forged just by typing the text in after DATA.
So sending mail from <santaclaus@northpole.org> is a matter of connecting to the mail server, typing MAIL FROM:<santaclaus@northpole.org>, RCPT TO:<somebody@nowhere.com>, DATA, and then your text. Some mail headers can't be changed, notably the topmost Received: header, but there are ways to make it seem like mail originated from the forged address and then passed through other servers before coming to you.
That's the first part of the problem. The second is that MTAs actually "relay" mail between servers; they don't deliver it unless the address is "local." When you send mail from your local email client, the mail server first analyzes the RCPT TO address and does one of two things. If the address is local - a user who has a mailbox on that server - the MTA uses a Mail Delivery Agent (MDA) to add the mail to a mailbox. The mailbox is generally a file to which the MTA can append more information. This is true for Unix mailboxes (generally accessed via POP, or Post Office Protocol) and also proprietary systems like QuickMail, Microsoft Exchange, and cc:Mail.
If the address in RCPT TO isn't a local mailbox, the MTA acts just like a mail client and connects to another mail server that receives mail for the RCPT TO domain. It then repeats the transaction of MAIL FROM and RCPT TO. This process is relaying - your email client sends the message to your mail server, which in turn relays the message to another mail server that receives the recipient's mail. This usually involves only your mail server and the recipient's mail server. In rare cases, mail can go through several intermediate relays if the servers on either end are in large organizations or aren't connected to the Internet full-time.
Spammers subvert this process by using your mail server without permission. They specify your SMTP server as their outbound server and send mail to it; the SMTP server accepts the mail and then tries to deliver it. So, if a spammer can connect to the Internet at all, he or she can just appropriate time and space on any SMTP server.
Fortunately, users don't have to just sit back and be spammed. Let's take a look at possible solutions that you, your company, or your ISP can try.
Turn Off Relaying for Non-Local Users -- At least a million machines are running some kind of mail server or delivery agent, and sendmail V8 (also called 8.x) is the freeware Unix MTA program of choice. Some simple rules can be added to its configuration to prevent users who don't connect from previously specified addresses or domains from sending mail to users other than those to whom the mail server can deliver locally. In other words, your ISP can prevent anyone but local users from sending mail through its SMTP server.
Point your ISP to, or take a look at, the Web page below, which is a highly technical (but hopefully readable) counterpart to this article that I've aimed at those who implement these kinds of solutions. With an hour of work, any sendmail V8 configuration can be secured. There are also other resources which are referenced on this page.
<http://www.glenns.org/sendmail.antispam.html>
Systems running MTAs other than sendmail should have options to allow connections only from specific addresses as well. Often, this restriction is turned off by default. If the mail package your company or ISP uses doesn't allow this restriction, the ISP or your company might rethink the choice of MTA or lobby the vendor for this feature. Qualcomm's latest beta of Eudora Internet Mail Server for Macintosh and Eudora WorldMail Server for Windows NT 4 both support filtering, though the Mac server is currently more flexible. The popular commercial Post.Office mailserver from Software.com for Windows NT and Unix offers relay filters starting with version 3.1.
<http://www.eudora.com/betas/eims2.html>
<http://www.eudora.com/worldmail/>
<http://www.software.com/products/Post.Office/PO31-WhatsNew.html>
Removing relaying can cause problems, however. A few months ago, the large ISP EarthLink Network turned off relaying for users who weren't dialed into their network in an effort to reduce spam. However, many people use EarthLink for email while at work, and they were cut off from sending email via EarthLink's mail servers. Retrieving email still worked, because it's secure: you must use your password to retrieve your email through a standard POP server. Of course, these users could use their companies' SMTP mail servers, but many companies have policies restricting use.
The benefit of eliminating relaying is enormous, though. Any site that spammers employ to relay mail can suffer twofold: first, from the added and unexpected gush of incoming spam mail that could slow down or crash a mail server; and second, from retaliation - many spam victims look at the last relay as the source of the spam and react by sending, say, 1,000 huge files or other "email bombs" in an attempt to take down the mail server. I don't condone such retaliation, in large part because it's usually misdirected at innocent by-servers.
Netcom, another large ISP, in a recent reply from their Internet abuse department, told me that they were still experimenting with what they called a "spam trap" to stop relaying. When Netcom stopped allowing non-Netcom addresses to use their mail servers, the servers bounced four million messages per week. Netcom then tackled users who bought accounts from them and spammed using other people's servers; three million messages per week were captured through this method.
Build Site-wide Spam Filters -- Spam can be rejected from specific IP numbers, domains, and email addresses. AOL pioneered this technique, although it's hard to tell whether it works. The sendmail filters noted above enable the mail server to bounce mail back to its return address automatically (with or without a custom bounce message, like, "We hate spam") instead of delivering it to local users.
Adam and I disagree over the utility of this kind of filter slightly, as he notes in his article last issue. Spammers are growing more and more clever, and usually forge return addresses uniquely each time. But spam filters can easily exclude "legitimate" spammers, like Cyber Promotions, which doesn't make any attempt at disguise. It's still difficult to identify and delete mail from <do.not.reply@nowhere.site> because the next time they'll masquerade as someone else, but I believe it's still worth trying.
Filtering on IP numbers works better, because faked addresses are ignored. IP numbers can't be spoofed easily (especially in mail transactions) since they involve low-level protocols. However, spammers often set up slash-and-burn ISP accounts at major providers from which they can send spam before the ISP realizes and cancels the account. If you filter on an IP number range that belonged to a major ISP because of this, you could risk filtering out a vast quantity of legitimate email.
One particular sendmail switch only allows incoming mail from real domain names. However, not all sites on the Internet have domain name resolution set up correctly, and you can wind up losing mail from people without knowing it. If you ban them from sending you email, it becomes a Dilbert cartoon: how do they send you email telling you that?
Make Money Slow -- You can take back your time and energy and put it into other things - like enjoying the sun and surf from your newly quiet patch of Internet sand. Or maybe getting some work done.
[Glenn Fleishman has been using Internet email since 1988. He receives about 40 spams a day and manages to reject 35 of them without ever seeing their subject lines.]
Question: How Can I Set My Clock Automatically? A number of readers asked that I reveal how you can make sure your computer's system clock correct itself by checking an atomic clock over the Internet. Do note that if your clock is radically wrong and continues to be wrong after you've set it, your computer probably needs a new clock battery or similar fix.
Answer: Along with the vast number of other protocols on the Internet, the Network Time Protocol (NTP) is used to synchronize the time of a computer with another computer running an NTP server. NTP can provide accuracy to several milliseconds, although the farther away you are from the source of the time (atomic clocks and the Global Positioning System (GPS)), the less accurate your results will be. Of course, this lack of accuracy is meaningless for normal use - I doubt many people other than researchers care about millisecond accuracy.
To use NTP, you need an NTP client program. For the Mac, check out Network Time (which hasn't been updated for a long time but still works fine for me) and Vremya, among others. For Windows 95, try Tardis95 or AtomTime95, among numerous others. For a more complete list, plus programs for Unix, DOS, Windows 3.1, Windows NT, and Novell NetWare, check out the Time Synchronization Software Web page. There's also a Macintosh Time FAQ.
<http://www.eecis.udel.edu/~ntp/software.html>
<http://www.foresight.com/rich/ComputerClock.html>
Once you've downloaded and installed an NTP client, you must point it at an NTP server, ideally one operated by your Internet provider. If your ISP doesn't run an NTP server, you can use a public NTP server, but respect the fact that you're a guest on the server. In particular, if you don't need the highest accuracy, use one of the secondary servers (called stratum 2) NTP servers in the list in the Public NTP Time Servers page.
<http://www.eecis.udel.edu/~mills/ntp/servers.html>
For more information about NTP in general, check out the Time Server Web page and the Time Service Department of the U.S. Naval Observatory, which includes both historical information and information about future types of clocks. [ACE]
<http://www.eecis.udel.edu/~ntp/>
<http://tycho.usno.navy.mil/>
Question: Which ISDN Hardware Combination Works Best? I'm planning to upgrade my small office from using PPP via a modem to using ISDN connected to our small Ethernet network. One method includes buying a router/ISDN modem combination unit for $600 or more. I noticed that five-port routers for small Ethernet networks can be had for about $50, and I've seen ISDN modems by themselves for $200 or so. Can I do the separate purchase plan and save big, or am I missing some key element? -Peter Commons <commons@scruznet.com>
Answer: The five-port device you're describing is actually an Ethernet hub, which has different function than a router. In this equation, the ISDN device itself is almost incidental. Ethernet, as we discussed in NetBITS-001, Hey, I'm Talking to You, Part 1, is essentially one big electrical system. Every packet on a given Ethernet segment has to reach every machine on that segment. A $50 Ethernet hub is just a way to cross-connect machines on a network using four copper wires (called 10Base-T or twisted-pair Ethernet). For $50, you're not buying much intelligence; it's kind of like an extension cord.
A router, on the other hand, transfers traffic between different networks or protocols. For example, all the machines on a local area network (LAN) that need to reach the rest of the Internet know the address of the router in their TCP/IP configuration. When you send a request for a Web page from a remote server, for instance, your machine figures out it has to send the stream of packets making up the request to the router. The router has at least two interfaces - one interface connects to your LAN using the same kind of network protocol (like Ethernet). The other interface hooks up to the Internet via ISDN, a serial connection leading to a T1 line, or even a modem. The router takes the packets from the LAN interface, removes the LAN packaging, rewraps them in the right format for the outgoing protocol, and then sends them out through the Internet interface. The same process happens in reverse for incoming requests.
It's possible to buy cheap routers with serial interfaces and cheap ISDN serial devices. However, from personal experience, we don't recommend this. It's hard to get dissimilar devices to communicate properly with one another for a reliable connection. An integrated ISDN router includes everything you need in a package that's guaranteed to work with all the different parts it contains. ISDN routers come from Cisco Systems (their new 760 and 770), Farallon (the Netopia line), and many others. Some even include a built-in Ethernet hub, and the Farallon line has an AppleTalk option that lets you combine an Ethernet network, a LocalTalk network, and a link to the rest of the Internet using ISDN. [GF]
<http://www.cisco.com/warp/public/779/smoff.html>
<http://www.farallon.com/product/netopia/dialup/isdn_router.html#models>
[Please send us any and all Internet questions at <faqtoids@netbits.net>, and include your full name and email address. Questions may be edited for content and length. We cannot guarantee publication or a reply.]
Stop Your Glottis -- In response to our GIF pronunciation pronouncement in FAQtoids 003, Joe Clark <joeclark@interlog.com> wrote us a stern note about phonology that makes everything much clearer.
There is no such thing as a hard or soft g, no matter what your second grade teacher told you. The sound represented by g in the word gift is a voiced velar stop, while the sound represented by g in the word gentle (and GIF) is a voiced palato-alveolar affricate. Calling these phonemes hard or soft g is like telling your mom you just bought a computer with 20 MB of storage and 2 GB of memory.
Omissions -- Robert Morse <morse@northcoast.com> notes an omission from our list of excellent Web resources for developing sites and writing scripts in NetBITS-002:
Not taking umbrage, but adding a reference. I was surprised you didn't mention Web Review, one of my favorite sources for web development information.
[Please send letters to the editor to us at <letters@netbits.net>. Please include your full name and email address. Letters may be edited for content, grammar, and length. All letters become the property of TidBITS Electronic Publishing. We cannot guarantee publication or a reply.]
Non-profit, non-commercial publications and Web sites may reprint or link to articles if full credit is given. Others please contact us. We do not guarantee accuracy of articles. Caveat lector. Publication, product, and company names may be registered trademarks of their companies.
Previous Issue | Search NetBITS | NetBITS Home Page | Next Issue