Previous Issue | Search NetBITS | NetBITS Home Page | Next Issue
Tired of trying to explain to your co-workers why their Web browsers keep alerting them to the presence of baked goods? For this issue of NetBITS, we've cooked up a batch of information about Web cookies, those sometimes annoying, often confusing morsels of data used to store unique information about your Web explorations. We also cover email address verification, responses to Net legislation issues, and a brief tour of the United Kingdom phone system.
Contents:
Copyright 1997 TidBITS Electronic Publishing. All rights reserved. To subscribe to our weekly list, email <netbits-on@netbits.net>. Thanks to our sponsors for their financial support of NetBITS.
NetBITS Complexity -- We've received a few notes that NetBITS was starting to cause some readers to swoon from technical complexity. Our goal is to explain and offer practical information on topics that are normally brushed over or described incorrectly, and sometimes that requires technical background. Glossing over the details would be doing you a disservice. That said, we're still learning too, so we'll try to steer more carefully between being too technical and providing information that's so light as to be useless. [GF]
Told Your Friends Yet? For NetBITS to survive and prosper, we need to reach more people, and the best method of achieving that for a tiny organization like ours is by word of mouth. Our goal is for NetBITS to support itself financially by advertising, so the more subscribers we have, the better. If you find NetBITS useful, interesting, and informative, please spread the word that it's simple to subscribe: send email to <netbits-on@netbits.net>. That's all you need do. No commands, no funny syntax - just a single message to a single address. And, if you're either interested in advertising in NetBITS or interested in representing NetBITS to potential advertisers, drop us a note at <sponsors@netbits.net>.
Also, if you're interested in printing or posting a NetBITS article elsewhere, please see the blurb at the end of each issue for information. Generally, we're happy to give permission - but we do ask that you credit the author and provide links to NetBITS subscriptions and Web site. [GF]
AOL Revises Spam Controls -- According to a story on News.com, America Online has changed how users can control what email they receive. Prior to this, AOL's PreferredMail option allowed AOL subscribers to opt in or out of banning mail from a large list of domains and addresses that AOL constantly modified. Critics of this filtering process noted that some sites were being added incorrectly, mail server hijacking being a key reason for the mis-identification of a spam's origin. The new filter, called Mail Controls, enables users to specify their own banned or accepted domains and IP addresses, block attachments, choose to receive mail only from other AOL members, from other online services, or from a specific list of addresses or domains. Although the one-at-a-time interface makes it difficult to create and manage anything but the simplest mail filters, these controls have the finest granularity of any major service provider or commercial service, and will hopefully improve in the future. [GF]
<http://www.news.com/News/Item/0,4,15787,00.html>
8-bit or QP? -- David Knudsen <dknudsen@perham.k12.ispn.net> read last week's article in NetBITS-005 on MIME encoding and Quoted-Printable text, and noted that his Netscape mail client offers settings for both "Allow 8-bit" and "MIME Compliant (Quoted Printable)". He wondered why you would choose one rather than the other, given that both attempt to enable you to send 8-bit characters (which include non-standard characters, such as accented vowels and special symbols) in email. Will Mayall <mayall@fogcity.com>, who contributed last week's piece, replied:
In general, you want to stay away from sending 8-bit email unless you know that all the servers and gateways between you and the destination can manage 8-bit email. The one time when this is relatively safe is in an intranet where you know the mail server that is being used.
by Jeff Carlson <jeffc@netbits.net>
I've been on the Internet for a while, so it didn't surprise me when I retrieved email for the webmaster address of one of my clients and had a message waiting with shouting, capital letters.
"STOP ALL THE COOKIES!!! Email me when you take out some of them, and then I and everyone else I told not to got to this site might come back."
If you're not familiar with the term, a "cookie" or "magic cookie" is a short stream of text that a Web server can send to your Web browser. Once the text is received, your browser stores that information in a special file called "MagicCookie" or "Cookie" or "cookie.txt" (depending on your computer and software configuration - Microsoft Internet Explorer 4.0 for Windows, for example, stores cookie information as individual files in the "\Windows\Temporary Internet Files\" directory.
Exciting, isn't it? Isn't your anger just boiling from the knowledge that hundreds of other sites are sending and receiving information via your Web browser? The nerve! How dare they?
Of course, there's more to it than that - so much more, in fact, that the use of cookies has pushed many people to take sides as to whether cookies represent the Web's salvation or its speedy decline into the real-time-rendered fiery pits of digital hell. Sound a little more exciting now? Hold tight and read on.
The Path to Web Nirvana Is Paved with... Cookies? If you've been to a site that gives you the ability to customize the information on its main pages, most likely you've run across one or more cookies. Yahoo's personal news service My Yahoo is a good example: after choosing to create a new account by supplying a user name and a password, Yahoo's Web server adds some information to your cookie file that assigns an identification number representing your Web browser.
Here's how cookies work. During every Web transaction, your Web browser sends a small number of HTTP (HyperText Transfer Protocol) headers, identifying what kind of software it is, the version number of HTTP it understands (most know version 1.1), and what page or file it wants. The Web server to which you're connecting responds by sending back a number of headers, including one that defines what kind of document is being returned - an image in GIF format or text in HTML, for instance. The server then follows those headers with the actual data of the Web page.
To change the data in your cookie file, the Web server sends a Set-Cookie header as part of its transaction with the Web browser. When you set up a My Yahoo account, the page that is retrieved contains a line that might look something like this fabricated example.
Set-Cookie: id="343a432h"; expires="01-Dec-98 GMT"; path="/"; domain="my.yahoo.com"
The first part is the cookie name ("id") and its value ("343a542h"). The expires attribute defines when the cookie should be deleted from your cookie file. The path and domain indicate that the Web browser should retrieve only this cookie when the browser is accessing "my.yahoo.com," but it doesn't matter where on the site. If the cookie should be retrieved only when you're in the "/antimatter/" subdirectory, the Set-Cookie header would also include path="/antimatter/" in it. Alternately, if the domain was set to just ".yahoo.com", visiting any machine in the Yahoo domain would trigger the cookie being sent.
The next time you go to My Yahoo, your browser automatically sends a Cookie header to the server that contains your ID code. The Cookie header might look like this:
Cookie: id=343a432h
There can be as many of these cookie headers as there are cookies for the site. The server checks for your id number, retrieves your record, and then sends the customized page to your browser.
It's important to note that cookies are kept with a specific Web browser on a specific machine. If you use several different browsers on more than one machine, as I do, My Yahoo will not recognize you when you switch browsers or machines, because each browser uses its own cookie file. There's currently no built-in or simple way either to share your cookies among all the machines and browsers you work with, or to limit access to your cookies if multiple people use the same machine. (I'll mention some add-on solutions later, though.)
Another way cookies are used is the "shopping cart" metaphor used by many sites that allow electronic ordering and payment. Sites such as Music Boulevard and Amazon.com use cookies to assign ID numbers to identify return customers and remember which products they ordered (even if the customer leaves the site and returns days later). So, if you've chosen a CD or book - but aren't ready to buy it yet - you won't have to search for it again the next time you visit. It should just be there when you check your shopping cart.
A third method, which seems the most prevalent on the Web, is the use of cookies to track people's progress through a Web site.
You're a "Unique Visitor" Here -- Many tracking software packages, such as Microsoft Usage Analyst (part of Microsoft BackOffice), have a server-side module that causes the server to try to set a unique visitor ID when a user visits any page on a site - unless they already have one set on that visit or a previous one. (Some servers come with this ability built in, like the Apache Web server). Most Web servers can also be configured to record the cookies sent by a browser for a specific request to the site.
<http://www.microsoft.com/backofficeserver/>
<http://www.apache.org/>
This means that the log for the server contains the page request, the datestamp, and the unique visitor identity for each hit on the site. The logs grow huge, but they also enable site owners to identify the number of individual Web browsers - and by extension, the approximate number of different people - that have visited a site in a given period of time.
If the server can't set a unique visitor cookie - or the user rejects it through one of the methods described below - the tracking and analysis software must rely on visitors' IP addresses to determine uniqueness, which isn't as accurate in cases of multiple users sharing the same IP address (such as dynamic addressing and AOL gateways).
If I were to stop here, your impression of cookies would most likely be favorable: cookies add functionality, both for site producers (tracking) and visitors (personalized environments). But that still doesn't explain why I received the SCREAMING EMAIL!!!
Following the Crumbs Straight to Hell? People have problems with cookie use primarily in regard to privacy. It's already fairly easy to find someone's email address using address-grabbing robots in Usenet groups or even some free white-pages directories on the Web. What happens if any Web site you visit can grab your personal information?
Fortunately, there are ways to prevent such unauthorized intrusions. If a site's cookie behaves correctly, it won't write any personal information to your cookie file (such as a name or password to the site), or it will encrypt such information so that it can only be read by the server. Because a site is limited to 20 cookies each, which individually can use no more than 4,096 characters, there isn't room to store much sensitive information. (Netscape's cookie specification says that browsers should store a maximum of 300 cookies, then start deleting the oldest ones.)
In most cases, a cookie entry will contain nothing more than an ID number and a time stamp. To view the contents of your cookie file, open it with any text editor or word processor. You can often open it in your Web browser, which will display it as a text file. In the case of Microsoft Internet Explorer, the only way you can easily view cookies and their contents through the Preferences dialog box by selecting "Cookies."
Another method of ensuring privacy is that a cookie always specifies its originating domain name; if it's omitted, the machine that sent it is dropped in by default. If you've visited nefariouscookies.com, your browser will only reply to a request for that information from nefariouscookies.com. As noted above, a cookie can be set for an entire domain (like any machine that has yahoo.com at the end of it) or for just one machine (like my.yahoo.com).
What seems to make most people uncomfortable, however, is the capability of having their movements on the Web tracked using cookies. I've heard analogies made that being tracked by cookies is like having an employee of your favorite bookstore (for example) walk around and note everything that you've picked up or grabbed for purchase.
Some advertising management sites, like DoubleClick, have caused the hairs on the backs of some folks' necks to stand up by their use of cookies across many sites. Even though a cookie will only be fed back out by a browser to the domain or machine for which the Set-Cookie header originally set the domain to, DoubleClick circumvents this by feeding all ads for their clients from central machines at their domain. This means that a banner on a Web page can set a cookie for an ad management site, even though you're not visiting that site. (The means to do this is simple; in HTML, any IMG tag that references an image can point either at a local file or any image anywhere on the Internet.) Netscape Navigator 4.0 for Windows and Macintosh has implemented a solution to this, allowing cookies to be set only by the site that sent the HTML page itself, rather than any image sites.
<http://www.wired.com/news/news/business/story/2615.html>
In my view, tracking squeamishness is strictly a touchy-feely response, and is based mostly on one's own comfort level. Nearly all of the sites I run across that use cookies to track my whereabouts are sites whose creators I trust, and who are often able to tailor my experience to my interests. If I always purchase classical and alternative-rock CDs, why should they bother tossing the best-selling country CD in my face? Not many sites have developed this level of individual selectivity, but it's not that far off.
What Can You Do? In my opinion, the biggest problem with cookies right now is that they work in the shadows. Netscape Navigator and Microsoft Internet Explorer default to accept all cookies automatically, so you may not even realize all this is happening. The most recent versions of these browsers offer several configuration options, such as rejecting all cookies, or prompting to accept or decline each cookie. Microsoft added an option in just the Macintosh 4.0 preview release that allows you to approve all cookies for a given site when you receive the first cookie from that location.
At first, I configured my browser to ask me before accepting any cookie. This was good for curiosity's sake, but I quickly tired of sifting through the number of requests being made to my browser and shut off the option.
On my ever-expanding list of wishes for Web browsers (such as smaller RAM footprints and universal compatibility of HTML tags, for example), I've added the capability to be able to tell quickly and easily what a given cookie plans to do. In the meantime, you can choose among several different ways to block access to your cookie file entirely. You can delete your cookie file each time you use your computer. Or, you can lock your cookie file so that nothing can write to it. You can also use any of a dozen-plus freeware and shareware applications for Windows and Macintosh. (I've referenced the comprehensive pages at Cookie Central for these files. The first is for PCs; the second for Macs.)
<http://www.cookiecentral.com/files.htm>
<http://www.cookiecentral.com/macfiles.htm>
Cookies can be powerful and help add interactivity and functionality to sites that use them correctly. As with most of the emerging Internet technologies like JavaScript, Java, ActiveX, and others, there are some still some problems that are being worked out. Until then, you'll have to decide for yourself how comfortable you are with cookies. Personally, I don't mind them, although I'd like them better with chocolate chips.
[Jeff Carlson is the managing editor of NetBITS and TidBITS. This article is adapted from "The Cookie Monster," which originally appeared under his byline in the 15-Feb-97 issue of adobe.mag. Adapted with permission from Adobe Systems, Inc. NetBITS editor in chief Glenn Fleishman also contributed substantially to this article.]
Question: Are addresses checked when sending mail? Kurt M. Scudder <kurs@novo.dk> writes from Denmark wondering if mail programs - Eudora specifically - contact the recipient's computer when sending outgoing mail. "I can see this in the progress window. In fact, on some occasions, Eudora will freeze there, returning a message some minutes later saying that it couldn't contact the recipient. I thought that when I sent mail, it simply went to my SMTP host, and I didn't have to worry about it after that."
Answer: Kurt's right. Most Internet email programs send messages via an SMTP (Simple Mail Transfer Protocol) server that then connects with remote mail systems to deliver mail. The SMTP server queues mail - that is, receives it, puts it in line to go out, and then sends it.
The problem Kurt is noticing is that most SMTP mailers verify two things. First, they check to see that the full domain name of the recipient - the entire part following the @ in an email address - has a corresponding mail server. Second, they look to see if the username exists when it's an email address for someone that has a mailbox on the same mail server. Some servers do this "live" - as the mail is being sent from your mail program - and others - like Qualcomm's Eudora Internet Mail Server (EIMS) - do it after receiving the mail from you.
So, let's say you send mail to <editors@netbits.net>. Your mail program connects to the SMTP server that you specified in your preferences and tells it the destination address. That SMTP server then looks up whether a mail server is listed for netbits.net using DNS. (DNS or domain naming system is a distributed method of pairing a name - like netbits.net - with an Internet address or mail server.) If the SMTP server finds an appropriate mail server record for netbits.net, it then accepts the mail from you and sticks it in a queue because it now knows that it can send mail to netbits.net.
So the delay Kurt sees is when either the DNS server that holds the information for that particular domain is down or slow - or the connections over the Internet between Kurt's ISP and the remote system are sluggish. The standard Unix SMTP server, sendmail, won't accept email when it can't find the destination machine, while EIMS will accept it and later send an error message back to you.
The second problem described above occurs when you're sending mail to another user whose mailbox is on the same server. If, for instance, you mis-type the user's name or the mailbox has been deleted, you'll get an instant error telling you that no such user exists. [GF]
[Please send us any and all Internet questions at <faqtoids@netbits.net>, and include your full name and email address. Questions may be edited for content and length. We cannot guarantee publication or a reply.]
Forward My Calls -- Paul Durrant <pdurrant@durrant.demon.co.uk> responded to the FAQtoids 005 item on how ISPs have local dialing numbers outside their main network center. He noted that Demon, a large ISP in the UK, actually works with British Telecom to have the phone call itself forwarded. In the U.S., phone companies will forward calls but charge the recipient a significant amount per minute; however, in most other countries, consumers and businesses already pay a charge for every minute used even for local service - making it more profitable for the phone company to offer services that increase local calls. By having local numbers that are forwarded, Demon makes its 6,000-modem pool available to the entire country equally. Paul writes:
Demon now has four main phone numbers - two for modems and two for ISDN; that's one of each from two separate telecom suppliers to provide a bit of redundancy in case of problems. Local phone calls in the UK are subject to a per-minute charge. This does mean, however, that ISPs can offer unlimited access for a fixed monthly fee without getting into the sort of trouble that AOL did. Unfortunately, it also means that I pay far more to the phone companies than I do to Demon for my access.
Fear of an Encrypted Planet -- In response to Brady Johnson's first installment on computer crime in NetBITS-005, Edward Reid <edward@paleo.greensboro.fl.us> responds to a point on key escrow:
Key escrow arose in response to regulatory agencies' fear of being unable to decode private messages, especially encrypted telephone conversations. There's absolutely no issue of interest to the general public which key escrow addresses. In fact, the major related issue of public concern is privacy - which key escrow diminishes. Whether key escrow is a reasonable response to the regulatory agencies' concerns is a separate question.
Our Man in Botswana -- Phil Hudson <phil.hudson@iname.com> writes from Gaborone, Botswana, with thoughts on who legislation on computer crime really affects:
The increasingly important issue of computer crime needs to be debated with a better, clearer understanding of the main perpetrators, and that includes, unfortunately, nation-states, their governments, administrations, law enforcement agencies, and spies. The state may be part of the solution, with legislation (let's hope so) but it is already part of the problem - slightly less so in America, with its freedom of information legislation, but definitely in Europe and Asia. Would you trust the UK never to infringe your rights? You'd better - you have no way of checking on them and no comeback if they do abuse you. This is not a new idea for Americans - the Constitution is designed to provide the citizen with redress for all sorts of administrative wrongdoings - but a lot of people elsewhere in the world have never thought in those terms (or never have been allowed to).
[Please send letters to the editor to us at <letters@netbits.net>. Please include your full name and email address. Letters may be edited for content, grammar, and length. All letters become the property of TidBITS Electronic Publishing. We cannot guarantee publication or a reply.]
Non-profit, non-commercial publications and Web sites may reprint or link to articles if full credit is given. Others please contact us. We do not guarantee accuracy of articles. Caveat lector. Publication, product, and company names may be registered trademarks of their companies. NetBITS ISSN 1096-4908.
Previous Issue | Search NetBITS | NetBITS Home Page | Next Issue