Previous Issue | Search NetBITS | NetBITS Home Page | Next Issue
Speed kills - but so does boredom. What can we do about decreasing the latency in modems to make our surfing sessions more productive? In this week's issue, Stuart Cheshire finishes his two-part series on latency by offering some solutions for the future. We also report on recent changes to the major Web browsers, explain why email file attachments are often difficult, and offer more options for multiple email accounts.
Contents:
Copyright 1998 TidBITS Electronic Publishing. All rights reserved. To subscribe to our weekly list, email <netbits-on@netbits.net>. Thanks to our sponsors for their financial support of NetBITS.
Search, and Ye Shall Find -- We've opened up our NetBITS search engine, so you should have no trouble finding old articles from now on. We've put a simple full-text search form on the NetBITS home page, and a more advanced search form is also available via the navigation bar. Note that if you wish to link to a specific article, there's a short URL specifically for that purpose at the bottom of every article served from the search engine. [ACE]
<http://www.netbits.net/search/>
ADSL Stands for "Alright! Dig that Service, Laddie" -- We received a number of terrific letters about our NetBITS Update last issue (see "And Who's Paying for This?" in NetBITS-015) about the coming standard for ADSL service being proposed by a group of computer companies, regional Bell operating companies (RBOCs or Baby Bells), and networking companies. The group has called itself the "Universal ADSL Working Group" and has a Web site, though with little hard information as yet. A better source of information is Aware, Inc.'s Web site, since Aware is the company whose DSL Lite technology is at the heart of the UAWG proposal.
<http://db.netbits.net/getbits.acgi?nbart=04656>
<http://www.uawg.org/>
<http://www.aware.com/>
We're planning a longer article on ADSL as currently deployed, along with some of the issues surrounding cable modems, in the 05-Feb-98 issue of NetBITS, at which point we'll run a number of the letters. The main point to be made is that, as frequently happens, Canada is ahead of the United States in rolling out service. [GF]
Browser Vendors Cave In, Millions Trapped -- In the latest bizarre turn of events in the browser wars between Microsoft and Netscape, Microsoft has agreed to the Justice Department's demand that Windows 95 be available without Internet Explorer bundled with it, while Netscape has made its browser free and promises to release source code for the upcoming Navigator 5.0.
Coincidentally, Microsoft released its fourth quarter 1997 earnings, showing that they netted over a billion dollars in that quarter alone. Netscape, around the same time, announced losses, laid off several hundred employees, and their stock price dropped to a level near its initial public offering.
What's the world coming to? A regrouping of forces, that's for sure. Netscape wasn't making most of its profit from browser sales. Abandoning all revenue from the browser market might enable them to compete on even ground with Microsoft in gaining the hearts and minds of corporations and PC makers. Prior to this, Netscape required site licenses for companies and per-machine fees for OEMs (original equipment manufacturers, such as Compaq, Dell, and Toshiba).
Microsoft, on the other hand, has now agreed to offer these same OEMs a version of Windows 95 that has all obvious instances of Internet Explorer removed. Internet Explorer will still be installed, but the user will have to ferret it out. OEMs can also choose a version of Windows 95 with the majority of Internet Explorer files and resources removed and that will boot, but it will lack almost all built-in Internet functionality. It's unclear if this includes utilities as simple as the PPP software used to connect to an ISP.
Netscape's decision to release source code for Navigator 5.0 appears to be outside the context of the rest of these announcements and events. In the Unix world, freeware and source code is almost a given for the majority of software used on servers. The emergence of the Web was the beginning of a real market for selling Unix server software rather than releasing code from universities and cooperative projects. In opposition to the growing commercialization of Unix server software, many developers have banded together in projects like Apache, which is a superb freeware Web server created and maintained by an army of programmers devoting their time gratis to the idea that having a good free server is in the best interests of the world. Apache is one of the most widely used Web servers, outpacing Netscape's various releases, Microsoft's Internet Information Server, and several other major offerings.
Netscape might be trying to hearken back to an earlier time when source code flowed freely from machine to machine, but it's not a clear motive. Perhaps by putting their mature pieces into the hands of this worldwide programming community they can have their ideas spread even further. But without control of the branding, development, and other components of the browser, I, for one, don't see what Netscape gets out of it.
On the one hand, we might be seeing the last futile cries of Netscape before the company is sucked into the maw of an Oracle, Sun, Motorola, or another industry giant. Microsoft might be poised to own 80 or 90 percent of the browser market share by the end of 1998. On the other hand, these major shifts in the landscape may turbocharge Netscape's continued advances into the corporate "enterprise" - the set of computers, networks, and servers that make up the information systems of a company. No outcome is clear yet, but tectonic shifts often cause new continents to rise out of the murky deep. Opera, anyone? [GF]
<http://db.netbits.net/getbits.acgi?nbart=04593>
Multiple Personalities -- Readers sent in many additional options for obtaining multiple email addresses without buying a separate dial-up account from an ISP.
John Belamaric <jdbel@jupiter.istc.com> notes: "EarthLink Network provides additional email addresses for $4.95 per month."
<http://www.earthlink.com/company/getmailbox.html>
Albert Smith <albert.smith@yale.edu> chimes in, "IBM.net allows up to five independent email addresses under one account. They are worldwide and seem great so far (I just set my mom up with them)."
John Roberts <johnr@cnet.com> comments, "If you have an account with Best Internet (an ISP in the Bay Area), you can get extra POP mail boxes at $5 per month (the price goes down as the quantity goes up)."
<http://www.best.com/services/pop/>
Steve Cavanaugh <appleblossom@delphi.com> suggests: "Delphi allows you to get a POP account for just $34.97 per year or $6.95 a month; it also gets you 10 MB of Web storage space and a few other services."
<http://www.delphi.com/dir-html/benefits/register.html>
Karl-Johan Noren <k-j-nore@dsv.su.se> writes, "In the Swedish market almost every larger ISP provides up to five separate email accounts with every Internet account. All this happened during the last year. Now, the Swedish market can't be compared with the American (lots smaller, and dominated by four or five large ISPs with a few local ones), but it shows a possible trend."
John Lucas-Williams <john@alysian.demon.co.uk> notes, "My ISP, Demon Internet Ltd. of London, has always provided dial-up Internet access that puts your machine on the Internet with its own static IP address. I can have an unlimited number of users online with their own email addresses. Using Eudora it is easy to have multiple mailboxes, and I have several, with a couple of friends having access as well. There is even an email address for my pet Border Terrier named Gyp <gyp@alysian.demon.co.uk> who "subscribes" to the Border Terrier List."
<http://www.demon.net/services/mail/pop3.html>
by Stuart Cheshire <cheshire@cs.stanford.edu>
[In NetBITS-015, Stuart examined issues of latency and delay in typical modem-based Internet communications. This week, he discusses how bandwidth can be used more efficiently, and how it affects the overall latency of a connection.]
<http://db.netbits.net/getbits.acgi?nbart=04658>
Last week, I asked readers to imagine a world where the only network connection you can get to your house is a modem running over a telephone line at 33.6 Kbps. Now, imagine that this is not enough bandwidth for your needs. You have a problem.
Making Bandwidth Is Easy -- Technically, the solution is simple. You can install two telephone lines and use them in parallel, giving you a total of 67.2 Kbps; a few companies are already shipping multiple-modem systems that let you combine two or more modems into a single parallel hunk of bandwidth. If you need more capacity you can install 10 telephone lines, for a total of 336 Kbps. Sure, it's expensive, having 10 modems in a pile is inconvenient, and you may have to write networking software to share the data evenly between the 10 lines.
But if it was sufficiently important, it could be done. People with ISDN lines already do this using a process called BONDING by some (which is short for "Bandwidth ON Demand INteroperability Group") or Multilink PPP (MPPP) by others, which enables them to use two 64 Kbps ISDN channels in parallel for a combined throughput of 128 Kbps (in some areas, it's two 56 Kbps channels that combine for 112 Kbps).
Getting additional bandwidth is possible, even if it's not always economical. However, an equally important fact is that making limited bandwidth go further is easy.
Compression -- Compression is an easy way to increase bandwidth. You can apply general purpose compression (such as StuffIt) to the data. Even better, you can apply data-specific compression (such as JPEG for still images and MPEG for video), which can provide much higher compression ratios.
These compression techniques trade off use of CPU power for lower bandwidth requirements. However, there's no equivalent way to trade off use of extra CPU power to make up for poor latency.
All modern modems use internal compression algorithms. Unfortunately, having your modem do compression is nowhere near as good as having your computer do it. Your computer has a powerful, expensive, fast CPU, whereas your modem has a feeble, cheap, slow processor. In addition, as we noted last week, a modem must hold on to data until it has a block big enough to compress effectively. This requirement adds latency, and once added, latency can't be eliminated. Also, since the modem doesn't know what kind of data you're sending, it can't use superior data-specific compression algorithms. In fact, since most images and sounds on Web pages are already compressed, a modem's attempts to compress the data a second time adds more latency without any benefit.
This is not to say that having a modem do compression never helps. When the host software at the endpoints of the connection is not smart and doesn't compress data appropriately, then the modem's own compression can compensate somewhat and improve throughput. The bottom line is that modem compression helps only dumb software; it hurts smart software by adding extra delay.
Send Less Data -- Another way to cope with limited bandwidth is to write programs that take care not to waste bandwidth. For example, to reduce packet size, wherever possible Bolo (my interactive network tank game for the Macintosh) uses bytes instead of 16-bit or 32-bit words.
<http://rescomp.stanford.edu/~cheshire/Bolo.html>
For many kinds of interactive software like games, it's not important to carry a lot of data. What's important is that when the little bits of data are delivered, they are delivered quickly. Bolo was originally developed running over serial ports at 4,800 bps and could support eight players that way. Over 28.8 Kbps modems it can barely support two players with acceptable response time. Why? A direct-connect serial port at 4,800 bps has a latency of 2 ms. A 28.8 Kbps modem has a latency of 100 ms, 50 times worse than the 4,800 bps serial connection.
Software can cope with limited bandwidth by sending less data. If a program doesn't have enough bandwidth to send high-resolution pictures, it could use a lower resolution. If a program doesn't have enough bandwidth to send color images, it could send black-and-white images, or images with dramatically reduced color detail (which is what NTSC television does). If there isn't enough bandwidth to send 30 frames per second, the software could send 15 fps, 5 fps, or fewer.
These trade-offs aren't pleasant, but they are possible. You can pay for more bandwidth or send less data to stay within your limited available bandwidth. However, if the latency is not good enough to meet your needs, you don't have the same option. Running multiple circuits in parallel won't improve latency, and sending less data won't help either.
Caching -- One of the most effective techniques for improving computer and network performance is caching. If you visit a Web site, your browser can copy the text and images to your hard disk. If you visit the site again, the browser verifies that the stored copies are up-to-date, and - if so - the browser just displays the local copies.
Checking the date and time a file was last modified is a tiny request to send across the network - so small that modem throughput makes no difference. Latency is all that matters and throughput is virtually irrelevant.
Latency Workarounds -- ISDN has a latency of about 10 ms. Its throughput may be twice that of a modem, but its latency is 10 times better, and that's the key reason why browsing the Web over an ISDN link feels faster than over a modem.
One reason standard modems have such poor latency is that they don't know what you're doing with your computer, or why. An external modem is usually connected through a serial port, and all it sees is an unstructured stream of bytes coming down the serial port.
Ironically, the much-maligned Apple GeoPort Telecom Adapter could have solved this problem. The Apple GeoPort Telecom Adapter connects your computer to a telephone line, but it's not a modem. Instead, all modem functions are performed by software running on the Mac. The main reason for the criticism is that this extra software takes up memory and slows down the Mac, but in theory it could offer an advantage no external modem could match. When you use the GeoPort Telecom Adapter, the modem software is running on the same CPU as your TCP/IP software and your Web browser, so it could know exactly what you are doing. When your Web browser sends a TCP packet, the GeoPort modem software doesn't have to mimic the behaviour of current modems. It could take that packet, encode it, and send it over the telephone line immediately, with almost zero latency.
Sending 36 bytes of data, a typical game-sized packet, over an Apple GeoPort Telecom Adapter running at 28.8 Kbps could take as little as 10 ms, making it as fast as ISDN, and ten times faster than the best modem you can buy today. For less than the price of a typical modem, the GeoPort Telecom Adapter could give you Web browsing performance close to that of ISDN. Even better, people who already own Apple GeoPort Telecom Adapters would need only a software upgrade. There's no telling if this will ever happen.
Bandwidth Still Matters -- Having said all this, you should not conclude that I believe bandwidth is unimportant. It is very important, but not in the way most people think. Bandwidth is valuable for its own sake, but also for its effect on overall latency - the important issue is the total end-to-end transmission delay for a data packet.
Remember the example in the first part of this article comparing the capacity of a Boeing 747 to a 737? Here's a real world example of the same issue. Many people believe that a private 64 Kbps ISDN connection is as good (or even better) as a 1/160 share of a 10 Mbps Ethernet connection. Telephone companies argue that ISDN is as good as new technologies like cable modems because though cable modems have much higher bandwidth, that bandwidth is shared between lots of users, so the average works out the same. This reasoning is flawed, as the following example will show.
Say we have a game where the data representing the game's overall state amounts to 40K. We have a game server, and in this simple example, the server transmits the entire game state to a player once every ten seconds. That's 40K every 10 seconds, an average of 4K per second or 32 Kbps. That's only half the capacity of a 64 Kbps ISDN line, and 160 users doing this on an Ethernet network will utilize only half the capacity of the Ethernet. So far so good: both links are running at 50 percent capacity, so the performance should be the same, right?
Wrong. On the Ethernet, when the server sends the 40K to a player, the player can receive that data as little as 32 ms later (40K / 10 Mbps). If the game server is not the only machine sending packets on the Ethernet, then there could be contention for the shared medium, but even in that case the average delay before the player receives the data is only 64 ms. On the ISDN line, when the server sends the 40K to a player, the player receives that data five seconds later (40K / 64 Kbps). In both cases the users have the same average bandwidth, but the actual performance is different. In the Ethernet case, the player receives the data almost instantly because of the connection's high capacity. But in the ISDN case, the connection's lower capacity means the information is already 5 seconds old when the player receives it.
The problem is that sending a 40K chunk every ten seconds and sending data at a uniform rate of 4K per second are not the same thing. If they were, ISDN, ATM, and other telephone company schemes would be good ideas. Telephone companies assume all communications are like the flow of fluid in a pipe. You just tell them the rate of flow you need, and they tell you how big the pipe has to be. Voice calls work like the flow of fluid in a pipe, but computer data does not. Computer data comes in lumps. A common mistake is to think that sending 60K of data once per minute is exactly the same as sending 1K per second. It's not. A 1K per second connection may be sufficient capacity to carry the amount of data you're sending, but that doesn't mean it will deliver the entire 60K in a timely fashion. It won't. By the time the lump finishes arriving, it will be one minute old.
The conclusion here is obvious: the capacity of a connection has a profound affect on its performance. If you're given the choice between a low bandwidth private connection, or a small share of a larger bandwidth connection, take the small share. Again, this is painfully obvious outside the computer world. If a government said it would build either a large shared freeway, or a million, tiny, separate footpaths, one reserved for each citizen, which would you vote for?
What Can You Do? I've received numerous messages from people who want to know what they can do to spread the word about these latency and bandwidth problems. I've found that calling modem vendors directly is futile, so I recommend that you circulate these two articles to friends who might find them interesting, and most important, send letters to editors of major magazines asking them to include latency times via ping and traceroute when testing modems for review. Perhaps if we can raise awareness about the horrible latency problems that all modems suffer, modem manufacturers will start putting effort into decreasing latency instead of just increasing throughput.
[Editor's note: Many readers wrote us following the first part of this article to ask if any modems featured the ideas that Stuart had mentioned. As far as we know, these ideas are still unimplemented. -Glenn]
[Portions of this article come from Stuart Cheshire's white paper entitled "Latency and the Quest for Interactivity," commissioned by Volpe Welty Asset Management, L.L.C.]
Question: How do you deal with attaching files to email? Many of you have asked about how to attach files to email so that the recipient receives them reliably. Louis-Philippe Thouin <phil@icrdl.net> said it most succinctly: "This is an oldie, but where can one find exhaustive information on how mail attachments are treated by different client and server (ISP) mail programs?"
Answer: There are dozens of possible combinations of email clients, attachment methods, and encoding systems. As far as we've seen, it's always a matter of experimentation to get the right mix in any given situation.
The root of the problem is how email itself is sent. Email is just a stream of data with a recipient and certain standard headers on the front. MIME (Multipurpose Internet Mail Extensions) was developed to create a uniform method of encapsulating attached files into this data stream. However, it appears that each mail client (like Eudora, Netscape Messenger, Claris Emailer, and so forth) handles MIME slightly differently. There are also differences in which encoding formats are supported by different mail clients. You'd think there would be at least one standard in common, but that's still to come.
<http://www.fwb.gulf.net/~dwsauder/mime.html>
Before MIME, you could attach files only when working within proprietary mail systems (like QuickMail, cc:Mail, or Microsoft Mail) or commercial online services (AOL, CompuServe, and others). If you wanted to include a file in email going over the Internet or between different mail systems or services, you would encode it and then paste it into the body of a mail message. Many older mail programs limited message size, which resulted in some amazing attempts to send large files in multiple pieces and reassemble them on the other side.
Encoding a file was necessary only for binary data, which, as you might recall from previous issues of NetBITS, is data that includes bytes which aren't pure text. Binary data is eight bits long and can represent a number from 0 to 255. ASCII data is described typically as seven bits, or values from 0 to 127. You would encode a binary file using either uuencode (a Unix standard) or BinHex (a Mac standard). Both methods take a stream of binary numbers and turn them into a longer stream of ASCII numbers. A decoder reverses the process. (Another reason to encode data was that most mail servers wouldn't handle binary data in mail messages and would corrupt the attachment.)
<http://www.dnai.com/~wussery/attach.html>
<http://www.natural-innovations.com/boo/binhex.html>
Now that we're in modern times, you'd think that this was all a thing of the past. In fact, it persists to an alarming degree, explaining the quantity and variety of mail we've received on the subject. Here's a simple example. If I'm using Eudora Pro on a Mac, I have four options for encoding attachments: BinHex, Uuencode, AppleSingle, and AppleDouble. The two Apple options are methods of sending a Macintosh file, with its two forks, as a binary attachment. If I send a file encoded with AppleSingle to a Netscape Messenger user, they're stymied. Netscape Messenger doesn't understand how to reassemble a MIME attachment that identifies itself as AppleSingle. Likewise, if I send a uuencoded file, the recipient is still stymied. Even more irritating is that uuencode is fairly universally supported - but it's not an option in the freeware Eudora Light on the Mac, though Eudora Light under Windows does support it. You see the problem.
A couple of years ago, I was regularly sending Microsoft Word files to my editor at Adobe Magazine, part of Adobe Systems. At that time, Adobe used cc:Mail with a gateway server that handled Internet mail and interacted with the proprietary cc:Mail servers. No matter what combination of encoding formats I tried and no matter what my recipient did, the files rarely came through intact. Fortunately, Adobe switched to Qualcomm's Eudora mail client, and my problems disappeared.
Our recommendation for solving this problem is to create a test file of the sort in question that you can send using a variety of methods to your recipient. Have that person report on which files (it's a good idea to name the files with the name of the encoding format you're using so your recipient can keep them straight) come through successfully and keep careful notes. Sometimes, and this is sad, FedEx is the fastest method of sending an electronic file (with help from a floppy disk) and making sure it arrives in the same form it left your machine.
All that said, MacUser Magazine last year published a comprehensive chart listing which encoding formats to use based on the sending and receiving email programs. It may be a bit out of date but is still useful. [GF]
<http://www.zdnet.com/macuser/mu_0297/handson/table.html>
[Please send any questions regarding the State of the Union and how it relates to the Internet to us at <faqtoids@netbits.net>, and include your full name and email address. Questions may be edited for content and length. We cannot guarantee publication or a reply.]
Special Acrobatics -- Francois Pottier <francois.pottier@inria.fr> passes along news of a free method of creating Acrobat PDF files (see NetBITS-014):
For people who don't want to buy the full Acrobat package, it's also possible to create PDF files using free tools, like a recent version of GhostScript, which can convert PostScript files to PDF files. (I don't know if it allows including hyperlinks, though.) The PostScript file itself can be created with a free tool such as TeX.
<http://www.cs.wisc.edu/~ghost/>
Fall from the High Wire -- Ray Davis <raydavis@wco.com> offers an excellent rebuttal to Mike Lee's view of Adobe Acrobat in NetBITS-014:
<http://db.netbits.net/getbits.acgi?nbart=04637>
I was disappointed to see the unqualified rave for Adobe's PDF electronic document format. In my experience, Acrobat is far more popular with print-experienced designers who rarely use the Web than it is with Web users.
For a painful example of PDF overuse, check Adobe's own Web site. The last time I visited, the company was so intent on publicizing its proprietary format that even its press releases (which by definition must be layout independent) were only available in PDF!
What most print-trained designers don't understand is that the "lack of precise layout" they complain of in HTML is one of HTML's greatest advantages. By using HTML, your documents can adjust themselves neatly to the viewer's monitor width, monitor resolution, and font choices. You can make text-only information easily available to text-only viewers (or even vision-impaired "listeners"). And, since computer users are used to vertical scrolling and use of vertical white space, you can divide your content into logical sections rather than into arbitrarily sized pages.
On the other hand, by using PDF, your documents may become illegible or unavailable. At the very least, they'll cause a break in the viewing experience and their download times will increase. PDF does make sense when precise layout of an entire document is truly required - but such occasions are far rarer on the Web than many print designers seem to believe.
[Please send letters to the editor to us at <letters@netbits.net>. Please include your full name and email address. Letters may be edited for content, grammar, and length. All letters become the property of TidBITS Electronic Publishing. We cannot guarantee publication or a reply.]
Non-profit, non-commercial publications and Web sites may reprint or link to articles if full credit is given. Others please contact us. We do not guarantee accuracy of articles. Caveat lector. Publication, product, and company names may be registered trademarks of their companies. NetBITS ISSN 1096-4908.
Previous Issue | Search NetBITS | NetBITS Home Page | Next Issue