The Cable Modem Traffic Jam

Tie-ups that slow broadband Internet access to a crawl are a reality--but solutions are near at hand
The Cable Modem Traffic Jam

Broadband access to the Internet by cable modem promises users lightning-fast download speeds and an always-on connection. And recent converts to broadband from dial-up technology are thrilled with complex Web screens that download before their coffee gets cold.

But, these days, earlier converts to broadband are noticing something different. They are seeing their Internet access rates slow down, instead of speed up. They are sitting in a cable modem traffic jam. In fact, today, a 56K dial-up modem can at times be faster than a cable modem and access can be more reliable.

Other broadband service providers--digital subscriber line (DSL), integrated-services digital networks (ISDNs), satellite high-speed data, and microwave high-speed data--have their own problems. In some cases, service is simply not available; in other situations, installation takes months, or the costs are wildly out of proportion. Some DSL installations work fine until a saturation point of data subscribers per bundle of twisted pairs is reached, when the crosstalk between the pairs can be a problem. 

In terms of market share, the leaders in providing broadband service are cable modems and DSL as shown below:

But because the cable modem was the first broadband access technology to gain wide popularity, it is the first to face widespread traffic tie-ups. These tie-ups have been made visible by amusing advertisements run by competitors, describing the "bandwidth hog" moving into the neighborhood. In one advertisement, for example, a new family with teenagers is seen as a strain on the shared cable modem interconnection and is picketed. (The message is that this won't happen with DSL, although that is only a half-truth.)

So, today, the cable-modem traffic jam is all too real in many cable systems. In severe cases, even the always-on capability is lost. Still, it is not a permanent limitation of the system. It is a temporary problem with technical solutions, if the resources are available to implement the fixes. But during the period before the corrections are made, the traffic jam can be a headache.

Cable modem fundamentals

Today's traffic jam stems from the rapid acceptance of cable broadband services by consumers. A major factor in that acceptance was the 1997 standardization of modem technology that allowed consumers to own the in-home hardware and be happy that their investment would not be orphaned by a change to another cable service provider.

A cable modem system can be viewed as having several components:

The cable modem connects to the subscriber's personal computer through the computer's Ethernet port. The purpose of this connection is to facilitate a safe hardware installation without the need for the cable technician to open the consumer's PC. If the PC does not have an Ethernet socket, commercially available hardware and software can be installed by the subscriber or by someone hired by the subscriber.

Downstream communication (from cable company headend to cable subscriber's modem) is accomplished with the same modulation systems used for cable digital television. There are two options, both using packetized data and quadrature amplitude modulation (QAM) in a 6-MHz channel, the bandwidth of an analog television channel. QAM consists of two sinusoidal carriers that are phase shifted 90 degrees with respect to each other (that is, the carriers are in quadrature with each other) and each is amplitude modulated by half of the data. The slower system uses 64 QAM with an approximate raw data rate of 30 Mb/s and a 27-Mb/s payload information rate (which is the actual usable data throughput after all error correction and system control bits are removed). The faster system uses 256 QAM with an approximate raw data rate of 43 Mb/s and a payload information rate of 39 Mb/s.

With 64 QAM, each carrier is amplitude modulated with one of eight amplitude levels. The product of the two numbers of possible amplitude levels is 64, meaning that one of 64 possible pieces of information can be transmitted at a time. Since 2^6 is 64, with 64 QAM modulation, 6 bits of data are transmitted simultaneously. Similarly, with 256 QAM, each carrier conveys one of 16 amplitude levels, and since 256 is 2^8, 8 bits of data are transmitted simultaneously. The higher speed is appropriate for newer or upgraded cable plant, while the lower speed is more tolerant of plant imperfections, such as the ingress of interfering signals and reflected signals from transmission line impedance discontinuities.

The upstream communications path (from cable modem to cable headend) resides in a narrower, more challenged spectrum. A large number of sources of interference limits the upstream communication options and speeds. Signals leak into the cable system through consumer-owned devices, through the in-home wiring, the cable drop, and the distribution cable. Fortunately, most modern cable systems connect the neighborhood to theheadend with optical fiber, which is essentially immune to interfering electromagnetic signals. A separate fiber is usually used for the upstream communications from each neighborhood. Also, the upstream bandwidth is not rigorously partitioned into 6-MHz segments.

Depending on the nature of the cable system, one or more of a dozen options for upstream communications are utilized. The upstream bandwidth and frequency are chosen by the cable operator so as to avoid strong interfering signals.

The cable modem termination system (CMTS) is an intelligent controller that manages the system operation. Managing the upstream communications is a major challenge because all of the cable modems in the subscriber's area are potentially simultaneous users of that communications path. Of course, only one cable modem can instantaneously communicate upstream on one RF channel at a time. Since the signals are packetized, the packets can be interleaved, but they must be timed to avoid collisions.

The 1997 cable modem standard included the possibility of an upstream telephone communications path for cable systems that have not implemented two-way cable. Such one-way cables have not implemented an upstream communications path from subscriber to headend. Using a dial-up modem is a practical solution since most applications involve upstream signals that are mainly keystrokes, while the downstream communications includes much more data-intensive messages that fill the screen with colorful graphics and photographs and even moving pictures and sound. The CMTS system interfaces with a billing system to ensure that an authorized subscriber is using the cable modem and that the subscriber is correctly billed.

The CMTS manages the interface to the Internet so that cable subscribers have access to more than just other cable subscribers' modems. This is accomplished with a router that links the cable system to the Internet service provider (ISP), which in turn links to the Internet. The cable company often dictates the ISP or may allow subscribers to choose from among several authorized ISPs. The largest cable ISP is @Home, which was founded in 1995 by TCI (now owned by AT&T), Cox Communications, Comcast, and others. Another ISP, Road Runner, was created by Time Warner Cable and MediaOne, which AT&T recently purchased.

Cable companies serving 80 percent of all North American households have signed exclusive service agreements with @Home or Road Runner. Two more cable ISPs--High Speed Access Corp. and ISP Channel--serve the remaining U.S. and Canadian broadband households. And other major cable companies, CableVision and Adelphia in the United States and Videotron in Canada, offer their own cable modem service.

Cable modem bottlenecks

If there were just one cable modem in operation, it could in principle have an ultimate data download capacity of 27 Mb/s in a 64 QAM cable system or 39 Mb/s in a 256 QAM cable system. While the 256 is four times 64, the data capacity does not scale by this factor since the 8 bits simultaneously transmitted by 256 QAM are not four times the 6 bits simultaneously transmitted by 64 QAM. The 256 QAM data rates are only about 50 percent larger than the 64 QAM rates. Of course, if the cable modem is not built into a PC but is instead connected with an Ethernet link, the Ethernet connection is a bottleneck, albeit at 10 Mb/s. In any case, neither of these bottlenecks is likely to bring any complaints since downloads at these speeds would be wonderful.

A much more likely bottleneck is in the cable system's connection to the Internet or in the Internet itself or even the ultimate Web site. For example, Ellis Island recently opened its Web site to citizens to let them search for their ancestors' immigration records, and huge numbers of interested users immediately bogged down the site. No method of subscriber broadband access could help this situation since the traffic jam is at the information source. A chain is only as strong as its weakest link; if the link between the cable operator and the ISP has insufficient capacity to accommodate the traffic requested by subscribers, it will be overloaded and present a bottleneck.

This situation is not unique to a cable modem system. Any system that connects subscribers to the Internet will have to contract for capacity with an ISP or a provider of connections to the Internet backbone, and that capacity must be shared by all the service's subscribers. If too little capacity has been ordered, there will be a bottleneck. This limitation applies to digital subscriber line systems and their connections to the Internet just as it does to cable systems. If the cable operator has contracted with an ISP, the ISP's Internet connection is a potential bottleneck, because it also serves other customers. Of course, the Internet itself can be overloaded as it races to build infrastructure in step with user growth.

Recognizing that the Internet itself can slow things down, cable operators have created systems that cache popular Web sites closer to the user and that contain local sites of high interest. These sites reside on servers close to the subscriber and reduce dependence on access to the Internet. Such systems have been called walled gardens because they attempt to provide a large quantity of interesting Web pages to serve the subscriber's needs from just a local server. Keeping the subscriber within the walled garden not only reduces the demand on the Internet connection, but can also make money for the provider through the sale of local advertising and services. This technique can become overloaded as well. But curing this overload is relatively easy with the addition of more server capacity (hardware) at the cache site.

Two cable ISPs, Road Runner and @Home, were designed to minimize or avoid Internet bottlenecks. They do it by leasing virtual private networks (VPNs) to provide nationwide coverage. VPNs consist of guaranteed, dedicated capacity, which will ensure acceptable levels of nationwide data transport to local cable systems. @Home employs a national high-speed data backbone through leased capacity from AT&T. Early on, a number of problems caused traffic jams, but these are now solved.

Other potential bottlenecks are the backend systems that control billing and authorization of the subscriber's service. As cable modem subscriber numbers grow, these systems must be able to handle the load.

The capacity on the cable system is shared by all the cable modems connected to a particular channel on a particular node. Cable systems are divided into physical areas of several hundred to a few thousand subscribers, each of which is served by a node. The node converts optical signals coming from (and going to) the cable system's headend into radio frequency signals appropriate for the coaxial cable system that serves the homes in the node area:

Only the cable modems being used at a particular time fight for sizable amounts of the capacity. Modems that are connected but idle are not a serious problem, as they use minimal capacity for routine purposes.

Clearly, success on the part of a cable company can be a source of difficulty if it sells too many cable modems to its subscribers for the installed capacity. The capacity of a given 6-MHz channel assigned to the subscribers' neighborhood and into their premises is limited to the amounts previously discussed (27 Mb/s in a 64 QAM cable system or 39 Mb/s in a 256 QAM cable system) and the demand for service can exceed that capacity. Both upstream and downstream bandwidth limitations can hinder performance. Upstream access is required to request downloads and to upload files. Downstream access provides the desired information.

Usually, it is the downstream slowdown that is noticed. Some browsers (the software that interprets the data and paints the images on the computer screen) include so-called fuel gages or animated bar graphs that display the progress of the download. They can be satisfying when they zip along briskly, but rub salt in the wound when they crawl slowly and remind the user that time is wasting.

Bandwidth hogs in a subscriber's neighborhood can be a big nuisance. As subscribers attempt to share large files, like music, photos, or home movies, they load up the system. One of the rewards of high-speed Internet connections is the ability to enjoy streaming video and audio. Yet these applications are a heavy load on all parts of the system, not just the final link. System capacity must keep up with both the number of subscribers and the kinds of applications they demand. As the Internet begins to look more like television with higher-quality video and audio, it will require massive downstream capacity to support the data throughput. As the Internet provides more compelling content, it will attract even more subscribers. So the number of subscribers grows and the bandwidth each demands also grows. Keeping up with this growth is a challenge.

Impact of open access

Open access is the result of a fear on the part of the government regulators that cable system operators will be so successful in providing high-speed access to the Internet that other ISPs will be unable to compete. The political remedy is to require cable operators to permit competitive ISPs to operate on their systems. Major issues include how many ISPs to allow, how to integrate them into the cable system, and how to charge them for access. The details of how open access is implemented may add to the traffic jam.

A key component in dealing with open access is the CMTS. The ports on the backend of this equipment connect to the ISPs. But sometimes too few ports are designed into the CMTS for the number of ISPs wishing access. More recent CMTS designs accommodate this need. However, these are expensive pieces of equipment, ranging up to several hundreds of thousands of dollars. An investment in an earlier unit cannot be abandoned without great financial loss.

If the cost of using cable modem access is fairly partitioned between the cost of using the cable system and the access fees charged by the cable company, then the cable operator is fairly compensated for the traffic. With more ISPs promoting service, the likelihood is that there will be more cable modem subscribers and higher usage. This, of course, will contribute to the traffic jam. In addition, the backend processing of billing and cable modem authorization can be a strain on the system.

What to do about the traffic jam?

The most important development in dealing with all these traffic delays is the release of the latest version of the cable modem technical standard. Docsis Release 1.1 (issued by CableLabs in 1999) includes many new capabilities, of which the most pertinent in this context is quality of service (QoS). In most aspects of life, the management of expectations is critical to success. When early adopters of cable modem service shared a lightly loaded service, they became accustomed to lightning access. When more subscribers were added, the loading of the system lowered speed noticeably for each subscriber in peak service times.

Similarly, the difference between peak usage times and the late night or early morning hours can be substantial. It is not human nature to feel grateful for the good times while they last, but rather to feel entitled to good times all the time. The grades of service provided by QoS prevent the buildup of unreasonable expectations and afford the opportunity to contract for guaranteed levels of service. Subscribers with a real need for speed can get it on a reliable basis by paying a higher fee while those with more modest needs can pay a lower price. First class, business class, and economy can be implemented with prices to match.

Beefing up to meet demand

Network traffic engineering is the design and allocation of resources to satisfy demand on a statistical basis. Any economic system must deal with peak loads while not being wasteful at average usage times. Consumers find it difficult to get a dial tone on Mother's Day, because it would be impractically expensive to have a phone system that never failed to provide dial tone. The same is true of a cable modem system. At unusually high peaks, service may be temporarily delayed or even unavailable.

An economic design matches the capacity of all of the system elements so that no element is underutilized while other elements are under constant strain. This means that a properly designed cable modem system will not have one element reach its maximum capacity substantially before other elements are stressed. There should be no weakest links. All links should be of relatively the same capacity.

More subscribers can be handled by allocating more bandwidth. Instead of just one 6-MHz channel for cable modem service, two or more can be allocated along with the hardware and software to support this bandwidth. Since many cable systems are capacity limited, the addition of another 6-MHz channel can be accomplished only by sacrificing the service already assigned to it. A typical modem cable system would have a maximum frequency of about 750 MHz. This allows for 111 or so 6-MHz channels to be allocated to conflicting demands. Perhaps 60-75 of them carry analog television. The remainder are assigned to digital services such as digital television, video on demand, broadband cable service, and telephony.

Canceling service to free up bandwidth for cable modems may cause other subscriber frustrations. While adding another 6-MHz channel solves the downstream capacity problem, if the upstream capacity is the limiting factor in a particular cable system, merely adding more 6-MHz channels will still leave a traffic jam. The extra channels help with only one of the traffic directions.

Cable nodalization is another important option in cable system design for accommodating subscriber demand. Nodalization is essentially the dividing up of the cable system into smaller cable systems, each with its own path to the cable headend. The neighborhood termination of that path is called a node. In effect, then, several cables, instead of a single cable, come out of the headend to serve the neighborhoods.

Cable system nodes cater to anywhere from several thousand subscribers to just a few hundred. Putting in more nodes is costly, but the advantage of nodalization is that the same spectrum can be used differently at each node. A specific 6-MHz channel may carry cable modem bits to the users in one node while the same 6-MHz channel carries completely different cable modem bits to other users in an adjacent node. This has been called space-division multiplexing since it permits different messages to be carried, depending on the subscriber's spatial location.

An early example of this principle was deployed in the Time Warner Cable television system in Queens, New York City. Queens is a melting pot of nationalities. The immigrants there tend to cluster in neighborhoods where they have relatives and friends who can help them make the transition to the new world. The fiber paths to these neighborhoods can use the same 6-MHz channel for programs in different languages. So a given channel number can carry Chinese programming on the fiber serving that neighborhood, Korean programming on another fiber, and Japanese programming on still another fiber. As the 747s fly into the John F. Kennedy International Airport in Queens each night, they bring tapes from participating broadcasters in other countries that become the next day's programming for the various neighborhoods. (Note that this technique is impossible in a broadcast or satellite transmission system since such systems serve the entire broadcast area and cannot employ nodalization.)

The same concept of spectrum reuse is applied to the cable modem. A 6-MHz channel set aside for this purpose carries the cable modem traffic for the neighborhood served by its respective node. While most channels carry the same programming to all nodes, just the channel(s) assigned to the modem service carry specialized information directed to the individual nodes. Importantly, nodalization reuses the upstream spectrum as well as the downstream spectrum. So, given enough nodes, traffic jams are avoided in both directions.

However, nodalization is costly. Optical-fiber paths must be installed from the headend to the individual nodes. The fiber paths require lasers and receivers to convert the optical signals into electrical signals for the coaxial cable in the neighborhood. Additional modulators per node are required at the cable headend , as well as routers to direct the signals to their respective lasers. The capital investment is substantial. However, it is technically possible to solve the problem. (In principle, nodalization could be implemented in a fully coaxial cable system. But in practice coaxial cable has a lot higher losses than fiber and incurs even greater expense in the form of amplifiers and their power supplies.)

Other techniques for alleviating the traffic jam include upgrading the cable system so that 256 QAM can be used instead of 64 QAM downstream and 16 QAM can be used upstream instead of QPSK. If the ISP's connection to the Internet is part of the problem, a larger data capacity connection to the Internet backbone can be installed.

Also, non-Docsis high-speed access systems are under development for very heavy users. These systems will provide guaranteed ultrahigh speeds of multiple megabits per second in the downstream direction while avoiding the loading of the Docsis cable modem channels. The service can then be partitioned into commercial and residential or small business services that do not limit each other's capabilities.

Speculations on the future

The cable modem traffic jam is due to rapid growth that sometimes outpaces the resources available to upgrade the cable system. But solutions may be near at hand.

The next wave of standardization, Docsis 1.1 released in 1999, provides for quality-of-service segmentation of the market. Now that the standard is released, products are in development by suppliers and being certified by CableLabs. Release 1.1 products will migrate into the subscriber base over the next several years. Subscribers will then be able to choose the capacity they require for their purposes and pay an appropriate fee. The effect will be to discourage bandwidth hogs and ensure that those who need high capacity, and are willing to pay for it, get it. And market segmentation will provide financial justification to implement even more comprehensive nodalization. After enough time has passed for these system upgrades to be deployed, the traffic jam should resolve itself.