From the September, 2013 Issue of Cabling Installation & Maintenance Magazine
The industry must develop cabling systems, networking devices, standards and field-test instruments suitable for the technology.
By Harshang Pandya, Psiber Data
Data center network infrastructure is witnessing a transformation, driven by growing bandwidth and network-performance demand. 10 Gigabit Ethernet is the de-facto standard in todays’ data center with growing adoption of 40G. While 40G Ethernet standards already exist for singlemode fiber and MPO-based multimode fiber cabling, standards bodies are currently developing 40GBase-T Ethernet over twisted-pair copper cabling systems. Such high networking speed imposes strict performance requirements for cable components and cabling systems. This article examines challenges in ensuring adequate performance of installed cabling, with specific attention to certification testing in the field.
Physical layer options
Several 40G physical-layer options already exist, and are briefly described here.
Singlemode fiber--Due to its long reach and superior transmission performance, singlemode fiber is specified for carrying 40-Gbit/sec data up to a distance as long as 10 kilometers, via 40GBase-LR4. The physical layer electronics and optics consist of four channels, each carrying 10-Gbit/sec data with different wavelengths. Singlemode fiber is the preferred option where budget is not a constraint, or when the link distances require it.
Multimode fiber--Multimode fiber with parallel optics and the MPO interface is the most popular medium for 40G Ethernet today, in the form of 40GBase-SR4. The networking hardware is less costly than long-wavelength, singlemode-based options, and it supports all typical link lengths (up to 100 meters for Om3 cabling and 125 meters for Om4 cabling) in a data center network.
Copper twinax--For short- reach channels up to a length of 7 meters, the 40GBase-CR4 standard specifies the use of twinaxial copper cable assemblies. The typical use of 40GBase-CR4 is interconnecting networked devices that are physically located adjacent to each other.
While copper twisted-pair is not an existing 40G option, recent developments suggest that copper structured cabling systems are here to stay, and that 40GBase-T will emerge as an important alternative to fiber links for 40G. Twisted-pair copper cabling is likely to retain its cost advantage over fiber, at least for the next several years. Copper cables are perceived to be easier to install and maintain. Importantly, Base-T networking standards over twisted-pair cabling are backward-compatible with autonegotiation capability. This enables organizations to upgrade to higher speeds incrementally, enabling greater control over capital expenses.
In 2012 the Institute of Electrical and Electronics Engineers (IEEE; www.ieee.org) initiated a formal project for defining a 40GBase-T standard using twisted-pair cabling. Cabling standardization bodies are not falling behind. The Telecommunications Industry Association (TIA; www.tiaonline.org) is developing specifications for Category 8 cabling systems suitable for 40GBase-T. The International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) has a similar project that aims to define two variants of cabling systems that will support 40GBase-T. These new cabling systems are being called Class I (using Category 6A-like components with higher capacity) and Class II (using Category 7A-like components with higher capacity). Additionally, ISO/IEC is defining recommendations about using existing cabling system such as Class FA for the application of 40GBase-T.
Lessons learned, moving forward
One of the key tasks in defining Ethernet standards is to determine appropriate radio-frequency (RF) bandwidth for communication. For example, 10GBase-T uses 400-MHz bandwidth, which roughly means that every Hz of RF spectrum carries 25 bits of binary data; in other words, channel capacity utilization is 25 bits/Hz. Higher-order and more-complex modulation schemes can increase capacity utilization. There is a maximum limit to channel capacity, known as Shannon Capacity. This limit is determined by electromagnetic noise experienced by the channel. Noise comes from external and internal sources. Examples of internal noise sources are crosstalk and return loss. Ethernet physical layer devices use sophisticated signal-processing techniques to predict and cancel the effects of internal noise sources, thereby increasing the channel’s capacity. However, there is a price to pay for such a capacity increase: higher power consumption, which leads to heat generation.
High power consumption was the single largest reason why 10GBase-T adoption lagged all predictions that were made when the standard was released in 2006. This problem has been largely overcome through innovative designs and semiconductor-technology advancements. With this 10GBase-T experience fresh in mind, the experts developing the 40GBase-T standard are not inclined to significantly increase the target for capacity utilization, because doing so would result in more signal processing and therefore greater power consumption.
But 40G transmits four times the data that 10G does. The way to support this data rate without the power-consumption penalty that results from significantly changing the modulation density (i.e. capacity utilization), is to increase bandwidth. In other words, if you cannot increase bits/Hz, then you must increase the Hz, which is the bandwidth. In this case, it would mean a fourfold increase of bandwidth, from 400 MHz to 1,600 MHz. The creators of 40GBase-T appear to be driving toward this bandwidth level.
One significant issue that comes along with increased bandwidth is that on twisted-pair cables, signal attenuates more rapidly as frequency increases. This means a received signal at 1,600 MHz is significantly smaller than a received signal at 100 MHz. This phenomenon imposes a restriction on the length of the cable. With a 100-meter twisted-pair cabling system, the signals received at high frequencies would be buried in the noise. Therefore, a compromise must be made on the maximum supportable link distance.
The net effects of all these considerations are as follows.
- 40GBase-T will use bandwidth spectrum from 1 MHz to approximately 1,600 MHz.
- The maximum length of cabling will be limited to around 30 meters.
- The cabling channel is likely to be specified for two rather than the usual four connections.
The good news is that a large percentage of data center link lengths are well within the 30-meter constraint. Studies have indicated that more than 80 percent of data center links are 30 meters or shorter, and therefore eligible to benefit from 40GBase-T.
Field testing 40G copper cabling
While cabling technologies and semiconductor technologies can ensure feasibility of supporting 40-Gbit/sec Ethernet over twisted-pair copper cabling, widespread market adoption calls for additional considerations. One of the key elements is the availability of field-test instruments to characterize and certify installed cabling for its suitability for 40G.
Just like a 300-millileter soft drink needs a bottle with 400-mL capacity for easy filling, cabling systems built to support 1,600-MHz transmission will be specified to 2,000 MHz. And field testers will typically support even higher measurement bandwidths. Although many aspects of 40GBase-T still are under the early stages of development, field testing is a noteworthy exception. Field-test capability already exists to provide sufficient measurement bandwidth to qualify 40GBase-T cabling during draft standardization stages as well as after standardization. A number of IEEE studies on RF performance of cabling systems have been done using this existing testing technology. Its availability has been a key for the research-and-development teams of cabling vendors to develop Category 8 components and subsystems in a fast, cost-effective manner.
Despite the growth in wireless and fiber infrastructure, copper cabling will still be the dominant media for enterprise networks in the foreseeable future. When designing infrastructure for use over the next 15 to 20 years, one must consider that there is a high likelihood that 40GBase-T systems will be defined and become commonplace over the next 5 to 10 years. There are technical challenges inherent in supporting such high data rates, a primary one being complexity of physical layer devices. In order to create a complete ecosystem for adoption of technologies like 40GBase-T, the industry will need cabling systems, networking devices, standardization and field-test instruments suitable for the technology. Field testing over wider bandwidth has been constrained in the past due to several factors, but now commercially available test equipment features the capability to certify cabling systems to bandwidths higher than 2,000 MHz, which is expected to meet field-testing needs for future 40GBase-T systems. ::
Harshang Pandya is managing director at Psiber Data Singapore. (www.psiber.com). Psiber has established a 40GBase-T Resource Center website, accessible from the company’s home page, which includes standards-committee documents, articles, white papers and other materials. This article is derived from a document in Psiber’s 40GBase-T Resource Center.
Archived CIM Issues