The Road to 100G Networking
Date: Tuesday , December 30, 2008
Rapid movement within the information age continues to create challenges in the communication landscape. Application requirements and user demand are driving the bandwidth growth across backbone networks, well past the single 10G capacity to support multiple 10G connections.
Additionally, the networking landscape is shifting toward Ethernet-based infrastructures as the demand for high-bandwidth services increases, resulting in a lower cost infrastructure that supports 10G connectivity. With data demands increasing exponentially, the current generation of 10 Gb/s networking has become insufficient to meet tomorrow’s networking needs. This increased demand follows a pattern established with the adoption of 40 Gb/s standards for transport in the late 1990s, and the 40 Gb/s IP router and Dense Wavelength Division Multiplexing (DWDM) field trials in around 2004. Although 40 Gb/s service is entering a growth phase, it is insufficient for many future network applications. Currently, the industry is working to standardize 100G technologies to take advantage of greater scalability and a more efficient convergence of optical data rates. The result of this push may restrict the use of the 40G technology to limited areas of network capacity until 100G is commercially available. For these reasons vendors, industry standards bodies, and network operators favor 100 Gb/s transport speed for future networking. This paper outlines the requirement for 100G technologies and introduces the underlying fundamental technologies that enable the support of 100G.
Convergence of Transport and Ethernet
As networks migrate to IP packet-based networks, transport and Ethernet data rates experience a paradigm shift. Ethernet rates historically have increased by a factor of ten and are currently defined up to 10 Gb/s. Alternatively, SONET/SDH/Optical Transport Network (OTN) rates, currently defined up to 40 Gb/s, traditionally increased by a factor of four; the convergence began at 10 Gb/s. With recent developments, this convergence continues at 40 and 100 Gb/s, as shown in Figure 1.
The emerging standards support this convergence and provide the framework for next-generation networking. Currently, 40GbE — originally proposed for data center aggregation and enterprise computing — is likely to be used in the transport network, an increase by a factor of four instead of the traditional factor of ten for Ethernet. For backbone transport networks, 100 Gb/s is proposed for an increase of two and a half times on the previous 40 Gb/s SONET/SDH/OTN rate, instead of an increase by four times which is the previous rate. This convergence between transport and Ethernet, shown in Figure 2, is changing the networking landscape rapidly. Additionally, support for machine protocols, such as Infiniband over the Wide Area Network (WAN), are best suited for 100 Gb/s speeds, potentially driving the requirements for 100 Gb/s networking as well.
Robust, secure, resilient, and high-capacity networked storage replication solutions are business imperatives. The right network solutions must provide reliable, timely, and accurate application-layer replication over a network that cost-effectively delivers high performance connectivity to fit application requirements.
In response to the communications industry’s recognition of the need for increased bandwidth, the IEEE 802.3 Working Group created a High Speed Study Group (HSSG) in July 2006 to examine the need for next generation bandwidth connectivity. In July 2007, the group approved a request for two Media Access Control (MAC) rates: 40GbE to address the server-to-server and server-to-switch applications; and 100GbE for the switch-to-switch applications, including point-to-point links between campuses. Figure 3 shows the requirement objectives approved for the Physical (PHY) interface. Other approved objectives included were: * Support for full-duplex operation only * Preserving the 802.3 Ethernet frame format utilizing the 802.3 MAC * Preserving maximum and minimum frame size of current 802.3 standard * Supporting a Bit Error Rate (BER) better than or equal to 10^-12 at the
* Signaling (PLS) service interface * Providing appropriate support for OTN In December 2007, the P802.3ba task force was formed to commence development of the standard, with the target completion date of 2010 for the IEEE 802.3 40 and 100GbE standards, as shown in Figure 4. In addition, the International Telecommunications Union (ITU) Study Group 15 (SG 15) investigates the requirements for next generation transport rates beyond 40 Gb/s. In March 2007, SG 15 approved extending the G.709 OTN standard to the next higher rate beyond the current 43 Gb/s, defined as an OTU-4. Proposals have been submitted for a 3 x ODU-3 at approximately 130 Gb/s, as well as a rate optimized for transporting 100GbE — approximately 112 Gb/s.
Network Requirements for 100 Gb/s Transport
To accommodate the low-revenue-per-bit data required by IP networks, the networks must support 100 Gb/s high-speed connectivity to 100GbE communities at a very low cost. Network operators cannot afford to build new overlay networks, and are adopting a phased approach to add 100 Gb/s capacity seamlessly to networks, alongside 10 Gb/s and 40 Gb/s wavelengths, without re-engineering the network, as shown in Figure 5. To add capacity to existing networks, the spectral efficiency must increase: 100 Gb/s must be single wavelength, not inversed muxed parallel lanes of 10 or 25 Gb/s. Spectral efficiency increases with the bit rate, as seen in Figure 6. Inverse multiplexing increases network complexity due to the difficulty of finding four or ten adjacent waves; while planning and management become even more complex if the system allows non-adjacent wave groups. In addition, troubleshooting becomes more complicated when one wave has bit errors. Furthermore, inverse-multiplexing uses up expensive Re-configurable Add/Drop Multiplexer (ROADM) ports on all-optical networks but does not add more capacity. Availability of 10 Gb/s is limited because the consumption of ten waves at a time quickly limits the ability to offer 100 Gb/s capacity using the 10 x 10 Gb/s multiplexing approach.
100 Gb/s Transport Technology
Defining 100 Gb/s protocols and optical modulation techniques that operate cost-effectively over existing fiber and maintain design rules for lower speed networks is challenging. In the past, new technology was required to transport data over a single wavelength for each higher DWDM bit rate, since chromatic dispersion increases by the square of the bit rate, and polarization mode dispersion increases linearly with the bit rate. These dispersion issues lead to inverse muxing solutions, in which a 10 Gb/s signal may be split into four 2.5 Gb/s for transport.
Network operators regularly reject these solutions, though initially cost-effective, due to operational complexity and low spectral efficiency. As technology matures to next generation, the total network cost per bit decreases as the bandwidth rate increases, as shown in Figure 7. The invention of erbium-doped fiber amplifiers and the application of Forward Error Correction (FEC) have enabled DWDM networking at 2.5 Gb/s, lowering the overall cost of networking. The dispersion compensation fiber, gain equalization, enhanced FEC, and Raman amplification were employed with 10 Gb/s to achieve performance similar to that of 2.5 Gb/s DWDM systems. Tunable dispersion compensation, polarization mode dispersion compensation, and the new modulation formats allowed 40 Gb/s DWDM systems to achieve the same performance as 10 Gb/s systems. Polarization muxing and further-enhanced modulation formats will be used to achieve single wavelength 100 Gb/s networking, using engineering rules similar to those employed with 10 Gb/s systems.
Enabling 100 Gb/s with Advanced Modulation Formats
Advanced modulation formats are key to enabling 40 and 100 Gb/s networking. Formerly, modulation formats for 10 Gb/s and below were simple digital encoding using Non Return to Zero (NRZ) modulation formats. Phase-shaped binary transmission (duobinary) and Differential Phase Shift Keying (DPSK), which provide the foundation for 100 Gb/s are modulation formats employed to improve performance for first-generation 40 Gb/s solutions. DPSK uses phase-shift keying to approximately double the unregenerated reach over NRZ modulation schemes.
As shown in Figure 8, data is superimposed on a carrier wave and phase-shifted, providing a DPSK output. Although DPSK is useful for 40 Gb/s, Polarization Mode Dispersion (PMD) issues limit the unregenerated reach at 100 Gb/s. However, DPSK provides the foundation for Differential Quadrature Phase Shift Keying (DQPSK).
For 100 Gb/s signals, DQPSK splits the streams into two data channels, each equivalent to 50 Gb/s, as illustrated in Figure 9. This rate is 25 percent higher than 40 Gb/s; consequently, performance of this modulation can allow 100 Gb/s transmission on fibers not significantly PMD-impaired.
The addition of Polarization Multiplexing for DQPSK (PMDQPSK) for more impaired fiber links improves performance of a 40 Gb/s or 100 Gb/s signal. By splitting the DQPSK modulated waves into respective polarizations, as shown in Figure 10, the effective symbol rate is 25 Gb/s for 100 Gb/s signals, which operates better than the current 40 Gb/s DPSK systems. Although the PMD at this rate is about two and a half times greater than at 10 Gb/s, many fibers deployed since the mid-nineties on long-haul routes can accommodate this symbol rate for 40G and 100G with limited reach. Chromatic dispersion increases by the square of the bit rate and requires dispersion compensation, as on current 40 Gb/s systems. PMD compensation would be required only on PMD-impaired routes.
Performance at 100 Gb/s can be further improved with the addition of a coherent receiver to PM-DQPSK, known as PM-QPSK with a coherent receiver. The PM-QPSK modulation format is the same as PM-DQPSK on the transmitter side, as shown in Figure 9. The major difference in PM-QPSK lies with the receiver. Instead of a differential detection circuit, a coherent receiver is employed where the incoming signal is coupled with a local oscillator and detected. Coherent receivers have superior sensitivity over incoherent detection, eliminating the need for dispersion compensation but requiring development of digital signal processing ASICs. As a result, initial implementations of 100G are likely to be PM-DQPSK with dispersion compensation, with a limited reach over existing 10G infrastructure. However, this solution provides the essential building blocks for PM-QPSK with a coherent receiver, which industry leaders considered the modulation format of choice for 100 Gb/s signals because of superior performance and absence of need for dispersion compensation.
100 Gb/s Implementation Agreements
The market acceptance of 40G was delayed by the cost of the solution. Many suppliers chose different implementations, creating fragmentation of the market for components and resulting in overall higher costs. So far for 40G, the industry has been unable to achieve a price/bandwidth ratio of two and a half times the price for a fourfold increase in bandwidth. Industry leaders for 100G recognized this disparity early, knowing that a large investment would be required to bring 100G to fruition in a cost-effective manner. Consequently, a number of component and equipment manufacturers came together through the Optical Internetworking Forum (OIF) in May 2008 to evolve a consensus on a technical approach for 100G DWDM transmission in Ultra Long-haul (ULH) applications. DQPSK modulation with a coherent receiver was the chosen approach. In July 2008, the OIF created two new projects to specify additional aspects of this implementation direction. A photonic integration project will specify both transmitter and receiver photonic modules for incorporation into 100G DWDM transceivers. Integrating many photonic components into a small number of modules will be critical to reduce the cost, size, and power dissipation of a 100G transceiver. An FEC project aims to develop a code with improved performance over the current industry practice to help meet the stringent performance goals of the 100G ULH applications.
While agreements on key technologies for 100G ULH transmissions ultimately can enable interoperable solutions, full DWDM interoperability is outside the scope of the current OIF work. Instead, the OIF is focusing on creating a larger market for component suppliers by virtue of the common implementation approach and interoperability of the component building blocks within a 100G DWDM transceiver, and providing opportunities for product differentiation. This differentiation is expected to result largely from choices vendors make in the digital signal processing modules of the coherent receiver. The OIF 100G project will help foster a healthy 100G ecosystem that includes network operators, networking equipment vendors, subsystem vendors, and component suppliers. This broad inclusion will result in the accelerated introduction of 100G ULH solutions that will meet industry performance, size, cost, and power requirements.
Large-scale networks are increasingly challenged by bandwidth constraints of the backbone network while attempting to accommodate demands for new, high-speed services. The aggregation of 10GbE on IP routers, coupled with fiber plant exhaust, necessitates 100 Gb/s ports on DWDM equipment and 100GbE ports for switch-to-switch interconnection. With 100 Gb/s proposed as the economical solution to the emerging bandwidth crisis caused by the demand for new services, Ciena leads the development of timely, technically innovative 100 Gb/s solutions that operate over the existing 10 Gb/s infrastructures. By focusing on inventive solutions built for long-term value, Ciena provides investment protection, flexibility, and cost savings to solidify lasting business relationships and allow customer networks to readily adapt to new requirements.
The author is Managing Director, Ciena India