DoE Energy Science Network turns on 46.1-Tbps ESnet6 infrastructure

The U.S. Department of Energy’s (DoE’s) Energy Sciences Network (ESnet) formally launched ESnet6, the latest generation of its intra-laboratory and supercomputing backbone network, during an Oct. 11 ceremony at Lawrence Berkeley National Laboratory.

ESnet6 will offer the facilities it connects 46.1 Tbps of capacity based on an advanced architecture that enables rapid link building and high resiliency.

The ESnet infrastructure links all of DoE’s national laboratories, tens of thousands of DoE-funded researchers, and the department’s scientific instruments and supercomputing centers.

ESnet1 launched in 1986; ESnet5 has met the DoE’s needs since 2012. ESnet6 was developed to keep pace with traffic levels that are expected to quickly exceed the greater than 1.4 exabytes experienced over the past 12 months.

The new network also will provide increased resiliency and reliability as well as lay a foundation to support future science workflows, ESnet Executive Director Inder Monga told attendees at the launch ceremony.

The new network features what Monga described as a “hollow” core optical transport network with “smart edge” intelligence and processing capabilities at the onramps.

Automation-enabling orchestration and monitoring capabilities are layered on top. The result is the ability to establish connectivity on-demand using a few commands, which Monga and his team demonstrated during the ceremony.

15K miles of greenfield fiber-optic cable

The ESnet6 network features 15,000 miles of fiber-optic cable, over which connections from 400 Gbps to 1 Tbps can travel.

Among the firsts ESnet6 represents is ESnet’s first fully greenfield network construction as well as the first time the ESnet staff designed, built, and will manage its own optical infrastructure, Monga said.

That doesn’t mean ESnet was alone on the project. Vendor partners included:

Ciena, which supplied its Waveserver 5 platform with WaveLogic 5 Extreme coherent optics.
Infinera, which supplied its FlexILS and GX Series line systems.
Nokia, which supplied switches and routers.
Lumen, which provided the fiber infrastructure and related colocation facilities.
AMD, which supplied its Alveo U280 FPGA-based network-attached accelerator cards for use at the core network switching nodes as part of the network’s packet monitoring system.

Barbara Helland, associate director of the DoE Office of Science's Advanced Scientific Computing Research program, said:

“As scientific instruments grow in complexity and supercomputers simulate scientific phenomena at higher resolutions, the science community is facing a growing challenge: data volumes that are increasing exponentially, coupled with the need to move, share, and process this data globally and faster than ever before. With ESnet6, DoE researchers are equipped with the most sophisticated technology to help tackle the grand challenges we face today in areas like climate science, clean energy, semiconductor production, microelectronics, the discovery of quantum information science, and more.”