100G Data Center Switches: Pioneering the Next Generation of Open Networks
https://cloudswit.ch/blogs/100g-data-center-switches-of-open-networking/
What is 100G Switch?
The 100G Ethernet Switch is a high-performance network device engineered to provide throughput of up to 100 gigabits per second (Gbps). Acting as a central switch within a network, it interconnects multiple devices, enabling swift and efficient data exchange. By leveraging 100G Ethernet technology, these switches offer substantially increased transmission capacity with minimized latency, significantly boosting data transfer speeds in data centers. This capability makes them particularly well-suited for managing the growing demands of contemporary high-bandwidth applications.
How to choose a 100G Data Center Switch?
When selecting a 100G data center switch, it’s essential to consider several pivotal factors to ensure the device not only meets your current needs but also scales for future demands. Here’s a more dynamic look at what to keep in mind:
Performance: The heart of any 100G data center switch is its performance. The chipset and CPU are the driving forces behind its ability to process data, manage high-speed traffic, and handle vast volumes without bottlenecks. Opt for switches equipped with cutting-edge ASICs (Application-Specific Integrated Circuits) that promise high throughput and low latency. Ensure the CPU can adeptly manage control plane tasks, allowing the switch to efficiently handle both routine and complex operations in a dynamic data center environment.
Hardware Design:Hardware design plays a crucial role in both maintenance ease and switch reliability. Look for hot-swappable components like power supplies and fans to minimize downtime during maintenance or upgrades. Switches also need robust thermal capacity to handle high heat loads, making the design of internal cooling channels vital. Proper fan system design and layout ensure optimal airflow, reducing hotspots and keeping temperatures in check. You can even choose the direction of fan airflow for data center switches.
Plus, the number of ports directly impacts scalability—high port density switches support more connections, cutting down the need for additional switches.
Software: Software capabilities define the functionality and manageability of your switch. Look for a switch with a robust network operating system that delivers comprehensive management tools, automation, and telemetry features. As we advance into the AI era, the demand for higher bandwidth, lower latency, and stable data transmission is crucial for large-scale model training. With the rise of open networking, SONiC stands out as a flexible, scalable network OS with a robust ecosystem, widely supported and utilized in the open-source community. It’s an excellent choice when considering data center switches.
Asterfusion 100G Data Center Switches powered by open network
Asterfusion’s CX-N series switches are designed for ultra-low latency, offering industry-leading forwarding speeds. Equipped with enterprise-grade SONiC NOS, they unlock advanced features for data centers. The series includes two 100G switches: the CX532P-N, featuring 32*100G ports, and the CX564P-N, boasting 64*100G ports. These switches cater to a variety of user needs, ensuring flexibility and performance in demanding environments.
| CX532P-N | CX564P-N | |
| Ports | 32x 100G QSFP282x 10G SFP+ | 64x 100G QSFP282x 10G SFP+ |
| Switch ASIC | Marvell Teralynx 7 IVM77500 | |
| Switching Capacity | 3.2 Tbps | 6.4Tbps |
| Packet forwarding rate (bidirectional) | 6.35Bpps | 7.62Bpps |
| Max. Power Consumption | <710W | <820W |
The features of 100G Data Center Switches
Hardware features
Marvell Teralynx 7: Unmatched Low-Latency Power
The Asterfusion CX532P-N and CX564P-N switches deliver lightning-fast performance powered by the ultra-efficient Marvell Teralynx 7 chipset.This cutting-edge chipset minimizes forwarding latency by combining a high-speed hardware acceleration engine with a non-blocking switching architecture, ensuring that packets move through the switch with minimal delay for exceptional network speed and efficiency.
With a mere 520ns latency for the CX532P-N and an even faster 400ns for the CX564P-N, these switches excel in environments where every nanosecond counts. Perfect for high-frequency trading, financial data centers, and cutting-edge AI or machine learning applications, they deliver the real-time performance that mission-critical operations demand.
At the end of the article there are forwarding latency related test reports.
Versatile Interface Options for Asterfusion 100G Data Center Switches
The CX532P-N and CX564P-N 100G data center switches offer versatile port options. The 100G ports can backwards compatible to 40G or break out into four 25G or four 10G, giving network administrators flexible configuration choices.
- CX532P-N: 32x 100G ports, ideal for small data centers with tight budgets.
- CX564P-N: 64x 100G ports, perfect for large data centers needing high-density connectivity and scalability.
CX532P-N
CX564P-N
Both switches include an RJ45 management port, RJ45 console port, and USB2.0 port for easy configuration and monitoring. What’s more,they also feature two additional 10G ports for management tasks like INT Telemetry.
Software Features: Pre-installed Enterprise-grade SONiC NOS
Asterfusion 100G switches are equipped with an Enterprise SONiC NOS, finely tuned for data center environments. Compared to the community version, it provides advanced features, enhanced stability, and dedicated after-sales support. The device includes the KLISH command line, allowing traditional network engineers to transition smoothly to open networking, reducing learning costs. Since 2017, Asterfusion’s R&D team of over 100 experts has focused on perfecting the enterprise version of SONiC. Over the years, this version has accumulated valuable experience and found success in AI, education,telecommunications, and cloud providers.
RoCEv2 Enables Lossless Transmission in Data Center Networks
The Asterfusion CX-N series supports RoCEv2.
The CX532P-N and CX564P-N switches bring the power of Remote Direct Memory Access (RDMA) over Ethernet to your network, tackling server-side data computation latency and freeing up the CPU for critical tasks like running applications and processing large datasets. This technology boosts bandwidth while slashing latency, jitter, and CPU load.
With Priority Flow Control (PFC), RoCE flows get their own priority lanes, while Explicit Congestion Notification (ECN) smartly predicts and prevents network congestion. Enhanced Transmission Selection (ETS) ensures consistent service quality for similar traffic types, fully guaranteeing bandwidth, and Data Center Bridging Exchange (DCBX) streamlines the configuration of DCB parameters for RoCE devices.
Plus, RoCEv2 delivers ultra-low-latency, lossless networking at less than one-third the price of infiniband, giving customers exceptional performance without breaking the bank.
For more information on ROCE and Infiniband and which one you should choose, refer to https://cloudswit.ch/blogs/how-to-choose-from-infiniband-and-roce-for-aigc/
EVPN Multihoming Optimizes Network Load Balancing
The CX532P-N and CX564P-N integrate advanced technologies like EVPN Multi-homing for redundancy and load balancing, BGP EVPN for managing network information, and VXLAN for large-scale network expansion through tunneling. In practical scenarios, these switches can offload virtual networks and distributed network function gateways from the computing space, directly hosting them on the devices. This enhances server computing power, simplifies the creation of virtual nodes, and maximizes CPU efficiency, resulting in an efficient, reliable, and scalable data center network. For more details, please refer to https://cloudswit.ch/blogs/asterfusion-enterprise-sonic-nos-evpn-multi-homing/#why-evpn-multi-homing
Field test : Ultra-Low Latency Ethernet Switch VS. Infiniband Switch
The following is a comparison test in a distributed storage network scenario, and the devices participating in the test are all 32x100G specifications. Comparing the test results, we can clearly see that the overall performance of Asterfusion’s CX532P-N is better than Mellanox’s SB7700, and the latency level is lower.
In this storage scenario test, the CX532P-N switch outperforms the Infiniband switch (SB7700) in various features, and the CX-N series, which is a general-purpose Ethernet hardware, is only about 50% of the price of the dedicated device.
RoCEv2 Switches Test Result in HPC Network
We build an HPC test network using an Asterfusion CX564P-N ultra-low latency switch and a Mellanox InifiBand switch to compare their performance.
| Item | Model | Specs | Quantity | Remark |
| Switch | AsterfusionCX564P-N（64-port 100G） | \ | 1 | |
| Server | X86_64 | CPU: Intel Xeon Gold 6348Memory: 512G | 2 | \ |
| Optical Module | 100G | QSFP28 | 6 | \ |
| Fiber | Multi-mode | \ | 3 | \ |
| NIC | MCX455A-ECA_AX | Mellanox CX-4 Lx | 6 | Switch to Ethernet Mode |
| OS | CentOS 7.9.2009 | \ | \ | Kernel Version > 3.10.0-957.el7 |
| Mellanox NIC Driver | MLNX_OFED-5.7 | \ | \ | \ |
| Switch OS | AsterNOS （SONiC201911.R0314P02） | \ | \ | \ |
| Kernel | 3.10.0-1160.el7.x86_64 | \ | \ | \ |
| WRF | V4.0 | \ | \ | \ |
Test Result
Layer 2 network, RoCEv2 enabled on CX532P-N, HPC node (Mellanox NICs) optimized for RDMA
E2E forwarding performance test
| E2E Latency | Switch Model | T1 | T2 | T3 | AVG |
| Asterfusion | CX564N-P | 1580ns | 1570ns | 1570ns | 1573.3ns |
| Mellanox | IB Switch | 1210ns | 1220ns | 1220ns | 1213.3ns |
HPC application performance test (WRF)
| WRF/s | T1 | T2 | T3 | AVG |
| Single Device 24 core | 1063.22 | 1066.62 | 1066.39 | 1065.41 |
| Dual Device 24 core NIC direct connection | 1106.95 | 1110.73 | 1107.31 | 1108.33 |
| Dual Device 24 core RoCE | 1117.21 | 1114.32 | 1114.34 | 1115.29 |
| Dual Device 24 core IB | 1108.35 | 1110.44 | 1110.55 | 1109.78 |
For more text in HPC and AI
Click here to view additional test data on Asterfusion data center switches in AI: https://cloudswit.ch/blogs/sonic-switches-rocev2-test-in-aigc-storage-hpc/
Click here to view additional test data on Asterfusion data center switches in HPC scenarios:https://cloudswit.ch/blogs/sonic-switches-rocev2-test-in-aigc-storage-hpc/#ro-c-ev-2-switches-test-result-in-aigc-network‘
48*25G+8*100G QSFP28 uplinks Data Center Leaf (TOR) Switch
We’ve already explored the spine switch, so now let’s dive into the impressive leaf switch: the CX308P-N. This powerhouse features 8 upstream 100G ports and 48 downstream 25G ports, making it a versatile choice for your network needs.
What sets the CX308P-N apart from models like the CX532P-N and CX564P-N is its use of the Marvell Falcon chip, along with the Enterprise SONiC and Intel Broadwell DE processor. While it’s perfect as a Leaf (TOR) Switch for data centers, it can also serve as a Core Switch in enterprise campuses.
For more detailed information, please refer to https://cloudswit.ch/blogs/open-network-switch-25g-10g-asterfusion/#cx-308-p-48-y-n-v-2-data-center-grade-25-g-campus-switch
Speedy Little Light Beams: The Magic of 100G Optical Modules in Data Centers
To upgrade a data center to 100G Ethernet, in addition to the switch itself, a 100G optical module is required to interconnect the switches.
Asterfusion offers a wide range of 100G optical modules with different specifications for different transmission distances, single-mode/multi-mode, and AOC/DOC cables to fully meet the needs of different users.
评论
发表评论