Why high-speed connections are only part of the equation—and how packet analysis helps engineers keep AI workloads moving.
AI Is Changing the Network
Artificial intelligence is changing the way data centers are designed, built and operated. Training large AI models requires thousands of accelerators to exchange enormous volumes of data, while inference services must deliver results with predictable latency and high availability.
As these workloads grow, network infrastructure becomes a critical component of overall computing performance. Faster connections help, but they do not automatically guarantee that data will reach its destination efficiently.
Cisco’s recent networking announcements highlight this shift. The company has introduced advances in networking silicon, high-capacity switching and optical connectivity intended to support the demands of large-scale AI infrastructure. These developments reflect an important industry trend: the next generation of data center networking must deliver not only more bandwidth, but also better traffic management, predictable performance and efficient use of network resources.
For network engineers, that raises a practical question: how do you determine whether an AI workload is being limited by the network, and what can you do about it?
The answer starts with understanding what happens to packets as they move through the infrastructure.
1. Why AI Workloads Place New Demands on Data Center Networks
Traditional enterprise networks often carry a mixture of web traffic, file transfers, database queries and application communications. Although these workloads can be demanding, many do not require thousands of computing devices to coordinate continuously at high speed.
AI training introduces a different pattern.
Large language models and other distributed AI workloads divide computation across multiple GPUs or accelerators. These devices must exchange intermediate results, synchronize gradients and coordinate work across servers.
Depending on the model and parallelization strategy, this communication can involve substantial bursts of traffic between many endpoints at once.
Three characteristics make these workloads particularly challenging:
High bandwidth demand. Accelerators can generate large volumes of traffic during collective communication operations. Network links must move this data without becoming a bottleneck.
Sensitivity to latency. Distributed training frequently involves synchronization. If one worker finishes its communication later than the others, faster workers may have to wait, reducing overall accelerator utilization.
Traffic bursts and congestion. A network may have sufficient aggregate bandwidth but still experience congestion when many sources transmit toward the same destination or share the same links simultaneously.
Inference introduces its own challenges. Interactive AI services may require consistently low latency, while large-scale inference systems need to distribute requests and exchange data efficiently across accelerators and servers.
In both cases, the network must be designed around the traffic patterns of the workload—not just the advertised speed of its interfaces.
2. Cisco’s Networking Silicon: Building Capacity Into the Fabric
Cisco has been investing in high-capacity networking silicon designed for demanding data center environments. Its Silicon One family combines programmable networking capabilities with high-throughput packet processing, enabling equipment manufacturers and network operators to build high-performance switching and routing systems.
One of the company's reported developments is the Silicon One G302, a 51.2-terabit-per-second switch chip intended for high-density data center switching.
That figure represents the chip's aggregate switching capacity. It should not be confused with the speed of an individual Ethernet port or the throughput that a particular application will achieve.
Why does this distinction matter?
Consider a server connected to a switch through a 400-gigabit-per-second interface. Even if the switch can process traffic across many ports at high aggregate rates, an application may still experience poor performance if several servers send data toward a destination that cannot accept it at the same rate.
The problem is not necessarily insufficient switching capacity. It may be a mismatch between traffic demand and available capacity along a particular path.
High-performance switching silicon provides the foundation for moving packets quickly, but the surrounding network must also distribute traffic effectively, manage congestion and maintain predictable performance under load.
For AI infrastructure, that means evaluating the entire network fabric, including the connections between accelerators, top-of-rack switches, spine switches and storage systems.
Copper connections remain useful for many short-distance data center links, but optical technology becomes increasingly important as link speeds and interconnect distances grow.
Cisco has also highlighted optical interconnect developments intended to support the demands of large AI systems. Optical technologies can help provide high-bandwidth connectivity while addressing challenges involving reach, signal integrity, power consumption and physical cabling density.
Optical connectivity is particularly relevant when data must move between racks, across larger facilities or between different sections of an AI computing cluster.
However, increasing the capacity of an optical link does not eliminate congestion elsewhere in the network.
Imagine that several GPU servers can each transmit at hundreds of gigabits per second. If their traffic converges on an oversubscribed uplink, a storage service or a congested network path, the faster optical connections feeding that bottleneck may simply deliver packets to it more quickly.
This is why optical bandwidth and traffic engineering must be considered together.
Engineers should evaluate link utilization, oversubscription, traffic distribution, packet loss and queueing behaviour when determining whether an AI network can support its intended workload.
Optical technology expands the available capacity. Effective network design determines how well that capacity is used.
4. Why Raw Port Speed Is Only Part of the Story
A network interface's advertised speed describes its nominal transmission capacity. It does not describe the complete performance experienced by an application.
Several other factors influence actual results.
Congestion and queueing
When packets arrive faster than a link can transmit them, they may accumulate in a queue. This creates additional latency. If buffers fill, packets may be dropped.
Even brief periods of congestion can matter when distributed training processes must synchronize across many workers.
Packet loss and retransmissions
Lost packets can trigger retransmission mechanisms, consume additional bandwidth and delay the completion of communication operations.
For TCP-based traffic, retransmissions may be visible in a packet capture. For other protocols, loss detection and recovery may operate differently, so engineers must understand the transport and application protocols involved.
Uneven traffic distribution
A network can have substantial unused capacity overall while a particular link is saturated.
This can occur when multiple flows follow the same path, traffic is distributed unevenly across available links or many senders transmit toward a common receiver.
Aggregate utilization statistics may not reveal the problem clearly enough to identify its cause.
Latency variation
Average latency is useful, but it can conceal short periods of very high delay. Tail latency—the slower end of the latency distribution—can be especially important for synchronized workloads.
Oversubscription
Oversubscription occurs when the potential traffic entering a network section exceeds the capacity available farther downstream.
It is not always a design failure; it can be a deliberate cost and capacity trade-off. However, oversubscription ratios must be evaluated against actual workload behaviour.
These factors explain why a network built entirely with high-speed interfaces can still deliver disappointing AI performance.
The objective is not simply to maximize link speed. It is to ensure that the complete communication path delivers the throughput, latency and reliability the workload requires.
5. Using Packet Captures to Find AI Network Bottlenecks
Packet captures provide engineers with a way to investigate what is actually travelling across the network.
Switch counters and telemetry reveal important information about utilization, errors, drops and queue occupancy. Packet captures complement those measurements by exposing protocol behaviour, individual flows and timing relationships.
Tools such as Wireshark and tcpdump can help engineers determine whether observed application delays coincide with retransmissions, connection problems, unexpected traffic patterns or other network symptoms.
Step 1: Identify the affected workload
Start by establishing which application, server or accelerator group is experiencing poor performance.
Record when the problem occurs, which endpoints communicate and whether the slowdown affects all workloads or only a particular training job or inference service.
For distributed AI systems, coordinate the investigation with workload logs and application-level metrics. A packet capture is much more useful when the engineer knows what the application was doing at the time.
Step 2: Capture traffic at the right location
Capture traffic at a point where the relevant communication is visible.
Possible locations include a server's network interface, a switch mirror port, a network tap or a supported packet-broker platform.
A capture from one endpoint may reveal retransmissions or delayed acknowledgements, but it may not establish where packets were delayed or lost. Comparing captures at multiple points can help narrow down the location of a problem.
For example, if a packet appears in an ingress capture but is absent from an appropriately configured egress capture, the missing traffic provides a useful lead. Engineers must still account for capture loss, filtering, packet transformations and other possible explanations.
Step 3: Look for retransmissions and loss symptoms
In Wireshark, TCP analysis can reveal retransmissions, duplicate acknowledgements and periods of limited progress.
Useful display filters include:
- `tcp.analysis.retransmission`
- `tcp.analysis.fast_retransmission`
- `tcp.analysis.duplicate_ack`
- `tcp.analysis.lost_segment`
These filters help identify traffic worth examining. They are not conclusive proof that the physical network is responsible for a problem.
For instance, a reported missing segment may result from packets being dropped before reaching the capture point, asymmetric routing or limitations in the capture itself.
Correlate packet observations with interface counters, switch drop statistics, host metrics and application logs before deciding on a remedy.
Step 4: Measure timing and flow behaviour
Use Wireshark's flow analysis, packet timestamps, conversations view and I/O graphs to investigate when traffic slows down and how communication patterns change.
For TCP traffic, engineers can examine round-trip-time estimates, retransmission behaviour and periods of reduced sending progress.
For UDP-based transports, including some modern data center communication designs, engineers may need additional telemetry and protocol-specific analysis because TCP retransmission indicators are not applicable.
A useful investigation compares normal operation with a period of degraded performance. The goal is to determine whether the slowdown coincides with a change in traffic volume, a particular flow, an endpoint problem or a network event.
Step 5: Correlate packets with infrastructure telemetry
Packet captures are most effective when used alongside measurements from the switches, hosts and applications.
Check:
- Interface utilization and errors.
- Switch queue occupancy and drop counters.
- ECN markings and congestion-control telemetry, where supported.
- Host CPU utilization, NIC statistics and driver counters.
- GPU utilization and distributed-training synchronization times.
- Application throughput and latency.
For example, high retransmission activity combined with switch egress drops and a saturated uplink supports a congestion hypothesis. Retransmissions without corresponding congestion evidence may require investigation of the host, physical link, routing path or capture setup.
The objective is to build a consistent explanation from multiple sources rather than relying on a single packet-level symptom.
6. A Practical Example: When Faster Links Do Not Improve Training Time
Consider an AI training cluster containing multiple GPU servers connected through high-speed Ethernet.
After upgrading several links, an administrator expects training time to improve. Instead, the overall result changes very little.
What could be happening?
One possibility is that multiple servers are sending data toward the same switch uplink. The upgraded server interfaces can transmit more traffic, but the downstream link remains the limiting factor.
Another possibility is that the workload is sensitive to synchronization delays. A subset of workers experiences packet loss or queueing, causing the entire group to wait for those workers to finish communicating.
A third possibility is that the network is functioning correctly, but the workload is limited by GPU compute, storage access, CPU processing or application scheduling.
A structured investigation would look something like this:
1. Compare training throughput and synchronization timing before and after the upgrade.
2. Examine switch utilization, queue drops and congestion telemetry.
3. Capture representative traffic during both normal and degraded periods.
4. Investigate relevant retransmissions, latency changes and flow patterns.
5. Correlate network findings with GPU, host and application metrics.
6. Test a targeted change and measure whether it improves end-to-end workload performance.
The solution might involve adjusting traffic distribution, changing an oversubscription ratio, correcting an MTU mismatch, tuning a supported congestion-control configuration or addressing a host-side issue.
It may also turn out that no network change is required.
The important point is to identify the actual bottleneck before investing in additional hardware or changing a working configuration.
7. Building Better Visibility Into AI Networks
As AI clusters scale, manual packet captures alone become insufficient for continuous monitoring. Engineers need a combination of always-on telemetry, event-driven captures and targeted protocol analysis.
A practical visibility strategy includes:
- Continuous telemetry: Monitor link utilization, queue drops, latency and errors.
- Targeted packet captures: Collect traffic around specific incidents or performance tests.
- Flow analysis: Identify communication patterns and heavily utilized paths.
- Application correlation: Compare network events with training throughput and inference latency.
- Repeatable testing: Measure performance before and after each significant network change.
Packet captures also require careful handling. AI infrastructure may carry sensitive application data, authentication material or proprietary information. Capture filters, restricted access, encryption-aware monitoring and appropriate retention policies help reduce unnecessary exposure.
Where encryption prevents inspection of application payloads, engineers can still examine available transport metadata, packet timing, sizes and endpoint relationships, subject to the protocol and capture location.
The best monitoring strategy balances useful visibility with performance overhead, privacy and operational complexity.
Conclusion: Faster Networks Need Better Observability
Cisco's advances in switching silicon and optical technology reflect the growing need for high-capacity networking in AI data centers. More bandwidth and better interconnects can help support larger clusters and more demanding communication patterns.
But speed alone does not guarantee performance.
Congestion, packet loss, uneven traffic distribution, latency variation and host-side bottlenecks can all prevent applications from realizing the full benefit of high-speed networking.
For network engineers, the path to better AI performance involves combining capable hardware with sound traffic engineering, infrastructure telemetry and packet-level investigation.
Wireshark and tcpdump remain useful tools in that process. They help engineers investigate communication behaviour, test hypotheses and correlate network symptoms with what applications experience.
As AI infrastructure continues to evolve, the most effective networks will not simply move packets faster. They will make it easier to understand where those packets are going, how efficiently they are moving and what happens when performance begins to deteriorate.
That is where network observability turns raw bandwidth into measurable application performance.
References
1. Cisco — Advances Scalability and Openness at OCP 2026
Cisco's official announcement covering Silicon One G300 and G302 switching silicon, AI networking infrastructure, and planned optical innovations with NVIDIA. This is the primary source for the article's discussion of Cisco's technology.
2. SiliconANGLE — Cisco Scales Programmable AI Network Capacity with New Silicon and Repluggable Optics Innovations
Industry coverage of Cisco's AI networking announcements, including high-capacity switching, optical interconnects, bandwidth density and power efficiency.
3. Wireshark — User's Guide: Filtering Packets While Viewing
The official Wireshark documentation explains how display filters isolate relevant traffic by protocol, field values and other packet characteristics. It supports the article's practical guidance on investigating network behaviour.
Read the Wireshark filtering guide
4. Wireshark — TCP Display Filter Reference
A technical reference for TCP analysis fields, including retransmissions, duplicate acknowledgements, round-trip-time measurements and suspected lost segments. Useful for readers following the article's packet troubleshooting examples.
Browse the TCP display filter reference
5. Wireshark — Command-Line Manual and Packet Analysis Capabilities
The official manual documents packet capture and analysis capabilities, including capture files, display filters, conversation statistics and traffic analysis options. It provides additional technical background for engineers investigating network performance.
