A Hop Away From Everywhere — Long-haul Links in Today’s Internet

Picture of Esteban Carisimo
Guest Author | Northwestern University
Twitter logo
LinkedIn logo
Facebook logo
June 25, 2024

Intercontinental Internet communications lie on top of a complex network of submarine cables forming its global communication backbone. As part of ongoing work focused on the criticality of the submarine cable network, fellow researchers from Northwestern University and the University of Wisconsin-Madison and I mapped traceroute measurements to submarine cables as a first step toward understanding its potential vulnerabilities. 

We initially set out to follow an approach first introduced in a position paper, building on the assumption that given a traceroute, one could identify the hop or link with the most significant latency and, if the associated routers were found near submarine landings, that this link could be mapped to one or a handful of cables. Unfortunately, a preliminary analysis of traceroute datasets changed this assumption.

While we found traceroutes matching these expectations, we also found many others in which the routers associated with a submarine-traversing link were far inland from the closest landing points, some as far as 700 km. Further analysis revealed many of these long-haul links cover distances larger than 10,000 km and connect every country worldwide.

A map showing a long-haul link connecting Seattle, US, with Singapore and a current link between the two cities, via Japan
Figure 1 — A long-haul link connecting Seattle, US with Singapore. Long-haul links can be a concatenation of several submarine cable segments, such as PC-1, JIH, and ASE in this case.

Motivated by these early observations, we conducted a longitudinal study of intercontinental long-haul links and their preferred destinations. In a paper that appeared at SIGMETRICS 2024, we report on the first-ever study of this long-haul infrastructure using a large corpus of traceroute data collected at the network’s edge and spanning over seven years. 

Our study revealed a vast network with links spanning thousands of kilometers and central nodes connecting as many as 45 countries.

While long-haul links can be thought of as the network-layer manifestation of critical transoceanic cables, we find that they are substantially longer than any intercontinental submarine cable segment, with a median Round-Trip Time (RTT) of 130ms—nearly 84% larger than the median RTT of submarine cable segments (70.76ms).

Cumulative distribution frequency graph comparing latency between long-haul links and submarine cable segments
Figure 2 — The cumulative latency distributions (in milliseconds) for both long-haul links and the computed latency of submarine cable segments. The latency equivalent for submarine cable segments presents a smoother distribution than that of long-haul links, with a median RTT of 70.76ms, compared with the median RTT of 130ms for long-haul links.

The termination points of these links are quite far from the nearest on-route landing points. Specifically, 64% of them are 500 km from the closest landing point, and in 10% of cases, that distance exceeds 3,513 km. Chicago, a popular ‘destination,’ is more than 1,000 km away from the nearest landing point and yet connects over 60 countries within one hop.

World map showing the cable landing destinations that Chicago, USA is one hop away from via submarine cables.
Figure 3 — Chicago, USA, is one hop away from 60 countries.

Our study contributes to the community effort to create consistent maps across layers of the Internet, from AS-level to logical and physical connectivity, critical to a range of important analyses from performance and robustness to security.

This new perspective opens a wide range of promising directions for future research, from alternative views of long-haul infrastructure to exploring its key properties and temporal stability.

To learn more about our study, read our paper ‘A Hop Away from Everywhere: A View of the Intercontinental long-haul Infrastructure,’ which we presented at the 2024 ACM SIGMETRICS Conference.

Contributors: Fabián E. Bustamante (Northwestern University), Caleb J. Wang (Northwestern University), Mia Weaver (U. of Wisconsin-Madison) and Paul Barford (U. of Wisconsin-Madison).

Esteban Carisimo is a Postdoctoral Researcher at AquaLab, a group at the Department of Computer Science of Northwestern University, whose research takes an interdisciplinary approach to Internet topology, including IXPs, CDNs, transit diversity, and network congestion.

Adapted from the original post that first appeared on the APNIC Blog.

Photo by Julia P VIA Flickr