Exploring Windows Azure Network Latency

13 May 2012

I wanted to understand the latency between the various Windows Azure data centres, so I set up six deployments, one in in each of the principal locations:

• europe-north
• europe-west
• us-north
• us-south
• asia-east
• asia-southeast

I installed an out-of-the-box copy of AzureRunMe using a small VM. I started IIS on each VM using:

> net start w3svc

We can't use ping to test latency because the Windows Azure VMs don't respond to the Internet Control Message Protocol (ICMP). Even if we turn off the firewall on the VM, ICMP is (quite rightly) blocked at the Windows Azure boundary.

So I started to experiment with tcping, a console app that uses TCP rather than ICMP.

tcping -i 5 -n 12 xxx.cloudapp.net 80

This runs twelve tests with an interval of five seconds; I took the average.

I conducted my tests during the early afternoon on Sunday 13 May 2012.

On each host, I ran the following test.bat file:

start /b tcping.exe -i 5 -n 12 europe-north.cloudapp.net 80 > europe-north.txt
start /b tcping.exe -i 5 -n 12 europe-west.cloudapp.net 80 > europe-west.txt
start /b tcping.exe -i 5 -n 12 us-north.cloudapp.net 80 > us-north.txt
start /b tcping.exe -i 5 -n 12 us-south.cloudapp.net 80 > us-south.txt
start /b tcping.exe -i 5 -n 12 asia-east.cloudapp.net 80 > asia-east.txt
start /b tcping.exe -i 5 -n 12 asia-southeast.cloudapp.net 80 > asia-southeast.txt

I occasionally saw a big spike in the results (one of the timings being about four times higher). I couldn't explain these results, so I confess; I just ignored that data and re-ran the test.

Here are my results to the nearest millisecond:

\	europe-north	europe-west	us-north	us-south	asia-east	asia-southeast
home	45	45	126	140	311	296
europe-north	16	31	105	125	311	346
europe-west	32	15	110	125	313	345
us-north	105	110	15	46	221	250
us-south	131	126	47	15	205	235
asia-east	311	313	220	204	15	47
asia-southeast	333	343	250	234	47	15

As you would expect, the latency between data centres within the same region is small - 45 ms for USA and Asia, 30 ms for Europe. I wonder why the European backbone seems fastest?

The time to ping a deployment from itself seems to be about 15 ms for all data centres. I'm seeing just less that 15 ms when I run the same test between two machines on my own gigabit ethernet switch. That suggests that the hyper-visor, VLAN and load balancer are not a big overhead.

My two nearest data centres are Dublin and Amsterdam; they seem to be neck-and-neck in terms of latency.

I thought it would be fun to play with tracert, but of course we soon run into problems with blocked ICMP. Modern Unix traceroutes have TCP and layer four options, but alas not Windows.

Instead, I installed NMAP, which now supports TCP based traceroute like this:

 nmap -sS -p 80 -Pn --traceroute europe-north.cloudapp.net

From home, I get this

TRACEROUTE (using port 80/tcp)
HOP RTT      ADDRESS
1   31.00 ms homeportal (192.168.1.254)
2   31.00 ms homeportal (192.168.1.254)
3   46.00 ms 213.123.107.186
4   46.00 ms 213.123.107.161
5   46.00 ms 213.1.69.66
6   46.00 ms 217.41.168.35
7   46.00 ms 217.41.168.107
8   46.00 ms 109.159.249.110
9   31.00 ms core1-te0-7-0-6.faraday.ukcore.bt.net (109.159.249.19)
10  31.00 ms 213.131.193.54
11  32.00 ms peer1-xe1-1-0.telehouse.ukcore.bt.net (109.159.254.136)
12  32.00 ms 195.99.126.42
13  32.00 ms ge-3-3-0-0.lts-64cb-1a.ntwk.msn.net (207.46.42.18)
14  32.00 ms xe-0-0-0-0.lts-96cbe-1a.ntwk.msn.net (207.46.42.168)
15  31.00 ms xe-8-2-0-0.db3-96c-1b.ntwk.msn.net (207.46.42.190)
16  ... 18
19  47.00 ms 65.52.76.146

But you can see that hops 16 to 18 are missing. It looks like those clever Windows Azure administrators have configured things so that you can't find any details about the infrastructure. Indeed this is confirmed if you try NMAP from the Windows Azure VM itself.

TRACEROUTE (using port 80/tcp)
HOP RTT     ADDRESS
1   ... 4
5   0.00 ms 65.52.76.146

The details f the hops might be missing, but at least we know the number of hops:

\	europe-north	europe-west	us-north	us-south	asia-east	asia-southeast
europe-north	5	9	11	13	13	16

Another thing I noticed was that if I used NMAP to portscan my xxx.cloudapp.net host, the RDP connection dropped. Looks to me like there are quite a lot of smarts in the Windows Azure security and denial of service attack protection ;-)

NMAP also now comes with Zenmap, but I'll leave the pretty pictures for another day.