After Dave's blog post
there have been lots of discussions and speculations whether Verizon is throttling AWS and Netflix for its customers. With a help of over 30,000 volunteers we have gathered speed test data to analyse what exactly is happening.
Our experiment run from 6th of February till 10th of February. Some data was collected before and after, however there were not enough testers to make statistically significant analysis. During the recorded days we gathered just over 60,000 data samples, out of which 1/6 were provided by Verizon customers. After the initial data analysis, we have decided that the best way to determine whether Verizon is limiting AWS bandwidth is to compare average speed (more precisely median as it is less polluted) recorded on servers that are close to each other. That is, AWS East vs Linode East and AWS California vs Linode California.
As you can see from the chart above the median speed on AWS East and Linode East does not differ much and does not indicate any severe traffic limitation that Dave had experienced.
The same story can be seen in California datacenters above with more variation between Linode and AWS which we suspect has more to do with lower popularity of California servers than Verizon limitations.
For fun, we have also included a comparison of the remaining 5/6 of the data sample of all other providers bundled together. Of course this comparison is not precise as we are bundling lots of different data access technologies which compare rather poorly to the majority of Verizon customers that are running on FiOS.
Finally, we were interested whether time of the day has any impact on the performance. Here, we do not see any surprises. Peak time (expected to be between 6PM - 9PM CST) has seen a drop in performance by about 12% on AWS and 18% on Linode for Verizon customers. For all other providers the peak time was not as severe with 4.5% drop on AWS and 10% drop on Linode.
As anticipated by us and also confirmed officially by Verizon, there is no evidence that the speed tests would show severe limitation of AWS connections. We do not rule out that Verizon is limiting in some ways Netflix, as most of the Netflix traffic does not flow through AWS but via Openconnect peering agreements
. The degradation of Netflix experience that some of the Verizon customers may be seeing is not unique to Verizon and other providers have problems
coping with Netflix data volumes too.
Nonetheless, for us this experiment does not end. We are preparing better ways to measure net neutrality violations. Stay tuned :)
Email: [email protected]