Towards faster web downloads and some interesting data
Decreasing the time required to download web pages is an obsession for a large number of content providers. They rely on various tricks to speed up the download of web pages. Many of these tricks are widely known, although they are not implemented on all web servers. Some depend on the content itself. A smaller web page will always load faster than a large web page. This is the reason why major content providers optimise their HTML content to reduce the unnecessary tags or compress their javascript scripts. They also heavily rely on cacheable information such as images, CSS, javascript, … Some also use gzip-based compression to dynamically compress the data that needs to be transmitted over the wire. This is particularly useful when web pages are delivered over low bandwidth links such as to mobile phones.
Two years ago, at the beginning of their work on SPDY, google published interesting data about the size of web accessible content on https://developers.google.com/speed/articles/web-metrics
The analysis was based on the web pages collected by googlebot. It reflects thus a large fraction of the public web. Some of the key findings of this analysis include :
- the average web page results in the transmission of 320 KBytes
- there are on average more than 40 GET Requests per web page and a web page is retrieved by contacting 7 different hostnames. The average page contains 29 images, 7 scripts and 3 CSS
- on average, each HTTP GET results in the retrieval of only 7 KBytes of data
The public web is thus widely fragmented and this fragmentation has an impact on its performance.
Last year, Yahoo researchers presented an interesting analysis of the work they did to optimise the performance of the web pages served by Yahoo [AFERG11]. This analysis focuses on the yahoo web pages but still provides some very interesting insights on the performance of the web. Some of the key findings of this analysis include :
around 90% of the objects served by yahoo are smaller than 25 Kbytes
despite the utilisation of HTTP/1.1, the average number of requests per TCP connection is still only 2.24 and 90% of the TCP connections do not carry more than 4 GETs
web page download is heavily affected by packet losses and packet losses occur. 30% of the TCP connections are slowed by retransmissions
Packet retransmission rate observed on the yahoo! CDN Source [AFERG11]
increasing the initial TCP window size as recently proposed and implemented on Linux reduces the page download time, even when packet losses occur
increasing the initial TCP window size may cause some unfairness problems given the small duration of TCP connections
There is still a lot of work to be done to reduce page download times and many factors influence the perceived performance of the web.
| [AFERG11] | (1, 2) Mohammad Al-Fares, Khaled Elmeleegy, Benjamin Reed, and Igor Gashinsky. Overclocking the yahoo!: cdn for faster web page loads. In Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference, IMC ‘11, 569–584. New York, NY, USA, 2011. ACM. URL: http://doi.acm.org/10.1145/2068816.2068869, doi:10.1145/2068816.2068869. |
Understanding the dropbox protocol and quantifying the usage of cloud storage services
Measurement studies show that video is consuming more and more bandwidth in the global Internet. Another service whose usage is growing are the cloud-based storage services like dropbox, icloud, skydrive or GDrive. These cloud storage services use proprietary protocols and allow users to exchange files and share folders efficiently. dropbox is probably one of the most widely known storage services. It heavily relies on the amazon EC2 and AWS services. The dropbox application is easy to use, but few is known open the operation of the underlying protocol. In a paper that will be presented this fall at IMC’12, Idilio Drago and his colleagues provide a very detailed analysis of the dropbox protocol and its usage in home and campus networks [DMMunafo+12].
Several of the findings of this paper are very interesting. First, despite its popularity, dropbox is still provided from servers located mainly in the US. This implies a long round-trip-time for the large population of dropbox users who do not reside in North America. Since dropbox uses the Amazon infrastructure, it is surprising that they do not seem to use Amazon datacenters outisde the US. All the files that you store in your dropbox folder are likely stored on US servers. Another surprising result is that dropbox divides the files to be transferred in chunks of 4 MBytes and each chunk needs be acknowledged by the application. Coupled with the long round-trip-time, this results in a surprisingly low transfer rate of about 500 Kbps. This performance issue seems to have been solved recently by dropbox with the ability to send chunks in batches.
[DMMunafo+12] also provides an analysis of the operation of the main dropbox protocol. dropbox uses mainly servers hosted on Amazon datacenters for various types of operation. Although dropbox uses TLS to encrypt the data, the authors used SSLBump running on squid to perform a man-in-the-middle attack between a dropbox client and the official servers.
An example of a storage operation with dropbox (source [DMMunafo+12])
Another interesting information provided in [DMMunafo+12] is an analysis of the dropbox traffic in campus and home networks. This analysis performed by using tstat shows that cloud storage services already contribute to a large volume of data in the global Internet. The analysis also considers the percentage of clients that are uploading, downloading and silent. Users who have installed dropbox but are not using it should be aware that the dropbox client always opens connections to dropbox servers, even if no data needs to be exchanged. The entire dataset collected for is available from http://www.simpleweb.org/wiki/Dropbox_Traces
| [DMMunafo+12] | (1, 2, 3, 4) Idilio Drago, Marco Mellia, Maurizio M. Munafò, Anna Sperotto, Ramin Sadre, and Aiko Pras. Inside Dropbox: Understanding Personal Cloud Storage Services. In Proceedings of the 12th ACM SIGCOMM Conference on Internet Measurement, IMC’12. 2012. |
An interesting retrospective of a decade of transport protocol research with SCTP
TCP is still the dominant transport protocol on the Internet. However, since the late 1990s, researchers and IETFers have worked on developing and improving another reliable transport protocol for the Internet : SCTP. The Stream Control Transport Protocol (SCTP) RFC 4960 provides several additional features compared to the venerable TCP, including :
- multihoming and failover support
- partially reliable data transfer
- preservation of message boundaries
- support for multiple concurrent streams
The main motivation for the design of SCTP has been the support of IP Telephony signaling applications that required a reliable delivery with small delays and fast failover in case of failures. SCTP has evolved to support other types of applications.
Since the publication of the first RFC on SCTP, various research groups have proposed extensions and improvements to SCTP. SCTP’s flexibility and extensibility has enabled researchers to explore various new techniques and solutions to improve transport protocols. A recently published survey paper [BGBFerrus12] analyses almost a decade of transport protocol research by looking at over 430 SCTP related publications. Like most survey papers, it provides an introduction to the paper topic, in this case SCTP and briefly compares some of the studied papers.
[BGBFerrus12] goes beyond a simple summary of research papers and the approach chosen by the authors could probably be applicable in other fields. In order to understand the evolution of the main topics of SCTP research, the authors of [BGBFerrus12] classified each paper along four different dimensions :
- protocol features (handover, load sharing, congestion, partial reliability, …)
- application (signaling, multimedia, bulk transfer, web, …)
- network environment (wireless, satellite, best effort, …)
- study approach (analytical, simulation, emulation, live measurements, …)
By using these four dimensions [BGBFerrus12], provides a quick snapshot of the past SCTP research and its evolution over the years. Not surprisingly, simulation is more popular than live measurements or bulk data transfer is more often explored than signaling. Furthermore, [BGBFerrus12] provides interesting visualization such as the figure below on the chosen study approach.
Fourth dimension : the study approach used for SCTP papers during the last decade [BGBFerrus12]
[BGBFerrus12] is a very interesting starting point for any researcher interested in transport protocol research. The taxonomy and the presentation could also inspire researchers in other fields. A web interface for the taxonomy is also available. Unfortunately, it does not seem to have been maintained after the finalization of the paper.
| [BGBFerrus12] | (1, 2, 3, 4, 5, 6, 7) \Lukasz Budzisz, Johan Garcia, Anna Brunstrom, and Ramon Ferrús. A taxonomy and survey of sctp research. ACM Comput. Surv., 44(4):18:1–18:36, September 2012. http://doi.acm.org/10.1145/2333112.2333113. doi:10.1145/2333112.2333113. |
Less than best effort : congestion control schemes do not always try to optimize goodput
The TCP congestion control scheme aims at providing a fair distribution of the network resources among the competing hosts while still achieving the highest possible throughput for all sources. In some sense, TCP’s congestion control scheme considers that all sources are equal and should obtain the same fraction of the available resources. In practice, this is not completely true since it is known that TCP is unfair and favors sources with a low round-trip-time compare to sources with a high round-trip-time. Furthermore, TCP’s congestion control scheme operates by filling the available buffers in the routers. This mode of operation results in an increase in the end-to-end delay perceived by the applications. This increased delay can be penalizing for interactive applications.
However, TCP’s congestion control RFC 5681 is only one possible design point in the space of congestion control schemes. Some congestion control schemes start from a different assumption than TCP. The Low Extra Delay Background Transport (LEDBAT) IETF working group is exploring such an alternate design point. Instead of assuming that all sources are equal, LEDBAT assumes that some sources are background sources that should benefit from the available network resources within creating unnecessary delays that would affect the other sources. The LEDBAT congestion control scheme is an example of a delay based congestion controller. LEDBAT operates by estimating the one-way delay between the source and the destination. This delay is estimated by measuring the minimum delay and assuming that this minimum delay can serve as a reference to evaluate the one-way delay. The figure below (from [RW12]) illustrates clearly the estimation of the minimum one-way-delay and then the estimation of the current one-way-delay. LEDBAT uses a congestion window and adjusts it every time the measured delay changes. If the measured delay increases, this indicates that the network is becoming more congested and thus background sources need to backoff and reduce their congestion window.
Estimation of the queueing delay in LEDBAT [RW12]
LEDBAT is only one example of congestion control schemes that can be used by less-than best effort applications. A recent survey [RW12] RFC 6297 summarizes the main features and properties of many of these congestion control schemes. Furthermore, [RW12] provides pointers to implementations of such controllers. Such congestion control schemes have notably been implemented inside Bittorrent clients. An analysis of the performance of such p2p clients may be found in [RTV12]
| [RW12] | (1, 2, 3, 4) D. Ros and M. Welzl. Less-than-best-effort service: a survey of end-to-end approaches. Communications Surveys Tutorials, IEEE, PP(99):1 –11, 2012. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6226797. doi:10.1109/SURV.2012.060912.00176. |
| [RTV12] | D. Rossi, C. Testa, and S. Valenti. Yes, we ledbat: playing with the new bittorrent congestion control algorithm. In Passive and Active Measurement (PAM’10). Zurich, Switzerland, April 2012. http://www.pam2010.ethz.ch/papers/full-length/4.pdf. |
Getting a better understanding of the Internet via the lenses of RIPE Atlas
The Internet is probably the most complex system to have been built by humans. All the devices and software that compose the Internet interact in various ways. Most of the times, these interactions are positive and allow to improve the network performance. However, some interactions can cause losses of connectivity, decreased performance or other types of problems.
Understanding all the factors that affect the performance of the Internet is complex. One way to approach the problem is to collect measurements about various metrics such as delay, bandwidth or paths through the network. Several research projects and companies are currently collecting large amounts of data about the Internet.
A very interesting project is RIPE Atlas . RIPE is a non-profit organisation mainly composed of network operators whose objective is to allocate IP addresses in Europe. In addition to this address allocation activity, they allow carry various projects that are useful for their members. Atlas is one of their recent projects. To obtain a better understanding on the performance and the connectivity of Internet nodes, RIPE engineers have developed a very small network probe that contains an embedded operating system, has an Ethernet plug an is powered via USB. A photo of the Atlas probe is available here.
This embedded system has low power and low performance, but it can be deployed at a large scale. As of this writing, more than 1800 probes are connected to the Internet and new ones are added on a daily basis. This large number of nodes places RIPE in a very good position to obtain very good data about the performance of the network since they can run various types of measurements including ping and traceroute. As of this writing, Atlas is mainly used to check network connectivity since Atlas hosts can request their own measurements. In this future, it can be expected that Atlas hosts will be able to program various forms of measurements and RIPE has developed a credit system that allows hosts to obtain credits based on the number of Atlas probes that they host.
Atlas already cover a large fraction of the Internet. In can check on https://atlas.ripe.net/ the probes that have been activated near your location. If you live in an area without Atlas probes and have permanent Internet connectivity, you can apply on https://atlas.ripe.net/pre-register/
DNS injection can pollute the entire Internet
The Domain Name System is one of the key applications on the Internet since it enables the translation of domain names into IP addresses. There are many usages of the DNS and unfortunately some abuses as well. Since DNS allows to map domain names into IP addresses, a simple attack on the DNS system consist in providing incorrect answers to DNS queries. This can be performed by attackers willing to launch a Man in the Middle Attack but also by some ISPs and some governments to block some websites. Various countries have laws that force ISPs to block some specific websites, for various purposes. In Belgium this technique has been used several times to block a small number of websites, see e.g. http://blog.rootshell.be/2011/10/04/the-great-firewall-of-belgium-is-back/
Some countries have more ambitious goals than Belgium and try to block a large number of websites. Chine is a well-known example with the Great Firewall of China. One of the techniques used in China is DNS injection. In a few words, a DNS injector is a device that is strategically placed inside the network to capture DNS requests. Every time the injector sees a DNS request that matches a blocked domain, it sends a fake DNS reply containing invalid information. A recent article published in SIGCOMM CCR analyses the impact of such injectors [Ano12]. Surprisingly, DNS injectors can lead to pollution of DNS resolvers outside of the country where the injection takes place. This is partially because the DNS is a hierarchy and when a resolver sends a DNS request, it usually needs to query top-level domain name servers. When the path to such a server passes through a DNS injector, even if the actual data traffic will not pass through the DNS injector later, the injector will inject a fake reply and the website will not be reachable by any client of this resolver during the lifetime of the cached information. The analysis presented in the paper shows that this DNS injection technique can pollute a large number of resolvers abroad. The article reports that domains in belonging to .de are affected by the Great Firewall of China.
DNS injection (source : [Ano12])
The article on The Collateral Damage of Internet Censorship by DNS Injection should become a must read for ISP operators who are forced by their governments to deploy DNS injectors. In parallel, this should also be a strong motivation to deploy DNSSEC that will enable resolvers to detect and ignore these DNS injections.
| [Ano12] | (1, 2) Anonymous. The collateral damage of internet censorship by dns injection. SIGCOMM Comput. Commun. Rev., 42(3):21–27, 2012. http://doi.acm.org/10.1145/2317307.2317311. doi:10.1145/2317307.2317311. |
Interesting networking magazines
During their studies, students learn the basics of networking. After one of maybe two networking courses, they know a very small subset of the networking field. This subset should allow them to continue learning new protocols, architectures and algorithms. However, finding information about these new techniques can be difficult. Here is a short list of magazines and journals that publish regularly interesting information about the evolution of the networking field.
A first source of information are the magazines. There are various trade press magazines that usually present new products and provide information on new commercial deployments. A networking student should not limit his/her networking knowledge to the information published in the trade press. Here is a subset of the magazines that I often try to read :
- the Internet Protocol Journal published by Cisco is a very useful and very interesting source of tutorial papers. Each issue usually contains articles about new Internet protocols written by an expert. The papers are easy to read and do not contain marketing information.
- the IETF Journal published by the Internet Society publishes short papers about the evolution of the protocols standardized by the IETF
These two magazines are freely available to anyone.
Scientific societies also publish magazines with networking content. These magazines are not available at the newsstand, but good university libraries should store them.
The IEEE publishes several magazines with some networking content :
- IEEE Network Magazine publishes tutorial papers in the broad networking field.
- IEEE Communications Magazine publishes tutorial articles on various topics of interest to telecommunication engineers, with sometime networking sections or articles
- IEEE Internet Computing Magazine publishes articles mainly on Internet-based applications with sometimes articles on lower layers
- IEEE Security and Privacy publishes articles on new advances in security and privacy in general. Some of the published articles are related to networking problems.
The ACM publishes various magazines and journals that cover most areas of Computer Science.
- Communications of the ACM is ACM’s main magazine. This magazine is an essential read for any computer scientist willing to track the evolution of his/her scientific field. It sometimes publishes networking articles. ACM Queue is a special section of the magazine that is devoted to more practically oriented articles. It has published very interesting networking articles and furthermore all content on ACM Queue is publicly available.
If you read other networking magazines that are of interest for networking students, let me know. I will cover networking conferences and networking journals in subsequent posts.
Multipath TCP : beyond grandmother’s TCP
TCP, the Transmission Control Protocol, is the default transport protocol in the TCP/IP protocol suite. TCP is essentially based on research carried out during the 1970s that resulted in the publication of RFC 793. Since then, TCP has slowly evolved. A major step in TCP’s history is the congestion control scheme proposed and implemented by Van Jacobson [Jac88]. RFC 1323 proposed a few years later extended TCP to support larger windows, a key extension for today’s high speed networks. Various changes have been included in TCP over the years and RFC 4614 provides pointers to many of the TCP standardisation documents.
In the late 1990s and early 2000s, the IETF developed the Stream Control Transmission Protocol (SCTP) RFC 4960. Compared to TCP, SCTP is a more advanced transport protocol that provides a richer API to the application. SCTP was also designed with multihoming and mind and can support hosts with multiple interfaces, something that TCP cannot easily do. Unfortunately, as of this writing, SCTP has not yet been widely deployed despite being implemented in major operating systems. The key difficulties in deploying SCTP appear to be :
- the lack of SCTP support in middleboxes such as NAT, firewalls, …
- the need to update applications to support SCTP, although this is changing with RFC 6458
SCTP seems to be stuck in a classical chicken and egg problem. As there are not enough SCTP applications, middleboxes vendor do not support it and application developers do not use SCTP since middleboxes do not support. Multipath TCP is a major extension to TCP whose specification is currently being finalised by the MPTCP working group of the IETF. Multipath TCP allows a TCP connection to be spread over several interfaces during the lifetime of the connection. Multipath TCP has several possible use cases :
The design of Multipath TCP [FRHB12] has been more complicated than expected due to the difficulty of supporting various types of middleboxes [RPB+12], but the protocol is now ready and you can even try our implementation in the Linux kernel from http://www.multipath-tcp.org
| [FRHB12] | Alan Ford, Costin Raiciu, Mark Handley, and Olivier Bonaventure. Tcp extensions for multipath operation with multiple addresses. Internet draft, draft-ietf-mptcp-multiaddressed-07, March 2012. URL: http://tools.ietf.org/html/draft-ietf-mptcp-multiaddressed-09. |
| [Jac88] | V. Jacobson. Congestion avoidance and control. In Symposium proceedings on Communications architectures and protocols, SIGCOMM ‘88, 314–329. New York, NY, USA, 1988. ACM. URL: http://doi.acm.org/10.1145/52324.52356, doi:10.1145/52324.52356. |
| [PDD+12] | Christoph Paasch, Gregory Detal, Fabien Duchene, Costin Raiciu, and Olivier Bonaventure. Exploring mobile/wifi handover with multipath tcp. In ACM SIGCOMM workshop on Cellular Networks (Cellnet12). 2012. URL: http://inl.info.ucl.ac.be/publications/exploring-mobilewifi-handover-multipath-tcp. |
| [RBP+11] | Costin Raiciu, Sebastien Barre, Christopher Pluntke, Adam Greenhalgh, Damon Wischik, and Mark Handley. Improving datacenter performance and robustness with multipath tcp. In Proceedings of the ACM SIGCOMM 2011 conference, SIGCOMM ‘11, 266–277. New York, NY, USA, 2011. ACM. URL: http://doi.acm.org/10.1145/2018436.2018467, doi:10.1145/2018436.2018467. |
| [RPB+12] | Costin Raiciu, Christoph Paasch, Sebastien Barre, Alan Ford, Michio Honda, Fabien Duchene, Olivier Bonaventure, and Mark Handley. How hard can it be? designing and implementing a deployable multipath tcp. In USENIX Symposium of Networked Systems Design and Implementation (NSDI’12), San Jose (CA). 2012. |
Anatomy of a Large European IXP
The Internet is still evolving and measurements allow us to better understand its evolution. In [LIJM+10], Craig Labovitz and his colleagues used extensive measurements to show the growing importance of large content providers such as google, yahoo or content distribution networks such as Akamai. This paper forced researchers to reconsider some of their assumptions on the organisation of the Internet with fully meshed Tier-1 ISPs serving Tier-2 ISPs that are serving Tier-3 ISPs and content providers. [LIJM+10] showed that many of the traffic sources are directly connected to the last mile ISPs.
In a recent paper [ACF+12] presented during SIGCOMM 2012 , Bernard Ager and his colleagues used statistics collected at one of the largest Internet eXchange Points (IXP) in Europe. IXPs are locations were various Internet providers place routers to exchange traffic via a switched network. While the first IXPs where simple Ethernet switches in the back of a room, current IXPs are huge. For example, AMS-IX gathers more than 500 different ISPs on more than 800 different ports. As of this writing, its peak traffic has been larger than 1.5 Terabits per second ! IXPs play a crucial role in the distribution of Internet traffic and in particular in Europe where there are many large IXPs. [ACF+12] highlights many unknown factors about these IXPs such as the large number of peering links that exist on each ISP, the application mix or the traffic matrices. [ACF+12] will become a classic paper for those willing to understand the organisation of the Internet.
| [ACF+12] | (1, 2, 3) B. Ager, N. Chatzis, A. Feldmann, N. Sarrar, S. Uhlig, and W. Willinger. Anatomy of a large european ixp. In SIGCOMM 2012. Helsinki, Finland, April 2012. |
| [BNC11] | H. Babiker, I. Nikolova, and K. Chittimaneni. Deploying ipv6 in the google enterprise network. lessons learned. In USENIX LISA 2011. 2011. URL: http://static.usenix.org/events/lisa11/tech/full_papers/Babiker.pdf. |
| [LIJM+10] | (1, 2) Craig Labovitz, Scott Iekel-Johnson, Danny McPherson, Jon Oberheide, and Farnam Jahanian. Internet inter-domain traffic. SIGCOMM Comput. Commun. Rev., 41(4):–, August 2010. URL: http://dl.acm.org/citation.cfm?id=2043164.1851194. |
| [TLB+12] | T. Tsou, D. Lopez, J. Brzozowski, C. Popoviciu, C. Perkins, and D. Cheng. Exploring ipv6 deployment in the enterprise: experiences of the it department of futurewei technologies. IETF Journal, June 2012. URL: http://www.internetsociety.org/articles/exploring-ipv6-deployment-enterprise-experiences-it-department-futurewei-technologies. |
Deploying IPv6 in enterprise networks
The Internet is slowly moving towards IPv6. IPv6 traffic is now growing and more and more enterprise networks are migrating to IPv6. Migrating all enterprise networks to support IPv6 will be slow. One of the main difficulties faced by network administrators when migrating to IPv6 is that it is not sufficient to migrate the hosts (most operating systems already support IPv6) and configure the routers are dual-stack. All devices that process packets need to be verified or updated to support IPv6.
Network administrators who perform IPv6 migrations sometimes document their findings in articles. These articles are interesting for other networ administrators who might face similar problems.
In [BNC11], google engineers explain the issues that they faced when migrating the enterprise networks of google premisses to support IPv6. Surprisingly, one of the main bottleneck in this migration was the support of IPv6 on the transparent proxies that they use to accelerate web access.
In [TLB+12], researchers and network administrators from FutureWei report their experience in adding IPv6 connectivity to their network. The report is short on enabling IPv6, but discusses more recent solutions that are being developed within the IETF.
Another interesting viewpoint is the one discussed in RFC 6585. In this RFC, Jari Arkko and Ari Keranen report all the issues that they faced while running an IPv6 only network in their lab and using it on a daily basis to access the Internet which is still mainly IPv4.
| [BNC11] | H. Babiker, I. Nikolova, and K. Chittimaneni. Deploying ipv6 in the google enterprise network. lessons learned. In USENIX LISA 2011. 2011. URL: http://static.usenix.org/events/lisa11/tech/full_papers/Babiker.pdf. |
| [TLB+12] | T. Tsou, D. Lopez, J. Brzozowski, C. Popoviciu, C. Perkins, and D. Cheng. Exploring ipv6 deployment in the enterprise: experiences of the it department of futurewei technologies. IETF Journal, June 2012. URL: http://www.internetsociety.org/articles/exploring-ipv6-deployment-enterprise-experiences-it-department-futurewei-technologies. |