The usage of Varnish revisited

Varnish is a high-performance HTTP accelerator. Working with Varnish is part of my day job. Among other things, I maintain the packages for Fedora and EPEL.

Some months have passed, and it is time to run my poking scripts again, looking for sites that run Varnish. There is no deep magic here. I just parse the available top lists that I know of, and peek at the HTML headers of the sites that are listed. If there are subsites linked from the front page of the site, I scan them too. This means that twitter.com shows up, though Twitter only runs Varnish on its search site. Subsites with a Varnish match are shown in parenthesis in the results.

For the Nordic countries, I have found quite good lists, that is, upload result lists from the probably most visited media sites in the respective countries. Remember of course, that these are generally pay-to-be-included lists, and there may exist sites with far more hits than the ones listed.

For the rest, I have used Alexa and Netcraft’s toolbar users’ lists with main sites summed up. With a few exceptions, they are a bit less interesting, as global .com and .net sites tend to cover most of the top 100 entries anyway.

Now for the results. Varnish is sponsored by large Norwegian sites, so it is no big surprise that there are a lot of hits in Norway. Of the TNS Gallup top list, Varnish runs at some every third site.

For Denmark, I use FDIM‘s list. The results are more or less the same as in my last run. For Finland, I use TNS’ numbers again. There are +1 site in the TNS top list. For Sweden, I use the KIA Index list. There is -1 Varnish site there since last.

For a global overview, I have used Alexa’s top 500 list. This is the first time I have got any hit on that list. That is interesting. Twitter is probably the most well known site outside the geekosphere running Varnish now.

The whole World: Alexa's global top 500 list
Place  39 Varnish running on twitter.com (search.twitter.com)         
Place  83 Varnish running on globo.com (globo.com)                       
Place 166 Varnish running on ifeng.com (ifeng.com)                         
Place 194 Varnish running on people.com.cn (bbs1.people.com.cn)         
Place 227 Varnish running on hulu.com (hulu.com)                      
Place 282 Varnish running on wikia.com (wikia.com)                          
Place 467 Varnish running on perezhilton.com (perezhilton.com)

Probing the rest of the World would probably take too much time, so I have done a scan of Alexa’s and Netcraft’s lists for their toolbar users in the European countries.

All the gory details are available here.

Other worth mentioned sites that uses Varnish that does not show up on my Alexa’s global list may be Slashdot, WAT TV, The Pirate Bay, JDownloader, e.Republik, WOWwiki, the Brazilian news site Globo.com, the German IDG site PC Welt, and the Chinese computer tech site Csdn.net.

Do you know of other famous sites running Varnish? Use the comments.

Tags:

10 Responses to “The usage of Varnish revisited”

  1. Jason Cook says:

    Wowwiki is part of wikia and generally counted under our numbers. You can find the vcl we use for varnish at

    http://code.google.com/p/wikia/source/browse/#svn/utils/varnishhtcpd

    The vcl also includes Artur’s hack for a geoip service on top of the maxmind database implemented as an error handler.

    Jason
    Wikia Ops

  2. Hi, we use Varnish on several of our sites here (the largest of which is BlackPlanet.com). We’re also looking to roll it out soon across our other sites – by the end of June 2009 we should be running it in some place on pretty much every site we have.

    One reason we don’t show up now, and probably other sites don’t as often, is that we use a CDN in front of our varnish caches, which are in front of the static servers and semi-static servers. (By semi-static servers I mean servers that dynamically generate ‘static’ content, such as image-resizing.)

    One question: will epel 5/ x86_64 be getting updated w/ varnish 2.0.4 sometime soon?

  3. ingvar says:

    As 2.0.4 is not a critical security update, it is still in the testing tree, waiting for the next pull. Updates are pulled from testing about once a month. If you want to start using the 2.0.4 EPEL package now, you can get it from the testing tree here:

    http://download.fedora.redhat.com/pub/epel/testing/5/x86_64/

    Ingvar

  4. tilo says:

    Big German news site:

    curl -I http://www.n-tv.de

  5. karl says:

    Video hosting site:

    curl -I http://www.funnyordie.com

  6. Not a major site in terms of traffic, but we just launched the new http://www.norden.org last week, which now uses Varnish (and Plone).

  7. [...] Some months have passed, and it is time to run my poking scripts again, looking for sites that run Varnish. There is no deep magic here. I just parse the available top lists that I know of, and peek at the HTML headers of the sites that are listed. If there are subsites linked from the front page of the site, I scan them too. This means that twitter.com shows up, though Twitter only runs Varnish on its search site. Subsites with a Varnish match are shown in parenthesis in the results. [...]

  8. [...] Some months have passed, and it is time to run my poking scripts again, looking for sites that run Varnish. There is no deep magic here. I just parse the available top lists that I know of, and peek at the HTML headers of the sites that are listed. If there are subsites linked from the front page of the site, I scan them too. This means that twitter.com shows up, though Twitter only runs Varnish on its search site. Subsites with a Varnish match are shown in parenthesis in the results. [...]

  9. Jose says:

    Globo.com is a brazilian site, not spanish

  10. ingvar says:

    Jose, thank you, I’ll update the posting.

    Ingvar

Leave a Reply