Archive for the ‘varnish’ Category

rpm packages of vmod-ipcast

Thursday, January 8th, 2015

Still on varnish-3.0? Missing the ability to filter X-Forwarded-For through ACLs? Use vmod ipcast by Lasse Karstensen.

I cleaned up and rolled an rpm package of vmod-ipcast-1.2 for varnish-3.0.6 on el6. It’s available here: http://users.linpro.no/ingvar/varnish/vmod-ipcast/.

Note that the usage has changed a bit since the last version. You are now longer permitted to change client.ip (and that’s probably a good thing). Now it’s called like this, returning an IP address object:

ipcast.ip("string","fallback_ip");

If the string does not resemble an IP address, the fallback ip is returned. Note that if the fallback ip is an unvalid address, varnishd will crash!

So, if you want to filter X-Forwarded-For through an ACL, you would something like this:

import ipcast;
sub vcl_recv {
   # Add some code to sanitize X-Forwarded-For above here, so it resembles one single IP address
   if ( ipcast.ip(req.http.X-Forwarded-For, "198.51.100.255") ~ someacl ) {
     # Do something special
   }
}

And that’s all for today.

Varnish Cache is powerful and feature rich front side web cache. It is also very fast, that is, Fast as in on steroids, and powered by The Dark Side of the Force.

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at redpill-linpro.com.

varnish-3.0.2 for fedora

Thursday, March 8th, 2012

I finally got around to wrap up varnish-3.0.2 for fedora 17 and rawhide. Please test and report karma.

In this release, I have merged changes from the upstream rpm, and added native systemd support for f17 and rawhide. It also builds nicely for epel5 and epel6, providing packages quite similar to those available from the varnish project repo.

As epel does not allow changes in a package API after release, varnish-3.0.2 won’t be available through epel5 or epel6, so use the varnish project repo, or my precompiled packages for epel 4, 5 and 6 available here.

As always, feedback is very welcome.

rpm packages of varnish-3.0.0

Friday, August 26th, 2011

Varnish is a state of the art http accelerator, or frontside cache, if you like.

varnish-3.0.0 was released some weeks ago. I have built packages for Fedora and epel4/5/6. Packages may be found at the usual http://users.linpro.no/ingvar/varnish/. The rhel packages require some dependencies pulled from epel.

Varnish Software produces their own packages, based on the specfile I maintain for Fedora. The changes from their rpm spec are mostly cosmetic to fit better to Fedora’s packaging standards.

The usage of Varnish revisited

Wednesday, June 22nd, 2011

Varnish is a high-performance HTTP accelerator, or frontside cache if you like. Working with Varnish is part of my day job. Among other things, I maintain the packages for Fedora and EPEL.

To celebrate the release of Varnish version 3, I decided to poke around lists again, to look for Varnish in common use.

This is more or less a repost, with updated numbers. There is no deep magic here. I just parse some of the available top lists that I know of, and peek at the HTML headers of the sites that are listed. If there are subsites linked from the front page of the site, I scan them too. Subsites with a Varnish match are shown in parenthesis in the results.

For the Nordic countries, I have quite good lists, that is, upload result lists from the probably most visited media sites in the respective countries. Remember of course, that these are generally pay-to-be-included lists, and there may exist sites with far more hits than the ones listed.

For a global overview, I have used Alexa and Google’s Top 1000 lists.

Now for the results. Varnish is sponsored by large Norwegian sites, so it is no big surprise that there are a lot of hits in Norway. Of the TNS Gallup top list, Varnish runs at stunning 51 of the top 100 sites. That’s 15 up since my last probe.

For Denmark, I use FDIM‘s list. Sorted on page hits, we now rule 15 of the top 100, and 29 in the top 200.

For Sweden, I use the KIA Index list. It is a bit harder to parse, but I think I got it right. Sorting on page hits, in the top 100, we are up to 13, and in the top 200, we find 26 sites running Varnish.

Iceland is finally on the list, with one single item on Modernus’ top list. The lucky site is www.vb.is, which looks like a financial publication.

I haven’t got results for Finland yet, I have to rebuild my parser, it seems.

For what it’s worth, I’ll toss in Germany as well. Four sites in the Google’s top 100 sites for Germany, and 13 on the Netcraft toolbar users’ list sounds like a good start to me. And Der Spiegel and Der Zeit are well-known publications.

For the Alexa’s World top 500 list, we have 17 instances of Varnish in the top 500. That is the same result as last year. Still no World domination. Google shows us a similar result, with 32 sites running Varnish in its top 1000 list.

We know Facebook, the World’s most visited site, runs Varnish for several of their services, but it is hidden from my probes.

All the gory details are available here.

Other more or less worth mentioned sites that has been reported to use Varnish but does not show up in my lists, may be Slashdot, The Pirate Bay, e.Republik, WOWwiki, Globo.com, PCWelt.de, BlackPlanet, funnyordie.com, n-tv.de, 20minutos.es, theglobeandmail.com and hackint0sh.org, to name a few.

Do you know of other famous sites running Varnish? Use the comments.

rpm packages of varnish-2.1.5

Friday, February 4th, 2011

Varnish is a state of the art http accelerator, or frontside cache, if you like.

varnish-2.1.5 was released the other day. I have updated my packages in Fedora and epel6. Builds for rhel4 and rhel5 may be found at the usual http://users.linpro.no/ingvar/varnish/. The rhel5 packages require some dependencies pulled from epel5.

Varnish Software produces their own packages, based on the specfile I maintain for Fedora. The only important change is that my spins link against a system installed jemalloc, instead of the one provided with the source. This gives us the opportunity to update jemalloc to the latest version without recompiling varnish.

I also build packages for rhel4. While probably unsupported from Varnish Software, it compiles and runs the test suite after some small fixes to the build. jemalloc packages are provided as well.

Updated packages of varnish-2.1.4 revisited

Friday, November 5th, 2010

An extra update of the varnish-2.1.4 packages was pushed to Fedora (rawhide, f14, f13, epel6) yesterday, including a bugfix.

I also finally got around and fixed make-initscript-reload-do-load-and-switch-vcl, by popular request. I had a look at the script in Debian, but found it too magical. My version uses explicit configuration in /etc/sysconfig/varnish.

Updated packages for RHEL4 and 5 available at the usual place.

Most of this will end up upstream rsn, srsly.

Updated packages of varnish-2.1.4

Friday, October 29th, 2010

Varnish is a state of the art HTTP accelerator, used to keep sites like Twitter, Wikia and Facebook up to speed.

Varnish version 2.1.4 was released the other day. Users of RHEL5 and clones may now use Varnish Software’s own repo directly.

I have done a few improvements over the specfile in the released version. They will be synced upstream soon. I have built updated packages for Fedora Rawhide, 14, 13, and epel6. You can find them in their respective testing repos from where they will eventually trickle down to stable. I have also built updated packages for el4 and el5. You can find them at http://users.linpro.no/ingvar/varnish

If you use Varnish Software’s own repo for rhel5, the only changes to these newer packages are related to documentation and the building process, and there is no need to upgrade to these packages for stability or bugfix reasons.

Accelerating the Internet (or actually, Squid) with Varnish

Wednesday, May 26th, 2010

Squid is an old, working-horse, caching proxy server that can be configurated to act as a reverse proxy. Varnish is the opposite, it’s an extremely fast http accellerator that’s configurated to be, well, just that. So I thought, just for the fun of it, what about configurating Varnish to cache the Internet for me, that is, use it as a general forwarding caching proxy server.

Obviously, we can’t define varnish backends for the entire world. But Squid can do that. So I used our corporate Squid proxy, and put a local varnish cache in front of it. The vcl is very simple:

backend default {
  # This is squidbox
  .host = "11.22.33.44";
  .port = "3128";
}

That’s it, actually. Start up varnish, and use that varnish instance’s address and http port as proxy in your web browser.

Then, using an ugly little perl script “proxytest” for testing, we found these quite interesting results:

$ for i in "" squidbox:3128 varnishbox:6081; do ./proxytest $i http://www.slashdot.org/ 10; done
1.0836112s
0.3773585s
0.0352446s

Lesson learned: Varnish is some 10 times faster than Squid, when caching the Internet!

With thanks to eric for playing with settings.

RPM packages of varnish-2.1.2

Wednesday, May 5th, 2010

Varnish is a high performance web accelerator. For more information about varnish, see the project homepage.

I had just added varnish-2.1.1 to rawhide when the upstream team released a bugfix update varnish-2.1.2.

Rawhide is updated. I also built packages for RHEL5. You can download them at http://users.linpro.no/ingvar/varnish/2.1.2/, or from SourceForge.

The usage of varnish revisited

Thursday, January 21st, 2010

Varnish is a high-performance HTTP accelerator. Working with Varnish is part of my day job. Among other things, I maintain the packages for Fedora and EPEL.

This is more or less a repost, with updated numbers.

Some months have passed, and it is time to run my poking scripts again, looking for sites that run Varnish. There is no deep magic here. I just parse the available top lists that I know of, and peek at the HTML headers of the sites that are listed. If there are subsites linked from the front page of the site, I scan them too. This means that twitter.com shows up, though Twitter only runs Varnish on its search site. Subsites with a Varnish match are shown in parenthesis in the results.

For the Nordic countries, I have found quite good lists, that is, upload result lists from the probably most visited media sites in the respective countries. Remember of course, that these are generally pay-to-be-included lists, and there may exist sites with far more hits than the ones listed.

For a global overview, I have used Alexa.

Now for the results. Varnish is sponsored by large Norwegian sites, so it is no big surprise that there are a lot of hits in Norway. Of the TNS Gallup top list, Varnish runs at 36 of the top 100 sites. That’s 3 up since my last probe.

For Denmark, I use FDIM‘s list. From May, we are up from 3 to 11 sites in the top 100. For Finland, I use TNS’ numbers again. No changes there. For Sweden, I use the KIA Index list. I now probe the 200 top sites, so there are several more varnish sites on the list. In the top 100, we are up from 8 to 9.

For the Alexa’s World top 500 list, I have tweaked my filters a bit, and the list is up from 7 to 17 sites since my last probe in May. World Domination, here we come!

The whole World: Alexa's global top 500 list
Place  12 Varnish running on twitter.com (http://integratedsearch.twitter.com/search.html)
Place  47 Varnish running on photobucket.com (http://blog.photobucket.com/)
Place  90 Varnish running on mixi.jp (http://img.mixi.jp/static/css/basic/logout_quirks) (and others)
Place 101 Varnish running on weather.com (http://www.weather.com/) (weather.com)
Place 107 Varnish running on globo.com (globo.com) (http://www.globo.com/)
Place 111 Varnish running on ifeng.com (http://big5.ifeng.com/gate/big5/www)
Place 138 Varnish running on answers.com (http://www.answers.com/) (answers.com)
Place 170 Varnish running on orange.fr (http://r.orange.fr/r/Eorangepublicite)
Place 179 Varnish running on hulu.com (hulu.com)
Place 199 Varnish running on wikia.com (wikia.com)
Place 213 Varnish running on xinhuanet.com (http://big5.xinhuanet.com/gate/big5/www)
Place 290 Varnish running on people.com.cn (http://bbs1.people.com.cn/boardList) (and others)
Place 344 Varnish running on tuenti.com (http://estaticos1.tuenti.com/layout/web2/images/favicon)
Place 428 Varnish running on chinanews.com.cn (http://www.chinanews.com.cn/test/index_back_d.html) (and others)
Place 433 Varnish running on mercadolivre.com.br (http://veiculos.mercadolivre.com.br/) (and others)
Place 447 Varnish running on mercadolibre.com.mx (http://tendencias.mercadolibre.com.mx/) (and others)
Place 456 Varnish running on break.com (break.com)

In my last probe, I poked sites all over Europe. With a few exceptions, that was a bit less interesting. Global .com and .net sites tend to cover most of the top 100 entries, as I had only toolbar lists, and it’s not that spectacular that for example people in Serbia and Monte Negro are browsing Twitter, like the rest of the World does. So I have skipped other countries. If you know of any good top list for you country that is not toolbar based, please let me know.

All the gory details are available here.

Other more or less worth mentioned sites that is reported to use Varnish but does not show up in my lists, may be Slashdot, WAT TV, The Pirate Bay, JDownloader, e.Republik, WOWwiki, Globo.com, PCWelt.de, BlackPlanet, funnyordie.com, n-tv.de and 20minutos.es to name a few.

Do you know of other famous sites running Varnish? Use the comments.