Archive for the ‘varnish’ Category

hitch-1.0.0-beta for Fedora and EPEL

Friday, June 26th, 2015

The Varnish project has a new little free software baby arriving soon: Hitch, a scalable TLS proxy. It will also be made available with support by Varnish Software as part of their Varnish Plus product.

A bit of background:

Varnish is a high-performance HTTP accelerator, widely used over the Internet. To use varnish with https, it is often fronted by other general http/proxy servers like nginx or apache, though a more specific proxy-only high-performance tool would be preferable. So they looked at stud.

hitch is a fork of stud. The fork is maintained by the Varnish development team, as stud seems abandoned by its creators, after the project was taken over by Google, with no new commits after 2012.

I wrapped hitch for fedora, epel6 and epel7, and submitted them for Fedora and EPEL. Please test the latest builds and add feedback: https://admin.fedoraproject.org/updates/search/hitch . The default config is for a single instance of hitch.

The package has been reviewed and was recently accepted into Fedora and EPEL (bz #1235305). Update august 2015: Packages are pushed for testing. They will trickle down to stable eventually.

Note that there also exists as a fedora package of the (old) version of stud. If you use stud on fedora and want to test hitch, the two packages may coexist, and should be able to install in parallel.

To test hitch in front of varnish, in front of apache, you may do something like this (tested on el7):

  • Install varnish, httpd and hitch
      sudo yum install httpd varnish
      sudo yum --enablerepo=epel-testing install hitch || sudo yum --enablerepo=updates-testing install hitch
    
  • Start apache
      sudo systemctl start httpd.service
    
  • Edit the varnish config to point to the local httpd, that is, change the default backend definition in /etc/varnish/default.vcl , like this:
      backend default {
        .host = "127.0.0.1";
        .port = "80";
      }
    
  • Start varnish
      sudo systemctl start varnish.service
    
  • Add an ssl certificate to the hitch config. For a dummy certificate,
    the example.com certificate from the hitch source may be used:

      sudo wget -O /etc/pki/tls/private/default.example.com.pem http://users.linpro.no/ingvar/varnish/hitch/default.example.com.pem
    
  • Edit /etc/hitch/hitch.conf. Change the pem-file option to use that cert
      pem-file = "/etc/pki/tls/private/default.example.com.pem"
    
  • Start hitch
      sudo systemctl start hitch.service
    
  • Open your local firewall if necessary, by something like this:
      sudo firewall-cmd --zone=public --add-port=8443/tcp
    
  • Point your web browser to https://localhost:8443/ . You should be greeted with a warning about a non-official certificate. Past that, you will get the apache frontpage through varnish and hitch.

    Enjoy, and let me hear about any interesting test results.

    Ingvar

    Varnish Cache is powerful and feature rich front side web cache. It is also very fast, that is, Fast as in on steroids, and powered by The Dark Side of the Force.

    Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at www.redpill-linpro.com.

  • varnish-4.0.3 for Fedora and EPEL

    Thursday, March 5th, 2015

    varnish-4.0.3 was released recently. I have wrapped packages for Fedora and EPEL, and requested updates for epel7, f21 and f22. They will trickle down as stable updates within some days. I have also built packages for el6, and after som small patching, even for el5. These builds are based on the Fedora package, but should be only cosmetically different from the el6 and el7 packages available from http://varnish-cache.org/.

    Also note that Red Hat finally caught up, and imported the necessary selinux-policy changes for Varnish from fedora into el7. With selinux-policy-3.13.1-23.el7, Varnish starts fine in enforcing mode. See RHBA-2015-0458.

    My builds for el5 and el6 are available here: http://users.linpro.no/ingvar/varnish/4.0.3/. Note that they need other packages from EPEL to work.

    Update 1: I also provide an selinux module for those running varnish-4.0 on el6. It should work for all versions of varnish-4.0, including mine and the ones from varnish-cache.org.

    Update 2: Updated builds with a patch for bugzilla ticket 1200034 are pushed for testing in f21, f22 and epel7. el5 and el6 builds are available on link above.

    Enjoy.

    Ingvar

    Varnish Cache is powerful and feature rich front side web cache. It is also very fast, that is, Fast as in on steroids, and powered by The Dark Side of the Force.

    Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at redpill-linpro.com.

    rpm packages of vmod-ipcast

    Thursday, January 8th, 2015

    Still on varnish-3.0? Missing the ability to filter X-Forwarded-For through ACLs? Use vmod ipcast by Lasse Karstensen.

    I cleaned up and rolled an rpm package of vmod-ipcast-1.2 for varnish-3.0.6 on el6. It’s available here: http://users.linpro.no/ingvar/varnish/vmod-ipcast/.

    Note that the usage has changed a bit since the last version. You are now longer permitted to change client.ip (and that’s probably a good thing). Now it’s called like this, returning an IP address object:

    ipcast.ip("string","fallback_ip");

    If the string does not resemble an IP address, the fallback ip is returned. Note that if the fallback ip is an unvalid address, varnishd will crash!

    So, if you want to filter X-Forwarded-For through an ACL, you would something like this:

    import ipcast;
    sub vcl_recv {
       # Add some code to sanitize X-Forwarded-For above here, so it resembles one single IP address
       if ( ipcast.ip(req.http.X-Forwarded-For, "198.51.100.255") ~ someacl ) {
         # Do something special
       }
    }

    And that’s all for today.

    Varnish Cache is powerful and feature rich front side web cache. It is also very fast, that is, Fast as in on steroids, and powered by The Dark Side of the Force.

    Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at redpill-linpro.com.

    varnish-3.0.2 for fedora

    Thursday, March 8th, 2012

    I finally got around to wrap up varnish-3.0.2 for fedora 17 and rawhide. Please test and report karma.

    In this release, I have merged changes from the upstream rpm, and added native systemd support for f17 and rawhide. It also builds nicely for epel5 and epel6, providing packages quite similar to those available from the varnish project repo.

    As epel does not allow changes in a package API after release, varnish-3.0.2 won’t be available through epel5 or epel6, so use the varnish project repo, or my precompiled packages for epel 4, 5 and 6 available here.

    As always, feedback is very welcome.

    rpm packages of varnish-3.0.0

    Friday, August 26th, 2011

    Varnish is a state of the art http accelerator, or frontside cache, if you like.

    varnish-3.0.0 was released some weeks ago. I have built packages for Fedora and epel4/5/6. Packages may be found at the usual http://users.linpro.no/ingvar/varnish/. The rhel packages require some dependencies pulled from epel.

    Varnish Software produces their own packages, based on the specfile I maintain for Fedora. The changes from their rpm spec are mostly cosmetic to fit better to Fedora’s packaging standards.

    The usage of Varnish revisited

    Wednesday, June 22nd, 2011

    Varnish is a high-performance HTTP accelerator, or frontside cache if you like. Working with Varnish is part of my day job. Among other things, I maintain the packages for Fedora and EPEL.

    To celebrate the release of Varnish version 3, I decided to poke around lists again, to look for Varnish in common use.

    This is more or less a repost, with updated numbers. There is no deep magic here. I just parse some of the available top lists that I know of, and peek at the HTML headers of the sites that are listed. If there are subsites linked from the front page of the site, I scan them too. Subsites with a Varnish match are shown in parenthesis in the results.

    For the Nordic countries, I have quite good lists, that is, upload result lists from the probably most visited media sites in the respective countries. Remember of course, that these are generally pay-to-be-included lists, and there may exist sites with far more hits than the ones listed.

    For a global overview, I have used Alexa and Google’s Top 1000 lists.

    Now for the results. Varnish is sponsored by large Norwegian sites, so it is no big surprise that there are a lot of hits in Norway. Of the TNS Gallup top list, Varnish runs at stunning 51 of the top 100 sites. That’s 15 up since my last probe.

    For Denmark, I use FDIM‘s list. Sorted on page hits, we now rule 15 of the top 100, and 29 in the top 200.

    For Sweden, I use the KIA Index list. It is a bit harder to parse, but I think I got it right. Sorting on page hits, in the top 100, we are up to 13, and in the top 200, we find 26 sites running Varnish.

    Iceland is finally on the list, with one single item on Modernus’ top list. The lucky site is www.vb.is, which looks like a financial publication.

    I haven’t got results for Finland yet, I have to rebuild my parser, it seems.

    For what it’s worth, I’ll toss in Germany as well. Four sites in the Google’s top 100 sites for Germany, and 13 on the Netcraft toolbar users’ list sounds like a good start to me. And Der Spiegel and Der Zeit are well-known publications.

    For the Alexa’s World top 500 list, we have 17 instances of Varnish in the top 500. That is the same result as last year. Still no World domination. Google shows us a similar result, with 32 sites running Varnish in its top 1000 list.

    We know Facebook, the World’s most visited site, runs Varnish for several of their services, but it is hidden from my probes.

    All the gory details are available here.

    Other more or less worth mentioned sites that has been reported to use Varnish but does not show up in my lists, may be Slashdot, The Pirate Bay, e.Republik, WOWwiki, Globo.com, PCWelt.de, BlackPlanet, funnyordie.com, n-tv.de, 20minutos.es, theglobeandmail.com and hackint0sh.org, to name a few.

    Do you know of other famous sites running Varnish? Use the comments.

    rpm packages of varnish-2.1.5

    Friday, February 4th, 2011

    Varnish is a state of the art http accelerator, or frontside cache, if you like.

    varnish-2.1.5 was released the other day. I have updated my packages in Fedora and epel6. Builds for rhel4 and rhel5 may be found at the usual http://users.linpro.no/ingvar/varnish/. The rhel5 packages require some dependencies pulled from epel5.

    Varnish Software produces their own packages, based on the specfile I maintain for Fedora. The only important change is that my spins link against a system installed jemalloc, instead of the one provided with the source. This gives us the opportunity to update jemalloc to the latest version without recompiling varnish.

    I also build packages for rhel4. While probably unsupported from Varnish Software, it compiles and runs the test suite after some small fixes to the build. jemalloc packages are provided as well.

    Updated packages of varnish-2.1.4 revisited

    Friday, November 5th, 2010

    An extra update of the varnish-2.1.4 packages was pushed to Fedora (rawhide, f14, f13, epel6) yesterday, including a bugfix.

    I also finally got around and fixed make-initscript-reload-do-load-and-switch-vcl, by popular request. I had a look at the script in Debian, but found it too magical. My version uses explicit configuration in /etc/sysconfig/varnish.

    Updated packages for RHEL4 and 5 available at the usual place.

    Most of this will end up upstream rsn, srsly.

    Updated packages of varnish-2.1.4

    Friday, October 29th, 2010

    Varnish is a state of the art HTTP accelerator, used to keep sites like Twitter, Wikia and Facebook up to speed.

    Varnish version 2.1.4 was released the other day. Users of RHEL5 and clones may now use Varnish Software’s own repo directly.

    I have done a few improvements over the specfile in the released version. They will be synced upstream soon. I have built updated packages for Fedora Rawhide, 14, 13, and epel6. You can find them in their respective testing repos from where they will eventually trickle down to stable. I have also built updated packages for el4 and el5. You can find them at http://users.linpro.no/ingvar/varnish

    If you use Varnish Software’s own repo for rhel5, the only changes to these newer packages are related to documentation and the building process, and there is no need to upgrade to these packages for stability or bugfix reasons.

    Accelerating the Internet (or actually, Squid) with Varnish

    Wednesday, May 26th, 2010

    Squid is an old, working-horse, caching proxy server that can be configurated to act as a reverse proxy. Varnish is the opposite, it’s an extremely fast http accellerator that’s configurated to be, well, just that. So I thought, just for the fun of it, what about configurating Varnish to cache the Internet for me, that is, use it as a general forwarding caching proxy server.

    Obviously, we can’t define varnish backends for the entire world. But Squid can do that. So I used our corporate Squid proxy, and put a local varnish cache in front of it. The vcl is very simple:

    backend default {
      # This is squidbox
      .host = "11.22.33.44";
      .port = "3128";
    }
    

    That’s it, actually. Start up varnish, and use that varnish instance’s address and http port as proxy in your web browser.

    Then, using an ugly little perl script “proxytest” for testing, we found these quite interesting results:

    $ for i in "" squidbox:3128 varnishbox:6081; do ./proxytest $i http://www.slashdot.org/ 10; done
    1.0836112s
    0.3773585s
    0.0352446s
    

    Lesson learned: Varnish is some 10 times faster than Squid, when caching the Internet!

    With thanks to eric for playing with settings.