Updated packages of varnish-6.0.2 matching vmods, for el6 and el7

November 28th, 2018

Recently, the Varnish Cache project released an updated upstream version 6.0.2 of Varnish Cache. This is a maintenance and stability release of varnish 6.0, which you may consider as the current “LTS” branch of varnish. I have updated the fedora rawhide package, and also updated the varnish 6.0 copr repo with packages for el6 and el7 based on the fedora package. A selection of matching vmods is also included in the copr repo.

Packages are available at https://copr.fedorainfracloud.org/coprs/ingvar/varnish60/

The following vmods are available:

Included in varnish-modules:
vmod-bodyaccess
vmod-cookie
vmod-header
vmod-saintmode
vmod-tcp
vmod-var
vmod-vsthrottle
vmod-xkey

Packaged separately:
vmod-curl
vmod-digest
vmod-geoip
vmod-memcached
vmod-querystring
vmod-uuid

Please test and report bugs. If there is enough interest, I may consider pushing these to fedora as well.

Varnish Cache is a powerful and feature rich front side web cache. It is also very fast, and that is, fast as in powered by The Dark Side of the Force. On steroids. And it is Free Software.

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, data center, and cloud, contact us at www.redpill-linpro.com.

Updated packages of varnish-6.0.1 with matching vmods, for el6 and el7

October 11th, 2018

Recently, the Varnish Cache project released an updated upstream version 6.0.1 of Varnish Cache. This is a maintenance and stability release of varnish 6.0. I have updated the fedora rawhide package, and also updated the varnish 6.0 copr repo with packages for el6 and el7 based on the fedora package. A selection of matching vmods is also included in the copr repo.

Packages are available at https://copr.fedorainfracloud.org/coprs/ingvar/varnish60/

The following vmods are available:

Included in varnish-modules:
vmod-bodyaccess
vmod-cookie
vmod-header
vmod-saintmode
vmod-tcp
vmod-var
vmod-vsthrottle
vmod-xkey

Packaged separately:
vmod-curl
vmod-digest
vmod-geoip
vmod-memcached
vmod-querystring
vmod-uuid

Please test and report bugs. If there is enough interest, I may consider pushing these to fedora as well.

Varnish Cache is a powerful and feature rich front side web cache. It is also very fast, and that is, fast as in powered by The Dark Side of the Force. On steroids. And it is Free Software.

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, data center, and cloud, contact us at www.redpill-linpro.com.

Five valuable considerations while moving services to the cloud

August 17th, 2018

Also posted at Redpill Linpro’s Operations and DevOps blog.

Going cloud is the new black, and has been for a few years already. Customers ask us: What is your Cloud Strategy? Can you help us moving to the cloud? Are your services Cloud Compliant? (Is that even a valid term?)

Burning the bridges and moving everything to the cloud may sound compelling, and public and private cloud services may be quite rewarding. No hardware responsibility. Pay for what you actually use, not what you may use. Scale up. Scale down. Scale out. Infinite storage. Multi data center. Multi location. High availability. Global location based load balancing. Functions as a service. Databases as a service. Anything as a service! Why wait?

All players in the IT field should consider cloud technologies, but while planning the setup of a new stack, there are issues to take into consideration. Not all services suits any cloud setup. Here are a few real-life scenarios:

1. Apps in the cloud, cache locally

A media house had reimplemented their production stack at a public cloud provider. The apps worked great, the developers were happy, the performance was satisfactory, and the users content. But the costs went up, as the cloud provider’s network traffic toll was quite high. Using a CDN could be a solution, but was considered too costly and unnecessary, as most users were local to a few central locations.

Keeping the stack in the cloud, and adding local web caches to our data centers ended up being a good solution, keeping high traffic volumes on low cost lines, while sending backend traffic to the cloud.

For high volume sites, consider caching in local data centers

2. Apps locally, cache in the cloud

Another media house had a classic in-house publication system, but had readers around the globe. The volume was quite low, but with high quality content to paying customers worldwide. Users in South-East Asia or the West Coast of USA got high latency and slow content loading. Actually building a scaled-down CDN service, we put cache servers in public cloud provider locations close to the users, and got happy readers. Using the Varnish Plus product, local users got cached content even though protected by a paywall.

For low volume sites, consider local caching in the cloud

3. Legal storage

A media content provider was moving their services and content library to the cloud. They considered using their existing platform for building a library product, delivering storage and search for images and video data, directed at public service usage, and asked for advice. Their platform used Amazon’s AWS S3 for storage and AWS Glacier for backup. Then it occurred that storing data for public services abroad might have legal consequences, and we had to search legal advice on document storage. A proposed solution was to use Ceph based S3 compatible storage services in our data centers within Norway. With this solution, network traffic expenses became a factor.

Storing data abroad may have consequences

4. Anything as a service, cost by call

Skipping the overhead of creating and maintaining virtual machines is tempting. Public cloud providers offer Database as a Service variants compatible with well-known databases, including most SQL and noSQL variants. Include Functions as a Service, and you may be able to build a complete serverless solution including data backend and APIs.

A customer built a system using cloud provided services for database and message queue. It worked flawlessly for development and test, but when adding production traffic to the solution, cost became a large issue, as the services were tolled by request. Biting the bullet, they admitted the work of maintaining virtual machines for some of the services, paying for cpu, memory, network, and man hours.

When using anything as a service, consider traffic driven costs against the overhead of server management.

Also, «serverless» computing is of course a lie. The servers are there, the interface just hides the database setup. In a test using one public cloud provider’s MySQL variant, a multi database slave instance setup was shown to be quite non-resilient against sudden death, with downtime for the service while the slaves were resynced.

There is no such thing as «serverless» computing, just another level of abstraction in front of another computer

5. Trust the cloud provider, the cloud provider is your friend

A well-known story tells how a complete site was taken down by a public cloud provider’s robots looking for suspicious activity. With customer chat down, and no on-call service available, the developers were forced to handle the incident by waking up the CFO, and send credit card information manually to the provider. Read the full story at https://bit.ly/2yWQBnD

Last year, a major part of Amazon’s S3 storage system went down, and was unavailable for hours, making trouble for thousands of sites. Read the details at https://amzn.to/2melOup

Nobody is perfect. Not even public cloud providers. Also small fish are … small, so who are you gonna call?

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at www.redpill-linpro.com.

Packages of varnish-6.0 with matching vmods, for el6 and el7

August 9th, 2018

Some time ago, the Varnish Cache project released a new upstream version 6.0 of Varnish Cache. I updated the fedora rawhide package a few weeks ago. I have also built a copr repo with varnish packages for el6 and el7 based on the fedora package. A selection of matching vmods is also included.

Packages are available at https://copr.fedorainfracloud.org/coprs/ingvar/varnish60/

The following vmods are available:

Included in varnish-modules:
vmod-bodyaccess
vmod-cookie
vmod-header
vmod-saintmode
vmod-tcp
vmod-var
vmod-vsthrottle
vmod-xkey

Packaged separately:
vmod-curl
vmod-digest
vmod-geoip
vmod-memcached
vmod-querystring
vmod-uuid

Please test and report bugs. If there is enough interest, I may consider pushing these to fedora as well.

J.R.R. Tolkien: The Silmarillion

December 29th, 2017

I read Tolkien’s “canon”, that is, The Hobbit, The Lord of the Rings, and The Silmarillion, around Christmas every year. So also this year.

One of the most fascinating stories in The Silmarillion is of course the story of Túrin Turambar. He is regarded as one of the major heroes of his age. At the Council of Elrond, Elrond himself lists the great men and elf-friends of old, Hador and Húrin and Túrin and Beren. But while reading through the Silmarillion, there are few among mortal men that have also added so much pain and disaster to the elves. While a great war hero, Húrin was also responsible for the slaying of the greatest hunter of the elves, Beleg Cúthalion, the strong bow. Being the war hero, he turned the people of Nargothrond away from the wisdom of their history, and even their king, and made the hidden kingdom available for the enemy. How many elves were cruelly slain or taken to captivity in Angband because of Turin’s pride? Thousands! Perhaps even tens of thousands? So how come the elves, ages later, still reckoned Túrin son of Húrin as one of the great elf-friends?

In a Nordic saga style stunt, Túrin finally slew his greatest enemy, Glaurung the great fire-breathing dragon. Glaurung had been a continous danger to all peoples of Middle-Earth, and the end of that worm was of course a great relief to all the elves, even Elrond’s ancestors, the kings of Doriath and Gondolin. Also, we must remember that the lives of the elves are different from that of men. When the elves’ bodies die, their spirits go to Mandos, where they sit in the shadow of their thought, and from where they may even return, like Glorfindel of both Gondolin and Rivendell. But when men die, they go to somewhere else, and are not bound to the world. It seems that elves are more willing to forgive and let grief rest for wisdom over time, than are men’s wont. Even the Noldor who survived the passing of the Helcaraxë forgave and united the Noldor of Fëanor’s people that left them at the burning of the ships at Losgar.

Perhaps that is one of the lessons learned from the tragic story of Túrin. From all his unhappy life, good things happened, and afterwards, the elves forgave and even mourned him and his family.

J.R.R. Tolkien: The Lord of the Rings

December 25th, 2017

I read Tolkien’s “canon”, that is, The Hobbit, The Lord of the Rings, and The Silmarillion, around Christmas every year. So also this year.

2017 was a great year for Tolkien fans. It was the 125th anniversary of the Professor’s birth, and the 80th anniversary for the Hobbit. We also got the magnificent news that Amazon will produce a TV series based on “previously unexplored stories based on J.R.R. Tolkien’s original writings“. So what storylines would that be? A reboot of the 2001-03 trilogy is out of the question, as Peter Jackson explored and extended more than enough already. So, what do we have left? A lot! Let’s have a look.

The Lord of the Rings and its appendices tells stories in several different timelines. Long before (as in hundreds, and even thousands of years) before the main story, just before the main story (like a few decennials), parallel to the main story, and after.

One storyline could follow the ancient history of Gondor and Arnor. There are lots and lots of substories there. If I should pick one I would like to see, it would be the stories of the kings Arvedui of Arnor and  Eärnil II of Gondor, perhaps started with the Firiel incident. There are lots of exciting points to pick up there. Gondor throne heiritage politics, the war against, and the prediction of the downfall of the Witch King, the flight to Forochel, with the disastrous ship’s wreck in the ice, and the loss of the palantiri.

For the “near history” before The War of the Ring, the obvious choice would be a “The young Aragorn” series, where we could follow Aragorn in his many guises, riding with the Rohirrim, going on raids with Gondor against Harad, in and in constant conflict with Denethor. And his love life, of course, with his meeting and very long-term relationship with Arwen. And speaking of Arwen, her family story is a good storyline, with the love of Celebrían and Elrond, travelling from Lorien to Rivendell, and her abduction, and Elladan and Elrohir’s rescue of her from the orcs. Parallel to that, the story I would most love to see, would be, the story of Denethor. His tragic life is worth a season alone. Another storyline from the years just before The War of the Ring, could be Balin’s attempt to retake Moria, and build  a colony of dwarves. Lots of gore and killing of goblins to depict!

Parallel to the War of the Ring, there are a lot of things going on, that are merely mentioned in the book, and completely forgotten in the movies. The fight in Dale. The Ents’ war against the orcs after the capture of Isengard, the loss of Osgiliath and Cair Andros, to name just a few.

And of course, even after the the War of the Ring, and the Return of the King, there are stories to follow up. Aragorn’s “negotiations” for peace with his neighbouring peoples, with armed battle as alternative, supported by Eomer of Rohan. The sweet but bitter death of Aragorn and Arwen. The reign of King Eldarion.

I’m optimistic! This is going to be great!

J.R.R. Tolkien: The Hobbit

December 24th, 2017

I read Tolkien’s “Canon”, that is, The Hobbit, The Lord of the Rings, and The Silmarillion, every year about Christmas. These year, it’s even The Hobbit’s 80th Anniversary, and to celebrate, I have of course read through The Hobbit again.

So many have said so much about this book, so I’d rather show off my newest addition to my Tolkien bookshelf. This is the Swedish 1962 edition of The Hobbit, Bilbo, En Hobbits Äventyr (Bilbo, A Hobbit’s Adventure), and it has quite an interesting history.

In the 50s and 60s, Astrid Lindgren, maybe most famous for her children’s books about Pippi Longstocking, worked as an editor at the department for Children’s literature at Rabén & Sjögren, who published Tolkien’s works in Sweden. Lindgren was very interested in Tolkien’s work, and while she later denied Tolkien as an inspiration for it, she published the quite Lord of the Rings reminiscing Mio my Son in 1954, and later the world beloved classic children’s fantasy novels The Brothers Lionheart and Ronia, the Robber’s daughter.

In the early 60s Lindgren was not content* with the current Swedish translation of The Hobbit, Hompen (translation by Tore Zetterholm, 1947), and wanted to better it. So she opted for a new translation and got hold of Britt G. Hallqvist for the job. For illustrations, she contacted her friend Tove Jansson, now World famous for her Moomin Valley universe. Jansson had already had success with her Moomintrolls, and had previously made illustrations for a Swedish edition of Lewis Carrol’s classic poem Snarkjakten (The Hunting of the Snark, 1959), so a successful publication seemed likely.

Hallqvist translated, Jansson drew, Lindgren published it, and it flopped! Tolkien fans didn’t enjoy Jansson’s drawings much, and the illustrations were not used** again before 1994. By then, the 1962 version was cherished by Tove Jansson fans and Tolkien collectors over the World, and it had become quite hard to find. The 1994 edition was sold out in a jiffy. The illustrations were finally “blessed” by the Tolkien Estate, when they were used for the 2016 Tolkien Calendar.

Jansson’s illustrations were also used in the 2016 Tolkien calendar, which I’m, afraid to say, have not acquired (yet).

I was lucky and found a decent copy of the 1962 edition in a Japanese(!) bookstore on the Net. Now I LOVE this book. Its illustrations are absolutely gorgeous.

20171219_003402.jpg.small

20171219_144844.small

20171219_003429.jpg.small

The destruction of Lake Town and the death of Smaug are my personal favourites

The destruction of Lake Town and the death of Smaug is my personal favourite

It makes a great additon to my ever growing list of Hobbits.

This book makes a great additon to my ever growing list of Hobbits.

It would be a pity to let this book stay alone without decent Janssonic company, so I searched a few weeks, was lucky again and found a nice copy of the mentioned Snarkjakten by Lewis Carrol, and an almost mint copy of the absolutely fantastic (in all meanings of that word) Swedish 1966 edition of Alice i underlandet (Alice in Wonderland). If you enjoy Alice, you will love Janssons’ illustrations, even outshining her work on The Hobbit.

Janssons illustrations of <i>Alice</i> were later used in a lot of versions, among them, Finnish, American, British, and Norwegian editions.

Janssons illustrations of Alice were later used in a lot of versions, among them, Finnish, American, British, and Norwegian editions.

For an intensely interesting read about Jansson’s artistic work on these classics: Read Olga Holownia’s essay at barnboken.net.

That’s it. Merry Christmas and happy Youletide everybody!

*) Neither was Tolkien himself. He specially disliked the translation of Elvish names into Swedish, like Esgaroth -> Snigelby (ie. Snail Town!!!). Also interesting: Svensson, Louise, Lost in Translation? – A Comparative Study of Three Swedish Translations of J.R.R. Tolkien’s ‘The Hobbit’, Lund University 2016

**) Actually, there were other versions with Jansson’s illustrations; the Finnish Hobbit Lohikäärme-vouri (The Dragon mountain) from 1973, and the updated Finnish translation in 2003. The illustrations were also used in this year’s Finnish 80th Anniversary edition of The Hobbit.

12 days of Varnish

December 19th, 2017

While Varnish is most famous for its speedy caching capabilities, it is also a general swiss army knife of web serving. In the spirit of Christmas, here’s Twelve Days of Varnish Cache, or at least, twelve Varnish use cases. Read the rest of this post on Redpill Linpro’s Sysadvent calendar.

Dynamic DNS helper scripts

December 1st, 2017

While dynamic DNS is a wonderful tool for automation and orchestration, tools for easy cleaning up and logging changes are needed. This post describes a couple of scripts that may help.

Read the rest of this post on Redpill Linpro SysAdvent Calendar.

copr packages of varnish-5.2, varnish-modules and miscellaneous vmods for el6 and el7

October 27th, 2017

Some weeks ago, the Varnish Cache project released a new upstream version 5.2 of varnish cache. I have built a copr repo with varnish packages for el6 and el7 based on the fedora package, and a selection of matching vmods.

The following vmods are available:

Included in varnish-modules:
vmod-cookie
vmod-header
vmod-saintmode
vmod-softpurge
vmod-tcp
vmod-var
vmod-vsthrottle
vmod-xkey

Packaged separately:
vmod-geoip
vmod-basicauth
vmod-curl
vmod-digest
vmod-memcached
vmod-querystring
vmod-rfc6052
vmod-uuid

Please test and report bugs. If there are enough interest, I may consider pushing these to fedora as well. Packages are available at https://copr.fedorainfracloud.org/coprs/ingvar/varnish52/