Archive for the ‘comp’ Category

varnish-5.0, varnish-modules-0.9.2 and hitch-1.4.1, packages for Fedora and EPEL

Thursday, October 20th, 2016

The Varnish Cache project recently released varnish-5.0, and Varnish Software released hitch-1.4.1. I have wrapped packages for Fedora and EPEL.

varnish-5.0 has configuration changes, so the updated package has been pushed to rawhide, but will not replace the ones currently in EPEL nor in Fedora stable. Those who need varnish-5.0 for EPEL may use my COPR repos at https://copr.fedorainfracloud.org/coprs/ingvar/varnish50/. They include the varnish-5.0 and matching varnish-modules packages, and are compatible with EPEL 5, 6, and 7.

hitch-1.4.1 is configure file compatible with earlier releases, so packages for Fedora and EPEL are available in their respective repos, or will be once they trickle down to stable.

As always, feedback is warmly welcome. Please report via Red Hat’s Bugzilla or, while the packages are cooking in testing, Fedora’s Package Update System.

Varnish Cache is a powerful and feature rich front side web cache. It is also very fast, and that is, fast as in powered by The Dark Side of the Force. On steroids. And it is Free Software.

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at www.redpill-linpro.com.

IPV6: clatd, a component of 464XLAT, for Fedora and EPEL

Friday, September 2nd, 2016

The World is running out of IPv4 addresses, but luckily, we have IPv6 here now, and running the whole data center on IPv6 only is not just happening, it’s becoming the standard. But what if you have an app, a daemon, or a container that actually needs IPv4 connectivity? Then you may use 464XLAT to provide an IPv4 tunnel through your IPv6 only infrastructure. clatd is one component in 464XLAT.

clatd is a CLAT / SIIT-DC Edge Relay implementation for Linux. From the github wash label:

clatd implements the CLAT component of the 464XLAT network architecture specified in RFC 6877. It allows an IPv6-only host to have IPv4 connectivity that is translated to IPv6 before being routed to an upstream PLAT (which is typically a Stateful NAT64 operated by the ISP) and there translated back to IPv4 before being routed to the IPv4 internet. This is especially useful when local applications on the host requires actual IPv4 connectivity or cannot make use of DNS64 (…) clatd may also be used to implement an SIIT-DC Edge Relay as described in RFC 7756.

Note that clatd relies on Tayga for the actual translation of packets between IPv4 and IPv6.

Yesterday, I pushed clatd for fedora testing and epel testing. Please test and report feedback by bugzilla.

For more information on clatd, see the documentation included in the package, or the clatd github home. For more info on Tayga, visit http://www.litech.org/tayga/.

For general information about the process of transisioning to the britght future of IPv6, consider https://en.wikipedia.org/wiki/IPv6_transition_mechanism

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at www.redpill-linpro.com.

varnish-4.1.3 and varnish-modules-0.9.1 for fedora and epel

Wednesday, August 10th, 2016

The Varnish Cache project recently released varnish-4.1.3 and varnish-modules-0.9.1. Of course, we want updated rpms for Fedora and EPEL.

While there are official packages for el6 and el7, I tend to like to use my Fedora downstream package, also for EPEL. So I have pushed updates for Fedora, and updated copr builds for epel5, epel6, and epel7.

An update of the official supported bundle of varnish modules, varnish-modules-0.9.1, was also released a few weeks ago. I did recently wrap it for Fedora, and am waiting for its review in BZ #1324863. Packages for epel5, epel6, and epel7 are in copr as well.

Fedora updates for varnish-4.1.3 may be found at https://bodhi.fedoraproject.org/updates/?packages=varnish

The Copr repos for epel are here: https://copr.fedorainfracloud.org/coprs/ingvar/varnish41/

Test and reports are very welcome.

Varnish Cache is a powerful and feature rich front side web cache. It is also very fast, and that is, fast as in powered by The Dark Side of the Force. On steroids. And it is Free Software.

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at www.redpill-linpro.com.

hitch-1.2.0 for fedora and epel

Thursday, April 28th, 2016

Hitch is a libev-based high performance SSL/TLS proxy. It is developed by Varnish Software, and may be used for adding https to Varnish cache.

hitch-1.2.0 was recently released. Among the new features in 1.2.0, might be mentioned more granular per-site configuration. Packages for Fedora and EPEL6/7 were requested for testing today. Please test and report feedback.

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at www.redpill-linpro.com.

Tayga, stateless NAT64 implementation

Monday, January 25th, 2016

If you are planning for an IPv6 only Data Center (and if you plan for the future, you are doing that) you may have noticed that there are applications out there that are just not ready for IPv6 yet. So you need some kind of 6-4 translation, either locally or in the network. From the Tayga wash label:

TAYGA is an out-of-kernel stateless NAT64 implementation for Linux that uses the TUN driver to exchange IPv4 and IPv6 packets with the kernel. It is intended to provide production-quality NAT64 service for networks where dedicated NAT64 hardware would be overkill.

Tayga is production quality software. We use it for ipv4 access for large amounts of production nodes every day. It is for example well suited for giving 6-to-4 network access for docker nodes. You may find more information about Tayga on it’s homepage: http://www.litech.org/tayga/

I pushed tayga-0.9.2-3 to Fedora 22 and 23 stable today. It will trickle down to your local mirrors in a couple of days. I have also forked tayga for epel5, epel6, and epel7. Please contribute by testing tayga for EPEL: https://bodhi.fedoraproject.org/updates/?packages=tayga

Update: tayga is now available in EPEL

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at www.redpill-linpro.com.

J.R.R. Tolkien: The Hobbit, TBOFA extended ed.

Wednesday, December 23rd, 2015

I read J.R.R. Tolkien’s “canon”, that is, The Hobbit, The Lord of the Rings, and the Silmarillion, every Christmas. So also this year.

Not much to post about The Hobbit this year, except that I also watched the extended edition of The Battle of the Five Armies some time ago. And I enjoyed it.

There are things to say about Peter Jackson’s Hobbit project, and I’ve actually already said a bit about the theater version. The extended edition, in plain 2D on a decent TV screen is a better film. There are things to dislike. How come Galadriel is the most powerful of the White Counsil? (Or is she?) The bunny sleigh is always annoying, and Legolas running up falling rocks is still a bit too disneyish for my taste. But hey, we also got more Beorn, more Esgaroth, and more Dale. That counterweights a lot. But what gave me most in this version, compared to the theater one, is the feeling of closure. We get Thorin, Fili and Kili’s funeral. Thorin has the Arkenstone on his breast, and Daín is crowned king. This is very satisfactory, and was reason enough for me to watch the movie.

Of Balin and Thrór’s Ring (J.R.R Tolkien: The Lord of the Rings)

Wednesday, December 23rd, 2015

I read Tolkien’s canon (The Silmarillion, The Hobbit, The Lord of the Rings) every year about christmas. This year’s pondering is over Balin and Gandalf and Thrór’s ring.

Thrór possessed one of the seven rings that the dwarves got from Sauron of old. Inherited from father to son through generations, it was an heirloom of immense value for the Durin line. It passed to Thrain, who was Thrór’s son, and Thorin Oakenshield’s father. When Sauron woke again during the Third Age, Thrain was taken captive in Dol Guldur, and the ring taken from him. He perished there before Gandalf could resuce him. All this Gandalf told in the council of Elrond.

Now, by the same council, Glóin reveals that one of Balin’s main reasons for attempting to recolonize Moria, was to find Thrór’s ring. But Gandalf knew that it was not in Moria, as it was taken from Thrain in Dol Guldur. When Gandalf knew this, it is quite obvious that Thorin knew too. Gandalf would not keep information hidden about Thrain’s condition and death from his only son. So both Gandalf and Thorin must have known that Thrór’s ring was taken. Still, Balin, did not know, even though he was a close friend and companion of both Thorin and Gandalf. Consider the last scene in the Hobbit, where Gandalf and Balin, on a journey all the way from The Lonely Mountain, visit Bilbo. It is a meeting between close friends. Yet, Balin knew not. So he went with his followers to seek for the ring, and the whole colony was killed cruelly, fighting a last stand against the orcs of Moria.

In retrospect, a bit more openess about the ring would perhaps have been advisable. But the keeping and the keeper of the ring was constantly kept a tight secret in the Durin line. No one knew for sure who had the ring, until it was given to its next keeper. The appendices tell us that the dwarven rings were treacherous. Though not making the dwarves into shadows and slaves of Sauron, the ring keepers of the dwarves became jealous, and a constant hunger for more gold was set in them. Thus, the ring was often the base for a large hoard of treasure, which in turn could cause grieves like wars and dragon plunder.

Perhaps Gandalf considered this, when he kept his knowledge about Thrór’s ring hidden. It is still a bit of a mystery to me though.

Poor man’s VPN via ssh socks proxy

Sunday, December 13th, 2015

This was also posted on Redpill Linpro’s Sysadvent blog

It is late night. You have just arrived at your Grandparents, when the SMS beeper goes off. There is a problem with a SAN controller, and the on-call person know you fixed it the last time. Now, if you only had documented it.

You know you have to fix this yourself, but you have no VPN access. You don’t even have an Internet connection, except your 3G mobile phone, and you really need access to that admin web gui. There is an emergency ssh port available, but no other port is open. X-forwarding over 3G? Not an option. ssh port-forwarding and fix /etc/hosts. Doable perhaps? VNC over ssh? Awkward. Enter the ssh socks proxy!

Emergency web access

Simply run:

$ ssh -D 1080 login.example.com

Now, you have a local port 1080 that creates a SOCKS proxy to the server side. Firefox has support for that proxy.

Settings -> Advanced -> Network -> Configure how Firefox connects to the Internet -> Manual settings, Socks: localhost, Port: 1080

If you need to resolve addresses from the server side, add that to the config. In the URL field, type about:config , then search for key

network.proxy.socks_remote_dns

Set it to true. That is all. You are now surfing as if Firefox was running locally on the login server. Remember to reset your settings after you have finished your session, or Firefox will not work properly when you close your SOCKS proxy ssh shell.

Not just surfing

But wait, there’s more. With a local SOCKS proxy, you may also use other programs, and they don’t even have to support SOCKS themselves. Install tsocks, and set localhost as the socks proxy host:

$ sudo yum install tsocks || sudo apt-get install tsocks
$ echo "server = 127.0.0.1" | sudo tee /etc/tsocks.conf

tsocks is a little gem of a program. It hooks into other programs, and redirects network traffic to the local SOCKS proxy. Now, while the ssh SOCKS proxy is still running (the ssh -D1080 command), just use tsocks to run your favourite program through the proxy:

# Log into a server on a closed network behind the firewall
$ tsocks ssh server.behind.firewall.example.com
# Run a local psql shell against a remote server through the SOCKS proxy
$ tsocks psql -U pg_admin_user -W -h database.behind.firewall.example.com -W template1

or to run a whole session of commands through the socks proxy, start with “. tsocks on” (note the leading dot), and stop it with “. tsocks off”

$. tsocks on
$ command
$ command
$ command 
$. tsocks off

To run Firefox through the SOCKS proxy, but without changing its configuration:

$ tsocks firefox http://ripe.net   # Stop firefox first

To check tsocks status, run

$ tsocks show

If the LD_PRELOAD variable is empty, tsocks is disabled for this shell.

Note that all Internet traffic is not routed via tsocks. For example, ICMP is not.

Bash process substitution

Saturday, December 12th, 2015

Also posted on Redpill Linpro’s sysadvent blog

In bash, we often use redirects (that is < and > ) to get output from a command to a file, or input from a file to a command. But sometimes, commands takes two or more files as input. Then our ordinary scheme does not work anymore.

Let’s say you want to diff(1) the output of two commands. For example, compare the contents of two directories. You may run the two commands, and redirect the output to files, then diff the files, and finally remove the files. Awkward.

 $ ls dir1 | sort > file1
 $ ls dir2 | sort > file2
 $ diff -u file1 file2
 $ rm file1 file2

Since diff can take stdin as one input via the special filename ‘-‘, we might cut down to one file, but this is still awkward.

 $ ls dir1 | sort > file1
 $ ls dir2 | sort | diff -u file1 -
 $ rm file1

Bash has (of course) a better solution: Process Substition, that is, treat the output (or input) of commands as files. Enter the process substitution operators:

 >(command list) # Input
 <(command list) # Output

Now, let us solve our diff challenge with a simple oneliner:

 $ diff -u <( ls dir1 | sort)  <( ls dir2 | sort )

Neat, isn’t it? I use this all the time!

Bonus: Avoid subshell scripting

The following bash shell loop is a pitfall often missed, leading to subtle bugs that are hard to spot. Pipe to a while loop runs in a subshell, so global variables goes out of scope when they are changed inside the loop.

 #!/bin/bash
 global=0

echo "Outside loop, global=$global"

 for n in 1 2 3; do echo $n; done | \
 while read i; do
     global=$i
     echo "Inside loop: global=$global"
 done
 
 echo "Outside loop, global=$global again :-("

Using command substitution, we avoid this elegantly:

 #!/bin/bash
 global=0
 
 echo "Outside loop, global=$global"
 
 while read i; do
     global=$i
     echo "Inside loop: global=$global"
 done < <( for n in 1 2 3; do echo $n; done )
 
 echo "Outside loop, global=$global still :-)"

Varnish-4.1.0 released, packages for fedora and epel

Monday, October 12th, 2015

Varnish-4.1.0 was recently released, and as usual, I have patched and wrapped up packages for fedora and epel. As 4.1.0 is not api/abi compatible with varnish-4.0, packages for stable releases of epel and fedora are not updated. Varnish-4.1.x will be available in a stable Fedora at latest from f24, though the package recompiles fine on anything from el5 to f23 as well.

Prebuilt packages for epel5, epel6, and epel7 are available here: http://users.linpro.no/ingvar/varnish/4.1.0/.

If you are a fedora contributor, please test the f23 package. The package should install directly on el7 and all supported fedoras, including f23. Then report feedback and add karma points. With a little luck, varnish-4.1 will go into fedora 23 before it freezes.

Ingvar

Varnish Cache is a powerful and feature rich front side web cache. It is also very fast, and that is, fast as in powered by The Dark Side of the Force. On steroids. And it is Free Software.

Redpill Linpro is the market leader for professional Open Source and Free Software solutions in the Nordics, though we have customers from all over. For professional managed services, all the way from small web apps, to massive IPv4/IPv6 multi data center media hosting, and everything through container solutions, in-house, cloud, and data center, contact us at www.redpill-linpro.com.