Today’s munin tip: Reading the future

Last time: Adding info from the past. This time: Looking into the future. As Munin is one of the ravens of Odin, one of the Norse gods, this script is called Skuld, the future Norne, one of the three female destiny deities from Norse mythology

Using rrdtool, perl and some simple statistical analysis, the linear least squares method, we try to predict when a filesystem reaches 100%. Very useful for system administrators.

Script can be downloaded from http://users.linpro.no/ingvar/munin/skuld.pl.txt

UPDATE: An even more flashy version, with optional --time and --value available at above URL.

#!/usr/bin/perl
#
# skuld - a perl script looking into the future
#
# This software is covered by the GNU General Public License version 2 or later.
# See http://www.gnu.org/copyleft/gpl.html for details.
#

use strict;
use Statistics::Descriptive;

unless (defined $ARGV[0]) {
    print "usage: skuld some_munin_df_file.rrd\n";
    exit 1;
}
my $debug=0;

my $rrdtool="/usr/bin/rrdtool";
my $rrd=$ARGV[0];
my $type="";
my $name="";

my @dayx  = (); my $dayy  = Statistics::Descriptive::Full->new();
my @weekx = (); my $weeky = Statistics::Descriptive::Full->new();
my @monthx= (); my $monthy= Statistics::Descriptive::Full->new();
my @yearx = (); my $yeary = Statistics::Descriptive::Full->new();

open (DUMP,"LANG=C $rrdtool dump $rrd |") 
    or die "Unable to open a pipe from $rrdtool dump, $!";

while (<DUMP>) 
{
    # Search for correct name
    if ( /\<ds\>/ ) {
	while ( <DUMP> ) {
	    /\<name\> (\w+) \<\/name\>/ and $name=$1;
	    last if /\<\/ds\>/
	}
	unless ( $name eq "42" ) {
	    print "No munin rrd file\n";
	    exit 2;
	}
    }

    if ( /\<rra\>/ ) {
	while ( <DUMP> ) {
	    # Only parse MAX values
	    if ( /\<cf\> MAX \<\/cf\>/ ) {
		my $in=<DUMP>;
		if ( $in =~ /\<pdp_per_row\> (\d+) \<\/pdp_per_row\>/ ) {
		    my $pdp=$1;
		    if    ( $pdp ==   1 ) { $type = "d"; print "day\n"   if $debug; }
		    elsif ( $pdp ==   6 ) { $type = "w"; print "week\n"  if $debug; }
		    elsif ( $pdp ==  24 ) { $type = "m"; print "month\n" if $debug; }
		    elsif ( $pdp == 288 ) { $type = "y"; print "year\n"  if $debug; }
		} 
		else { $type=""; }	    
	    }
	    if ( /\<database\>/ and ! $type eq "" ) {
		while (<DUMP>) {
		    if ( /\<\!\-\-.+ \/ (\d+) \-\-\> \<row\>\<v\> (\d+\.\d+e.\d+) \<\/v\>\<\/row>$/ ) {
			print "type: $type, time=$1, val=$2\n" if $debug;

			if ( $type eq "d" ) { push @dayx,  ($1); $dayy->add_data($2)   };
			if ( $type eq "w" ) { push @weekx, ($1); $weeky->add_data($2)  };
			if ( $type eq "m" ) { push @monthx,($1); $monthy->add_data($2) };
			if ( $type eq "y" ) { push @yearx, ($1); $yeary->add_data($2)  };
		    }
		    last if /\<\/database\>/;				   
		}
	    }
	    last if /\<\/rra\>/;
	}
    }
}

my ($dq, $dm, $dr, $drms)=$dayy->least_squares_fit(@dayx);
my ($wq, $wm, $wr, $wrms)=$weeky->least_squares_fit(@weekx);
my ($mq, $mm, $mr, $mrms)=$monthy->least_squares_fit(@monthx);
my ($yq, $ym, $yr, $yrms)=$yeary->least_squares_fit(@yearx);

print "Daily forecast: Passes 100% at ";
print localtime((100-$dq)/$dm) . "\n";

print "Weekly forecast: Passes 100% at ";
print localtime((100-$wq)/$wm) . "\n";

print "Monthly forecast: Passes 100% at ";
print localtime((100-$mq)/$mm) . "\n";

print "Yearly forecast: Passes 100% at ";
print localtime((100-$yq)/$ym) . "\n";

5 Responses to “Today’s munin tip: Reading the future”

  1. [...] since you have Munininstalled, this could of course be avoided, by watching the graphs, and even looking into the future. But since you are responsible for a lot of other things, browsing those graphs may be [...]

  2. xavier says:

    Hi,

    All my rrd files arent correctly parsed this script, would it be possible for me to send you a rrd sample ?

    thanks.

    here it is in case it goes through the submit.

    cat < adn-df-_dev_hda7-g.rrd
    QlpoOTFBWSZTWVJg42EAQLP///////////////////////3/////////////////////4BI+d7Fx6VPuAAD5RD7y3TZq2RqqlGNpAAD48u7AO5wAs4QZxE0gmQlPCJ4E0bQAkY0nimTTJkwyNT00jaT0xE9NCYYo8RhDU9G0ib0pjSemm0k3pT0IPRgnpGjAmmiaepshpptTU9NGmIm2ppPTJPBTNNJ4oRCCm0JgVPZMp6Q0PSj1PKe0iNmpPyptTxTepqejU/KTanhPSPUxDZEanjSnp6o02oxNNN6p4po3qnlMmhtqmjT08oxT1N6RPKfqBPUPUY0jT1MJ6mAPSnoajajR6ah6g0yJBGkU2KeU00PUGmjQaafqhpo00AAAGRoAAAaAGjQAB6QAAAA00B6Jo0wQAaAAAGgAADQAAhUSio2RGn6k/VAB6Q9QYI00AAANBkaPUGmepGEAAGgAAAAA0GR6hoMmgAaGmgaAGIYgGgNMgNAAAhUlNBKf6p6U/URpkfqnqD1APUBiPU9TQbUA9Q0eoD1AGhpoaDIBoAABtQAADT1D1GgaGg0Gmg9QNAA9QAGgaAAABSkCYmEwAAEwAmAAAAjAAAJghgAAABMmmTEwCaYAAAAAAjBMTAEyNMEaYJkwTAHF4BAU1Nuon/pUUyUEWBTEiCI6LkkzgAVEglyViGoiMbHf60CMLKTTytH0AFJLX2GpbdtBJINaxXBJbkUL4SHVQHZ6KgCKPcJF194xz/+8KKdZlQUN89htNxBFE4Db4VpQBxKIBifeO0N83LvG7F13itXeZxwoC9J2nDbb0eYERevaQqOAVE9dxfut3udERRDRRFROD8ry3LZPFPTNEXpuWvkICcBCDskBShomEEZUQQ5bX+FstHUxhSAD7Cl7OBNYxrkJEUcQgGA3iE0bwGPzWXa13OQT4+hypE4Vg2+GXkQUiYGiFt615lKPzQveyRMKBK27W4M9oSAHeLE4z3BUak/0oxlJ94AqcW9AHwZ6mANeYmeBlx8BBMaS+w90T+rOQmgP3w7O8ik4BKRv3S5oLRCBkqbNsHlji5gblpbWhYlbYjuuhR5bIKUmxce5+dW7PCht2OFMzoeyHtUKjMYn0TTLBqBZeOXsccPfpSveoASDDU8bU5nqIRWwbctKUVXKhkOX7nW2uSpEbPbyL4E7JTPTiqVod6L+pruHR9VhGEBgU1+xVL9AR6m3lL6tMTHv9JqIF2Qb71zMBSL9yzbU1yrvHu+LvGPV2T/15UgrfyEuEqj026qiKwriWSYAJFH+tDvvDN145wa/cT6AIuLr23wnai6j2oTehe6Mzs5o1xljnByLH2nzU2Y41/5t6jIL/k8eswM1n+PbClghcXWDnqG0VBBPBl8iR5WIvOiUYiqCgKPWkqRQK8GQcDCIAiq2yl5Iy/N72kI132lKFVvybaameoIKUJZDwHE2fyZZAtjdkqp4wKiZYpz9gsjLTKZJPC+aiSoFXaJuh1VhsX3nK26nP/qFglhr0FYeQAAknaIBwgEGgApgva1kRXCQgAABSABCAGWEKhkQTKCZVQyChlUQyrmQoaVpRpCRMwJQjQlJIuYClaQqEUF0lUpRAoAcukNDTErFQUpQtKaJStCUUFU1VVSpS5AiCIClippo0qgGlAKKVoBopWlaBpRoRmCQgCYUaRKCYUmBKWgQJmZFqgCYZhgkKZmYKWZWYZlZgmFmRZmYFJlSCGYZhiqAhpZIEJgRmVmEpEpCYZgpVmYIBJhqgGWZqaCqQqCCXJDixONjBv3hsIIhpACUDQqNJSKUoNANAFClKUAwhMgEzMCkykSTIzCJQUAlBMMwCUglANBS0KUUoEyjHHDEUByQKaU5TvM4UQQE6BJBcJNBDklzDsYLk15IPEnEFEMSSjEICRhoKSMJzJjNuSKvJyJg0wqBvBCY1UpjQsUAigipi7gsikiydBAguERQUS6CQDoEJinRSSRCiCXgwCUkHSIcu6hi4ILmA7O7pJRARAhiDBh4gu7kPCBEOnOmDOaoxWI0xg0xpmsUTjFgoBBTQmBiGZoaYJGYGlgmmgZmmQhpimKghmQoZoYlmIIlgqaWqpiqYYSKqiSYShYiJaVQ03QD0/KfG5fe/Qp5wYPABJe3XLeo4LogobmxffVlLfVETpzhCDPaIiFQII9cAbfumOHzm+K5C3NgCgjzNUyWK9E2Wr39dGp/AAPeZvLszO8h2gkLWRiQPnHh/WPUasm2z1HWdXA732bS44LwnRtnlt7cxn5Pjf0+KzAu599lLiCjuMuJqk5zCYkopaoKPAS+vk2rn5eQMxqFuYLAZwt0xA/o7xg0f4/W1gfjnKnw5QtO2neN0Vd7n+0Kxka3boeOw+H1u/9R2NxDx+zuXy5hzH3IMzaYNvGWg7ZAYLEfBkGnoVry4kqjxEatsk0Ood5w6QkTu5Dezmk19nh6fJpaGIkKeEjHSsJnzgsSwAHdkAEuBvk7XYGDnupvN0XbHlYvlrFhZDy3exsPt2/IVVpqED9ULyoFb60hLKJ7vyed3OetKYcVQJatG+m6tl59V5kprF7Peg8Jt/q/h3zogO4QmtbXicI6SUmkYtWTJZkxdSknBBYgi/ljYTGymAslSQJDYjMycMEZyBOIOSgxbMMGKmhLSxQvZNfl8u2lMXW+DwbghqyFC1T1mQ6rFBwdiRzL/n7HpdjqNpZ+FpvbjHOKt7Q5LY67y068yD2ozyfv6NEwt3wU1FPq1vMt7mao3M1FOWyVFGJ6ibloz72MzrECYVAq1Gat8vx2Glb8DQ7UliWCIBLI2SEWyL799HYYTNU8JG0gMqI0o3U0faLPZgYmJiWg1uMwpidlAmUlGhCdksW1u9SbPTw9wnTjfbjnMnNBsI38uaW1NYMthf357kqtK8hC7WUXSEcwg4lUxIhmQ3WQzR2yRzSpx8o95kNCV0tHCsyjQAU0hMJNDrYDaQc/Ht420OqEpFSTCEwAbuVMEpQzKzCjn3mADMzMEtUhwjBiBmiioRmFmVJkU1qgMMp0P5WETVgdBmmqAqpkJgqYmpCJKgKpopmKSYaeka9CUFAG3uz2x0UyLzO42GiqaPXML7mVDsLK60gYZtXCiapip5HX3TTQ6eZXWkNJCqFoiAcYwES0WZNGQFmYYpMyC00IFIhS0CtCqUCUCv9pAMSMwoAYgsxpQZSLBMcBJkY0imMvOJPM4CCwJv/8qJjYqaMtFf7YnFeDp+BnNw24YCm2hpmopSkpRD2lnBPYhXlMDR41SFtdlVb0mHofBP/2J1Oi2MqBxtxwd1KU/RqifmXeUzjZ7q+sS9Ak8TA4haFiylcQO6xgKBxYNVZhmGqQmBkmSRiSoKSGKCqIqpCqdPAOGgcpwslS1a8YogJmgKHQkMSRVTFSnVoDZWkGjVDKarUOJEtXBqSJpJqQ1qWYUpMwbFoSJmVxBqQoYIx/jCpmBaADRQaQ7bAmYXoMhiR0nEjS4kwS4nNSqZ+l+zbcfrfk2mx+D8epxXIdJ+xuu50UiUo0IhMswDMAxATIk0wJDIEwzIJDCTMyEwhMpTEMxMDIQpEizJ7GVwwswyEzKEs0oRUCULDKsyzCwyDMDDCEQTCTBEITMNLREFQCzAzMyMykwsysyRVRU1KEwMzDKxIlM1UJUSyxQSUNEMwMMMzM1BLIUsS0FCUDQ0BEDUQkQtINCFAsEwhMowzMrMAEwzDMzAtIACQSEAAc112+WfngvnmLBwKyMtsUjAAbp/zyz46psQsh6iUApbtpgQPhBemSxTGxGccuH9LXybfsCG+EqUMS8/LhSDXW5NzwGQBrU5yUpCdm5xQ9cQdnXcrxQ7nyqLo8ETsz+/NwawNOhV+MstD2VN4hm86t9CJL0eNcFJ3MFtfjQvRuKqig5YzXRLLz7zP0LTKw4Hyazb7Mue0+qKAG1bg3DiIhMwc6//EEGa3OTLuFpDAs0mgx/Tzsaw7msQmT1It3Esl627442lMTamhHy+WxiyqV0UC8dfJquZbXTD6u6jyFUGksujpWtoRtRDD2wJ582sykq0JvleklRXsRJtk4h/kg4i3UCSCcQAzB0shEdPmvQc7lw+ZV2vGYdSVTewjSCD6gkCKFShGhKCUM41eZ5zU+PfQN0/pxPtPdbn5a+2ITiXqgZiDRsJdbtO1218uqmUFKd4M/Gf2JmsvuC/tx0ctrashv9se4p3u7PdMf3jzWvc04c7MnNQ6GOEvT1ZbTKz5hzKXmMpXJhvLMBv1p6Reu1klHwCvgI8zN9vXXLrwwOq0GwTrEuqrtXgd5cmCG8PPeQBYnAl3zMFRRwtA6NFKlahf5PTu8hhOFg8O9em5V9lzM1vrXX987wKMuZhi557IKaSqqiA3CARqO6RyOpNULAgK3oPrjZrOIlo5SFAFxpvivZk3FELeo7tZjO0TYEC+RJI/xs4d1OS4pWd7sDGLRaRUzOUSkovopNz6InodugmNwmxevQ8adfvS+pqJKRXqUGOat5xJQdeSacj4t0kCFfS9PXs3UjDmpZD4cJOHzEI/GE8E8Rez6nBw0nKYg7Cdb2vzp4A6uefLQdGGBhIZf+TG2B1azUi/B/SEy3JntG/cEK6fVsE7ZcDIX49IWO2LHKIvJ0TcfPry6XN5TlsUqCx6QwUVoqLdDoJxsU12HqVtN5vyWOcOwPFiinknUlAllZLcifTE/JdfgsIYAA4QaOAeYlVfkyAJ2qRUD5HOdl9rymYRDPCJnlBP3wACe/kVKRmRJgGZEmQCYChQpYZiGSggWYCYCYGlgmCapSKKiiIAJgJgCYCiGGmakCZZoiZSoIoiIqiIYigmZBmAJlUmSKClhaWoKmGCCZKSQoaWkkqkiaGpkaKhpZiGogkgpCqWIoEiCYKAIISlJkCgAKChAKQoBYlCgCkCVhISigaCqGZGYSYFmA/LIYCVN5CvjLfWNf4/+7qGy7l8Xddn+Jq6bstqLRiaXZQF721ZIjw2MHjpuzogfIS46dIesVN75OBoRTGzOk3RAiMUmfvYDDvLF3JxV89pHD1y9RIETrDyiQrmJfbKP45kdv4E4FH9zHF+2HkWMWFmOfD8vWmMTqY0yverNJ/kjUL/OGql3z2Nh8DiMFtymcrRgSONtF23dkmni3My6BPcxuq1B5N/EbODws+etZ9apI7Xy7n8X6tIm490g3TpwVI8FzqGB1x7a4lWmRTfFU05hN6VVOtLehJ8ewSukCUKkXBDRmaYdbeksVYGq/+qDN4DJnN23nRmTXbKWgBV7KxKEMJIzf7Ft3CnQsYRPCL6xixfvDOK+QKadShXkk7SY5vS2TDJkZibZRvOrv5bYd/mrKGbAUAYhgj2uY3ucp832JBbLnYzu+4w6RGEDU1BX2N066d5v6X+h/HEQMh3N5kPSlMyawmlnscDjEutpZsyKKyQ+O3bs9h5afHXkxIjlKu/Mvxq1OBoG6QN+7FrhXY0azU2SmdE+m2yeN4mxbxklbw2t4JtX2oc1N5S7t2Ei07Om2nj9fV2tS1uUoNMq/U2RkRYls6mUw8Lk0mNVWD1SIJRzre30+wllQWZozfd1cACBWMBNRgIKNW/yGROotNXcMx7Q1OFh36ZJcnAx5bfH9BqatAdFQaWMPzZQqH58yMfSheCh8Tq5K8EiYn63ePSfC1DbfS7d28R20u4EUQOY4bq2h4a4U1Z9VB0vf+P0ed1uTy1QRRAyBVfYsqJRQAqUKgOf/i7kinChIKTBxsI=
    EOF1

  3. xavier says:

    oops, forgot the output :

    Illegal division by zero at /usr/local/src/munin-prediction-df-skuld.pl line 126, line 6233.
    Daily forecast: Reaches 100 at

    line with cat above should read :
    cat less less EOF1 pipe base64 -d pipe bunzip2 superior adn-df-_dev_hda7-g.rrd
    replace less with key code 0x3C of course, same for pipe.

  4. Vincent says:

    Hi,

    I am planning to use this wonderful script.
    Could you please give some insight, at least, on the div by zero error?
    Appearently, $dq and $dm are not initialized properly.

    In some cases, I got no error at all and correct output values.
    But not lately. I use real-life data and I don’t know what can cause this in the script (maybe some negative values or something like this?).

    Anyone, any idea?
    Thanks’ anyway!

  5. ingvar says:

    Hi, Vincent. I updated the script today with a fix for newer versions of rrdtool. Please try the new version. http://users.linpro.no/ingvar/munin/skuld.pl.txt

Leave a Reply