Today’s munin tip: Reading the future

Last time: Adding info from the past. This time: Looking into the future. As Munin is one of the ravens of Odin, one of the Norse gods, this script is called Skuld, the future Norne, one of the three female destiny deities from Norse mythology

Using rrdtool, perl and some simple statistical analysis, the linear least squares method, we try to predict when a filesystem reaches 100%. Very useful for system administrators.

Script can be downloaded from http://users.linpro.no/ingvar/munin/skuld.pl.txt

UPDATE: An even more flashy version, with optional --time and --value available at above URL.

#!/usr/bin/perl
#
# skuld - a perl script looking into the future
#
# This software is covered by the GNU General Public License version 2 or later.
# See http://www.gnu.org/copyleft/gpl.html for details.
#

use strict;
use Statistics::Descriptive;

unless (defined $ARGV[0]) {
    print "usage: skuld some_munin_df_file.rrd\n";
    exit 1;
}
my $debug=0;

my $rrdtool="/usr/bin/rrdtool";
my $rrd=$ARGV[0];
my $type="";
my $name="";

my @dayx  = (); my $dayy  = Statistics::Descriptive::Full->new();
my @weekx = (); my $weeky = Statistics::Descriptive::Full->new();
my @monthx= (); my $monthy= Statistics::Descriptive::Full->new();
my @yearx = (); my $yeary = Statistics::Descriptive::Full->new();

open (DUMP,"LANG=C $rrdtool dump $rrd |") 
    or die "Unable to open a pipe from $rrdtool dump, $!";

while (<DUMP>) 
{
    # Search for correct name
    if ( /\<ds\>/ ) {
	while ( <DUMP> ) {
	    /\<name\> (\w+) \<\/name\>/ and $name=$1;
	    last if /\<\/ds\>/
	}
	unless ( $name eq "42" ) {
	    print "No munin rrd file\n";
	    exit 2;
	}
    }

    if ( /\<rra\>/ ) {
	while ( <DUMP> ) {
	    # Only parse MAX values
	    if ( /\<cf\> MAX \<\/cf\>/ ) {
		my $in=<DUMP>;
		if ( $in =~ /\<pdp_per_row\> (\d+) \<\/pdp_per_row\>/ ) {
		    my $pdp=$1;
		    if    ( $pdp ==   1 ) { $type = "d"; print "day\n"   if $debug; }
		    elsif ( $pdp ==   6 ) { $type = "w"; print "week\n"  if $debug; }
		    elsif ( $pdp ==  24 ) { $type = "m"; print "month\n" if $debug; }
		    elsif ( $pdp == 288 ) { $type = "y"; print "year\n"  if $debug; }
		} 
		else { $type=""; }	    
	    }
	    if ( /\<database\>/ and ! $type eq "" ) {
		while (<DUMP>) {
		    if ( /\<\!\-\-.+ \/ (\d+) \-\-\> \<row\>\<v\> (\d+\.\d+e.\d+) \<\/v\>\<\/row>$/ ) {
			print "type: $type, time=$1, val=$2\n" if $debug;

			if ( $type eq "d" ) { push @dayx,  ($1); $dayy->add_data($2)   };
			if ( $type eq "w" ) { push @weekx, ($1); $weeky->add_data($2)  };
			if ( $type eq "m" ) { push @monthx,($1); $monthy->add_data($2) };
			if ( $type eq "y" ) { push @yearx, ($1); $yeary->add_data($2)  };
		    }
		    last if /\<\/database\>/;				   
		}
	    }
	    last if /\<\/rra\>/;
	}
    }
}

my ($dq, $dm, $dr, $drms)=$dayy->least_squares_fit(@dayx);
my ($wq, $wm, $wr, $wrms)=$weeky->least_squares_fit(@weekx);
my ($mq, $mm, $mr, $mrms)=$monthy->least_squares_fit(@monthx);
my ($yq, $ym, $yr, $yrms)=$yeary->least_squares_fit(@yearx);

print "Daily forecast: Passes 100% at ";
print localtime((100-$dq)/$dm) . "\n";

print "Weekly forecast: Passes 100% at ";
print localtime((100-$wq)/$wm) . "\n";

print "Monthly forecast: Passes 100% at ";
print localtime((100-$mq)/$mm) . "\n";

print "Yearly forecast: Passes 100% at ";
print localtime((100-$yq)/$ym) . "\n";

This entry was posted on Thursday, November 13th, 2008 at 16:36 and is filed under comp, munin, sysadmin. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

5 Responses to “Today’s munin tip: Reading the future”

On the third side » Blog Archive » Today’s munin tip: Collecting similar graphs says:

March 25, 2009 at 13:29

[...] since you have Munininstalled, this could of course be avoided, by watching the graphs, and even looking into the future. But since you are responsible for a lot of other things, browsing those graphs may be [...]
xavier says:

June 14, 2011 at 18:38

Hi,

All my rrd files arent correctly parsed this script, would it be possible for me to send you a rrd sample ?

thanks.

here it is in case it goes through the submit.

cat < adn-df-_dev_hda7-g.rrd
QlpoOTFBWSZTWVJg42EAQLP///////////////////////3/////////////////////4BI+d7Fx6VPuAAD5RD7y3TZq2RqqlGNpAAD48u7AO5wAs4QZxE0gmQlPCJ4E0bQAkY0nimTTJkwyNT00jaT0xE9NCYYo8RhDU9G0ib0pjSemm0k3pT0IPRgnpGjAmmiaepshpptTU9NGmIm2ppPTJPBTNNJ4oRCCm0JgVPZMp6Q0PSj1PKe0iNmpPyptTxTepqejU/KTanhPSPUxDZEanjSnp6o02oxNNN6p4po3qnlMmhtqmjT08oxT1N6RPKfqBPUPUY0jT1MJ6mAPSnoajajR6ah6g0yJBGkU2KeU00PUGmjQaafqhpo00AAAGRoAAAaAGjQAB6QAAAA00B6Jo0wQAaAAAGgAADQAAhUSio2RGn6k/VAB6Q9QYI00AAANBkaPUGmepGEAAGgAAAAA0GR6hoMmgAaGmgaAGIYgGgNMgNAAAhUlNBKf6p6U/URpkfqnqD1APUBiPU9TQbUA9Q0eoD1AGhpoaDIBoAABtQAADT1D1GgaGg0Gmg9QNAA9QAGgaAAABSkCYmEwAAEwAmAAAAjAAAJghgAAABMmmTEwCaYAAAAAAjBMTAEyNMEaYJkwTAHF4BAU1Nuon/pUUyUEWBTEiCI6LkkzgAVEglyViGoiMbHf60CMLKTTytH0AFJLX2GpbdtBJINaxXBJbkUL4SHVQHZ6KgCKPcJF194xz/+8KKdZlQUN89htNxBFE4Db4VpQBxKIBifeO0N83LvG7F13itXeZxwoC9J2nDbb0eYERevaQqOAVE9dxfut3udERRDRRFROD8ry3LZPFPTNEXpuWvkICcBCDskBShomEEZUQQ5bX+FstHUxhSAD7Cl7OBNYxrkJEUcQgGA3iE0bwGPzWXa13OQT4+hypE4Vg2+GXkQUiYGiFt615lKPzQveyRMKBK27W4M9oSAHeLE4z3BUak/0oxlJ94AqcW9AHwZ6mANeYmeBlx8BBMaS+w90T+rOQmgP3w7O8ik4BKRv3S5oLRCBkqbNsHlji5gblpbWhYlbYjuuhR5bIKUmxce5+dW7PCht2OFMzoeyHtUKjMYn0TTLBqBZeOXsccPfpSveoASDDU8bU5nqIRWwbctKUVXKhkOX7nW2uSpEbPbyL4E7JTPTiqVod6L+pruHR9VhGEBgU1+xVL9AR6m3lL6tMTHv9JqIF2Qb71zMBSL9yzbU1yrvHu+LvGPV2T/15UgrfyEuEqj026qiKwriWSYAJFH+tDvvDN145wa/cT6AIuLr23wnai6j2oTehe6Mzs5o1xljnByLH2nzU2Y41/5t6jIL/k8eswM1n+PbClghcXWDnqG0VBBPBl8iR5WIvOiUYiqCgKPWkqRQK8GQcDCIAiq2yl5Iy/N72kI132lKFVvybaameoIKUJZDwHE2fyZZAtjdkqp4wKiZYpz9gsjLTKZJPC+aiSoFXaJuh1VhsX3nK26nP/qFglhr0FYeQAAknaIBwgEGgApgva1kRXCQgAABSABCAGWEKhkQTKCZVQyChlUQyrmQoaVpRpCRMwJQjQlJIuYClaQqEUF0lUpRAoAcukNDTErFQUpQtKaJStCUUFU1VVSpS5AiCIClippo0qgGlAKKVoBopWlaBpRoRmCQgCYUaRKCYUmBKWgQJmZFqgCYZhgkKZmYKWZWYZlZgmFmRZmYFJlSCGYZhiqAhpZIEJgRmVmEpEpCYZgpVmYIBJhqgGWZqaCqQqCCXJDixONjBv3hsIIhpACUDQqNJSKUoNANAFClKUAwhMgEzMCkykSTIzCJQUAlBMMwCUglANBS0KUUoEyjHHDEUByQKaU5TvM4UQQE6BJBcJNBDklzDsYLk15IPEnEFEMSSjEICRhoKSMJzJjNuSKvJyJg0wqBvBCY1UpjQsUAigipi7gsikiydBAguERQUS6CQDoEJinRSSRCiCXgwCUkHSIcu6hi4ILmA7O7pJRARAhiDBh4gu7kPCBEOnOmDOaoxWI0xg0xpmsUTjFgoBBTQmBiGZoaYJGYGlgmmgZmmQhpimKghmQoZoYlmIIlgqaWqpiqYYSKqiSYShYiJaVQ03QD0/KfG5fe/Qp5wYPABJe3XLeo4LogobmxffVlLfVETpzhCDPaIiFQII9cAbfumOHzm+K5C3NgCgjzNUyWK9E2Wr39dGp/AAPeZvLszO8h2gkLWRiQPnHh/WPUasm2z1HWdXA732bS44LwnRtnlt7cxn5Pjf0+KzAu599lLiCjuMuJqk5zCYkopaoKPAS+vk2rn5eQMxqFuYLAZwt0xA/o7xg0f4/W1gfjnKnw5QtO2neN0Vd7n+0Kxka3boeOw+H1u/9R2NxDx+zuXy5hzH3IMzaYNvGWg7ZAYLEfBkGnoVry4kqjxEatsk0Ood5w6QkTu5Dezmk19nh6fJpaGIkKeEjHSsJnzgsSwAHdkAEuBvk7XYGDnupvN0XbHlYvlrFhZDy3exsPt2/IVVpqED9ULyoFb60hLKJ7vyed3OetKYcVQJatG+m6tl59V5kprF7Peg8Jt/q/h3zogO4QmtbXicI6SUmkYtWTJZkxdSknBBYgi/ljYTGymAslSQJDYjMycMEZyBOIOSgxbMMGKmhLSxQvZNfl8u2lMXW+DwbghqyFC1T1mQ6rFBwdiRzL/n7HpdjqNpZ+FpvbjHOKt7Q5LY67y068yD2ozyfv6NEwt3wU1FPq1vMt7mao3M1FOWyVFGJ6ibloz72MzrECYVAq1Gat8vx2Glb8DQ7UliWCIBLI2SEWyL799HYYTNU8JG0gMqI0o3U0faLPZgYmJiWg1uMwpidlAmUlGhCdksW1u9SbPTw9wnTjfbjnMnNBsI38uaW1NYMthf357kqtK8hC7WUXSEcwg4lUxIhmQ3WQzR2yRzSpx8o95kNCV0tHCsyjQAU0hMJNDrYDaQc/Ht420OqEpFSTCEwAbuVMEpQzKzCjn3mADMzMEtUhwjBiBmiioRmFmVJkU1qgMMp0P5WETVgdBmmqAqpkJgqYmpCJKgKpopmKSYaeka9CUFAG3uz2x0UyLzO42GiqaPXML7mVDsLK60gYZtXCiapip5HX3TTQ6eZXWkNJCqFoiAcYwES0WZNGQFmYYpMyC00IFIhS0CtCqUCUCv9pAMSMwoAYgsxpQZSLBMcBJkY0imMvOJPM4CCwJv/8qJjYqaMtFf7YnFeDp+BnNw24YCm2hpmopSkpRD2lnBPYhXlMDR41SFtdlVb0mHofBP/2J1Oi2MqBxtxwd1KU/RqifmXeUzjZ7q+sS9Ak8TA4haFiylcQO6xgKBxYNVZhmGqQmBkmSRiSoKSGKCqIqpCqdPAOGgcpwslS1a8YogJmgKHQkMSRVTFSnVoDZWkGjVDKarUOJEtXBqSJpJqQ1qWYUpMwbFoSJmVxBqQoYIx/jCpmBaADRQaQ7bAmYXoMhiR0nEjS4kwS4nNSqZ+l+zbcfrfk2mx+D8epxXIdJ+xuu50UiUo0IhMswDMAxATIk0wJDIEwzIJDCTMyEwhMpTEMxMDIQpEizJ7GVwwswyEzKEs0oRUCULDKsyzCwyDMDDCEQTCTBEITMNLREFQCzAzMyMykwsysyRVRU1KEwMzDKxIlM1UJUSyxQSUNEMwMMMzM1BLIUsS0FCUDQ0BEDUQkQtINCFAsEwhMowzMrMAEwzDMzAtIACQSEAAc112+WfngvnmLBwKyMtsUjAAbp/zyz46psQsh6iUApbtpgQPhBemSxTGxGccuH9LXybfsCG+EqUMS8/LhSDXW5NzwGQBrU5yUpCdm5xQ9cQdnXcrxQ7nyqLo8ETsz+/NwawNOhV+MstD2VN4hm86t9CJL0eNcFJ3MFtfjQvRuKqig5YzXRLLz7zP0LTKw4Hyazb7Mue0+qKAG1bg3DiIhMwc6//EEGa3OTLuFpDAs0mgx/Tzsaw7msQmT1It3Esl627442lMTamhHy+WxiyqV0UC8dfJquZbXTD6u6jyFUGksujpWtoRtRDD2wJ582sykq0JvleklRXsRJtk4h/kg4i3UCSCcQAzB0shEdPmvQc7lw+ZV2vGYdSVTewjSCD6gkCKFShGhKCUM41eZ5zU+PfQN0/pxPtPdbn5a+2ITiXqgZiDRsJdbtO1218uqmUFKd4M/Gf2JmsvuC/tx0ctrashv9se4p3u7PdMf3jzWvc04c7MnNQ6GOEvT1ZbTKz5hzKXmMpXJhvLMBv1p6Reu1klHwCvgI8zN9vXXLrwwOq0GwTrEuqrtXgd5cmCG8PPeQBYnAl3zMFRRwtA6NFKlahf5PTu8hhOFg8O9em5V9lzM1vrXX987wKMuZhi557IKaSqqiA3CARqO6RyOpNULAgK3oPrjZrOIlo5SFAFxpvivZk3FELeo7tZjO0TYEC+RJI/xs4d1OS4pWd7sDGLRaRUzOUSkovopNz6InodugmNwmxevQ8adfvS+pqJKRXqUGOat5xJQdeSacj4t0kCFfS9PXs3UjDmpZD4cJOHzEI/GE8E8Rez6nBw0nKYg7Cdb2vzp4A6uefLQdGGBhIZf+TG2B1azUi/B/SEy3JntG/cEK6fVsE7ZcDIX49IWO2LHKIvJ0TcfPry6XN5TlsUqCx6QwUVoqLdDoJxsU12HqVtN5vyWOcOwPFiinknUlAllZLcifTE/JdfgsIYAA4QaOAeYlVfkyAJ2qRUD5HOdl9rymYRDPCJnlBP3wACe/kVKRmRJgGZEmQCYChQpYZiGSggWYCYCYGlgmCapSKKiiIAJgJgCYCiGGmakCZZoiZSoIoiIqiIYigmZBmAJlUmSKClhaWoKmGCCZKSQoaWkkqkiaGpkaKhpZiGogkgpCqWIoEiCYKAIISlJkCgAKChAKQoBYlCgCkCVhISigaCqGZGYSYFmA/LIYCVN5CvjLfWNf4/+7qGy7l8Xddn+Jq6bstqLRiaXZQF721ZIjw2MHjpuzogfIS46dIesVN75OBoRTGzOk3RAiMUmfvYDDvLF3JxV89pHD1y9RIETrDyiQrmJfbKP45kdv4E4FH9zHF+2HkWMWFmOfD8vWmMTqY0yverNJ/kjUL/OGql3z2Nh8DiMFtymcrRgSONtF23dkmni3My6BPcxuq1B5N/EbODws+etZ9apI7Xy7n8X6tIm490g3TpwVI8FzqGB1x7a4lWmRTfFU05hN6VVOtLehJ8ewSukCUKkXBDRmaYdbeksVYGq/+qDN4DJnN23nRmTXbKWgBV7KxKEMJIzf7Ft3CnQsYRPCL6xixfvDOK+QKadShXkk7SY5vS2TDJkZibZRvOrv5bYd/mrKGbAUAYhgj2uY3ucp832JBbLnYzu+4w6RGEDU1BX2N066d5v6X+h/HEQMh3N5kPSlMyawmlnscDjEutpZsyKKyQ+O3bs9h5afHXkxIjlKu/Mvxq1OBoG6QN+7FrhXY0azU2SmdE+m2yeN4mxbxklbw2t4JtX2oc1N5S7t2Ei07Om2nj9fV2tS1uUoNMq/U2RkRYls6mUw8Lk0mNVWD1SIJRzre30+wllQWZozfd1cACBWMBNRgIKNW/yGROotNXcMx7Q1OFh36ZJcnAx5bfH9BqatAdFQaWMPzZQqH58yMfSheCh8Tq5K8EiYn63ePSfC1DbfS7d28R20u4EUQOY4bq2h4a4U1Z9VB0vf+P0ed1uTy1QRRAyBVfYsqJRQAqUKgOf/i7kinChIKTBxsI=
EOF1
xavier says:

June 14, 2011 at 18:51

oops, forgot the output :

Illegal division by zero at /usr/local/src/munin-prediction-df-skuld.pl line 126, line 6233.
Daily forecast: Reaches 100 at

line with cat above should read :
cat less less EOF1 pipe base64 -d pipe bunzip2 superior adn-df-_dev_hda7-g.rrd
replace less with key code 0x3C of course, same for pipe.
Vincent says:

August 26, 2013 at 13:25

Hi,

I am planning to use this wonderful script.
Could you please give some insight, at least, on the div by zero error?
Appearently, $dq and $dm are not initialized properly.

In some cases, I got no error at all and correct output values.
But not lately. I use real-life data and I don’t know what can cause this in the script (maybe some negative values or something like this?).

Anyone, any idea?
Thanks’ anyway!
ingvar says:

September 6, 2013 at 12:27

Hi, Vincent. I updated the script today with a fix for newer versions of rrdtool. Please try the new version. http://users.linpro.no/ingvar/munin/skuld.pl.txt

Today’s munin tip: Reading the future

5 Responses to “Today’s munin tip: Reading the future”

Leave a Reply