I’ve been wanting to change the Monolith polling (poller.pl) script for a while. It’s written in perl and is called by the agent.pl script. This ideally runs from Cron on hourly or daily intervals – depending on how often you want to collect DB statistics for OLAP needs.
Problem:
I am currently polling >80 servers and the original script runs those checks in serial. Some DBs take longer to poll than others given the load on the server and the number of schemas that need information gathered from them. For example most servers poll in a matter of seconds, but one server has over 300 schemas running on it and the data+index size gathering takes over 10 minutes to poll.
Solution:
migrate the agent.pl script to utilize the threads library that is included in Perl 5.8 (possibly older versions as well but that’s what I’m running).
Method:
I have the agent.pl pull down a list of hostnames to poll, push the various variables for database connection into a array, and send that nested array to a thread_execution function. It spawns a thread for each connection and the time for polling is decreased.
Code:
the important parts, some code left out here. Also the code tag on wordpress sucks and isn’t formatting my tabs.
push(@Array , ([ "$ip_address", "$username", "$pass", "$port", "$host_id" ]));}
while($i < $count) {
my $thr = new threads \&
thread_exec, @Array, $i;
my $id=threads->tid;
$thr->detach;
threads->
yield();
threads->
list();$i++;}}
sub thread_exec {
my $id=threads->tid;
my $ip_address = $Array["$i"]->[0];
my $username = $Array["$i"]->[1];
my $pass = $Array["$i"]->[2];
my $port = $Array["$i"]->[3];
my $host_id = $Array["$i"]->[4];
print "THREAD ID: $id , IP: $ip_address, USR: $username, PASS: $pass, PORT: $port, HOST: $host_id\n";go_poller($ip_address,$username,$pass,$port,$host_id)
Solution:
While this is still a bit buggy I have the following results.
Single Thread Timings
real 47m49.729s
user 0m21.100s
sys 0m5.870s
Multi-Threaded Timings
real 0m35.318
suser 0m23.560s
sys 0m8.270s
So as you see, quite a bit of savings there. Now I just need to figure out why this keeps popping up….
Attempt to free unreferenced scalar during global destruction.
