Kontrollbase gets Memcache support

After working with Memcache on another application I figured that Kontrollbase could benefit from it as well. Expect in the next release (coming this week), easily configurable support for memcache. This makes use of the memcache library for CodeIgniter talked about here: http://codeigniter.com/forums/viewthread/72538/

In the config.php file you just enable it and set the proper connection parameters.

$config['memcache_enabled'] = TRUE;
$config['memcache_ip'] = '127.0.0.1';
$config['memcache_port'] = '11211';

In the main controller I added the following code to the index and host functions.

$this->load->library('cache');
// memcache library information
$memcache = $this->config->item('memcache_enabled');
if($memcache == TRUE) {
$memcache_ip = $this->config->item('memcache_ip');
$memcache_port = $this->config->item('memcache_port');
$memcache = $this->cache->useMemcache($memcache_ip, $memcache_port);
if(!$memcache) {
log_message('debug', "Memcache connection failure.");
show_error("Memcache library not enabled correctly.");
}
else {
log_message('debug', "Memcache enabled!");
}
}
else {
log_message('debug', "Memcache not enabled in config.");
}
//end memcache

And then to load the view into the cache we see this later on…

$this->cache->save('cachedMain',$this->load->view('main/main', $g, TRUE),NULL,3600);

Dev Ops: Speeding up webapps

Some quick notes on useful apps for speeding up webapps. I’ve been working on one application that was suffering some performance issues and needed immediate help. So, with a combination of the following three apps and some hours of configuration tuning, I have the page loads down from 10 seconds to 2 seconds. Not too bad. Still more tuning to go of course.

PHP Acceleration: http://eaccelerator.net
Memcached http://www.danga.com/memcached
XtraDB http://www.percona.com/docs/wiki/percona-xtradb:start

JFS and RHEL5

Quick one here; If you’re thinking to yourself “why can’t I make the MySQL data partition JFS on my RHEL5 server…” here’s a good how to: http://phaq.phunsites.net/2008/02/04/enabling-reiserfs-xfs-jfs-on-redhat-enterprise-linux/

Simple Linux: Is this RPM 32bit or 64bit?

Simple question with a simple answer. Let’s say you’re on Redhat 4 or 5 and you need to know if an installed RPM is 32bit or 64bit…

shell> rpm -q --qf '%{NAME}-%{VERSION}(%{ARCH})\n' libaio
libaio-0.3.107(x86_64)

MySQL Common: shared-compat libraries are very handy


install_driver(mysql) failed: Can't load '/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/DBD/mysql/mysql.so' for module DBD::mysql: /usr/lib64/libmysqlclient.so.15: version `libmysqlclient_15' not found (required by /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/DBD/mysql/mysql.so) at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm line 230.
at (eval 8) line 3

After a lot of mucking around I noticed that the MySQL-shared-community RPM was installed, not the MySQL-shared-compat RPM like my usual installs. So…


sudo /etc/init.d/mysqld stop
sudo rpm -e MySQL-shared-5.1.33-0.rhel5.x86_64.rpm
sudo rpm -Uvh MySQL-shared-compat-5.1.33-0.rhel5.x86_64.rpm
sudo /etc/init.d/mysqld start

Kontrollbase gets new tabs for server cnf/stats/vars

Added some new code to Kontrollbase to allow you to view the cnf file on each host, as well as all of the global variables and global status information that was collected from the most recent polling period. Here are some screencaps.

cnf file display

global statistics display

global variables display

Reviewed: In praise of JSLint

Short post here, just wanted to give some praise to the JSLint website for offering an incredibly useful tool. I recently saved me a lot of time debugging some ExtJS code, so next time you can’t find that missing comma or extra semi-colon in your minified JS code, give it a shot. http://www.jslint.com

Change control generator

Perhaps you have found yourself in the middle of planning a large maintenance that involves many servers and many clients and all of the work is the same and you noticed that you don’t want to write a change control document for each client or each server.

Well, here’s some code that will write one for you. Just fill in your array of clients/hosts and then the content of the change control document – then run the file with PHP and you’ll have your change control docs all ready to go.

Note that all blank lines have been removed because wordpress’ code tag sucks and breaks on empty lines in code

function write_cr($content,$client,$hostname,$today) {
$FULLPATH="/tmp";
$filename=("$FULLPATH/cr/$client.$hostname.$today.txt");
if (!file_exists("$filename")) { touch("$filename");}
if (is_writable($filename)) {
if (!$handle = fopen($filename, 'a')) {
echo "Cannot open file ($filename)";
exit;
}
else {
if (fwrite($handle, $content) === FALSE) {
echo "Cannot write to file [$filename]";
exit;
}
}
fclose($handle);
}
else {
echo "The file [$filename] is not writable.";
}
}
function crfile($client,$hostname,$today) {
$contents="
CHANGE CONTROL DOCUMENT FOR: $client-$hostname-$today
Date: $today
Name: $client
Phone:
Email:
Downtime Required:
Summary of Changes:
Reason for change:
Testing done prior to change:
Reveral plan:
Triggers for reversal:
Reversal plan time requirement:
Test Plan:
Who will test the changes:
Who will sign off on the changes:
";
write_cr($contents,$client,$hostname,$today);
}
$today = date('d');
$list = array(
"client1" => "hostname1",
"client2" => "hostname2");
foreach($list as $k => $v) {
$client = $k;
$hostname = $v;
crfile($client,$hostname,$today);
}
?>

A quick rundown of per-thread buffers

I saw something interesting today when helping out someone on the #mysql IRC channel. It was a cnf file that was designed to destroy a server. Before I get into the why-not, here are the goods:

...snip...
read_buffer = 128M
join_buffer = 128M
key_buffer = 512M
max_allowed_packet = 200M
thread_stack = 192K
thread_concurrency = 8
thread_cache_size = 64
query_cache_limit = 256M
query_cache_size = 256M
table_cache = 8192
query_cache_type = 1
sort_buffer = 128M
record_buffer = 128M
myisam_sort_buffer_size = 128M
thread_cache = 64
max_user_connections = 500
wait_timeout = 200
max_connections = 4096
tmp_table_size = 1000M
max_heap_table_size = 1000M
...snip...

Now, you may ask why these settings are bad. I will tell you. First, an equation for calculating per-thread buffer memory usage.

total per-thread buffer usage = (read_buffer_size + read_rnd_buffer_size + sort_buffer_size + thread_stack + join_buffer_size + binlog_cache_size) * max_connections

So in this case we have the following memory usage in per-thread buffers. Any buffers or settings not specifically mentioned above take the MySQL defaults.

1537GB = ( 128M + 256K + 128M + 192K + 128M + 32K) * 4096

Yes, you read that correctly. This server is setup to allow 1537GB of ram usage on the per-thread buffer level. This server only has 4GB of ram. You tell me what would happen if 4096 connections were made and all of them, or even just a few of them, tried to max out their available buffer space. So folks, remember to always check your per-thread variables and make sure you’re not over allocating your memory resources.

MySQL Monitoring: SNMP for metrics collection

A nice write up here: http://www.masterzen.fr/2009/04/13/introducing-mysql-snmp/ “It’s a Net-SNMP perl subagent that connects to your MySQL server, and reports various statistics (from show status or show innodb status, or even replication) through SNMP.”

This might find its way into Kontrollbase soon…