DDR-Drive gets 300,000 IOPS

Sure it’s expensive and hard to justify for most budgets but this is the future of drive technology happening now. http://www.ddrdrive.com

Simple SQL: growth-over-time query

I’ve been messing around with the Kontrollbase schema for the last couple of days, writing various queries for the daily reporting scripts that will eventually be an automated pdf report. I’ll give you examples of two of the queries, the first being overall environment stats, and the second being single-host growth over time.

Overall environment stats
select ((((MAX(os_mem_used)) / 1024 ) / 1024) / 1024) max_os_mem_used, ((((MIN(os_mem_used)) / 1024 ) / 1024) / 1024) min_os_mem_used, ((((AVG(os_mem_used)) / 1024 ) / 1024) / 1024) avg_os_mem_used, ((((STDDEV_POP(os_mem_used)) / 1024 ) / 1024) / 1024) stdev_os_mem_used, ((((MAX(length_data + length_index)) / 1024 ) / 1024) / 1024) max_size, ((((MIN(length_data + length_index)) / 1024 ) / 1024) / 1024) min_size, ((((AVG(length_data + length_index)) / 1024 ) / 1024) / 1024) avg_size, ((((STDDEV_POP(length_data + length_index)) / 1024 ) / 1024) / 1024) stdev_size, MAX(num_connections) max_connections, MIN(num_connections) min_connections, AVG(num_connections) avg_connections, STDDEV_POP(num_connections) stdev_connections, MAX(queries_per_second) max_qps, MIN(queries_per_second) min_qps, AVG(queries_per_second) avg_qps, STDDEV_POP(queries_per_second) stdev_qps from server_statistics;
*************************** 1. row ***************************
max_os_mem_used: 50.743488311768
min_os_mem_used: 0.023044586182
avg_os_mem_used: 1.5759627057922952
stdev_os_mem_used: 2.4064208226596184
max_size: 283.815660957247
min_size: 0.000492287800
avg_size: 10.8213909777686940
stdev_size: 39.9443980717105447
max_connections: 435
min_connections: 0
avg_connections: 16.9734
stdev_connections: 37.3269
stdev_os_mem_used: 2.4064208226596184
max_qps: 9243.6533203125
min_qps: 0.00011409764556447
avg_qps: 216.421064774444
stdev_qps: 1071.72792986232
1 row in set (0.00 sec)

single-host growth over time
mysql> select (select (((length_data + length_index) / 1024) / 1024) as curr_size from server_statistics where server_list_id='44' and CURDATE() <= Creation_time order by Creation_time asc limit 1) as 0_day_size_mb, (select (((length_data + length_index) / 1024) / 1024) as curr_size from server_statistics where server_list_id='44' and DATE_SUB(CURDATE(),INTERVAL 30 DAY) <= Creation_time order by Creation_time asc limit 1) as 30_day_size_mb, ( (select (((length_data + length_index) / 1024) / 1024) as curr_size from server_statistics where server_list_id='44' and CURDATE() <= Creation_time order by Creation_time asc limit 1) - (select (((length_data + length_index) / 1024) / 1024) as curr_size from server_statistics where server_list_id='44' and DATE_SUB(CURDATE(),INTERVAL 30 DAY) <= Creation_time order by Creation_time asc limit 1)) as difference, ( select (select ((select (((length_data + length_index) / 1024) / 1024) as curr_size from server_statistics where server_list_id='44' and CURDATE() <= Creation_time order by Creation_time asc limit 1) - (select (((length_data + length_index) / 1024) / 1024) as curr_size from server_statistics where server_list_id='44' and DATE_SUB(CURDATE(),INTERVAL 30 DAY) <= Creation_time order by Creation_time asc limit 1)) as difference) / (select(select (((length_data + length_index) / 1024) / 1024) as curr_size from server_statistics where server_list_id='44' and CURDATE() <= Creation_time order by Creation_time asc limit 1) as 0_day_size_mb ) * 100) as percent_growth;
+---------------+----------------+--------------+----------------+
| 0_day_size_mb | 30_day_size_mb | difference | percent_growth |
+---------------+----------------+--------------+----------------+
| 9100.43986130 | 8199.39263916 | 901.04722214 | 9.901139240197 |
+---------------+----------------+--------------+----------------+
1 row in set (0.01 sec)

Interesting links that predict the future of data storage

Ok, they aren’t like the 2012 nonsense or anything but they are interesting articles that will be important topics in the near future for expanding our knowledge and usefulness in the job market. Being a DBA does not limit you to relational database systems. I expect to see more requests for non-relational or No-SQL type of data stores as they can have many advantages over an RDBMS. Stability, scalability, simplification of administration, higher performance on lesser hardware for larger datasets, and many other items make no-sql engines very interesting. Here are some useful links:

http://en.wikipedia.org/wiki/NoSQL

http://www.linux-mag.com/cache/7579/1.html

http://www.computerworld.com/s/article/9135086/No_to_SQL_Anti_database_movement_gains_steam_

http://www.viget.com/extend/nosql-misconceptions/

How can I pass up talking about the various Cloud offerings again. Here are some interesting news stories and a shameless link to the cloud service that my employer runs.

http://aws.typepad.com/aws/2009/10/two-new-ec2-instance-types-additional-memory.html

http://www.opsourcecloud.net/

http://virtualization.sys-con.com/node/770174

This is not related to no-sql or the cloud but I’m going to post it anyway: http://www.continuent.com/community/all-projects

Notes for installing MySQL 5.4.2 on OSX 10.5.8 PPC

Some notes from my recent installation of MySQL 5.4.2-beta on a PPC G5 box running OSX 10.5.8. Hopefully these notes will save someone a bit of time when they begin the “I want to make my mac a development box” process. I’m not going to cover installing/configuring Apache and PHP since the web is full of those articles already.  So, first things first,  I downloaded the pkg file  – not the tarball – from the MySQL downloads site and installed the following files in the following order:

  1. mysql-5.4.2-beta-osx10.5-powerpc-64bit.pkg
  2. MySQLStartupItem.pkg
  3. MySQL.prefPane

CNF File Settings

Here are some interesting settings that you may want to be aware of with 5.4 and OSX

  • Do not set innodb_flush_method=fdatasync – it’s not a valid option, and the default after install is actually blank.
  • Do not set “nice = value” in the [mysqld_safe] section as this is not recognized and give the error ‘nice: illegal option — -
  • Set the variable lower_case_table_names=2 because the HFS+ file system is case insensitive – isn’t that special.
  • The variable ‘log_slow_queries’ will work but you should use ‘slow_query_log’ or ‘slow_query_log_file’ instead since it’s getting removed in *gasp* MySQL 7.0 so we do have a bit of time before that becomes something to be concerned about but you will see it in the error log if you don’t correct it now.

Preference Pane service controller

In order to get the Preference Pane controller to work (if you are getting the following error in /var/log/system.log – “Could not setup Mach task special port 9: (os/kern) no access”) you need to correct the permissions for the installation directory as such
(root@g5)-(/usr/local)-
shell> chown -R _mysql:wheel /usr/local/mysql*

This will still generate the ‘(os/kern) no access‘ message in the system.log but the server will be started/stopped by the pane controller and that’s what we really care about. There may be some more notes along the way as I’m going to be testing 5.4 quite a bit for various application related matters.

An easy method for hiding wordpress pages

I needed to hide a page from the page list – the header panel that shows all of the main site pages. Problem was that I needed it accessible to users via links but not listed in the main header. This page from a google search helped me out:

The last way is to hardcode your menu function. WordPress created the menu with this function: wp_list_pages();
You can either remove that function and add your own links, or add the "exclude" argument to wp_list_pages() like so:
wp_list_pages(exclude=17,38);

So my header.php code ended up looking like so:

$pages = wp_list_pages('sort_column=menu_order&title_li=&echo=0&exclude=377');
$pages = preg_replace('%]+)>%U','', $pages);
$pages = str_replace('
','', $pages);
echo $pages;
?>

Bash Basics: Fun with the pipe

Of course there are better ways to manage security on webservers, but I was playing around with the apache error log file today and wanted to get all of the subnets for the script kiddies that are hammering on this dev box. A quick one liner and we’re done!

ROOT-(0)> grep "error" error_log |awk -F[ {'print $4'}| sort | sed -e 's/]//g' | grep -v "/var/www/html/" | grep -v "/usr/local/src/code" | awk {'print $2'} | grep -v "75.142.246.205" |uniq | awk -F. {'print $1"."$2"."$3"/24"'} |uniq
114.80.93/24
119.63.193/24
124.115.0/24
205.234.253/24
208.115.111/24
208.80.195/24
221.194.132/24
58.61.164/24
65.55.106/24
65.55.207/24
66.249.71/24
67.202.34/24
72.14.199/24
74.125.44/24
76.73.37/24
88.131.106/24
92.52.69/24

How to make an editable data grid using ExtJS

I’m not going to go into a great amount of detail here, but I will post the three files required for the process. If you know PHP (this uses the CodeIgniter framework) and JS (this uses the ExtJS framework) then it will make sense. Hopefully this saves people some time and will serve as a basis for a good project.

Model: http://themattreid.pastebin.com/fd163b0
View: http://themattreid.pastebin.com/f2a14ba54
Controller: http://themattreid.pastebin.com/f40599353

Kontrollbase gets a new default theme

I couldn’t take staring at the default theme anymore so I enabled the Slate CSS. Unless I get sick of this one, it will be the default from here on out. Maybe I’ll make a theme chooser at some point as well. Here’s a screenshot of the Slate theme enabled.  If you’re inclined to enable this on your older version it’s very simple. There are two files to touch: system/application/views/header.php and header_host.php

Add the following to line 41 on header.php and line 44 on header_host.php
<link rel="stylesheet" type="text/css" href="$nroot/includes/extjs/resources/css/xtheme-slate.css" />

If your line numbers are different, basically you just want that css file include to be directly below the “ext-all.css” css include file. If you break it then you probably did something wrong. I recommend making a backup of the files before editing just to be sure. Of course you’ll want to make sure that the “includes/extjs/resources/css/xtheme-slate.css” file exists before trying to include it.

Slate theme

Kontrollbase screenshots are posted

It’s taken a while – for no good reason – but the Kontrollbase screenshots are online for viewing pleasure. There are a couple of the overall environment as well as several of the host-specific pages. Check them out here: http://kontrollsoft.com/screenshots

Kontrollbase gets Memcache support

After working with Memcache on another application I figured that Kontrollbase could benefit from it as well. Expect in the next release (coming this week), easily configurable support for memcache. This makes use of the memcache library for CodeIgniter talked about here: http://codeigniter.com/forums/viewthread/72538/

In the config.php file you just enable it and set the proper connection parameters.

$config['memcache_enabled'] = TRUE;
$config['memcache_ip'] = '127.0.0.1';
$config['memcache_port'] = '11211';

In the main controller I added the following code to the index and host functions.

$this->load->library('cache');
// memcache library information
$memcache = $this->config->item('memcache_enabled');
if($memcache == TRUE) {
$memcache_ip = $this->config->item('memcache_ip');
$memcache_port = $this->config->item('memcache_port');
$memcache = $this->cache->useMemcache($memcache_ip, $memcache_port);
if(!$memcache) {
log_message('debug', "Memcache connection failure.");
show_error("Memcache library not enabled correctly.");
}
else {
log_message('debug', "Memcache enabled!");
}
}
else {
log_message('debug', "Memcache not enabled in config.");
}
//end memcache

And then to load the view into the cache we see this later on…

$this->cache->save('cachedMain',$this->load->view('main/main', $g, TRUE),NULL,3600);