Filed under Uncategorized

Win a free book at the February Python Book Contest

This month is a special month. It’s not because of Valentines day or even the exciting day where we see groundhogs. No, this month is special because I’m have a book contest where you, the reader, get to win something free for doing absolutely nothing more than posting a comment saying that you want one of the several books I have available in the contest.

So without getting into boring details I’ll keep this short. I’ve been reviewing a lot of books lately and I think it’s time to get some books into people’s hands to enjoy themselves. This month the giveaways are all Python oriented.

So, all you have to do is take a look at the following titles and post a comment here saying that you want one of them. At the end of the month two readers will be chosen via a random list sorting python script I’ve whipped up for just this purpose. You will then get an email from the publisher who will send a brand new e-copy of the book free of charge. I’ll also be reviewing these books at a later date for those that do not win the contest.

Python Text Processing with NLTK 2.0 Cookbook

Python 2.6 Text Processing: Beginners Guide

Python 2.6 Graphics Cookbook

Post a comment now and tell me which book you want!

Easy Python: multi-threading MySQL queries [updated]

There are many times when writing an application that single threaded database operations are simply too slow. In these cases it’s a matter of course that you’ll use multi-threading or forking to spawn secondary processes to handle the database actions. In this simple example for Python multi-threading you’ll see the how simple it is to improve the performance of your python app.

#!/usr/bin/env python
@author: Matt Reid
@date: 2012-01-20
@purpose: example to show thread-safe MySQL connection pooling with multi-threaded SQL execution
@license: new BSD:
@requires: Python 2.6, MySQL Server 5.0+, MySQL-python connector
@SQL table create statement:
  CREATE TABLE `test` (
  `col_a` varchar(255) DEFAULT NULL,
  `col_b` varchar(255) DEFAULT NULL,
  `col_c` varchar(255) DEFAULT NULL,

import threading
import sys
import MySQLdb
import pdb

class threader(threading.Thread):
    def __init__(self,db):
        self.db = db
    def run(self):

def run_insert(db):
    sql = "INSERT INTO `test` (`id`,`col_a`,`col_b`,`col_c`) VALUES (NULL,'0','0','0');"
    print "thread executing sql:%s"%(sql)
        cursor = db.cursor()
        print "insert failed"


def init_thread(): 
    backgrounds = []
    for db in connections:
        print "connection: %s"%(db)
        background = threader(db)

    for background in backgrounds:

def main():
        print "failed to initiate threads"

if __name__ == "__main__":
    mysql_host = "localhost" 
    mysql_user = "root" 
    mysql_pass = "password" 
    mysql_port = int(3306)
    mysql_db = "test" 
    threads = int(4) #quantity of execution threads and size of connection pool

    connections = []
    for thread in range(threads):
            connections.append(MySQLdb.connect(host=mysql_host, user=mysql_user, passwd=mysql_pass, db=mysql_db, port=mysql_port))

        except MySQLdb.Error, e:
            print "Error %d: %s"%(e.args[0], e.args[1])
            sys.exit (1)



$> python ./ 
connection: <_mysql.connection open to 'localhost' at 2389850>
thread executing sql:INSERT INTO `test` (`id`,`col_a`,`col_b`,`col_c`) VALUES (NULL,'0','0','0');
connection: <_mysql.connection open to 'localhost' at 240da70>
thread executing sql:INSERT INTO `test` (`id`,`col_a`,`col_b`,`col_c`) VALUES (NULL,'0','0','0');
connection: <_mysql.connection open to 'localhost' at 24c6d70>
thread executing sql:INSERT INTO `test` (`id`,`col_a`,`col_b`,`col_c`) VALUES (NULL,'0','0','0');
connection: <_mysql.connection open to 'localhost' at 23c6290>
thread executing sql:INSERT INTO `test` (`id`,`col_a`,`col_b`,`col_c`) VALUES (NULL,'0','0','0');

* this script has been updated

How to install MongoDB on CentOS 5.4 / RHEL5 and interface with PHP 5

If you’ve been reading up on the various NoSQL offerings and have wanted to try out one but don’t know how to get started, this is one of the easiest ways. I chose MongoDB for this example because I’m going to start using it for a project that needs features that MySQL isn’t as fast at: namely denormalized data with billions of rows. MongoDB has plenty of drivers for other scripting and high-level languages but I’ll focus on the PHP driver today. If there is interest I can do a write up on Python usage later. This example is limited to CentOS, Fedora, and Redhat 5 servers that use the yum package management system. For more information you can reference their download page:

First install the prerequisites:

  • sudo yum install gcc php php-pear

Then install the mogo php extension via pecl and add the extension to the ini file. (Increase the php.ini memory_limit to 32M if necessary). Then we restart apache to have the extension loaded. You can check your extensions via the phpinfo(); command in php.

  • sudo pecl install mongo
  • sudo echo “” >> /etc/php.ini
  • sudo /etc/init.d/httpd restart

For the last part we add the yum repo and get the actual MongoDB server and client installed

  • cd /etc/yum.repos.d
  • sudo emacs mongodb.10gen.repo
  • paste in the following. This uses the repo for 64bit servers. See their download page for other architecture repo settings.
  • [10gen]
    name=10gen Repository

  • sudo yum install mongo-stable-server
  • sudo /etc/init.d/monod start
  • chkconfig mongod on

That’s it, you’re ready to start using Mongo! You can check the logfile to make sure things are running smoothly: “sudo cat /var/log/mongo/mongod.log”