python

C-Style Unions And Python

So, you’re creating a C Union (used to create a variant type) and writing it to a file, socket, etc and want to read that in Python. There are two ways to deal with this.

Assume the following union definition in C

In C, reading the value represented by this is easy.  Since its 4 bytes, you simply read 4 bytes and then reference the appropriate element.  In Python, if your looking for functionality to closely match C, it seems not so straight forward.

struct.pack and struct.unpack

The first thing you try to do is look at the struct module and see if pack and unpack can come close to doing what you want.  The problem with pack and unpack is that it requires a data type.

This works just as well as anything and is completely straightforward, the big problem here is speed.  First, we have to do an if around each call to unpack to get the appropriate option.  Second, its faster to pull in arrays in python than single values.

A ctypes addition to struct.pack and struct.unpack

Using ctypes, you can approach a functionality similar C.  Take the following code for example.

Notice that it is always unpacking the data as an integer into the integer part of the union.  This approach has a few advantages.  One, it functions the same as the C version of the code would.  Two, you can unpack entire arrays at once which can be faster.

Conclusions

The first code sample seems to be the simplest and most straightforward though potentially slower.  However, depending on the situation, you may want an implementation similar to the second.

Posted by Chad Dotson in Programming, 0 comments

A Better AsyncTestCase for Python

If you need asynchronous support in python unit tests, there are a few possibilities.  A simple google search points you to a few packages, one being tornado.  Tornado’s tornado.testing.AsyncTestCase seems straight-forward.  However it does have a shortcoming: it’s not thread-safe.  Trying to use it with a thread will result in a sporadic exception.  Consider this thread-safe, functionally similar alternative.

 

 

 

Posted by Chad Dotson in Doing Things Better, Programming, 0 comments

Automating Pylint with Gulp.js

Automating Pylint (and other Python Tasks) can be achieved with several viable python-based methods, but what if we used Gulp.js?  The following code snippet gathers runs Pylint on the set of python files defined by pySource.

Notes:

  • This is just a first cut.  I may find a better way.
  • I am aware that I could have simply used gulp-shell to call pylint with a collection of directories.
  • I am open to feedback on this.  Let me know if I’m doing something wrong or inefficient.
Posted by Chad Dotson in Doing Things Better, Software Engineering, Tips, 0 comments

Generating JSON Documents From SQLite Databases In Python

Special Note

This article assumes that you do not wish to use a more sophisticated ORM tool such as SQLAlchemy.

Some Setup

Let’s start with a Q&D sqlite database given the following sql.

You can create the sqlite database given the following command.

Some Different Methods

For this example we want each record returned via the sql select statement to be its on JSON document.  There are several ways of doing this.  All of them solve the problem reasonably well but I was in search of the best way.  In checking python.org, I discovered that the sqlite connection object has an attribute falled row_factory.  This attribute can be modified provide selection results in a more advanced way.

Method 1 – My Preferred Method

From the python docs, we find that they already have a good factory for generating dictionaries.  It is my opinion that this functionality to should be more explicitly enabled in the language.

In this method, we override the row_factory attribute with a callable function that generates the python dictionary from the results.

 Method 2 – Almost As Good As Method 1

This method is just about as good as method 1.  Matter of fact, you can get away with this one and be just fine.  Functionally, the methods are almost identical.  With this method, the records can be accessed via index or via column name.  The biggest difference is that unlike method 1, these results don’t have the full functionality of a python dictionary.  For most people, this might be enough.

Putting It All Together

The following code snippet will extract a group of dictionaries based on the select statement from the sqlite database and dump it to JSON for display.

The Code

 The Results

 

 

Posted by Chad Dotson in Programming, Tips, 3 comments
My First Raspberry Pi Project

My First Raspberry Pi Project

View of the whole setup

View of the whole setup

Almost every year, I try to select a fun project to do while off for Christmas. This year the project I selected was based on Raspberry Pi.  Specifically I decided to build a device that would, when motion is detected, find the range to the motion using an ultrasonic rangefinder, take a picture using the camera module, and finally email me the results.

After just a few hours of work, I had separate proofs of concept working for each sensor written in Python.  A short time after, I started combining all the proofs of concept into a consolidated script which I got working.  Right now I am perfecting the classes for interfacing with the range finder and the motion detector and I am currently using picamera 0.8 for interfacing with the camera module.  I’ve included a link to the git repository I’m using for source control below.

Equipment:

  • Raspberry Pi Model B
  • Raspberry Pi 5MP Camera Board Module
  • HC-SR501 Human Sensor Module Pyroelectric Infrared
  • SainSmart HC-SR04 Ranging Detector Mod Distance Sensor
  • 5 180 ohm resistors
  • Breadboard and miscellaneous connectors

Source Code: https://github.com/chaddotson/PiPy

 

Posted by Chad Dotson in Featured, Hobbies, Neat Stuff, Raspberry Pi, 0 comments

DateTakenPhotoSorter.py – A python script for sorting photos

I decided this script was a necessity since I’ve been arranging my photos based on date taken since I bought my first Canon point and shoot.  After buying my Nikon 5000, I was disappointed to find the Nikon’s sync software did not break photos out into directories based on the date they were taken.  Yesterday, I finally had time to sit down and code it up.  This script should work on any platform and can work via either command line or via drag-and-drop.  Simply give it one or more directories using either of those methods and it will sort the photos in each directory based on the “Image DateTaken” Exif field.

Download Link: DateTakenPhotoSorter

SVN: http://cdotson-utilities.googlecode.com/svn/trunk/DateTakenPhotoSorter

Posted by Chad Dotson in Photography, Programming, 0 comments