Chad Dotson - Part 2

Jan

2017

Pip Installs From Github

Pip installs From github

Installing a package from github is fairly simple. The following are examples of installing packages from github.

# Install repo package from the url, master branch
pip3 install git+https://github.com/username/repo.git

@ Install repo from the specified branch
pip3 install git+https://github.com/username/repo.git@branch

# Install repo from the specified commit.
pip3 install git+https://github.com/username/repo.git@commit

# Install repo package from the url, master branch

pip3 install git+https://github.com/username/repo.git

@ Install repo from the specified branch

pip3 install git+https://github.com/username/repo.git@branch

# Install repo from the specified commit.

pip3 install git+https://github.com/username/repo.git@commit

requirements.txt

At some point, you will find yourself wanting to list a dependency in the requirements.txt file that resides on github. This is fairly straight forward If you only plan to use it in requirements.txt (not processed for usage in setup.py). Note that I’ve specified the master branch of repo and given it a version id.

https://github.com/username/repo/tarball/master#egg=repo-0.1.1

1	https://github.com/username/repo/tarball/master#egg=repo-0.1.1

setup.py and requirements.txt

Some people will process their requirements.txt files to generate the install_requires parameter for the setup function called in setup.py. This works fine until you have a repository on github. Setup will fail to find your dependencies if your requirements.txt has a line like the one above. To remedy this we must do two things.

Parse the line to create a named python dependency for install requires. Given the file above, install_requires would equal ["repo==0.0.1"] .
Specify the dependency_links argument to setup. ["https://github.com/chaddotson/repo/tarball/master#egg=repo-0.1.1"] for this example.

An Example

This is an example of a setup.py that properly processes requirements.txt dependencies that are located on github. It’s probably not complete, but it works for what I need. Feel free to take and adapt. See the repo here (python3).

from setuptools import setup

version = '0.1.0'

temp_install_reqs = []
install_reqs = []
dependency_links = []

with open("requirements.txt", "r") as f:
    temp_install_reqs = list(map(str.strip, f.readlines()))

for req in temp_install_reqs:
    # This if should be expanded with all the other possibilities that can exist.  However, this
    # simple version works for this program.
    if req.startswith("https://"):
        dependency_links.append(req)
        install_reqs.append(req[req.find("egg=") + 4:].replace("-", "==", 1))
    else:
        install_reqs.append(req)

setup(
    name='tweeter-repeater',
    version=version,
    packages=['bin'],
    url='',
    license='',
    author='Chad Dotson',
    author_email='chad@cdotson.com',
    description='',
    install_requires=install_reqs,
    dependency_links=dependency_links,
    entry_points={
        'console_scripts': [
            'tweeter-repeater = bin.main:main',
        ]
    },
)

from setuptools import setup

version = '0.1.0'

temp_install_reqs = []

install_reqs = []

dependency_links = []

with open("requirements.txt", "r") as f:

temp_install_reqs = list(map(str.strip, f.readlines()))

for req in temp_install_reqs:

# This if should be expanded with all the other possibilities that can exist. However, this

# simple version works for this program.

if req.startswith("https://"):

dependency_links.append(req)

install_reqs.append(req[req.find("egg=") + 4:].replace("-", "==", 1))

else:

install_reqs.append(req)

setup(

name='tweeter-repeater',

version=version,

packages=['bin'],

url='',

license='',

author='Chad Dotson',

author_email='chad@cdotson.com',

description='',

install_requires=install_reqs,

dependency_links=dependency_links,

entry_points={

'console_scripts': [

'tweeter-repeater = bin.main:main',

]

)

Jan

2017

Jack Daniels 2016 Barrel Tree in Lynchburg, TN

Jul

2016

July 8, 2016 Lightning

These are the first lightning photos in several months. It was a good electrical storm but the lightning wasn’t so photogenic.

Jun

2016

Raspberry Pi 3 Mini Cluster

I recently constructed a mini cluster consisting of 3 Raspberry Pi 3s. I am planning on using these machines for various little projects. I’ve installed some services on one of the nodes. Currently that node hosts Redis and RabbitMQ instances. I’m sure that I’ll have other services to install in the near future. I may even use it to tinker with clustering those services. Below are some pictures of the cluster as I was putting it together and a parts list.

: Initial node. RabbitMQ, Redis, etc

: First 2 nodes

: 2 nodes assembled

: All 3 nodes assembled

: Cluster wired up ready to go

: Final build. Wired lan removed until needed

Materials:

3x Raspberry Pi 3s – https://www.amazon.com/gp/product/B01CD5VC92/
3x 32GB Class 10 MicroSDs – https://www.amazon.com/gp/product/B00IVPU786/
3x Micro-USB to USB – https://www.amazon.com/gp/product/B00NH13O7K/
60W 6 port USB Charger- https://www.amazon.com/gp/product/B0115MVRO4/
GeauxRobot Enclosure – https://www.amazon.com/gp/product/B01D90TX1O/

Apr

2016

Odd Behavior in Python 2.7

I was tinkering around with replacing the print statement with the print function in a Python 2 script when I ran across this peculiar oddity.

>>> from __future__ import print_function as print
  File "<stdin>", line 1
    from __future__ import print_function as print
                                                 ^
SyntaxError: invalid syntax
>>> from __future__ import print_function as print_
>>> from __future__ import print_function as print

>>> from __future__ import print_function as print

File "<stdin>", line 1

from __future__ import print_function as print

SyntaxError: invalid syntax

>>> from __future__ import print_function as print_

>>> from __future__ import print_function as print

Notice that sequence is just importing the future print function as different names in each iteration. The oddity is that the first import fails but the third (which is exactly the same) succeeds after performing the second.

** Note: I don’t know if replacing print with the print from the futures module is a wise thing to do. I was simply using it while trying out some code. **

Versions tested: Python 2.7.10 (OSX), Python 2.7.6 (Ubuntu 14.04).

Apr

2016

Saving Console Messages in JavaScript

Have you ever encountered a situation where it would be beneficial to save whatever is written to the console across browser sessions? If you have, you know there are not many solutions. I recently decided that localStorage would be a good candidate to implement a solution to this problem with.

logsaver.js

logsaver.js is a lightweight, simple log preservation script. It utilizes the flexibility of JavaScript to replace the default console.log function with a new, customized one. Along with writing messages to the original console.log, this custom console.log also writes messages to the specified localStorage key.

To start saving console.log messages, call startSavingLog().
To stop saving console.log messages, call stopSavingLog().
To clear saved console.log messages, call clearSavedLog().
To completely remove the localStorage key used by logsaver, call removeSavedLog().

For more information on options and how to use them, checkout the readme.

While you can copy the source below, I strongly recommend grabbing the latest from github here: logsaver.js.

Notes:

It doesn’t handle objects well. Results in [Object object] appearing in the localStorage key value. This is because localStorage is limited to strings.
It is possible to exceed the maximum space allocated to you in localStorage. For desktop browsers, Chrome and Firefox have a 10 MB limit, while Safari and IE both have a 5 MB limit. These number differs for mobile browsers. More details. In this event, logsaver.js will stop trying to save the log and display additional error messages in the console.

v0.1.0 Source

/*
logsaver.js v0.1.0
https://github.com/chaddotson/logsaver/
(c) 2016 Chad Dotson
logsaver.js may be freely distributed under the GNUv3 license.
*/

(function(root) {
    'use strict';
        
    var _root = root;
    
    var _localStorageKeyForLog = "console.log.saved";
    var _includeTimestamp = false;
    var _originalLog;
        
    function _isEnabled() {
        return _originalLog !== undefined;
    }
    
    function _zeroPad(str) {
        str = str + ""; // force it to be a string.
        return ((str.length < 2) ? "0" + str : str);
    
    }
    
    function _formatDate() {
       return this.getFullYear() + "/" + _zeroPad(this.getMonth()) + "/" + _zeroPad(this.getDay()) + " - " + _zeroPad(this.getHours()) + ":" + _zeroPad(this.getMinutes()) + ":" + _zeroPad(this.getSeconds());
    }
    
    _root.clearSavedLog = function () {        
        if(localStorage[_localStorageKeyForLog]) {
            localStorage[_localStorageKeyForLog] = "";
            return true;
        }
        return false;
    };
    
    _root.removeSavedLog = function () {
        if(localStorage[_localStorageKeyForLog] !== undefined) {
            localStorage.removeItem(_localStorageKeyForLog);
            return true;
        }
        return false;
    };
    
    _root.stopSavingLog = function () {
        if(!_isEnabled()) {
            console.log("Not currently saving console.log to localStorage.");
            return false;
        }
        
        console.log = _originalLog;
        _originalLog = undefined;
        console.log("No longer saving console.log to localStorage.");
        return true;
    };
    
    _root.startSavingLog = function (options) {
        options = options || {};
                
        if(_isEnabled()) {
            _originalLog.call(console, "Already saving console.log. localStorage Key:" + _localStorageKeyForLog);
            return false;
        }
                
        _localStorageKeyForLog = options.keyForLocalStorage || _localStorageKeyForLog;
        var timestampFormatter = options.timestampFormatter || _formatDate; 
        
        _originalLog = console.log;
        
        console.log = function(){
            _originalLog.apply(console, arguments);
                                    
            var formattedTimestamp = timestampFormatter.apply(new Date());
                        
            var message =  "\n "+ formattedTimestamp + " :: " + Array.prototype.join.call(arguments, ", ");
            
            try {
                if(!localStorage[_localStorageKeyForLog]) {
                    localStorage[_localStorageKeyForLog] = "";
                }
                localStorage[_localStorageKeyForLog] += message;    
            } catch (e) {
                if (e == QUOTA_EXCEEDED_ERR) {
                    _originalLog.apply(console, arguments.splice(0, 0, "SAVE FAILED - LOCAL STORAGE QUOTA EXCEEDED"));
                } else {
                    _originalLog.apply(console, arguments.splice(0, 0, "SAVE FAILED"));
                }
                
            }
        }
        
        _originalLog.call(console, "Now saving console.log to localStorage Key:" + _localStorageKeyForLog);
        return true;
    };
        
})(this);

logsaver.js v0.1.0

https://github.com/chaddotson/logsaver/

logsaver.js may be freely distributed under the GNUv3 license.

(function(root) {

'use strict';

var _root = root;

var _localStorageKeyForLog = "console.log.saved";

var _includeTimestamp = false;

var _originalLog;

function _isEnabled() {

return _originalLog !== undefined;

}

function _zeroPad(str) {

str = str + ""; // force it to be a string.

return ((str.length < 2) ? "0" + str : str);

}

function _formatDate() {

return this.getFullYear() + "/" + _zeroPad(this.getMonth()) + "/" + _zeroPad(this.getDay()) + " - " + _zeroPad(this.getHours()) + ":" + _zeroPad(this.getMinutes()) + ":" + _zeroPad(this.getSeconds());

}

_root.clearSavedLog = function () {

if(localStorage[_localStorageKeyForLog]) {

localStorage[_localStorageKeyForLog] = "";

return true;

}

return false;

};

_root.removeSavedLog = function () {

if(localStorage[_localStorageKeyForLog] !== undefined) {

localStorage.removeItem(_localStorageKeyForLog);

return true;

}

return false;

};

_root.stopSavingLog = function () {

if(!_isEnabled()) {

console.log("Not currently saving console.log to localStorage.");

return false;

}

console.log = _originalLog;

_originalLog = undefined;

console.log("No longer saving console.log to localStorage.");

return true;

};

_root.startSavingLog = function (options) {

options = options || {};

if(_isEnabled()) {

_originalLog.call(console, "Already saving console.log. localStorage Key:" + _localStorageKeyForLog);

return false;

}

_localStorageKeyForLog = options.keyForLocalStorage || _localStorageKeyForLog;

var timestampFormatter = options.timestampFormatter || _formatDate;

_originalLog = console.log;

console.log = function(){

_originalLog.apply(console, arguments);

var formattedTimestamp = timestampFormatter.apply(new Date());

var message = "\n "+ formattedTimestamp + " :: " + Array.prototype.join.call(arguments, ", ");

try {

if(!localStorage[_localStorageKeyForLog]) {

localStorage[_localStorageKeyForLog] = "";

}

localStorage[_localStorageKeyForLog] += message;

} catch (e) {

if (e == QUOTA_EXCEEDED_ERR) {

_originalLog.apply(console, arguments.splice(0, 0, "SAVE FAILED - LOCAL STORAGE QUOTA EXCEEDED"));

} else {

_originalLog.apply(console, arguments.splice(0, 0, "SAVE FAILED"));

}

_originalLog.call(console, "Now saving console.log to localStorage Key:" + _localStorageKeyForLog);

return true;

};

})(this);

Feb

2016

Raspberry Pi 7″ Touchscreen Unboxing and Thoughts

This week I ordered and received the official Raspberry 7″ touchscreen ($60). I’ve been tinkering with some ideas for it (some car based projects) and wanted it on hand to try out.

Opening the Box

: Excellently packaged 7″ touch display

: Standard safety guide in many languages.

: 4 Screws, DSI Cable, and Jumpers.

: The screen in anti-static container.

The screen is packaged and presented really well. It comes with all that you need to connect and power it from the Pi itself. It comes with 4 screws to mount the Pi, the ribbon cable to connect it to the Pi’s DSI port, and 4 jumper wires, 2 of which are for powering the display on the Raspberry Pi 2 B+. The other 2 are used when attaching the display to an older model A or B.

The Screen Itself

: The touchscreen

: The controller for the display.

The screen itself seems really well made, I’m really impressed by it. The LCD side seems pretty sturdy and the backside seems better than I would have expected. Mine has a little give around the are the ribbons from the control board enter the LCD itself, but its nothing to complain about.

Connecting It All

: The controller for the display.

: The Raspberry Pi mounted to the screen.

In this case with the Raspberry Pi 2 B+, the screen connects easily via DSI ribbon and 2 jumpers to 5V and ground. Once wired its an easy job to mount the Pi onto the screen. In these pics you also see my cheap wifi and old keyboard dongle. I think I will want to buy a good Bluetooth adapter and keyboard shortly. I found a really need looking one here: http://www.amazon.com/gp/product/B00BX0YKX4.

Up and Running

: Raspbian booted up to console.

: Raspbian GUI up and running.

Once all connected, the Pi boots right up to the console. At this point you need the external keyboard to actually login and run startx, unless you’ve already configured Raspbian to do so. You will also need to install a virtual keyboard like Florence or Matchbox.

Using It

The biggest shortcoming is the need for a physical keyboard, like I mentioned in the preceding section, and it is probably the biggest drawback. The out of the box touch features seems only to augment physical keyboard functionality. That’s because unlike Android or iOS, the virtual keyboard isn’t dynamic, it is either always there or not. I suppose that a custom interface would be more touch oriented. The one oddity I have found is that counter to other touch screens, the hotspot for a touch seems off and to me it seems more exaggerated near the edges of the screen. It’s not a show stopper, but takes some getting used to.

Overall Opinion

After getting it all hooked up, I found that it seems a bit clunky. It left me wondering if a better solution isn’t just a raspberry pi running as a server and using a Android or iOS device as a head. However, assuming you stick with the Raspberry driving its own display, a longer cable might be desirable, depending on your project. In that case a regular HDMI touchscreen with usb connections might be the better fit.

The lack of good cases is also a problem. Nothing I have found yet, fully encloses the pi and the display. Coincidentally, I had an old project box laying around that seems to fit almost perfectly. At 7.5″ x 4.3″, the Hammond 1591ESBK‘s dimensions are only a little shy of the actual dimensions of the displays bezel (less than 0.25″). I’ll post new photos if / when I do that.

Other Resources:

Official Announcement – https://www.raspberrypi.org/blog/the-eagerly-awaited-raspberry-pi-display/
Buying the display: http://www.mcmelectronics.com/product/RASPBERRY-PI-RASPBERRYPI-DISPLAY-/83-16872
Pimoroni’s FAQ – http://forums.pimoroni.com/t/official-7-raspberry-pi-touch-screen-faq/959

: Excellently packaged 7″ touch display

: Standard safety guide in many languages.

: 4 Screws, DSI Cable, and Jumpers.

: The screen in anti-static container.

: The touchscreen

: The controller for the display.

: The Raspberry Pi mounted to the screen.

: Raspbian booted up to console.

: Raspbian GUI up and running.

: Hammond project box that seems to fit the Raspberry Pi display almost perfectly.

Feb

2016

Bringing the Internet to Mars and Beyond

With the exploration of space and the eventual colonization of Mars humanity needs to establish a method for improving internet connectivity for people on the surface.

One method of achieving this is to place one or more satellites in high orbit around the planet. These would use interplanetary internet(delay tolerant) technology and protocols to service requests as well as actively mirror content for primary sites located on Earth. The goal of this strategy is to reduce the latency of user requests from 8-48 minutes down to at most a few seconds for popular content.

What is needed for a single node:

High bandwidth communications equipment similar to what would have been on the Mars Telecommunications Orbiter.
A cluster of servers in an a spacecraft the the size of a supply module. Something similar to Microsoft’s Project Natick would be about right for a first generation. There would be a redundant capacity built in to cover eventual server failure.
A heat management system.
A nuclear power generation system.

Now we just need NASA, Microsoft, SpaceX, Amazon (w/Blue Origin) AWS – Mars, or Google to make it happen!