Saving those Magic Ubuntu Environment Variables

Hello World!

This one is more for me than for you.  I often find a piece of software that needs just some magic environment variable set with some magic path that never seems to get properly configured during installation.  Below is an example of how to get that path set, and then ensure it is always set when you log on to the server from then on out.

# These instructions are for bash
$ echo $SHELL
/bin/bash

# Check the current value of your envvar
$ echo $CAFFE_ROOT

# Add the envvar to ~/.profile so it will load automatically when you login
$ echo "export CAFFE_ROOT=/home/username/caffe/" >> ~/.profile

# Load the new configuration
$ source ~/.profile

# Check the new envvar value
$ echo $CAFFE_ROOT
/home/username/caffe/

Dealing with Pesky Image Names in Cocos

CocoImage

Hello World!

Coco_2

If you are not familiar with Microsoft CoCos, you should be.  Its a treasure trove of data for your learning pleasure!  There just happens to be one pesky problem with it, and that is the fact that when attempting to find the files for training/testing; the Annotation file that ships with MS CoCo does not include the actual file name, but rather the image id.  This sounds fine, except the data when you download it has a bunch of trailing stuff!  In this article we will go through how to get it ready.

Continue reading

Microsoft Cognitive Toolkit + VS Code = Awesome

CNTKIntellisense

Hello World!

In this article I’m going to go through how to set up CNTK with Visual Studio Code and take advantage of those PASCAL GPUs I know everybody has these days.  I will also do a breif overview of what CNTK and Visual Studio Code are and why they are so incredible for machine learning scientists.

CNTKIntellisense

Continue reading

Linking CuDNN

Hello World,

This is quick and dirty post, because I run into this problem all the time and need a place to find the answer quickly.

Here is what happens:

ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory

Here is the answer:

drcrook@BigBen:/usr/local/cuda$ sudo cp include/cudnn.h /usr/include
drcrook@BigBen:/usr/local/cuda$ sudo cp lib64/libcudnn* /usr/lib/x86_64-linux-gnu/
drcrook@BigBen:/usr/local/cuda$ sudo chmod a+r /usr/lib/x86_64-linux-gnu/libcudnn*

If you are struggling getting your GPU initialized with Theano, Tensorflow or really any deep learning framework, this is probably something you may want to do.

Operationalizing SKLearn with Azure Machine Learning

web_servic-ified

Hello World!

So I just completed an incredible project with Brain Thermal Tunnel Genix, where I learned so much about pattern recognition, machine learning and taking research and algorithms and pushing those into a production environment where it can be integrated into a real product.  Today’s article takes those lessons and provides a sample on how to perform complex modelling and operationalize it in the cloud.  The accompanying Gallery Example can be found here.

Continue reading

Arduino + Reinforcement Learning = Autonomous Robot

Herby_V1

Hello World,

So there are a ton of articles out there on the theory of Reinforcement Learning, but very few with an actual application.  I watched a few lectures from Berkley, and read a few articles by NVidia and thought, “Well, lets just give this a shot”.  8 hours later, this is what I had.

Herby_V1

Herby V1 simply learns to go forward as much as possible while avoiding obstacles.

Continue reading

Functional Coding in Python – The right way

Hello World!

So I’ve been on a bender with this Python thing not having proper functional piping.  I just can’t beleive it was not built in.  Every language should have this.  It cleans up your code so much its unbeleive-able.  Fear not however, I have built one for Python :D.  Now I just need to figure out how to create pip packages.  Anyways, lets just do a quick walk through on what it is, how it works etc.

Continue reading

Decoding Woes Solved – Python

Hello World,

This is a short post.  Basically I had a data set come in, where there were some funky characters involved.  I was getting “Can’t read this; doesn’t appear to be UTF-8”.  Looked around on stackoverflow for a while to little avail.  I came up with this, which works.

dataPath = "C:\\data\\CompanyA\\DavidCrook\\davidData_Session1.csv"
fil = open(dataPath)
txt = fil.readlines()
txt = ''.join(txt)
works = pd.read_csv(StringIO(txt), index_col = 0)
doesntWork = pd.read_csv(dataPath, index_col = 0)

Just read the sucker with the standard file open and line reader, push it into a StringIO and then read into a data frame.  Guess what I’m doing from now on.

#MicroBlogPost 🙂

Standardize Continuous Data Shape for Neural Networks

Hello World,

So this is an interesting problem.  You are collecting data from somewhere and you want to feed it into a neural network for classification.  There is one main problem with this.  The shape of the data!  Neural networks and really just anything require specifically shaped data, you can’t just like give it something of ambiguous size.  There are tons of papers out there on dimensionality reduction, but nothing on dimensionality reduction to a specified size.  This article explains my approach.

Continue reading

Time Series Discovery with Python

Hello World,

This article is loosely based on a time series challenge from customer data.  I have fabricated 3 data files such that they represent the same challenge and we will go through the process of discovering that data.  The primary challenge in this data set is that it is from a sleep study and the researchers left the date portion of the time stamp off.  What this means is that at midnight, the data plots at the beginning of the x-axis.  The second challenge is lining up data to see if there is anything interesting with the time.  So yes, you can simply plot using the index that python generates, however I’m also interested in the actual time itself as this is a study involving humans.

Continue reading