Linear Regression from Scratch using Linear Algebra

Hello World!

So I wrote an article earlier “Linear Regression From Scratch”.  Many folks have pointed out that this is in fact not the optimal approach.  Now being the perfectionist I decided to re-implement.  Not to mention it works great in my own libraries.  The following article discussing converting the original code into code that uses linear algebra.  Beyond this, it still works in PCL for xamarin,  Hoo-Rah Xamarin!

Continue reading

Linear Regression from Scratch

Hello World,

So today we will do a quick conversion from mathematical notations of Algebra into a real algorithm that can be executed.  Note we will not be covering gradient descent, but rather only cost functions, errors and execution of these to provide the framework for gradient descent.  Gradient descent has so many flavors that it deserves its own article.

So to the mathematical representation.

LinearRegression

Continue reading

Convert Latitude Longitude to Degree Minutes Seconds for Maps

Hello World!

If you have ever done mapping applications, you may have encountered needing to do this.  It takes a lot of looking around the internet to finally find the right equation etc.  For our application, we need to do this for google maps, as it does not take a latitude/longitude combination like bing maps.  If you choose to support ONLY bing maps, your job is easy, as here is the format: http://www.bing.com/maps/default.aspx?q=LATITUDE%2c+LONGITUDE (include negative signs if necessary).  However Google maps requires more work (UGH!) https://www.google.com/maps/place/LATITUDE <directional> LONGITUDE <directional>, ZOOM (with various encoded separators).

This article goes through the code that converts latitude longitude like you will pull from a phone’s gps into the DMS format needed by google maps.  Again, you don’t even need to bother with this conversion if you choose to use bing maps, it is simply as stated above.

Continue reading

Using Census Data to Help Pick your Child’s Name

Hello World!

So I had a life changing event this past Sunday at 8:55am 5/24/2015.  My first child was born!  Both child and wife are healthy and happy.  Everything is good in life.  Like many couples though, my wife and I struggled to find the right name for our child.  We didn’t want something too common, or was an old person name, or so rare and funky that nobody could spell it.  We also realized we just had a general lack in knowing what names were out there.  So after much debate and discussion over what to name her, I started doing a bit of an analysis using some census data.  I want to thank Jamie Dixon for providing the data that he found for use in his Dinner Nerds article.  The data itself can be found here.  This article will discuss the code used to go through all of the data and provide insights into child names.

Continue reading

How to Datamine Zillow

Hello World,

As many of you may know at this point, I am relocating to South Florida.  Final location to be determined, but will probably be renting around Pompano Beach or Fort Lauderdale while working out of Venture Hive and the Microsoft Fort Lauderdale Offices.  So what does this have to do with Zillow?  Well, It has EVERYTHING to do with Zillow.  What I’ve found while searching for homes is that between Realtors, Zillow and Trulia, they really just don’t have a predictive analytics solution that works for me.  So I decided to give a shot at AzureML to mash together a few datasets to send me notifications more to my liking than is currently being sent.  So step 1 in this plan is to data mine Zillow.  Luckily, Zillow has an api for that.  Or if you are feeling particularly frisky, Zillow gets their data from ArcGIS (example for Raleigh).  So lets get cracking…

Continue reading

So you want to be an Analytics Developer

Hello World!

I get a series of questions all the time.

  1. How do I switch careers to be a developer?
  2. How do I become a data scientist?
  3. How do I add intelligence to my code?
  4. How do I get a job in distributed computing?
  5. How do I code more analytically?

The answer to these questions are pretty much all the same.  Step 1, learn about it and build one piece of software focused on that goal.  Step 2, go for it, just do it.  So that said, Microsoft has a fantastic resource, Microsoft Virtual Academy, which provides free training around various topics from entry level to advanced.  This article focuses on a learning plan with MVA to attain the goal of becoming an Analytics Developer.

Continue reading

Mocking IoT Telemetry Data with Azure

Hello World!

This should prove to be an interesting series of posts coming up, as I am working on a new project that is very unique and interesting. The idea is to use incoming data from Arduinos, Raspberry Pis, Gallileos, Edisons and other assortments of IoT type devices connected to oil and gas pipelines to determine if a leak is currently in progress and also predict if a leak is likely to occur in the future based on current and trending conditions.

My part in the project is all back end analytics, and I have very little to do with the actual telemetry and hardware. The telemetry will be posted using Azure Event Hubs, and thus my portion of the project begins with mocking that real time data at a large enough geo dispersed scale that I can develop a system that can handle it, and then switch my configurations to consume from the production event hubs. Since I am no longer a consultant working on projects with trade secrets and everything these days is about the elevation of skills in the community, I have posted everything on github that you can download and peruse at your leisure. Please note that this is in progress and well the github source may not necessarily work when you look. I’ll try to enforce a standard to comment “working – comment” on pushes to the repository. The git repository is located here: https://github.com/drcrook1/OilGas

Continue reading

Adopting F#

Hello World!

You can see from my blog that I have been increasing my adoption of F# and my love of the language is growing.  I find the language very conducive to cloud computing.  F# at its core seems to be built for distributed computing.  If you program with F# in an idiomatic way, you are setting yourself up for success on the cloud, as the cloud is a variably sized distributed system that you are deploying production code to.  That is what F# excels at.

As much as I have loved F#, it has not been all roses and sunshine.  There have been issues.  This article at its core is really an ask from the community to support me in building the case and evangelizing F# in such a way it can be recognized and adopted not only by the community, but by those who make the decisions to expend resources on additional tooling and support.  I must iterate that the contents of this article are in no way the opinion of Microsoft, but merely my own observations.  From these observations I have developed a strategy and action items that we the F# community must do to achieve these goals.

First, we must step outside of our normal mindset as F# developers and begin looking at the language from the outside perspective.  We must also look at some trends.

Continue reading