Exploratory IoT Analysis with R

Hello World!

These days I need to make videos instead of written articles, so I am going to post a few of those here.

In this video we will do an initial exploratory analysis on a water flow data set that came from a prototype that I built. The prototype consists of a water pump, a valve and a flow meter. The data set exists in SQL Azure. We will use R and R Studio to perform the analysis from an Azure virtual machine.

The code is:

install.packages("RODBC")
require("RODBC")
d <- odbcDriverConnect(connection = "Driver={SQL Server Native Client 11.0};Server=tcp:qvr9aar3i6.database.windows.net,1433;Database=TelemetryDB;Uid=drcrook@qvr9aar3i6;Pwd=David!2345;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;")
sqlColumns(d, "WaterFlows")
flows <- as.data.frame(sqlFetch(d, "WaterFlows"))
summary(flows)
flows
flows["WaterFlow"]
summary(flows["WaterFlow"]) 

var(flows["WaterFlow"])
sd(flows$WaterFlow) 

nrow(flows["WaterFlow"])
plot(flows["WaterFlow"])
nrow(flows[flows$WaterFlow < 9, ])
plot(flows[flows$WaterFlow < 9, ]["WaterFlow"])

#initial boxplot showing outliers
boxplot(
  x = flows$WaterFlow,
  xlab = "Flow/Minute",
  horizontal = TRUE
)
#calculate outlier threshold
outlierThreshold <- sd(flows$WaterFlow) * 1.5
#calculate mean
m <- mean(flows$WaterFlow)
#filter out anything under threshold
filtFlows <- flows[flows$WaterFlow > (m - outlierThreshold),]
#filter out anything over threshold
filtFlows <- filtFlows[filtFlows$WaterFlow < (m + outlierThreshold),]
summary(flows$WaterFlow)
summary(filtFlows$WaterFlow)
#new box plot, no outliers
boxplot(
  x = filtFlows$WaterFlow,
  xlab = "Flow/Minute",
  horizontal = TRUE
)
#line graph w/outliers
plot(
  y = flows$WaterFlow,
  x = flows$CollectionTime,
  type = "l"
)
#line graph no outliers
plot(
  y = filtFlows$WaterFlow,
  x = filtFlows$CollectionTime,
  type = "l"
)
t <- ts(flows$WaterFlow, frequency = 6307200)
plot(t)
plot(density(flows$WaterFlow))
plot(density(filtFlows$WaterFlow))

Leave a Reply

Your email address will not be published. Required fields are marked *