Format HoboWare output for use with `ContDataQC` package. Works on single files. Imports, modifies, and saves the new file.
Single file (or vector of files) to perform functions.
Directory for import data. Default is current working directory.
Directory for export data. Default is current working directory.
Date format of Hoboware output, excluding a delimiter (e.g., DMY not D/M/Y). Default is NULL and no transformation of the dates is performed.
Configuration file to use for this data analysis. The default is always loaded first so only "new" values need to be included. This is the easiest way to control time zones.
No data frames are returned. A CSV file ready for use with the ContDataQC QC function will be generated in the specified output directory.
1. Imports a HoboWare output from a folder. Minimal tweaks from within HOBOware.
2. Reformats it using defaults from the ContDataQC config file
3. Exports a CSV to the provided folder for use with ContDataQC
Below are the default data directories assumed to exist in the working directory. These can be created with code in the example. Using this function as an example, files will be imported from Data0_Original and exported to Data1_RAW.
* ./Data0_Original/ = Unmodified data logger files.
* ./Data1_RAW/ = Data logger files modified for use with library. Modifications for extra rows and file and column names.
* ./Data2_QC/ = Repository for library output for QCed files.
* ./Data3_Aggregated/ = Repository for library output for aggregated (or split) files.
* ./Data4_Stats/ = Repository for library output for statistical summary files.
File format should be "SiteID_SensorType_StartDate_EndData.csv".
* SiteID = no spaces or underscores
* SensorType = Air, Water, or AW (Air + Water in the same file)
* Dates = YYYYMMDD (no delimiter)
* Delimiter = underscore (as specified in the config file)
Column names are inspected with regular expressions (R not Perl) to find matches before being renamed. If column names do not match the criteria below they will not be formatted properly.
* Date = "date"
* Air Temperature = "air.\*temp" or "temp.\*air"
* Water Temperature = "water.\*temp" or "temp.\*water"
* Air Presssure = "barom.\*pres" or "pres.\*barom"
* Water Pressure = "pres" (search for Air Pressure first)
* Sensor Depth = "depth"
* Water Level = "level"
* Dissolved Oxygen (conc) = "do conc"
* Dissolved Oxygen (adj conc) = "do" & "adj" (searches for both)
* Dissolved Oxygen (
HOBOware will output ambiguous dates with only 2 digits. There are two delimiters (/ and -) and three formats (MDY, DMY, and YMD) resulting in six possible formats. If the user provides input for fun.HoboDateFormat the function will modify the data to proper 4 digit years. The default of NULL will not modify the date format.
It is assumed the user has a single Date Time field rather than two fields (Date and Time).
if (FALSE) {
# Parameters
Selection.Operation <- c("GetGageData"
, "QCRaw"
, "Aggregate"
, "SummaryStats")
Selection.Type <- c("Air", "Water", "AW", "Gage", "AWG", "AG", "WG")
Selection.SUB <- c("Data0_Original"
, "Data1_RAW"
, "Data2_QC"
, "Data3_Aggregated"
, "Data4_Stats")
myDir.BASE <- tempdir()
# Create data directories
myDir.create <- file.path(myDir.BASE, Selection.SUB[1])
ifelse(dir.exists(myDir.create) == FALSE
, dir.create(myDir.create)
, "Directory already exists")
myDir.create <- file.path(myDir.BASE, Selection.SUB[2])
ifelse(dir.exists(myDir.create) == FALSE
, dir.create(myDir.create)
, "Directory already exists")
myDir.create <- file.path(myDir.BASE, Selection.SUB[3])
ifelse(dir.exists(myDir.create) == FALSE
, dir.create(myDir.create)
, "Directory already exists")
myDir.create <- file.path(myDir.BASE, Selection.SUB[4])
ifelse(dir.exists(myDir.create) == FALSE
, dir.create(myDir.create)
, "Directory already exists")
myDir.create <- file.path(myDir.BASE, Selection.SUB[5])
ifelse(dir.exists(myDir.create) == FALSE
, dir.create(myDir.create)
, "Directory already exists")
# Save example data (assumes directory ./Data0_Original/ exists)
fn_1 <- "Charlies_Air_20170726_20170926.csv"
fn_2 <- "Charlies_AW_20170726_20170926.csv"
fn_3 <- "Charlies_Water_20170726_20170926.csv"
fn_4 <- "ECO66G12_AW_20160128_20160418.csv"
fn_5 <- "EXAMPLE_DO_RUSSWOOD--02M_DO_20180918_20190610.csv"
lapply(c(fn_1, fn_2, fn_3, fn_4, fn_5), function(x)
file.copy(system.file("extdata", x, package="ContDataQC")
, file.path(myDir.BASE, Selection.SUB[1], x)))
# Function Inputs
myFiles <- c("Charlies_Air_20170726_20170926.csv"
, "Charlies_AW_20170726_20170926.csv"
, "Charlies_Water_20170726_20170926.csv")
myDir.import <- file.path(myDir.BASE, "Data0_Original")
myDir.export <- file.path(myDir.BASE, "Data1_RAW")
# Run Function (with default config)
formatHobo(myFiles, myDir.import, myDir.export)
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# QC Data
myData.Operation <- "QCRaw" #Selection.Operation[2]
myFile <- myFiles
myDir.import <- file.path(myDir.BASE, "Data1_RAW")
myDir.export <- file.path(myDir.BASE, "Data2_QC")
myReport.format <- "html"
ContDataQC(myData.Operation
, fun.myDir.import = myDir.import
, fun.myDir.export = myDir.export
, fun.myFile = myFile
, fun.myReport.format = myReport.format)
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Example with unmodified dates
myFiles <- "ECO66G12_AW_20160128_20160418.csv"
myDir.import <- file.path(myDir.BASE, "Data0_Original")
myDir.export <- file.path(myDir.BASE, "Data1_RAW")
HoboDateFormat <- "MDY"
# Run Function (with default config)
formatHobo(myFiles, myDir.import, myDir.export, HoboDateFormat)
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Example with multiple DO fields
myFiles <- "EXAMPLE_DO_RUSSWOOD--02M_DO_20180918_20190610.csv"
myDir.import <- file.path(myDir.BASE, "Data0_Original")
myDir.export <- file.path(myDir.BASE, "Data1_RAW")
HoboDateFormat <- "MDY"
# Run Function (with default config)
formatHobo(myFiles, myDir.import, myDir.export, HoboDateFormat)
}