---
title: "How to use `rvdat`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{How to use `rvdat`}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---



This vignette is a working draft. There will be lots of changes as `rvdat` gets fleshed out, so please check back often!

The most recent changes were made on 2024-07-08.

## Getting started


```r
library(rvdat)
```

First, as we should always do when beginning a `rvdat` workflow, we tell R where our `vdat.exe` is located. Mine is within an installed version of [Fathom Connect](https://support.fishtracking.innovasea.com/s/downloads).


```r
vdat_here("c:/program files/innovasea/fathom/vdat.exe")
#> ℹ vdat.exe is located at c:/program files/innovasea/fathom/vdat.exe
```

If you don't want to do this every time you re-open R, you can add `VDAT_EXE="PATH_TO_VDAT.EXE"` to your `.Renviron` file. `rvdat` will automatically look for it there when you restart R. You can do this manually, but I find that the easiest way to edit `.Renviron` is using [`usethis::edit_r_environ()`](https://usethis.r-lib.org/reference/edit.html).

Next, let's download a sample VRL file from the `glatos` package using `utils::download.file`. I'll be keeping everything in a temporary directory just to keep tidy.


```r
td <- file.path(
  tempdir(),
  "vignette"
)

dir.create(
  td
)

download.file(
  url = file.path(
    "https://github.com/ocean-tracking-network/glatos/raw/dev/inst/extdata",
    "detection_files_raw/VR2AR_546310_20190613_1.vrl"
  ),
  destfile = file.path(
    td,
    "VR2AR_546310_20190613_1.vrl"
  ),
  mode = "wb",
  quiet = TRUE
)
```

## Check the file's metadata

To take a quick glimpse of the VDAT file's metadata, we can use `vdat_inspect`. Note that this only works for VDAT files (`.vrl` and `.vdat`), not any exported CSVs. `vdat_insepct` converts this information to a data.frame if you'd like to keep it around.


```r
inspect_df <- vdat_inspect(
  file.path(td, "VR2AR_546310_20190613_1.vrl")
)
#> ==============================================================================

#>                                      VRL                                      

#> ==============================================================================

#> File:      VR2AR_546310_20190613_1.vrl

#> Original:  VR2AR_546310_20190613_1.vrl

#> Container: VR2AR VRL file (com.vemco.file.vrl.0207.ff02.ff02/5.0.1)

#> Created:   2019-06-13T13:30:54

#> Data UUID: ffee6ee7-450a-db42-b54c-334b672fddf5

#> Rx Model:  VR2AR-69

#> Rx Serial: 546310

#> 

#> ==============================================================================

#>                                     Device                                    

#> ==============================================================================

#> Decoding Map:      MAP-113

#> Blanking Interval: 260 ms

#> 

#> 


head(inspect_df)
#>    variable                                                    value
#> 1      File                              VR2AR_546310_20190613_1.vrl
#> 2  Original                              VR2AR_546310_20190613_1.vrl
#> 3 Container VR2AR VRL file (com.vemco.file.vrl.0207.ff02.ff02/5.0.1)
#> 4   Created                                      2019-06-13T13:30:54
#> 5 Data UUID                     ffee6ee7-450a-db42-b54c-334b672fddf5
#> 6  Rx Model                                                 VR2AR-69
#>   section
#> 1     VRL
#> 2     VRL
#> 3     VRL
#> 4     VRL
#> 5     VRL
#> 6     VRL
```

## Converting VDAT file to a CSV

Now that we have a VRL to play with and know something about it, let's convert it to a csv. It's pretty quick with `vdat_to_csv`.


```r
vdat_to_csv(
  file.path(td, "VR2AR_546310_20190613_1.vrl"),
  outdir = td
)
#> ✔ File converted:
#>   C:\Users\darpa2\AppData\Local\Temp\RtmpuIFd1a/vignette/VR2AR_546310_20190613_1.vrl
#> ℹ File saved in:
#>   C:\Users\darpa2\AppData\Local\Temp\RtmpuIFd1a/vignette/VR2AR_546310_20190613_1.csv

list.files(td, pattern = "VR2AR")
#> [1] "VR2AR_546310_20190613_1.csv" "VR2AR_546310_20190613_1.vrl"
```
`vdat` is very respectful of your data in that it never overwrites files that exist in the same directory. It'll just tack on a time stamp to the name of the newly-exported file.


```r
vdat_to_csv(
  file.path(td, "VR2AR_546310_20190613_1.vrl"),
  outdir = td
)
#> ✔ File converted:
#>   C:\Users\darpa2\AppData\Local\Temp\RtmpuIFd1a/vignette/VR2AR_546310_20190613_1.vrl
#> ℹ File saved in:
#>   C:\Users\darpa2\AppData\Local\Temp\RtmpuIFd1a/vignette/VR2AR_546310_20190613_1[20240708-12-50-27-xxxxxx].csv

list.files(td, pattern = "VR2AR")
#> [1] "VR2AR_546310_20190613_1.csv"                          
#> [2] "VR2AR_546310_20190613_1.vrl"                          
#> [3] "VR2AR_546310_20190613_1[20240708-12-50-28-002297].csv"
```
## Converting a VDAT file to a CSV with time correction

Adding the `time_corrected = TRUE` argument to `vdat_to_csv` corrects for clock drift using `vdat`'s internal algorithm:


```r
vdat_to_csv(
  file.path(td, "VR2AR_546310_20190613_1.vrl"),
  outdir = td,
  time_corrected = TRUE
)
#> ✔ File converted:
#>   C:\Users\darpa2\AppData\Local\Temp\RtmpuIFd1a/vignette/VR2AR_546310_20190613_1.vrl
#> ℹ File saved in:
#>   C:\Users\darpa2\AppData\Local\Temp\RtmpuIFd1a/vignette/VR2AR_546310_20190613_1[20240708-12-50-28-xxxxxx].csv
```

A new column is added with time corrected down to the microsecond.


```r
corrected <- read.csv(
  list.files(td, pattern = "VR2AR.*\\]\\.csv$", full.names = TRUE)[2],
  header = FALSE
)

corrected[46, c(2:3)]
#>                     V2                         V3
#> 46 2019-06-11 02:47:00 2019-06-11 02:47:00.000725
```


## Splitting a VDAT file into a folder of CSVs

As outlined in `vignette('vdat-data-structure')`, the flat CSV exported via `vdat_to_csv` is a mush-together of all available data types in the VDAT file. `vdat.exe` can split these into a folder of files according to data type for you, accessible via `rvdat` in a few ways:

  -   `vdat_to_folder("PATH_TO_FILE")`, which is a convenience wrapper around...
  -   `vdat_to_csv("PATH_TO_FILE", folder = TRUE)`, which is a convenience wrapper around...
  -   `vdat_call(c('convert', '--format=csv.fathom.split', 'PATH_TO_FILE'))`.


```r
vdat_to_folder(
  file.path(td, "VR2AR_546310_20190613_1.vrl"),
  outdir = td
)
#> ✔ File converted:
#>   C:\Users\darpa2\AppData\Local\Temp\RtmpuIFd1a/vignette/VR2AR_546310_20190613_1.vrl
#> ℹ Files saved in:
#>   C:\Users\darpa2\AppData\Local\Temp\RtmpuIFd1a/vignette/VR2AR_546310_20190613_1.csv-fathom-split

list.dirs(td, full.names = F, recursive = F)
#> [1] "VR2AR_546310_20190613_1.csv-fathom-split"

list.files(
  list.dirs(td, full.names = T, recursive = F)
)
#>  [1] "ATTITUDE.csv"         "BATTERY.csv"         
#>  [3] "CFG_CHANNEL.csv"      "CFG_TRANSMITTER.csv" 
#>  [5] "CLOCK_REF.csv"        "DATA_SOURCE_FILE.csv"
#>  [7] "DEPTH.csv"            "DET.csv"             
#>  [9] "DIAG.csv"             "DIAG_FAST.csv"       
#> [11] "EVENT_INIT.csv"       "EVENT_OFFLOAD.csv"   
#> [13] "HEALTH_VR2AR.csv"     "TEMP.csv"
```


## Other file types

Though we just used a VR2AR's `VRL` file here, this works for multiple different receiver styles and file types. For example, a HR-180 `VDAT` file:


```r
download.file(
  url = file.path(
    "https://github.com/ocean-tracking-network/glatos/raw/dev/inst/extdata",
    "detection_files_raw/HR2-180%20461396%202021-04-20%20173145.vdat"
  ),
  destfile = file.path(
    td,
    "HR2-180 461396 2021-04-20 173145.vdat"
  ),
  mode = "wb",
  quiet = TRUE
)

hr_output <- vdat_inspect(file.path(td, "HR2-180 461396 2021-04-20 173145.vdat"))
#> ==============================================================================

#>                                      VDAT                                     

#> ==============================================================================

#> File:      HR2-180 461396 2021-04-20 173145.vdat

#> Original:  HR2-180 461396 2021-04-20 173145.vdat

#> Container: Vemco Data File (com.vemco.file.vdat/1.0.0)

#> Content:   HR2 Receiver Data Pack (com.vemco.file.vrdp.vrhr2/1.0.0)

#> Created:   2021-04-20T17:31:45

#> File UUID: f1a18604-46ea-4985-b57c-24809d37ad46

#> Data UUID: 1a595886-99ea-4ee9-a996-2e4d92dc01c3

#> Size:      39370

#> Blocks:    17

#> Block Types: 

#>            VRDP Header (com.vemco.file.vrdp.header/1.1.0)

#>            RxLog General Summary (com.vemco.protobuf.rxlog.summary.general/1.0.0)

#>            HR2 Offload Header (com.vemco.device.vrhr2.offload.header/1.0.0)

#>            HR2 Offload Footer (com.vemco.device.vrhr2.offload.footer/1.0.0)

#>            HR2 Detection Log Block (com.vemco.device.vrhr2.log.detect/1.0.0)

#>            HR2 Ping Log Block (com.vemco.device.vrhr2.log.ping/1.0.0)

#> Codecs: 

#>            zlib compression (com.vemco.file.vdat.codec.zlib/1.2.11)

#> 

#> ==============================================================================

#>                                      VRDP                                     

#> ==============================================================================

#> Creator:   fathom-2.5.0-19700101--release

#> Source:    HR2-180 (0814)

#> 

#> ==============================================================================

#>                                    Offload                                    

#> ==============================================================================

#> Firmware Version: 2.0.1

#> Receiver Time @ Init: 2021-04-20T17:20:05Z

#> Client Time @ Init: 2021-04-20T17:20:06Z

#> Client Time Zone @ Init: -04:00

#> Receiver Time @ Offload: 2021-04-20T21:31:45Z

#> Client Time @ Offload: 2021-04-20T21:31:45Z

#> Client Time Zone @ Offload: -04:00

#> 

#> 

head(hr_output)
#>    variable                                                    value
#> 1      File                    HR2-180 461396 2021-04-20 173145.vdat
#> 2  Original                    HR2-180 461396 2021-04-20 173145.vdat
#> 3 Container              Vemco Data File (com.vemco.file.vdat/1.0.0)
#> 4   Content HR2 Receiver Data Pack (com.vemco.file.vrdp.vrhr2/1.0.0)
#> 5   Created                                      2021-04-20T17:31:45
#> 6 File UUID                     f1a18604-46ea-4985-b57c-24809d37ad46
#>   section
#> 1    VDAT
#> 2    VDAT
#> 3    VDAT
#> 4    VDAT
#> 5    VDAT
#> 6    VDAT
```