`R/preprocessing.R`

`timeseries_dataset_from_array.Rd`

Creates a dataset of sliding windows over a timeseries provided as array

```
timeseries_dataset_from_array(
data,
targets,
sequence_length,
sequence_stride = 1L,
sampling_rate = 1L,
batch_size = 128L,
shuffle = FALSE,
...,
seed = NULL,
start_index = NULL,
end_index = NULL
)
```

- data
array or eager tensor containing consecutive data points (timesteps). The first axis is expected to be the time dimension.

- targets
Targets corresponding to timesteps in

`data`

.`targets[i]`

should be the target corresponding to the window that starts at index`i`

(see example 2 below). Pass NULL if you don't have target data (in this case the dataset will only yield the input data).- sequence_length
Length of the output sequences (in number of timesteps).

- sequence_stride
Period between successive output sequences. For stride

`s`

, output samples would start at index`data[i]`

,`data[i + s]`

,`data[i + (2 * s)]`

, etc.- sampling_rate
Period between successive individual timesteps within sequences. For rate

`r`

, timesteps`data[i], data[i + r], ... data[i + sequence_length]`

are used for create a sample sequence.- batch_size
Number of timeseries samples in each batch (except maybe the last one).

- shuffle
Whether to shuffle output samples, or instead draw them in chronological order.

- ...
For backwards and forwards compatibility, ignored presently.

- seed
Optional int; random seed for shuffling.

- start_index, end_index
Optional int (1 based); data points earlier than

`start_index`

or later then`end_index`

will not be used in the output sequences. This is useful to reserve part of the data for test or validation.

A `tf.data.Dataset`

instance. If `targets`

was passed, the
dataset yields batches of two items: `(batch_of_sequences, batch_of_targets)`

. If not, the dataset yields only
`batch_of_sequences`

.

This function takes in a sequence of data-points gathered at equal intervals, along with time series parameters such as length of the sequences/windows, spacing between two sequence/windows, etc., to produce batches of timeseries inputs and targets.

Consider indices `0:99`

. With `sequence_length=10`

, `sampling_rate=2`

,
`sequence_stride=3`

, `shuffle=FALSE`

, the dataset will yield batches of
sequences composed of the following indices:

```
First sequence: 0 2 4 6 8 10 12 14 16 18
Second sequence: 3 5 7 9 11 13 15 17 19 21
Third sequence: 6 8 10 12 14 16 18 20 22 24
...
Last sequence: 78 80 82 84 86 88 90 92 94 96
```

In this case the last 3 data points are discarded since no full sequence can be generated to include them (the next sequence would have started at index 81, and thus its last step would have gone over 99).

Temporal regression.

Consider an array `data`

of scalar values, of shape `(steps)`

.
To generate a dataset that uses the past 10
timesteps to predict the next timestep, you would use:

```
steps <- 100
# data is integer seq with some noise
data <- array(1:steps + abs(rnorm(steps, sd = .25)))
inputs_data <- head(data, -10) # drop last 10
targets <- tail(data, -10) # drop first 10
dataset <- timeseries_dataset_from_array(
inputs_data, targets, sequence_length=10)
library(tfdatasets)
dataset_iterator <- as_iterator(dataset)
repeat {
batch <- iter_next(dataset_iterator)
if(is.null(batch)) break
c(input, target) %<-% batch
stopifnot(exprs = {
# First sequence: steps [1-10]
# Corresponding target: step 11
all.equal(as.array(input[1, ]), data[1:10])
all.equal(as.array(target[1]), data[11])
all.equal(as.array(input[2, ]), data[2:11])
all.equal(as.array(target[2]), data[12])
all.equal(as.array(input[3, ]), data[3:12])
all.equal(as.array(target[3]), data[13])
})
}
```

Temporal regression for many-to-many architectures.

Consider two arrays of scalar values `X`

and `Y`

,
both of shape `(100)`

. The resulting dataset should consist of samples with
20 timestamps each. The samples should not overlap.
To generate a dataset that uses the current timestamp
to predict the corresponding target timestep, you would use:

```
X <- seq(100)
Y <- X*2
sample_length <- 20
input_dataset <- timeseries_dataset_from_array(
X, NULL, sequence_length=sample_length, sequence_stride=sample_length)
target_dataset <- timeseries_dataset_from_array(
Y, NULL, sequence_length=sample_length, sequence_stride=sample_length)
library(tfdatasets)
dataset_iterator <-
zip_datasets(input_dataset, target_dataset) %>%
as_array_iterator()
while(!is.null(batch <- iter_next(dataset_iterator))) {
c(inputs, targets) %<-% batch
stopifnot(
all.equal(inputs[1,], X[1:sample_length]),
all.equal(targets[1,], Y[1:sample_length]),
# second sample equals output timestamps 20-40
all.equal(inputs[2,], X[(1:sample_length) + sample_length]),
all.equal(targets[2,], Y[(1:sample_length) + sample_length])
)
}
```

```
int_sequence <- seq(20)
dummy_dataset <- timeseries_dataset_from_array(
data = head(int_sequence, -3), # drop last 3
targets = tail(int_sequence, -3), # drop first 3
sequence_length = 3,
start_index = 3,
end_index = 9,
batch_size = 2
)
library(tfdatasets)
dummy_dataset_iterator <- as_array_iterator(dummy_dataset)
repeat {
batch <- iter_next(dummy_dataset_iterator)
if (is.null(batch)) # iterator exhausted
break
c(inputs, targets) %<-% batch
for (r in 1:nrow(inputs))
cat(sprintf("input: [ %s ] target: %s\n",
paste(inputs[r,], collapse = " "), targets[r]))
cat("---------------------------\n") # demark batchs
}
```

Will give output like:

```
input: [ 3 4 5 ] target: 6
input: [ 4 5 6 ] target: 7
---------------------------
input: [ 5 6 7 ] target: 8
input: [ 6 7 8 ] target: 9
---------------------------
input: [ 7 8 9 ] target: 10
```