---
title: "Continuous variables"
description: > 
  Learn the workflow for loading and analysing data collected during the
  elicitation of continuous variables.
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Continuous variables}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set( #nolint
  collapse = TRUE,
  comment = "#>"
)
```

```{r setup}
library(elicitr)
```

## Datasets

There are two datasets that can be used for practising and demonstrating. These datasets contains simulated data for two rounds of elicitation of continuous variables. The first dataset is called `?round_1` and the second `?round_2`. Both datasets contain the same variables, but the values are less disperse in the second round.

```{r round-1}
round_1
```

```{r round-2}
round_2
```

The first column in the dataset stores the name of the expert (randomly generated for demonstration only). The second column contains estimates of a variable collected with a _one point_ elicitation, where only the best estimate is provided. Columns 3 to 5 contain estimates of another variable collected with a _three points_ elicitation, where minimum, maximum, and best values are provided. Finally, columns 6 to 9 contain estimates of another variable collected with a _four points_ elicitation, for which minimum, maximum, best, and confidence are given.

## Load data

Any analysis of continuous variables starts by creating the `?elic_cont` object with the function `cont_start()`. This object stores the matadata of the elicitation process that are then used to validate the data belonging to each elicitation round. To build this `elic_cont` object, four parameters must be specified: 

* `var_names`: the name of the variables estimated during the elicitation process
* `var_types`: the type of the variable for each of these variables (many options are available, ranging from real numbers to probabilities)
* `elic_types` the type of elicitation for each of these variables (three options are available: one, three, and four points elicitation)
* `experts` the number of experts that participated to the elicitation
* `title` used to bind a name to the elicitation process

```{r cont-start}
my_elicitation <- cont_start(var_names = c("var1", "var2", "var3"),
                             var_types = "ZNp",
                             elic_types = "134",
                             experts = 6,
                             title = "Continuous elicitation example")
my_elicitation
```

Variable and elicitation types must be provided as string containing short codes. For example, the first variable is expected to be an integer (Z), the second a positive integer (N), and the third a probability (p). Similarly, the first variable has been collected with a _one point_ elicitation process (1), the second with a _three points_ elicitation elicitation process (3), and the third a _four points_ elicitation process (4).

Once the `elic_cont` object has been created, data belonging to each round can be added with the `cont_add_data()` function. Data can be added from different sources, like a data frame, a CSV or xlsx file, or a Google Sheets file. The function will check that the data is consistent with the metadata of the `elic_cont` object. Here we add the data available in the package:

```{r add-data}
my_elicitation <- cont_add_data(my_elicitation,
                                data_source = round_1,
                                round = 1) |>
  cont_add_data(data_source = round_2,
                round = 2)
my_elicitation
```

Notice that the commands can be piped together.
When we load the data into the `elic_cont` object, the name of the experts are anonymised and replaced with an identifier. This is automatic but optional and can be disabled by setting the argument `anonymise = FALSE` in the `cont_add_data()` function.

As explained above, the metadata are used to validate the data added to the `elic_cont` object. If the data is not consistent with the metadata, an error message will be displayed. For example, if we try to add a data frame with a different number of columns:

```{r add-wrong-data, error=TRUE}
cont_add_data(my_elicitation,
              data_source = round_1[, 1:5],
              round = 1)
```

Or if we try to add data where the variable has an incorrect type:

```{r add-wrong-type, error=TRUE}
malformed_data <- round_1
malformed_data[, 2] <- malformed_data[, 2] * 0.1
cont_add_data(my_elicitation,
              data_source = malformed_data,
              round = 1)
```

In this case the first variable is expected to have integer values (`var_type = Z`), but the data contains real numbers.

## Get data

Data can be retrieved from the `elic_cont` object with the `cont_get_data()` function:

```{r get-data}
cont_get_data(my_elicitation, round = 1)
```

Notice that the name of the expert has been anonymised and assigned to the column `id`. If you add the data from a file, the name of the experts will never reach the `elic_cont` object. 

Data can be retrieved only for given variables or for given variable or elicitation types. For example, to get data for the first variable only:

```{r get-data-var1}
cont_get_data(my_elicitation, round = 1, var = "var1")
```

## Data analysis

### Plot raw data

Now that the data has been loaded in R, it can be analysed. First let's plot the data for the first round and the first variable:

```{r plot-data-var1}
#| fig.alt: >
#|   Estimates of variable 1 collected with a one point elicitation process.
plot(my_elicitation, round = 1, var = "var1")
```

When the variable has been collected with a _three_ or _four points_ elicitation process, the plot shows the range of the estimates (from the minimum estimated value to the maximum estimated value, with the best guess estimate represented by a point):

```{r plot-data-var2}
#| fig.alt: >
#|  Estimates of variable 2 collected with a three points elicitation process.
#|  The error bars represent the range of the estimates.
plot(my_elicitation, round = 1, var = "var2")
```

When the variable has been collected with a _four points_ elicitation process, the estimates are rescaled based on the confidence provided by the expert. If the estimates are probabilities, the rescaled values are the constrained to the [0, 1] interval:

```{r plot-data-var3}
#| fig.alt: >
#|   Estimates of variable 3 collected with a four points elicitation process.
plot(my_elicitation, round = 2, var = "var3")
```

Finally, the group mean and 2.5th–97.5th percentile interval of expert estimates can be added to the plot:

```{r plot-data-var3-group}
#| fig.alt: >
#|  Estimates of variable 3 collected with a four points elicitation process
#|  with group mean.
plot(my_elicitation, round = 2, var = "var1", group = TRUE)
```

When the elicitation process is part of a workshop and is used for demonstration, it can be useful to show a truth argument on the plot. This argument can be added as a list of estimates.

```{r plot-data-var3-truth}
#| fig.alt: >
#|  Estimates of variable 3 collected with a four points elicitation process
#|  with group mean and truth.
plot(my_elicitation, round = 1, var = "var2",
     group = TRUE,
     truth = list(min = 10, max = 20, best = 15))
```

Expert names or identifiers can also be changed on the plot:

```{r plot-data-var3-custom-id}
#| fig.alt: >
#|  Estimates of variable 3 collected with a four points elicitation process
#|  with custom expert identifiers.
plot(my_elicitation, round = 1, var = "var2",
     group = TRUE,
     truth = list(min = 10, max = 20, best = 15),
     expert_names = 1:6)
```

### Sample data

Data can be sampled using the function `cont_sample_data()`. 

* one point elicitation: the estimate provided by each expert is repeated `n_votes` number of times. The `weights` argument can be used to weight the estimates of each expert (give a certain number of vote to each expert) in the overall distribution.

* three points elicitation: the minimum, best, and maximum estimates of each expert are used as scaling parameters of the PERT distribution from which the data are sampled. The `weights` argument can be used to weight the estimates of each expert (give a certain number of vote to each expert) in the overall distribution.

* four points elicitation: the minimum, best, and maximum estimates of each expert are rescaled according to their confidence and used as scaling parameters of the PERT distribution from which the data are sampled. Furthermore, their confidence is used as the `weights` argument to weight the estimates of each expert (give a number of vote to each expert) in the overall distribution. Weights can also be provided to avoid this confidence scaling. See the function documentation for the explanation of the sampling methods. 

Here we sample 1000 values for each variable:

```{r sample}
samp <- cont_sample_data(my_elicitation, round = 2)
samp
```

Sampled data can be summarised:

```{r sample-summary}
summary(samp)
```

Plotted as violin plots:

```{r sample-plot-violin}
#| fig.alt: >
#|  Violin plot of the sampled data for variable 2.
plot(samp, var = "var2", type = "violin")
```

Plotted as density plots:

```{r sample-plot-density}
#| fig.alt: >
#|  Density plot of the sampled data for variable 3.
plot(samp, var = "var3", type = "density")
```

Or plotted as beeswarm plots and adapted as needed:

```{r sample-plot-beeswarm}
#| fig.alt: >
#|  Beeswarm plot of the sampled data for variable 1.
plot(samp, var = "var2", type = "beeswarm")
```

```{r sample-plot-beeswarm-wrap}
#| fig.alt: >
#|  Beeswarm plot of the sampled data for variable 1, wrapped.
plot(samp, var = "var2", type = "beeswarm",
     beeswarm_cex = 0.9, beeswarm_corral = "wrap")
```

The same plots can be generated for the group (aggregated data) with violin plots, density plots or beeswarm plots.

```{r sample-plot-violin-group}
#| fig.alt: >
#|  Violin plot of the sampled data for variable 2 with group mean.
plot(samp, var = "var2", type = "violin", group = TRUE)
```

```{r sample-plot-density-group}
#| fig.alt: >
#|  Density plot of the sampled data for variable 3 with group mean.
plot(samp, var = "var3", type = "density", group = TRUE)
```

```{r sample-plot-beeswarm-group}
#| fig.alt: >
#|  Beeswarm plot of the sampled data for variable 3 with group mean.
plot(samp, var = "var3", type = "beeswarm", group = TRUE)
```