---
title: "`FIESTA` Manual - Module Estimates"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{`FIESTA` Manual - Module Estimates}
  %\VignetteEngine{knitr::rmarkdown}
  \usepackage[utf8]{inputenc}
---

```{r setup, include = F}
library(knitr)
knitr::opts_chunk$set(message = F, warning = F)
```

```{r, include=FALSE}
# Sets up output folding
hooks = knitr::knit_hooks$get()
hook_foldable = function(type) {
  force(type)
  function(x, options) {
    res = hooks[[type]](x, options)
    
    if (isFALSE(options[[paste0("fold.", type)]])) return(res)
    
    paste0(
      "<details><summary>", type, "</summary>\n\n",
      res,
      "\n\n</details>"
    )
  }
}
knitr::knit_hooks$set(
  output = hook_foldable("output"),
  plot = hook_foldable("plot")
)
```

```{r, echo=-1}
data.table::setDTthreads(2)
```

## Overview

FIESTA's Estimation Modules combine multiple functions from FIESTA or other packages to generate estimates across a user-defined population(s) using different estimation strategies. Each module has an associated mod\*pop function for compiling the population data and calculations, including adjustments for nonresponse and standardizing auxiliary data. The output form the mod\*pop functions are input directly into the mod\* estimation modules.

All Estimation Modules require similar inputs, including a set of population data output from associated mod\*pop functions and other parameters defining filters and row and/or column domains. 
Refer to the following vignettes for specific examples.


* [FIESTA_tutorial_GB](FIESTA_tutorial_GB.html)
* [FIESTA_tutorial_MA](FIESTA_tutorial_MA.html)
* [FIESTA_tutorial_SA](FIESTA_tutorial_SA.html)


## Module Parameters {#input}


The parameters for FIESTA modules are organized by different categories based on population data and resulting estimates.

[Estimates](#estout):

1. [Plot/condition filters to subset table records](#filt)
2. [Tree information for tree estimates - modGBtree()](#tree) 
3. [Tree information for ratio estimates - modGBratio()](#rat)
4. [Output table information](#out1)
5. [Output table specifications](#out2)
6. [Output table title(s)](#title)




### Estimates {#estout}

<a name="filt"></a>
**Estimation Filters**

Filters subset the area of the sample population for the desired output. Filters do no change the population data used for estimate (e.g., number of plots, strata weights). Some filters are defined internally, depending on the dataset, such as land area of interest (landarea) or inclusion of ACI (All Condition Inventory) data. Others are specified according to the input table, such as pfilter or cfilter. The nonsamp.filter is for removing nonsampled conditions and is applied, internally, unless otherwise stated. *All filter expressions must be in R syntax*.

* **landarea**       - A predefined cond filter depending on desired sample land area {ALL, FOREST (COND_STATUS_CD=1), TIMBERLAND (SITECLCD in(1:6) & RESERVCD=0)}.
* **pcfilter**    - Plot/Condition-level filter.


<a name="tree"></a>
**Tree information for tree estimates - modGBtree()**

Tree information is used for calculating estimates for derived tree data, such as basal area (BA), volume (e.g., VOLCFNET), or number of trees (i.e., TPA_UNADJ). All variables except number of tree variables are extrapolated to the acre plot size by multiplying by TPA_*). 

* **estvar**        - Name of estimate variable in tree (e.g., VOLCFNET, TPA_UNADJ).
* **estvar.filter** - Tree-level filter (e.g., 'STATUS_CD == 1 & DIA >= 5.0').
* **estvar.name**   - A name for aggregated estvar (Default = estvar'_SUM').

<a name="rat"></a>
**Tree information for ratio estimates - modGBratio()**

Tree information is used for calculating per-acre ratio estimates (numerator) and per-tree ratio estimates (numerator and denominator) from derived tree data, such as basal area (BA), volume (e.g., VOLCFNET), or number of trees (i.e., TPA_UNADJ). All variables except number of tree variables are extrapolated to the acre plot size by multiplying by TPA_*).

* **ratiotype** - If ratio estimate, the ratio type ("PERACRE", "PERTREE").
* **estvar(n)**        - Numerator - Name of estimate variable (e.g., VOLCFNET, TPA_UNADJ)
* **estvar(n).filter** - Numerator - Tree-level filter(s).
* **estvar(n).name**   - Numerator - A name for aggregated estvar (Default = estvar(n)'_SUM').
* **estvard**        - Denominator - Name of estimate variable (e.g., VOLCFNET, TPA_UNADJ)
* **estvard.filter** - Denominator - Tree-level filter(s).
* **estvard.name**   - Denominator - A name for aggregated estvar (Default = estvard'_SUM').


**Table estimates**

Estimates can be broken down by categorical domains for further detail about a population. These details are presented as rows or rows and columns in a matrix table, with a separate, associated sampling error for each cell in the table. If the area of interest (i.e., population) has more than one estimation unit (i.e., subpopulation), estimates may be presented by estimation unit or summed to the population. 

* **rowvar**    - Name of row variable (domain). If area estimate, rowvar must be from cond (e.g., FORTYPCD). If tree or ratio estimate, rowvar can be from cond or tree (e.g., SPCD).
* **colvar**    - Name of column variable (domain). If area estimate, colvar must be from cond (e.g., FORTYPCD). If tree or ratio estimate, colvar can be from cond or tree (e.g, SPCD).
* **sumunits**  - TRUE, sum estimates by estimation units (sub-populations) into one estimate (e.g., Counties to State).


**Other**

* **returntitle**  - TRUE, return table titles (See title_opts for more title parameters)
* **savedata**  - TRUE, save data to outfolder (See savedata_opts for more savedata parameters).


<a name="table_opts"></a>
**Output table options (table_opts)**

The following parameters provide more detail about the presentation and aesthetics of the final output table, including: using code names instead of codes; applying specific ordering; adding categories not in the data set with 0 values; displaying percent standard errors and estimates together or separate; and rounding options. These parameters are set by supplying a list to the `table_opts` parameter. The possible parameters that can be set within the `table_opts` parameter can be seen by running `help(table_options)`.

* **row.FIAname**   - TRUE, gets FIA reference name for rowvar, if available (`FIESTA::getRefcodes(rowvar)` for availability).
* **col.FIAname**   - TRUE, gets FIA reference name for colvar, if available (`FIESTA::getRefcodes(rowvar)` for availability).
* **row.orderby**   - Name of variable to order values of rowvar (if row.FIAname=FALSE).
* **col.orderby**   - Name of variable to order values of colvar (if col.FIAname=FALSE).
* **row.add0**      - TRUE, add 0 value in table for row values that occur in `FIESTA::getRefcodes(rowvar)` or in rowlut, but are not in final estimate.
* **col.add0**      - TRUE, add 0 value in table for column values that occur in `FIESTA::getRefcodes(colvar)` or in collut, but are not in final estimate.
* **rowlut**        - A lookup table with variable codes and descriptions to include in rows of output table (See note following for more details).
* **collut**        - A lookup table with specific variable codes and descriptions to include in columns of output table (See note following for more details).
* **rawonly**       - If TRUE, only raw data are returned (this option is more efficient for estimates with many estimation units, rows, and/columns).
* **raw.keep0**     - If TRUE, keeps 0 values in raw data tables. If FALSE, removes 0 values for efficient output.
* **allin1**     - TRUE, table cells include: estimates (% sample error).
* **metric**     - TRUE, output is returned in metric units.
* **estround**   - Number of digits to round estimates to.
* **pseround**   - Number of digits to round percent sample error (pse) to.
* **estnull**    - Number or character to indicate 'not sampled' for estimate.
* **psenull**    - Number or character to indicate 'not sampled' for percent sample error (pse).
* **divideby**   - Conversion of output numbers ('hundred', 'thousand', 'million').


Note: 
rowlut/collut - There are several objectives for including rowlut/collut look-up tables: 1) to include descriptive names that match row/column codes in the input table; 2) to use number codes that match row/column names in the input table for ordering rows; 3) to add rows and/or columns with 0 values for consistency. No duplicate names are allowed.

Include 2 columns in the table: 1-the merging variable with same name as the variable in the input merge table; 2-the ordering or descriptive variable. If the ordering variable is the rowvar/colvar in the input table and the descriptive variable is in rowlut/collut, set row.orderby/col.orderby equal to rowvar/colvar. If the descriptive variable is the rowvar/colvar in the input table, and the ordering code variable is in rowlut/collut, set row.orderby/col.orderby equal to the variable name of the code variable in rowlut/collut. 


<a name="title_opts"></a>
**Title parameters (title_opts)**

If returntitle=TRUE, a default title is generated based on the estimation parameters above. The following parameters allow user-defined inputs for generating customized titles. These parameters are set by supplying a list to the `title_opts` parameter. The possible parameters that can be set within the `title_opts` parameter can be seen by running `help(title_options)`.

* **title.main** - Full title for table.
* **title.ref**  - The ending text for table title. If not NULL, included with title.main.
* **title.rowvar**  - Pretty name for rowvar for table title (If NULL, default = rowvar).
* **title.colvar**  - Pretty name for colvar for table title (If NULL, default = colvar).
* **title.unitvar** - Pretty name for unitvar for table title (If NULL and sumunits=FALSE, default = unitvar).
* **title.estvar**  - Pretty name for estvar for table title (If NULL, default = estvar). For ratio estimates, title.estvarn (default = estvarn) and title.estvard (default = estvard).
* **title.filter**  - Pretty name for any filters for table title.


<a name="savedata_opts"></a>
**Saving data specifications (savedata_opts)**

If savedata=TRUE, output is saved using default settings. The following parameters specify different options for saving output. These parameters are set by supplying a list to the `savedata_opts` parameter. The possible parameters that can be set within the `savedata_opts` parameter can be seen by running `help(savedata_options)`.

* **outfolder**  - Name of folder to output data to (Default = working directory or window to browse).
* **out_fmt**    - Format of output data tables ('sqlite', 'gpkg', 'csv', 'gdb').
* **out_dsn**    - TRUE, data source name of database to output data tables.
* **outfn.pre**  - Add a prefix to output name (e.g., '01_*').
* **outfn.date** - Add date (YYYYMMDD) to output name (e.g., '*_20200220').
* **addtitle**   - TRUE, add title to tables.
* **raw_fmt**    - Format of raw data tables ('sqlite', 'gpkg', 'csv', 'gdb').
* **raw_dsn**    - TRUE, data source name of database to output raw data tables.
* **overwrite_dsn**  - TRUE, overwrites existing data source name database files (e.g., sqlite).
* **overwrite_layer**  - TRUE, overwrites layer within a database or file in outfolder, if exists.
* **append_layer**  - TRUE, appends output to existing layer in database or file in outfolder.





<a name="output"></a>

## Output values from `FIESTA` modules

Estimates with percent sampling error for the row domain (and column domain) specified by the input parameters. This can be in the form of one table or two separate tables, depending on the number of domains and on allin1 parameter. 

A list object with one or more of the following components. If savedata=TRUE, all output data frames are written to outfolder.

* **$est** - Data frame with estimates by rowvar, colvar (and estimation unit). If sumunits=TRUE or one estimation unit and colvar=NULL, estimates and percent sampling error (confidence level 68%) are all in est. For 95% percent confidence level multiply percent sampling error by 1.96.
* **$pse** - Data frame with percent sampling errors corresponding to est.
* **$raw** - A list of raw data used in the estimation process.
* **$titlelst** - If returntitle=TRUE, a list with one or two titles for est and pse, depending on number of output data frames (see below for list components).



### Raw data used for producing estimates (If rawdata=TRUE, raw$):

The raw data includes the domain-level data set used for estimation and separate data frames with calculated variables used in estimation process. The number of processing tables depends on the input parameters. The tables include:

* **raw$domdat**      - Plot domain data used for estimation.
* **raw$unit_totest** - Total by estimation unit 
* **raw$unit_rowest** - If rowvar != NULL, rowvar totals by estimation unit
* **raw$unit_colvar** - If colvar != NULL, colvar totals by estimation unit
* **raw$unit_grpvar** - If colvar != NULL, a combination of rowvar and colvar by estimation unit
* If sumunits=TRUE, the raw data for the summed estimation units are also included: (totest, rowest, colest, grpest, respectively). These tables do not included estimate proportions (nhat and nhat.var). See below for variable descriptions.
* **raw$totest** - Total by estimation unit, summed to population 
* **raw$rowest** - If rowvar != NULL, rowvar totals by estimation unit, summed to population
* **raw$colvar** - If colvar != NULL, colvar totals by estimation unit, summed to population
* **raw$grpvar** - If colvar != NULL, a combination of rowvar and colvar by estimation unit, summed to population 


```{r, results = 'asis', echo=FALSE}
nonratio <- data.frame(Variable = c("nhat", "nhat.var", "est", "est.var"), Description = c("Estimated proportion", "Variance estimate of estimated proportion", "Estimated acres { nhat * ACRES }", "Variance estimate of estimated acres { nhat * ACRES^2 }"), stringsAsFactors = FALSE)

ratio <- data.frame(Variable = c("nhat", "nhat.var", "dhat", "dhat.var", "covar", "estn", "estd", "estn.var", "estn.se", "estn.cv", "estn.pse", "estd.var", "estd.se", "estd.cv", "estd.pse", "est.covar", "rhat", "rhat.var", "rhat.se", "rhat.cv", "est", "est.var"), Description = c("Estimated proportion of land, for numerator", "Variance estimate of estimated proportion of land, for numerator", "Estimated proportion of land, for denominator", "Variance estimate of estimated proportion of land, for denominator", "Covariance of estimated proportion of numerator and denominator", "Estimated acres, for numerator", "Estimated acres, for denominator", "Variance estimate of estimate acres, for numerator", "Standard error estimated acres, for numerator", "Coefficient of variation of estimated acres, for numerator", "Percent sampling error of estimate, for numerator", "Variance estimate of estimate acres, for denominator", "Standard error estimated acres, for denominator", "Coefficient of variation of estimated acres, for denominator", "Percent sampling error of estimate, for denominator", "Covariance of estimated acres of numerator and denominator", "Ratio of estimated proportions (numerator/denominator)", "Variance of ratio of estimated proportions", "Standard error of ratio of estimated proportions { rhat.se/rhat }", "Coefficient of variation of ratio of estimated proportions { sqrt(rhat.var) }", "Estimated percent cover of land { rhat*100 }", "Variance of estimated percent cover of land { rhat.var*100^2 }"), stringsAsFactors = FALSE)

all <- data.frame(Variable = c("NBRPLT.gt0", "ACRES", "est.se", "est.cv", "pse", "CI99left", "CI99right", "CI95left", "CI95right", "CI68left", "CI68right"), Description = c("Number of nonzero plots used in estimate", "Total acres for estimation unit", "Standard error of estimated acres { sqrt(est.var) }", "Coefficient of variation of estimated acres { est.se/est }", "Percent sampling error of estimate { est.cv * 100 }", "Left tail of 99% confidence interval for estimate { est - (2.58*est.se) }", "Right tail of 99% confidence interval for estimate { est + (2.58*est.se) }", "Left tail of 95% confidence interval for estimate { est - (1.96*est.se) }", "Right tail of 95% confidence interval for estimate { est + (1.96*est.se) }", "Left tail of 68% confidence interval for estimate { est - (0.97*est.se) }", "Right tail of 68% confidence interval for estimate { est + (0.97*est.se) }"), stringsAsFactors = FALSE)



kable(nonratio,
  format = "pandoc",   # default
  caption = "Description of variables in nonratio tables.",
  col.names = names(nonratio),
  row.names = FALSE,
  align = c("l"),       # align = c("c", "c", "c", "r")
  # padding = 2         # inner spacing
) 


kable(ratio,
  format = "pandoc",   # default
  caption = "Description of variables in nonratio tables.",
  col.names = names(ratio),
  row.names = FALSE,
  align = c("l"),       # align = c("c", "c", "c", "r")
  # padding = 2         # inner spacing
) 


kable(all,
  format = "pandoc",   # default
  caption = "Description of variables in nonratio and ratio tables.",
  col.names = names(all),
  row.names = FALSE,
  align = c("l"),       # align = c("c", "c", "c", "r")
  # padding = 2         # inner spacing
) 

```



<a name="ref"></a>

## Reference Tables in `FIESTA`

There are several reference tables stored in `FIESTA`, including table variable descriptions, code definitions, and estimation categories. 

* Table variable descriptions - ref_plt, ref_cond, ref_tree
* Code definitions - ref_codes
* Domain variables - ref_domain
* Estimation variables - ref_estvar

getdomain()
getestvar()



