Tutorial 4: Variable Creation and Temporal Aggregation

Objective

By the end of this tutorial you will be able to transform individual microdata into a time series ready for analysis. There are two chained operations and two visualization steps:

Create derived variables — sus_data_create_variables() — generating age groups (user-defined, climate risk-based, and IBGE five-year bands) and calendar variables (year, month, quarter, season);
Aggregate over time and space — sus_data_aggregate() — converting the individual records into counts per period and per municipality (daily, monthly, seasonal, etc.);
Visualize the time series — sus_data_plot_aggregate_ts() — with epidemic curves, seasonal boxplots, calendar heatmaps and annual trends;
Visualize the spatial distribution — sus_data_plot_aggregate_map() — with bubble maps or municipal choropleths.

This is the fourth stage of the pipeline (derive → aggregate). It occupies a critical place: this is where the data stop being a table of individual events and take on the shape required by the climate–health models of Tutorial 9 (a count $Y_t$ per time unit $t$ ).

Scientific context

Time series studies in environmental epidemiology model the outcome count per period (e.g., deaths per day) as a function of environmental exposure (temperature, pollution). To do so, microdata must be aggregated onto a regular temporal grid and, often, stratified by age group, since vulnerability to heat and cold varies greatly with age (Gasparrini et al., 2015; Bhaskaran et al., 2013).

Two methodological decisions at this stage have direct consequences for the analysis:

The temporal aggregation unit (time_unit) defines the study’s resolution. Daily series capture short-term acute effects (ideal for DLNM); monthly or seasonal series highlight long-term patterns and seasonality (Bhaskaran et al., 2013).
Filling in missing dates (complete_dates = TRUE) guarantees a series without gaps: days with no deaths receive a zero count. Without this, time series models would mistakenly interpret the absence of a date as an absence of information, rather than as a genuine zero.

Age groups follow recognized categorizations — including the IBGE five-year structure (0–4, 5–9, …, 80+), which ensures national and international comparability — and a climate risk band (0–4, 5–64, 65+) designed to highlight the most vulnerable groups (IBGE, 2010; WHO, 2014).

Guiding case. We investigate temperature and health outcomes in the elderly (60+) in the São Paulo Metropolitan Region, 2014–2019. Starting from the elderly respiratory deaths already filtered in Tutorial 3 (caso_filtrado.rds), we create age groups (60–69, 70–79, 80+) and calendar variables, and aggregate into a daily series (all deaths) and a monthly series by age group. The resulting series (caso_serie.rds) feeds into the spatial join (Tutorial 5) and, later, the modeling (Tutorial 9).

Data used

Item	Detail
Input	`dados/caso_filtrado.rds` (output of Tutorial 3)
Outcome	Respiratory deaths (ICD-10 J00–J99) in elderly 60+
System	SIM-DO (mortality)
Scope	São Paulo Metropolitan Region, 2014–2019
Output	`dados/caso_serie.rds` (input for Tutorial 5)

If Tutorial 3 has not yet been run, the generation script warns and proceeds with a minimal synthetic example that has the same standardized column structure (data_obito, codigo_municipio_residencia, idade, sexo, causa_basica) — so the stage is illustrated end to end.

Complete pipeline

The blocks below reproduce what the script scripts/tutorial_04_variaveis_agregacao.R runs. Start by loading the package:

library(climasus4r)

Load the output from the previous stage

caso_filtrado <- sus_data_read("output/caso_filtrado.parquet")
# or, in the tutorial workflow, readRDS("vignettes-pt/dados/caso_filtrado.rds")
obitos <- caso_filtrado            # climasus_df at stage "filter_cid"/"stand"

Create derived variables

sus_data_create_variables() creates age groups and calendar variables in a single call. Here we use bands geared to the elderly age range of the guiding case and link the seasonal variables to the "sudeste" (Southeast) climate region.

obitos_vars <- sus_data_create_variables(
  obitos,
  create_age_groups    = TRUE,
  age_breaks           = c(60, 70, 80, Inf),     # young-old / 70-79 / 80+
  age_labels           = c("60-69", "70-79", "80+"),
  create_calendar_vars = TRUE,                   # year, month, quarter, ...
  create_climate_vars  = TRUE,                   # season of the year
  climate_region       = "sudeste",              # climate region of the case
  hemisphere           = "south",                # Brazil
  backend              = "tibble",
  lang                 = "pt"
)

In addition to age_group (defined by age_breaks/age_labels), the function automatically creates the climate risk band (grupo_risco_climatico) and the IBGE five-year band (faixa_etaria_ibge), and — with create_calendar_vars = TRUE — the variables ano, mes, trimestre (year, month, quarter) and the season of the year (estacao_climatica + estacao_seca_chuvosa when climate_region is provided; otherwise, estacao_astronomica).

Aggregate into a daily time series

sus_data_aggregate() converts the individual records into counts. With fun = "count" and the SIM system, the aggregated column is automatically named n_obitos. complete_dates = TRUE fills days with no deaths with zero.

serie_diaria <- sus_data_aggregate(
  obitos_vars,
  time_unit      = "day",      # one observation per day
  fun            = "count",    # count of deaths
  group_by       = NULL,       # aggregates on the detected municipality code
  complete_dates = TRUE,       # fills days with no event with zero
  backend        = "tibble",
  lang           = "pt"
)

Aggregate into a monthly series by age group

The same function, now stratified by age_group. This is the series saved for the next tutorial.

serie_mensal <- sus_data_aggregate(
  obitos_vars,
  time_unit      = "month",
  fun            = "count",
  group_by       = "age_group",   # age group created above
  complete_dates = TRUE,
  backend        = "tibble",
  lang           = "pt"
)

Visualize the time series

sus_data_plot_aggregate_ts() takes a climasus_df at the "aggregate" stage and produces publication-quality plots. plot_type accepts "epidemic", "seasonal", "heatmap" and "trend" (and combinations, via patchwork).

# Daily epidemic curve with loess smoothing
sus_data_plot_aggregate_ts(
  serie_diaria,
  plot_type     = "epidemic",
  smooth_method = "loess",
  palette       = "lancet",
  lang          = "pt"
)

# Boxplot of the monthly seasonal distribution
sus_data_plot_aggregate_ts(
  serie_diaria,
  plot_type = "seasonal",
  lang      = "pt"
)

Visualize the spatial distribution

sus_data_plot_aggregate_map() maps the counts by municipality. To do so, we aggregate while preserving the municipality code as the grouping variable.

serie_muni <- sus_data_aggregate(
  obitos_vars,
  time_unit = "month",
  fun       = "count",
  group_by  = "codigo_municipio_residencia",
  backend   = "tibble",
  lang      = "pt"
)

sus_data_plot_aggregate_map(
  serie_muni,
  map_type  = "bubble",     # does not require geobr/sf
  log_scale = TRUE,
  palette   = "YlOrRd",
  lang      = "pt"
)

At the end of this stage we have serie_mensal (by age group) and serie_muni (by municipality) — the temporal/spatial base that feeds the next tutorials.

Function features

`sus_data_create_variables()`

Creates derived variables from age and event date. Requires the data to be at the "stand" stage (or later).

Argument	Essential?	Description
`df`	yes	Standardized `climasus_df` (stage `"stand"`+)
`create_age_groups`	yes	Creates `age_group`, `grupo_risco_climatico` and `faixa_etaria_ibge` (default `TRUE`)
`age_breaks`	optional	Age group cut points; default `c(0, 5, 15, 60, Inf)`
`age_labels`	optional	Group labels; if `NULL`, generated from `age_breaks`
`create_calendar_vars`	yes	Creates `ano`, `mes`, `trimestre`, etc. (default `TRUE`)
`create_climate_vars`	optional	Creates the season of the year; requires `create_calendar_vars = TRUE`
`climate_region`	optional	Climate region (`"norte"`, `"nordeste"`, `"centro-oeste"`, `"sudeste"`, `"sul"`); defines dry/wet season
`hemisphere`	optional	`"south"` (default, Brazil) or `"north"`
`age_col` / `date_col`	optional	Force the age/date column; if `NULL`, auto-detects
`backend`	optional	`"arrow"` (lazy, default) or `"tibble"`
`lang` / `verbose`	optional	Language (`"pt"`/`"en"`/`"es"`) and messages

Return: the climasus_df with the new columns and stage advanced to "derive". Age is determined by a hierarchy: (1) existing age column, (2) calculation from birth date and event date (gold standard), (3) decoding of the DATASUS age code (fallback).

`sus_data_aggregate()`

Aggregates individual microdata into a time series of counts. Requires stage "stand" (or later).

Argument	Essential?	Description
`df`	yes	`climasus_df` (output of standardization/filtering/derivation)
`time_unit`	yes	`"day"`, `"week"`, `"month"`, `"quarter"`, `"year"`, `"season"`, or multiples such as `"5 days"`, `"14 days"`, `"3 months"` (default `"day"`)
`fun`	optional	`"count"` (default), `"sum"`, `"mean"`, `"median"`, `"min"`, `"max"`, `"sd"`, `"q25"`, `"q75"`, `"q95"`, `"q99"`, or a named list
`group_by`	optional	Stratification columns (e.g., `c("age_group", "sexo")`); `NULL` aggregates on the detected municipality
`value_col`	only if `fun != "count"`	Numeric column to aggregate
`complete_dates`	recommended	`TRUE` fills periods with no event with zero (default `FALSE`)
`date_col`	optional	Date column; if `NULL`, auto-detects by system
`backend`	optional	`"arrow"` (lazy, default) or `"tibble"`
`lang` / `verbose`	optional	Language and messages

Return: a climasus_df at the "aggregate" stage with a date column (start of the period), the group_by columns, and the count column named intelligently by system: n_obitos (SIM), n_internacoes (SIH), n_nascimentos (SINASC), n_casos (SINAN), n_procedimentos (SIA), n_estabelecimentos (CNES).

`sus_data_plot_aggregate_ts()`

Time series plots from a climasus_df at the "aggregate" stage.

Argument	Description
`df`	Aggregated `climasus_df`
`value_col`	Outcome column; if `NULL`, auto-detects (`n_obitos`, `n_casos`, …)
`plot_type`	`"epidemic"`, `"seasonal"`, `"heatmap"`, `"trend"` (accepts several, via `patchwork`)
`group_col` / `facet_col`	Column for color (epidemic/seasonal) and for panels (`facet_wrap`)
`smooth_method` / `smooth_span`	Smoothing of the epidemic curve: `"loess"` (default), `"gam"`, `"lm"`, `"none"`
`log_transform`	Applies `log1p` to the y-axis / fill
`palette`	`"lancet"` (default), `"nejm"`, `"jco"`, `"uchicago"`, `"viridis"`
`city`	Filters municipalities (name or IBGE code) before plotting
`interactive`	`TRUE` returns a `plotly` widget
`lang` / `verbose`	Language and messages

Return: the ggplot2 object (or plotly if interactive = TRUE), invisibly; it also prints the plot. Does not change the stage of df.

`sus_data_plot_aggregate_map()`

Municipal map of the aggregated counts.

Argument	Description
`df`	Aggregated `climasus_df` with a municipality code column
`value_col`	Outcome column; if `NULL`, auto-detects
`map_type`	`"bubble"` (default, without `geobr`/`sf`), `"choropleth"`, `"quantile_choropleth"`
`rate_per_100k`	`TRUE` calculates incidence per 100,000 inhabitants (uses `pop_25`)
`period`	`Date` or `c(start, end)` to filter before aggregating
`city` / `top_n`	Restricts to municipalities; labels the `top_n` largest
`state_borders`	Overlays state borders (via `geobr`)
`palette`	`RColorBrewer` palette (`"YlOrRd"`, `"Blues"`, …) or `"lancet"`/`"nejm"`
`log_scale`	`log1p` on the color scale (default `TRUE`)
`interactive`	`TRUE` returns `plotly`
`use_cache` / `cache_dir`	Cache of municipal metadata
`lang` / `verbose`	Language and messages

Return: a ggplot2 object (or plotly). The choropleth requires the Suggests packages geobr and sf; if absent, the function falls back to the bubble map with a warning. It does not modify df nor advance the stage.

Formulas

The aggregated count of an outcome in period $t$ (with fun = "count") is simply the number of records whose date falls in that period:

$Y_t \;=\; \sum_{i=1}^{N} \mathbf{1}\!\left[\,\lfloor d_i \rfloor_{u} = t\,\right]$

where $d_i$ is the date of event $i$ , $\lfloor \cdot \rfloor_{u}$ rounds down to the start of the period at unit $u$ (time_unit), and $\mathbf{1}[\cdot]$ is the indicator function.

The annual trend figure (plot_type = "trend") reports the year-over-year (YoY) percent change:

$\Delta\%_{a} \;=\; \left( \frac{Y_{a}}{Y_{a-1}} - 1 \right) \times 100$

When rate_per_100k = TRUE in the map, the municipal incidence rate is

$\text{taxa}_{m} \;=\; \frac{\text{casos}_{m}}{\text{pop}_{m}} \times 10^{5}$

Illustrated results

Figure 1. Daily epidemic curve of respiratory deaths in the elderly (60+) in the São Paulo Metropolitan Region, 2014–2019, generated by sus_data_plot_aggregate_ts(plot_type = "epidemic"). The area and the line show the daily count; the smoothed (loess) curve highlights the underlying trend.

Figure 2. Monthly seasonal distribution (sus_data_plot_aggregate_ts(plot_type = "seasonal")). The boxplots by month show the winter seasonality (June–August) of elderly respiratory mortality — the starting point of the climate–health hypothesis of the guiding case.

Figure 3. Monthly series stratified by the age groups created in sus_data_create_variables() (60–69, 70–79, 80+). It is this stratified structure that is saved to caso_serie.rds for the following tutorials.

Figure 4. Spatial distribution of deaths by municipality (sus_data_plot_aggregate_map(map_type = "bubble")). The size and color of the bubbles encode the total number of events per municipality in the São Paulo Metropolitan Region (log1p scale).

Tabela 1. Amostra das variáveis derivadas por sus_data_create_variables() (faixas etárias e calendário).
data_obito	idade	age_years	age_group	grupo_risco_climatico	faixa_etaria_ibge	ano	mes	trimestre	estacao_climatica	estacao_seca_chuvosa
2018-05-21	85	85	80+	Alto Risco (65+)	80+	2018	5	2	Seca	Seca
2015-12-01	86	86	80+	Alto Risco (65+)	80+	2015	12	4	Chuvosa	Chuvosa
2016-07-01	78	78	70-79	Alto Risco (65+)	75-79	2016	7	3	Seca	Seca
2014-04-15	75	75	70-79	Alto Risco (65+)	75-79	2014	4	2	Seca	Seca
2016-08-19	63	63	60-69	Risco Padrao (5-64)	60-64	2016	8	3	Seca	Seca
2016-11-18	82	82	80+	Alto Risco (65+)	80+	2016	11	4	Chuvosa	Chuvosa
2018-06-28	93	93	80+	Alto Risco (65+)	80+	2018	6	2	Seca	Seca
2015-02-02	84	84	80+	Alto Risco (65+)	80+	2015	2	1	Chuvosa	Chuvosa
2014-07-22	85	85	80+	Alto Risco (65+)	80+	2014	7	3	Seca	Seca
2017-10-16	61	61	60-69	Risco Padrao (5-64)	60-64	2017	10	4	Chuvosa	Chuvosa

Tabela 2. Resumo das séries agregadas: número de períodos, total de eventos e intervalo de datas.
Serie	Unidade_temporal	N_periodos	Total_eventos	Inicio	Fim
Diaria (todos os obitos)	dia	2191	6000	2014-01-01	2019-12-31
Mensal (por faixa etaria)	mes	72	6000	2014-01-01	2019-12-01

References

Bhaskaran, K., Gasparrini, A., Hajat, S., Smeeth, L., & Armstrong, B. (2013). Time series regression studies in environmental epidemiology. International Journal of Epidemiology, 42(4), 1187–1195.
Gasparrini, A., Guo, Y., Hashizume, M., Lavigne, E., Zanobetti, A., Schwartz, J., et al. (2015). Mortality risk attributable to high and low ambient temperature: a multicountry observational study. The Lancet, 386(9991), 369–375.
Instituto Brasileiro de Geografia e Estatística (IBGE). (2010). Censo Demográfico 2010. Rio de Janeiro: IBGE.
World Health Organization (WHO). (2014). Quantitative risk assessment of the effects of climate change on selected causes of death, 2030s and 2050s. Geneva: WHO.

Back to tutorial hub
Previous: Tutorial 3 — Filtering (ICD-10 and demographics)
Next: Tutorial 5 — Spatial Join

climasus4r Team

17/10/2026