Skip to contents

Objective

By the end of this tutorial you will be able to import microdata from the DATASUS Health Information Systems using a single function — sus_data_import() — and to control its main features: the choice of system (SIM, SIH, SINAN, SINASC, CNES, SIA) and its subsystems, the spatial scope (by state, by a predefined region, or by municipality), the difference between annual and monthly systems, the choice of processing backend, and — essential for reproducible, large-volume work — smart caching and parallel processing.

Import is the first stage of the climasus4r pipeline. Everything that comes afterward (standardization, filtering, variable creation, aggregation, spatial join, socioeconomic linkage, climate integration, and modeling) starts from the raw base built here.

Scientific context

Climate-health research in Brazil relies on the Health Information Systems (SIS) maintained by the Ministry of Health and made available through DATASUS. Each system records one type of event, with national coverage and municipal granularity:

  • SIM — Mortality Information System (death certificates);
  • SIH — Hospital Information System (hospitalizations/AIH);
  • SINAN — Notifiable Diseases Information System (notifiable diseases, such as dengue, zika, and chikungunya);
  • SINASC — Live Birth Information System;
  • CNES — National Registry of Health Establishments;
  • SIA — Outpatient Information System.

Programmatic access to these .dbc files is mediated by the microdatasus package (Saldanha, Bastos & Barcellos, 2019). climasus4r wraps this access in sus_data_import(), adding caching (so the same data is not re-downloaded on every run), parallelism (to speed up downloads across many states/years), and multilingual messages. Importing reproducibly — recording the system, year, and spatial scope — is what makes a study auditable and replicable.

Guiding case. Throughout this tutorial series we investigate temperature and health outcomes in older adults (60+) in the São Paulo Metropolitan Region, 2014–2019. In this first tutorial we assemble the multi-system raw base: deaths (SIM-DO), hospitalizations (SIH-RD), and, as a secondary case on climate × arboviruses, dengue (SINAN-DENGUE). The objects imported here are saved and will serve as input for Tutorial 2 (Encoding and Standardization).

Data used

System system = Periodicity Role in the case
SIM-DO (deaths) "SIM-DO" Annual Primary outcome (mortality)
SIH-RD (hospitalizations) "SIH-RD" Monthly (requires month) Hospital morbidity
SINAN-DENGUE "SINAN-DENGUE" Annual Secondary case (climate × dengue)

Spatial scope: the state of São Paulo (uf = "SP"), 2014–2019. All data are public and real, obtained from DATASUS via microdatasus.

Complete pipeline

The blocks below reproduce what the scripts/tutorial_01_importacao.R script runs. Start by loading the package:

library(climasus4r)

Importing an annual system (SIM-DO)

SIM is annual: the month argument is ignored. We request all six years at once and leave caching and parallelism turned on.

obitos <- sus_data_import(
  uf        = "SP",
  year      = 2014:2019,
  system    = "SIM-DO",
  backend   = "arrow",   # "arrow" (lazy, padrão) ou "tibble" (em memória)
  use_cache = TRUE,        # reaproveita downloads anteriores
  parallel  = TRUE,        # baixa anos em paralelo
  workers   = 4,
  lang      = "pt"
)

View the data

dplyr::glimpse(obitos)

Importing a monthly system (SIH-RD)

SIH is monthly: the month argument is mandatory. Here we request only the winter months (June to August), when respiratory mortality in older adults tends to be higher.

internacoes <- sus_data_import(
  uf        = "SP",
  year      = 2014:2019,
  month     = 6:8,         # obrigatório para SIH, CNES e SIA
  system    = "SIH-RD",
  use_cache = TRUE,
  parallel  = TRUE,
  workers   = 4,
  lang      = "pt"
)

View the data

dplyr::glimpse(internacoes)

Importing notifications (SINAN-DENGUE)

dengue <- sus_data_import(
  uf        = "SP",
  year      = 2014:2019,
  system    = "SINAN-DENGUE",
  use_cache = TRUE,
  lang      = "pt"
)

View the data

dplyr::glimpse(dengue)

Spatial scope: by region or by municipality

Instead of listing states, you can use a predefined group in region (macro-regions, biomes, or river basins) — useful for regional studies:

# Toda a macrorregião Sudeste (ES, MG, RJ, SP) de uma vez:
obitos_sudeste <- sus_data_import(
  region = "sudeste",
  year   = 2019,
  system = "SIM-DO",
  lang   = "pt"
)

View the data

dplyr::glimpse(obitos_sudeste)

Or restrict to a single municipality by IBGE code (or by name in city). The filter is resolved to the IBGE code before the download:

# Apenas o município de São Paulo (código IBGE 3550308):
obitos_muni_sp <- sus_data_import(
  uf                = "SP",
  year              = 2014:2019,
  system            = "SIM-DO",
  municipality_code = "3550308",
  lang              = "pt"
)

View the data

dplyr::glimpse(obitos_muni_sp)

(Optional) Re-reading already exported data

After exporting the base (with sus_data_export()), you can read it back in future sessions without downloading it again, using sus_data_read() — which automatically detects the format:

obitos <- sus_data_read("output/sim_sp_2014_2019.parquet")

At the end of this stage we have obitos, internacoes, and dengue — the multi-system raw base that will feed the next tutorials.

Function features

sus_data_import()

The single entry point for the import stage. It wraps microdatasus::fetch_datasus (Saldanha, Bastos & Barcellos, 2019) with caching and parallelism.

Argument Essential? Description
uf / region yes (one of them) State(s) ("SP", c("SP","RJ"), "all") or a predefined group ("sudeste", "amazonia_legal", …)
year yes 4-digit year(s) (e.g., 2014:2019)
system yes System/subsystem (see systems table below)
month monthly only Month(s) 1–12; mandatory for SIH, CNES, and SIA; ignored for SIM, SINAN, and SINASC
backend optional "arrow" (lazy, default, very fast) or "tibble" (in memory)
city / municipality_code optional Restricts to municipalities (name or 6/7-digit IBGE code)
use_cache recommended TRUE reuses downloads; cache_dir sets the folder
force_redownload optional TRUE ignores the cache and re-downloads everything
parallel / workers optional Parallelizes state/year combinations; respects the user’s future::plan()
lang / verbose optional Language ("pt", "en", "es") and message level

Return: a tibble/data.frame (or an Arrow object, if backend = "arrow") with the system’s original variables, plus source_system and download_timestamp, dates as Date objects, and UTF-8 text.

Smart caching. Calls are identified by a SHA-256 key of the parameters (state, year, month, system). Results are stored in cache_dir (~/.climasus4r_cache/data by default) as compressed RDS and are reused automatically. The cache expires after ~30 days for dynamic systems (CNES, SIA, SIH) and ~365 days for static ones (SIM, SINAN, SINASC).

sus_data_read()

Re-reads already exported files without going back to the source. Auto-detects the format.

Argument Description
path Path to a file, a vector of files, or a folder
format NULL (auto-detects) or "parquet", "csv", "dbc", "rds", "shapefile", …
parallel / workers Parallel reading of many files
read_metadata Recovers companion metadata as attributes

Return: the table read (or an sf object, for spatial data); batch reads are combined with dplyr::bind_rows().

Supported systems and subsystems

Table 1. DATASUS systems accessible via sus_data_import().
Sistema Descricao Exemplos_system Periodicidade
SIM Mortalidade (declaracoes de obito) SIM-DO, SIM-DOFET, SIM-DOINF, SIM-DOMAT, SIM-DOEXT Anual (month ignorado)
SIH Internacoes hospitalares (AIH) SIH-RD, SIH-RJ, SIH-SP, SIH-ER Mensal (exige month)
SINASC Nascidos vivos SINASC Anual (month ignorado)
SINAN Agravos de notificacao (doencas notificaveis) SINAN-DENGUE, SINAN-ZIKA, SINAN-CHIKUNGUNYA, SINAN-MALARIA, … Anual (month ignorado)
CNES Estabelecimentos de saude CNES-LT, CNES-ST, CNES-EQ, CNES-PF, … Mensal (exige month)
SIA Atendimentos ambulatoriais SIA-PA, SIA-AB, SIA-AD, … Mensal (exige month)

region shortcuts (download several states at once):

Table 2. Examples of predefined groups accepted in region.
Grupo Tipo UFs
sudeste Macrorregiao IBGE ES, MG, RJ, SP
sul Macrorregiao IBGE PR, RS, SC
nordeste Macrorregiao IBGE AL, BA, CE, MA, PB, PE, PI, RN, SE
norte Macrorregiao IBGE AC, AP, AM, PA, RO, RR, TO
centro_oeste Macrorregiao IBGE DF, GO, MT, MS
amazonia_legal Bioma/fronteira ecologica AC, AP, AM, PA, RO, RR, MT, MA, TO
mata_atlantica Bioma AL, BA, CE, ES, GO, MA, MG, MS, PB, PE, PI, PR, RJ, RN, RS, SC, SE, SP
cerrado Bioma BA, DF, GO, MA, MG, MS, MT, PA, PI, PR, RO, SP, TO
bacia_sao_francisco Bacia hidrografica AL, BA, DF, GO, MG, PE, SE
semi_arido Recorte climatico AL, BA, CE, MA, PB, PE, PI, RN, SE, MG

Typical structure of SIM-DO right after import (underlying cause in CAUSABAS, age coded in IDADE, municipality of residence in CODMUNRES):

Table 3. Sample of key columns from the imported SIM-DO.
DTOBITO DTNASC SEXO IDADE RACACOR CAUSABAS CODMUNRES CODMUNOCOR
08072017 17071962 2 454 1 C383 352230 352230
13062017 02091986 2 430 4 C099 353870 353870
05072017 14051971 2 446 4 C710 351170 353870
23062017 24081940 1 476 4 R960 351030 351030
07062017 21041928 1 489 1 R960 351030 351030
13052017 04091945 1 471 1 I119 351030 351030
28052017 02031949 2 468 4 I110 351030 351030
24062017 03101950 2 466 1 I120 351030 351030
13032017 28051917 2 499 1 I500 351040 351040
13032017 01031953 2 464 1 C538 351040 351040

Performance: caching and parallelism

Two features of sus_data_import() make importing reproducible and fast. The relative gain (speedup) is the ratio between the time without the feature and the time with it:

speedup=tsem recursotcom recurso \text{speedup} = \frac{t_{\text{sem recurso}}}{t_{\text{com recurso}}}

Speedup values above 1 indicate acceleration. The figures below show real measurements taken by the generation script.

Figure 1. Cache benchmark. The first call downloads the data from the network; the second reads from the local cache, typically orders of magnitude faster — without any change to the analysis code.

Figure 2. Parallelism benchmark. Importing the six years with workers = 4 is faster than doing so in series (workers = 1). The gain depends on the number of available cores; sus_data_import() warns if workers exceeds the machine’s cores.

Illustrated results

Figure 3. Number of records imported per system for São Paulo, 2014–2019 (SIH-RD was restricted to winter, June–August, to keep the example light). The very different order of magnitude across systems helps in sizing memory and download time.

Figure 4. Monthly series of deaths (all causes) from SIM-DO in São Paulo, 2014–2019. Winter seasonality is already visible in the raw base and will be the starting point for the subsets (older adults, respiratory causes) in the next tutorials.

References

  • Saldanha, R. de F., Bastos, R. R., & Barcellos, C. (2019). Microdatasus: pacote para download e pré-processamento de microdados do Departamento de Informática do SUS (DATASUS). Cadernos de Saúde Pública, 35(9), e00032419.
  • Ministério da Saúde (Brasil). DATASUS — Departamento de Informática do SUS. Available at: http://datasus.saude.gov.br.