Skip to contents

Filter Available Datasets in IRW

Source: R/filter.R
irw_filter.Rd

Returns the names of datasets in the Item Response Warehouse (IRW) that match user-specified metadata, tag values, variable presence, and license criteria.

Usage

irw_filter(
  n_responses = NULL,
  n_categories = NULL,
  n_participants = NULL,
  n_items = NULL,
  responses_per_participant = NULL,
  responses_per_item = NULL,
  difficulty_spread = NULL,
  mean_item_rest_r = NULL,
  mean_abs_item_rest_r = NULL,
  prop_items_extreme = NULL,
  density = c(0.5, Inf),
  var = NULL,
  age_range = NULL,
  child_age__for_child_focused_studies_ = NULL,
  construct_type = NULL,
  construct_name = NULL,
  sample = NULL,
  measurement_tool = NULL,
  item_format = NULL,
  primary_language_s_ = NULL,
  longitudinal = NULL,
  n_actors = NULL,
  n_respondents = NULL,
  n_attributes = NULL,
  outcome = NULL,
  country = NULL,
  domain = NULL,
  choice_object = NULL,
  levels_labelled = NULL,
  source = "core",
  comp = FALSE,
  sim = FALSE,
  nom = FALSE,
  license = NULL,
  collection = NULL
)

Arguments

n_responses

Numeric vector of length 1 or 2. Filters datasets by total number of responses.

  • Length 1: exact value (e.g., n_responses = 1000)

  • Length 2: range (e.g., n_responses = c(1000, Inf))

n_categories

Numeric vector of length 1 or 2. Filters by number of unique response categories.

n_participants

Numeric vector of length 1 or 2. Filters by number of unique participants (id).

n_items

Numeric vector of length 1 or 2. Filters by number of unique items.

responses_per_participant

Numeric vector of length 1 or 2. Filters by average responses per participant.

responses_per_item

Numeric vector of length 1 or 2. Filters by average responses per item.

difficulty_spread

Numeric vector of length 1 or 2. Filters by the share of the table's observed response range that its item means cover – the range of item p-values on a dichotomous table, and normalised by the response range so a 1-5 Likert table is comparable. Near 0 means every item sits at the same difficulty; near 1 means they span the scale. core only.

mean_item_rest_r

Numeric vector of length 1 or 2. Filters by the mean item-rest correlation, SIGNED. IRW does not harmonise response direction, so a scale with reverse-keyed items averages toward zero here; use mean_abs_item_rest_r to ask whether the items hang together at all. core only.

mean_abs_item_rest_r

Numeric vector of length 1 or 2. Filters by the mean ABSOLUTE item-rest correlation, which is direction-invariant. A wide gap between this and mean_item_rest_r indicates reverse-keyed items. core only.

prop_items_extreme

Numeric vector of length 1 or 2. Filters by the share of items whose mean sits within 5% of either end of the response range, i.e. floor or ceiling items. prop_items_extreme = c(0, 0) keeps tables with none. core only.

density

Numeric vector of length 1 or 2, or NULL. Filters on density, which is responses per person-item cell – n_responses / (n_participants * n_items) – and is therefore not bounded by 1. A value above 1 means the table measures a person-item pair more than once: 768 of 6,720 core tables are above 1 (measured 2026-10-25), 700 of them carrying an occasion column (wave, rater, trial_*, session, ...), and the largest is 3,624.6 for a go/no-go task. It is not the proportion of cells that are filled, which is a different quantity IRW does not publish (ben-domingue/irw#3222).

  • Default is c(0.5, Inf): a lower bound only, to exclude sparse matrices. The default used to cap at 1, which silently dropped every repeated-measures table in the corpus.

  • Use NULL to disable this filter.

  • The default is skipped for a source whose metadata has no density column ("nom"). Passing density, var, longitudinal or a numeric filter whose column that source lacks is an error.

var

Character vector. Filters datasets by presence of variables.

  • Use exact names (e.g., "rt", "wave"), or

  • Use a prefix (e.g., "cov_") to match any variable starting with that prefix.

age_range

Character vector. Filters by participant age group (e.g., "Adult (18+)"). See irw_tag_options("age_range") for values.

child_age__for_child_focused_studies_

Character vector. Filters by child age subgroup. See irw_tag_options("child_age__for_child_focused_studies_") for values.

construct_type

Character vector. Filters by high-level construct category (e.g., "Affective/mental health"). See irw_tag_options("construct_type").

construct_name

Character vector. Filters by specific construct (e.g., "Big Five"). See irw_tag_options("construct_name").

sample

Character vector. Filters by sample type or recruitment method (e.g., "Educational", "Clinical"). See irw_tag_options("sample").

measurement_tool

Character vector. Filters by instrument type (e.g., "Survey/questionnaire"). See irw_tag_options("measurement_tool").

item_format

Character vector. Filters by item format (e.g., "Likert Scale/selected response"). See irw_tag_options("item_format").

primary_language_s_

Character vector. Filters by language used (e.g., "eng"). See irw_tag_options("primary_language_s_").

longitudinal

Logical or NULL. Filters longitudinal datasets with variables wave or date.

  • TRUE: include only datasets flagged as longitudinal

  • FALSE: exclude datasets flagged as longitudinal

  • NULL (default): no filter

n_actors

Numeric vector of length 1 or 2. Competition-only filter for the number of actors. Only used when source = "comp".

n_respondents

Numeric vector of length 1 or 2. Conjoint-only: number of respondents. Only used when source = "conj".

n_attributes

Numeric vector of length 1 or 2. Conjoint-only: number of attributes varied on each profile.

outcome

Character, "choice" and/or "rating". Conjoint-only: keep tables that have every outcome type named (a table with both matches either).

country

Character vector of ISO 3166 alpha-2 codes (e.g. "US"). Conjoint-only: keep tables fielded in any of them; a table pooling several countries matches each.

domain

Character vector. Conjoint-only: what the study is about, its setting rather than the attribute it manipulates (a candidate conjoint that varies gender is "candidate_choice"). A table carries one or more domains, primary first; it matches when any value named is among them. The vocabulary: candidate_choice, immigration_asylum, climate_energy_environment, international_economy, democracy_institutions, foreign_policy_security, health_covid, tax_welfare_economy, consumer_products, social_identity_interpersonal, labour_hiring, crime_justice_policing, technology_ai_privacy, political_communication, parties_platforms, public_administration, corruption_accountability, housing_neighbourhoods, other. A value no table carries is an error that lists the values in use. table(irw_metadata(source = "conj")$domain) gives the counts.

choice_object

Character vector. Conjoint-only: what one profile represents, one value per table: person (candidate, applicant, patient, neighbour), policy, scenario (a situation or case), product_service, place, organisation or message (a text or media artefact judged as such). Any value named matches.

levels_labelled

Character vector. Conjoint-only: whether a table's attribute levels are the text respondents saw. "full": every attribute holds displayed text. "partial": some attributes hold only the deposit's codes, in columns named attr_<name>_code, because no source gives their text. "none": every attribute is coded. Codes say which profiles share a level, which is enough for position, task-order and heterogeneity analyses and for effects within a table, but not what a level was, so they cannot be compared across tables. Any value named matches; the default NULL keeps every table. Pass "full" for tables whose levels are all text.

source

Character. The data family: "core" (default), "nom", "sim", "comp", "conj", or "rank". For "rank" the usable filter is license; the other criteria name columns its metadata does not have and are refused by name. Filter on the design facts (method, seats, positions offered, ...) with irw_metadata(source = "rank") directly. For "conj" the filters are n_respondents, n_attributes, outcome, country, domain, choice_object, levels_labelled and license; see irw_metadata() with source = "conj" for the other design facts. The two tag columns are defined in the irw repo's data/conjoint/README.md ("Topic tags"). Tag filters require "core" or "nom".

comp

Deprecated. Use source = "comp" instead.

sim

Deprecated. Use source = "sim" instead.

nom

Deprecated. Use source = "nom" instead.

license

Character vector. Filters datasets by license (e.g., "CC BY 4.0"). See irw_license_options() for available values.

collection

Character vector. Filters datasets by collection membership (e.g., "rct", "big_five", "depression"). Matching is OR within the argument: collection = c("rct", "response_time") returns the union. For the intersection, use intersect(irw_collection("rct"), irw_collection("response_time")). See irw_collections() for available collections and their coverage. Core source only.

Value

A sorted character vector of dataset names that match all specified filters, or character(0) if no match is found.

Details

Filtering is based on:

  • Numeric metadata: number of responses, participants, items, etc.

  • Tag metadata: e.g., construct type, sample, measurement tool

  • Variable presence: e.g., rt, wave, cov_

  • License type: e.g., "CC BY 4.0", "CC0 1.0"

Psychometric Filtering

Four columns describe how a table behaves rather than how big it is: difficulty_spread, mean_item_rest_r, mean_abs_item_rest_r and prop_items_extreme. They are classical and model-free – no IRT model is fitted anywhere – so they are defined for every core table, polytomous and sparse included, and they take no position on what the items measure. They are computed per item and published per table; the per-item rows behind them are the item_classical table of irw_meta.

Two cautions. Response direction is not harmonised across IRW, so a signed mean item-rest correlation near zero can mean a reverse-keyed scale rather than an incoherent one – compare it with the absolute version. And where a respondent sees a handful of items out of thousands, rest scores are nearly disjoint between respondents, so the correlation is a weaker statement about a sparse bank than about a 20-item questionnaire.

Metadata and Tag-Based Filtering

To explore available metadata:

  • summary(irw_metadata()) — numeric summaries (e.g., n_responses, density)

  • irw_tags() — full tag metadata table (1 row per dataset)

  • irw_tag_options("column_name") — valid values (with counts) for any tag column

  • irw_license_options() — available license values with frequencies

  • irw_collections() — the 22 collections, with kind, definition and coverage

  • irw_collection("name") — the table names in one collection

Tag-based metadata (e.g., construct_type, sample, item_format) can be passed directly as named arguments. See the parameter list below for supported tag columns. Tag filters are available for source = "core" and source = "nom"; "comp" and "sim" have no tags by design and reject them.

Examples

if (FALSE) { # irw_has_credentials() || nzchar(Sys.getenv("IRW_RUN_EXAMPLES"))
# \donttest{
# Numeric filters
irw_filter(n_responses = c(1000, Inf), n_items = c(10, 50))
irw_filter(n_participants = c(500, Inf), density = c(0.3, 0.9))

# Variable presence
irw_filter(var = "rt")
irw_filter(var = c("wave", "cov_"), density = NULL)

# Tag metadata filtering
irw_filter(construct_type = "Affective/mental health", sample = "Educational")
irw_tag_options("construct_type")  # view tag values

# License filtering
irw_license_options()
irw_filter(license = "CC BY 4.0")

# Collection filtering
irw_collections()                                    # what exists
irw_filter(collection = "rct")
irw_filter(collection = "depression", n_participants = c(500, Inf))
irw_filter(collection = c("rct", "response_time"))   # union, not intersection

# Filter by response category complexity
irw_filter(n_categories = 2, density = NULL)           # binary
irw_filter(n_categories = c(3, 5), density = NULL)     # small multi-category
irw_filter(n_categories = c(10, Inf), density = NULL)  # large category sets
# Filter by psychometric property, not just shape (irw_meta tier-0 columns)
irw_filter(mean_abs_item_rest_r = c(0.4, 1))            # items that hang together
irw_filter(prop_items_extreme = c(0, 0),                # no floor or ceiling items
           difficulty_spread = c(0.3, 1))               # spanning the scale
irw_filter(mean_item_rest_r = c(-1, 0.1),               # signed near zero but
           mean_abs_item_rest_r = c(0.3, 1))            # absolute high: reverse-keyed
irw_filter(source = "sim", n_items = c(10, Inf))
irw_filter(source = "comp", n_actors = c(2, 10))
irw_filter(source = "conj", outcome = "rating", country = c("US", "GB"))
irw_filter(source = "conj", domain = "immigration_asylum")          # the immigration conjoints
irw_filter(source = "conj", domain = c("candidate_choice", "parties_platforms"),
           choice_object = "person")
irw_filter(source = "conj", levels_labelled = "full")               # level text throughout
# }
}