Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,45 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [Unreleased]

## [v1.8.4] - 2026-10-28

Custom concepts with a standard equivalent move to the standard concept, and one missing answer is added. The candidates came from the vocabulary review of v1.8.3 against `omop_vocab` (`validate_concept_map_vocab.sql` V8 and a name search ignoring case and punctuation); each target's class, standard status and validity were confirmed in `omop_vocab`. Seven rows are edited in place and one is appended (line 2144), so every existing line number is the same as in v1.8.3. The ETL must apply these changes (OBSERVATION and CONDITION_OCCURRENCE concepts).

### Changed — custom → standard

| Line | Row | v1.8.3 | v1.8.4 |
|---|---|---|---|
| 929 | `pxne16` question "The crime rate in my neighborhood makes it unsafe to go on walks during the day…" | 2005200121 (custom) | **40192414** (PPI `ips_16`, Question, standard; same wording), exactMatch 1 |
| 957, 964 | `pxne4`, `pxne5` answer 88 "Does not apply to my neighborhood" | 2005200268 (custom) | **40192520** (PPI `SDOH_9`, Answer, standard; same wording), exactMatch 1 |
| 1869 | `mh_a1c` "Yes" (Condition) "Elevated A1C levels (elevated blood sugars)" | 2005200547 (custom) | **4214376** *Hyperglycemia* (SNOMED 80394007, Condition, standard), closeMatch 0.75 |
| 1849 | `mh_a1c` question `MODIFIER` | 2005200547 | 4214376 (follows line 1869, as for the v1.5 health-history conditions) |

- `TARGET_DOMAIN_ID` of the two answer rows stays *Meas Value*: the answer is written as `value_as_concept_id` of the OBSERVATION. The vocabulary domain of 40192520 is Observation (PPI answers), so `validate_concept_map_vocab.sql` V2 lists these two rows as a deliberate domain override.
- `mh_a1c` "Yes" now writes a standard CONDITION_OCCURRENCE (hyperglycemia) that OHDSI cohorts can find. closeMatch 0.75: the answer reports elevated A1C (blood sugar), which `mhterm_predm` and `mhterm_dm2` complement as separate items.
- The custom concepts 2005200121, 2005200268 and 2005200547 are no longer referenced by the map but are kept in `concept_manual.csv` (IDs are never reused).
- Reviewed and not changed: the other `pxne*` items (PPI `ips_1`–`ips_7` are non-standard and map to the PhenX LOINC questions the map already uses; `pxne8`–`pxne15` and `pxne17` have no PPI question); the insurance block `pxhic1`–`pxhic7` (All of Us models insurance as one non-standard question, `basics_11a`, with the insurance types as answers, so `cope_a_193` *Medicare* is an Answer, not a question concept); `viaomthd` *EVA* (the same-name standard concept 42066053 is a Geography place); `pxhic*` *Covered* (SNOMED 255334000 is a general qualifier, not insurance coverage); `mhoccur_circ`, `mhoccur_cvdot` (compound answers; only very broad standard parents exist).

### Added

- `eos_dsvst` answer 3 "Devices Returned" (line 2144) → **2005200589** *Device returned* (existing AI-READI custom concept, as used for `dcvgmenyn` and `dvenvenyn`), exactMatch 1, in the format of line 267. Stages 0–2 were mapped and stage 3 was not; the current data has 3 records with stage 3.

### Changed — metadata

- `SRC_CODE` filled for the two imported ratio questions that had it blank: line 554 `import_buncreatinineratio` → 31045-8, line 584 `import_ag_ratio` → 1759-0 (the LOINC code, as for the other imported labs). The ETL and the unit lookup match on `FIELD_ID`, so this changes nothing in the data.

### Added — release check script

- `redCapDataDictionaryWithExtension/scripts/validate_concept_map_against_data.sql` (read-only, Databricks): checks the map against the OMOP survey data and the vocabulary, alongside `validate_concept_map_vocab.sql`. Row count; targets non-standard, invalid or missing in `omop_vocab`; answer and question fan-out (one value row per REDCap answer); the v1.8.x changes in place; data fields with no map row; answer codes in the data with no map row; unit coverage, no fan-out and Unit-domain units; question rows pointing at Answer-class concepts and the reverse. Each check states its expected result.

### Changed — master REDCap data dictionary

- Eight master rows take the new values (`Origin` = Changed, `Date_of_Variable_Modification` 10/28/2026), including the existing `eos_dsvst` 3 row (now `Map_to_OMOP` = Yes). 3,150 rows; every concept map row has a master row with the same values.

### Regenerated

- map2omop (2,113 rows) and open-source public (1,998) versions, SSSOM (2,068 rows; 75 incomplete) and LinkML (2,143 records). `unit_concept_lookup.csv` and `unit_concept_lookup_extended.csv` unchanged.
- Counts: standard 1,519 (403 concepts); custom 486 (339); mapped 2,005 (69.0%, 742); designated but unmapped 138; not designated 763. SNOMED 172/75; PPI 8/7. Standard predicates: exact 1,258; close 259 (170 at 1, 89 at 0.75); related 2. Audit A7 (custom Condition targets) 3 → 2. `count_concept_map.sql` expected values and `validate_concept_map_vocab.sql` V0 (2,143) updated.

## [v1.8.3] - 2026-10-27

Two lab unit rows are corrected to the units the data is reported in. A check of the delivered OMOP survey data against the v1.8.2 unit lookup found that the unit text in the data and the value ranges disagree with the map for C-peptide and urine albumin; the field key (which the join applies first) would have labeled C-peptide 1,000× and urine albumin 10× off. Two rows are edited in place, so every line number is the same as in v1.8.2. The ETL must apply this change (`MEASUREMENT.unit_concept_id`).
Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ repository-code: "https://github.com/AI-READI/DataElementMaps"
repository-artifact: "https://github.com/AI-READI/DataElementMaps/blob/main/manuscript-tvst2026/ai_readi_redcap_value_set_mapping_common.csv"
url: "https://aireadi.org"
license: MIT
version: "v1.8.3"
date-released: "2026-10-27"
version: "v1.8.4"
date-released: "2026-10-28"
identifiers:
- type: doi
value: "10.5281/zenodo.23375723"
Expand Down
6 changes: 3 additions & 3 deletions codemeta.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
"codeRepository": "https://github.com/AI-READI/DataElementMaps",
"downloadUrl": "https://github.com/AI-READI/DataElementMaps",
"issueTracker": "https://github.com/AI-READI/DataElementMaps/issues",
"version": "v1.8.3",
"datePublished": "2026-10-27",
"version": "v1.8.4",
"datePublished": "2026-10-28",
"releaseNotes": "Concept map v1.6: maps the A/G ratio, BUN/creatinine ratio (stored as creatinine/BUN), BMI and waist circumference to standard LOINC concepts, adds the unit ratio for the three ratio fields, and excludes the leftover answers of two removed questions. ETL re-run required. See CHANGELOG.md.",
"dateCreated": "2023-06-28",
"dateModified": "2026-10-27",
"dateModified": "2026-10-28",
"applicationCategory": "Scientific",
"developmentStatus": "active",
"isPartOf": "https://aireadi.org/",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11632,6 +11632,7 @@ mappings:
field_id: import_buncreatinineratio
field_id_num: 2960
question_or_answer: Question
src_code: 31045-8
src_code_id: BUN/Creatinine ratio
src_cd_description: BUN/Creatinine ratio
target_concept_id: 3033837
Expand Down Expand Up @@ -12153,6 +12154,7 @@ mappings:
field_id: import_ag_ratio
field_id_num: 2990
question_or_answer: Question
src_code: 1759-0
src_code_id: A/G Ratio (calculated field)
src_cd_description: Albumin/Globulin ratio
target_concept_id: 3020509
Expand Down Expand Up @@ -19421,13 +19423,13 @@ mappings:
src_code: pxne16
src_cd_description: The crime rate in my neighborhood makes it unsafe to go on walks during the day. Would you say that
you...
target_concept_id: 2005200121
local_concept_id: 2005200121
target_concept_id: 40192414
target_concept_name: The crime rate in my neighborhood makes it unsafe to go on walks during the day. Would you say that
you...
target_domain_id: Observation
target_vocabulary_id: AIREADI
target_concept_class_id: Survey
target_vocabulary_id: PPI
target_concept_class_id: Question
target_concept_code: ips_16
target_standard_concept: S
predicate_id: skos:exactMatch
confidence: 1.0
Expand Down Expand Up @@ -20037,12 +20039,12 @@ mappings:
src_code: '88'
src_code_id: pxne4_88
src_cd_description: Does not apply to my neighborhood
target_concept_id: 2005200268
local_concept_id: 2005200268
target_concept_id: 40192520
target_concept_name: Does not apply to my neighborhood
target_domain_id: Meas Value
target_vocabulary_id: AIREADI
target_vocabulary_id: PPI
target_concept_class_id: Answer
target_concept_code: SDOH_9
target_standard_concept: S
predicate_id: skos:exactMatch
confidence: 1.0
Expand Down Expand Up @@ -20191,12 +20193,12 @@ mappings:
src_code: '88'
src_code_id: pxne5_88
src_cd_description: Does not apply to my neighborhood
target_concept_id: 2005200268
local_concept_id: 2005200268
target_concept_id: 40192520
target_concept_name: Does not apply to my neighborhood
target_domain_id: Meas Value
target_vocabulary_id: AIREADI
target_vocabulary_id: PPI
target_concept_class_id: Answer
target_concept_code: SDOH_9
target_standard_concept: S
predicate_id: skos:exactMatch
confidence: 1.0
Expand Down Expand Up @@ -39782,7 +39784,7 @@ mappings:
target_standard_concept: S
predicate_id: skos:exactMatch
confidence: 1.0
modifier: '2005200547'
modifier: '4214376'
qualifier_concept_id: 2005200547
qualifier_source_value: Elevated A1C levels (elevated blood sugars)
- mapping_record_id: AIREADI_MAPPING_1849
Expand Down Expand Up @@ -40207,15 +40209,15 @@ mappings:
src_code: '1'
src_code_id: mh_a1c_1
src_cd_description: 'Yes'
target_concept_id: 2005200547
local_concept_id: 2005200547
target_concept_name: Elevated A1C levels (elevated blood sugars)
target_concept_id: 4214376
target_concept_name: Hyperglycemia
target_domain_id: Condition
target_vocabulary_id: AIREADI
target_vocabulary_id: SNOMED
target_concept_class_id: Disorder
target_concept_code: '80394007'
target_standard_concept: S
predicate_id: skos:exactMatch
confidence: 1.0
predicate_id: skos:closeMatch
confidence: 0.75
- mapping_record_id: AIREADI_MAPPING_1869
map_to_omop: 'Yes'
transferred_to_fairhub_redcap_dataset: 'Yes'
Expand Down Expand Up @@ -46196,3 +46198,24 @@ mappings:
target_standard_concept: S
predicate_id: skos:exactMatch
confidence: 1.0
- mapping_record_id: AIREADI_MAPPING_2143
map_to_omop: 'Yes'
transferred_to_fairhub_redcap_dataset: 'Yes'
protected_or_open_source: open-source
extension_needed: 'Yes'
form_name: disposition
field_type: radio
field_id: eos_dsvst
question_or_answer: Answer
src_code: '3'
src_code_id: eos_dsvst_3
src_cd_description: Devices Returned
target_concept_id: 2005200589
local_concept_id: 2005200589
target_concept_name: Device returned
target_domain_id: Observation
target_vocabulary_id: AIREADI
target_concept_class_id: Qualifier Value
target_standard_concept: S
predicate_id: skos:exactMatch
confidence: 1.0
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# AI-READI REDCap Value Set Mapping LinkML Conversion Summary

- Source file: `_ai_readi_redcap_value_set_mapping_common.csv`
- Rows converted: 2142
- Rows converted: 2143
- Source columns: 26

## Generated files
Expand All @@ -23,7 +23,7 @@

### TARGET_DOMAIN_ID
- Meas Value: 1240
- Observation: 442
- Observation: 443
- Unit: 153
- Measurement: 116
- [blank]: 75
Expand All @@ -35,24 +35,24 @@

### TARGET_VOCABULARY_ID
- LOINC: 1198
- AIREADI: 424
- SNOMED: 171
- AIREADI: 421
- SNOMED: 172
- UCUM: 153
- [blank]: 75
- AIREADI-Vision: 69
- AIREADI-MOCA: 30
- CPT4: 12
- PPI: 5
- PPI: 8
- Ethnicity: 4
- OMOP Extension: 1

### PREDICATE_ID
- skos:exactMatch: 1779
- skos:closeMatch: 286
- skos:closeMatch: 287
- [blank]: 75
- skos:relatedMatch: 2

### Extension_Needed
- No: 1618
- Yes: 523
- Yes: 524
- [blank]: 1
Loading
Loading