System Overview

High-level architecture and data flow of IMPACT-VIS
Authors
Affiliation

Nicholas Boehler

University of Toronto Mississauga

Hai-Ying Mary Cheng

University of Toronto Mississauga

Published

December 18, 2025

1 Introduction

IMPACT-VIS is an R Shiny application implemented following Rhinoverse guidelines for modularity and reproducibility. It serves as the visualization and curation component of the broader IMPACT pipeline, supporting interactive interpretation of preprocessed genomic variants.

ImportantSystem Scope

IMPACT-VIS Responsibilities (This Application):

  • Load and validate preprocessed variant files (GDS, AnnotSV TSV, CNV TXT)
  • Apply user-requested filtering and sorting on loaded variants
  • Render the interactive IMPACT Plot with unified genome visualization
  • Enable variant-level curation and annotation

Upstream IMPACT Modules (Separate Pipelines):

  • IMPACT-SNV: VCF normalization → GDS conversion → FAVOR annotation → IMPACT scoring
  • IMPACT-SV: SV VCF → AnnotSV annotation → Phenotype-aware filtering → ACMG classification
  • IMPACT-CNV: CNV VCF → SCIP processing → Read-depth validation → Phenotype prioritization

Key Point: IMPACT-VIS assumes all input files are already processed and scored by upstream modules. It does NOT re-process raw VCF files, perform annotation, or apply scoring. See SNV/Indel Processing, SV Processing, and CNV Processing for upstream pipeline details.

1.1 Motivation

The IMPACT workflow produces comprehensive preprocessed variant files (SNVs, SVs, CNVs) with phenotype-specific prioritization. IMPACT-VIS was developed to enable intuitive exploration of these outputs through interactive visualization and annotation:

  1. Unified Interface: Enable simultaneous exploration of SNVs/Indels, SVs, and CNVs alongside phenotype-driven prioritization
  2. Reproducible Curation: Persist user classifications and annotations for reproducible variant review workflows
  3. Phenotype-Aware Prioritization: Visualize IMPACT tier-based scoring and gene–disease associations from Open Targets

1.2 Design Philosophy

IMPACT-VIS is designed with the following principles:

  • Hierarchical Prioritization: Surface high-priority variants first while maintaining access to lower-priority candidates
  • Multi-Step Review: Implement structured variant curation workflow (quick filtering → detailed inspection → classification)
  • Persistent State: Save annotations and curation decisions for continuity across sessions
  • Integrated Evidence: Display variant annotations, external database links, and phenotype context simultaneously
  • Efficiency: Enable responsive filtering and visualization even for large datasets (1000+ variants per sample)

2 System Architecture Diagram

graph TB
    A["IMPACT Workflows<br/>(SNV/SV/CNV)"] 
    B["Data Layer<br/>(GDS, AnnotSV TSV,<br/>CNV TXT)"]
    C["IMPACT-VIS<br/>(Rhino Shiny)"]
    D["Logic Layer<br/>(Processing,<br/>Filtering,<br/>Ranking)"]
    E["View Layer<br/>(UI Modules)"]
    F["Visualization Layer<br/>(ggplot2 + plotly)"]
    G["Export<br/>(CSV, PNG, SVG)"]
    
    A -->|Preprocessing| B
    B -->|Load & Validate| C
    C -->|Orchestrate| D
    D -->|Render| E
    D -->|Plot| F
    E -->|Interact| F
    F -->|Export| G
    
    style A fill:#e1f5ff
    style B fill:#e1f5ff
    style C fill:#fff3e0
    style D fill:#fff3e0
    style E fill:#fff3e0
    style F fill:#e8f5e9
    style G fill:#f3e5f5

3 Data Flow

3.1 High-Level Pipeline

The following sequence diagram illustrates the typical data flow in IMPACT-VIS:

sequenceDiagram
    actor User
    participant UI as UI Layer<br/>shiny.fluent
    participant Logic as Logic Layer<br/>Box Modules
    participant SeqArray as SeqArray<br/>On-Disk Access
    participant Plot as Visualization<br/>ggplot2 + plotly
    participant Storage as Persistent<br/>State RDS
    
    User->>UI: Select sample & filters
    UI->>Logic: Trigger reactive update
    Logic->>SeqArray: Open GDS, apply filters
    SeqArray->>Logic: Return filtered variants
    Logic->>Logic: Rank & subset (top N)
    Logic->>Plot: Pass processed data
    Plot->>Plot: Generate plot object
    Plot->>UI: Render interactive plot
    UI->>User: Display visualization
    User->>UI: Annotate variant
    UI->>Logic: Store annotation
    Logic->>Storage: Save RDS

4 Architectural Components

4.1 1. User Interface Layer (app/view/)

The UI layer consists of modular Shiny components built with shiny.fluent (Microsoft Fluent Design System):

Module Purpose Inputs
sample_input Sample picker & file loader File path
plot_settings Plot customization (type, scale, colors) Sample data
filters_ui Variant filtering controls Active filters
main_panel Central visualization area Processed data
variant_modal SNV detail inspector Variant ID
sv_variant_modal SV detail inspector SV ID
cnv_variant_modal CNV detail inspector CNV ID
export_ui Export controls & options Plotly object

Design Principle: Modules are stateless; all reactivity and state management flows through app/main.R.

4.2 2. Logic Layer (app/logic/)

The logic layer contains all computational workflows:

Module Responsibility
data_manager.R Load GDS, AnnotSV, CNV files; on-disk filtering
plot_gds.R SNV/Indel visualization (bottom panel of IMPACT Plot)
plot_sv.R SV visualization (top panel of IMPACT Plot: 5 SV type tracks)
plot_cnv.R CNV visualization (middle panel of IMPACT Plot)
plot_combine.R Multi-panel composition and synchronized interaction
validators.R File validation for each format
sample_loader.R Sample discovery & metadata
error_handler.R Consistent error/warning/success messaging
variant_modal_helpers.R Variant detail retrieval

Design Principle: Logic modules are pure functions; given the same inputs, they produce identical outputs.

4.3 3. Data Access Pattern

IMPACT-VIS implements a lazy-load, on-disk filtering pattern to minimize memory usage:

graph LR
    A["GDS File<br/>~100 MB"] -->|SeqArray| B["Open Handle<br/>(on-disk)"]
    B -->|setFilter| C["Filter<br/>(disk)"]
    C -->|getAfter filter| D["Subset Data<br/>(RAM)"]
    D -->|Rank & slice| E["Top N variants<br/>(display)"]
    
    style A fill:#e1f5ff
    style B fill:#fff3e0
    style C fill:#fff3e0
    style D fill:#e8f5e9
    style E fill:#e8f5e9

Key Advantages:

  • Full GDS file never loaded into memory
  • Filtering happens on-disk via SeqArray C++ backend
  • Only final subset transferred to R
  • Substantial reduction in memory footprint

5 Reactive Programming Model

IMPACT-VIS uses Shiny’s reactive framework with explicit dependency management:

# Example: Reactive flow in app/main.R

# 1. Input reactives
sample_input_data <- sample_input$server("sample_input")  
# Returns: reactive list with $gds, $sv, $cnv paths

# 2. Derived reactives
plot_settings_data <- plot_settings$server("plot_settings", sample_input_data)
# Depends on: sample selection

# 3. Filtered data
filtered_data <- reactive({
  data_manager$load_gds_data(
    sample_input_data$gds_path(),
    filters = plot_settings_data$filters(),
    bravo_thresh = plot_settings_data$bravo_threshold()
  )
})

# 4. Visualization
observe({
  plot_object <- plot_gds$plot_snv_scatter(filtered_data())
  output$main_plot <- renderPlotly(plot_object)
})

6 Module Dependency Graph

The following diagram illustrates the complete module dependency structure of IMPACT-VIS, organized by layer.

%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#fff3e0', 'primaryBorderColor': '#ff9800', 'secondaryColor': '#e8f5e9', 'tertiaryColor': '#e1f5ff'}, 'flowchart': { 'nodeSpacing': 40, 'rankSpacing': 50, 'useMaxWidth': false}}}%%
flowchart TB
    subgraph orchestration["<b>Orchestration Layer</b>"]
        direction TB
        main["<b>main.R</b><br/><i>Application Entry Point</i>"]
    end

    subgraph views["<b>View Layer</b> <i>(app/view/)</i>"]
        direction TB
        header["header_panel"]
        sample["sample_input"]
        settings["plot_settings"]
        filters["filters_ui"]
        tooltip["tooltip_ui"]
        export["export_ui"]
        mainpanel["<b>main_panel</b><br/><i>Central Visualization</i>"]
        
        subgraph modals["Variant Modals"]
            direction LR
            vm["variant_modal<br/><i>(SNV)</i>"]
            svm["sv_variant_modal<br/><i>(SV)</i>"]
            cvm["cnv_variant_modal<br/><i>(CNV)</i>"]
        end
        
        react["react<br/><i>(ToggleSection)</i>"]
    end

    subgraph logic["<b>Logic Layer</b> <i>(app/logic/)</i>"]
        direction TB
        
        subgraph dataproc["Data Processing"]
            dm["<b>data_manager</b><br/><i>Load & Filter</i>"]
            sl["sample_loader<br/><i>File Discovery</i>"]
            val["validators<br/><i>Schema Validation</i>"]
        end
        
        subgraph plotting["Visualization"]
            direction LR
            pgds["plot_gds<br/><i>(SNV/Indel)</i>"]
            psv["plot_sv<br/><i>(Structural Variants)</i>"]
            pcnv["plot_cnv<br/><i>(Copy Number)</i>"]
            pcomb["plot_combine<br/><i>(Multi-Panel)</i>"]
        end
        
        subgraph helpers["Helpers"]
            direction LR
            err["error_handler<br/><i>UI Notifications</i>"]
            vmh["variant_modal_helpers<br/><i>Field Extraction</i>"]
        end
    end

    %% Orchestration → View connections
    main --> header
    main --> sample
    main --> settings
    main --> filters
    main --> tooltip
    main --> export
    main --> mainpanel

    %% View internal connections
    settings -.->|uses| react
    filters -.->|uses| react
    tooltip -.->|uses| react
    mainpanel --> vm
    mainpanel --> svm
    mainpanel --> cvm

    %% View → Logic connections
    sample --->|loads samples| sl
    filters -->|reads metadata| dm
    filters -->|file paths| sl
    mainpanel -->|loads data| dm
    mainpanel -->|renders| pgds
    mainpanel -->|renders| psv
    mainpanel -->|renders| pcnv
    mainpanel -->|composes| pcomb
    mainpanel -->|notifications| err

    %% Modal → Helper connections
    vm -->|field utils| vmh
    svm -->|field utils| vmh
    cvm -->|field utils| vmh

    %% Logic internal connections
    sl -->|calls loaders| dm
    dm -->|validates| val
    dm -->|error handling| err
    pcomb -->|chr_lengths| pgds
    psv -->|chr_lengths| pgds
    pcnv -->|chr_lengths| pgds

    %% Styling
    style main fill:#fff3e0,stroke:#ff9800,stroke-width:3px
    style mainpanel fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    style dm fill:#fff3e0,stroke:#ff9800,stroke-width:2px
    
    style header fill:#e8f5e9,stroke:#4caf50
    style sample fill:#e8f5e9,stroke:#4caf50
    style settings fill:#e8f5e9,stroke:#4caf50
    style filters fill:#e8f5e9,stroke:#4caf50
    style tooltip fill:#e8f5e9,stroke:#4caf50
    style export fill:#e8f5e9,stroke:#4caf50
    style react fill:#c8e6c9,stroke:#388e3c
    
    style vm fill:#e3f2fd,stroke:#1976d2
    style svm fill:#e3f2fd,stroke:#1976d2
    style cvm fill:#e3f2fd,stroke:#1976d2
    
    style sl fill:#e1f5ff,stroke:#0288d1
    style val fill:#e1f5ff,stroke:#0288d1
    style pgds fill:#f3e5f5,stroke:#7b1fa2
    style psv fill:#f3e5f5,stroke:#7b1fa2
    style pcnv fill:#f3e5f5,stroke:#7b1fa2
    style pcomb fill:#f3e5f5,stroke:#7b1fa2
    style err fill:#ffebee,stroke:#c62828
    style vmh fill:#fff8e1,stroke:#f9a825

    style orchestration fill:#fafafa,stroke:#9e9e9e,stroke-dasharray: 5 5
    style views fill:#f1f8e9,stroke:#689f38,stroke-dasharray: 5 5
    style logic fill:#e8eaf6,stroke:#3f51b5,stroke-dasharray: 5 5
    style modals fill:#e3f2fd,stroke:#1976d2,stroke-dasharray: 3 3
    style dataproc fill:#e0f7fa,stroke:#00838f,stroke-dasharray: 3 3
    style plotting fill:#fce4ec,stroke:#c2185b,stroke-dasharray: 3 3
    style helpers fill:#fff8e1,stroke:#fbc02d,stroke-dasharray: 3 3

6.1 Dependency Legend

Color Layer Description
🟠 Orange fill Core modules main.R, main_panel, data_manager — primary control flow
🟢 Green fill View modules UI components in app/view/
🔵 Blue fill Modal dialogs Variant detail inspectors
🟣 Purple fill Plot modules Visualization generators
🔴 Red fill Error handling User notification system
🟡 Yellow fill Helpers Utility functions
⬜ Cyan fill Data utilities Validators and file discovery

7 Error Handling Strategy

IMPACT-VIS implements layered error handling:

  1. File Validation Layer (validators.R)
    • Schema validation before data loading
    • Returns list(is_valid = TRUE/FALSE, message = "...")
    • Prevents silent failures
  2. Load Layer (data_manager.R)
    • Returns NULL + warning() on failure
    • Caller can check for NULL and handle gracefully
  3. Presentation Layer (error_handler.R)
    • Converts errors/warnings to user-friendly notifications
    • Non-blocking warnings (yellow toast)
    • Blocking errors (modal dialog)

Example Flow:

# User selects invalid GDS
result <- load_gds_data(gds_path)
if (is.null(result)) {
  show_error("GDS file validation failed. Check format specifications.")
} else {
  # Continue with visualization
}

8 Reproducibility & Versioning

IMPACT-VIS ensures reproducibility through:

  1. Dependency Pinning: renv.lock specifies exact versions of all R packages
  2. Runtime Versioning: Session info captured at runtime
  3. Docker Containerization: Full OS + R environment encapsulated

9 Design Principles

9.1 1. Modularity

  • Each R module (Box) encapsulates a single responsibility
  • Clear, documented interfaces between modules
  • Facilitates testing and maintenance

9.2 2. Memory Efficiency

  • On-disk processing via SeqArray preferred over in-memory subsetting
  • Lazy evaluation for large datasets
  • Garbage collection optimizations

9.3 3. User-Centric Design

  • Fluent UI components familiar to Windows/Office users
  • Interactive tooltips with full variant details
  • Non-blocking error handling where possible

10 Key Technologies

Technology Role Version
R Language runtime ≥ 4.0
Shiny Web framework 1.7+
Rhino Project structure Latest
Box Module system Latest
SeqArray Genomic data access Latest
ggplot2 Visualization 3.4+
plotly Interactivity 4.10+
shiny.fluent UI components Latest

11 Next Steps

For detailed exploration of each subsystem:

  1. For SNV/Indel Processing: See SNV/Indel Processing
  2. For SV Processing: See SV Processing
  3. For CNV Processing: See CNV Processing
  4. For Architecture Deep Dive: See Architecture & Design

Document Version: 1.0.0
Last Updated: 2025-12-10