scvi-tools Deep Learning Skill

intent

Use this skill when working with single-cell RNA-seq, ATAC-seq, CITE-seq, multiome, or spatial transcriptomics data and you need probabilistic deep learning models for batch correction, data integration, multi-modal analysis, label transfer, or RNA velocity. scvi-tools is the framework to reach for when variational autoencoders (VAEs) or related methods are mentioned, when integrating data across multiple technologies or batches, when mapping query samples to reference models, or when extracting latent representations for downstream analysis.

inputs

Python environment:

• Python 3.8+
• scvi-tools 0.20.0+ (install via pip install scvi-tools or conda install -c conda-forge scvi-tools)
• scanpy, anndata, numpy, scipy, pandas for data handling
• Optional: GPU support (CUDA 11.8+, cuDNN) for faster training on large datasets

Data requirements:

• AnnData object (.h5ad format) with raw integer count data stored in a dedicated layer (e.g., adata.layers["counts"])
• Minimum 2000-4000 highly variable genes selected before model setup
• Batch/condition metadata as a column in adata.obs (e.g., batch_key="batch")
• For label transfer workflows: pre-trained reference model (file path or HuggingFace model ID)
• For spatial analysis: spatial coordinates and optionally a reference single-cell dataset

External resources (optional):

• HuggingFace Model Hub access for downloading pre-trained models (https://huggingface.co/scvi-tools)
• scvi-tools documentation (https://docs.scvi-tools.org/)

Model selection context:

• Data modality (scRNA-seq, scATAC-seq, CITE-seq, multiome, spatial)
• Availability of cell type labels for semi-supervised learning
• Whether cross-technology batch correction is needed
• Presence of pre-trained reference models for query mapping

procedure

1. Validate and prepare input data

   • Input: raw .h5ad file or AnnData object
   • Run QC: filter cells (min genes, max mitochondrial %, min counts), filter genes (min cells, max genes)
   • Output: QC metrics logged, low-quality cells/genes removed
   • Edge case: if data is already normalized or log-transformed, flag for warning and preserve raw counts in separate layer

2. Select highly variable genes (HVGs)

   • Input: filtered AnnData, batch_key if multi-batch
   • Run sc.pp.highly_variable_genes() with flavor="seurat_v3", n_top_genes=2000 (adjust 2000-4000 based on dataset size and sparsity)
   • Output: adata.var["highly_variable"] boolean column, filtered AnnData subset to HVGs
   • Edge case: if batch_key is provided, compute HVGs per batch then take union to avoid losing batch-specific signals

3. Prepare raw count layer

   • Input: HVG-filtered AnnData
   • Ensure adata.layers["counts"] contains original integer counts (not normalized, not log-transformed)
   • Store normalized/log counts in adata.X if needed for other downstream tools
   • Output: adata.layers["counts"] verified as raw counts, shape (n_obs, n_vars_hvg)
   • Edge case: if counts layer missing, check if adata.X is raw; if not, fail and request raw data

4. Select appropriate scvi-tools model based on data type and task

   • Input: data modality, presence of labels, batch structure, downstream goal
   • Decision branch (see "decision points" section below)
   • Output: model class selected (scVI, scANVI, totalVI, PeakVI, MultiVI, DestVI, veloVI, or sysVI)

5. Setup AnnData for the selected model

   • Input: AnnData, model class, batch_key, optional label_key for scANVI
   • Call model.setup_anndata() with appropriate arguments (e.g., scvi.model.SCVI.setup_anndata(adata, layer="counts", batch_key="batch"))
   • Output: model-specific setup metadata stored in adata.uns["scvi"] (or equivalent)
   • Edge case: if setup_anndata() fails due to missing columns, provide clear error indicating which .obs or .var columns are required

6. Initialize and train the model

   • Input: setup AnnData, hyperparameters (hidden_dim, latent_dim, n_layers, dropout_rate, learning_rate, n_epochs)
   • Create model instance: model = SCVI(adata, ...)
   • Train: model.train(max_epochs=N, early_stopping=True, early_stopping_patience=20)
   • Output: trained model object, training history (loss curves), saved model checkpoint (directory or .pth file)
   • Edge case: if GPU out-of-memory, reduce batch_size or latent_dim; if training loss plateaus early, increase n_epochs or adjust learning_rate; if training diverges, reduce learning_rate by 10x

7. Extract and visualize latent representations

   • Input: trained model, AnnData
   • Get latent codes: latent = model.get_latent_representation()
   • Compute neighbors, UMAP, and Leiden clustering: sc.pp.neighbors(adata, use_rep="X_scvi"), sc.tl.umap(adata), sc.tl.leiden(adata, resolution=0.8)
   • Output: latent representation stored in adata.obsm["X_scvi"], UMAP in adata.obsm["X_umap"], cluster labels in adata.obs["leiden"]
   • Edge case: if UMAP computation is slow on large datasets (>100k cells), subsample for visualization or use alternative (e.g., trimap, PCA)

8. Perform task-specific downstream analysis (differential expression, label transfer, deconvolution, etc.)

   • Input varies by task; see task-specific reference files
   • Output: task-dependent (DE results table, transferred labels, deconvolution weights, velocity vectors)
   • Edge case: ensure model is kept in memory or reloaded from checkpoint before downstream steps

9. Validate integration quality and save results

   • Input: integrated AnnData, model, task-specific outputs
   • Compute integration metrics if multi-batch: kBET score, iLISI, cLISI, batch correction metrics
   • Save: model (model.save(dirpath)), AnnData with results (adata.write_h5ad(path)), plots (UMAP, DE volcano, etc.)
   • Output: validated results directory with model checkpoint, processed AnnData, metrics CSV, visualizations
   • Edge case: if model directory already exists, move or version old checkpoint to avoid overwrite; if save fails due to disk space, provide size estimate

decision points

Workflow selection (data type + task):

• If data is scRNA-seq only and you have cell type labels, use scANVI for semi-supervised integration and label transfer (reference: label_transfer.md)
• Else if data is scRNA-seq only and you have no labels, use scVI for unsupervised batch correction and integration (reference: scrna_integration.md)
• If data is CITE-seq (RNA + surface proteins), use totalVI for multi-modal integration (reference: citeseq_totalvi.md)
• Else if data is multiome (RNA + ATAC from same cells), use MultiVI for joint modality learning (reference: multiome_multivi.md)
• Else if data is scATAC-seq only, use PeakVI for chromatin accessibility analysis (reference: atac_peakvi.md)
• Else if data is spatial transcriptomics with a single-cell reference, use DestVI for cell type deconvolution (reference: spatial_deconvolution.md)
• Else if you need RNA velocity / transcriptional dynamics, use veloVI (reference: rna_velocity_velovi.md)
• Else if you have strong cross-technology batch effects (e.g., 10x + Dropseq + SMART-seq), use sysVI (reference: batch_correction_sysvi.md)

Label transfer decision:

• If you have a pre-trained reference model and a new query dataset, use scArches for query-to-reference mapping (reference: scarches_mapping.md)
• Else if you have scANVI reference and query is unlabeled, retrain scANVI in semi-supervised mode on combined ref+query data

Hardware / environment decision:

• If training stalls or GPU memory errors occur, check references/environment_setup.md for CUDA/cuDNN versions, reduce batch_size, or switch to CPU training
• If you encounter dependency conflicts, consult references/troubleshooting.md for version pinning and virtual environment setup

Data preparation edge case:

• If raw count layer is missing or data appears pre-normalized, stop and request raw counts; do not attempt to reverse-engineer normalization

output contract

Model checkpoint:

• Directory containing model.pt, var_names.csv, setup_dict.json
• Location: user-specified (default: results/<model_name>_checkpoint/)

Processed AnnData:

• File: adata_processed.h5ad
• Contains: original raw counts in adata.layers["counts"], HVG-filtered genes, latent representation in adata.obsm["X_scvi"] (or model-specific key), UMAP/clustering in adata.obsm["X_umap"], adata.obs["leiden"]
• Format: HDF5-backed AnnData, lossless

Integration metrics (if multi-batch):

• File: integration_metrics.csv
• Columns: kBET_score, iLISI, cLISI, silhouette_score, batch_purity (if applicable)
• Single row or per-batch rows depending on metric

Task-specific outputs:

• Differential expression: CSV with columns [gene, log2fc, p_value, q_value]
• Label transfer: AnnData with transferred labels in adata.obs["transferred_labels"], confidence scores in adata.obs["transfer_confidence"]
• Spatial deconvolution: CSV with cell type proportions per spatial location
• RNA velocity: vector field visualization (PDF/PNG) and velocity vectors in adata.obsm["velocity"]

Logs and diagnostics:

• File: training.log (loss curves, convergence info)
• File: validation_report.txt (data QC summary, model hyperparameters, warnings)

outcome signal

• Training converged: ELBO loss plateaus and does not spike; training log shows stable loss for final 10+ epochs
• Latent space is meaningful: UMAP visualization clusters cells by cell type (if labels known) and batches are mixed (if integration is successful)
• Integration successful (multi-batch): kBET p-value > 0.05 (fails to reject null of random batch distribution), iLISI > 1.5, cLISI > 1.0 (cell type label impurity is low)
• Downstream task passes sanity checks: DE markers align with known biology, transferred labels match expected cell types (if reference is reliable), spatial deconvolution proportions sum to ~1.0 per location
• Model saves without error: model checkpoint directory exists with all required files; AnnData file is valid and readable via sc.read_h5ad()
• No warnings or errors in logs: validation_report.txt contains no "ERROR" or "CRITICAL" lines; any "WARNING" lines are expected (e.g., "convergence stalled but acceptable")