Software package & tutorials

GitHub repository Tutorial on GitHub Issues

Installation

The package is a plain Python source tree; clone it and install the dependencies. The preprocessing tools themselves (Guppy, minimap2, Nanopolish, Tombo, …) are external and listed on the pipeline page.

git clone https://github.com/nanobaselib/NanoBaseLib.git
cd NanoBaseLib
pip install h5py pandas numpy torch parasail

Package layout

NanoBaseLib/
├── dataprep/
│   ├── combine_nanopolish_eventalign.py   # merge eventalign rows per k-mer, build an index
│   └── extract_tombo_resquiggle.py        # export Tombo resquiggle results to text
├── base_calling/
│   ├── generate_dna_dataset.py            # signal chunks + ground truth from eventalign
│   ├── generate_rna_dataset.py
│   ├── base_calling_dataloader.py         # PyTorch Dataset / DataLoader
│   └── base_calling_evaluation.py         # M / X / I / D rates with parasail
├── segment_align/
│   ├── segment_align.py
│   ├── nanopolish_evaluation.py           # σ̂ and L̂ per tool
│   ├── tombo_evaluation.py
│   └── segpore_evaluation.py
├── rna_mod_detection/
│   ├── rna_mod_detection.py               # ROC / PR AUC against ground truth
│   ├── m6A.benchmark.csv, m5C.benchmark.csv
│   └── plot_m6A.py, plot_m5C.py
├── utils/
│   ├── sam_filter.py
│   └── stat_fast5.py
└── docs/

1. Data preparation

combine_nanopolish_eventalign

Nanopolish writes one line per raw event, so a single k-mer often spans several lines. This script merges them into one event per k-mer and writes an index into the original file for random access.

python dataprep/combine_nanopolish_eventalign.py \
    --input_file demo_dataset/4_nanopolish/eventalign.txt \
    --base_type rna --n_processes 10

Parameter	Description
--input_file	Path to the `eventalign.txt` produced by `nanopolish eventalign`.
--base_type	`rna` or `dna`.
--output_suffix	Suffix of the combined output file.
--n_processes	Number of worker processes.
--chunk_size	Lines of the eventalign file processed per chunk.

Outputs eventalign_combined.txt (one event per k-mer) and eventalign.index.

extract_tombo_resquiggle

python dataprep/extract_tombo_resquiggle.py --root_dir demo_dataset/5_tombo/single_reads --folder 0

Reads Tombo's per-read HDF5 results from --root_dir/--folder and writes tombo_resquiggle.txt (segments) and tombo_summary.txt (per-read attributes).

2. Base calling

generate_dna_dataset / generate_rna_dataset

Builds the base calling ground truth: raw-signal chunks matched to reference fragments, written as new fast5 files plus a label.csv.

mkdir demo_dataset_base_calling
python base_calling/generate_rna_dataset.py \
    --input_file demo_dataset/4_nanopolish/eventalign.txt \
    --summary_file demo_dataset/4_nanopolish/summary.txt \
    --reference_file demo_dataset/0_reference/ref.fa \
    --fast5_folder demo_dataset/1_raw_signal/multi_fast5 \
    --save_folder demo_dataset_base_calling

Parameter	Description
--input_file	Combined eventalign file.
--summary_file	Nanopolish eventalign summary.
--reference_file	Reference genome / transcriptome FASTA.
--fast5_folder	Original multi-fast5 folder.
--save_folder	Output folder for generated fast5 files and `label.csv`.

base_calling_dataloader

A PyTorch Dataset that pads signal chunks and label sequences and caches them as NumPy arrays. Use it directly in a training loop.

python base_calling/base_calling_dataloader.py --root_dir demo_dataset_base_calling

from base_calling.base_calling_dataloader import Seg2BaseDataset, dataset_prep
from torch.utils.data import DataLoader

dataset_prep("demo_dataset_base_calling/fast5", "demo_dataset_base_calling/label.csv",
             "demo_dataset_base_calling/output")   # writes seg_data_arr.npy, seg_len_arr.npy, base_seq_arr.npy, base_len_arr.npy
# then build Seg2BaseDataset(src_seqs, src_lens, tgt_seqs, tgt_lens) from the arrays
# items: {"src_seq", "src_len", "tgt_seq", "tgt_len"} as tensors; base ids N=0 A=1 T=2 C=3 G=4

Output file	Content
seg_data_arr.npy	Padded current-signal chunks.
seg_len_arr.npy	Length of each signal chunk.
base_seq_arr.npy	Ground-truth base sequences (integer-encoded).
base_len_arr.npy	Length of each ground-truth sequence.

base_calling_evaluation

Aligns predicted sequences to the ground truth with parasail and reports average match, mismatch, insertion and deletion rates. Readers for Dorado, Guppy, Rodan, Halcyon and Radian outputs are included.

python base_calling/base_calling_evaluation.py --root_dir bc_benchmark --base_type dna --model guppy_v6.0.1

3. PolyA detection

PolyA results from Nanopolish (4_nanopolish/polya.tsv) and Tailfindr (3_tailfindr/tails.csv) ship with every RNA dataset. A dedicated dataloader is planned; see the label files for the ground truth format.

4. Segmentation and event alignment

nanopolish_evaluation.py, tombo_evaluation.py and segpore_evaluation.py compute the average std σ̂ and average log-likelihood L̂ described on the segmentation page from each tool's output and the ONT k-mer table.

5. RNA modification detection

rna_mod_detection.py reads a benchmark CSV of per-site predictions and labels and computes ROC and PR AUC; plot_m6A.py and plot_m5C.py reproduce the figures on the modification detection page.

python rna_mod_detection/rna_mod_detection.py --benchmark_file rna_mod_detection/m6A.benchmark.csv

Extending NanoBaseLib with new data

When a new public dataset appears, run the unified pipeline on its raw fast5, place the outputs in the standard directory layout, and the package scripts work unchanged. Contributions of new datasets, tools or baselines are welcome through pull requests.