Help AI Agents answer questions and execute tasks for SpikeCV, an ultra-high-speed spike camera vision framework. Use when the user asks about spike cameras,...
---
name: spikecv-helper
description: Help AI Agents answer questions and execute tasks for SpikeCV, an ultra-high-speed spike camera vision framework. Use when the user asks about spike cameras, SpikeCV repository, spike dataset download, running vision task like tracking, reconstruction through spike vision, etc.
---
# SpikeCV Agent Skill
*last updated: 2026-04-28*
This document is designed to equip an AI Agent (or developer) with the necessary prior knowledge and instructions to answer questions and execute tasks within the SpikeCV repository.
Any command execution or conceptual question should be informed by the context in this document and the attached references.
> **Before reading**: If you're unfamiliar with spike cameras or SpikeCV's architecture, first read `references/About_SpikeCV.md`.
---
## ๐งญ Quick Lookup
Use this table to jump to the right section. If it's not here, the answer isn't in this skill.
| You want... | Go to... |
|---|---|
| What is SpikeCV? | `references/About_SpikeCV.md` |
| Install SpikeCV | [ยง Installation](#-installation) |
| SpikeCV API docs | [spikecv.readthedocs.io](https://spikecv.readthedocs.io) |
| SpikeCV Project directory structure | [ยง Project Layout](#-project-layout) |
| Run `spikecv` CLI (download, track, reconst) | [ยง CLI Reference](#-cli-reference) |
| Full publication list (up-to-date) | [spikecv.github.io/publications.html](https://spikecv.github.io/publications.html) |
| Algorithm and it's theory | search for `docs/source/ๆ ธๅฟๆไฝ.rst` ๐จ๐ณ or source code under `SpikeCV/spkProc/` ๐ฌ๐ง `SpikeCV/examples` ๐ฌ๐ง, you might get it wrong, so notify user to look up the details in publication for exact theory |
| SpikeCV event, competition, Team contact, contribution, hardware | `references/About_SpikeCV.md` |
| Dataset descriptions | `references/About_SpikeCV.md` |
| Troubleshooting | [ยง Gotchas & Common Errors](#-gotchas--common-errors) |
**Not in this skill?** โ Defer to the user or suggest SpikeCV [website](https://spikecv.github.io) and SpikeCV team contact (spikecv@outlook.com).
---
## ๐ฆ Installation
**Prerequisites**:
- Python โฅ 3.10
- Python virtual environment (**Conda** recommended): see [install_miniconda.md](references/install_miniconda.md) for a step-by-step setup guide from scratch.
```bash
git clone https://github.com/Zyj061/SpikeCV
cd SpikeCV
pip install .[cli]
```
Optional extras:
```bash
pip install .[tracking] # for SNNTracker/SpikeSORT
```
**Verify**:
```bash
spikecv --help
```
**Quick health check** (downloads ~40 MB and runs TFSTP reconstruction):
```bash
spikecv data download --dataset recVidarReal2019 --agent-used
cd datasets/
spikecv proc reconst --agent-used
```
---
## ๐ CLI Reference
SpikeCV provides the `spikecv` command for wrapper interactions. After installation, you can use it to download datasets and run processing algorithms.
All commands accept `--agent-used` to return structured JSON output. **Always include this flag.**
> **Note**: Feel free to explore other CLI options and parameters as needed. The following examples only showcase partial subcommands and parameters, the CLI may support additional features or algorithms. Always refer to `--help` for the most up-to-date command options and usage instructions.
### `spikecv data download`
Download a dataset from the OpenI platform:
```bash
spikecv data download --dataset <name> --local-dir <path> --agent-used
```
| Dataset | For Task |
|---|---|
| `recVidarReal2019` | Reconstruction |
| `motVidarReal2020` | Tracking |
**โ ๏ธ Path gotcha**: Data lands in `<local-dir>/<dataset>/`. Default local-dir is `datasets/`, so your dat files end up at `datasets/recVidarReal2019/classA/car-100kmh.dat`, NOT `recVidarReal2019/classA/car-100kmh.dat`.
### `spikecv proc reconst`
Reconstruct visible images from a raw `.dat` spike stream. Default algorithm: **TFSTP**.
```bash
# From the datasets/ directory:
cd datasets/
spikecv proc reconst --dat-file-path recVidarReal2019/classA/car-100kmh.dat --agent-used
# Or with an absolute/explicit path:
spikecv proc reconst --dat-file-path /path/to/recVidarReal2019/classA/car-100kmh.dat \
--yaml-file-path /path/to/recVidarReal2019/config.yaml \
--agent-used
```
**Key parameters**:
- `--dat-file-path`: Path to `.dat` spike stream file
- `--yaml-file-path`: Path to `config.yaml` for the dataset (default: `recVidarReal2019/config.yaml`)
- `--begin-idx`: Starting frame index (default: 500)
- `--block-len`: How many spike frames to process (default: 1500)
- `--stp-d`, `--stp-F`, `--stp-f`: STP model parameters
**โ ๏ธ Path gotcha**: The default `yaml-file-path` is `recVidarReal2019/config.yaml` (relative to cwd). If you download the dataset using the CLI, it lives under `datasets/recVidarReal2019/`. You must either `cd datasets/` first or pass explicit paths.
### `spikecv proc track`
Run multi-object tracking on spike data. Default algorithm: **SNNTracker**.
```bash
spikecv proc track --scene-idx 0 --metrics --agent-used
```
**Key parameters**:
- `--scene-idx`: Scene index 0โ6
- `--metrics`: (Optional) Calculate MOTA/IDF1 metrics (requires ground truth labels)
**โ ๏ธ Dataset**: Tracking uses `motVidarReal2020`, not `recVidarReal2019`.
---
## ๐ Project Layout
```
SpikeCV/
โโโ SpikeCV/
โ โโโ cli/ # CLI command definitions (typer)
โ โโโ device/ # Hardware driver for spike cameras
โ โโโ examples/ # Standalone test scripts (**NOT for agent use**, use CLI instead)
โ โโโ metrics/ # Quantitative evaluation (PSNR, SSIM, MOTA, IDF1, AEPE)
โ โโโ spkData/ # Data loaders + config.yaml parsing
โ โ โโโ load_dat.py # ParaDict generation, SpikeStream class
โ โโโ spkProc/ # โญ Core algorithms
โ โ โโโ filters/ # STP filter (background removal)
โ โ โโโ reconstruction/ # TFI, TFP, TFSTP, SSML
โ โ โโโ tracking/ # SNNTracker, SpikeSORT
โ โ โโโ detection/ # STDP, Motion-based detection
โ โ โโโ recognition/ # RPSNet, SVM, VGG
โ โ โโโ depth_estimation/ # SpikeT (Transformer)
โ โ โโโ optical_flow/ # SCFlow
โ โ โโโ augment/ # Data augmentation
โ โโโ utils/ # Path helpers
โ โโโ visualization/ # Video generation from spikes
โโโ docs/
โ โโโ spike_algo.md # deprecated
โ โโโ data_processing.md # deprecated
โ โโโ tools.md # deprecated
โ โโโ examples.md # deprecated
โ โโโ source/ # ๐จ๐ณ Sphinx sources (readthedocs)
โโโ Publications.md # ๐ Publication list (may be stale โ use website instead)
โโโ README.md / README_en.md
โโโ CONTRIBUTING.md / CONTRIBUTING_en.md
```
### ๐ Documentation by Language
| File(s) | Language | Best for |
|---|---|---|
| `docs/source/*.rst` | ๐จ๐ณ Chinese | API, little Algorithm theory, code examples |
| `SpikeCV/spkProc/**/*.py` (source code) | ๐ฌ๐ง English | Algorithm math + implementation details |
| `README_en.md`, `CONTRIBUTING_en.md` | ๐ฌ๐ง English | Setup, contribution workflow |
| `Publications.md` | ๐ฌ๐ง English | Publication list (stale โ see website) |
---
## ๐จ Rules (SHOULD NOT DO)
These are not suggestions โ following them prevents errors and hallucinations:
1. **Use CLI, not example scripts** โ If a task has a `spikecv` CLI command, use it. Do NOT run `SpikeCV/examples/test_snntracker.py` or similar standalone scripts directly.
2. **Always include `--agent-used`** โ Otherwise the CLI prints log text, not structured JSON.
3. **Do not modify SpikeCV code** โ Suggest the user contact the team instead (spikecv@outlook.com).
4. **Do not hallucinate external resources** โ For contacts, purchases, or contributions, only use info from `references/About_SpikeCV.md`.
5. **For publication/paper queries โ always use the website** โ `Publications.md` in the repo may be outdated. Point to [spikecv.github.io/publications.html](https://spikecv.github.io/publications.html) as the canonical source.
---
## ๐ Gotchas & Common Errors
| Symptom | Cause | Fix |
|---|---|---|
| `"Data path '...config.yaml' does not exist"` | `--yaml-file-path` is relative but you're not in the right directory | `cd datasets/` first, or pass full path |
| `spikecv proc track` fails | Wrong dataset โ using `recVidarReal2019` instead of `motVidarReal2020` | Download correct dataset first |
| `--scene-idx` out of range | `motVidarReal2020` only has scenes 0โ6 | Use a value in [0, 6] |
| `--scene-idx` in range but still broken | some of the scene data might not have uploaded by the developer, check carefully if the corresponding scene appear in the downloaded dataset | don't use the scene that's not downloaded |
| `spikecv` not found | CLI not installed or not in PATH | Run `pip install .[cli]` from SpikeCV root |
| `--agent-used` not working (no JSON output) | Using an older SpikeCV version | Update to latest, or check `spikecv --help` for flag name |don't have the plugin yet? install it then click "run inline in claude" again.