back
loading skill details...
Download OpenStreetMap features via Overpass API. Query by bbox and tag filter, output GeoJSON and Shapefile. Supports roads, buildings, POIs, landuse, natur...
---
name: osm-data-download
display_name: OSM数据下载工具
version: 0.1.0
author: ruiduobao
license: MIT-0
description: |
Download OpenStreetMap features via Overpass API. Query by bbox and tag filter,
output GeoJSON and Shapefile. Supports roads, buildings, POIs, landuse, natural features.
runtime: python>=3.8
tags: [gis, osm, openstreetmap, overpass, download]
---
# OSM Data Download
Download OpenStreetMap features via the Overpass API. Query by bounding box and tag filter, output as GeoJSON or Shapefile.
## Features
- **By bbox + tag**: Download roads, buildings, POIs, landuse, natural features
- **Custom Overpass QL**: Run your own queries
- **Multiple outputs**: GeoJSON and Shapefile
- **Rate limiting**: Built-in delays to respect API limits
- **Tag reference**: Built-in list of common OSM tags
## Common Feature Types
| Type | OSM Tag | Examples |
|------|---------|----------|
| Roads | `highway=*` | motorway, primary, residential |
| Buildings | `building=*` | yes, residential, commercial |
| POIs | `amenity=*` | restaurant, school, hospital |
| Landuse | `landuse=*` | residential, forest, farmland |
| Natural | `natural=*` | water, wood, grassland |
| Waterways | `waterway=*` | river, stream, canal |
## Usage
### Download roads in a bounding box
```bash
python scripts/osm-data-download.py download \
--bbox "116.0,39.5,116.8,40.2" \
--feature highway --output roads.geojson
```
### Download buildings
```bash
python scripts/osm-data-download.py download \
--bbox "116.3,39.8,116.5,40.0" \
--feature building --output buildings.geojson --format geojson
```
### Custom Overpass QL query
```bash
python scripts/osm-data-download.py query \
--query '[out:json][timeout:60];(node["amenity"="restaurant"](39.8,116.3,40.0,116.5););out body;' \
--output restaurants.geojson
```
### List common tags
```bash
python scripts/osm-data-download.py list-tags
```
## Installation
```bash
pip install requests>=2.28.0 tqdm>=4.64.0
# Or: pip install -r scripts/requirements.txt
```
## Parameters
- `--bbox`: Bounding box as `lon_min,lat_min,lon_max,lat_max`
- `--feature`: Feature type (`highway`, `building`, `amenity`, `shop`, `tourism`, `landuse`, `natural`, `waterway`)
- `--value`: Specific tag value (e.g., `restaurant`, `motorway`). Omit for all values.
- `--output`: Output file path
- `--format`: Output format (`geojson`, `shapefile`)
- `--query`: Custom Overpass QL query string
- `--timeout`: API timeout in seconds (default: 60)
- `--rate-delay`: Delay between requests in seconds (default: 1.0)
## Output
- **GeoJSON**: Standard GeoJSON with OSM tags as properties
- **Shapefile**: ESRI Shapefile with attribute table
## Geometry Types
OSM features come in three geometry types — choose based on your use case:
| Type | OSM Element | Typical Features | Use For |
|------|-------------|-----------------|---------|
| Points | `node` | POIs, amenities, shops | Point-based analysis, heatmaps |
| Lines | `way` (open) | roads, rivers, boundaries | Network analysis, routing |
| Polygons | `way` (closed), `relation` | buildings, landuse, lakes | Area calculations, spatial join |
The tool automatically detects geometry type. Use `--geometry-type` to filter.
## Maximum Bounding Box Size
Large bounding boxes cause timeouts and excessive data:
| Area Size | Recommendation |
|-----------|---------------|
| <0.25°×0.25° | Safe for most queries |
| 0.25°–0.5°×0.25°–0.5° | Recommended maximum for dense urban areas |
| >0.5°×0.5° | Split into smaller tiles; use `--split-bbox 4` |
```bash
# Auto-split large bbox into 4 sub-queries
python scripts/osm-data-download.py download \
--bbox "115.5,39.0,117.5,41.0" \
--feature building --output buildings.geojson --split-bbox 4
```
## Shapefile Output
Export directly to ESRI Shapefile format:
```bash
python scripts/osm-data-download.py download \
--bbox "116.3,39.8,116.5,40.0" \
--feature building --output buildings.shp --format shapefile
```
**Note**: Shapefile column names are truncated to 10 characters. Use `--format geojson` for full attribute names.
## Character Encoding
Shapefile attribute tables use UTF-8 encoding by default. If you see garbled text in ArcGIS:
- Set environment variable: `SHAPE_ENCODING=UTF-8`
- Or open in QGIS (handles UTF-8 natively)
- GeoJSON output is always UTF-8
## Error Handling and Retry Logic
The tool handles common HTTP errors automatically:
| HTTP Code | Meaning | Tool Behavior |
|-----------|---------|---------------|
| 400 | Bad query syntax | Reports error, suggests fixes |
| 429 | Rate limit exceeded | Waits 60s, retries up to 3 times |
| 504 | Server timeout | Increases timeout, retries up to 3 times |
| 500 | Server error | Waits 30s, retries |
Use `--max-retries 5` and `--retry-delay 120` to customize retry behavior.
## Alternative Overpass Endpoints
If the primary endpoint is slow or unavailable:
| Endpoint | Location | Notes |
|----------|----------|-------|
| `https://overpass-api.de/api/interpreter` | Germany | Default, most stable |
| `https://z.overpass-api.de/api/interpreter` | Germany | Mirror |
| `https://lz4.overpass-api.de/api/interpreter` | Germany | Mirror |
| `https://overpass.kumi.systems/api/interpreter` | Finland | Alternative |
| `https://overpass.openstreetmap.ru/api/interpreter` | Russia | Alternative |
Specify with `--endpoint https://overpass.kumi.systems/api/interpreter`.
## Empty Results Handling
If a query returns no features:
1. Verify bbox coordinates (lon/lat order, sign)
2. Check tag spelling against OSM wiki
3. Try larger bbox — the area may have no mapped features
4. Use `list-tags` to see available features in the area
The tool prints a warning and exits gracefully on empty results.
## Citation
Please cite OpenStreetMap data (required by ODbL license):
```bibtex
@misc{osm_contributors,
author = {{OpenStreetMap contributors}},
title = {OpenStreetMap Data},
howpublished = {\url{https://www.openstreetmap.org}},
year = {2024},
note = {ODbL License}
}
@software{osm_data_download,
author = {ruiduobao},
title = {OSM Data Download Tool},
url = {https://github.com/ruiduobao/osm-data-download},
version = {0.1.0},
year = {2024},
}
```
When using OSM data, display: `© OpenStreetMap contributors (ODbL)`.
## Troubleshooting
| Error | Cause | Solution |
|-------|-------|----------|
| `ConnectionError` | Network issue | Check internet, retry |
| `HTTP 429` | Rate limit | Wait 60s, retry |
| `ValueError` | Invalid bbox format | Check `lon_min,lat_min,lon_max,lat_max` |
| Empty output | No features in area | Verify bbox, check tag spelling |
| `ModuleNotFoundError` | Missing dep | Run pip install |
| `HTTP 504` | Server timeout | Reduce bbox size, increase `--timeout` |
| Garbled text in ArcGIS | Encoding issue | Use UTF-8 or output GeoJSON |
## API Information
- **Endpoint**: `https://overpass-api.de/api/interpreter`
- **No API key required**
- **Rate limits**: Please be respectful. Large queries may take time.
- **Data license**: ODbL (OpenStreetMap contributors)
## Dependencies
```
requests>=2.28.0
tqdm>=4.64.0
```
## Data Source
OpenStreetMap via Overpass API. Data © OpenStreetMap contributors (ODbL).
## Visualization
- **QGIS**: Load output GeoJSON/Shapefile directly → Layer → Add Layer → Add Vector Layer
- **Python (geopandas)**: `gdf = gpd.read_file('output.geojson'); gdf.plot()`
- **Leaflet/Mapbox**: Convert to GeoJSON and load in web map
- **Kepler.gl**: Drag-and-drop GeoJSON for interactive visualization
---
## Advanced Usage
### Batch Multi-City Download
```bash
# Download buildings for multiple cities
declare -A cities=( ["北京"]="116.0 39.8 116.8 40.2" ["上海"]="121.0 30.8 122.0 31.5" )
for city in "${!cities[@]}"; do
python scripts/osm_download.py download --feature building --bbox ${cities[$city]} --output osm_${city}_buildings.geojson
sleep 2
done
```
### CI/CD Integration (GitHub Actions)
```yaml
# .github/workflows/update-osm.yml
name: Update OSM Data
on:
schedule:
- cron: '0 0 1 * *' # Monthly
jobs:
download:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- run: pip install requests
- run: |
python scripts/osm_download.py download \
--feature building --bbox 116.0 39.8 116.8 40.2 \
--output data/beijing_buildings.geojson
```
### PostGIS Import
```bash
python scripts/osm_download.py download --feature building --bbox 116.0 39.8 116.8 40.2 --output buildings.geojson
ogr2ogr -f PostgreSQL PG:"dbname=gis_db" buildings.geojson -nln osm_buildings
```
### Performance Tips
- Add `--rate-delay 2` for complex queries to avoid rate limiting
- Use `--timeout 300` for large country-sized queries
- Custom Overpass QL: use `--query` with raw QL for unsupported feature types
---
## 中文说明
通过 Overpass API 下载 OpenStreetMap 数据。按边界框和标签筛选，输出 GeoJSON 或 Shapefile。
## 功能特性
- **按 bbox + 标签**：下载道路、建筑、POI、土地利用、自然地物
- **自定义 Overpass QL**：运行自定义查询
- **多种输出**：GeoJSON 和 Shapefile
- **速率限制**：内置延迟，尊重 API 限制
- **标签参考**：内置常见 OSM 标签列表
## 常见要素类型
| 类型 | OSM 标签 | 示例 |
|------|---------|------|
| 道路 | `highway=*` | motorway, primary, residential |
| 建筑 | `building=*` | yes, residential, commercial |
| POI | `amenity=*` | restaurant, school, hospital |
| 土地利用 | `landuse=*` | residential, forest, farmland |
| 自然地物 | `natural=*` | water, wood, grassland |
| 水系 | `waterway=*` | river, stream, canal |
## 使用示例
### 下载道路
```bash
python scripts/osm-data-download.py download \
--bbox "116.0,39.5,116.8,40.2" \
--feature highway --output roads.geojson
```
### 下载建筑
```bash
python scripts/osm-data-download.py download \
--bbox "116.3,39.8,116.5,40.0" \
--feature building --output buildings.geojson --format geojson
```
### 自定义查询
```bash
python scripts/osm-data-download.py query \
--query '[out:json][timeout:60];(node["amenity"="restaurant"](39.8,116.3,40.0,116.5););out body;' \
--output restaurants.geojson
```
### 列出常见标签
```bash
python scripts/osm-data-download.py list-tags
```
## 安装
```bash
pip install requests>=2.28.0 tqdm>=4.64.0
# 或: pip install -r scripts/requirements.txt
```
## 参数说明
- `--bbox`: 边界框 `lon_min,lat_min,lon_max,lat_max`
- `--feature`: 要素类型
- `--value`: 特定标签值（如 `restaurant`）
- `--output`: 输出文件路径
- `--format`: 输出格式（`geojson`, `shapefile`）
- `--query`: 自定义 Overpass QL 查询
- `--timeout`: API 超时秒数（默认: 60）
- `--rate-delay`: 请求间隔秒数（默认: 1.0）
## 输出结果
- **GeoJSON**: 标准 GeoJSON，OSM 标签作为属性
- **Shapefile**: ESRI Shapefile 含属性表
## API 信息
- **端点**: `https://overpass-api.de/api/interpreter`
- **无需 API 密钥**
- **速率限制**: 请合理使用，大查询可能需要时间
- **数据许可**: ODbL (OpenStreetMap contributors)
## 依赖库
```
requests>=2.28.0
tqdm>=4.64.0
```
## 几何类型
OSM 要素有三种几何类型 — 根据用途选择：
| 类型 | OSM 元素 | 典型要素 | 用途 |
|------|---------|---------|------|
| 点 | `node` | POI、设施、商铺 | 点分析、热力图 |
| 线 | `way`（开放） | 道路、河流、边界 | 网络分析、路径规划 |
| 面 | `way`（闭合）、`relation` | 建筑、土地利用、湖泊 | 面积计算、空间连接 |
工具自动检测几何类型。使用 `--geometry-type` 过滤。
## 最大边界框尺寸
过大的边界框会导致超时和过量数据：
| 区域大小 | 建议 |
|---------|------|
| <0.25°×0.25° | 大多数查询安全 |
| 0.25°–0.5°×0.25°–0.5° | 密集城区建议最大值 |
| >0.5°×0.5° | 分割为小块；使用 `--split-bbox 4` |
```bash
# 自动将大边界框拆分为 4 个子查询
python scripts/osm-data-download.py download \
--bbox "115.5,39.0,117.5,41.0" \
--feature building --output buildings.geojson --split-bbox 4
```
## Shapefile 输出
直接导出为 ESRI Shapefile 格式：
```bash
python scripts/osm-data-download.py download \
--bbox "116.3,39.8,116.5,40.0" \
--feature building --output buildings.shp --format shapefile
```
**注意**：Shapefile 列名截断为 10 字符。需要完整属性名时请使用 `--format geojson`。
## 字符编码
Shapefile 属性表默认使用 UTF-8 编码。如在 ArcGIS 中显示乱码：
- 设置环境变量：`SHAPE_ENCODING=UTF-8`
- 或在 QGIS 中打开（原生支持 UTF-8）
- GeoJSON 输出始终为 UTF-8
## 错误处理与重试逻辑
工具自动处理常见 HTTP 错误：
| HTTP 代码 | 含义 | 工具行为 |
|-----------|------|---------|
| 400 | 查询语法错误 | 报告错误，建议修复 |
| 429 | 超出速率限制 | 等待 60 秒，最多重试 3 次 |
| 504 | 服务器超时 | 增加超时，最多重试 3 次 |
| 500 | 服务器错误 | 等待 30 秒，重试 |
使用 `--max-retries 5` 和 `--retry-delay 120` 自定义重试行为。
## 备用 Overpass 端点
主端点缓慢或不可用时：
| 端点 | 位置 | 说明 |
|------|------|------|
| `https://overpass-api.de/api/interpreter` | 德国 | 默认，最稳定 |
| `https://z.overpass-api.de/api/interpreter` | 德国 | 镜像 |
| `https://lz4.overpass-api.de/api/interpreter` | 德国 | 镜像 |
| `https://overpass.kumi.systems/api/interpreter` | 芬兰 | 备用 |
| `https://overpass.openstreetmap.ru/api/interpreter` | 俄罗斯 | 备用 |
使用 `--endpoint https://overpass.kumi.systems/api/interpreter` 指定。
## 空结果处理
如果查询无要素：
1. 验证边界框坐标（lon/lat 顺序、符号）
2. 检查标签拼写（对照 OSM Wiki）
3. 尝试更大的边界框 — 该区域可能无映射要素
4. 使用 `list-tags` 查看区域内可用要素
工具在空结果时打印警告并正常退出。
## 引用格式
使用 OSM 数据时请引用（ODbL 许可要求）：
```bibtex
@misc{osm_contributors,
author = {{OpenStreetMap contributors}},
title = {OpenStreetMap Data},
howpublished = {\url{https://www.openstreetmap.org}},
year = {2024},
note = {ODbL License}
}
@software{osm_data_download,
author = {ruiduobao},
title = {OSM Data Download Tool},
url = {https://github.com/ruiduobao/osm-data-download},
version = {0.1.0},
year = {2024},
}
```
使用 OSM 数据时请显示：`© OpenStreetMap contributors (ODbL)`。
## 故障排除
| 错误 | 原因 | 解决方案 |
|------|------|---------|
| `ConnectionError` | 网络问题 | 检查网络，重试 |
| `HTTP 429` | 速率限制 | 等待 60 秒后重试 |
| `ValueError` | 边界框格式错误 | 检查 `lon_min,lat_min,lon_max,lat_max` |
| 无输出 | 区域内无要素 | 验证边界框，检查标签拼写 |
| `ModuleNotFoundError` | 缺少依赖 | 运行 pip install |
| `HTTP 504` | 服务器超时 | 缩小边界框，增加 `--timeout` |
| ArcGIS 中乱码 | 编码问题 | 使用 UTF-8 或输出 GeoJSON |
## 数据来源
OpenStreetMap via Overpass API. 数据 © OpenStreetMap contributors (ODbL)。
don't have the plugin yet? install it then click "run inline in claude" again.