5.2 KiB
danable backend
Backend API for danable, a computer-vision data labeling platform (classification, object detection, segmentation). Metadata lives in PostgreSQL; image bytes, datasets, models and exports live in Cloudflare R2 (S3-compatible). A Redis-backed action-event pipeline records every user operation for tracelog and replay.
Tech stack
- Python 3.10+, FastAPI, SQLAlchemy 2 (asyncio) + asyncpg, Alembic (reserved)
- PostgreSQL 16, Redis 7, Celery (reserved for heavy workers)
- boto3 for R2 (S3-compatible), PyJWT + bcrypt for auth, Pillow for thumbnails
- pytest + pytest-asyncio + aiosqlite for tests
Quickstart
cd backend
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env # then fill R2 keys (see Configuration)
docker compose up -d postgres redis
uvicorn src.main:app --host 0.0.0.0 --port 8000 --reload
curl -X POST localhost:8000/api/v1/admin/init-db
init-db creates all tables and seeds the dev admin:
- email:
admin@danable.local - password:
Admin123!
Change or remove this account before any shared deployment. API docs (Swagger):
http://localhost:8000/docs.
Detached runs (servers survive the shell):
bash restart_api.sh # kill + start API on :8000, health-checks itself
Configuration
Secrets live ONLY in .env (gitignored). Everything structural lives in
config.yaml. The application never reads secrets from yaml, and never reads
shape/thresholds from env.
.env key |
Purpose | Default |
|---|---|---|
DATABASE_URL |
Postgres DSN (asyncpg) | postgresql+asyncpg://danable:danable@localhost:5432/danable |
REDIS_URL |
Redis for the event queue | redis://localhost:6379/0 |
JWT_SECRET |
Signs access/refresh tokens | change-me (must override) |
JWT_EXPIRE_MINUTES |
Access token lifetime | 720 |
R2_ENDPOINT |
https://<account>.r2.cloudflarestorage.com |
account endpoint |
R2_ACCESS_KEY_ID / R2_SECRET_ACCESS_KEY |
R2 API token | empty (R2 calls return 503 until set) |
R2_BUCKET |
Main private bucket | danagble |
R2_PUBLIC_BASE_URL |
Optional public/CDN base for reads | empty (presigned URLs only) |
config.yaml holds: API prefix and page sizes, R2 key layout
(datasets/{project}/..., models/..., artifacts/...), presigned TTLs,
per-task settings, the AI model registry, traffic budgets, and thumbnail
settings (enabled, max_side: 512, quality: 70, micro 128px).
Project structure
backend/
config.yaml / config.example.yaml # non-secret shape (see above)
docker-compose.yml # postgres + redis + api
requirements.txt
models/ # SQLAlchemy tables (Postgres metadata only)
shared/ # cross-cutting: logger, tracing, traffic, r2, keys,
# thumbnails, events, schemas
src/
main.py # app wiring, middleware, lifespan (event drainer)
core/ # config (.env + yaml), db engine, security (JWT/bcrypt)
api/v1/ # thin routers: auth, users, projects, images,
# annotations, events, activities, system
services/ # business logic (routers never touch the DB directly)
stores/ # local dev mirror of the R2 layout (see stores/README.md)
tests/ # pytest suite (sqlite-backed, no external services)
Layering rule: routers parse/validate and call src/services/*; shared code
never imports routers. Background work goes through the Redis event queue,
never inline in requests.
Usage workflows
- Label flow:
POST /projects->POST .../images/presign-> PUT bytes straight to R2 ->POST .../images(confirm mints 512px + 128px thumbs) ->PUT .../annotationper image ->PATCH .../images/{id}?status=. - Image lists are paginated (
limit<= 200,offset,totalin response) with server-sidestatus/search/assigneefilters. The UI never fetches the whole dataset. - Backfill thumbnails for pre-existing rows:
POST /api/v1/admin/backfill-thumbnails?limit=100. - Every mutating call emits an action event (Redis ->
action_eventstable); the activity feed reads that table. Client-only actions arrive viaPOST /api/v1/eventswith asession_id. - Tracing: send
X-Session-ID(one per visit) and theAuthorizationheader; every response echoesX-Session-ID/X-Trace-IDand JSON logs carrysession_id,trace_id,user_idfor replay.
Full endpoint reference: docs/API.md.
Testing
.venv/bin/python -m pytest tests/ -q
Tests are sqlite-backed and need no running services. Current coverage: auth (hash/token/route protection), annotations (server defaults, 404), tracing middleware + event ingest, thumbnail resize bounds, pagination (totals, filters, limit clamp).
Notes and limits
POST /admin/*endpoints are dev-only: protect or remove them in any shared deployment.- Thumbnails are minted inline at confirm time (fail-open; the UI falls back to the full image). Move to a worker when uploads spike.
- R2 free tier is account-wide: watch Class A/B request counts once many annotators browse thousand-image projects; the UI paginates (50/page) and caches immutable thumbs for a year to stay inside it.
License
Apache-2.0, see LICENSE.