# danable backend Backend API for danable, a computer-vision data labeling platform (classification, object detection, segmentation). Metadata lives in PostgreSQL; image bytes, datasets, models and exports live in Cloudflare R2 (S3-compatible). A Redis-backed action-event pipeline records every user operation for tracelog and replay. ## Tech stack - Python 3.10+, FastAPI, SQLAlchemy 2 (asyncio) + asyncpg, Alembic (reserved) - PostgreSQL 16, Redis 7, Celery (reserved for heavy workers) - boto3 for R2 (S3-compatible), PyJWT + bcrypt for auth, Pillow for thumbnails - pytest + pytest-asyncio + aiosqlite for tests ## Quickstart ```bash cd backend python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt cp .env.example .env # then fill R2 keys (see Configuration) docker compose up -d postgres redis uvicorn src.main:app --host 0.0.0.0 --port 8000 --reload curl -X POST localhost:8000/api/v1/admin/init-db ``` `init-db` creates all tables and seeds the dev admin: - email: `admin@danable.local` - password: `Admin123!` Change or remove this account before any shared deployment. API docs (Swagger): `http://localhost:8000/docs`. Detached runs (servers survive the shell): ```bash bash restart_api.sh # kill + start API on :8000, health-checks itself ``` ## Configuration Secrets live ONLY in `.env` (gitignored). Everything structural lives in `config.yaml`. The application never reads secrets from yaml, and never reads shape/thresholds from env. | `.env` key | Purpose | Default | |---|---|---| | `DATABASE_URL` | Postgres DSN (asyncpg) | `postgresql+asyncpg://danable:danable@localhost:5432/danable` | | `REDIS_URL` | Redis for the event queue | `redis://localhost:6379/0` | | `JWT_SECRET` | Signs access/refresh tokens | `change-me` (must override) | | `JWT_EXPIRE_MINUTES` | Access token lifetime | `720` | | `R2_ENDPOINT` | `https://.r2.cloudflarestorage.com` | account endpoint | | `R2_ACCESS_KEY_ID` / `R2_SECRET_ACCESS_KEY` | R2 API token | empty (R2 calls return 503 until set) | | `R2_BUCKET` | Main private bucket | `danagble` | | `R2_PUBLIC_BASE_URL` | Optional public/CDN base for reads | empty (presigned URLs only) | `config.yaml` holds: API prefix and page sizes, R2 key layout (`datasets/{project}/...`, `models/...`, `artifacts/...`), presigned TTLs, per-task settings, the AI model registry, traffic budgets, and thumbnail settings (`enabled`, `max_side: 512`, `quality: 70`, micro `128px`). ## Project structure ```text backend/ config.yaml / config.example.yaml # non-secret shape (see above) docker-compose.yml # postgres + redis + api requirements.txt models/ # SQLAlchemy tables (Postgres metadata only) shared/ # cross-cutting: logger, tracing, traffic, r2, keys, # thumbnails, events, schemas src/ main.py # app wiring, middleware, lifespan (event drainer) core/ # config (.env + yaml), db engine, security (JWT/bcrypt) api/v1/ # thin routers: auth, users, projects, images, # annotations, events, activities, system services/ # business logic (routers never touch the DB directly) stores/ # local dev mirror of the R2 layout (see stores/README.md) tests/ # pytest suite (sqlite-backed, no external services) ``` Layering rule: routers parse/validate and call `src/services/*`; shared code never imports routers. Background work goes through the Redis event queue, never inline in requests. ## Usage workflows - Label flow: `POST /projects` -> `POST .../images/presign` -> PUT bytes straight to R2 -> `POST .../images` (confirm mints 512px + 128px thumbs) -> `PUT .../annotation` per image -> `PATCH .../images/{id}?status=`. - Image lists are paginated (`limit` <= 200, `offset`, `total` in response) with server-side `status` / `search` / `assignee` filters. The UI never fetches the whole dataset. - Backfill thumbnails for pre-existing rows: `POST /api/v1/admin/backfill-thumbnails?limit=100`. - Every mutating call emits an action event (Redis -> `action_events` table); the activity feed reads that table. Client-only actions arrive via `POST /api/v1/events` with a `session_id`. - Tracing: send `X-Session-ID` (one per visit) and the `Authorization` header; every response echoes `X-Session-ID` / `X-Trace-ID` and JSON logs carry `session_id`, `trace_id`, `user_id` for replay. Full endpoint reference: [docs/API.md](docs/API.md). ## Testing ```bash .venv/bin/python -m pytest tests/ -q ``` Tests are sqlite-backed and need no running services. Current coverage: auth (hash/token/route protection), annotations (server defaults, 404), tracing middleware + event ingest, thumbnail resize bounds, pagination (totals, filters, limit clamp). ## Notes and limits - `POST /admin/*` endpoints are dev-only: protect or remove them in any shared deployment. - Thumbnails are minted inline at confirm time (fail-open; the UI falls back to the full image). Move to a worker when uploads spike. - R2 free tier is account-wide: watch Class A/B request counts once many annotators browse thousand-image projects; the UI paginates (50/page) and caches immutable thumbs for a year to stay inside it. ## License Apache-2.0, see [LICENSE](LICENSE).