# Skill: Production Deployment

## Identity
- **Skill ID**: `deployment-dockerization`
- **Domain**: DevOps, Deployment, Podman Containers
- **Source Agent**: `backend-app-expert.md`

## CRITICAL: What Requires What

| What changed | What to do |
|---|---|
| `admin-panel/src/**` (frontend only) | Build locally with `VITE_API_URL=/api`, commit `dist/`, push, `git pull` on remote |
| `api/src/**` (backend only) | Push, `git pull` on remote, `podman-compose up -d --build api worker` |
| Both frontend + backend | Do both of the above |

**WARNING**: The API runs inside a Podman container. A `git pull` alone does NOT update the running API. You MUST rebuild the image and restart containers for any `api/` changes to take effect.

## Production Server

- **SSH**: `ssh root@100.116.176.87`
- **Tailscale IP**: `100.116.176.87` (used for direct access before DNS is configured)
- **User**: `press-api` (run commands via `su - press-api -c '...'`)
- **Backend location**: `/home/press-api/Press.Zone-Works/press-zone-backend/`
- **API dir**: `/home/press-api/Press.Zone-Works/press-zone-backend/api/`
- **Admin panel dir**: `/home/press-api/Press.Zone-Works/press-zone-backend/admin-panel/`
- **Env file**: `/home/press-api/Press.Zone-Works/press-zone-backend/.env` (at backend root, NOT in api/)
- **Git remote**: `git@github.com:avi-ezra/Press.Zone-Works.git` (branch: `master`)
- **API URL**: `https://api.press.zone` (proxied via nginx to localhost:3000)
- **Admin panel URL**: `https://admin.translate.press.zone` (also accessible via Tailscale IP)
- **Health check**: `curl https://api.press.zone/health`

## Architecture

All 4 services run in Podman Compose with a shared bridge network (`tpz-backend`).
Managed via a single `podman-compose.yml` at the backend root.

| Service | Container | Details |
|---------|-----------|---------|
| PostgreSQL | `tpz-postgres` | `postgres:15-alpine`, port `127.0.0.1:5432` (host-accessible for backups + Prisma Studio) |
| Redis | `tpz-redis` | `redis:7-alpine`, internal only (no host port) |
| API | `presszone-api` | Image `localhost/presszone-api`, port `127.0.0.1:3000` (for nginx proxy) |
| Worker | `presszone-worker` | Same image, runs `node dist/worker.js` |
| Admin panel | Static files via nginx | `admin-panel/dist/` served by nginx directly |

### Network

- Bridge network `tpz-backend` — containers reference each other by service name (`postgres`, `redis`)
- `DATABASE_URL` and `REDIS_URL` are overridden in compose to use container hostnames
- Host ports are bound to `127.0.0.1` only (not externally accessible)

### Data Persistence

- PostgreSQL data: named volume `tpz-postgres-data`
- Redis data: named volume `tpz-redis-data`
- API logs: bind mount `./api/logs`
- Prisma migrations: bind mount `./api/prisma`

## Deployment Procedure

### Admin Panel Only Deploy (frontend-only changes)

The admin panel `dist/` is committed to git and served directly by nginx. No container rebuild needed.

**IMPORTANT**: Always build with `VITE_API_URL=/api` so the frontend uses nginx's `/api/` proxy (which forwards to `localhost:3000` and strips CORS). Without this, the build bakes in `http://localhost:3000` and requests fail with CORS errors.

```bash
# 1. Build locally (MUST set VITE_API_URL)
cd press-zone-backend/admin-panel && VITE_API_URL=/api npm run build

# 2. Commit and push (dist/ is tracked in git)
cd .. && git add admin-panel/dist/ && git commit -m "build: admin panel" && git push origin master

# 3. SSH and pull - nginx serves the new files immediately
ssh root@100.116.176.87
su - press-api -c 'cd ~/Press.Zone-Works && git pull origin master'
```

### API Deploy (backend code changes, no schema changes)

```bash
# 1. Push changes to GitHub from local
cd press-zone-backend && git push origin master

# 2. SSH and pull
ssh root@100.116.176.87
su - press-api -c 'cd ~/Press.Zone-Works && git pull origin master'

# 3. Rebuild and restart API + Worker only
su - press-api -c 'cd ~/Press.Zone-Works/press-zone-backend && podman-compose up -d --build api worker'

# 4. Verify (wait ~15s for startup + migrations)
sleep 15
curl https://api.press.zone/health
su - press-api -c 'podman ps --format "{{.Names}} {{.Status}}"'
```

### Full Deploy (API + Admin Panel)

Combine both: build admin panel locally, commit dist, push, then on remote pull + rebuild API containers.

### Migration Deploy (schema changes)

Same as API deploy. The API container command includes `npx prisma migrate deploy` which runs before `npm start`. The prisma/ directory is mounted from the host so it always uses the latest migration files.

### Full Stack Rebuild (nuclear option)

```bash
ssh root@100.116.176.87
su - press-api -c 'cd ~/Press.Zone-Works && git pull origin master'
su - press-api -c 'cd ~/Press.Zone-Works/press-zone-backend && podman-compose down && podman-compose up -d --build'

# Verify all 4 containers are running
sleep 15
curl https://api.press.zone/health
su - press-api -c 'podman-compose -f ~/Press.Zone-Works/press-zone-backend/podman-compose.yml ps'
```

## Backup

```bash
# Manual backup
su - press-api -c 'bash ~/Press.Zone-Works/press-zone-backend/backup/backup.sh'

# Ad-hoc PostgreSQL dump
su - press-api -c 'podman exec tpz-postgres pg_dump -U translate_user -d translate_db -F c > ~/backup.dump'

# Restore from backup
su - press-api -c 'bash ~/Press.Zone-Works/press-zone-backend/backup/restore.sh ~/press-zone-backend/backup/postgres_YYYYMMDD_HHMMSS.dump.gz'
```

## Troubleshooting

### Check container logs
```bash
su - press-api -c 'podman-compose -f ~/Press.Zone-Works/press-zone-backend/podman-compose.yml logs --tail 50 api'
su - press-api -c 'podman-compose -f ~/Press.Zone-Works/press-zone-backend/podman-compose.yml logs --tail 50 worker'
su - press-api -c 'podman-compose -f ~/Press.Zone-Works/press-zone-backend/podman-compose.yml logs --tail 50 postgres'
```

### Check running containers
```bash
su - press-api -c 'podman ps --format "{{.Names}} {{.Status}}"'
```

### Restart single service
```bash
su - press-api -c 'cd ~/Press.Zone-Works/press-zone-backend && podman-compose restart api'
```

### Database access (containerized PostgreSQL)
```bash
# Interactive psql via container
su - press-api -c 'podman exec -it tpz-postgres psql -U translate_user -d translate_db'

# Or via host port (localhost:5432 is mapped)
su - press-api -c 'psql -h 127.0.0.1 -U translate_user -d translate_db'

# Prisma Studio (from API container)
su - press-api -c 'podman exec presszone-api npx prisma studio'
```

### Test bridge network DNS
```bash
su - press-api -c 'podman exec presszone-api ping -c 1 postgres'
su - press-api -c 'podman exec presszone-api ping -c 1 redis'
```

### Run ad-hoc script in container
```bash
su - press-api -c 'podman cp /tmp/script.js presszone-api:/app/script.js && podman exec presszone-api node script.js'
```

## Admin API Access (creating test licenses etc.)

```bash
# Login to get JWT
TOKEN=$(curl -s https://api.press.zone/v1/admin/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"email":"admin@translate.press.zone","password":"admin123"}' \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['access_token'])")

# Create translate license
curl -s https://api.press.zone/v1/admin/licenses \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"email":"user@example.com","plan_tier":"enterprise","sites_allowed":-1,"languages_allowed":-1,"expires_at":"2027-03-07T00:00:00Z","plugin":"translate"}'

# Create multilingual license
curl -s https://api.press.zone/v1/admin/licenses \
  -H 'Content-Type: application/json' \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"email":"user@example.com","plan_tier":"enterprise","sites_allowed":-1,"languages_allowed":-1,"expires_at":"2027-03-07T00:00:00Z","plugin":"multilingual"}'
```

## Rollback

```bash
ssh root@100.116.176.87
su - press-api -c 'cd ~/Press.Zone-Works && git log --oneline -5'
# Revert to specific commit
su - press-api -c 'cd ~/Press.Zone-Works && git checkout <commit-hash>'
# Rebuild and restart
su - press-api -c 'cd ~/Press.Zone-Works/press-zone-backend && podman-compose up -d --build'
```

## Validation Checklist

### Admin Panel Deploy
- [ ] `npm run build` in `admin-panel/` succeeded locally
- [ ] `admin-panel/dist/` committed and pushed
- [ ] `git pull` on production succeeded
- [ ] Admin panel loads at Tailscale IP or `admin.translate.press.zone`

### API Deploy
- [ ] `git push origin master` succeeded
- [ ] `git pull` on production succeeded
- [ ] `podman-compose up -d --build api worker` completed
- [ ] `curl https://api.press.zone/health` returns healthy
- [ ] All 4 containers are up: `tpz-postgres`, `tpz-redis`, `presszone-api`, `presszone-worker`