# Modal.com Translation Service

ML-powered translation service deployed on Modal.com infrastructure.

## Models

- **4B Model**: Fast, cost-effective translations using `google/flan-t5-base` (T4 GPU)
- **27B Model**: Higher quality translations using `google/flan-t5-xxl` (A100 GPU)

## Features

- HTML tag preservation during translation
- Multiple tone options (neutral, formal, casual)
- Token usage tracking
- Automatic model caching after first run
- Error handling and logging

## Setup

### 1. Install Modal CLI

```bash
pip install modal
```

### 2. Authenticate

```bash
modal token new
```

This will open a browser window for OAuth authentication.

### 3. Deploy Service

```bash
modal deploy translate_service.py
```

### 4. Get Endpoint URL

After deployment, Modal will output:

```
✓ Created web endpoint => https://your-username--translate-gemma-translate.modal.run
```

Copy this URL and add it to your `.env` file as `MODAL_API_URL`.

### 5. Generate API Token

```bash
modal token new --profile api
```

Copy the generated token and add it to your `.env` file as `MODAL_API_KEY`.

## API Usage

### Translation Endpoint

**POST** `/translate`

**Request:**
```json
{
  "content": "Hello world",
  "source_lang": "en",
  "target_lang": "es",
  "model": "4b",
  "tone": "neutral"
}
```

**Response:**
```json
{
  "translation": "Hola mundo",
  "tokens_used": 12,
  "processing_time_ms": 1500,
  "model": "4b"
}
```

### Health Check

**GET** `/health`

**Response:**
```json
{
  "status": "healthy",
  "service": "translate-gemma",
  "models": ["4b", "27b"]
}
```

## HTML Tag Preservation

The service automatically preserves HTML tags during translation:

**Input:**
```html
<p>Hello <strong>world</strong>!</p>
```

**Process:**
1. Extract tags: `__TAG_0__ Hello __TAG_1__ world __TAG_2__ ! __TAG_3__`
2. Translate: `__TAG_0__ Hola __TAG_1__ mundo __TAG_2__ ! __TAG_3__`
3. Restore tags: `<p>Hola <strong>mundo</strong>!</p>`

## Costs

### Free Tier
- 30 GPU hours/month
- Suitable for development and testing
- Approximately 1,000-2,000 translations/hour on 4B model

### Paid Tier
- T4 GPU: ~$0.60/hour (4B model)
- A100 GPU: ~$3.00/hour (27B model)

## Development

### Local Testing

```bash
# Run function locally (uses Modal's local execution)
modal run translate_service.py::translate_4b --content "Hello" --source-lang "en" --target-lang "es" --tone "neutral"
```

### View Logs

```bash
modal logs translate-gemma
```

### Redeploy

```bash
modal deploy translate_service.py
```

Changes take effect immediately.

## Troubleshooting

### Cold Start Times

First request after inactivity may take 10-30 seconds as Modal spins up the GPU instance. Subsequent requests are fast.

**Solution**: Implement keep-alive pings from backend API (every 5 minutes) to keep instance warm.

### Out of Memory

If using 27B model with very long content:

- Reduce `max_length` parameter
- Use 8-bit quantization (already enabled)
- Split content into chunks

### Model Loading Timeout

If model loading exceeds timeout:

- Increase `timeout` parameter in `@app.function` decorator
- Use Modal's volume caching for faster model loading

## Environment Variables

Set in Modal dashboard or via `modal secret create`:

```bash
# Optional: For Sentry error tracking
SENTRY_DSN=https://xxx@sentry.io/xxx

# Optional: For custom model paths
HUGGINGFACE_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxx
```

## Support

For Modal-specific issues: https://modal.com/docs
For service issues: https://github.com/your-repo/issues