red/teressa-copeland-homes/.planning/research/ARCHITECTURE.md

# Architecture Patterns: Document Templates

**Domain:** Real-estate agent portal — PDF signing with field placement
**Researched:** 2026-04-06
**Confidence:** HIGH (based on direct code inspection)

---

## Context: What Was Inspected

| File | Purpose |
|------|---------|
| `src/lib/db/schema.ts` | Full Drizzle schema — all tables, types, helpers |
| `src/app/api/documents/route.ts` | Document creation (two paths: library copy + custom upload) |
| `src/app/api/documents/[id]/fields/route.ts` | GET/PUT for `signatureFields` JSONB |
| `src/app/api/forms-library/route.ts` | Lists `form_templates` rows |
| `documents/[docId]/_components/DocumentPageClient.tsx` | State owner for document editing page |
| `documents/[docId]/_components/FieldPlacer.tsx` | Drag-drop field placement (822 lines) |
| `documents/[docId]/_components/PreparePanel.tsx` | Prepare + send side panel |
| `portal/_components/AddDocumentModal.tsx` | "Add Document" modal — uses form_templates |
| `portal/_components/ClientProfileClient.tsx` | Client profile page with document list |
| `drizzle/` | 12 migration files (0000–0011) |

---

## Current Architecture

### Schema Summary

```
form_templates          (id, name, filename, createdAt, updatedAt)
  — raw PDF registry only, no field data

documents               (id, name, clientId, status, formTemplateId FK,
                         filePath, signatureFields JSONB, textFillData JSONB,
                         signers JSONB, preparedFilePath, signedFilePath,
                         emailAddresses, pdfHash, sentAt, signedAt,
                         completionTriggeredAt, assignedClientId)
  — per-client working copy, carries all field placement state

clients                 (id, name, email, contacts JSONB, propertyAddress,
                         createdAt, updatedAt)

signing_tokens          (jti, documentId FK, signerEmail, expiresAt, usedAt)
audit_events            (id, documentId FK, eventType, metadata JSONB, ...)
users                   (id, email, passwordHash, agentSignatureData,
                         agentInitialsData)
```

### Document Creation Flow (current)

```
AddDocumentModal
  → POST /api/documents
      ├── (library path)  formTemplateId provided
      │     → copy seed PDF to uploads/clients/{clientId}/{docId}.pdf
      │     → INSERT documents (no signatureFields — empty, to be placed)
      └── (upload path)   custom file
            → write file to uploads/clients/{clientId}/{docId}.pdf
            → INSERT documents (no signatureFields)
```

After creation, agent opens `/portal/documents/{docId}`, uses FieldPlacer to place fields
(drag-drop or AI auto-place), then uses PreparePanel to fill text values and send.

### FieldPlacer Integration Point

`FieldPlacer` is tightly coupled to `docId`:

- Fetches fields from `GET /api/documents/{docId}/fields`
- Persists changes to `PUT /api/documents/{docId}/fields`
- The `aiPlacementKey` prop triggers a re-fetch after AI auto-place

The component has **no concept of a "template source"** — it only reads/writes document fields.
That is the integration seam where template pre-loading must be injected.

---

## Schema Decision: New Table vs Extend `form_templates`

**Decision: New `document_templates` table. Do not extend `form_templates`.**

### Reasoning

`form_templates` is currently a **file registry** — a pointer to a PDF filename on disk.
Every part of the system treats it that way:

- `forms-library/route.ts` returns `{ id, name, filename }` only
- `documents/route.ts` uses it only to resolve `template.filename` for `copyFile()`
- `AddDocumentModal` renders it as a name list for file selection
- `documents.formTemplateId` is a FK used only for audit/traceability

Adding `signatureFields` + `signers` JSONB to `form_templates` would:
1. Break the clear single-responsibility of that table
2. Force every consumer that reads `form_templates` to handle nullable field data
3. Make the "template has fields" state conflated with "template is just a PDF"
4. Create confusion: which `form_templates` rows have fields? All? Some?

A separate `document_templates` table is clean:
- Represents an intentionally-prepared template (PDF + fields + signer role slots)
- Can be soft-deleted, versioned, or duplicated without touching `form_templates`
- `form_templates` continues working exactly as it does today
- The new table has an optional FK to `form_templates` for traceability

### Proposed Schema Addition

```typescript
// New interface — signer role slot (no email yet, filled at document creation)
export interface TemplateSignerSlot {
  role: string;    // e.g. "Buyer", "Seller", "Agent"
  color: string;   // display color, matches FieldPlacer SIGNER_COLORS
}

// SignatureFieldData already has signerEmail — for templates, signerEmail
// holds the ROLE string (e.g. "Seller") rather than an actual email.
// This is a deliberate dual-use: roles are resolved to emails at apply time.

export const documentTemplates = pgTable("document_templates", {
  id: text("id").primaryKey().$defaultFn(() => crypto.randomUUID()),
  name: text("name").notNull(),
  // Optional link back to the form_templates raw PDF
  formTemplateId: text("form_template_id").references(() => formTemplates.id),
  // Stored field layout — same shape as documents.signatureFields
  // signerEmail values are ROLE strings at this stage
  signatureFields: jsonb("signature_fields").$type<SignatureFieldData[]>(),
  // Signer role slots — defines which roles exist and their colors
  signerSlots: jsonb("signer_slots").$type<TemplateSignerSlot[]>(),
  createdAt: timestamp("created_at").defaultNow().notNull(),
  updatedAt: timestamp("updated_at").defaultNow().notNull(),
});
```

**Backwards compatibility:** Zero impact on existing documents. `form_templates` is unchanged.
All existing documents with `signatureFields: null` continue working (FieldPlacer handles empty arrays).

---

## Component Boundaries

### New Components

| Component | Location | Responsibility |
|-----------|----------|---------------|
| `TemplatesPageClient` | `portal/_components/` | List templates, launch editor, delete |
| `TemplateEditorPage` (server) | `portal/(protected)/templates/[templateId]/page.tsx` | Loads template data, renders editor shell |
| `TemplateEditorClient` | `portal/(protected)/templates/[templateId]/_components/` | Thin wrapper — composes FieldPlacer + TemplatePanel |
| `TemplatePanel` | same directory | Sidebar: signer slot management, save, AI auto-place |
| `TemplatePicker` | `portal/_components/` | Dropdown/modal used in AddDocumentModal to choose a document_template |

### Modified Components

| Component | Change |
|-----------|--------|
| `AddDocumentModal` | Add "Start from template" option that calls a new `POST /api/templates/{id}/apply` endpoint |
| `FieldPlacer` | Add optional `templateId` prop (or make `docId` accept either); add `onSaveAsTemplate` callback. Alternatively: keep FieldPlacer document-only and build a thin TemplateFieldPlacer wrapper that remaps the API calls. |

**Recommendation on FieldPlacer:** Keep FieldPlacer document-only. Build `TemplateFieldPlacer` as a thin wrapper that intercepts the `persistFields` call (currently hardcoded to `/api/documents/{docId}/fields`) by accepting an `onPersist` callback prop, or by adding a `persistUrl` override prop. This is a 5-line change to FieldPlacer and avoids branching logic inside the 822-line component.

---

## New API Routes

| Route | Method | Purpose |
|-------|--------|---------|
| `GET /api/templates` | GET | List all document_templates |
| `POST /api/templates` | POST | Create new document_template (blank or from form_template) |
| `GET /api/templates/[id]` | GET | Fetch single template (with signatureFields) |
| `PATCH /api/templates/[id]` | PATCH | Save field layout + signer slots |
| `DELETE /api/templates/[id]` | DELETE | Soft or hard delete |
| `POST /api/templates/[id]/apply` | POST | Apply template to a new document (see flow below) |

---

## "Apply Template to Document" Operation

This is the core integration between templates and the existing creation flow.

```
POST /api/templates/{templateId}/apply
Body: { clientId, name, signerRoleMap: { "Seller": "email@client.com", ... } }

Server:
1. Fetch document_template row (fields + signerSlots)
2. Fetch form_templates row for the PDF filename (via template.formTemplateId)
3. Copy PDF file to uploads/clients/{clientId}/{docId}.pdf
4. Deep-clone signatureFields array
5. For each field: replace signerEmail (role string) with actual email from signerRoleMap
6. INSERT documents with: clientId, name, formTemplateId, filePath, signatureFields (resolved),
   signers (built from signerRoleMap + slot colors)
7. Return created document

Client (AddDocumentModal):
- Shows template picker (list from GET /api/templates)
- Shows role-to-email assignment UI (one row per signerSlot, pre-populated from client.contacts)
- On submit: calls POST /api/templates/{id}/apply
- On 201: router.refresh() + onClose() — same pattern as current flow
```

The document created this way is indistinguishable from a hand-placed document.
The agent can still edit fields after creation — FieldPlacer works identically.

---

## Template Editor Architecture

```
/portal/(protected)/templates/[templateId]/page.tsx   (Server Component)
  → fetches document_template row
  → renders TemplateEditorClient

TemplateEditorClient
  → state: fields (from template.signatureFields), signerSlots, dirty flag
  → layout: same 2-column grid as DocumentPageClient (PDF left, panel right)

  Left: TemplateFieldPlacer
    → wraps FieldPlacer with persistUrl="/api/templates/{templateId}/fields"
    → OR: FieldPlacer gets onPersist callback prop (cleaner)

  Right: TemplatePanel
    → Signer slot list (add/remove roles, assign colors)
    → Save button (PATCH /api/templates/{templateId})
    → AI Auto-Place button (reuses existing /api/documents/{docId}/ai-prepare pattern,
       needs a parallel /api/templates/{id}/ai-prepare endpoint)
```

**AI auto-place for templates:** The existing `ai-prepare` route writes to `documents.signatureFields`. A parallel `POST /api/templates/{id}/ai-prepare` writes to `document_templates.signatureFields` instead. The extraction + classification logic in `src/lib/ai/` is document-agnostic and can be called from either route without modification.

---

## Data Flow Diagram

```
form_templates (PDF registry)
    │
    │ formTemplateId FK (optional)
    ▼
document_templates (field layout + signer slots)
    │
    │ POST /api/templates/{id}/apply
    │ → resolves role → email
    ▼
documents (working copy, per client)
    │
    ├── signatureFields JSONB  ← FieldPlacer reads/writes
    ├── signers JSONB          ← PreparePanel reads/writes
    └── textFillData JSONB     ← PreparePanel reads/writes
```

---

## Scalability Considerations

| Concern | Now (one agent) | Future (multi-agent SaaS) |
|---------|-----------------|--------------------------|
| Template ownership | All templates belong to one agent | Add `userId` FK to `document_templates` |
| Template sharing | N/A | Separate `shared_templates` or `userId: null` convention |
| Template versioning | Not needed | Add `version` int or duplicate-on-edit pattern |
| PDF storage | Local filesystem (uploads/) | Already abstracted in route — swap `copyFile` for S3 copy |

---

## Build Order

Dependencies flow downward — each step enables the next.

### Step 1: Schema migration
- Add `document_templates` table to `schema.ts`
- Add `TemplateSignerSlot` interface
- Write drizzle migration (`0012_document_templates.sql`)
- **No changes to existing tables. Safe to deploy alongside live system.**

### Step 2: Template CRUD API
- `GET /api/templates` — list
- `POST /api/templates` — create (blank shell, optionally linked to form_template)
- `GET /api/templates/[id]` — fetch
- `PATCH /api/templates/[id]` — save fields + signerSlots
- `DELETE /api/templates/[id]` — delete
- These routes follow the exact same auth + db pattern as `forms-library/route.ts` and `documents/[id]/route.ts`

### Step 3: FieldPlacer persistence abstraction
- Add `onPersist?: (fields: SignatureFieldData[]) => Promise<void>` prop to FieldPlacer
- When prop is provided, use it instead of the hardcoded `persistFields(docId, ...)` call
- Fallback: keep existing behavior when prop is absent
- **This is a non-breaking change — all existing consumers pass no prop, behavior unchanged**

### Step 4: Template editor UI
- `TemplateEditorClient` + `TemplatePanel` components
- `/portal/(protected)/templates/[templateId]/page.tsx` server page
- `/portal/(protected)/templates/page.tsx` list page
- Portal nav link ("Templates")

### Step 5: Apply template API + AddDocumentModal update
- `POST /api/templates/[id]/apply` — the apply operation (step 5 depends on step 1+2)
- Update `AddDocumentModal` to add "Start from template" path
- Role-to-email mapping UI (pre-seeded from `client.contacts`)

### Step 6: AI auto-place for templates (optional, can defer)
- `POST /api/templates/[id]/ai-prepare` — mirrors existing route, writes to `document_templates`
- Reuse `extractBlanks()` and `classifyFieldsWithAI()` from `src/lib/ai/` unchanged

---

## Anti-Patterns to Avoid

### Anti-Pattern 1: Extending `form_templates` with field data
**What goes wrong:** `form_templates` becomes overloaded. Some rows have fields, some don't. Every consumer must handle both states. The forms-library UI must differentiate "raw PDF" from "template with fields".
**Instead:** New `document_templates` table with an optional FK back to `form_templates`.

### Anti-Pattern 2: FieldPlacer knows about templates
**What goes wrong:** 822-line component gets template-vs-document branching throughout. Hard to test, easy to regress.
**Instead:** FieldPlacer gets a single `onPersist` callback. The caller decides what to do with the fields.

### Anti-Pattern 3: Storing actual signer emails in template fields
**What goes wrong:** Templates must be edited when a client's email changes. Template becomes client-specific, defeating the purpose.
**Instead:** Store role strings ("Seller", "Buyer") in `signerEmail`. Resolve to actual emails at apply time.

### Anti-Pattern 4: Applying template by mutating the template's PDF
**What goes wrong:** Templates get corrupted or have prepared PDF artifacts baked in.
**Instead:** Apply = copy raw PDF + clone + resolve fields. Template's PDF path is never touched.

### Anti-Pattern 5: Building a new field placer from scratch for templates
**What goes wrong:** Two diverging implementations of the same complex drag-drop logic.
**Instead:** Reuse FieldPlacer via the `onPersist` prop abstraction in Step 3.

---

## Sources

- Direct code inspection of all listed files (HIGH confidence — current state)
- Schema introspection of all 12 drizzle migrations
- No external sources needed — this is pure architecture derivation from existing patterns