# Pitfalls Research

**Domain:** Document signing app — adding template management to existing system
**Researched:** 2026-04-06
**Confidence:** HIGH (derived from direct codebase analysis + domain knowledge)

---

## Critical Pitfalls

### Pitfall 1: Field ID Collision Between Template Fields and Document Fields

**What goes wrong:**
When a template is applied to create a document, the template's pre-authored `SignatureFieldData` is copied into `documents.signatureFields`. If template fields carry stable/predictable IDs (e.g., sequential strings, hash-of-position, or any ID format that could be reused), two documents created from the same template end up with the same field IDs. Everything works until multi-document views, audit correlation, or `textFillData` lookups appear — then the wrong fill values surface on the wrong document.

**Why it happens:**
Developers author template fields by hand or via an AI-prepare step and assign IDs at template-save time rather than at document-creation time. They assume fields belong to a document and never reconsider that templates are shared parents.

**How to avoid:**
Assign fresh `crypto.randomUUID()` IDs to every field at the moment a document is created from a template, not when the template is saved. The template stores field geometry/type/hint but carries NO IDs. The document creation route (`POST /api/documents`) performs the ID assignment during the copy step.

In the current codebase `documents.signatureFields` is `SignatureFieldData[]` with a required `id: string` on each field (schema.ts line 17). The template table (`formTemplates`) has no `signatureFields` column today. When that column is added, store fields as a type without IDs, and stamp fresh UUIDs on copy.

**Warning signs:**
- `textFillData` (keyed by field ID) returns the wrong prefill values for a document
- Agent edits a field on document A and document B's prepare screen shows the change
- Duplicate `id` values found in `signatureFields` arrays across two documents in the DB

**Phase to address:**
The phase that adds `templateFields` to the `formTemplates` table and the document-creation route that copies them.

---

### Pitfall 2: Template Deletion Breaks Existing Documents' Audit / Reference Integrity

**What goes wrong:**
`documents.formTemplateId` is a nullable FK to `formTemplates.id` (schema.ts line 116). If a template is deleted, all documents that were created from it lose their lineage reference silently — the FK becomes a dangling null. In the current schema there is no `ON DELETE` constraint on `formTemplateId`, so Postgres will reject the delete with a FK violation, not silently nullify it. A naive delete route will throw a 500.

More subtly: even with `ON DELETE SET NULL`, the signed PDF and audit trail remain intact (they reference `documents.id`, not `formTemplateId`), but the agent loses the ability to see which form was used, which matters for compliance and support.

**Why it happens:**
Template management UIs typically have a "Delete" button that calls a simple `DELETE FROM form_templates WHERE id = ?`. No one thinks about the 47 existing documents that were created from that template.

**How to avoid:**
- Never hard-delete templates. Soft-delete only: add `archivedAt timestamp` to `formTemplates`. Archived templates are hidden from the library but remain in the DB.
- If hard-delete is required, block it when `documents` rows reference the template: check `SELECT COUNT(*) FROM documents WHERE form_template_id = ?` before allowing deletion.
- Do NOT use `ON DELETE CASCADE` on `documents.formTemplateId` — that would delete all documents created from a template when the template is removed.

**Warning signs:**
- 500 errors from a template delete route
- Agent reports that document history no longer shows which form was used
- `SELECT * FROM documents WHERE form_template_id IS NULL AND created_at > [template_launch_date]` returns unexpected rows

**Phase to address:**
The phase that adds template CRUD (create, list, archive/delete). Must precede any phase that exposes a "Delete template" button.

---

### Pitfall 3: Template Edit Retroactively Changes Fields on Existing Documents

**What goes wrong:**
If `documents.signatureFields` is stored as a reference to (or derived live from) the template's fields rather than as an independent copy, editing the template — adding a new field, removing a date field, changing a field type — retroactively alters the field layout of every document already created from that template. A document that was prepared and sent with 12 fields suddenly has 14. Signers on active signing sessions see a different document than the one the agent reviewed.

**Why it happens:**
Developers store only `formTemplateId` in the document and resolve fields from the template at runtime to avoid duplication. This seems efficient but breaks document immutability.

**How to avoid:**
Fields must be snapshotted at document creation. `documents.signatureFields` holds the authoritative field list for that document for life. The `formTemplateId` is metadata (lineage), not a live pointer. Today this is already the correct pattern — `signatureFields` lives on the `documents` row — and adding templates must not change it.

Do not add any code path that reads `formTemplates.signatureFields` to resolve a document's fields after document creation.

**Warning signs:**
- A route that does `JOIN form_templates ON documents.form_template_id = form_templates.id` and reads template fields instead of document fields
- A prepare screen that fetches fields from the template endpoint rather than `GET /api/documents/[id]/fields`
- "My client's signing link broke after I updated the template" reports

**Phase to address:**
Template authoring phase. The snapshot rule must be written into the document-creation route and into code review guidelines for all future template-related routes.

---

### Pitfall 4: Agent-Typed Hint Values Stored in textFillData Instead of hint Field

**What goes wrong:**
`SignatureFieldData.hint` is the label shown to a signer for `client-text` / `client-checkbox` fields (schema.ts line 24). `documents.textFillData` is a `Record<string, string>` keyed by field ID that stores agent-typed fill values for `text` fields that are burned into the prepared PDF. If a developer uses `textFillData[fieldId] = "Enter your address here"` to store what is actually a signer hint, the prepared PDF will have the hint text printed as content rather than displayed as a placeholder. The signer sees a field already filled with the instruction text, not an empty field with a hint.

**Why it happens:**
Both `hint` and `textFillData` are string values associated with a field ID. The naming is close enough that a developer authoring a "template defaults" feature stores instructional text in the wrong bucket.

**How to avoid:**
- `textFillData` = agent fills agent-typed values at prepare time; these are burned into the PDF as visible text.
- `hint` on `SignatureFieldData` = a label shown in the signing UI as a placeholder; never written to the PDF.
- Template defaults (pre-seeded agent fill values) go into `textFillData`. Signer instructions go into `SignatureFieldData.hint`.
- When a template stores default field configurations, store `hint` on the field struct, not in a separate `textFillData`-like map on the template.

**Warning signs:**
- The prepared PDF contains placeholder text like "Enter address here" as printed content
- The signer sees a field that appears pre-filled with an instruction rather than empty
- The AI classify step (`classifyFieldsWithAI`) returns `prefillValue` for a field and a developer copies it directly into `hint` instead of `textFillData`

**Phase to address:**
Template field authoring phase AND any phase that adds signer-instruction hints to fields.

---

### Pitfall 5: Signer Role Slots Mixed Up With Actual Signer Emails

**What goes wrong:**
Templates define roles ("Seller 1", "Seller 2", "Agent"). Documents bind actual emails to those roles at prepare time. If role assignment is skipped or the binding is applied to the wrong field, `field.signerEmail` ends up with a role label string ("Seller 1") instead of an email address. The signing flow uses `signerEmail` to route tokens and color-code fields. A role label string passes string validation silently but breaks every `signerEmail` comparison downstream: token routing fails, the wrong signer is asked to sign agent fields, and `getSignerEmail()` (schema.ts line 52) returns the role string as the fallback email.

**Why it happens:**
Template fields need to reference a role without knowing the actual email. Developers use the role label as a placeholder value and forget to replace it during the document-creation step. Or they add a `roleName` property to `SignatureFieldData` but never strip it before saving to the document.

**How to avoid:**
- Templates store `signerRole: string` (not `signerEmail`) on field definitions. This is a separate property, never the same field as `signerEmail`.
- At document creation, a role-to-email map is required and validated. If any role has no email, block document creation — do not fall through to writing a role label into `signerEmail`.
- `SignatureFieldData` in the document must have `signerEmail` as a real email or undefined (legacy fallback). Validate with a regex or `z.string().email()` before DB write.
- The `getSignerEmail(field, fallbackEmail)` helper (schema.ts line 52) is the right pattern for legacy fallback, but is not a safe guard against role-label corruption.

**Warning signs:**
- `field.signerEmail` values contain spaces or capital letters (e.g., "Seller 1")
- Signing tokens are created with a non-email `signerEmail` value
- The multi-signer signing page routes a client to sign agent-owned fields

**Phase to address:**
Template field definition phase AND the document-creation route that binds roles to emails. Role validation must be an explicit step, not an afterthought.

---

### Pitfall 6: Template PDF File Gone But DB Row Persists (or Vice Versa)

**What goes wrong:**
`formTemplates.filename` maps to a file in `seeds/forms/`. If the file is renamed, moved, or deleted on disk while the DB row exists (or the row is inserted with the wrong filename), `copyFile(srcPath, destPath)` in `POST /api/documents` (documents/route.ts line 64) throws `ENOENT`. The agent gets a 500 with no explanation. Because `seeds/forms/` is under version control (per gitStatus), this happens most often during a migration that renames form files.

**Why it happens:**
The DB row and the file are not managed as a unit. A migration script inserts rows, the filename in the row has a typo, or someone renames a PDF without updating the DB.

**How to avoid:**
- On startup (or in a health-check route), validate that every `formTemplates` row's `filename` resolves to an existing file. Log mismatches loudly.
- The `POST /api/documents` route should return a 422 with a meaningful message when `ENOENT` is caught, not fall through to a 500.
- When seeding templates (or adding new forms), use a migration that inserts the DB row and checks file existence atomically in a script, never separate steps.

**Warning signs:**
- Unhandled `ENOENT` errors in the documents creation API
- Agent reports "template not found" but the template appears in the library list
- `formTemplates` rows exist with filenames that differ from actual files in `seeds/forms/`

**Phase to address:**
Template seeding / library management phase.

---

## Technical Debt Patterns

| Shortcut | Immediate Benefit | Long-term Cost | When Acceptable |
|----------|-------------------|----------------|-----------------|
| Store field IDs in template at save time | Skip ID-generation step at doc creation | ID collisions across documents; corrupted textFillData | Never |
| Use `ON DELETE CASCADE` on `formTemplateId` | No orphan rows | Deleting a template deletes all documents created from it | Never |
| Live-resolve fields from template at runtime | Single source of truth for field layout | Template edits retroactively break sent/signed documents | Never |
| Store role labels ("Seller 1") in `signerEmail` | Simpler template field schema | Silent routing failures in signing flow | Never |
| Skip snapshot of textFillData defaults when creating doc from template | Less code at creation time | Agent's fill values come from template, diverge when template is edited | Never acceptable after Sent status |
| Hard-delete templates | Smaller DB | Agent can't trace which form a historical document used | MVP only if template count is tiny and no compliance need |

---

## Integration Gotchas

| Integration | Common Mistake | Correct Approach |
|-------------|----------------|------------------|
| pdfjs-dist + template field storage | Extract blanks from template PDF at save time and store AI-classified fields permanently on the template | Store only the template PDF filename; always re-run `extractBlanks` + `classifyFieldsWithAI` at document creation, or store field geometry without IDs for later stamping |
| OpenAI AI-prepare + templates | Run AI-prepare once, save result to template, assume it applies to all documents | AI-prepare personalizes prefill values (client name, property address) — these must be per-document. Template may store field geometry hints but prefill values must be computed at document creation time |
| Drizzle FK + soft-delete | Add `archivedAt` column but forget to filter it in all `formTemplates` queries | Add a `where(isNull(formTemplates.archivedAt))` default filter or use a query helper that always excludes archived rows |
| `documents.signers` + template roles | Populate `signers` from template role definitions without binding real emails | `signers` must contain real `{ email, color }` pairs; roles are template-side only and must be resolved to emails before the `signers` array is written to the DB |

---

## Performance Traps

| Trap | Symptoms | Prevention | When It Breaks |
|------|----------|------------|----------------|
| AI-prepare on every document created from template | Slow doc creation (1-3s OpenAI call); high API cost | Cache AI-classified field geometry on the template (without IDs or client-specific prefill); run AI-prepare only for field type/geometry, run prefill substitution locally | At ~50+ documents/day created from same template |
| Loading all `signatureFields` JSONB for every document list query | Portal page slow as document count grows | Never SELECT signatureFields in list queries; only fetch when opening a specific document | At ~500+ documents |
| Copying template PDF for every new document (current pattern) | Disk fills with identical PDFs | Store the template PDF once; only copy when the document will be mutated (prepare step). Pre-prepare creates the mutated `_prepared.pdf` from the canonical seed | At ~1000+ documents from same template |

---

## Security Mistakes

| Mistake | Risk | Prevention |
|---------|------|------------|
| Allowing agents to upload arbitrary PDFs as "templates" | Malicious PDF embedded in template library, served to all clients | Keep template PDFs in `seeds/forms/` under version control, not in `uploads/`. Agent-uploaded custom PDFs go to `uploads/clients/` only and are not promoted to templates |
| Template field `hint` containing HTML/script content | XSS in the signing UI if hint is rendered as HTML | Render hint as plain text only (`textContent`, never `dangerouslySetInnerHTML`) |
| Missing auth on template CRUD routes | Any authenticated user (in a multi-agent system) can modify shared templates | Add ownership or role check on template write routes; read-only access is fine for all authenticated agents |
| Path traversal via template `filename` field | Agent-crafted filename reads arbitrary files from server | Validate that `path.join(SEEDS_DIR, template.filename)` starts with `SEEDS_DIR` — same guard pattern already used in `documents/route.ts` line 47 |

---

## UX Pitfalls

| Pitfall | User Impact | Better Approach |
|---------|-------------|-----------------|
| Template library shows archived/deleted templates | Agent adds form from a stale template; document creation fails | Filter `archivedAt IS NULL` in library API; show "archived" badge on template management screen, not in the add-document flow |
| No preview of template fields before creating a document | Agent creates document, runs AI-prepare, finds wrong fields, deletes and recreates | Show a field-count and type summary on the template card in the library; optional: thumbnail of first page |
| Prepare screen pre-populates client data from template defaults instead of the assigned client | Wrong name/address in text fields for documents with multiple clients sharing a template | Prefill values are always derived from the assigned `clientId` at prepare time, never from the template |
| Role slot UI during prepare requires understanding "Seller 1 = email" mapping | Agent assigns wrong signer to wrong role; wrong party signs | Show the client's name (from `clients` table) next to each role slot during preparation, not just the email address |

---

## "Looks Done But Isn't" Checklist

- [ ] **Template field IDs:** Verify field IDs in `documents.signatureFields` are freshly generated at doc creation — not copied from template. Query two documents from same template and confirm no shared field IDs.
- [ ] **Template deletion guard:** Verify DELETE endpoint returns 4xx (not 5xx and not 2xx) when documents reference the template. Test by creating a document from a template, then attempting to delete the template.
- [ ] **Snapshot independence:** Edit a template's fields after creating a document from it. Verify the existing document's `signatureFields` is unchanged.
- [ ] **textFillData vs hint:** Open a prepared PDF for a `client-text` field that has a hint. Confirm the PDF does not contain the hint string as printed text.
- [ ] **Role-to-email binding:** Confirm that after document creation no `signerEmail` value on any field contains a space or non-email string.
- [ ] **ENOENT handling:** Rename a template PDF, attempt to create a document from it, verify a 4xx with a clear message (not a 500).
- [ ] **Archived templates:** Archive a template, confirm it disappears from the library API response but its historical documents remain accessible.

---

## Recovery Strategies

| Pitfall | Recovery Cost | Recovery Steps |
|---------|---------------|----------------|
| Field ID collision discovered post-launch | HIGH | Write a migration that assigns fresh UUIDs to all `signatureFields` on every document; cross-reference with `textFillData` keys and rekey them atomically |
| Template deletion cascaded to documents (if wrongly configured) | CRITICAL | Restore from backup; no in-place recovery possible. Prevention is the only option. |
| Hint text burned into prepared PDFs as content | MEDIUM | Re-run prepare step (only possible if document is still Draft); for Sent/Signed documents, manual intervention required |
| Role label stored in signerEmail | MEDIUM | Write a one-time migration: query documents where any `signatureFields[].signerEmail` does not match email pattern; surface them to the agent for manual correction |
| Template PDF and DB row out of sync | LOW | Add a validation script that walks `formTemplates` rows and checks file existence; run during deploys |

---

## Pitfall-to-Phase Mapping

| Pitfall | Prevention Phase | Verification |
|---------|------------------|--------------|
| Field ID collision | Template fields schema design phase (adding `signatureFields` to `formTemplates`) | Assert no duplicate `id` values across two documents created from same template |
| Template deletion breaking documents | Template CRUD phase | Attempt delete of template with associated documents; expect 409 or soft-delete only |
| Template edit retroactively changing documents | Template authoring phase + document creation route | Edit template fields; confirm existing document `signatureFields` unchanged in DB |
| Hint vs textFillData confusion | Template field authoring phase | Inspect prepared PDF bytes; confirm hint text absent from PDF content stream |
| Signer role vs email confusion | Template field definition + document creation binding phase | Inspect `signatureFields` after doc creation; confirm all `signerEmail` values pass email regex |
| Template file/DB desync | Template seeding / library management phase | Remove a seed file; confirm creation attempt returns 422 not 500 |

---

## Sources

- Direct codebase analysis: `src/lib/db/schema.ts`, `src/app/api/documents/route.ts`, `src/app/api/documents/[id]/prepare/route.ts`, `src/app/api/documents/[id]/ai-prepare/route.ts`, `src/lib/ai/field-placement.ts`, `src/lib/ai/extract-text.ts`
- Schema observation: `documents.formTemplateId` has no `ON DELETE` action specified (schema.ts line 116) — Postgres defaults to `RESTRICT`
- `SignatureFieldData` interface (schema.ts lines 15-25) — `hint`, `signerEmail`, `type` fields and their optionality
- `textFillData: Record<string, string>` (schema.ts line 119) — keyed by field ID
- Milestone context: template field IDs, deletion safety, hint vs fill, role vs email

---
*Pitfalls research for: document signing app — template management addition*
*Researched: 2026-04-06*