red/.planning/phases/05-pdf-fill-and-field-mapping/05-01-SUMMARY.md

---
phase: 05-pdf-fill-and-field-mapping
plan: 01
subsystem: api, database, pdf
tags: [drizzle, postgresql, jsonb, pdf-lib, jest, typescript, next.js]

# Dependency graph
requires:
  - phase: 04-pdf-ingest
    provides: documents table with filePath, file upload API, uploads/ storage pattern

provides:
  - documents table with 4 new nullable JSONB/text columns (signatureFields, textFillData, assignedClientId, preparedFilePath)
  - SignatureFieldData TypeScript interface exported from schema.ts
  - GET/PUT /api/documents/[id]/fields for signature field coordinate storage
  - POST /api/documents/[id]/prepare that mutates PDF and transitions status to Sent
  - preparePdf utility with atomic tmp->rename write and AcroForm text fill
  - Migration 0003_cool_natasha_romanoff.sql applied to database
  - Unit test suite for Y-flip coordinate conversion formula (10 tests passing)

affects: [05-pdf-fill-and-field-mapping, field-placer-ui, text-fill-ui, signing-flow]

# Tech tracking
tech-stack:
  added:
    - "@cantoo/pdf-lib ^2.6.3 — server-side PDF mutation (NOT pdf-lib — they conflict)"
    - "jest ^29.7.0 — unit test runner"
    - "ts-jest ^29.4.6 — TypeScript preprocessor for jest"
    - "@types/jest ^30.0.0 — jest type definitions"
  patterns:
    - "Atomic PDF write: writeFile to .tmp path, verify %PDF magic bytes, rename to final — prevents partial writes"
    - "form.flatten() called BEFORE drawing signature rectangles to avoid overlay conflicts"
    - "params is a Promise in Next.js 16 — always await before destructuring"
    - "UPLOADS_DIR path traversal guard on all file path operations"

key-files:
  created:
    - teressa-copeland-homes/src/lib/pdf/prepare-document.ts
    - teressa-copeland-homes/src/app/api/documents/[id]/fields/route.ts
    - teressa-copeland-homes/src/app/api/documents/[id]/prepare/route.ts
    - teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts
    - teressa-copeland-homes/drizzle/0003_cool_natasha_romanoff.sql
  modified:
    - teressa-copeland-homes/src/lib/db/schema.ts

key-decisions:
  - "@cantoo/pdf-lib used instead of pdf-lib — packages conflict; @cantoo is the maintained fork"
  - "signatureFields and textFillData stored as JSONB in documents table — flexible schema for field arrays and key/value maps"
  - "Atomic write (tmp → rename) for prepared PDFs — prevents corrupting the source PDF on failure"
  - "form.flatten() before rectangle drawing — required order to avoid AcroForm overlay conflicts"
  - "jest + ts-jest chosen over vitest — straightforward TypeScript test support without ESM complications"
  - "preparedRelPath uses .replace(/\\.pdf$/, '_prepared.pdf') — keeps prepared variant next to source with clear naming"

patterns-established:
  - "PDF atomic write: write to {destPath}.tmp, verify %PDF header, rename to {destPath}"
  - "Y-flip formula: pdfY = ((renderedH - screenY) / renderedH) * originalHeight — scale-invariant"
  - "API auth guard: const session = await auth(); if (!session) return new Response('Unauthorized', { status: 401 })"

requirements-completed: [DOC-04, DOC-05, DOC-06]

# Metrics
duration: 20min
completed: 2026-03-19
---

# Phase 5 Plan 01: PDF Field Mapping Foundation Summary

**Drizzle migration 0003 adds 4 JSONB/text columns to documents, @cantoo/pdf-lib preparePdf utility with atomic write, GET/PUT fields API, POST prepare API transitioning to Sent status, and 10-test Y-flip coordinate suite all passing**

## Performance

- **Duration:** ~20 min
- **Started:** 2026-03-19T00:00:00Z
- **Completed:** 2026-03-19
- **Tasks:** 3 of 3
- **Files modified:** 6

## Accomplishments

- Extended documents table with signatureFields (JSONB), textFillData (JSONB), assignedClientId (text), preparedFilePath (text) nullable columns and applied migration 0003 to the local PostgreSQL database
- Created preparePdf server utility using @cantoo/pdf-lib with AcroForm text fill, signature rectangle drawing, and atomic tmp-to-rename write pattern
- Created GET/PUT /api/documents/[id]/fields and POST /api/documents/[id]/prepare API routes with 401 auth guards, 422 validation, and 403 path traversal protection
- Added 10-test Jest suite verifying Y-flip coordinate conversion formula at both 1:1 scale and 50% zoom

## Task Commits

Each task was committed atomically:

1. **Task 1: Extend schema + generate and apply migration 0003** - `d67130d` (feat)
2. **Task 2: Create pdf prepare-document utility + API routes for fields and prepare** - `c81e8ea` (feat)
3. **Task 3: Write unit tests for Y-flip coordinate conversion formula** - `34ed0ba` (test)

**Plan metadata:** TBD (docs: complete plan)

## Files Created/Modified

- `teressa-copeland-homes/src/lib/db/schema.ts` - Added SignatureFieldData interface and 4 new nullable columns to documents table, added jsonb import
- `teressa-copeland-homes/drizzle/0003_cool_natasha_romanoff.sql` - Migration adding 4 columns (ALTER TABLE ... ADD COLUMN x4)
- `teressa-copeland-homes/src/lib/pdf/prepare-document.ts` - preparePdf async function: loads PDF, fills AcroForm, flattens, draws signature rectangles, atomic write
- `teressa-copeland-homes/src/app/api/documents/[id]/fields/route.ts` - GET returns signatureFields array, PUT stores new array and returns it
- `teressa-copeland-homes/src/app/api/documents/[id]/prepare/route.ts` - POST calls preparePdf, updates preparedFilePath/textFillData/assignedClientId/status/sentAt
- `teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts` - 10 tests for screenToPdfY and screenToPdfX formulas

## Decisions Made

- Used @cantoo/pdf-lib (NOT pdf-lib) — maintained fork, packages conflict if both installed
- JSONB columns for signatureFields and textFillData — flexible schema supports arbitrary field arrays without schema changes
- Atomic write pattern (tmp → rename) for preparePdf — protects against partial writes corrupting the prepared PDF
- form.flatten() before drawing rectangles — if reversed, AcroForm overlay can obscure the drawn rectangles
- jest + ts-jest for unit tests — no ESM configuration needed for pure TypeScript utility functions
- Jest config placed in package.json jest field (not jest.config.ts) — simpler setup for single test suite

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 3 - Blocking] Fixed pre-existing TypeScript error in scripts/debug-inspect2.ts**
- **Found during:** Task 2 (build verification)
- **Issue:** `el.textContent()` returns `string | null`; concatenation at line 22 failed strict TypeScript check. This file was included in tsconfig `**/*.ts` glob and caused `npm run build` to fail.
- **Fix:** Changed `text + href` to `(text ?? '') + (href ?? '')` — null coalescing guards both values
- **Files modified:** teressa-copeland-homes/scripts/debug-inspect2.ts
- **Verification:** `npm run build` completed successfully after fix; all new routes visible in build output
- **Committed in:** `c81e8ea` (Task 2 commit)

---

**Total deviations:** 1 auto-fixed (1 blocking pre-existing bug)
**Impact on plan:** Pre-existing bug in unrelated debug script blocked build verification. Trivial null-coalescing fix restored build. No scope creep.

## Issues Encountered

- `npm run db:migrate` requires DATABASE_URL env var which was in `.env.local` not `.env`. Ran migration with explicit `DATABASE_URL=postgresql://postgres:postgres@localhost:5432/teressa` env override to apply successfully.

## User Setup Required

None — no external service configuration required. Database migration applied automatically.

## Next Phase Readiness

- Plan 02 (field-placer UI) can now import `SignatureFieldData` from `@/lib/db/schema` and consume `GET/PUT /api/documents/[id]/fields`
- Plan 03 (text-fill UI) can use `POST /api/documents/[id]/prepare` with `textFillData` payload
- All three API routes return 401 unauthenticated (verified by build output showing routes as dynamic server-rendered)
- `preparePdf` utility is ready for integration testing once an actual PDF file exists in uploads/

---
*Phase: 05-pdf-fill-and-field-mapping*
*Completed: 2026-03-19*