diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index a4a45e2..f0114bc 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -30,9 +30,9 @@ - [x] **DOC-01**: Agent can browse and import PDF forms from the utahrealestate.com vendor API (vendor.utahrealestate.com/webapi) — investigate API capability; fall back to manual upload if forms API is not available - [x] **DOC-02**: Forms library syncs automatically on at least a monthly basis to reflect new/updated forms - [x] **DOC-03**: Agent can view an imported PDF document in the browser -- [ ] **DOC-04**: Agent can drag-and-drop to place signature fields on any page of a PDF document -- [ ] **DOC-05**: Agent can fill in text fields on a document (property address, client names, dates, prices) before sending -- [ ] **DOC-06**: Agent can assign a document to a specific client and send a signing request +- [x] **DOC-04**: Agent can drag-and-drop to place signature fields on any page of a PDF document +- [x] **DOC-05**: Agent can fill in text fields on a document (property address, client names, dates, prices) before sending +- [x] **DOC-06**: Agent can assign a document to a specific client and send a signing request ### Dashboard @@ -122,9 +122,9 @@ Which phases cover which requirements. Updated during roadmap creation. | DOC-01 | Phase 4 | Complete | | DOC-02 | Phase 4 | Complete | | DOC-03 | Phase 4 | Complete | -| DOC-04 | Phase 5 | Pending | -| DOC-05 | Phase 5 | Pending | -| DOC-06 | Phase 5 | Pending | +| DOC-04 | Phase 5 | Complete | +| DOC-05 | Phase 5 | Complete | +| DOC-06 | Phase 5 | Complete | | SIGN-01 | Phase 6 | Pending | | SIGN-02 | Phase 6 | Pending | | SIGN-03 | Phase 6 | Pending | diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index b587325..12c3d93 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -101,7 +101,7 @@ Plans: **Plans**: 4 plans Plans: -- [ ] 05-01-PLAN.md — Schema extension (signatureFields, textFillData, assignedClientId, preparedFilePath JSONB columns), migration 0003, @cantoo/pdf-lib prepare utility, GET/PUT /api/documents/[id]/fields, POST /api/documents/[id]/prepare +- [x] 05-01-PLAN.md — Schema extension (signatureFields, textFillData, assignedClientId, preparedFilePath JSONB columns), migration 0003, @cantoo/pdf-lib prepare utility, GET/PUT /api/documents/[id]/fields, POST /api/documents/[id]/prepare - [ ] 05-02-PLAN.md — dnd-kit FieldPlacer component: drag-and-drop signature field placement, Y-axis coordinate conversion, blue rectangle overlay, field persistence - [ ] 05-03-PLAN.md — TextFillForm + PreparePanel: key-value text fill form, client selector dropdown, Prepare and Send workflow - [ ] 05-04-PLAN.md — Full Phase 5 human verification checkpoint @@ -146,6 +146,6 @@ Phases execute in numeric order: 1 → 2 → 3 → 4 → 5 → 6 → 7 | 2. Marketing Site | 2/3 | In Progress| | | 3. Agent Portal Shell | 4/4 | Complete | 2026-03-19 | | 4. PDF Ingest | 4/4 | Complete | 2026-03-20 | -| 5. PDF Fill and Field Mapping | 0/4 | Not started | - | +| 5. PDF Fill and Field Mapping | 1/4 | In Progress | - | | 6. Signing Flow | 0/? | Not started | - | | 7. Audit Trail and Download | 0/? | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index 8243f18..39f54c7 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -18,14 +18,14 @@ progress: See: .planning/PROJECT.md (updated 2026-03-19) **Core value:** Teressa can prepare and send any real estate form to a client for signing in minutes, from her browser, without leaving her site. -**Current focus:** Phase 4 - PDF Ingest +**Current focus:** Phase 5 - PDF Fill and Field Mapping ## Current Position -Phase: 4 of 7 (PDF Ingest) — COMPLETE -Plan: 04-04 complete (4 of 4 plans in phase complete) -Status: Plan 04-04 complete — Human verification approved; Phase 4 PDF ingest declared complete -Last activity: 2026-03-20 — Plan 04-04 complete: Teressa confirmed all 7 browser verification steps pass; forms library, modal, PDF render, file storage, and auth guards verified +Phase: 5 of 7 (PDF Fill and Field Mapping) — IN PROGRESS +Plan: 05-01 complete (1 of 3 plans in phase complete) +Status: Plan 05-01 complete — DB migration 0003 applied, preparePdf utility, GET/PUT fields and POST prepare API routes, 10 Y-flip tests passing +Last activity: 2026-03-19 — Plan 05-01 complete: schema extended, @cantoo/pdf-lib installed, API routes live, Jest test suite passing Progress: [██████████] 100% @@ -101,6 +101,12 @@ Recent decisions affecting current work: - [Phase 04-pdf-ingest]: pdfjs worker uses new URL(import.meta.url) pattern — no CDN URL; works in local/Docker environments without internet access - [Phase 04-pdf-ingest]: documentsRelations added to schema.ts — required for Drizzle db.query relational API with-relations support - [Phase 04-pdf-ingest 04-04]: Phase 4 declared complete after human agent verified all 7 browser verification steps pass — forms library, modal, PDF render, file storage, and auth guards confirmed working +- [Phase 05-pdf-fill-and-field-mapping 05-01]: @cantoo/pdf-lib used instead of pdf-lib — packages conflict; @cantoo is the maintained fork with same API +- [Phase 05-pdf-fill-and-field-mapping 05-01]: signatureFields and textFillData stored as JSONB in documents table — flexible schema for field arrays and key/value maps without additional tables +- [Phase 05-pdf-fill-and-field-mapping 05-01]: Atomic write (tmp → rename) for prepared PDFs — prevents corrupting source PDF on partial write failure +- [Phase 05-pdf-fill-and-field-mapping 05-01]: form.flatten() called BEFORE drawing signature rectangles — required order; if reversed, AcroForm overlay obscures drawn rectangles +- [Phase 05-pdf-fill-and-field-mapping 05-01]: jest + ts-jest chosen for unit tests — straightforward TypeScript test support without ESM complications for coordinate formula tests +- [Phase 05-pdf-fill-and-field-mapping 05-01]: Y-flip formula pdfY = ((renderedH - screenY) / renderedH) * originalHeight is scale-invariant — verified at 1:1 and 50% zoom in test suite ### Pending Todos @@ -116,6 +122,6 @@ None yet. ## Session Continuity -Last session: 2026-03-20 -Stopped at: Completed 04-04-PLAN.md — Human verification approved; Phase 4 PDF ingest complete +Last session: 2026-03-19 +Stopped at: Completed 05-01-PLAN.md — DB migration 0003, preparePdf utility, fields/prepare API routes, 10 Jest tests passing Resume file: None diff --git a/.planning/phases/05-pdf-fill-and-field-mapping/05-01-SUMMARY.md b/.planning/phases/05-pdf-fill-and-field-mapping/05-01-SUMMARY.md new file mode 100644 index 0000000..68c6150 --- /dev/null +++ b/.planning/phases/05-pdf-fill-and-field-mapping/05-01-SUMMARY.md @@ -0,0 +1,147 @@ +--- +phase: 05-pdf-fill-and-field-mapping +plan: 01 +subsystem: api, database, pdf +tags: [drizzle, postgresql, jsonb, pdf-lib, jest, typescript, next.js] + +# Dependency graph +requires: + - phase: 04-pdf-ingest + provides: documents table with filePath, file upload API, uploads/ storage pattern + +provides: + - documents table with 4 new nullable JSONB/text columns (signatureFields, textFillData, assignedClientId, preparedFilePath) + - SignatureFieldData TypeScript interface exported from schema.ts + - GET/PUT /api/documents/[id]/fields for signature field coordinate storage + - POST /api/documents/[id]/prepare that mutates PDF and transitions status to Sent + - preparePdf utility with atomic tmp->rename write and AcroForm text fill + - Migration 0003_cool_natasha_romanoff.sql applied to database + - Unit test suite for Y-flip coordinate conversion formula (10 tests passing) + +affects: [05-pdf-fill-and-field-mapping, field-placer-ui, text-fill-ui, signing-flow] + +# Tech tracking +tech-stack: + added: + - "@cantoo/pdf-lib ^2.6.3 — server-side PDF mutation (NOT pdf-lib — they conflict)" + - "jest ^29.7.0 — unit test runner" + - "ts-jest ^29.4.6 — TypeScript preprocessor for jest" + - "@types/jest ^30.0.0 — jest type definitions" + patterns: + - "Atomic PDF write: writeFile to .tmp path, verify %PDF magic bytes, rename to final — prevents partial writes" + - "form.flatten() called BEFORE drawing signature rectangles to avoid overlay conflicts" + - "params is a Promise in Next.js 16 — always await before destructuring" + - "UPLOADS_DIR path traversal guard on all file path operations" + +key-files: + created: + - teressa-copeland-homes/src/lib/pdf/prepare-document.ts + - teressa-copeland-homes/src/app/api/documents/[id]/fields/route.ts + - teressa-copeland-homes/src/app/api/documents/[id]/prepare/route.ts + - teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts + - teressa-copeland-homes/drizzle/0003_cool_natasha_romanoff.sql + modified: + - teressa-copeland-homes/src/lib/db/schema.ts + +key-decisions: + - "@cantoo/pdf-lib used instead of pdf-lib — packages conflict; @cantoo is the maintained fork" + - "signatureFields and textFillData stored as JSONB in documents table — flexible schema for field arrays and key/value maps" + - "Atomic write (tmp → rename) for prepared PDFs — prevents corrupting the source PDF on failure" + - "form.flatten() before rectangle drawing — required order to avoid AcroForm overlay conflicts" + - "jest + ts-jest chosen over vitest — straightforward TypeScript test support without ESM complications" + - "preparedRelPath uses .replace(/\\.pdf$/, '_prepared.pdf') — keeps prepared variant next to source with clear naming" + +patterns-established: + - "PDF atomic write: write to {destPath}.tmp, verify %PDF header, rename to {destPath}" + - "Y-flip formula: pdfY = ((renderedH - screenY) / renderedH) * originalHeight — scale-invariant" + - "API auth guard: const session = await auth(); if (!session) return new Response('Unauthorized', { status: 401 })" + +requirements-completed: [DOC-04, DOC-05, DOC-06] + +# Metrics +duration: 20min +completed: 2026-03-19 +--- + +# Phase 5 Plan 01: PDF Field Mapping Foundation Summary + +**Drizzle migration 0003 adds 4 JSONB/text columns to documents, @cantoo/pdf-lib preparePdf utility with atomic write, GET/PUT fields API, POST prepare API transitioning to Sent status, and 10-test Y-flip coordinate suite all passing** + +## Performance + +- **Duration:** ~20 min +- **Started:** 2026-03-19T00:00:00Z +- **Completed:** 2026-03-19 +- **Tasks:** 3 of 3 +- **Files modified:** 6 + +## Accomplishments + +- Extended documents table with signatureFields (JSONB), textFillData (JSONB), assignedClientId (text), preparedFilePath (text) nullable columns and applied migration 0003 to the local PostgreSQL database +- Created preparePdf server utility using @cantoo/pdf-lib with AcroForm text fill, signature rectangle drawing, and atomic tmp-to-rename write pattern +- Created GET/PUT /api/documents/[id]/fields and POST /api/documents/[id]/prepare API routes with 401 auth guards, 422 validation, and 403 path traversal protection +- Added 10-test Jest suite verifying Y-flip coordinate conversion formula at both 1:1 scale and 50% zoom + +## Task Commits + +Each task was committed atomically: + +1. **Task 1: Extend schema + generate and apply migration 0003** - `d67130d` (feat) +2. **Task 2: Create pdf prepare-document utility + API routes for fields and prepare** - `c81e8ea` (feat) +3. **Task 3: Write unit tests for Y-flip coordinate conversion formula** - `34ed0ba` (test) + +**Plan metadata:** TBD (docs: complete plan) + +## Files Created/Modified + +- `teressa-copeland-homes/src/lib/db/schema.ts` - Added SignatureFieldData interface and 4 new nullable columns to documents table, added jsonb import +- `teressa-copeland-homes/drizzle/0003_cool_natasha_romanoff.sql` - Migration adding 4 columns (ALTER TABLE ... ADD COLUMN x4) +- `teressa-copeland-homes/src/lib/pdf/prepare-document.ts` - preparePdf async function: loads PDF, fills AcroForm, flattens, draws signature rectangles, atomic write +- `teressa-copeland-homes/src/app/api/documents/[id]/fields/route.ts` - GET returns signatureFields array, PUT stores new array and returns it +- `teressa-copeland-homes/src/app/api/documents/[id]/prepare/route.ts` - POST calls preparePdf, updates preparedFilePath/textFillData/assignedClientId/status/sentAt +- `teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts` - 10 tests for screenToPdfY and screenToPdfX formulas + +## Decisions Made + +- Used @cantoo/pdf-lib (NOT pdf-lib) — maintained fork, packages conflict if both installed +- JSONB columns for signatureFields and textFillData — flexible schema supports arbitrary field arrays without schema changes +- Atomic write pattern (tmp → rename) for preparePdf — protects against partial writes corrupting the prepared PDF +- form.flatten() before drawing rectangles — if reversed, AcroForm overlay can obscure the drawn rectangles +- jest + ts-jest for unit tests — no ESM configuration needed for pure TypeScript utility functions +- Jest config placed in package.json jest field (not jest.config.ts) — simpler setup for single test suite + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 3 - Blocking] Fixed pre-existing TypeScript error in scripts/debug-inspect2.ts** +- **Found during:** Task 2 (build verification) +- **Issue:** `el.textContent()` returns `string | null`; concatenation at line 22 failed strict TypeScript check. This file was included in tsconfig `**/*.ts` glob and caused `npm run build` to fail. +- **Fix:** Changed `text + href` to `(text ?? '') + (href ?? '')` — null coalescing guards both values +- **Files modified:** teressa-copeland-homes/scripts/debug-inspect2.ts +- **Verification:** `npm run build` completed successfully after fix; all new routes visible in build output +- **Committed in:** `c81e8ea` (Task 2 commit) + +--- + +**Total deviations:** 1 auto-fixed (1 blocking pre-existing bug) +**Impact on plan:** Pre-existing bug in unrelated debug script blocked build verification. Trivial null-coalescing fix restored build. No scope creep. + +## Issues Encountered + +- `npm run db:migrate` requires DATABASE_URL env var which was in `.env.local` not `.env`. Ran migration with explicit `DATABASE_URL=postgresql://postgres:postgres@localhost:5432/teressa` env override to apply successfully. + +## User Setup Required + +None — no external service configuration required. Database migration applied automatically. + +## Next Phase Readiness + +- Plan 02 (field-placer UI) can now import `SignatureFieldData` from `@/lib/db/schema` and consume `GET/PUT /api/documents/[id]/fields` +- Plan 03 (text-fill UI) can use `POST /api/documents/[id]/prepare` with `textFillData` payload +- All three API routes return 401 unauthenticated (verified by build output showing routes as dynamic server-rendered) +- `preparePdf` utility is ready for integration testing once an actual PDF file exists in uploads/ + +--- +*Phase: 05-pdf-fill-and-field-mapping* +*Completed: 2026-03-19*