diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index ce719a8..51991fc 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -291,8 +291,8 @@ Plans: Plans: - [ ] 13-01-PLAN.md — lib/ai/extract-text.ts (pdfjs-dist legacy build, server-only), lib/ai/field-placement.ts (GPT-4o-mini structured output, manual JSON schema, server-only guard), aiCoordsToPagePdfSpace() utility + unit test -- [ ] 13-02-PLAN.md — POST /api/documents/[id]/ai-prepare route (orchestrate extract + AI call + coordinate conversion + field write), "AI Auto-place" button in PreparePanel with loading state -- [ ] 13-03-PLAN.md — AI pre-fill: map client profile fields (name, property address, date) to placed text fields; agent review step before fields are committed +- [ ] 13-02-PLAN.md — POST /api/documents/[id]/ai-prepare route: auth + OPENAI_API_KEY guards, extract PDF text, call classifyFieldsWithAI, write fields to DB, return { fields, textFillData } +- [ ] 13-03-PLAN.md — UI wiring: aiPlacementKey prop in FieldPlacer + PdfViewerWrapper thread-through; "AI Auto-place Fields" button (violet) in PreparePanel; DocumentPageClient handleAiAutoPlace with textFillData merge and preview reset - [ ] 13-04-PLAN.md — Integration test with full 20-page Utah REPC + full Phase 13 human verification checkpoint ## Progress diff --git a/.planning/phases/13-ai-field-placement-and-pre-fill/13-01-PLAN.md b/.planning/phases/13-ai-field-placement-and-pre-fill/13-01-PLAN.md new file mode 100644 index 0000000..acd7c51 --- /dev/null +++ b/.planning/phases/13-ai-field-placement-and-pre-fill/13-01-PLAN.md @@ -0,0 +1,255 @@ +--- +phase: 13-ai-field-placement-and-pre-fill +plan: 01 +type: tdd +wave: 1 +depends_on: [] +files_modified: + - teressa-copeland-homes/src/lib/ai/extract-text.ts + - teressa-copeland-homes/src/lib/ai/field-placement.ts + - teressa-copeland-homes/src/lib/pdf/__tests__/ai-coords.test.ts +autonomous: true +requirements: + - AI-01 + - AI-02 + +must_haves: + truths: + - "pdfjs-dist extracts per-page text and page dimensions from a server-side file path" + - "aiCoordsToPagePdfSpace converts AI top-left percentage coordinates to PDF bottom-left point coordinates with correct Y-axis flip" + - "GPT-4o-mini structured output call returns a typed AiPlacedField[] array via manual json_schema response_format" + artifacts: + - path: "teressa-copeland-homes/src/lib/ai/extract-text.ts" + provides: "extractPdfText(filePath) returning PageText[] with page, text, width, height" + exports: ["extractPdfText", "PageText"] + - path: "teressa-copeland-homes/src/lib/ai/field-placement.ts" + provides: "aiCoordsToPagePdfSpace() + classifyFieldsWithAI() using GPT-4o-mini structured output" + exports: ["aiCoordsToPagePdfSpace", "classifyFieldsWithAI", "AiFieldCoords"] + - path: "teressa-copeland-homes/src/lib/pdf/__tests__/ai-coords.test.ts" + provides: "Unit tests for aiCoordsToPagePdfSpace with US Letter dimensions" + contains: "aiCoordsToPagePdfSpace" + key_links: + - from: "teressa-copeland-homes/src/lib/ai/field-placement.ts" + to: "teressa-copeland-homes/src/lib/ai/extract-text.ts" + via: "import { PageText } from './extract-text'" + pattern: "from.*extract-text" + - from: "teressa-copeland-homes/src/lib/ai/field-placement.ts" + to: "openai SDK" + via: "import OpenAI from 'openai'" + pattern: "import OpenAI" +--- + + +Install the openai SDK and create the two server-only AI utility modules plus a unit test for the coordinate conversion function. + +Purpose: These are the foundation layer for the AI auto-place feature. The route (Plan 02) and the UI button (Plan 03) both depend on these utilities existing and being correct. The coordinate conversion is the most failure-prone part of the feature — the unit test locks in correctness before it is wired into a live route. + +Output: +- `src/lib/ai/extract-text.ts` — pdfjs-dist server-side text extraction +- `src/lib/ai/field-placement.ts` — GPT-4o-mini structured output + aiCoordsToPagePdfSpace utility +- `src/lib/pdf/__tests__/ai-coords.test.ts` — unit test for aiCoordsToPagePdfSpace (TDD plan: write test first, then implement) + + + +@/Users/ccopeland/.claude/get-shit-done/workflows/execute-plan.md +@/Users/ccopeland/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-RESEARCH.md + +Read next.config.ts and check for Next.js guide in node_modules/next/dist/docs/ before writing route code. + + + + + +From teressa-copeland-homes/src/lib/db/schema.ts: +```typescript +export type SignatureFieldType = + | 'client-signature' + | 'initials' + | 'text' + | 'checkbox' + | 'date' + | 'agent-signature' + | 'agent-initials'; + +export interface SignatureFieldData { + id: string; + page: number; // 1-indexed + x: number; // PDF user space, bottom-left origin, points + y: number; // PDF user space, bottom-left origin, points + width: number; // PDF points (default: 144 — 2 inches) + height: number; // PDF points (default: 36 — 0.5 inches) + type?: SignatureFieldType; +} +``` + +From teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts (unit test pattern): +```typescript +// Jest + ts-jest (see package.json jest config) +// Test file pattern: src/lib/pdf/__tests__/*.test.ts +// Run: cd teressa-copeland-homes && npx jest src/lib/pdf/__tests__/ai-coords.test.ts +``` + +Existing coordinate formula from FieldPlacer.tsx (lines 289-290) — aiCoordsToPagePdfSpace MUST replicate this: +```typescript +const pdfX = (clampedX / renderedW) * pageInfo.originalWidth; +const pdfY = ((renderedH - (clampedY + fieldHpx)) / renderedH) * pageInfo.originalHeight; +// Where clampedX/clampedY are screen top-left origin, fieldHpx is field height in pixels +// Translated to percentage inputs: +// pdfX = (xPct / 100) * pageWidth +// screenY = (yPct / 100) * pageHeight (top-left origin from AI) +// fieldH = (heightPct / 100) * pageHeight +// pdfY = pageHeight - screenY - fieldH (bottom edge in PDF space) +``` + + + + + + Task 1: Write failing unit test for aiCoordsToPagePdfSpace + teressa-copeland-homes/src/lib/pdf/__tests__/ai-coords.test.ts + +Create the test file. Import `aiCoordsToPagePdfSpace` from `'../../../lib/ai/field-placement'` (it does not exist yet — the import will fail, making the test RED). Write these test cases covering US Letter (612×792 pts): + +Test group: "aiCoordsToPagePdfSpace — AI top-left percentage to PDF bottom-left points" + +Case 1 — text field near top of page: +- Input: { page:1, fieldType:'text', xPct:10, yPct:5, widthPct:30, heightPct:5, prefillValue:'' }, pageWidth:612, pageHeight:792 +- Expected: x ≈ 61.2, y ≈ 712.8, width ≈ 183.6, height ≈ 39.6 +- Formula: x=612*0.1=61.2; fieldH=792*0.05=39.6; screenY=792*0.05=39.6; y=792-39.6-39.6=712.8 + +Case 2 — checkbox near bottom-right: +- Input: { page:1, fieldType:'checkbox', xPct:90, yPct:95, widthPct:3, heightPct:3, prefillValue:'' }, pageWidth:612, pageHeight:792 +- Expected: x ≈ 550.8, y ≈ 15.84, width ≈ 18.36, height ≈ 23.76 +- Formula: x=612*0.9=550.8; fieldH=792*0.03=23.76; screenY=792*0.95=752.4; y=792-752.4-23.76=15.84 + +Case 3 — field at exact center: +- Input: { page:1, fieldType:'client-signature', xPct:50, yPct:50, widthPct:20, heightPct:5, prefillValue:'' }, pageWidth:612, pageHeight:792 +- Expected: x ≈ 306, y ≈ 357.84, width ≈ 122.4, height ≈ 39.6 +- Formula: x=306; fieldH=39.6; screenY=396; y=792-396-39.6=356.4 (use toBeCloseTo(1)) + +Run: `cd teressa-copeland-homes && npx jest src/lib/pdf/__tests__/ai-coords.test.ts --no-coverage` +Tests must FAIL (RED) because the module does not exist yet. Commit: `test(13-01): add failing aiCoordsToPagePdfSpace unit tests` + + + cd /Users/ccopeland/temp/red/teressa-copeland-homes && npx jest src/lib/pdf/__tests__/ai-coords.test.ts --no-coverage 2>&1 | tail -20 + + Test file exists, tests run and fail with "Cannot find module" or similar import error — RED phase confirmed + + + + Task 2: Implement extract-text.ts and field-placement.ts (GREEN) + + teressa-copeland-homes/src/lib/ai/extract-text.ts + teressa-copeland-homes/src/lib/ai/field-placement.ts + + +**Step 0: Install openai SDK** +```bash +cd /Users/ccopeland/temp/red/teressa-copeland-homes && npm install openai +``` + +**Step 1: Create `src/lib/ai/extract-text.ts`** + +Use the exact pattern from RESEARCH.md Pattern 1. Key points: +- Import from `'pdfjs-dist/legacy/build/pdf.mjs'` +- Set `GlobalWorkerOptions.workerSrc = ''` at module level (Node.js fake-worker mode — NOT browser pattern) +- Export `PageText` interface: `{ page: number; text: string; width: number; height: number }` +- Export `extractPdfText(filePath: string): Promise` +- Uses `readFile` from `node:fs/promises`, loads PDF bytes as `Uint8Array` +- Iterates all pages, calls `page.getViewport({ scale: 1.0 })` for dimensions, `getTextContent()` for text +- Joins text items by filtering `'str' in item` then mapping `.str`, joined with `' '` +- Cap text per page at 2000 chars with `text.slice(0, 2000)` to stay within GPT-4o-mini context +- Add comment: `// server-only — never import from client components` + +**Step 2: Create `src/lib/ai/field-placement.ts`** + +Exports: +```typescript +export interface AiFieldCoords { + page: number; + fieldType: SignatureFieldType; + xPct: number; // % from left, top-left origin (AI output) + yPct: number; // % from top, top-left origin (AI output) + widthPct: number; + heightPct: number; + prefillValue: string; +} + +export function aiCoordsToPagePdfSpace( + coords: AiFieldCoords, + pageWidth: number, + pageHeight: number, +): { x: number; y: number; width: number; height: number } +``` + +Implement `aiCoordsToPagePdfSpace` using the formula: +```typescript +const fieldWidth = (coords.widthPct / 100) * pageWidth; +const fieldHeight = (coords.heightPct / 100) * pageHeight; +const screenX = (coords.xPct / 100) * pageWidth; +const screenY = (coords.yPct / 100) * pageHeight; // screen Y from top +const x = screenX; +const y = pageHeight - screenY - fieldHeight; // Y-axis flip: bottom-left origin +return { x, y, width: fieldWidth, height: fieldHeight }; +``` + +Also export `classifyFieldsWithAI(pageTexts: PageText[], client: { name: string | null; propertyAddress: string | null } | null): Promise<{ fields: SignatureFieldData[], textFillData: Record }>` + +Implementation of `classifyFieldsWithAI`: +1. Build the FIELD_PLACEMENT_SCHEMA (exact schema from RESEARCH.md Pattern 2 — all fields required, `additionalProperties: false` at every nesting level, `strict: true`) +2. Build `pagesSummary` string: for each page, format as `Page {n} ({w}x{h}pt):\n{text}\n` — truncate text to 2000 chars per page +3. Create `const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })` +4. Check `if (!process.env.OPENAI_API_KEY) throw new Error('OPENAI_API_KEY not configured')` +5. Call `openai.chat.completions.create` with model `'gpt-4o-mini'`, manual json_schema response_format (NOT zodResponseFormat), system prompt from RESEARCH.md Pattern 2, user message includes client name, property address, pagesSummary +6. Parse response: `JSON.parse(response.choices[0].message.content!)` typed as `{ fields: AiFieldCoords[] }` +7. Map each AI field to `SignatureFieldData` using `aiCoordsToPagePdfSpace` with the page's dimensions from `pageTexts.find(p => p.page === field.page)` +8. Assign `id: crypto.randomUUID()` to each field, set `type: field.fieldType` +9. Default field sizes: checkbox=24×24pt, all others=144×36pt (ignore AI width/height for consistent sizing — override with standard sizes) +10. Build `textFillData: Record` — for each text field where `prefillValue` is non-empty string, add `{ [field.id]: prefillValue }` +11. Return `{ fields: SignatureFieldData[], textFillData }` + +Add comment: `// server-only — never import from client components` + +**Step 3: Run tests (GREEN)** +```bash +cd /Users/ccopeland/temp/red/teressa-copeland-homes && npx jest src/lib/pdf/__tests__/ai-coords.test.ts --no-coverage +``` +Tests must PASS. Commit: `feat(13-01): implement aiCoordsToPagePdfSpace and AI field utilities` + +Note on TypeScript import: if `'pdfjs-dist/legacy/build/pdf.mjs'` triggers type errors, add `// @ts-ignore` above the import or use the main types: `import type { PDFDocumentProxy, TextItem } from 'pdfjs-dist'`. + + + cd /Users/ccopeland/temp/red/teressa-copeland-homes && npx jest src/lib/pdf/__tests__/ai-coords.test.ts --no-coverage + + All 3+ aiCoordsToPagePdfSpace test cases pass (GREEN). extract-text.ts and field-placement.ts exist with correct exports. openai package is in package.json dependencies. + + + + + +- `npx jest src/lib/pdf/__tests__/ai-coords.test.ts --no-coverage` passes all cases +- `src/lib/ai/extract-text.ts` exports `extractPdfText` and `PageText` +- `src/lib/ai/field-placement.ts` exports `aiCoordsToPagePdfSpace`, `classifyFieldsWithAI`, `AiFieldCoords` +- `openai` is listed in `package.json` dependencies (not devDependencies) +- Neither file is imported from any client component (server-only comment guard present) +- `GlobalWorkerOptions.workerSrc = ''` set in extract-text.ts (NOT the browser `new URL(...)` pattern) +- Manual `json_schema` response_format used (NOT `zodResponseFormat`) + + + +- Unit test passes: aiCoordsToPagePdfSpace correctly flips Y-axis for all 3 test cases +- openai SDK installed and importable +- Both AI utility modules exist, export correct types, have server-only guard comments +- TypeScript compiles without errors: `npx tsc --noEmit` + + + +After completion, create `.planning/phases/13-ai-field-placement-and-pre-fill/13-01-SUMMARY.md` + diff --git a/.planning/phases/13-ai-field-placement-and-pre-fill/13-02-PLAN.md b/.planning/phases/13-ai-field-placement-and-pre-fill/13-02-PLAN.md new file mode 100644 index 0000000..448ce7f --- /dev/null +++ b/.planning/phases/13-ai-field-placement-and-pre-fill/13-02-PLAN.md @@ -0,0 +1,258 @@ +--- +phase: 13-ai-field-placement-and-pre-fill +plan: 02 +type: execute +wave: 2 +depends_on: + - 13-01 +files_modified: + - teressa-copeland-homes/src/app/api/documents/[id]/ai-prepare/route.ts +autonomous: true +requirements: + - AI-01 + - AI-02 + +must_haves: + truths: + - "POST /api/documents/[id]/ai-prepare extracts PDF text, calls GPT-4o-mini, converts coordinates, and writes SignatureFieldData[] to the DB" + - "Route returns { fields, textFillData } on success — client uses this to update FieldPlacer and pre-fill state" + - "Route returns 503 if OPENAI_API_KEY is not configured, 401 if unauthenticated, 404 if document not found, 403 if document is not Draft" + - "textFillData in response is keyed by field UUID (not by label string)" + artifacts: + - path: "teressa-copeland-homes/src/app/api/documents/[id]/ai-prepare/route.ts" + provides: "POST handler orchestrating extract → AI → convert → DB write → return" + exports: ["POST"] + key_links: + - from: "teressa-copeland-homes/src/app/api/documents/[id]/ai-prepare/route.ts" + to: "teressa-copeland-homes/src/lib/ai/extract-text.ts" + via: "import { extractPdfText } from '@/lib/ai/extract-text'" + pattern: "extractPdfText" + - from: "teressa-copeland-homes/src/app/api/documents/[id]/ai-prepare/route.ts" + to: "teressa-copeland-homes/src/lib/ai/field-placement.ts" + via: "import { classifyFieldsWithAI } from '@/lib/ai/field-placement'" + pattern: "classifyFieldsWithAI" + - from: "teressa-copeland-homes/src/app/api/documents/[id]/ai-prepare/route.ts" + to: "drizzle documents table" + via: "db.update(documents).set({ signatureFields: fields })" + pattern: "signatureFields" +--- + + +Create the `POST /api/documents/[id]/ai-prepare` route that orchestrates the full AI placement pipeline: extract PDF text, call GPT-4o-mini, convert AI percentage coordinates to PDF user-space points, and write the resulting fields to the database. + +Purpose: This is the server-side orchestration layer. The AI utility modules (Plan 01) provide the logic; this route wires them to the document DB record and returns the result to the client. + +Output: `src/app/api/documents/[id]/ai-prepare/route.ts` + + + +@/Users/ccopeland/.claude/get-shit-done/workflows/execute-plan.md +@/Users/ccopeland/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-RESEARCH.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-01-SUMMARY.md + +Read next.config.ts and check for Next.js guide in node_modules/next/dist/docs/ before writing route code. + + + + + +From teressa-copeland-homes/src/lib/db/schema.ts: +```typescript +export interface SignatureFieldData { + id: string; + page: number; + x: number; y: number; width: number; height: number; + type?: SignatureFieldType; +} +export const documents = pgTable("documents", { + id: text("id").primaryKey(), + clientId: text("client_id").notNull(), + status: documentStatusEnum("status").notNull().default("Draft"), + filePath: text("file_path"), + signatureFields: jsonb("signature_fields").$type(), + textFillData: jsonb("text_fill_data").$type>(), + // ... +}); +export const clients = pgTable("clients", { + id: text("id").primaryKey(), + name: text("name").notNull(), + propertyAddress: text("property_address"), + // ... +}); +export const documentsRelations = relations(documents, ({ one }) => ({ + client: one(clients, { fields: [documents.clientId], references: [clients.id] }), +})); +``` + +From existing prepare route pattern (src/app/api/documents/[id]/prepare/route.ts): +```typescript +import { auth } from '@/lib/auth'; +import { db } from '@/lib/db'; +import { documents, users } from '@/lib/db/schema'; +import { eq } from 'drizzle-orm'; +import path from 'node:path'; + +const UPLOADS_DIR = path.join(process.cwd(), 'uploads'); + +export async function POST( + req: Request, + { params }: { params: Promise<{ id: string }> } +) { + const session = await auth(); + if (!session?.user?.id) return new Response('Unauthorized', { status: 401 }); + const { id } = await params; + const doc = await db.query.documents.findFirst({ + where: eq(documents.id, id), + with: { client: true }, // use this to get client.name and client.propertyAddress + }); + if (!doc) return Response.json({ error: 'Not found' }, { status: 404 }); + // ... +} +``` + +From Plan 01 (13-01-SUMMARY.md): +```typescript +// src/lib/ai/extract-text.ts +export async function extractPdfText(filePath: string): Promise +export interface PageText { page: number; text: string; width: number; height: number; } + +// src/lib/ai/field-placement.ts +export async function classifyFieldsWithAI( + pageTexts: PageText[], + client: { name: string | null; propertyAddress: string | null } | null +): Promise<{ fields: SignatureFieldData[], textFillData: Record }> +``` + + + + + + Task 1: Create POST /api/documents/[id]/ai-prepare route + teressa-copeland-homes/src/app/api/documents/[id]/ai-prepare/route.ts + +Create the directory `src/app/api/documents/[id]/ai-prepare/` and the `route.ts` file. + +Implement: +```typescript +import { auth } from '@/lib/auth'; +import { db } from '@/lib/db'; +import { documents } from '@/lib/db/schema'; +import { eq } from 'drizzle-orm'; +import path from 'node:path'; +import { extractPdfText } from '@/lib/ai/extract-text'; +import { classifyFieldsWithAI } from '@/lib/ai/field-placement'; + +const UPLOADS_DIR = path.join(process.cwd(), 'uploads'); + +export async function POST( + req: Request, + { params }: { params: Promise<{ id: string }> } +) { + // 1. Auth guard — same pattern as prepare/route.ts + const session = await auth(); + if (!session?.user?.id) return new Response('Unauthorized', { status: 401 }); + + // 2. OPENAI_API_KEY guard — fail fast with 503 if not configured + if (!process.env.OPENAI_API_KEY) { + return Response.json( + { error: 'OPENAI_API_KEY not configured. Add it to .env.local.' }, + { status: 503 } + ); + } + + // 3. Load document with client relation + const { id } = await params; + const doc = await db.query.documents.findFirst({ + where: eq(documents.id, id), + with: { client: true }, + }); + if (!doc) return Response.json({ error: 'Not found' }, { status: 404 }); + if (!doc.filePath) return Response.json({ error: 'Document has no PDF file' }, { status: 422 }); + + // 4. Only Draft documents can be AI-prepared (same principle as prepare route) + if (doc.status !== 'Draft') { + return Response.json({ error: 'Document is locked — only Draft documents can use AI auto-place' }, { status: 403 }); + } + + // 5. Path traversal guard + const filePath = path.join(UPLOADS_DIR, doc.filePath); + if (!filePath.startsWith(UPLOADS_DIR)) { + return new Response('Forbidden', { status: 403 }); + } + + // 6. Extract text from PDF (server-side, pdfjs-dist legacy build) + const pageTexts = await extractPdfText(filePath); + + // 7. Classify fields with GPT-4o-mini + coordinate conversion + const client = (doc as typeof doc & { client: { name: string; propertyAddress: string | null } | null }).client; + const { fields, textFillData } = await classifyFieldsWithAI( + pageTexts, + client ? { name: client.name, propertyAddress: client.propertyAddress ?? null } : null, + ); + + // 8. Write fields to DB (replaces existing signatureFields — agent reviews after) + // Do NOT change document status — stays Draft so agent can review and adjust + const [updated] = await db + .update(documents) + .set({ signatureFields: fields }) + .where(eq(documents.id, id)) + .returning(); + + // 9. Return fields and textFillData for client state update + return Response.json({ + fields: updated.signatureFields ?? [], + textFillData, + }); +} +``` + +Key decisions to honor (from STATE.md and RESEARCH.md): +- Do NOT use zodResponseFormat (broken with Zod v4) — this route delegates to classifyFieldsWithAI which uses manual json_schema +- Do NOT change document status to anything other than Draft — only the prepare route moves status +- textFillData is keyed by field UUID (classifyFieldsWithAI handles this — see Plan 01) +- `doc.client` is available via Drizzle relation — use `with: { client: true }` in the query + +Error handling: wrap the extractPdfText + classifyFieldsWithAI calls in try/catch, return 500 with the error message if they throw (AI errors, PDF read errors, etc.). + + + cd /Users/ccopeland/temp/red/teressa-copeland-homes && npx tsc --noEmit 2>&1 | grep -E "ai-prepare|error" | head -20 + + + - `src/app/api/documents/[id]/ai-prepare/route.ts` exists and exports `POST` + - TypeScript compiles without errors for this file + - Route structure matches Next.js App Router pattern (params as Promise) + - All guards present: auth, OPENAI_API_KEY, not-found, no-filePath, non-Draft + - Fields written to DB, document status stays Draft + - Returns `{ fields: SignatureFieldData[], textFillData: Record }` + + + + + + +- `npx tsc --noEmit` passes with no errors related to ai-prepare route +- Route file exists at correct Next.js App Router path +- Drizzle query uses `with: { client: true }` to load client profile data +- Route does NOT call `extractPdfText` or `classifyFieldsWithAI` directly — only after all guards pass +- Document status remains Draft after successful AI placement +- `textFillData` in response is keyed by SignatureFieldData UUIDs (not labels) + + + +- `POST /api/documents/[id]/ai-prepare` returns 401 for unauthenticated requests +- Returns 503 when OPENAI_API_KEY is absent from environment +- Returns 403 for non-Draft documents +- Returns `{ fields, textFillData }` on success, with fields written to DB +- TypeScript compiles clean + + + +After completion, create `.planning/phases/13-ai-field-placement-and-pre-fill/13-02-SUMMARY.md` + diff --git a/.planning/phases/13-ai-field-placement-and-pre-fill/13-03-PLAN.md b/.planning/phases/13-ai-field-placement-and-pre-fill/13-03-PLAN.md new file mode 100644 index 0000000..b2d12da --- /dev/null +++ b/.planning/phases/13-ai-field-placement-and-pre-fill/13-03-PLAN.md @@ -0,0 +1,326 @@ +--- +phase: 13-ai-field-placement-and-pre-fill +plan: 03 +type: execute +wave: 3 +depends_on: + - 13-01 + - 13-02 +files_modified: + - teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/FieldPlacer.tsx + - teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/PdfViewerWrapper.tsx + - teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/PreparePanel.tsx + - teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/DocumentPageClient.tsx +autonomous: true +requirements: + - AI-01 + - AI-02 + +must_haves: + truths: + - "PreparePanel has an 'AI Auto-place' button that calls POST /api/documents/[id]/ai-prepare and shows loading state" + - "After AI placement succeeds, FieldPlacer re-fetches fields from DB and displays the AI-placed field overlays" + - "After AI placement, DocumentPageClient's textFillData is updated with the returned pre-fill map and previewToken is reset to null" + - "Agent can review, move, resize, or delete any AI-placed field after placement — fields are editable, not locked" + artifacts: + - path: "teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/FieldPlacer.tsx" + provides: "FieldPlacer with aiPlacementKey prop that triggers loadFields re-fetch" + contains: "aiPlacementKey" + - path: "teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/PdfViewerWrapper.tsx" + provides: "PdfViewerWrapper threads aiPlacementKey and onAiPlacementKeyChange to PdfViewer/FieldPlacer" + contains: "aiPlacementKey" + - path: "teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/PreparePanel.tsx" + provides: "AI Auto-place button with loading state, calls onAiAutoPlace callback" + contains: "AI Auto-place" + - path: "teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/DocumentPageClient.tsx" + provides: "handleAiAutoPlace that calls route, updates textFillData, increments aiPlacementKey, resets previewToken" + contains: "handleAiAutoPlace" + key_links: + - from: "teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/PreparePanel.tsx" + to: "/api/documents/[id]/ai-prepare" + via: "onAiAutoPlace callback → DocumentPageClient.handleAiAutoPlace → fetch POST" + pattern: "onAiAutoPlace|handleAiAutoPlace" + - from: "teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/DocumentPageClient.tsx" + to: "teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/FieldPlacer.tsx" + via: "aiPlacementKey prop threaded through PdfViewerWrapper" + pattern: "aiPlacementKey" +--- + + +Wire the "AI Auto-place" button into the PreparePanel and connect it to the FieldPlacer reload mechanism via DocumentPageClient state. + +Purpose: The AI utilities (Plan 01) and route (Plan 02) are server-side. This plan adds the client-side interaction: the button, the loading state, the post-success state updates (fields reload + textFillData update + preview reset), and the FieldPlacer re-fetch trigger. + +Output: Four files modified — FieldPlacer gets `aiPlacementKey` prop, PdfViewerWrapper threads it, PreparePanel gets "AI Auto-place" button, DocumentPageClient orchestrates the handler. + + + +@/Users/ccopeland/.claude/get-shit-done/workflows/execute-plan.md +@/Users/ccopeland/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-RESEARCH.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-01-SUMMARY.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-02-SUMMARY.md + +Read next.config.ts and check for Next.js guide in node_modules/next/dist/docs/ before writing component code. + + + + + +Current FieldPlacer props (FieldPlacer.tsx line 155-166): +```typescript +interface FieldPlacerProps { + docId: string; + pageInfo: PageInfo | null; + currentPage: number; + children: React.ReactNode; + readOnly?: boolean; + onFieldsChanged?: () => void; + selectedFieldId?: string | null; + textFillData?: Record; + onFieldSelect?: (fieldId: string | null) => void; + onFieldValueChange?: (fieldId: string, value: string) => void; +} +// FieldPlacer loads fields in useEffect([docId]) — adding aiPlacementKey to dependency array triggers re-fetch +``` + +Current PdfViewerWrapper props: +```typescript +// PdfViewerWrapper({ docId, docStatus, onFieldsChanged, selectedFieldId, textFillData, onFieldSelect, onFieldValueChange }) +// It wraps PdfViewer (dynamic import, ssr: false) which renders FieldPlacer +// PdfViewer is at src/app/portal/(protected)/documents/[docId]/_components/PdfViewer.tsx +// It passes all props through to FieldPlacer +``` + +Current PreparePanel props (PreparePanel.tsx lines 6-18): +```typescript +interface PreparePanelProps { + docId: string; + defaultEmail: string; + clientName: string; + currentStatus: string; + agentDownloadUrl?: string | null; + signedAt?: Date | null; + clientPropertyAddress?: string | null; + previewToken: string | null; + onPreviewTokenChange: (token: string | null) => void; + textFillData: Record; + selectedFieldId: string | null; + onQuickFill: (fieldId: string, value: string) => void; +} +``` + +Current DocumentPageClient state (DocumentPageClient.tsx): +```typescript +const [previewToken, setPreviewToken] = useState(null); +const [selectedFieldId, setSelectedFieldId] = useState(null); +const [textFillData, setTextFillData] = useState>({}); +// handleFieldsChanged, handleFieldValueChange, handleQuickFill already exist +``` + + + + + + Task 1: Add aiPlacementKey prop to FieldPlacer and thread through PdfViewerWrapper + + teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/FieldPlacer.tsx + teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/PdfViewerWrapper.tsx + + +**FieldPlacer.tsx changes:** + +1. Add `aiPlacementKey?: number` to `FieldPlacerProps` interface. + +2. Add `aiPlacementKey = 0` to the destructured props in the function signature. + +3. Update the `loadFields` useEffect dependency array to include `aiPlacementKey`: + ```typescript + // Change from: + useEffect(() => { ... loadFields(); }, [docId]); + // Change to: + useEffect(() => { ... loadFields(); }, [docId, aiPlacementKey]); + ``` + This causes FieldPlacer to re-fetch from DB whenever `aiPlacementKey` increments after AI placement. + +No other changes to FieldPlacer.tsx. + +**PdfViewerWrapper.tsx changes:** + +Read PdfViewer.tsx first to understand how it passes props to FieldPlacer — the thread needs to go: `DocumentPageClient → PdfViewerWrapper → PdfViewer → FieldPlacer`. + +Add `aiPlacementKey?: number` to PdfViewerWrapper's props interface and pass it through to PdfViewer: +```typescript +export function PdfViewerWrapper({ + docId, docStatus, onFieldsChanged, selectedFieldId, textFillData, + onFieldSelect, onFieldValueChange, + aiPlacementKey, // NEW +}: { + docId: string; + docStatus?: string; + onFieldsChanged?: () => void; + selectedFieldId?: string | null; + textFillData?: Record; + onFieldSelect?: (fieldId: string | null) => void; + onFieldValueChange?: (fieldId: string, value: string) => void; + aiPlacementKey?: number; // NEW +}) { + return ( + + ); +} +``` + +Also read PdfViewer.tsx and add `aiPlacementKey?: number` to its props + pass through to FieldPlacer. PdfViewer is the component that actually renders FieldPlacer wrapping the PDF page canvas. + + + cd /Users/ccopeland/temp/red/teressa-copeland-homes && npx tsc --noEmit 2>&1 | grep -E "FieldPlacer|PdfViewer|error TS" | head -20 + + + - FieldPlacer accepts and uses `aiPlacementKey` in the loadFields useEffect dependency array + - PdfViewerWrapper and PdfViewer thread `aiPlacementKey` through to FieldPlacer + - TypeScript compiles without errors for these files + + + + + Task 2: Add AI Auto-place button to PreparePanel and wire DocumentPageClient handler + + teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/PreparePanel.tsx + teressa-copeland-homes/src/app/portal/(protected)/documents/[docId]/_components/DocumentPageClient.tsx + + +**PreparePanel.tsx changes:** + +1. Add `onAiAutoPlace: () => Promise` to `PreparePanelProps` interface. + +2. Destructure `onAiAutoPlace` from props. + +3. Add a local `aiLoading` state: `const [aiLoading, setAiLoading] = useState(false)`. + +4. Add an `handleAiAutoPlaceClick` function in the component: + ```typescript + async function handleAiAutoPlaceClick() { + setAiLoading(true); + setResult(null); + try { + await onAiAutoPlace(); + } catch (e) { + setResult({ ok: false, message: String(e) }); + } finally { + setAiLoading(false); + } + } + ``` + +5. Add the "AI Auto-place" button to the JSX. Position it ABOVE the Preview button, at the top of the Draft panel (after the recipients textarea section, before the quick-fill section). Only show it when `currentStatus === 'Draft'`: + ```tsx + + ``` + Use violet (#7c3aed) to visually distinguish from the gray Preview button and blue Prepare and Send button. + +**DocumentPageClient.tsx changes:** + +1. Add `aiPlacementKey` state: `const [aiPlacementKey, setAiPlacementKey] = useState(0)`. + +2. Add `handleAiAutoPlace` callback: + ```typescript + const handleAiAutoPlace = useCallback(async () => { + const res = await fetch(`/api/documents/${docId}/ai-prepare`, { method: 'POST' }); + if (!res.ok) { + const err = await res.json().catch(() => ({ error: 'AI placement failed' })); + throw new Error(err.error ?? err.message ?? 'AI placement failed'); + } + const { textFillData: aiTextFill } = await res.json() as { + fields: unknown[]; + textFillData: Record; + }; + // Merge AI pre-fill into existing textFillData (AI values take precedence) + setTextFillData(prev => ({ ...prev, ...aiTextFill })); + // Trigger FieldPlacer to re-fetch from DB (fields were written server-side) + setAiPlacementKey(k => k + 1); + // Reset preview staleness — fields changed + setPreviewToken(null); + }, [docId]); + ``` + +3. Pass `aiPlacementKey` to `PdfViewerWrapper`: + ```tsx + + ``` + +4. Pass `onAiAutoPlace={handleAiAutoPlace}` to `PreparePanel`. + +Key design decisions: +- AI placement REPLACES all existing fields in the DB (the route's `db.update` overwrites `signatureFields`) +- FieldPlacer re-fetches from DB via the aiPlacementKey increment — this is the single source of truth +- textFillData is MERGED (not replaced) so any manually typed values are preserved alongside AI pre-fills +- previewToken is reset to null — agent must re-preview after AI placement +- Error from the route is thrown and caught in PreparePanel's handleAiAutoPlaceClick, displayed via the existing `result` state + + + cd /Users/ccopeland/temp/red/teressa-copeland-homes && npx tsc --noEmit 2>&1 | grep -E "PreparePanel|DocumentPageClient|error TS" | head -20 + + + - PreparePanel has "AI Auto-place Fields" button with violet color, loading state, error display + - DocumentPageClient has handleAiAutoPlace calling /api/documents/[id]/ai-prepare + - After success: textFillData merges AI pre-fills, aiPlacementKey increments, previewToken resets to null + - Error from route is surfaced to user via PreparePanel result state + - TypeScript compiles without errors across all 4 modified files + + + + + + +- `npx tsc --noEmit` passes with no errors +- FieldPlacer.tsx: `aiPlacementKey` in loadFields useEffect dependency array +- PdfViewerWrapper.tsx + PdfViewer.tsx: `aiPlacementKey` threaded through +- PreparePanel.tsx: "AI Auto-place Fields" button renders (violet, above Preview button) for Draft docs +- DocumentPageClient.tsx: `handleAiAutoPlace` callback, `aiPlacementKey` state, wired to both components +- Agent can still drag, move, resize, delete fields after AI placement (existing FieldPlacer behavior unchanged) + + + +- TypeScript compiles clean for all 4 files +- "AI Auto-place Fields" button is visible in PreparePanel for Draft documents +- Clicking the button calls POST /api/documents/[id]/ai-prepare +- On success: FieldPlacer shows AI-placed fields (via DB re-fetch), textFillData merges pre-fills, previewToken is null +- On error: error message displayed in PreparePanel result area +- Existing functionality unchanged: drag-and-drop, click-to-edit, quick-fill, preview, prepare-and-send + + + +After completion, create `.planning/phases/13-ai-field-placement-and-pre-fill/13-03-SUMMARY.md` + diff --git a/.planning/phases/13-ai-field-placement-and-pre-fill/13-04-PLAN.md b/.planning/phases/13-ai-field-placement-and-pre-fill/13-04-PLAN.md new file mode 100644 index 0000000..dcea5f1 --- /dev/null +++ b/.planning/phases/13-ai-field-placement-and-pre-fill/13-04-PLAN.md @@ -0,0 +1,200 @@ +--- +phase: 13-ai-field-placement-and-pre-fill +plan: 04 +type: execute +wave: 4 +depends_on: + - 13-01 + - 13-02 + - 13-03 +files_modified: [] +autonomous: false +requirements: + - AI-01 + - AI-02 + +must_haves: + truths: + - "Agent can click 'AI Auto-place Fields' on a real document and fields appear on the PDF canvas" + - "AI-placed text fields are pre-filled with client name and/or property address where available" + - "AI-placed fields are editable — agent can move, resize, and delete them after AI placement" + - "Coordinate conversion is correct — fields appear at the expected vertical positions (not inverted)" + - "Unit test passes: aiCoordsToPagePdfSpace produces correct Y-axis flip for US Letter dimensions" + artifacts: + - path: "teressa-copeland-homes/src/lib/pdf/__tests__/ai-coords.test.ts" + provides: "Passing unit tests for coordinate conversion" + contains: "aiCoordsToPagePdfSpace" + key_links: + - from: "AI Auto-place button" + to: "/api/documents/[id]/ai-prepare" + via: "PreparePanel onAiAutoPlace → DocumentPageClient handleAiAutoPlace → fetch POST" + pattern: "ai-prepare" + - from: "FieldPlacer" + to: "DB (signatureFields)" + via: "loadFields useEffect re-runs when aiPlacementKey increments" + pattern: "aiPlacementKey" +--- + + +Verify the complete AI field placement feature end-to-end: unit tests, integration test with a real document, coordinate accuracy check, and final human UAT sign-off. + +Purpose: AI coordinate accuracy on real Utah forms is listed as an explicit concern in STATE.md Blockers/Concerns. This plan is the mandatory verification gate before Phase 13 ships. + +Output: Human-verified AI field placement on a real document. Phase 13 complete. + + + +@/Users/ccopeland/.claude/get-shit-done/workflows/execute-plan.md +@/Users/ccopeland/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/STATE.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-RESEARCH.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-01-SUMMARY.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-02-SUMMARY.md +@.planning/phases/13-ai-field-placement-and-pre-fill/13-03-SUMMARY.md + + + + + + Task 1: Run unit tests and TypeScript build check + + +Run the full test suite for the AI coordinate conversion and confirm TypeScript compiles clean: + +```bash +cd /Users/ccopeland/temp/red/teressa-copeland-homes +npx jest src/lib/pdf/__tests__/ai-coords.test.ts --no-coverage --verbose +npx tsc --noEmit +``` + +If any test fails: +- Re-read the formula in `aiCoordsToPagePdfSpace` against FieldPlacer.tsx lines 289-290 +- The expected values in the test file are authoritative — fix the implementation, not the tests +- Re-run until GREEN + +If TypeScript errors: +- Fix them (likely missing prop types or import errors) +- Re-run `npx tsc --noEmit` until clean + +Do NOT proceed to the checkpoint task until both commands pass cleanly. + + + cd /Users/ccopeland/temp/red/teressa-copeland-homes && npx jest src/lib/pdf/__tests__/ai-coords.test.ts --no-coverage --verbose && npx tsc --noEmit + + All aiCoordsToPagePdfSpace test cases PASS. TypeScript compiles with no errors. Ready for human verification. + + + + Task 2: Human E2E verification of AI field placement + Human verifies the complete AI auto-place feature end-to-end on a real document at localhost:3000. + +Complete AI field placement feature: +- "AI Auto-place Fields" button (violet) in PreparePanel for all Draft documents +- POST /api/documents/[id]/ai-prepare route that extracts PDF text, calls GPT-4o-mini, converts coordinates, writes fields to DB +- FieldPlacer re-fetches from DB after AI placement (aiPlacementKey increment) +- Text fields pre-filled from client profile (name, property address) where available +- Unit test: aiCoordsToPagePdfSpace Y-axis flip verified for US Letter 612×792 pts + + +Before running this test, ensure: +- `.env.local` in the teressa-copeland-homes directory has `OPENAI_API_KEY=sk-...` set +- `npm run dev` is running at localhost:3000 + +**Step 1: Verify the "AI Auto-place Fields" button appears** +- Open any Draft document in the portal: http://localhost:3000/portal/documents/[any-draft-docId] +- Confirm the violet "AI Auto-place Fields" button is visible in the PreparePanel (above the gray "Preview" button) +- Confirm it is NOT visible for non-Draft documents (Sent/Signed) + +**Step 2: Test AI auto-place on a simple document** +- Choose a Draft document with a PDF that has visible form fields (any uploaded PDF) +- Click "AI Auto-place Fields" +- Expected: Button shows "AI placing fields..." during the request +- Expected: After completion, field overlays appear on the PDF canvas representing AI-placed fields +- Confirm field overlays appear — even if placement is imperfect, at least some fields should appear + +**Step 3: Verify field editability (AI-01 success criterion)** +- Click on one of the AI-placed text field overlays — confirm it selects (blue ring appears) +- Try moving a field to a different position — confirm drag-and-drop works +- Try deleting a field (click x) — confirm it deletes +- Confirm the remaining fields still show after delete + +**Step 4: Verify text pre-fill (AI-02)** +- If the document has an assigned client with a name and property address, open that document +- Click "AI Auto-place Fields" +- After completion, check the PreparePanel quick-fill area — click a text field +- Expected: The field value is already populated with client name or property address (if AI detected a matching field) +- Note: AI may not always pre-fill — this is best-effort. As long as the mechanism works when AI returns a prefillValue, it passes. + +**Step 5: Verify Y-axis coordinate accuracy (AI-01 success criterion)** +- Place a known field manually at 10% from top on a real PDF, note its visual position +- Use AI auto-place on the same PDF +- Expected: AI-placed fields should appear at roughly correct vertical positions (not inverted/flipped to bottom) +- If fields appear at bottom when they should be at top, flag as a coordinate bug (do not approve) + +**Step 6: Verify preview and prepare still work after AI placement** +- After AI auto-place, click "Preview" — confirm preview generates successfully +- Confirm the preview PDF shows the AI-placed fields +- Note: You do NOT need to click "Prepare and Send" for this verification + +**Step 7: Verify error state** +- Temporarily set an invalid OPENAI_API_KEY in .env.local (e.g., OPENAI_API_KEY=invalid) +- Restart dev server, click "AI Auto-place Fields" +- Expected: An error message appears in PreparePanel (red text) — it should NOT crash the page +- Restore the correct API key afterward + + + MISSING — this is a human-verify checkpoint; automated verification is Step 7 above (error state test) + + Human approves all 7 verification steps. AI-01 and AI-02 verified. Phase 13 complete. + +Type "approved" if all 7 steps pass. Describe any issues (especially Y-axis inversion in Step 5 or field editability failures in Step 3) and I will diagnose and fix. + +Known acceptable limitations: +- AI may not place fields on every form perfectly — imprecise positions are acceptable as long as they are in the correct vertical region +- AI may not pre-fill text fields if it does not detect matching labels — that is acceptable +- Very long PDFs (20+ pages) may take 10-20 seconds — that is acceptable + +Blocking failures (do NOT approve): +- Fields appear at inverted Y positions (Step 5) — this means aiCoordsToPagePdfSpace has a bug +- Fields cannot be moved or deleted after AI placement (Step 3) +- Page crashes or 500 error without user-visible error message +- "AI Auto-place Fields" button does not appear for Draft documents (Step 1) + + + + + + +- All aiCoordsToPagePdfSpace unit tests pass +- TypeScript compiles clean: npx tsc --noEmit +- AI Auto-place button visible in PreparePanel for Draft documents +- AI placement populates FieldPlacer with field overlays from DB +- Fields are movable/resizable/deletable after AI placement +- Text pre-fill from client profile works when AI detects matching fields +- Error state visible (not a crash) when API key is missing/invalid +- Human approved all 7 verification steps + + + +- Unit tests green (aiCoordsToPagePdfSpace) +- TypeScript clean build +- Human approves Steps 1-7: + - AI Auto-place button appears for Draft docs + - Fields appear on canvas after AI placement + - Fields are editable (move, resize, delete) + - Y-axis coordinates are correct (not inverted) + - Preview and prepare-and-send still work after AI placement + - Error state shown (not crash) when API key invalid +- AI-01 satisfied: one-click button places field types from PDF analysis +- AI-02 satisfied: text fields pre-filled with client data where AI detects match +- Phase 13 complete — v1.1 Smart Document Preparation milestone fully shipped + + + +After completion, create `.planning/phases/13-ai-field-placement-and-pre-fill/13-04-SUMMARY.md` +