red/.planning/phases/05-pdf-fill-and-field-mapping/05-01-PLAN.md

---
phase: 05-pdf-fill-and-field-mapping
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - teressa-copeland-homes/src/lib/db/schema.ts
  - teressa-copeland-homes/src/lib/pdf/prepare-document.ts
  - teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts
  - teressa-copeland-homes/src/app/api/documents/[id]/fields/route.ts
  - teressa-copeland-homes/src/app/api/documents/[id]/prepare/route.ts
autonomous: true
requirements: [DOC-04, DOC-05, DOC-06]

must_haves:
  truths:
    - "GET /api/documents/[id]/fields returns [] for a document with no placed fields"
    - "PUT /api/documents/[id]/fields with a SignatureFieldData[] body returns the stored array on the next GET"
    - "POST /api/documents/[id]/prepare transitions the document status to Sent and creates a file at uploads/clients/{clientId}/{docId}_prepared.pdf"
    - "POST /api/documents/[id]/prepare returns 422 if the document has no PDF file attached"
    - "A field placed at screen top (screenY=0) on a 792pt-tall page produces pdfY≈792 — Y-axis flip is correct"
  artifacts:
    - path: "teressa-copeland-homes/src/lib/db/schema.ts"
      provides: "Extended documents table with 4 new nullable columns"
      contains: "signatureFields"
    - path: "teressa-copeland-homes/src/lib/pdf/prepare-document.ts"
      provides: "Server-side PDF mutation utility using @cantoo/pdf-lib"
      exports: ["preparePdf"]
    - path: "teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts"
      provides: "Unit tests verifying Y-flip coordinate conversion formula against US Letter dimensions"
      exports: []
    - path: "teressa-copeland-homes/src/app/api/documents/[id]/fields/route.ts"
      provides: "GET/PUT API for signature field coordinates"
      exports: ["GET", "PUT"]
    - path: "teressa-copeland-homes/src/app/api/documents/[id]/prepare/route.ts"
      provides: "POST endpoint that mutates PDF and transitions status to Sent"
      exports: ["POST"]
  key_links:
    - from: "PUT /api/documents/[id]/fields"
      to: "documents.signatureFields (JSONB)"
      via: "drizzle db.update().set({ signatureFields })"
      pattern: "db\\.update.*signatureFields"
    - from: "POST /api/documents/[id]/prepare"
      to: "teressa-copeland-homes/src/lib/pdf/prepare-document.ts"
      via: "import { preparePdf } from '@/lib/pdf/prepare-document'"
      pattern: "preparePdf"
    - from: "prepare-document.ts"
      to: "uploads/clients/{clientId}/{docId}_prepared.pdf"
      via: "atomic write via tmp → rename"
      pattern: "rename.*tmp"
---

<objective>
Extend the database schema with four new columns on the documents table, generate and apply the Drizzle migration, create the server-side PDF preparation utility using @cantoo/pdf-lib, add two new API route files for signature field storage and document preparation, and add a unit test verifying the Y-flip coordinate conversion formula.

Purpose: Establish the data contracts and server logic that the field-placer UI (Plan 02) and text-fill UI (Plan 03) will consume. Everything in Phase 5 builds on these server endpoints.

Output: Migration 0003 applied, two new API routes live, @cantoo/pdf-lib utility with atomic write behavior, passing unit test for Y-flip formula.
</objective>

<execution_context>
@/Users/ccopeland/.claude/get-shit-done/workflows/execute-plan.md
@/Users/ccopeland/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/05-pdf-fill-and-field-mapping/05-RESEARCH.md

<interfaces>
<!-- Key types and contracts the executor needs. Extracted from existing codebase. -->

From teressa-copeland-homes/src/lib/db/schema.ts (current state — MODIFY this file):
```typescript
import { pgEnum, pgTable, text, timestamp } from "drizzle-orm/pg-core";
import { relations } from "drizzle-orm";

export const documentStatusEnum = pgEnum("document_status", ["Draft", "Sent", "Viewed", "Signed"]);

export const documents = pgTable("documents", {
  id: text("id").primaryKey().$defaultFn(() => crypto.randomUUID()),
  name: text("name").notNull(),
  clientId: text("client_id").notNull().references(() => clients.id, { onDelete: "cascade" }),
  status: documentStatusEnum("status").notNull().default("Draft"),
  sentAt: timestamp("sent_at"),
  createdAt: timestamp("created_at").defaultNow().notNull(),
  formTemplateId: text("form_template_id").references(() => formTemplates.id),
  filePath: text("file_path"),
  // ADD: signatureFields, textFillData, assignedClientId, preparedFilePath
});
```

From teressa-copeland-homes/src/app/api/documents/[id]/file/route.ts (auth pattern to mirror):
```typescript
import { auth } from '@/lib/auth';
// params is a Promise in Next.js 15 — always await before destructuring
export async function GET(req: Request, { params }: { params: Promise<{ id: string }> }) {
  const session = await auth();
  if (!session) return new Response('Unauthorized', { status: 401 });
  const { id } = await params;
  // ...
}
```

File path patterns from Phase 4 decisions:
- UPLOADS_DIR = path.join(process.cwd(), 'uploads')
- DB stores relative paths: 'clients/{clientId}/{docId}.pdf'
- Absolute path at read time: path.join(UPLOADS_DIR, relPath)
- New prepared file: 'clients/{clientId}/{docId}_prepared.pdf'
</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: Extend schema + generate and apply migration 0003</name>
  <files>
    teressa-copeland-homes/src/lib/db/schema.ts
    teressa-copeland-homes/drizzle/0003_*.sql (generated)
    teressa-copeland-homes/drizzle/meta/_journal.json (updated)
  </files>
  <action>
Add four nullable columns to the documents table in schema.ts. Import `jsonb` from 'drizzle-orm/pg-core'. Add these columns to the documents pgTable definition — place them after the existing `filePath` column:

```typescript
import { jsonb } from 'drizzle-orm/pg-core'; // add to existing import

// Add to documents table after filePath:
signatureFields: jsonb('signature_fields').$type<SignatureFieldData[]>(),
textFillData: jsonb('text_fill_data').$type<Record<string, string>>(),
assignedClientId: text('assigned_client_id'),
preparedFilePath: text('prepared_file_path'),
```

Also export this TypeScript interface from schema.ts (add near the top of the file, before the table definitions):

```typescript
export interface SignatureFieldData {
  id: string;
  page: number;   // 1-indexed
  x: number;      // PDF user space, bottom-left origin, points
  y: number;      // PDF user space, bottom-left origin, points
  width: number;  // PDF points (default: 144 — 2 inches)
  height: number; // PDF points (default: 36 — 0.5 inches)
}
```

Then run in the teressa-copeland-homes directory:
```bash
cd /Users/ccopeland/temp/red/teressa-copeland-homes
npm run db:generate
npm run db:migrate
```

The migration should add 4 columns (ALTER TABLE documents ADD COLUMN ...) — confirm the generated SQL looks correct before applying. The migration command will prompt if needed; migration applies via drizzle-kit migrate (not push — schema is controlled via migrations in this project).
  </action>
  <verify>
    <automated>cd /Users/ccopeland/temp/red/teressa-copeland-homes && npm run db:migrate 2>&1 | tail -5 && ls drizzle/0003_*.sql</automated>
  </verify>
  <done>schema.ts has all 4 new nullable columns typed correctly; drizzle/0003_*.sql exists; npm run db:migrate completes without error</done>
</task>

<task type="auto">
  <name>Task 2: Create pdf prepare-document utility + API routes for fields and prepare</name>
  <files>
    teressa-copeland-homes/src/lib/pdf/prepare-document.ts
    teressa-copeland-homes/src/app/api/documents/[id]/fields/route.ts
    teressa-copeland-homes/src/app/api/documents/[id]/prepare/route.ts
  </files>
  <action>
**Step A — Install @cantoo/pdf-lib** (do NOT install `pdf-lib` — they conflict):
```bash
cd /Users/ccopeland/temp/red/teressa-copeland-homes && npm install @cantoo/pdf-lib
```

**Step B — Create teressa-copeland-homes/src/lib/pdf/prepare-document.ts:**

```typescript
import { PDFDocument, StandardFonts, rgb } from '@cantoo/pdf-lib';
import { readFile, writeFile, rename } from 'node:fs/promises';
import type { SignatureFieldData } from '@/lib/db/schema';

/**
 * Fills AcroForm text fields and draws signature rectangles on a PDF.
 * Uses atomic write: write to tmp path, verify %PDF header, rename to final path.
 *
 * @param srcPath     - Absolute path to original PDF
 * @param destPath    - Absolute path to write prepared PDF (e.g. {docId}_prepared.pdf)
 * @param textFields  - Key/value map: { fieldNameOrLabel: value } (agent-provided)
 * @param sigFields   - Signature field array from SignatureFieldData[]
 */
export async function preparePdf(
  srcPath: string,
  destPath: string,
  textFields: Record<string, string>,
  sigFields: SignatureFieldData[],
): Promise<void> {
  const pdfBytes = await readFile(srcPath);
  const pdfDoc = await PDFDocument.load(pdfBytes);
  const helvetica = await pdfDoc.embedFont(StandardFonts.Helvetica);
  const pages = pdfDoc.getPages();

  // Strategy A: fill existing AcroForm fields by name
  // form.flatten() MUST happen before drawing rectangles
  try {
    const form = pdfDoc.getForm();
    for (const [fieldName, value] of Object.entries(textFields)) {
      try {
        form.getTextField(fieldName).setText(value);
      } catch {
        // Field name not found in AcroForm — silently skip
      }
    }
    form.flatten();
  } catch {
    // No AcroForm in this PDF — skip to rectangle drawing
  }

  // Draw signature field placeholders (blue rectangle + "Sign Here" label)
  for (const field of sigFields) {
    const page = pages[field.page - 1]; // page is 1-indexed
    if (!page) continue;
    page.drawRectangle({
      x: field.x,
      y: field.y,
      width: field.width,
      height: field.height,
      borderColor: rgb(0.15, 0.39, 0.92),
      borderWidth: 1.5,
      color: rgb(0.90, 0.94, 0.99),
    });
    page.drawText('Sign Here', {
      x: field.x + 4,
      y: field.y + 4,
      size: 8,
      font: helvetica,
      color: rgb(0.15, 0.39, 0.92),
    });
  }

  const modifiedBytes = await pdfDoc.save();

  // Atomic write: tmp → rename (prevents partial writes from corrupting the only copy)
  const tmpPath = `${destPath}.tmp`;
  await writeFile(tmpPath, modifiedBytes);

  // Verify output is a valid PDF (check magic bytes)
  const magic = modifiedBytes.slice(0, 4);
  const header = Buffer.from(magic).toString('ascii');
  if (!header.startsWith('%PDF')) {
    throw new Error('preparePdf: output is not a valid PDF (magic bytes check failed)');
  }

  await rename(tmpPath, destPath);
}
```

**Step C — Create teressa-copeland-homes/src/app/api/documents/[id]/fields/route.ts:**

GET returns the stored signatureFields array. PUT accepts a SignatureFieldData[] body and stores it.

```typescript
import { auth } from '@/lib/auth';
import { db } from '@/lib/db';
import { documents } from '@/lib/db/schema';
import { eq } from 'drizzle-orm';
import type { SignatureFieldData } from '@/lib/db/schema';

export async function GET(
  _req: Request,
  { params }: { params: Promise<{ id: string }> }
) {
  const session = await auth();
  if (!session) return new Response('Unauthorized', { status: 401 });

  const { id } = await params;
  const doc = await db.query.documents.findFirst({ where: eq(documents.id, id) });
  if (!doc) return Response.json({ error: 'Not found' }, { status: 404 });

  return Response.json(doc.signatureFields ?? []);
}

export async function PUT(
  req: Request,
  { params }: { params: Promise<{ id: string }> }
) {
  const session = await auth();
  if (!session) return new Response('Unauthorized', { status: 401 });

  const { id } = await params;
  const fields: SignatureFieldData[] = await req.json();

  const [updated] = await db
    .update(documents)
    .set({ signatureFields: fields })
    .where(eq(documents.id, id))
    .returning();

  if (!updated) return Response.json({ error: 'Not found' }, { status: 404 });
  return Response.json(updated.signatureFields ?? []);
}
```

**Step D — Create teressa-copeland-homes/src/app/api/documents/[id]/prepare/route.ts:**

POST: loads the document, resolves the src PDF, calls preparePdf, stores the preparedFilePath, updates textFillData, assignedClientId, status to 'Sent', and sets sentAt.

```typescript
import { auth } from '@/lib/auth';
import { db } from '@/lib/db';
import { documents } from '@/lib/db/schema';
import { eq } from 'drizzle-orm';
import { preparePdf } from '@/lib/pdf/prepare-document';
import path from 'node:path';

const UPLOADS_DIR = path.join(process.cwd(), 'uploads');

export async function POST(
  req: Request,
  { params }: { params: Promise<{ id: string }> }
) {
  const session = await auth();
  if (!session) return new Response('Unauthorized', { status: 401 });

  const { id } = await params;
  const body = await req.json() as {
    textFillData?: Record<string, string>;
    assignedClientId?: string;
  };

  const doc = await db.query.documents.findFirst({ where: eq(documents.id, id) });
  if (!doc) return Response.json({ error: 'Not found' }, { status: 404 });
  if (!doc.filePath) return Response.json({ error: 'Document has no PDF file' }, { status: 422 });

  const srcPath = path.join(UPLOADS_DIR, doc.filePath);
  // Prepared variant stored next to original with _prepared suffix
  const preparedRelPath = doc.filePath.replace(/\.pdf$/, '_prepared.pdf');
  const destPath = path.join(UPLOADS_DIR, preparedRelPath);

  // Path traversal guard
  if (!destPath.startsWith(UPLOADS_DIR)) {
    return new Response('Forbidden', { status: 403 });
  }

  const sigFields = (doc.signatureFields as import('@/lib/db/schema').SignatureFieldData[]) ?? [];
  const textFields = body.textFillData ?? {};

  await preparePdf(srcPath, destPath, textFields, sigFields);

  const [updated] = await db
    .update(documents)
    .set({
      preparedFilePath: preparedRelPath,
      textFillData: body.textFillData ?? null,
      assignedClientId: body.assignedClientId ?? doc.assignedClientId ?? null,
      status: 'Sent',
      sentAt: new Date(),
    })
    .where(eq(documents.id, id))
    .returning();

  return Response.json(updated);
}
```

**Verify build compiles cleanly after creating all files:**
```bash
cd /Users/ccopeland/temp/red/teressa-copeland-homes && npm run build 2>&1 | tail -20
```
  </action>
  <verify>
    <automated>cd /Users/ccopeland/temp/red/teressa-copeland-homes && npm run build 2>&1 | grep -E "(error|Error|FAILED|compiled successfully)" | head -10</automated>
  </verify>
  <done>
    - @cantoo/pdf-lib in package.json dependencies
    - src/lib/pdf/prepare-document.ts exports preparePdf function
    - GET /api/documents/[id]/fields returns 401 unauthenticated (verifiable via curl)
    - PUT /api/documents/[id]/fields returns 401 unauthenticated
    - POST /api/documents/[id]/prepare returns 401 unauthenticated
    - npm run build completes without TypeScript errors
  </done>
</task>

<task type="auto">
  <name>Task 3: Write unit tests for Y-flip coordinate conversion formula</name>
  <files>
    teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts
  </files>
  <action>
Create the directory if needed and write the test file. This test extracts the coordinate conversion logic (which will also exist inlined in FieldPlacer.tsx) as pure functions so they can be verified in isolation — no PDF loading, no DB, no network.

The coordinate conversion formulas are:
```
pdfY = ((renderedH - screenY) / renderedH) * originalHeight
pdfX = (screenX / renderedW) * originalWidth
```

For a US Letter PDF (612 × 792 pts) rendered at 1:1 scale (renderedW=612, renderedH=792):
- screenY=0 (visual top) → pdfY = ((792-0)/792) * 792 = 792 (high PDF Y = top of PDF space)
- screenY=792 (visual bottom) → pdfY = ((792-792)/792) * 792 = 0 (low PDF Y = bottom of PDF space)
- screenX=0 → pdfX = 0
- screenX=612 → pdfX = 612

Also test at a 50% zoom (renderedW=306, renderedH=396) — scaling must not affect the PDF coordinate output because the formula compensates for scale:
- screenY=0, renderedH=396, originalHeight=792 → pdfY = (396/396) * 792 = 792

Create teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts:

```typescript
/**
 * Unit tests for the Y-flip coordinate conversion formula used in FieldPlacer.tsx.
 *
 * PDF user space: origin at bottom-left, Y increases upward.
 * DOM/screen space: origin at top-left, Y increases downward.
 * Formula must flip the Y axis.
 *
 * US Letter: 612 × 792 pts at 72 DPI.
 */

// Pure conversion functions extracted from the FieldPlacer formula.
// These MUST match what is implemented in FieldPlacer.tsx exactly.
function screenToPdfY(screenY: number, renderedH: number, originalHeight: number): number {
  return ((renderedH - screenY) / renderedH) * originalHeight;
}

function screenToPdfX(screenX: number, renderedW: number, originalWidth: number): number {
  return (screenX / renderedW) * originalWidth;
}

const US_LETTER_W = 612;  // pts
const US_LETTER_H = 792;  // pts

describe('Y-flip coordinate conversion (US Letter 612×792)', () => {
  describe('at 1:1 scale (rendered = original dimensions)', () => {
    const rW = US_LETTER_W;
    const rH = US_LETTER_H;

    test('screenY=0 (visual top) produces pdfY≈792 (PDF top)', () => {
      expect(screenToPdfY(0, rH, US_LETTER_H)).toBeCloseTo(792, 1);
    });

    test('screenY=792 (visual bottom) produces pdfY≈0 (PDF bottom)', () => {
      expect(screenToPdfY(792, rH, US_LETTER_H)).toBeCloseTo(0, 1);
    });

    test('screenY=396 (visual center) produces pdfY≈396 (PDF center)', () => {
      expect(screenToPdfY(396, rH, US_LETTER_H)).toBeCloseTo(396, 1);
    });

    test('screenX=0 produces pdfX=0', () => {
      expect(screenToPdfX(0, rW, US_LETTER_W)).toBeCloseTo(0, 1);
    });

    test('screenX=612 (full width) produces pdfX≈612', () => {
      expect(screenToPdfX(612, rW, US_LETTER_W)).toBeCloseTo(612, 1);
    });

    test('screenX=306 (horizontal center) produces pdfX≈306', () => {
      expect(screenToPdfX(306, rW, US_LETTER_W)).toBeCloseTo(306, 1);
    });
  });

  describe('at 50% zoom (rendered = half original dimensions)', () => {
    const rW = US_LETTER_W / 2; // 306
    const rH = US_LETTER_H / 2; // 396

    test('screenY=0 at 50% zoom still produces pdfY≈792 (scale-invariant)', () => {
      expect(screenToPdfY(0, rH, US_LETTER_H)).toBeCloseTo(792, 1);
    });

    test('screenY=396 (visual bottom at 50% zoom) produces pdfY≈0', () => {
      expect(screenToPdfY(396, rH, US_LETTER_H)).toBeCloseTo(0, 1);
    });

    test('screenY=198 (visual center at 50% zoom) produces pdfY≈396', () => {
      expect(screenToPdfY(198, rH, US_LETTER_H)).toBeCloseTo(396, 1);
    });

    test('screenX=153 (quarter width at 50% zoom) produces pdfX≈306 (half PDF width)', () => {
      expect(screenToPdfX(153, rW, US_LETTER_W)).toBeCloseTo(306, 1);
    });
  });
});
```

No special Jest config should be needed — the project uses ts-jest or similar already (check package.json). If there is no test runner configured, add jest config as follows in package.json:
```json
"jest": {
  "preset": "ts-jest",
  "testEnvironment": "node"
}
```
And install if missing: `npm install --save-dev jest ts-jest @types/jest`
  </action>
  <verify>
    <automated>cd /Users/ccopeland/temp/red/teressa-copeland-homes && npx jest src/lib/pdf/__tests__/prepare-document.test.ts --no-coverage 2>&1 | tail -20</automated>
  </verify>
  <done>
    - src/lib/pdf/__tests__/prepare-document.test.ts exists with 10 test cases
    - All tests pass: npx jest src/lib/pdf/__tests__/prepare-document.test.ts --no-coverage exits 0
    - screenY=0 on a 792pt page produces pdfY≈792 (visual top = high PDF Y confirmed)
    - Tests pass at both 1:1 and 50% zoom to confirm scale-invariance of formula
  </done>
</task>

</tasks>

<verification>
After all tasks complete:
1. `ls /Users/ccopeland/temp/red/teressa-copeland-homes/drizzle/0003_*.sql` — migration file exists
2. `ls /Users/ccopeland/temp/red/teressa-copeland-homes/src/lib/pdf/prepare-document.ts` — utility exists
3. `ls /Users/ccopeland/temp/red/teressa-copeland-homes/src/lib/pdf/__tests__/prepare-document.test.ts` — test file exists
4. `ls /Users/ccopeland/temp/red/teressa-copeland-homes/src/app/api/documents/[id]/fields/route.ts` — route exists
5. `npm run build` — clean compile
6. `npx jest src/lib/pdf/__tests__/prepare-document.test.ts --no-coverage` — all 10 tests pass
7. `curl -s http://localhost:3000/api/documents/test-id/fields` — returns "Unauthorized" (server must be running)
</verification>

<success_criteria>
- Migration 0003 applied: documents table has 4 new nullable columns (signature_fields JSONB, text_fill_data JSONB, assigned_client_id TEXT, prepared_file_path TEXT)
- SignatureFieldData interface exported from schema.ts with id, page, x, y, width, height fields
- @cantoo/pdf-lib installed (NOT the original pdf-lib — they conflict)
- preparePdf utility uses atomic tmp→rename write pattern
- form.flatten() called BEFORE drawing signature rectangles
- GET /api/documents/[id]/fields returns 401 without auth, [] with auth for a document with no fields
- PUT /api/documents/[id]/fields stores the array and returns it
- POST /api/documents/[id]/prepare returns 401 without auth; with auth loads doc, calls preparePdf, transitions status to Sent, creates prepared PDF on disk
- POST /api/documents/[id]/prepare returns 422 if document has no filePath
- Unit tests pass: screenY=0 on 792pt page → pdfY≈792; formula is scale-invariant
- npm run build compiles without errors
</success_criteria>

<output>
After completion, create `.planning/phases/05-pdf-fill-and-field-mapping/05-01-SUMMARY.md`
</output>