red/.planning/research/ARCHITECTURE.md

# Architecture Research

**Domain:** Real estate agent website + PDF document signing web app — v1.1 integration
**Researched:** 2026-03-21
**Confidence:** HIGH

---

> **Scope note:** This document supersedes the v1.0 architecture research. It reflects the *actual* v1.0 codebase (Drizzle ORM, local `uploads/` directory, `@cantoo/pdf-lib`, Auth.js v5) and focuses specifically on how the four v1.1 feature areas integrate with what already exists. The previous research doc described a Prisma + S3 design that was never built — disregard it for implementation.

---

## Existing Architecture (Actual v1.0 State)

The actual implementation diverges from the v1.0 research document. Key facts confirmed by reading the codebase:

| Concern | Actual v1.0 |
|---------|-------------|
| ORM | Drizzle ORM (`drizzle-orm` + `drizzle-kit`) |
| DB | Local PostgreSQL in Docker |
| PDF write | `@cantoo/pdf-lib` (fork of pdf-lib, same API) |
| PDF view | `react-pdf` (pdfjs-dist backed) — client-side, `ssr: false` |
| PDF storage | `uploads/` at project root (never under `public/`) |
| Auth | Auth.js v5 (`next-auth@5.0.0-beta.30`) |
| Field types | Only `signature` — all fields in `signatureFields` JSONB as `SignatureFieldData[]` |
| Text fill | Free-form key/value pairs in `textFillData` JSONB — manual entry only |
| Agent signature | Stored in client `localStorage` (key: `teressa_homes_saved_signature`) — ephemeral |
| Field placement | `FieldPlacer.tsx` — dnd-kit palette + move/resize, persisted to DB via `PUT /api/documents/[id]/fields` |
| Signing flow | Client signs only; agent signature not embedded before sending |
| Preview | No dedicated preview step — agent goes straight from prepare to send |

---

## System Overview — v1.1 Additions

```
┌──────────────────────────────────────────────────────────────────────────────┐
│                        DOCUMENT PREPARE PAGE  /portal/documents/[docId]       │
│                                                                                │
│  ┌────────────────────────────────┐   ┌──────────────────────────────────┐   │
│  │  PdfViewerWrapper (existing)   │   │  PreparePanel (extended)         │   │
│  │  FieldPlacer (extended)        │   │                                  │   │
│  │                                │   │  [AI Auto-place] NEW             │   │
│  │  + new field type tokens in    │   │  [Text fill (AI pre-filled)] EXT │   │
│  │    palette: text, checkbox,    │   │  [Agent signature] NEW           │   │
│  │    initials, date, agent-sig   │   │  [Filled preview] NEW            │   │
│  │                                │   │  [Send] existing                 │   │
│  └────────────────────────────────┘   └──────────────────────────────────┘   │
│                                                                                │
│  NEW API Routes:                       NEW DB Columns:                         │
│  POST /api/documents/[id]/ai-prepare   users.agentSignatureData (text)        │
│  GET/PUT /api/agent/signature          clients.propertyAddress (text)         │
│  GET /api/documents/[id]/preview       signatureFields schema extended        │
│                                        (type discriminant on DocumentField)   │
└──────────────────────────────────────────────────────────────────────────────┘
             │                                        │
             ▼                                        ▼
┌────────────────────────┐            ┌──────────────────────────────────────┐
│  lib/ai/               │            │  lib/pdf/                            │
│  extract-text.ts  NEW  │            │  prepare-document.ts  (extended)     │
│  field-placement.ts NEW│            │  preview-document.ts  NEW            │
└────────────────────────┘            └──────────────────────────────────────┘
             │                                        │
             ▼                                        ▼
┌────────────────────────┐            ┌──────────────────────────────────────┐
│  OpenAI gpt-4o-mini    │            │  @cantoo/pdf-lib                     │
│  (server-only)         │            │  (existing, unchanged)               │
└────────────────────────┘            └──────────────────────────────────────┘
```

---

## Integration 1: OpenAI PDF Text Extraction and Field Placement

### Question answered: Which extraction library? What prompt structure? Multi-page handling?

### Extraction Library: pdfjs-dist (already installed)

`react-pdf` depends on `pdfjs-dist`, which is already in `node_modules`. Use it server-side for text extraction rather than adding a new dependency.

The correct server-side import uses the **legacy build** to avoid worker/canvas requirements:

```typescript
// lib/ai/extract-text.ts  (NEW FILE — server-only)
import * as pdfjsLib from 'pdfjs-dist/legacy/build/pdf.mjs';
import { readFile } from 'node:fs/promises';

export async function extractPdfText(filePath: string): Promise<string> {
  const data = new Uint8Array(await readFile(filePath));
  const pdf = await pdfjsLib.getDocument({ data }).promise;

  const pages: string[] = [];
  for (let i = 1; i <= pdf.numPages; i++) {
    const page = await pdf.getPage(i);
    const content = await page.getTextContent();
    const pageText = content.items
      .filter((item): item is { str: string } => 'str' in item)
      .map((item) => item.str)
      .join(' ');
    pages.push(`[Page ${i}]\n${pageText}`);
  }
  return pages.join('\n\n');
}
```

**Why not `pdf-parse`?** It wraps pdfjs-dist but adds a dependency that would be redundant. Use pdfjs-dist directly since it is already installed.

**Multi-page handling:** Extract all pages, prefix each with `[Page N]` so OpenAI can reference page numbers when placing fields. For real estate forms (typically 8–20 pages), total token count will be 2,000–8,000 tokens — well within gpt-4o-mini's 128k context window.

### OpenAI API Route: Server-Only, NOT a Server Action

Use a dedicated API route at `POST /api/documents/[id]/ai-prepare` for the following reasons:
1. The operation is long-running (OpenAI call + text extraction). Server Actions are better for quick mutations.
2. The route needs to return structured JSON (field placements + prefill data) that the client then applies.
3. The existing pattern in this codebase is API routes for document operations.

```typescript
// app/api/documents/[id]/ai-prepare/route.ts  (NEW FILE)
import { auth } from '@/lib/auth';
import { db } from '@/lib/db';
import { documents, clients } from '@/lib/db/schema';
import { eq } from 'drizzle-orm';
import path from 'node:path';
import { extractPdfText } from '@/lib/ai/extract-text';
import { callFieldPlacementAI } from '@/lib/ai/field-placement';

const UPLOADS_DIR = path.join(process.cwd(), 'uploads');

export async function POST(
  _req: Request,
  { params }: { params: Promise<{ id: string }> }
) {
  const session = await auth();
  if (!session) return new Response('Unauthorized', { status: 401 });

  const { id } = await params;
  const doc = await db.query.documents.findFirst({
    where: eq(documents.id, id),
    with: { client: true },
  });
  if (!doc || !doc.filePath) return Response.json({ error: 'Not found' }, { status: 404 });

  const srcPath = path.join(UPLOADS_DIR, doc.filePath);
  const pdfText = await extractPdfText(srcPath);

  const result = await callFieldPlacementAI(pdfText, {
    clientName: doc.client?.name ?? '',
    clientEmail: doc.client?.email ?? '',
    propertyAddress: doc.client?.propertyAddress ?? '',
  });

  return Response.json(result);
}
```

### Prompt Structure

Use OpenAI's Structured Outputs with `response_format: { type: "json_schema", strict: true }` to guarantee the schema. This eliminates validation and retry loops.

```typescript
// lib/ai/field-placement.ts  (NEW FILE)
// This file must NEVER be imported from client components.
// Place in lib/ (not app/) to enforce server-side execution.

interface ClientContext {
  clientName: string;
  clientEmail: string;
  propertyAddress: string;
}

export interface AiFieldPlacement {
  type: 'text' | 'checkbox' | 'initials' | 'date' | 'client-signature';
  label: string;       // human label, becomes the key in textFillData
  page: number;        // 1-indexed
  xPct: number;        // 0–100, percentage of page width from left
  yPct: number;        // 0–100, percentage of page height from bottom (PDF coords)
  prefillValue?: string; // AI-suggested value for text/date fields
}

export interface AiPrepareResult {
  fields: AiFieldPlacement[];
  prefillData: Record<string, string>;  // label → value for known fields
}

const FIELD_PLACEMENT_SCHEMA = {
  type: 'object',
  properties: {
    fields: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          type: { type: 'string', enum: ['text', 'checkbox', 'initials', 'date', 'client-signature'] },
          label: { type: 'string' },
          page: { type: 'integer' },
          xPct: { type: 'number' },
          yPct: { type: 'number' },
          prefillValue: { type: 'string' },
        },
        required: ['type', 'label', 'page', 'xPct', 'yPct'],
        additionalProperties: false,
      },
    },
    prefillData: {
      type: 'object',
      additionalProperties: { type: 'string' },
    },
  },
  required: ['fields', 'prefillData'],
  additionalProperties: false,
};

export async function callFieldPlacementAI(
  pdfText: string,
  context: ClientContext
): Promise<AiPrepareResult> {
  const systemPrompt = `You are a Utah real estate document assistant.
Analyze the provided PDF text and identify where form fields should be placed.
For each field, estimate its position as a percentage (0-100) of page width (xPct)
and page height from the bottom (yPct, because PDF coords start at bottom-left).
Pre-fill fields where you can infer the value from the client context provided.
Only place fields where blank lines, underscores, or form labels indicate input is expected.`;

  const userPrompt = `Client name: ${context.clientName}
Client email: ${context.clientEmail}
Property address: ${context.propertyAddress || 'unknown'}

PDF text content:
${pdfText}

Identify all form fields that need to be filled or signed.
For text fields, prefill any you can infer from the client context.
Return results as JSON matching the specified schema.`;

  const response = await fetch('https://api.openai.com/v1/chat/completions', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: 'gpt-4o-mini',
      messages: [
        { role: 'system', content: systemPrompt },
        { role: 'user', content: userPrompt },
      ],
      response_format: {
        type: 'json_schema',
        json_schema: {
          name: 'field_placement',
          schema: FIELD_PLACEMENT_SCHEMA,
          strict: true,
        },
      },
      temperature: 0.2,  // lower temperature for more deterministic placement
    }),
  });

  const data = await response.json();
  return JSON.parse(data.choices[0].message.content) as AiPrepareResult;
}
```

**Coordinate conversion note:** AI returns percentage-based coordinates (`xPct`, `yPct`) rather than absolute PDF points because the AI cannot know page dimensions from text alone. The conversion from percentages to PDF user-space points happens in the API route after reading the PDF with pdfjs-dist to get actual page dimensions.

**Multi-page strategy:** Send all pages in one request (prefixed `[Page N]`). For real estate forms, the full text fits in 8k tokens. Do NOT split into multiple requests — the AI needs the full document context to understand which pages need signatures vs. text fields.

---

## Integration 2: Extending signatureFields JSONB Schema

### Question answered: How to add field types without breaking existing client signature functionality?

### Analysis of Risk

The existing signing flow in `src/app/api/sign/[token]/route.ts` (POST handler, step 8) reads `signatureFields` and maps client-supplied `dataURL` values to each field using `field.id`. It then calls `embedSignatureInPdf` which draws an image at `field.x`, `field.y`, `field.width`, `field.height`, `field.page`.

The critical invariant: **the client signing page must know which fields require a drawn signature (canvas) versus which are already filled (text/checkbox/date/agent-sig).**

### Schema Extension Strategy: Discriminated Union with Backward Compatibility

Extend `SignatureFieldData` by adding an optional `type` property. When `type` is absent or `'client-signature'`, existing behavior is preserved exactly. All field types share the same geometry properties.

```typescript
// src/lib/db/schema.ts — MODIFY existing SignatureFieldData

// NEW: base interface — shared geometry, all field types
interface BaseFieldData {
  id: string;
  page: number;     // 1-indexed
  x: number;        // PDF user space, bottom-left origin, points
  y: number;
  width: number;
  height: number;
}

// NEW: discriminated union
export type DocumentField =
  | (BaseFieldData & { type?: 'client-signature' })  // type omitted = client-sig (backward compat)
  | (BaseFieldData & { type: 'text';         label: string; value?: string })
  | (BaseFieldData & { type: 'checkbox';     label: string; checked?: boolean })
  | (BaseFieldData & { type: 'initials';     label: string })        // client initials
  | (BaseFieldData & { type: 'date';         label: string; value?: string })
  | (BaseFieldData & { type: 'agent-signature'; value?: string })    // PNG dataURL, embedded before send

// Keep old name as alias for migration safety
export type SignatureFieldData = DocumentField;
```

**DB column:** No migration needed for the JSONB column itself. JSONB accepts any JSON; the schema change is TypeScript-only. The existing `signatureFields jsonb` column in `documents` stores the extended array.

**Backward compatibility rule:** Any `DocumentField` where `type` is `undefined` or `'client-signature'` is treated identically to the original `SignatureFieldData`. The existing `FieldPlacer.tsx` creates fields without `type` — those continue to work as client signature fields.

### Client Signing Page — Filter to Client-Only Fields

The client signing page (`SigningPageClient.tsx`) currently iterates `signatureFields` and presents every field for signature. With the extended schema, it must only present `client-signature` and `initials` fields:

```typescript
// src/app/sign/[token]/_components/SigningPageClient.tsx — MODIFY
// Filter to fields the client needs to interact with:
const clientFields = signatureFields.filter(
  (f) => !f.type || f.type === 'client-signature' || f.type === 'initials'
);
```

**What the client signing page does NOT change:**
- The `POST /api/sign/[token]` route uses `doc.signatureFields` from the DB and server-stored coordinates. Text/checkbox/date/agent-sig fields are already baked into the prepared PDF by the time the client signs. The signing API should filter `signatureFields` the same way — only embed images for `client-signature`/`initials` fields.

### FieldPlacer.tsx — Add New Field Type Tokens to Palette

Extend the palette with new draggable tokens, each creating a `DocumentField` with the appropriate `type`. The existing drag-drop, move, resize, and persist logic does not change — it operates on the shared geometry properties.

```typescript
// Palette additions in FieldPlacer.tsx — add to the palette row:
<DraggableToken id="text-token"      label="Text Field"    fieldType="text" />
<DraggableToken id="checkbox-token"  label="Checkbox"      fieldType="checkbox" />
<DraggableToken id="initials-token"  label="Initials"      fieldType="initials" />
<DraggableToken id="date-token"      label="Date"          fieldType="date" />
<DraggableToken id="agent-sig-token" label="Agent Sig"     fieldType="agent-signature" />
// Keep existing:
<DraggableToken id="signature-token" label="+ Signature"   fieldType="client-signature" />
```

### prepare-document.ts — Handle All Field Types

The existing `preparePdf` function draws a blue rectangle + "Sign Here" for every entry in `sigFields`. With extended types, it needs type-aware rendering:

- `client-signature` / no type: existing blue rectangle + "Sign Here" label (unchanged)
- `text` with `value`: stamp the value directly (use AcroForm fill if name matches, else drawText)
- `date` with `value`: stamp the date text
- `checkbox` with `checked`: draw checkmark glyph or an X
- `agent-signature` with `value` (dataURL): embed the PNG image (same logic as `embedSignatureInPdf`)
- `initials`: blue rectangle + "Initials" label

---

## Integration 3: Agent Saved Signature Persistence

### Question answered: DB BYTEA/text column vs file on disk? How served?

### Decision: `text` column on the `users` table (base64 PNG dataURL)

**Recommendation: Store as `TEXT` in the `users` table** — not BYTEA, not a file on disk.

**Rationale:**

1. **Size:** A signature drawn on a 400×140px canvas is typically 2–8 KB as a PNG dataURL string. PostgreSQL's 33% size penalty for base64 is negligible at this scale (8 KB becomes ~11 KB).

2. **Access pattern:** The signature is always fetched alongside an authenticated agent session. A single-row `SELECT` by user ID returns it immediately. No streaming, no presigned URLs.

3. **Existing stack:** The codebase already stores binary-ish data as JSONB text (`signatureFields` containing base64 in `embedSignatureInPdf`). Base64 `data:` URLs are the native format of `signature_pad.toDataURL()` and `canvas.toDataURL()` — no conversion needed.

4. **File on disk:** Rejected. Files on disk create path management complexity, require auth-gated API routes to serve, and must survive container restarts. The `uploads/` pattern works for documents (immutable blobs) but is overkill for a single small image per user.

5. **BYTEA:** Rejected. Drizzle ORM's BYTEA support requires additional type handling. The dataURL string is already the right format for `@cantoo/pdf-lib`'s `embedPng()` — no conversion needed.

### DB Migration

```typescript
// src/lib/db/schema.ts — ADD to users table:
export const users = pgTable('users', {
  id: text('id').primaryKey().$defaultFn(() => crypto.randomUUID()),
  email: text('email').notNull().unique(),
  passwordHash: text('password_hash').notNull(),
  agentSignatureData: text('agent_signature_data'),   // NEW: base64 PNG dataURL or null
  createdAt: timestamp('created_at').defaultNow().notNull(),
});
```

Run: `npm run db:generate && npm run db:migrate`

### API Routes for Agent Signature

```
GET  /api/agent/signature   — returns { dataURL: string | null }
PUT  /api/agent/signature   — body: { dataURL: string }, saves to DB
```

```typescript
// app/api/agent/signature/route.ts  (NEW FILE)
import { auth } from '@/lib/auth';
import { db } from '@/lib/db';
import { users } from '@/lib/db/schema';
import { eq } from 'drizzle-orm';

export async function GET() {
  const session = await auth();
  if (!session?.user?.id) return new Response('Unauthorized', { status: 401 });

  const user = await db.query.users.findFirst({
    where: eq(users.id, session.user.id),
    columns: { agentSignatureData: true },
  });
  return Response.json({ dataURL: user?.agentSignatureData ?? null });
}

export async function PUT(req: Request) {
  const session = await auth();
  if (!session?.user?.id) return new Response('Unauthorized', { status: 401 });

  const { dataURL } = await req.json() as { dataURL: string };
  // Validate: must be a PNG dataURL
  if (!dataURL.startsWith('data:image/png;base64,')) {
    return Response.json({ error: 'Invalid signature format' }, { status: 400 });
  }

  await db.update(users)
    .set({ agentSignatureData: dataURL })
    .where(eq(users.id, session.user.id));

  return Response.json({ ok: true });
}
```

### How Agent Signature Is Applied

When the agent places an `agent-signature` field and has a saved signature, `PreparePanel` sends the `agentSignatureData` as the `value` on that field when calling `POST /api/documents/[id]/prepare`. The `prepare-document.ts` function embeds it as a PNG image at the field's coordinates — exactly the same logic as `embedSignatureInPdf`.

**The agent signature is embedded during prepare, before the document is sent to the client.** The client sees the agent's signature already in the PDF as a real image, not a placeholder rectangle.

---

## Integration 4: Filled Preview Approach

### Question answered: Re-render via react-pdf with overlaid values, or generate a new temporary PDF server-side?

### Decision: Generate a temporary prepared PDF server-side; render with existing react-pdf viewer

**Rejected approach: Client-side overlay rendering**

Overlaying text/checkmarks on top of a react-pdf canvas in the browser is fragile. Text positions must be pixel-perfect, and the coordinate math between PDF user space and screen pixels is already complex (demonstrated by the existing `FieldPlacer.tsx`). Overlaid values would not be "in" the PDF — they would be CSS layers that look different from the final embedded result.

**Recommended approach: Reuse `preparePdf` in preview-only mode**

The existing `preparePdf` function already generates a complete prepared PDF from the source PDF + text values + field geometries. For preview, call the same function but write to a temporary path, then serve it through the existing `/api/documents/[id]/file` pattern.

```
PreparePanel clicks "Preview"
    │
    ▼
POST /api/documents/[id]/preview
    │ (auth-gated, agent only)
    │ body: { textFillData, fields (DocumentField[]) }
    ▼
preparePdf(srcPath, tmpPath, textFillData, fields)
    │ (same function, new tmp destination)
    ▼
Response: streams the tmp PDF bytes directly
    │ (or writes to uploads/{docId}_preview.pdf, serves via file route)
    ▼
PreparePanel opens preview in a modal with <Document> from react-pdf
    (same PdfViewerWrapper pattern — ssr: false)
```

**Preview route:**

```typescript
// app/api/documents/[id]/preview/route.ts  (NEW FILE)
import { auth } from '@/lib/auth';
import { db } from '@/lib/db';
import { documents } from '@/lib/db/schema';
import { eq } from 'drizzle-orm';
import { preparePdf } from '@/lib/pdf/prepare-document';
import { readFile } from 'node:fs/promises';
import path from 'node:path';
import type { DocumentField } from '@/lib/db/schema';

const UPLOADS_DIR = path.join(process.cwd(), 'uploads');

export async function POST(
  req: Request,
  { params }: { params: Promise<{ id: string }> }
) {
  const session = await auth();
  if (!session) return new Response('Unauthorized', { status: 401 });

  const { id } = await params;
  const body = await req.json() as {
    textFillData?: Record<string, string>;
    fields?: DocumentField[];
  };

  const doc = await db.query.documents.findFirst({ where: eq(documents.id, id) });
  if (!doc?.filePath) return Response.json({ error: 'Not found' }, { status: 404 });

  const srcPath = path.join(UPLOADS_DIR, doc.filePath);
  const previewPath = path.join(UPLOADS_DIR, doc.filePath.replace(/\.pdf$/, '_preview.pdf'));

  // Path traversal guard
  if (!previewPath.startsWith(UPLOADS_DIR)) return new Response('Forbidden', { status: 403 });

  await preparePdf(srcPath, previewPath, body.textFillData ?? {}, body.fields ?? []);

  const pdfBytes = await readFile(previewPath);
  return new Response(pdfBytes, {
    headers: {
      'Content-Type': 'application/pdf',
      'Content-Disposition': 'inline',
      'Cache-Control': 'no-store',
    },
  });
}
```

**PreparePanel preview modal:** Add a "Preview" button that calls `POST /api/documents/[id]/preview` with current field + text-fill state, receives the PDF bytes, converts to an object URL via `URL.createObjectURL`, and opens a modal containing `<Document file={objectUrl}>` from react-pdf. Uses the same `PdfViewerWrapper` pattern (dynamic import, `ssr: false`).

**Why not stream PDF to a new browser tab:** Object URL in a modal keeps the preview in-app and avoids browser popup blockers. The agent can review without leaving the prepare page.

**Preview file persistence:** The `_preview.pdf` file is overwritten each time the agent clicks Preview. It is not stored in the DB and is never sent to the client. It can be cleaned up on a schedule or simply overwritten on each preview request.

---

## Data Flow Changes

### New Flow: AI Auto-Prepare

```
Agent clicks "AI Auto-place"
    │
    ▼
Client: POST /api/documents/[id]/ai-prepare
    │
    ▼
Server: extractPdfText(filePath) — pdfjs-dist legacy build
    │
    ▼
Server: callFieldPlacementAI(text, clientContext) — OpenAI gpt-4o-mini
    │                                                 structured output
    ▼
Server: convert xPct/yPct → PDF points using page dimensions
    │
    ▼
Response: { fields: DocumentField[], prefillData: Record<string, string> }
    │
    ▼
Client: setFields(aiFields) + setTextFillData(prefillData)
    │   — renders in FieldPlacer (existing render path)
    ▼
Agent adjusts if needed, then saves via existing PUT /api/documents/[id]/fields
```

### New Flow: Agent Signature Draw & Save

```
Agent opens "My Signature" panel (new section in PreparePanel or sidebar)
    │
    ▼
Agent draws signature on SignaturePad canvas (reuse SignatureModal pattern)
    │
    ▼
Agent clicks "Save Signature"
    │
    ▼
Client: PUT /api/agent/signature { dataURL }
    │
    ▼
Server: UPDATE users SET agent_signature_data = ? WHERE id = session.user.id
    │
    ▼
Response: { ok: true }
    │
    ▼
Agent can now place "Agent Sig" tokens in FieldPlacer — on prepare, the saved
dataURL is fetched and embedded as a PNG image at those field coordinates.
```

### Modified Flow: Prepare (Extended)

```
Agent clicks "Prepare and Send" (existing button)
    │
    ▼
PreparePanel: POST /api/documents/[id]/prepare
    body: {
      textFillData: Record<string, string>,   // existing
      emailAddresses: string[],               // existing
      fields: DocumentField[],                // MODIFIED: full typed field array
    }
    │
    ▼
prepare/route.ts: fetch agent signature if any agent-sig fields present
    │  GET users.agentSignatureData WHERE id = session.user.id
    │
    ▼
preparePdf(srcPath, destPath, textFillData, fields, agentSigDataURL)
    │  — extended to handle all field types
    │
    ▼
[existing] DB update, audit log, redirect
```

---

## Component Boundaries

### New Files (create from scratch)

| File | Type | Purpose |
|------|------|---------|
| `src/lib/ai/extract-text.ts` | Server-only lib | pdfjs-dist text extraction for OpenAI input |
| `src/lib/ai/field-placement.ts` | Server-only lib | OpenAI structured output call, prompt, schema |
| `src/app/api/documents/[id]/ai-prepare/route.ts` | API route (auth-gated) | Orchestrates extract + AI call + coord conversion |
| `src/app/api/documents/[id]/preview/route.ts` | API route (auth-gated) | Calls preparePdf in preview mode, streams bytes |
| `src/app/api/agent/signature/route.ts` | API route (auth-gated) | GET/PUT agent saved signature from DB |
| `src/app/portal/(protected)/documents/[docId]/_components/AgentSignaturePanel.tsx` | Client component | Draw/save/display agent signature; calls PUT /api/agent/signature |
| `src/app/portal/(protected)/documents/[docId]/_components/PreviewModal.tsx` | Client component | Modal wrapping react-pdf Document for preview display |

### Modified Files (targeted additions only)

| File | Change | Constraint |
|------|--------|------------|
| `src/lib/db/schema.ts` | Add `DocumentField` discriminated union; add `agentSignatureData` to users table; add `propertyAddress` to clients table | Keep `SignatureFieldData` as type alias — zero breaking changes |
| `src/lib/pdf/prepare-document.ts` | Add type-aware rendering for text/checkbox/date/agent-sig/initials fields | Existing signature field path (no type / `client-signature`) must behave identically |
| `src/app/api/documents/[id]/fields/route.ts` | Accept `DocumentField[]` (union type) instead of `SignatureFieldData[]` — structurally identical, TypeScript type change only | No behavior change |
| `src/app/api/documents/[id]/prepare/route.ts` | Fetch agent signature from users table if any agent-sig fields present | Must remain backward-compatible with no-fields body |
| `src/app/portal/(protected)/documents/[docId]/_components/FieldPlacer.tsx` | Add new draggable tokens to palette (text, checkbox, initials, date, agent-sig); render type-specific labels and colors for placed fields | Do NOT change the drag/drop/move/resize/persist mechanics |
| `src/app/portal/(protected)/documents/[docId]/_components/PreparePanel.tsx` | Add "AI Auto-place" button + loading state; add "Preview" button + PreviewModal; add AgentSignaturePanel; connect new API routes | Do NOT change existing "Prepare and Send" flow |
| `src/app/sign/[token]/_components/SigningPageClient.tsx` | Filter signatureFields to client-interaction types only (`client-signature`, `initials`, no-type) | Do NOT change submit logic or embed-signature flow |
| `src/app/api/sign/[token]/route.ts` (POST) | Filter signatureFields to client-sig/initials before building `signaturesWithCoords` | Other field types are already baked into the prepared PDF |
| `src/app/portal/_components/ClientModal.tsx` | Add `propertyAddress` field | Straightforward form field addition |
| `src/lib/actions/clients.ts` | Add propertyAddress to create/update mutations | Simple Drizzle update |

### Files to Leave Completely Unchanged

| File | Reason |
|------|--------|
| `src/lib/signing/embed-signature.ts` | Agent signature embedding reuses this logic, but via `prepare-document.ts` not here. Client signing path is unchanged. |
| `src/lib/signing/audit.ts` | No new audit event types needed for v1.1 |
| `src/lib/signing/token.ts` | Signing token flow unchanged |
| `src/app/sign/[token]/_components/SignatureModal.tsx` | Client signing modal unchanged |
| `src/app/api/sign/[token]/download/route.ts` | Download route unchanged |
| `src/app/api/documents/[id]/send/route.ts` | Send flow unchanged |
| All public marketing pages | No changes to homepage, contact form, listings |

---

## Database Schema Changes

```
-- Migration: v1.1 additions
-- Generated by: npm run db:generate after schema.ts changes

ALTER TABLE "users"
  ADD COLUMN IF NOT EXISTS "agent_signature_data" text;          -- base64 PNG dataURL

ALTER TABLE "clients"
  ADD COLUMN IF NOT EXISTS "property_address" text;              -- for AI pre-fill

-- No migration needed for documents.signature_fields JSONB —
-- new field types are stored in existing column, backward-compatible
```

**Migration safety:** Both new columns are nullable with no default, so existing rows are unaffected. The `signatureFields` JSONB change requires no migration — JSONB stores arbitrary JSON.

---

## Coordinate Conversion: AI Output to PDF Points

The AI returns `xPct` / `yPct` (0–100). The API route must convert to PDF user-space points before returning to the client or writing to DB.

```typescript
// Inside ai-prepare/route.ts, after AI call:
import * as pdfjsLib from 'pdfjs-dist/legacy/build/pdf.mjs';

const pdfDoc = await pdfjsLib.getDocument({ data: new Uint8Array(await readFile(srcPath)) }).promise;
const convertedFields: DocumentField[] = [];

for (const aiField of result.fields) {
  const page = await pdfDoc.getPage(aiField.page);
  const viewport = page.getViewport({ scale: 1.0 });
  const pageW = viewport.width;   // PDF points
  const pageH = viewport.height;

  // Default dimensions by type
  const defaultW = aiField.type === 'checkbox' ? 14 : aiField.type === 'initials' ? 72 : 144;
  const defaultH = aiField.type === 'checkbox' ? 14 : 28;

  convertedFields.push({
    id: crypto.randomUUID(),
    type: aiField.type,
    label: aiField.label,
    page: aiField.page,
    x: (aiField.xPct / 100) * pageW,
    y: (aiField.yPct / 100) * pageH,
    width: defaultW,
    height: defaultH,
    ...(aiField.prefillValue ? { value: aiField.prefillValue } : {}),
  });
}
```

---

## Build Order for v1.1

Dependencies flow in this order. Each item can only start after its prerequisites.

```
Step 1 — Schema foundation (no deps within v1.1)
├── Extend DocumentField union in schema.ts
├── Add agentSignatureData to users table
├── Add propertyAddress to clients table
└── Run db:generate + db:migrate

Step 2 — Agent signature persistence (depends on Step 1)
├── GET/PUT /api/agent/signature route
└── AgentSignaturePanel component (draw + save + display)

Step 3 — New field types in FieldPlacer (depends on Step 1)
├── Extend FieldPlacer palette with 5 new token types
├── Update field rendering (type-specific colors/labels)
└── Update PUT /api/documents/[id]/fields to accept DocumentField[]

Step 4 — Extended prepare-document.ts (depends on Step 1, 2, 3)
├── Add type-aware rendering in preparePdf
├── Handle agent-sig field type: fetch + embed PNG
└── Update POST /api/documents/[id]/prepare to pass agent sig

Step 5 — Client signing page filter (depends on Step 3)
├── Filter signatureFields to client-interaction types in SigningPageClient
└── Filter in POST /api/sign/[token] before embedSignatureInPdf

Step 6 — Preview (depends on Step 4)
├── POST /api/documents/[id]/preview route
└── PreviewModal component + "Preview" button in PreparePanel

Step 7 — AI auto-place (depends on Step 1, 3)
├── lib/ai/extract-text.ts
├── lib/ai/field-placement.ts
├── POST /api/documents/[id]/ai-prepare route
└── "AI Auto-place" button + client-side field application in PreparePanel

Step 8 — Property address for AI (depends on Step 1)
├── Add propertyAddress field to ClientModal
└── Update clients server action to save propertyAddress
```

**Why this order:**
- Steps 1–3 establish the data foundation that everything else reads from.
- Step 4 (prepare) depends on knowing what all field types look like.
- Step 5 (client signing filter) must happen before Step 4 ships, otherwise clients see agent-sig/text fields as signature prompts.
- Step 6 (preview) depends on the extended prepare function being complete.
- Step 7 (AI) depends on field types being placeable and the FieldPlacer accepting them.

---

## Anti-Patterns to Avoid

### 1. Exposing OPENAI_API_KEY to the client

Never call the OpenAI API from a Client Component or import `lib/ai/*.ts` in a component. All OpenAI calls must go through `POST /api/documents/[id]/ai-prepare`. Add a `'server-only'` import to `lib/ai/field-placement.ts` to get a build error if accidentally imported on the client.

### 2. Storing agent signature in localStorage only (current v1.0 state)

localStorage is ephemeral (cleared on browser data wipe), not shared across devices, and not available server-side during prepare. Keeping the current localStorage fallback is fine as a UX shortcut during the signing session, but the source of truth must be the DB.

### 3. Changing the `embedSignatureInPdf` signing flow for client signatures

The client signing flow embeds signatures from client-drawn dataURLs. Do not modify `embed-signature.ts` or the POST `/api/sign/[token]` logic to handle new field types — handle agent-sig and text during `preparePdf` only. The signing route should filter fields before calling embed.

### 4. Making preview a full "saved" prepared file

The `_preview.pdf` file is ephemeral and not recorded in the DB. Do not confuse it with `preparedFilePath`. If the agent proceeds to send after previewing, the actual `POST /api/documents/[id]/prepare` generates a fresh `_prepared.pdf` as before. Preview is read-only and stateless.

### 5. Using AI field placement as authoritative without agent review

AI placement is a starting point. The "AI Auto-place" button fills the FieldPlacer with suggested fields, but the agent must be able to adjust before the fields are committed to the DB. Coordinates from the AI response should populate the client-side field state, not directly write to DB.

### 6. Skipping path traversal guard on new preview route

The preview route writes a file to `uploads/`. Apply the same `destPath.startsWith(UPLOADS_DIR)` guard used in the prepare route.

### 7. Using `pdf-parse` as an additional dependency

`pdfjs-dist` is already installed (dependency of `react-pdf`). Use the legacy build server-side. Adding `pdf-parse` would be a duplicate dependency with no benefit.

---

## Integration Points Summary

| New Feature | Touches | Does NOT Touch |
|-------------|---------|----------------|
| AI auto-place | lib/ai/* (new), ai-prepare route (new), FieldPlacer (palette only), PreparePanel (button only) | Signing flow, embed-signature, FieldPlacer drag logic |
| New field types | schema.ts (type union), FieldPlacer (palette + render), prepare-document.ts (type switch), sign route (filter) | Signature modal, signing token, audit log |
| Agent signature | users table (new column), signature route (new), PreparePanel (new panel), prepare-document.ts (embed agent sig) | Client signing, embed-signature.ts, signing token |
| Filled preview | preview route (new), PreviewModal (new), PreparePanel (button only), prepare-document.ts (reused as-is) | Prepare/send flow, signing flow, DB records |

---

## Sources

- [pdfjs-dist legacy build — Node.js text extraction](https://lirantal.com/blog/how-to-read-and-parse-pdfs-pdfjs-create-pdfs-pdf-lib-nodejs)
- [unpdf vs pdf-parse vs pdfjs-dist — 2026 comparison](https://www.pkgpulse.com/blog/unpdf-vs-pdf-parse-vs-pdfjs-dist-pdf-parsing-extraction-nodejs-2026)
- [OpenAI Structured Outputs — official docs](https://platform.openai.com/docs/guides/structured-outputs)
- [Introducing Structured Outputs in the API](https://openai.com/index/introducing-structured-outputs-in-the-api/)
- [Next.js 15 Server Actions vs API Routes — 2025 patterns](https://medium.com/@sparklewebhelp/server-actions-in-next-js-the-future-of-api-routes-06e51b22a59f)
- [PostgreSQL BYTEA vs TEXT for image storage](https://www.postgrespro.com/list/thread-id/1509166)
- [Drizzle ORM — add column migration](https://orm.drizzle.team/docs/drizzle-kit-generate)
- [react-pdf npm — v10 (current)](https://www.npmjs.com/package/react-pdf)

---

*Architecture research for: v1.1 Smart Document Preparation — integration with existing Next.js 15 app*
*Researched: 2026-03-21*