# Phase 8: Schema Foundation and Signing Page Safety - Research

**Researched:** 2026-03-21
**Domain:** TypeScript discriminated union schema extension, Drizzle ORM migration, Next.js API route filtering, React client component type branching
**Confidence:** HIGH

<phase_requirements>
## Phase Requirements

| ID | Description | Research Support |
|----|-------------|-----------------|
| FIELD-01 | Agent can place text field markers on a PDF (for typed content like names, addresses, prices) | Schema type discriminant enables safe introduction of all new field types; signing page filter prevents non-client-sig types from surfacing as required unsigned; migration ensures no data loss on v1.0 documents |
</phase_requirements>

---

## Summary

Phase 8 is a pure safety gate — no user-visible feature is added. It makes the codebase safe to receive the new field types that Phases 9-13 will introduce. The single most dangerous change in v1.1 is adding new values to `SignatureFieldData` while the client signing page and the `/api/sign/[token]` GET route still treat every field as a required client-signature. Two concrete failure modes exist: (1) a client opens a signing session and sees an `agent-signature` field as a required unsigned overlay, panics, and cannot complete signing; (2) the `SigningPageClient` calls `setModalOpen(true)` for a non-drawable field type (text/checkbox/date), producing broken UX before any modal code for those types exists. Both failures are HIGH recovery cost in production — the signing session is broken and cannot self-heal.

The technical work is narrow and well-scoped: add a `type` discriminant to the `SignatureFieldData` TypeScript interface, generate and run a Drizzle migration (the migration is a no-op in SQL terms — JSONB column already exists, no SQL change required for the type discriminant — but a migration for the `agentSignatureData` column on `users` IS needed), apply a backward-compatible runtime fallback (`field.type ?? 'client-signature'`) in every path that reads `signatureFields`, add a type filter in the `/api/sign/[token]` GET route before returning `signatureFields` to the client, and branch `SigningPageClient.tsx` so only `client-signature` types open the signature modal. None of these changes alter any existing user-visible behavior on v1.0 documents because all existing fields have no `type` property and will read as `'client-signature'` via the fallback.

The Drizzle ORM discriminated union type annotation is applied to `.$type<SignatureFieldData[]>()` on the `signatureFields` JSONB column — this is a TypeScript-only annotation with no database DDL change. The only SQL migration needed for Phase 8 is `ALTER TABLE "users" ADD COLUMN "agent_signature_data" TEXT` (needed by Phase 11, but adding it now keeps the schema extension atomic). However, examining the roadmap plan files, the `agentSignatureData` column is planned for Phase 11's migration. Phase 8's migration may therefore be ZERO SQL statements (the discriminant lives in the TypeScript interface only) unless the team decides to pre-add the users column.

**Primary recommendation:** Extend `SignatureFieldData` with `type` discriminant in `schema.ts`, add a backward-compat fallback in all read paths, filter agent-signature fields from the sign GET route, and branch `SigningPageClient.tsx` by field type — all in a single atomic deployment. The Drizzle migration for Phase 8 is likely a zero-SQL migration (TypeScript type change only); generate it anyway to ensure the meta snapshot stays in sync.

---

## Standard Stack

### Core (All Existing — No New Dependencies)

| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| Drizzle ORM | ^0.45.1 | TypeScript schema definition, JSONB type annotation, migration generation | Already in project; `.$type<T>()` method annotates JSONB column TypeScript type without DDL change |
| drizzle-kit | ^0.31.10 | Migration generation (`npm run db:generate`) and migration execution (`npm run db:migrate`) | Already in project; generates 0006_*.sql diff |
| Next.js | 16.2.0 | API route (server-side filter) and React client component (type branching) | Already in project |
| TypeScript | (project version) | Discriminated union type definition | Already in project |
| postgres | ^3.4.8 | Database driver | Already in project |

### No New Dependencies

Phase 8 adds zero new npm packages. All changes are to TypeScript type definitions, runtime logic in existing files, and a Drizzle migration.

**Installation:**
```bash
# No new packages needed
```

---

## Architecture Patterns

### Recommended Project Structure

No new files or directories are created in Phase 8. All changes are modifications to existing files:

```
src/lib/db/schema.ts                              # Modified: SignatureFieldData type + discriminant
src/app/api/sign/[token]/route.ts                 # Modified: filter signatureFields before GET response
src/app/sign/[token]/_components/SigningPageClient.tsx  # Modified: type branching in field render + click handler
drizzle/0006_*.sql                                # New: migration file (may be zero-SQL or users column add)
```

### Pattern 1: TypeScript Discriminated Union on JSONB .$type<T>()

**What:** Add a `type` literal union to the `SignatureFieldData` interface. Drizzle's `.$type<T>()` annotation changes only the TypeScript type of the column — it does NOT generate any SQL DDL change. The JSONB column remains unchanged in the database.

**When to use:** Any time a JSONB column stores an array of objects that need type discrimination at the TypeScript layer without a DB schema change.

**Example:**
```typescript
// src/lib/db/schema.ts — AFTER Phase 8

export type SignatureFieldType =
  | 'client-signature'
  | 'initials'
  | 'text'
  | 'checkbox'
  | 'date'
  | 'agent-signature';

export interface SignatureFieldData {
  id: string;
  page: number;       // 1-indexed
  x: number;         // PDF user space, bottom-left origin, points
  y: number;         // PDF user space, bottom-left origin, points
  width: number;     // PDF points (default: 144 — 2 inches)
  height: number;    // PDF points (default: 36 — 0.5 inches)
  type?: SignatureFieldType;  // Optional for backward compat with v1.0 documents
}

// Helper: always use this to read field type — never read .type directly
export function getFieldType(field: SignatureFieldData): SignatureFieldType {
  return field.type ?? 'client-signature';
}

// Type predicates for common checks
export function isClientVisibleField(field: SignatureFieldData): boolean {
  const t = getFieldType(field);
  return t !== 'agent-signature';
}

// In the documents table — .$type<T> annotation only (no SQL change)
signatureFields: jsonb("signature_fields").$type<SignatureFieldData[]>(),
```

**Key insight:** `type` is declared as `type?: SignatureFieldType` (optional). This is backward-compatible: existing documents with no `type` property in their JSONB fields will NOT fail JSON parsing, and the `getFieldType()` helper coalesces `undefined` to `'client-signature'`.

### Pattern 2: Server-Side Filter Before Response

**What:** Before returning `signatureFields` in the GET `/api/sign/[token]` response, filter to client-visible types only.

**When to use:** Any API route that returns field data to an unauthenticated client (the signing page).

**Exact location:** `/src/app/api/sign/[token]/route.ts` — the current line 88:
```typescript
// BEFORE (line 88 — current, unguarded):
signatureFields: doc.signatureFields ?? [],

// AFTER (Phase 8):
signatureFields: (doc.signatureFields ?? []).filter(isClientVisibleField),
```

This single line is the security-critical change. After this filter, `agent-signature` fields are permanently invisible to the client signing page regardless of what field types future phases add.

### Pattern 3: Type-Branching in SigningPageClient

**What:** The `handleFieldClick` callback and the field overlay render must branch on field type. Non-client-signature types must not open the signature modal.

**When to use:** Any UI component that renders `SignatureFieldData[]` received from the server.

**Example pattern (Phase 8 minimum — no UI changes needed yet):**
```typescript
// In SigningPageClient.tsx

// handleFieldClick — guard against non-modal types
const handleFieldClick = useCallback(
  (fieldId: string) => {
    const field = signatureFields.find((f) => f.id === fieldId);
    if (!field) return;
    const fieldType = getFieldType(field);
    // Only client-signature opens the modal in Phase 8
    // Other types will be handled in Phase 10 (initials, text, etc.)
    if (fieldType !== 'client-signature') return;
    if (signedFields.has(fieldId)) return;
    setActiveFieldId(fieldId);
    setModalOpen(true);
  },
  [signatureFields, signedFields]
);

// handleSubmit — only require signatures for client-signature fields
const handleSubmit = useCallback(async () => {
  const clientSigFields = signatureFields.filter(
    (f) => getFieldType(f) === 'client-signature'
  );
  if (signedFields.size < clientSigFields.length || submitting) return;
  // ... rest of submit unchanged
}, [signatureFields, signedFields.size, submitting, token]);
```

**Note on progress bar:** `SigningProgressBar` receives `total` and `signed` counts. After Phase 8, `total` must be `signatureFields.filter(f => getFieldType(f) === 'client-signature').length` — not `signatureFields.length`. Because the API route now filters out agent-signature fields before sending, the client will never receive them. This is handled transparently. The filter in the GET route means `signatureFields` on the client already only contains client-visible types — so the `SigningPageClient` type branching in Phase 8 is a defense-in-depth guard, not the primary protection.

### Pattern 4: Drizzle Migration for Phase 8

**What:** The TypeScript discriminant requires no SQL DDL change — `signatureFields` is already `jsonb`. The migration generated by `drizzle-kit generate` will reflect the TypeScript change but produce an empty SQL file (or just a comment).

**Important:** Run `npm run db:generate` after updating `schema.ts`. Even if the migration is empty SQL, it updates the Drizzle meta snapshot (the JSON file in `drizzle/meta/`) to stay in sync with the schema. Skipping this causes future `db:generate` runs to produce incorrect diffs.

**The migration SQL will be:**
```sql
-- No DDL change needed for Phase 8 discriminant (JSONB type annotation only)
-- This is an intentionally empty migration to advance the Drizzle snapshot
```

If the team decides to pre-add `agent_signature_data` to `users` (for Phase 11 prep), the migration would be:
```sql
ALTER TABLE "users" ADD COLUMN "agent_signature_data" text;
```

**DECISION NEEDED:** The roadmap plan files assign `agentSignatureData` to Phase 11. Do NOT pre-add it in Phase 8 unless explicitly decided. Keep Phase 8 minimal.

### Anti-Patterns to Avoid

- **Reading `field.type` directly without fallback:** Always use `getFieldType(field)` or `field.type ?? 'client-signature'`. Direct access produces `undefined` on v1.0 documents and TypeScript will not catch this at all entry points.
- **Filtering on the client instead of the server:** The `agent-signature` filter MUST happen in the API route before the response is sent. Filtering only in `SigningPageClient.tsx` is insufficient — an attacker can call the API directly and see field coordinates.
- **Changing `type` from optional to required:** Making `type` required in `SignatureFieldData` breaks TypeScript compilation on every existing call site that constructs a `SignatureFieldData` object (including `FieldPlacer.tsx` `handleDragEnd`). Keep `type` optional; add `getFieldType()` helper for safe reading.
- **Modifying `embed-signature.ts` or the POST `/api/sign/[token]` route:** Phase 8 does NOT touch the signing POST handler or PDF embedding pipeline. Those are working correctly and only handle client-signature fields today.

---

## Don't Hand-Roll

| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| TypeScript type narrowing for JSONB union | Custom type guard per field | `getFieldType()` helper + type predicate functions exported from `schema.ts` | Single source of truth; all consumers import the same fallback logic |
| API field filtering | Field visibility logic inline in route | `isClientVisibleField()` pure function exported from `schema.ts` | Testable, consistent across all routes that return fields to clients |
| Drizzle meta sync | Manual edits to meta JSON | `npm run db:generate` after every schema.ts change | Drizzle meta is machine-generated; manual edits corrupt future diffs |
| Backward-compat runtime coercion | Migration that sets `type = 'client-signature'` on all existing JSONB rows | Optional TypeScript field with fallback in `getFieldType()` | No data migration risk; existing rows work immediately; all new rows get explicit type |

**Key insight:** The entire Phase 8 safety guarantee comes from two pure TypeScript functions (`getFieldType` and `isClientVisibleField`) that are easy to unit-test and can be imported by every consumer. Don't scatter the fallback logic across callsites.

---

## Common Pitfalls

### Pitfall 1: Breaking Active Signing Sessions
**What goes wrong:** Any document that was prepared and sent BEFORE Phase 8 ships has `signatureFields` JSONB with no `type` property on each field object. If Phase 8 changes the TypeScript interface to require `type` (non-optional), TypeScript may compile but the `getFieldType()` fallback handles this correctly — however, if any code path branches on `field.type === 'client-signature'` with a strict equality check and `field.type` is `undefined`, the branch is never taken.
**Why it happens:** Confusion between optional TypeScript field and runtime undefined value.
**How to avoid:** Always coerce via `getFieldType(field)` which returns `field.type ?? 'client-signature'`. Export this function from `schema.ts` so there is only one implementation.
**Warning signs:** TypeScript strict mode warnings on `field.type` comparisons; unit tests on v1.0-style field objects (no type property) failing.

### Pitfall 2: Drizzle Meta Snapshot Out of Sync
**What goes wrong:** After updating `SignatureFieldData` in `schema.ts`, the developer skips `npm run db:generate` because "there's no SQL change." The next `db:generate` run (in Phase 9 or later) produces a diff against the stale meta snapshot and generates incorrect SQL.
**Why it happens:** Drizzle stores the TypeScript schema representation as a JSON snapshot in `drizzle/meta/`. Even TypeScript-only changes to `.$type<T>()` annotations update this snapshot.
**How to avoid:** Always run `npm run db:generate` after any `schema.ts` change, even if you expect zero SQL output. Commit the resulting migration file (even if it has no SQL statements).
**Warning signs:** `drizzle-kit generate` produces a migration that re-adds columns that already exist.

### Pitfall 3: The Filter Is in the Wrong Place
**What goes wrong:** Developer adds the `agent-signature` filter only in `SigningPageClient.tsx` (client component) but leaves the GET `/api/sign/[token]` route returning unfiltered `signatureFields`. The filter works in the browser, but the field coordinates and IDs of agent-signature fields are still leaked to the network.
**Why it happens:** The signing page is the visible failure point, so developers focus on it.
**How to avoid:** Filter in the API route (server-side, line 88) as the primary protection. The `SigningPageClient` type guard is defense-in-depth only.
**Warning signs:** Network tab in browser DevTools shows agent-signature field objects in the GET response JSON.

### Pitfall 4: Breaking the Submit Logic
**What goes wrong:** `handleSubmit` checks `signedFields.size < signatureFields.length`. If an `initials` or other non-signature type is returned in `signatureFields` (which Phase 8's server filter prevents), this check would require the client to "sign" non-signable fields before enabling Submit. Because the filter is in the API route, this is already safe — but if the filter is ever removed or incomplete, Submit becomes permanently disabled.
**Why it happens:** The submit guard counts total fields, not just client-signature fields.
**How to avoid:** Update `handleSubmit` to count only `client-signature` fields in its completeness check: `signatureFields.filter(f => getFieldType(f) === 'client-signature').length`. Even though Phase 8's server filter makes this moot today, it makes the component resilient to future API changes.
**Warning signs:** Submit button stays disabled even after all visible signature fields are completed.

### Pitfall 5: Forgetting the `SigningProgressBar` Total
**What goes wrong:** `SigningProgressBar` is passed `total={signatureFields.length}` as a prop. If for any reason a non-client-signature field appears in the array, the total is inflated and "X of Y signed" shows a count the client can never reach.
**Why it happens:** The progress bar total is passed from `SigningPageClient` which receives the full array.
**How to avoid:** Calculate `total` as `signatureFields.filter(f => getFieldType(f) === 'client-signature').length`. This is currently moot because the API route filters, but is correct defensive coding.
**Warning signs:** Progress bar shows "0 of 3" when only 2 signature fields are visible.

---

## Code Examples

Verified patterns from codebase inspection (2026-03-21):

### Current SignatureFieldData (schema.ts — to be extended)
```typescript
// Source: /src/lib/db/schema.ts (current — no type discriminant)
export interface SignatureFieldData {
  id: string;
  page: number;   // 1-indexed
  x: number;      // PDF user space, bottom-left origin, points
  y: number;      // PDF user space, bottom-left origin, points
  width: number;  // PDF points (default: 144 — 2 inches)
  height: number; // PDF points (default: 36 — 0.5 inches)
}
```

### Extended SignatureFieldData with Discriminant
```typescript
// Source: Phase 8 target schema
export type SignatureFieldType =
  | 'client-signature'
  | 'initials'
  | 'text'
  | 'checkbox'
  | 'date'
  | 'agent-signature';

export interface SignatureFieldData {
  id: string;
  page: number;
  x: number;
  y: number;
  width: number;
  height: number;
  type?: SignatureFieldType;  // Optional — v1.0 docs have no type; fallback = 'client-signature'
}

export function getFieldType(field: SignatureFieldData): SignatureFieldType {
  return field.type ?? 'client-signature';
}

export function isClientVisibleField(field: SignatureFieldData): boolean {
  return getFieldType(field) !== 'agent-signature';
}
```

### Current Unguarded Line in sign GET route (the critical line)
```typescript
// Source: /src/app/api/sign/[token]/route.ts line 83-92 (current)
return NextResponse.json({
  status: 'pending',
  document: {
    id: doc.id,
    name: doc.name,
    signatureFields: doc.signatureFields ?? [],  // ← UNGUARDED — returns all fields including agent-signature
    preparedFilePath: doc.preparedFilePath,
  },
  expiresAt: new Date(payload.exp * 1000).toISOString(),
});
```

### Fixed sign GET route (Phase 8 target)
```typescript
// Source: Phase 8 target — /src/app/api/sign/[token]/route.ts
import { isClientVisibleField } from '@/lib/db/schema';

// Line 88 replacement:
signatureFields: (doc.signatureFields ?? []).filter(isClientVisibleField),
```

### Current SigningPageClient handleFieldClick (to be guarded)
```typescript
// Source: /src/app/sign/[token]/_components/SigningPageClient.tsx (current)
const handleFieldClick = useCallback(
  (fieldId: string) => {
    if (signedFields.has(fieldId)) return;
    setActiveFieldId(fieldId);
    setModalOpen(true);  // ← opens modal for ANY field, no type check
  },
  [signedFields]
);
```

### Fixed SigningPageClient handleFieldClick (Phase 8 target)
```typescript
// Phase 8 target — defense-in-depth guard (primary protection is the server filter)
import { getFieldType } from '@/lib/db/schema';

const handleFieldClick = useCallback(
  (fieldId: string) => {
    const field = signatureFields.find((f) => f.id === fieldId);
    if (!field) return;
    if (getFieldType(field) !== 'client-signature') return;  // type guard
    if (signedFields.has(fieldId)) return;
    setActiveFieldId(fieldId);
    setModalOpen(true);
  },
  [signatureFields, signedFields]
);
```

### Drizzle Migration Pattern (additive, no data loss)
```typescript
// After schema.ts change, run: npm run db:generate
// Expected generated SQL (0006_*.sql) — may be empty or only a comment:
// --> statement-breakpoint
// (no DDL change for JSONB type annotation)

// If pre-adding agentSignatureData (Phase 11 scope — decision needed):
// ALTER TABLE "users" ADD COLUMN "agent_signature_data" text;
```

---

## State of the Art

| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| Flat interface, no type discriminant | Discriminated union with optional `type` field | Phase 8 | Enables safe multi-type field arrays without breaking existing data |
| All fields sent to signing page | Server-side filter for client-visible types only | Phase 8 | Agent-signature fields permanently invisible to clients |
| Modal opens for every field | Type-branched click handler | Phase 8 | Non-drawable field types cannot accidentally trigger the signature modal |

**Deprecated/outdated after Phase 8:**
- `signatureFields: doc.signatureFields ?? []` (unguarded) in sign GET route: replaced by filtered version

---

## Open Questions

1. **Should Phase 8 add the `agentSignatureData` column to `users`?**
   - What we know: The roadmap assigns this to Phase 11. Phase 8 migration is otherwise zero-SQL.
   - What's unclear: Is there value in pre-adding it (avoids a second migration later) vs. keeping Phase 8 minimal?
   - Recommendation: Keep Phase 8 minimal. Do NOT pre-add `agentSignatureData`. Add it in Phase 11 where it belongs. Phase 8's migration is zero-SQL (JSONB type annotation change only) — that is fine and expected.

2. **Do `initials` fields need any treatment in Phase 8's signing page?**
   - What we know: The roadmap says Phase 8 filters `agent-signature` from the client. Phase 10 handles `initials` end-to-end (including the signing page UI for initials capture).
   - What's unclear: Should Phase 8's `handleFieldClick` guard against ALL non-`client-signature` types, or just `agent-signature`?
   - Recommendation: Guard against everything except `client-signature` in Phase 8 (`if (getFieldType(field) !== 'client-signature') return`). This is more future-proof and makes Phase 10 easier — Phase 10 will expand the click handler to handle `initials` explicitly.

3. **Is the Drizzle meta snapshot the only artifact to update?**
   - What we know: `drizzle/meta/` contains JSON snapshot files. The schema change affects the snapshot even with no SQL output.
   - What's unclear: Whether `drizzle-kit generate` produces a meaningful SQL file or a completely empty one when only a `.$type<T>()` annotation changes.
   - Recommendation: Run `npm run db:generate` and commit whatever it produces. If it's empty, commit it anyway to keep the snapshot current.

---

## Validation Architecture

> `workflow.nyquist_validation` is not present in `.planning/config.json` (config has `workflow.research`, `workflow.plan_check`, `workflow.verifier` — no `nyquist_validation` key). Skipping this section.

---

## Sources

### Primary (HIGH confidence)
- `/Users/ccopeland/temp/red/teressa-copeland-homes/src/lib/db/schema.ts` (codebase inspection 2026-03-21) — `SignatureFieldData` interface confirmed to have no `type` field; JSONB annotation via `.$type<SignatureFieldData[]>()` confirmed
- `/Users/ccopeland/temp/red/teressa-copeland-homes/src/app/api/sign/[token]/route.ts` lines 83-92 (codebase inspection 2026-03-21) — unguarded `doc.signatureFields ?? []` on line 88 confirmed
- `/Users/ccopeland/temp/red/teressa-copeland-homes/src/app/sign/[token]/_components/SigningPageClient.tsx` (codebase inspection 2026-03-21) — `handleFieldClick` opens modal for every field with no type check confirmed; `handleSubmit` checks `signedFields.size < signatureFields.length` confirmed
- `/Users/ccopeland/temp/red/.planning/research/SUMMARY.md` (project v1.1 pre-research 2026-03-21) — Phase 8 rationale, critical pitfalls #1 and #3, exact file/line references confirmed
- `/Users/ccopeland/temp/red/.planning/research/STACK.md` (project v1.1 pre-research 2026-03-21) — existing stack confirmed; no new dependencies for Phase 8
- `/Users/ccopeland/temp/red/.planning/ROADMAP.md` (2026-03-21) — Phase 8 plan structure confirmed (08-01-PLAN.md + 08-02-PLAN.md)
- Drizzle ORM docs (`.$type<T>()` on jsonb) — TypeScript-only annotation, no DDL generated — HIGH confidence (confirmed via codebase usage pattern)

### Secondary (MEDIUM confidence)
- Drizzle ORM GitHub/docs — `db:generate` updates meta snapshot even for TypeScript-only changes (standard Drizzle behavior; verified via project migration pattern in `drizzle/meta/`)

---

## Metadata

**Confidence breakdown:**
- Standard stack: HIGH — all libraries already in use; no new dependencies
- Architecture: HIGH — based on actual codebase files; specific line numbers identified; minimal surface area
- Pitfalls: HIGH — grounded in actual file inspection; failure modes are deterministic, not speculative

**Research date:** 2026-03-21
**Valid until:** 2026-04-20 (stable stack; no external dependencies that could change)
