red/.planning/phases/08-schema-foundation-and-signing-page-safety/08-RESEARCH.md

Phase 8: Schema Foundation and Signing Page Safety - Research

Researched: 2026-03-21 Domain: TypeScript discriminated union schema extension, Drizzle ORM migration, Next.js API route filtering, React client component type branching Confidence: HIGH

<phase_requirements>

Phase Requirements

ID	Description	Research Support
FIELD-01	Agent can place text field markers on a PDF (for typed content like names, addresses, prices)	Schema type discriminant enables safe introduction of all new field types; signing page filter prevents non-client-sig types from surfacing as required unsigned; migration ensures no data loss on v1.0 documents
</phase_requirements>

---

Summary

Phase 8 is a pure safety gate — no user-visible feature is added. It makes the codebase safe to receive the new field types that Phases 9-13 will introduce. The single most dangerous change in v1.1 is adding new values to SignatureFieldData while the client signing page and the /api/sign/[token] GET route still treat every field as a required client-signature. Two concrete failure modes exist: (1) a client opens a signing session and sees an agent-signature field as a required unsigned overlay, panics, and cannot complete signing; (2) the SigningPageClient calls setModalOpen(true) for a non-drawable field type (text/checkbox/date), producing broken UX before any modal code for those types exists. Both failures are HIGH recovery cost in production — the signing session is broken and cannot self-heal.

The technical work is narrow and well-scoped: add a type discriminant to the SignatureFieldData TypeScript interface, generate and run a Drizzle migration (the migration is a no-op in SQL terms — JSONB column already exists, no SQL change required for the type discriminant — but a migration for the agentSignatureData column on users IS needed), apply a backward-compatible runtime fallback (field.type ?? 'client-signature') in every path that reads signatureFields, add a type filter in the /api/sign/[token] GET route before returning signatureFields to the client, and branch SigningPageClient.tsx so only client-signature types open the signature modal. None of these changes alter any existing user-visible behavior on v1.0 documents because all existing fields have no type property and will read as 'client-signature' via the fallback.

The Drizzle ORM discriminated union type annotation is applied to .$type<SignatureFieldData[]>() on the signatureFields JSONB column — this is a TypeScript-only annotation with no database DDL change. The only SQL migration needed for Phase 8 is ALTER TABLE "users" ADD COLUMN "agent_signature_data" TEXT (needed by Phase 11, but adding it now keeps the schema extension atomic). However, examining the roadmap plan files, the agentSignatureData column is planned for Phase 11's migration. Phase 8's migration may therefore be ZERO SQL statements (the discriminant lives in the TypeScript interface only) unless the team decides to pre-add the users column.

Primary recommendation: Extend SignatureFieldData with type discriminant in schema.ts, add a backward-compat fallback in all read paths, filter agent-signature fields from the sign GET route, and branch SigningPageClient.tsx by field type — all in a single atomic deployment. The Drizzle migration for Phase 8 is likely a zero-SQL migration (TypeScript type change only); generate it anyway to ensure the meta snapshot stays in sync.

---

Standard Stack

Core (All Existing — No New Dependencies)

Library	Version	Purpose	Why Standard
Drizzle ORM	^0.45.1	TypeScript schema definition, JSONB type annotation, migration generation	Already in project; .$type<T>() method annotates JSONB column TypeScript type without DDL change
drizzle-kit	^0.31.10	Migration generation (npm run db:generate) and migration execution (npm run db:migrate)	Already in project; generates 0006_*.sql diff
Next.js	16.2.0	API route (server-side filter) and React client component (type branching)	Already in project
TypeScript	(project version)	Discriminated union type definition	Already in project
postgres	^3.4.8	Database driver	Already in project

No New Dependencies

Phase 8 adds zero new npm packages. All changes are to TypeScript type definitions, runtime logic in existing files, and a Drizzle migration.

Installation:

# No new packages needed

---

Architecture Patterns

Recommended Project Structure

No new files or directories are created in Phase 8. All changes are modifications to existing files:

src/lib/db/schema.ts                              # Modified: SignatureFieldData type + discriminant
src/app/api/sign/[token]/route.ts                 # Modified: filter signatureFields before GET response
src/app/sign/[token]/_components/SigningPageClient.tsx  # Modified: type branching in field render + click handler
drizzle/0006_*.sql                                # New: migration file (may be zero-SQL or users column add)

Pattern 1: TypeScript Discriminated Union on JSONB .$type()

What: Add a type literal union to the SignatureFieldData interface. Drizzle's .$type<T>() annotation changes only the TypeScript type of the column — it does NOT generate any SQL DDL change. The JSONB column remains unchanged in the database.

When to use: Any time a JSONB column stores an array of objects that need type discrimination at the TypeScript layer without a DB schema change.

Example:

// src/lib/db/schema.ts — AFTER Phase 8

export type SignatureFieldType =
  | 'client-signature'
  | 'initials'
  | 'text'
  | 'checkbox'
  | 'date'
  | 'agent-signature';

export interface SignatureFieldData {
  id: string;
  page: number;       // 1-indexed
  x: number;         // PDF user space, bottom-left origin, points
  y: number;         // PDF user space, bottom-left origin, points
  width: number;     // PDF points (default: 144 — 2 inches)
  height: number;    // PDF points (default: 36 — 0.5 inches)
  type?: SignatureFieldType;  // Optional for backward compat with v1.0 documents
}

// Helper: always use this to read field type — never read .type directly
export function getFieldType(field: SignatureFieldData): SignatureFieldType {
  return field.type ?? 'client-signature';
}

// Type predicates for common checks
export function isClientVisibleField(field: SignatureFieldData): boolean {
  const t = getFieldType(field);
  return t !== 'agent-signature';
}

// In the documents table — .$type<T> annotation only (no SQL change)
signatureFields: jsonb("signature_fields").$type<SignatureFieldData[]>(),

Key insight: type is declared as type?: SignatureFieldType (optional). This is backward-compatible: existing documents with no type property in their JSONB fields will NOT fail JSON parsing, and the getFieldType() helper coalesces undefined to 'client-signature'.

Pattern 2: Server-Side Filter Before Response

What: Before returning signatureFields in the GET /api/sign/[token] response, filter to client-visible types only.

When to use: Any API route that returns field data to an unauthenticated client (the signing page).

Exact location: /src/app/api/sign/[token]/route.ts — the current line 88:

// BEFORE (line 88 — current, unguarded):
signatureFields: doc.signatureFields ?? [],

// AFTER (Phase 8):
signatureFields: (doc.signatureFields ?? []).filter(isClientVisibleField),

This single line is the security-critical change. After this filter, agent-signature fields are permanently invisible to the client signing page regardless of what field types future phases add.

Pattern 3: Type-Branching in SigningPageClient

What: The handleFieldClick callback and the field overlay render must branch on field type. Non-client-signature types must not open the signature modal.

When to use: Any UI component that renders SignatureFieldData[] received from the server.

Example pattern (Phase 8 minimum — no UI changes needed yet):

// In SigningPageClient.tsx

// handleFieldClick — guard against non-modal types
const handleFieldClick = useCallback(
  (fieldId: string) => {
    const field = signatureFields.find((f) => f.id === fieldId);
    if (!field) return;
    const fieldType = getFieldType(field);
    // Only client-signature opens the modal in Phase 8
    // Other types will be handled in Phase 10 (initials, text, etc.)
    if (fieldType !== 'client-signature') return;
    if (signedFields.has(fieldId)) return;
    setActiveFieldId(fieldId);
    setModalOpen(true);
  },
  [signatureFields, signedFields]
);

// handleSubmit — only require signatures for client-signature fields
const handleSubmit = useCallback(async () => {
  const clientSigFields = signatureFields.filter(
    (f) => getFieldType(f) === 'client-signature'
  );
  if (signedFields.size < clientSigFields.length || submitting) return;
  // ... rest of submit unchanged
}, [signatureFields, signedFields.size, submitting, token]);

Note on progress bar: SigningProgressBar receives total and signed counts. After Phase 8, total must be signatureFields.filter(f => getFieldType(f) === 'client-signature').length — not signatureFields.length. Because the API route now filters out agent-signature fields before sending, the client will never receive them. This is handled transparently. The filter in the GET route means signatureFields on the client already only contains client-visible types — so the SigningPageClient type branching in Phase 8 is a defense-in-depth guard, not the primary protection.

Pattern 4: Drizzle Migration for Phase 8

What: The TypeScript discriminant requires no SQL DDL change — signatureFields is already jsonb. The migration generated by drizzle-kit generate will reflect the TypeScript change but produce an empty SQL file (or just a comment).

Important: Run npm run db:generate after updating schema.ts. Even if the migration is empty SQL, it updates the Drizzle meta snapshot (the JSON file in drizzle/meta/) to stay in sync with the schema. Skipping this causes future db:generate runs to produce incorrect diffs.

The migration SQL will be:

-- No DDL change needed for Phase 8 discriminant (JSONB type annotation only)
-- This is an intentionally empty migration to advance the Drizzle snapshot

If the team decides to pre-add agent_signature_data to users (for Phase 11 prep), the migration would be:

ALTER TABLE "users" ADD COLUMN "agent_signature_data" text;

DECISION NEEDED: The roadmap plan files assign agentSignatureData to Phase 11. Do NOT pre-add it in Phase 8 unless explicitly decided. Keep Phase 8 minimal.

Anti-Patterns to Avoid

• Reading field.type directly without fallback: Always use getFieldType(field) or field.type ?? 'client-signature'. Direct access produces undefined on v1.0 documents and TypeScript will not catch this at all entry points.
• Filtering on the client instead of the server: The agent-signature filter MUST happen in the API route before the response is sent. Filtering only in SigningPageClient.tsx is insufficient — an attacker can call the API directly and see field coordinates.
• Changing type from optional to required: Making type required in SignatureFieldData breaks TypeScript compilation on every existing call site that constructs a SignatureFieldData object (including FieldPlacer.tsx handleDragEnd). Keep type optional; add getFieldType() helper for safe reading.
• Modifying embed-signature.ts or the POST /api/sign/[token] route: Phase 8 does NOT touch the signing POST handler or PDF embedding pipeline. Those are working correctly and only handle client-signature fields today.

---

Don't Hand-Roll

Problem	Don't Build	Use Instead	Why
TypeScript type narrowing for JSONB union	Custom type guard per field	getFieldType() helper + type predicate functions exported from schema.ts	Single source of truth; all consumers import the same fallback logic
API field filtering	Field visibility logic inline in route	isClientVisibleField() pure function exported from schema.ts	Testable, consistent across all routes that return fields to clients
Drizzle meta sync	Manual edits to meta JSON	npm run db:generate after every schema.ts change	Drizzle meta is machine-generated; manual edits corrupt future diffs
Backward-compat runtime coercion	Migration that sets type = 'client-signature' on all existing JSONB rows	Optional TypeScript field with fallback in getFieldType()	No data migration risk; existing rows work immediately; all new rows get explicit type

Key insight: The entire Phase 8 safety guarantee comes from two pure TypeScript functions (getFieldType and isClientVisibleField) that are easy to unit-test and can be imported by every consumer. Don't scatter the fallback logic across callsites.

---

Common Pitfalls

Pitfall 1: Breaking Active Signing Sessions

What goes wrong: Any document that was prepared and sent BEFORE Phase 8 ships has signatureFields JSONB with no type property on each field object. If Phase 8 changes the TypeScript interface to require type (non-optional), TypeScript may compile but the getFieldType() fallback handles this correctly — however, if any code path branches on field.type === 'client-signature' with a strict equality check and field.type is undefined, the branch is never taken. Why it happens: Confusion between optional TypeScript field and runtime undefined value. How to avoid: Always coerce via getFieldType(field) which returns field.type ?? 'client-signature'. Export this function from schema.ts so there is only one implementation. Warning signs: TypeScript strict mode warnings on field.type comparisons; unit tests on v1.0-style field objects (no type property) failing.

Pitfall 2: Drizzle Meta Snapshot Out of Sync

What goes wrong: After updating SignatureFieldData in schema.ts, the developer skips npm run db:generate because "there's no SQL change." The next db:generate run (in Phase 9 or later) produces a diff against the stale meta snapshot and generates incorrect SQL. Why it happens: Drizzle stores the TypeScript schema representation as a JSON snapshot in drizzle/meta/. Even TypeScript-only changes to .$type<T>() annotations update this snapshot. How to avoid: Always run npm run db:generate after any schema.ts change, even if you expect zero SQL output. Commit the resulting migration file (even if it has no SQL statements). Warning signs: drizzle-kit generate produces a migration that re-adds columns that already exist.

Pitfall 3: The Filter Is in the Wrong Place

What goes wrong: Developer adds the agent-signature filter only in SigningPageClient.tsx (client component) but leaves the GET /api/sign/[token] route returning unfiltered signatureFields. The filter works in the browser, but the field coordinates and IDs of agent-signature fields are still leaked to the network. Why it happens: The signing page is the visible failure point, so developers focus on it. How to avoid: Filter in the API route (server-side, line 88) as the primary protection. The SigningPageClient type guard is defense-in-depth only. Warning signs: Network tab in browser DevTools shows agent-signature field objects in the GET response JSON.

Pitfall 4: Breaking the Submit Logic

What goes wrong: handleSubmit checks signedFields.size < signatureFields.length. If an initials or other non-signature type is returned in signatureFields (which Phase 8's server filter prevents), this check would require the client to "sign" non-signable fields before enabling Submit. Because the filter is in the API route, this is already safe — but if the filter is ever removed or incomplete, Submit becomes permanently disabled. Why it happens: The submit guard counts total fields, not just client-signature fields. How to avoid: Update handleSubmit to count only client-signature fields in its completeness check: signatureFields.filter(f => getFieldType(f) === 'client-signature').length. Even though Phase 8's server filter makes this moot today, it makes the component resilient to future API changes. Warning signs: Submit button stays disabled even after all visible signature fields are completed.

Pitfall 5: Forgetting the SigningProgressBar Total

What goes wrong: SigningProgressBar is passed total={signatureFields.length} as a prop. If for any reason a non-client-signature field appears in the array, the total is inflated and "X of Y signed" shows a count the client can never reach. Why it happens: The progress bar total is passed from SigningPageClient which receives the full array. How to avoid: Calculate total as signatureFields.filter(f => getFieldType(f) === 'client-signature').length. This is currently moot because the API route filters, but is correct defensive coding. Warning signs: Progress bar shows "0 of 3" when only 2 signature fields are visible.

---

Code Examples

Verified patterns from codebase inspection (2026-03-21):

Current SignatureFieldData (schema.ts — to be extended)

// Source: /src/lib/db/schema.ts (current — no type discriminant)
export interface SignatureFieldData {
  id: string;
  page: number;   // 1-indexed
  x: number;      // PDF user space, bottom-left origin, points
  y: number;      // PDF user space, bottom-left origin, points
  width: number;  // PDF points (default: 144 — 2 inches)
  height: number; // PDF points (default: 36 — 0.5 inches)
}

Extended SignatureFieldData with Discriminant

// Source: Phase 8 target schema
export type SignatureFieldType =
  | 'client-signature'
  | 'initials'
  | 'text'
  | 'checkbox'
  | 'date'
  | 'agent-signature';

export interface SignatureFieldData {
  id: string;
  page: number;
  x: number;
  y: number;
  width: number;
  height: number;
  type?: SignatureFieldType;  // Optional — v1.0 docs have no type; fallback = 'client-signature'
}

export function getFieldType(field: SignatureFieldData): SignatureFieldType {
  return field.type ?? 'client-signature';
}

export function isClientVisibleField(field: SignatureFieldData): boolean {
  return getFieldType(field) !== 'agent-signature';
}

Current Unguarded Line in sign GET route (the critical line)

// Source: /src/app/api/sign/[token]/route.ts line 83-92 (current)
return NextResponse.json({
  status: 'pending',
  document: {
    id: doc.id,
    name: doc.name,
    signatureFields: doc.signatureFields ?? [],  // ← UNGUARDED — returns all fields including agent-signature
    preparedFilePath: doc.preparedFilePath,
  },
  expiresAt: new Date(payload.exp * 1000).toISOString(),
});

Fixed sign GET route (Phase 8 target)

// Source: Phase 8 target — /src/app/api/sign/[token]/route.ts
import { isClientVisibleField } from '@/lib/db/schema';

// Line 88 replacement:
signatureFields: (doc.signatureFields ?? []).filter(isClientVisibleField),

Current SigningPageClient handleFieldClick (to be guarded)

// Source: /src/app/sign/[token]/_components/SigningPageClient.tsx (current)
const handleFieldClick = useCallback(
  (fieldId: string) => {
    if (signedFields.has(fieldId)) return;
    setActiveFieldId(fieldId);
    setModalOpen(true);  // ← opens modal for ANY field, no type check
  },
  [signedFields]
);

Fixed SigningPageClient handleFieldClick (Phase 8 target)

// Phase 8 target — defense-in-depth guard (primary protection is the server filter)
import { getFieldType } from '@/lib/db/schema';

const handleFieldClick = useCallback(
  (fieldId: string) => {
    const field = signatureFields.find((f) => f.id === fieldId);
    if (!field) return;
    if (getFieldType(field) !== 'client-signature') return;  // type guard
    if (signedFields.has(fieldId)) return;
    setActiveFieldId(fieldId);
    setModalOpen(true);
  },
  [signatureFields, signedFields]
);

Drizzle Migration Pattern (additive, no data loss)

// After schema.ts change, run: npm run db:generate
// Expected generated SQL (0006_*.sql) — may be empty or only a comment:
// --> statement-breakpoint
// (no DDL change for JSONB type annotation)

// If pre-adding agentSignatureData (Phase 11 scope — decision needed):
// ALTER TABLE "users" ADD COLUMN "agent_signature_data" text;

---

State of the Art

Old Approach	Current Approach	When Changed	Impact
Flat interface, no type discriminant	Discriminated union with optional type field	Phase 8	Enables safe multi-type field arrays without breaking existing data
All fields sent to signing page	Server-side filter for client-visible types only	Phase 8	Agent-signature fields permanently invisible to clients
Modal opens for every field	Type-branched click handler	Phase 8	Non-drawable field types cannot accidentally trigger the signature modal

Deprecated/outdated after Phase 8:

• signatureFields: doc.signatureFields ?? [] (unguarded) in sign GET route: replaced by filtered version

---

Open Questions

1. Should Phase 8 add the agentSignatureData column to users?

   • What we know: The roadmap assigns this to Phase 11. Phase 8 migration is otherwise zero-SQL.
   • What's unclear: Is there value in pre-adding it (avoids a second migration later) vs. keeping Phase 8 minimal?
   • Recommendation: Keep Phase 8 minimal. Do NOT pre-add agentSignatureData. Add it in Phase 11 where it belongs. Phase 8's migration is zero-SQL (JSONB type annotation change only) — that is fine and expected.

2. Do initials fields need any treatment in Phase 8's signing page?

   • What we know: The roadmap says Phase 8 filters agent-signature from the client. Phase 10 handles initials end-to-end (including the signing page UI for initials capture).
   • What's unclear: Should Phase 8's handleFieldClick guard against ALL non-client-signature types, or just agent-signature?
   • Recommendation: Guard against everything except client-signature in Phase 8 (if (getFieldType(field) !== 'client-signature') return). This is more future-proof and makes Phase 10 easier — Phase 10 will expand the click handler to handle initials explicitly.

3. Is the Drizzle meta snapshot the only artifact to update?

   • What we know: drizzle/meta/ contains JSON snapshot files. The schema change affects the snapshot even with no SQL output.
   • What's unclear: Whether drizzle-kit generate produces a meaningful SQL file or a completely empty one when only a .$type<T>() annotation changes.
   • Recommendation: Run npm run db:generate and commit whatever it produces. If it's empty, commit it anyway to keep the snapshot current.

---

Validation Architecture

workflow.nyquist_validation is not present in .planning/config.json (config has workflow.research, workflow.plan_check, workflow.verifier — no nyquist_validation key). Skipping this section.

---

Sources

Primary (HIGH confidence)

• /Users/ccopeland/temp/red/teressa-copeland-homes/src/lib/db/schema.ts (codebase inspection 2026-03-21) — SignatureFieldData interface confirmed to have no type field; JSONB annotation via .$type<SignatureFieldData[]>() confirmed
• /Users/ccopeland/temp/red/teressa-copeland-homes/src/app/api/sign/[token]/route.ts lines 83-92 (codebase inspection 2026-03-21) — unguarded doc.signatureFields ?? [] on line 88 confirmed
• /Users/ccopeland/temp/red/teressa-copeland-homes/src/app/sign/[token]/_components/SigningPageClient.tsx (codebase inspection 2026-03-21) — handleFieldClick opens modal for every field with no type check confirmed; handleSubmit checks signedFields.size < signatureFields.length confirmed
• /Users/ccopeland/temp/red/.planning/research/SUMMARY.md (project v1.1 pre-research 2026-03-21) — Phase 8 rationale, critical pitfalls #1 and #3, exact file/line references confirmed
• /Users/ccopeland/temp/red/.planning/research/STACK.md (project v1.1 pre-research 2026-03-21) — existing stack confirmed; no new dependencies for Phase 8
• /Users/ccopeland/temp/red/.planning/ROADMAP.md (2026-03-21) — Phase 8 plan structure confirmed (08-01-PLAN.md + 08-02-PLAN.md)
• Drizzle ORM docs (.$type<T>() on jsonb) — TypeScript-only annotation, no DDL generated — HIGH confidence (confirmed via codebase usage pattern)

Secondary (MEDIUM confidence)

• Drizzle ORM GitHub/docs — db:generate updates meta snapshot even for TypeScript-only changes (standard Drizzle behavior; verified via project migration pattern in drizzle/meta/)

---

Metadata

Confidence breakdown:

• Standard stack: HIGH — all libraries already in use; no new dependencies
• Architecture: HIGH — based on actual codebase files; specific line numbers identified; minimal surface area
• Pitfalls: HIGH — grounded in actual file inspection; failure modes are deterministic, not speculative

Research date: 2026-03-21 Valid until: 2026-04-20 (stable stack; no external dependencies that could change)