diff --git a/.planning/phases/12-filled-document-preview/12-RESEARCH.md b/.planning/phases/12-filled-document-preview/12-RESEARCH.md new file mode 100644 index 0000000..c722a45 --- /dev/null +++ b/.planning/phases/12-filled-document-preview/12-RESEARCH.md @@ -0,0 +1,466 @@ +# Phase 12: Filled Document Preview - Research + +**Researched:** 2026-03-21 +**Domain:** PDF preview generation, React modal with react-pdf, Send-button gating with staleness detection +**Confidence:** HIGH + + +## Phase Requirements + +| ID | Description | Research Support | +|----|-------------|-----------------| +| PREV-01 | Agent sees a live filled preview of the fully-prepared document (text filled, signatures embedded) before sending to client | Preview route reuses `preparePdf()` in preview mode writing to a versioned path; PreviewModal uses react-pdf `Document` with `ArrayBuffer` file prop; Send-button gating via `previewToken` state; staleness detection by resetting token on field/text change | + + +## Summary + +Phase 12 adds a "Preview" button to the document prepare page that generates a temporary filled PDF — with all text, agent signatures, agent initials, checkboxes, and date stamps embedded — and displays it in a modal using the already-installed `react-pdf` library. The Send button becomes available only after the agent has generated at least one preview and is re-disabled the moment any field or text-fill value changes (staleness detection). + +The preview PDF is written to a unique versioned path (e.g., `{docId}_preview_{timestamp}.pdf`) in the uploads directory, completely separate from `_prepared.pdf`. This preserves the integrity of the final prepared document: the prepare route that actually sends to clients always writes to `_prepared.pdf` and is unchanged. The preview route is a thin POST endpoint that calls the existing `preparePdf()` utility with `destPath` pointing to the versioned preview path, then streams the bytes back to the client. + +The client side is straightforward: the `PreparePanel` component maintains a `previewToken` (a string or null) that represents whether the current field/text state has a valid preview. The Send button is enabled only when `previewToken !== null`. Any user action that changes fields (via `FieldPlacer.persistFields`) or text fill values fires a `onFieldsChanged` callback into `PreparePanel`, resetting the token to null and re-disabling Send. + +**Primary recommendation:** Add a POST `/api/documents/[id]/preview` route that calls `preparePdf()` to a versioned path and returns the bytes as `application/pdf`; render in a `PreviewModal` using `react-pdf` `` pattern; gate the Send button on `previewToken` state in `PreparePanel`. + +## Standard Stack + +### Core +| Library | Version | Purpose | Why Standard | +|---------|---------|---------|--------------| +| `@cantoo/pdf-lib` | 2.6.3 | Generates the preview PDF | Already used in `preparePdf()` — zero new dependency | +| `react-pdf` | 10.4.1 | Renders the preview PDF in the modal | Already installed; `Document` accepts `ArrayBuffer` | +| `pdfjs-dist` | 5.4.296 | Powers react-pdf rendering | Already installed; worker already configured in `PdfViewer.tsx` | + +### Supporting +| Library | Version | Purpose | When to Use | +|---------|---------|---------|-------------| +| Next.js Route Handler | 16.2.0 | POST `/api/documents/[id]/preview` returning PDF bytes | Follows exact pattern of `/api/documents/[id]/file` route | +| Node `fs/promises` | built-in | Write preview file, read bytes back | Same pattern as existing prepare route | +| `path` | built-in | Path construction + traversal guard | Same pattern as every other file-serving route | + +### Alternatives Considered +| Instead of | Could Use | Tradeoff | +|------------|-----------|----------| +| Returning `ArrayBuffer` from preview route | Generating a presigned URL to preview file | URL approach requires a second route + cleanup; ArrayBuffer is simpler since file is ephemeral | +| `ArrayBuffer` as react-pdf `file` prop | Blob URL via `URL.createObjectURL` | Both work; `ArrayBuffer` is simpler — no cleanup of blob URLs needed, and react-pdf accepts `ArrayBuffer` directly per its type definitions | +| In-memory PDF (never write to disk) | Write-then-read pattern | Write-to-disk is required so the preview path is distinct from `_prepared.pdf`; reusing `preparePdf()` as-is avoids reimplementing the atomic write logic | + +**Installation:** +```bash +# No new packages needed — all dependencies already installed +``` + +## Architecture Patterns + +### Recommended Project Structure +``` +src/ +├── app/api/documents/[id]/preview/ +│ └── route.ts # POST — generates preview PDF, returns bytes +├── app/portal/(protected)/documents/[docId]/_components/ +│ ├── PreparePanel.tsx # Modified: Preview button + Send gating + staleness +│ ├── PreviewModal.tsx # NEW: Modal wrapping react-pdf Document/Page +│ └── PdfViewer.tsx # Existing: exposes onFieldsChanged callback (new prop) +``` + +### Pattern 1: Preview Route — Reuse preparePdf with Versioned Path + +**What:** A POST route that calls the existing `preparePdf()` utility with a timestamp-versioned destination path and returns the resulting PDF bytes directly in the response body. + +**When to use:** Agent clicks the Preview button. Route is auth-guarded (session check), reads the same source PDF and field data the prepare route uses, and does NOT update any document DB columns. + +**Example:** +```typescript +// Source: adapted from /src/app/api/documents/[id]/prepare/route.ts +// File: src/app/api/documents/[id]/preview/route.ts +import { auth } from '@/lib/auth'; +import { db } from '@/lib/db'; +import { documents, users } from '@/lib/db/schema'; +import { eq } from 'drizzle-orm'; +import { preparePdf } from '@/lib/pdf/prepare-document'; +import path from 'node:path'; +import { readFile, unlink } from 'node:fs/promises'; + +const UPLOADS_DIR = path.join(process.cwd(), 'uploads'); + +export async function POST( + req: Request, + { params }: { params: Promise<{ id: string }> } +) { + const session = await auth(); + if (!session?.user?.id) return new Response('Unauthorized', { status: 401 }); + + const { id } = await params; + const body = await req.json() as { textFillData?: Record }; + + const doc = await db.query.documents.findFirst({ where: eq(documents.id, id) }); + if (!doc) return Response.json({ error: 'Not found' }, { status: 404 }); + if (!doc.filePath) return Response.json({ error: 'No source file' }, { status: 422 }); + + const srcPath = path.join(UPLOADS_DIR, doc.filePath); + + // Versioned preview path — never overwrites _prepared.pdf + const timestamp = Date.now(); + const previewRelPath = doc.filePath.replace(/\.pdf$/, `_preview_${timestamp}.pdf`); + const previewPath = path.join(UPLOADS_DIR, previewRelPath); + + // Path traversal guard + if (!previewPath.startsWith(UPLOADS_DIR)) { + return new Response('Forbidden', { status: 403 }); + } + + const sigFields = (doc.signatureFields as import('@/lib/db/schema').SignatureFieldData[]) ?? []; + const textFields = body.textFillData ?? {}; + + const agentUser = await db.query.users.findFirst({ + where: eq(users.id, session.user.id), + columns: { agentSignatureData: true, agentInitialsData: true }, + }); + const agentSignatureData = agentUser?.agentSignatureData ?? null; + const agentInitialsData = agentUser?.agentInitialsData ?? null; + + // Same 422 guards as prepare route + const { getFieldType } = await import('@/lib/db/schema'); + const hasAgentSigFields = sigFields.some(f => getFieldType(f) === 'agent-signature'); + if (hasAgentSigFields && !agentSignatureData) { + return Response.json({ error: 'agent-signature-missing', message: 'No agent signature saved.' }, { status: 422 }); + } + const hasAgentInitialsFields = sigFields.some(f => getFieldType(f) === 'agent-initials'); + if (hasAgentInitialsFields && !agentInitialsData) { + return Response.json({ error: 'agent-initials-missing', message: 'No agent initials saved.' }, { status: 422 }); + } + + await preparePdf(srcPath, previewPath, textFields, sigFields, agentSignatureData, agentInitialsData); + + // Read bytes and stream back; then clean up the preview file + const pdfBytes = await readFile(previewPath); + // Fire-and-forget cleanup — preview files are ephemeral + unlink(previewPath).catch(() => {}); + + return new Response(pdfBytes, { + headers: { 'Content-Type': 'application/pdf' }, + }); +} +``` + +### Pattern 2: PreviewModal — react-pdf with ArrayBuffer + +**What:** A client component modal that receives the PDF as `ArrayBuffer` from a `fetch()` response and passes it directly to `react-pdf`'s `` component. + +**When to use:** After the preview API call succeeds, `PreparePanel` stores the `ArrayBuffer` in state and renders `PreviewModal`. The `file` prop of react-pdf's `Document` accepts `ArrayBuffer` directly — confirmed in `node_modules/react-pdf/dist/shared/types.d.ts`: `export type File = string | ArrayBuffer | Blob | Source | null;`. + +**Example:** +```typescript +// Source: react-pdf Document.d.ts (installed v10.4.1) +// File: src/app/portal/(protected)/documents/[docId]/_components/PreviewModal.tsx +'use client'; +import { useState } from 'react'; +import { Document, Page, pdfjs } from 'react-pdf'; +import 'react-pdf/dist/Page/AnnotationLayer.css'; +import 'react-pdf/dist/Page/TextLayer.css'; + +// Worker must be configured here too (each 'use client' module is independent) +pdfjs.GlobalWorkerOptions.workerSrc = new URL( + 'pdfjs-dist/build/pdf.worker.min.mjs', + import.meta.url, +).toString(); + +interface PreviewModalProps { + pdfBytes: ArrayBuffer; + onClose: () => void; +} + +export function PreviewModal({ pdfBytes, onClose }: PreviewModalProps) { + const [numPages, setNumPages] = useState(0); + const [pageNumber, setPageNumber] = useState(1); + + // CRITICAL: pdfBytes must be memoized by the parent (passed as stable reference) + // react-pdf uses === comparison to detect file changes; new ArrayBuffer each render causes reload loop + return ( +
+
+
+
+ + {pageNumber} / {numPages || '?'} + +
+ +
+ setNumPages(numPages)} + > + + +
+
+ ); +} +``` + +### Pattern 3: Send Button Gating and Staleness Detection + +**What:** `PreparePanel` tracks a `previewToken` (string | null). Token is set to a timestamp string after a successful preview fetch. Token is reset to null whenever the agent changes any text fill or adds/removes fields. Send is disabled while `previewToken === null`. + +**When to use:** All Send-button gating lives in `PreparePanel`. FieldPlacer needs an `onFieldsChanged` callback prop so it can notify PreparePanel when fields are persisted. + +**Example:** +```typescript +// In PreparePanel.tsx +const [previewToken, setPreviewToken] = useState(null); +const [previewBytes, setPreviewBytes] = useState(null); +const [showPreview, setShowPreview] = useState(false); + +// When text fill changes: +function handleTextFillChange(data: Record) { + setTextFillData(data); + setPreviewToken(null); // Stale — re-preview required +} + +// When fields change (callback from FieldPlacer): +function handleFieldsChanged() { + setPreviewToken(null); // Stale +} + +async function handlePreview() { + setLoading(true); + const res = await fetch(`/api/documents/${docId}/preview`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ textFillData }), + }); + if (res.ok) { + const bytes = await res.arrayBuffer(); + setPreviewBytes(bytes); + setPreviewToken(Date.now().toString()); // Mark as fresh + setShowPreview(true); + } else { + // Show error + } + setLoading(false); +} + +// Send button is disabled until preview is fresh: + +{showPreview && previewBytes && ( + setShowPreview(false)} /> +)} + +``` + +### Pattern 4: FieldPlacer onFieldsChanged Callback + +**What:** `FieldPlacer` currently calls `persistFields()` internally after every drag/drop/delete/resize. Adding an `onFieldsChanged?: () => void` prop and calling it after every `persistFields()` call lets `PreparePanel` reset the stale token. + +**Example:** +```typescript +// FieldPlacer.tsx — add prop and call after persistFields +interface FieldPlacerProps { + docId: string; + pageInfo: PageInfo | null; + currentPage: number; + children: React.ReactNode; + readOnly?: boolean; + onFieldsChanged?: () => void; // NEW +} + +// Inside handleDragEnd, handleZonePointerUp after persistFields(): +persistFields(docId, next); +onFieldsChanged?.(); +``` + +### Anti-Patterns to Avoid + +- **Overwriting `_prepared.pdf` with preview**: The preview MUST use a different path. The prepare route writes to `_prepared.pdf`; the preview route must use a timestamp-versioned path. +- **Storing ArrayBuffer in a ref instead of state**: `react-pdf` uses `===` comparison on the `file` prop. If you store bytes in a ref and pass `ref.current` to `Document`, React will not detect the change. Use `useState`. +- **Not memoizing the `file` prop**: If `previewBytes` is recreated on every render (e.g., `new Uint8Array(...)` inline), react-pdf reloads the PDF on every render. Store the `ArrayBuffer` returned by `res.arrayBuffer()` directly in state and pass that state variable — it is stable between renders. +- **Configuring the pdfjs worker more than once**: If PreviewModal is its own component (not inside PdfViewer), it needs its own `pdfjs.GlobalWorkerOptions.workerSrc` assignment. This is fine — the global is idempotent when set to the same value. +- **Leaking preview files**: The preview route should delete the file after reading it (`unlink` after `readFile`). Fire-and-forget is acceptable since the file is ephemeral. Do not leave preview files accumulating in uploads/. +- **Re-enabling Send without a fresh preview when fields are locked**: The `readOnly` guard in FieldPlacer prevents field changes after status is not Draft. But PreparePanel's `currentStatus !== 'Draft'` branch already returns early, so this is not reachable in practice. + +## Don't Hand-Roll + +| Problem | Don't Build | Use Instead | Why | +|---------|-------------|-------------|-----| +| PDF rendering in modal | Custom canvas renderer | `react-pdf` `` + `` | Already installed; handles PDF.js worker, page scaling, canvas rendering | +| PDF bytes generation | Custom PDF manipulation | Existing `preparePdf()` in `lib/pdf/prepare-document.ts` | Already handles all field types, atomic write, magic-byte verification | +| Byte streaming from API | Custom file serving | Standard `new Response(pdfBytes, { headers: ... })` | Same pattern as `/api/documents/[id]/file` route | +| Staleness detection | Complex hash diffing of fields | Simple token reset on any change | Field positions change frequently; any mutation invalidates preview | + +**Key insight:** Every required building block already exists in the codebase. Phase 12 is primarily wiring: a new POST route, a new modal component, and state management in PreparePanel. + +## Common Pitfalls + +### Pitfall 1: ArrayBuffer Equality and react-pdf Reload Loop +**What goes wrong:** `react-pdf` re-fetches/re-renders the PDF every time the `file` prop reference changes (strict `===` comparison). If you do `file={new Uint8Array(previewBytes)}` or rebuild the bytes object on each render, the PDF continuously reloads. +**Why it happens:** react-pdf uses a memoized source hook internally that compares the `file` prop by reference. +**How to avoid:** Store the `ArrayBuffer` from `res.arrayBuffer()` directly in React state (`useState`). Pass the state variable directly — never wrap it in `new Uint8Array()` or `{ data: ... }` on every render. +**Warning signs:** PDF shows "Loading PDF..." repeatedly after first render; network tab shows repeated preview API calls. + +### Pitfall 2: pdfjs Worker Not Configured in PreviewModal +**What goes wrong:** `react-pdf` throws "Setting up fake worker failed" or shows "Failed to load PDF file." +**Why it happens:** The worker is configured in `PdfViewer.tsx` but that assignment is module-scoped to that file's execution context. A new component that imports `react-pdf` independently may execute before PdfViewer. +**How to avoid:** Set `pdfjs.GlobalWorkerOptions.workerSrc` at the top of PreviewModal.tsx exactly as done in PdfViewer.tsx (same `new URL('pdfjs-dist/build/pdf.worker.min.mjs', import.meta.url)` pattern). +**Warning signs:** Modal opens but shows error state immediately. + +### Pitfall 3: Preview File Path Collision with Prepared File +**What goes wrong:** Preview overwrites `_prepared.pdf`, corrupting the final prepared document. +**Why it happens:** Copying the path logic from the prepare route without changing the suffix. +**How to avoid:** Use a timestamp suffix: `doc.filePath.replace(/\.pdf$/, `_preview_${Date.now()}.pdf`)`. Never use `_prepared.pdf` as the preview destination. +**Warning signs:** Prepared document shows "draft preview" content (e.g., "Sign Here" placeholder) after agent sends. + +### Pitfall 4: Preview File Not Cleaned Up +**What goes wrong:** Preview PDFs accumulate in `uploads/` directory indefinitely. +**Why it happens:** Route writes the file for streaming but doesn't delete it after reading. +**How to avoid:** After `readFile(previewPath)` succeeds, call `unlink(previewPath).catch(() => {})` (fire-and-forget). The preview file is ephemeral — delete it immediately after reading bytes. +**Warning signs:** `uploads/` grows with `*_preview_*.pdf` files. + +### Pitfall 5: Send Button Gating Not Propagated Through FieldPlacer Changes +**What goes wrong:** Agent adds or moves a field, then clicks Send without re-previewing — the staleness token is not reset. +**Why it happens:** FieldPlacer calls `persistFields()` internally and has no way to notify PreparePanel that fields changed. +**How to avoid:** Add `onFieldsChanged?: () => void` to `FieldPlacerProps`. Call it after every `persistFields()` invocation (inside `handleDragEnd`, `handleZonePointerUp`, and the delete button handler). In PreparePanel, pass `onFieldsChanged={() => setPreviewToken(null)}` to FieldPlacer. +**Warning signs:** Manual test — place a new field after previewing; Send button stays enabled when it should be disabled. + +### Pitfall 6: Deployment Target — Vercel Serverless vs. Home Docker +**What goes wrong:** The write-then-read-then-unlink pattern silently fails on Vercel's serverless runtime (ephemeral/read-only filesystem outside /tmp). +**Why it happens:** Vercel's serverless functions have no persistent writable filesystem outside `/tmp`. `uploads/` is a Docker volume on the home server, not accessible on Vercel. +**How to avoid:** The STATE.md already flags this: "Deployment target (Vercel serverless vs. self-hosted container) must be confirmed before implementing preview route." Based on the entire architecture (uploads/, Docker-volume-based file storage, home server deployment), this app runs on a self-hosted Docker container — not Vercel. The write-to-disk pattern is correct for this environment. +**Warning signs:** If ever deploying to Vercel, the preview route would need to use `/tmp` as the scratch directory, not `uploads/`. + +### Pitfall 7: 422 Guards Mirror Mismatch +**What goes wrong:** Preview route and prepare route get out of sync on 422 guards (e.g., prepare route gets a new guard added in a future phase but preview doesn't). +**How to avoid:** The preview route should duplicate the same guards as the prepare route for agent-signature and agent-initials fields. Both must block if the relevant field type is present but the signature data is missing. + +## Code Examples + +Verified patterns from official sources: + +### react-pdf ArrayBuffer file prop (verified from installed v10.4.1 types) +```typescript +// Source: node_modules/react-pdf/dist/shared/types.d.ts +// export type File = string | ArrayBuffer | Blob | Source | null; + +// Correct: pass ArrayBuffer directly from res.arrayBuffer() +const bytes = await res.arrayBuffer(); +setPreviewBytes(bytes); // store in state + +// In render: + // previewBytes is stable (state reference) + + +``` + +### Next.js Route Handler returning PDF bytes (verified from existing codebase) +```typescript +// Source: /src/app/api/documents/[id]/file/route.ts (existing) +const buffer = await readFile(filePath); +return new Response(buffer, { + headers: { 'Content-Type': 'application/pdf' }, +}); +``` + +### pdfjs worker configuration (verified from PdfViewer.tsx) +```typescript +// Source: /src/app/portal/(protected)/documents/[docId]/_components/PdfViewer.tsx +pdfjs.GlobalWorkerOptions.workerSrc = new URL( + 'pdfjs-dist/build/pdf.worker.min.mjs', + import.meta.url, +).toString(); +``` + +### Versioned preview path pattern +```typescript +// Avoid collision with _prepared.pdf; use timestamp for uniqueness +const timestamp = Date.now(); +const previewRelPath = doc.filePath.replace(/\.pdf$/, `_preview_${timestamp}.pdf`); +const previewPath = path.join(UPLOADS_DIR, previewRelPath); + +// Path traversal guard (required on every file operation) +if (!previewPath.startsWith(UPLOADS_DIR)) { + return new Response('Forbidden', { status: 403 }); +} +``` + +### Ephemeral file cleanup +```typescript +// Read bytes first, then fire-and-forget delete +const pdfBytes = await readFile(previewPath); +unlink(previewPath).catch(() => {}); // ephemeral — delete after read +return new Response(pdfBytes, { headers: { 'Content-Type': 'application/pdf' } }); +``` + +### Staleness token pattern +```typescript +// In PreparePanel state: +const [previewToken, setPreviewToken] = useState(null); + +// Set on successful preview: +setPreviewToken(Date.now().toString()); + +// Reset on any state change: +setPreviewToken(null); + +// Gate the Send button: +disabled={loading || previewToken === null || parseEmails(recipients).length === 0} +``` + +## State of the Art + +| Old Approach | Current Approach | When Changed | Impact | +|--------------|------------------|--------------|--------| +| Generating preview as a separate pipeline | Reuse existing `preparePdf()` | Always (by design) | Zero duplication; preview is guaranteed to match the actual prepare output | +| Blob URL for PDF bytes | `ArrayBuffer` direct to react-pdf | react-pdf v5+ | No blob URL creation/revocation needed; cleaner lifecycle | + +**Deprecated/outdated:** +- **`pdfjs-dist` legacy build**: An earlier decision used the legacy build for server-side text extraction in Phase 13. Phase 12 uses the standard (non-legacy) build for client-side rendering — no change required since react-pdf already bundles the correct worker. + +## Open Questions + +1. **Preview file cleanup if route errors mid-stream** + - What we know: `unlink` is called after `readFile`, so if `preparePdf` fails, no file is written and nothing to clean up. If `readFile` fails (unusual), the file would linger. + - What's unclear: Whether a try/finally cleanup wrapper is warranted. + - Recommendation: Wrap in try/finally: `try { await preparePdf(...); const bytes = await readFile(previewPath); return new Response(bytes, ...); } finally { unlink(previewPath).catch(() => {}); }` — ensures cleanup even on unexpected errors. + +2. **Scroll-to-page in PreviewModal** + - What we know: The existing PdfViewer shows one page at a time with Prev/Next buttons. PreviewModal can use the same pattern. + - What's unclear: Whether the agent needs scroll-all-pages vs. page-by-page navigation in the modal. + - Recommendation: Use the same page-by-page Prev/Next pattern from PdfViewer — no need for scroll-all-pages in a modal. + +3. **Modal scroll on small screens** + - What we know: The modal will contain a multi-page PDF. + - Recommendation: Use `overflow-y: auto` on the modal container and `max-height: 90vh` — same approach as the sticky PreparePanel. + +## Sources + +### Primary (HIGH confidence) +- `node_modules/react-pdf/dist/shared/types.d.ts` — `File` type definition confirming `ArrayBuffer` is accepted +- `node_modules/react-pdf/dist/Document.d.ts` — `DocumentProps.file` prop docs +- `/src/lib/pdf/prepare-document.ts` — Existing `preparePdf()` function signature and behavior +- `/src/app/api/documents/[id]/prepare/route.ts` — Prepare route pattern (auth, guards, path construction) +- `/src/app/api/documents/[id]/file/route.ts` — PDF byte-streaming pattern +- `/src/app/portal/(protected)/documents/[docId]/_components/PdfViewer.tsx` — Worker configuration pattern +- `/src/app/portal/(protected)/documents/[docId]/_components/PreparePanel.tsx` — Current Send button logic +- `/src/app/portal/(protected)/documents/[docId]/_components/FieldPlacer.tsx` — `persistFields` call sites + +### Secondary (MEDIUM confidence) +- `.planning/STATE.md` — Phase 12 deployment concern (Vercel vs. home Docker server) +- `.planning/ROADMAP.md` — Phase 12 plan descriptions + +### Tertiary (LOW confidence) +- None + +## Metadata + +**Confidence breakdown:** +- Standard stack: HIGH — all dependencies already installed; type definitions verified from installed packages +- Architecture: HIGH — patterns lifted directly from existing codebase; `preparePdf()` reuse is confirmed +- Pitfalls: HIGH — ArrayBuffer equality trap verified from react-pdf type docs; file cleanup and path collision are verifiable from existing code patterns; deployment concern documented in STATE.md + +**Research date:** 2026-03-21 +**Valid until:** 2026-04-20 (stable libraries — react-pdf, pdf-lib are unlikely to change in 30 days)