Files
red/.planning/research/STACK.md
2026-04-03 14:47:06 -06:00

30 KiB

Technology Stack

Project: teressa-copeland-homes Researched (v1.1): 2026-03-21 | Updated (v1.2): 2026-04-03 Overall confidence: HIGH


v1.1 Stack Research (retained — do not re-research)

Scope: OpenAI integration, expanded field types, agent signature storage, filled preview

Existing Stack (Do Not Re-research)

Already validated and in package.json. Do not change these.

Technology Version in package.json Role
Next.js 16.2.0 Full-stack framework
React 19.2.4 UI
@cantoo/pdf-lib ^2.6.3 PDF modification (server-side)
react-pdf ^10.4.1 In-browser PDF rendering
signature_pad ^5.1.3 Canvas signature drawing
zod ^4.3.6 Schema validation
@vercel/blob ^2.3.1 File storage
Drizzle ORM + postgres ^0.45.1 / ^3.4.8 Database
Auth.js (next-auth) 5.0.0-beta.30 Authentication
nodemailer ^7.0.13 Transactional email (SMTP)
@react-email/components ^1.0.10 Typed React email templates
@react-email/render ^2.0.4 Server-side renders React email to HTML
@dnd-kit/core + @dnd-kit/utilities ^6.3.1 / ^3.2.2 Drag-drop field placement UI
pdfjs-dist (bundled in node_modules) PDF text extraction for AI pipeline — uses legacy/build/pdf.mjs to handle Node 20
@napi-rs/canvas ^0.1.97 Native canvas bindings (Node.js) — used server-side for canvas operations; architecture-specific prebuilt binary

Note on unpdf: The v1.1 research recommended unpdf as a safer serverless wrapper around PDF.js, but the implemented code uses pdfjs-dist/legacy/build/pdf.mjs directly with GlobalWorkerOptions.workerSrc pointing to the local worker file. unpdf is NOT in package.json and was not installed. Do not add it — the existing pdfjs-dist integration is working.

New Stack Additions for v1.1

Core New Dependency: OpenAI API

Technology Version Purpose Why Recommended
openai ^6.32.0 OpenAI API client for GPT calls Official SDK, current latest, TypeScript-native. Provides client.chat.completions.create() for structured JSON output via manual json_schema response format. Required for AI field placement and pre-fill.

No other new core dependencies are needed. The remaining v1.1 features extend capabilities already in @cantoo/pdf-lib, signature_pad, and react-pdf.

Supporting Libraries

Library Version Purpose When to Use
unpdf NOT INSTALLED — see note above Originally recommended, not used Do not add

No other new supporting libraries needed. See "What NOT to Add" below.

Development Tools

No new dev tooling required for v1.1 features.

Installation

# New dependencies for v1.1
npm install openai

That is the full installation delta for v1.1.

Feature-by-Feature Integration Notes

Feature 1: OpenAI PDF Analysis + Field Placement

Flow:

  1. API route receives document ID
  2. Fetch PDF bytes from Vercel Blob (@vercel/blob — already installed)
  3. Extract text per page using pdfjs-dist/legacy/build/pdf.mjs: getDocument() + page.getTextContent()
  4. Call OpenAI gpt-4.1 with extracted text + a manually defined JSON schema
  5. Parse structured response: array of { fieldType, label, pageNumber, x, y, width, height, suggestedValue }
  6. Save placement records to DB via Drizzle ORM

Why gpt-4.1 (not gpt-4o): The implemented code uses gpt-4.1 which was released after the v1.1 research was written. Use whatever model is set in the existing field-placement.ts implementation.

Why manual JSON schema (not zodResponseFormat): The project uses zod v4.3.6. The zodResponseFormat helper in openai/helpers/zod uses vendored zod-to-json-schema that still expects ZodFirstPartyTypeKind — removed in Zod v4. This is a confirmed open bug as of late 2025. Using zodResponseFormat with Zod v4 throws runtime exceptions. Use response_format: { type: "json_schema", json_schema: { name: "...", strict: true, schema: { ... } } } directly with plain TypeScript types instead.

// CORRECT for Zod v4 project — use manual JSON schema, not zodResponseFormat
const response = await openai.chat.completions.create({
  model: "gpt-4.1",
  messages: [{ role: "user", content: prompt }],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "field_placements",
      strict: true,
      schema: {
        type: "object",
        properties: {
          fields: {
            type: "array",
            items: {
              type: "object",
              properties: {
                fieldType: { type: "string", enum: ["text", "checkbox", "initials", "date", "signature"] },
                label: { type: "string" },
                pageNumber: { type: "number" },
                x: { type: "number" },
                y: { type: "number" },
                width: { type: "number" },
                height: { type: "number" },
                suggestedValue: { type: "string" }
              },
              required: ["fieldType", "label", "pageNumber", "x", "y", "width", "height", "suggestedValue"],
              additionalProperties: false
            }
          }
        },
        required: ["fields"],
        additionalProperties: false
      }
    }
  }
});
const result = JSON.parse(response.choices[0].message.content!);

Feature 2: Expanded Field Types in @cantoo/pdf-lib

No new library needed. @cantoo/pdf-lib v2.6.3 already supports all required field types natively:

Field Type @cantoo/pdf-lib API
Text form.createTextField(name).addToPage(page, options).setText(value)
Checkbox form.createCheckBox(name).addToPage(page, options).check() / .uncheck()
Initials No dedicated type — use createTextField with width/height appropriate for initials
Date No dedicated type — use createTextField, constrain value format in application logic
Agent Signature Use page.drawImage(embeddedPng, { x, y, width, height }) — see Feature 3

Key pattern for checkboxes:

const checkBox = form.createCheckBox('fieldName')
checkBox.addToPage(page, { x, y, width: 15, height: 15, borderWidth: 1 })
if (shouldBeChecked) checkBox.check()

Coordinate system note: @cantoo/pdf-lib uses PDF coordinate space where y=0 is the bottom of the page. If field positions come from pdfjs-dist (which uses y=0 at top), you must transform: pdfY = pageHeight - sourceY - fieldHeight.

Feature 3: Agent Signature Storage

No new library needed. The project already has signature_pad v5.1.3, @vercel/blob, and Drizzle ORM.

Architecture:

  1. Agent draws signature in browser using signature_pad (already installed)
  2. Call signaturePad.toDataURL('image/png') to get base64 PNG
  3. POST to API route; server converts base64 → Uint8Array → uploads to Vercel Blob at a stable path (e.g., /agents/{agentId}/signature.png)
  4. Save blob URL to agent record in DB (add signatureImageUrl column to Agent/User table via Drizzle migration)
  5. On "apply agent signature": server fetches blob URL, embeds PNG into PDF using @cantoo/pdf-lib

signature_pad v5 in React — use useRef on a <canvas> element directly:

import SignaturePad from 'signature_pad'
import { useRef, useEffect } from 'react'

export function SignatureDrawer() {
  const canvasRef = useRef<HTMLCanvasElement>(null)
  const padRef = useRef<SignaturePad | null>(null)

  useEffect(() => {
    if (canvasRef.current) {
      padRef.current = new SignaturePad(canvasRef.current)
    }
    return () => padRef.current?.off()
  }, [])

  const save = () => {
    const dataUrl = padRef.current?.toDataURL('image/png')
    // POST dataUrl to /api/agent/signature
  }

  return <canvas ref={canvasRef} width={400} height={150} />
}

Do NOT add react-signature-canvas. It wraps signature_pad at v1.1.0-alpha.2 (alpha status) and the project already has signature_pad directly. Use the raw library with a useRef.

Embedding the saved signature into PDF:

const sigBytes = await fetch(agentSignatureBlobUrl).then(r => r.arrayBuffer())
const sigImage = await pdfDoc.embedPng(new Uint8Array(sigBytes))
const dims = sigImage.scaleToFit(fieldWidth, fieldHeight)
page.drawImage(sigImage, { x: fieldX, y: fieldY, width: dims.width, height: dims.height })

Feature 4: Filled Document Preview

No new library needed. react-pdf v10.4.1 is already installed and supports rendering a PDF from an ArrayBuffer directly.

Architecture:

  1. Server Action: load original PDF from Vercel Blob, apply all field values (text, checkboxes, embedded signature image) using @cantoo/pdf-lib, return pdfDoc.save() bytes
  2. API route returns the bytes as application/pdf; client receives as ArrayBuffer
  3. Pass ArrayBuffer directly to react-pdf's <Document file={arrayBuffer}> — no upload required

Known issue with react-pdf v7+: ArrayBuffer becomes detached after first use. Always copy:

const safeCopy = (buf: ArrayBuffer) => {
  const copy = new ArrayBuffer(buf.byteLength)
  new Uint8Array(copy).set(new Uint8Array(buf))
  return copy
}
<Document file={safeCopy(previewBuffer)}>

react-pdf renders the flattened PDF accurately — all filled text fields, checked checkboxes, and embedded signature images will appear correctly because they are baked into the PDF bytes by @cantoo/pdf-lib before rendering.

Alternatives Considered (v1.1)

Recommended Alternative Why Not
pdfjs-dist/legacy/build/pdf.mjs directly unpdf wrapper unpdf was recommended in research but not actually installed; the legacy build path works correctly with Node 20 LTS.
pdfjs-dist/legacy/build/pdf.mjs directly pdf-parse pdf-parse is unmaintained (last publish 2019).
Manual JSON schema for OpenAI zodResponseFormat helper Broken with Zod v4 — open bug in openai-node as of Nov 2025. Manual schema avoids the dependency entirely.
gpt-4.1 gpt-4o Real estate form field extraction is a structured extraction task on templated documents. Upgrade only if accuracy on unusual forms is unacceptable.
page.drawImage() for agent signature PDFSignature AcroForm field @cantoo/pdf-lib has no createSignature() API — PDFSignature only reads existing signature fields and provides no image embedding. The correct approach is embedPng() + drawImage() at the field coordinates.

What NOT to Add (v1.1)

Avoid Why Use Instead
zodResponseFormat from openai/helpers/zod Broken at runtime with Zod v4.x (throws exceptions). Open bug, no fix merged as of 2026-03-21. Plain response_format: { type: "json_schema", ... } with hand-written schema
react-signature-canvas Alpha version (1.1.0-alpha.2); project already has signature_pad v5 directly — the wrapper adds nothing signature_pad + useRef<HTMLCanvasElement> directly
@signpdf/placeholder-pdf-lib For cryptographic PKCS#7 digital signatures (DocuSign-style). This project needs visual e-signatures (image embedded in PDF), not cryptographic signing. @cantoo/pdf-lib embedPng() + drawImage()
pdf2json Extracts spatial text data; useful for arbitrary document analysis. Overkill here — we only need raw text content to feed OpenAI. pdfjs-dist legacy build
unpdf Was in the v1.1 research recommendation but not installed. The existing pdfjs-dist/legacy/build/pdf.mjs usage works correctly in Node 20 — do not add unpdf retroactively. pdfjs-dist legacy build (already in use)
langchain / Vercel AI SDK Heavy abstractions for the simple use case of one structured extraction call per document. Adds bundle size and abstraction layers with no benefit here. openai SDK directly
A separate image processing library (sharp, jimp) Not needed — signature PNGs from signature_pad.toDataURL() are already correctly sized canvas exports. @cantoo/pdf-lib handles embedding without pre-processing. N/A

Version Compatibility (v1.1)

Package Compatible With Notes
openai@6.32.0 zod@4.x (manual schema only) Do NOT use zodResponseFormat helper — use raw json_schema response_format. The helper is broken with Zod v4.
openai@6.32.0 Node.js 20+ Requires Node 20 LTS or later.
pdfjs-dist (legacy build) Node.js 20+ Uses legacy/build/pdf.mjs path which handles Promise.withResolvers polyfill issues. Set GlobalWorkerOptions.workerSrc to local worker path.
@cantoo/pdf-lib@2.6.3 react-pdf@10.4.1 These do not interact at runtime — @cantoo/pdf-lib runs server-side, react-pdf runs client-side. No conflict.
signature_pad@5.1.3 React 19 Use as a plain class instantiated in useEffect with a useRef<HTMLCanvasElement>. No React wrapper needed.

Sources (v1.1)


v1.2 Stack Research — Multi-Signer + Docker Production

Scope: Multi-signer support, production Docker Compose, SMTP in Docker Confidence: HIGH for Docker patterns and schema approach, HIGH for email env vars (code-verified)


Summary

Multi-signer support requires zero new npm packages. It is a pure schema extension: a new document_signers junction table in the existing Drizzle/PostgreSQL setup, plus a signerEmail field added to the SignatureFieldData interface. The existing signingTokens table is extended with a signerEmail column. Everything else — token generation, field rendering, email sending, PDF merging — uses libraries already installed.

Docker production is a three-stage Dockerfile with Next.js standalone mode plus a docker-compose.yml with env_file for SMTP credentials. The known "email not working in Docker" failure mode is almost always environment variables not reaching the container — not a nodemailer bug.

Email provider confirmed: The project uses nodemailer v7.0.13 with SMTP (not Resend, not SendGrid). Both the contact form (contact-mailer.ts) and signing emails (signing-mailer.tsx) use nodemailer with shared env vars CONTACT_SMTP_HOST, CONTACT_SMTP_PORT, CONTACT_EMAIL_USER, CONTACT_EMAIL_PASS. The sendAgentNotificationEmail function already exists — it just needs to be called for the completion event. No email library changes needed.


New Dependencies for v1.2

Multi-Signer: None

Capability Already Covered By
Per-signer token signingTokens table — extend with signerEmail TEXT NOT NULL column
Per-signer field filtering Filter signatureFields JSONB by field.signerEmail at query time
Completion detection Query document_signers WHERE signedAt IS NULL
Parallel email dispatch nodemailer (already installed) — Promise.all([sendMail(...), sendMail(...)])
Final PDF merge after all sign @cantoo/pdf-lib (already installed)
Agent notification on completion sendAgentNotificationEmail() already implemented in signing-mailer.tsx
Final PDF to all parties nodemailer + @react-email/render (already installed)

Schema additions needed (pure Drizzle migration, no new packages):

  1. New table document_signers — one row per (document, signer email). Columns: id, documentId (FK → documents), signerEmail, signerName (optional), signedAt (nullable timestamp), ipAddress (captured at signing), tokenJti (FK → signingTokens).

  2. New field on SignatureFieldData interfacesignerEmail?: string. Fields without signerEmail are agent-only fields (already handled). Fields with signerEmail route to that signer's session.

  3. Extend documentStatusEnum — add 'PartialSigned' (some but not all signers complete). 'Signed' continues to mean all signers have completed.

  4. Extend auditEventTypeEnum — add 'all_signers_complete' for the completion notification trigger.

  5. Extend signingTokens table — add signerEmail text NOT NULL column so each token is scoped to one signer and the signing page can filter fields correctly.

Docker: No New Application Packages

The Docker setup is infrastructure only (Dockerfile + docker-compose.yml). No npm packages are added to the application.

One optional dev-only Docker image for local email testing:

Tool What It Is When to Use
maildev/maildev:latest Lightweight SMTP trap that catches all outbound mail and shows it in a web UI Add to a docker-compose.override.yml for local development only. Never deploy to production.

Docker Stack

Image Versions

Service Image Rationale
Next.js app node:20-alpine LTS, small. Do NOT use node:24 — @napi-rs/canvas ships prebuilt .node binaries and the build for node:24-alpine may not exist yet. Verify before upgrading.
PostgreSQL postgres:16-alpine Current stable, alpine keeps it small. Pin to 16-alpine explicitly — never use postgres:latest which silently upgrades major versions on docker pull.

Dockerfile Pattern — Three-Stage Standalone

Next.js output: 'standalone' in next.config.ts must be enabled. This generates .next/standalone/ with a minimal self-contained Node.js server. Reduces image from ~7 GB (naive) to ~300 MB (verified across multiple production reports).

Stage 1 — deps: npm ci --omit=dev with cache mounts. This layer is cached until package-lock.json changes, making subsequent builds fast.

Stage 2 — builder: Copy deps from Stage 1, copy source, run next build. Set NEXT_TELEMETRY_DISABLED=1 and NODE_ENV=production.

Stage 3 — runner: Copy .next/standalone/, .next/static/, and public/ from builder only. Set HOSTNAME=0.0.0.0 and PORT=3000. Run as non-root user. The uploads/ directory must be a named Docker volume — never baked into the image.

@napi-rs/canvas native binding note: This package includes a compiled .node binary for a specific OS and CPU architecture. The build stage and runner stage must use the same OS (both node:20-alpine) and the Docker build must run on the same CPU architecture as the deployment server (arm64 for Apple Silicon servers, amd64 for x86). Use docker buildx --platform linux/arm64 or linux/amd64 explicitly if building cross-platform. Cross-architecture builds will produce a binary that silently fails at runtime.

Database migrations on startup: Add an entrypoint.sh that runs npx drizzle-kit migrate && exec node server.js. This ensures schema migrations run before the application accepts traffic on every container start.

Docker Compose Structure

services:
  app:
    build: .
    restart: unless-stopped
    depends_on:
      db:
        condition: service_healthy
    env_file: .env.production        # SMTP creds, AUTH_SECRET, OPENAI_API_KEY etc.
    environment:
      NODE_ENV: production
      DATABASE_URL: postgresql://appuser:${POSTGRES_PASSWORD}@db:5432/tchmesapp
    volumes:
      - uploads:/app/uploads         # PDFs written at runtime
    ports:
      - "3000:3000"
    networks:
      - internal

  db:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: appuser
      POSTGRES_DB: tchmesapp
      POSTGRES_PASSWORD_FILE: /run/secrets/postgres_password
    secrets:
      - postgres_password
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U appuser -d tchmesapp"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - internal

volumes:
  pgdata:
  uploads:

networks:
  internal:
    driver: bridge

secrets:
  postgres_password:
    file: ./secrets/postgres_password.txt

Key decisions:

  • db is on the internal bridge network only — not exposed to the host or internet.
  • app waits for db to pass its health check before starting (condition: service_healthy), preventing migration failures on cold boot.
  • restart: unless-stopped survives server reboots without systemd service files.
  • Named volumes for pgdata and uploads survive container recreation.
  • PostgreSQL uses a Docker secret for its password because Postgres natively supports POSTGRES_PASSWORD_FILE. The app reads its DATABASE_URL from .env.production via env_file — no code change needed.

SMTP / Email in Docker

Actual Environment Variable Names (Code-Verified)

The signing mailer (src/lib/signing/signing-mailer.tsx) reads these exact env vars:

CONTACT_SMTP_HOST=smtp.gmail.com
CONTACT_SMTP_PORT=587
CONTACT_EMAIL_USER=teressa@tcopelandhomes.com
CONTACT_EMAIL_PASS=xxxx-xxxx-xxxx-xxxx
AGENT_EMAIL=teressa@tcopelandhomes.com

These same vars are used by the contact form mailer (src/lib/contact-mailer.ts). Both mailers share the same SMTP transport configuration via the same env var names.

The Actual Problem

The current "email not working in Docker" bug is almost certainly one of two causes:

Cause 1 — Environment variables not passed to the container. Docker does not inherit host environment variables. If CONTACT_SMTP_HOST, CONTACT_EMAIL_PASS etc. are in .env.local on the host, they are invisible inside the container unless explicitly injected via env_file or environment: in docker-compose.yml.

Cause 2 — DNS resolution failure (EAI_AGAIN). Docker containers use Docker's internal DNS resolver (127.0.0.11). This can intermittently fail to resolve external hostnames, producing getaddrinfo EAI_AGAIN smtp.gmail.com. The symptom is email that works locally but silently fails (or errors) in Docker.

nodemailer itself is reliable in Docker containers. The multiple open GitHub issues on this topic all trace back to environment or DNS configuration problems, not library bugs.

Solution: env_file for Application Secrets

Docker Compose native secrets (non-Swarm) mount plaintext files at /run/secrets/secret_name. Application code must explicitly read those file paths. nodemailer reads credentials from process.env string values — not file paths. Rewriting the transporter initialization to read from /run/secrets/ would require code changes for no meaningful security gain on a single-server setup.

The correct approach for this application is env_file:

  1. Create .env.production on the server (never commit, add to .gitignore):

    CONTACT_SMTP_HOST=smtp.gmail.com
    CONTACT_SMTP_PORT=587
    CONTACT_EMAIL_USER=teressa@tcopelandhomes.com
    CONTACT_EMAIL_PASS=xxxx-xxxx-xxxx-xxxx
    AGENT_EMAIL=teressa@tcopelandhomes.com
    AUTH_SECRET=<long-random-string>
    OPENAI_API_KEY=sk-...
    POSTGRES_PASSWORD=<db-password>
    NEXT_PUBLIC_BASE_URL=https://teressacopelandhomes.com
    
  2. In docker-compose.yml, reference it:

    services:
      app:
        env_file: .env.production
    
  3. All variables in that file are injected as process.env.* inside the container. nodemailer reads process.env.CONTACT_EMAIL_PASS exactly as in development. Zero code changes.

Why not Docker Swarm secrets for SMTP? Plain Compose secrets have no encryption — they are just bind-mounted plaintext files. The security profile is identical to a chmod 600 .env.production file. The complexity cost (code that reads from /run/secrets/) is not justified on a single-server home deployment. Use Docker secrets for PostgreSQL password only because PostgreSQL natively reads from POSTGRES_PASSWORD_FILE — no code change required.

DNS Fix for EAI_AGAIN

If SMTP resolves correctly in dev but fails in Docker, add to the app service:

services:
  app:
    dns:
      - 8.8.8.8
      - 1.1.1.1
    environment:
      NODE_OPTIONS: --dns-result-order=ipv4first

The dns: keys bypass Docker's internal resolver for external lookups. --dns-result-order=ipv4first tells Node.js to try IPv4 DNS results before IPv6, which resolves the most common Docker DNS timeout pattern (IPv6 path unreachable, long timeout before IPv4 fallback).

SMTP Provider

Gmail with an App Password (not the account password) is the recommended choice for a solo agent at low volume. Requires 2FA enabled on the Google account. The signing mailer already uses port logic: port 465 → secure: true; any other port → secure: false. Port 587 with STARTTLS is more reliable than port 465 implicit TLS in Docker environments — use CONTACT_SMTP_PORT=587.


What NOT to Add (v1.2)

Temptation Why to Avoid
Redis / BullMQ for email queuing Overkill. This app sends at most 5 emails per document. Promise.all([sendMail(...)]) is sufficient. Redis adds a third container, more ops burden, and more failure modes.
Resend / SendGrid / Postmark Adds a paid external dependency. nodemailer + Gmail App Password is free, already implemented, and reliable when env vars are correctly passed. Switch only if Gmail SMTP becomes a persistent problem.
Docker Swarm secrets for SMTP Requires code changes to read from file paths. No security benefit over a permission-restricted env_file on single-server non-Swarm setup.
postgres:latest image Will silently upgrade major versions on docker pull. Always pin to postgres:16-alpine.
Node.js 22 or 24 as base image @napi-rs/canvas ships prebuilt .node binaries. Verify the binding exists for the target node/alpine version before upgrading. Node 20 LTS is verified.
Sequential signing enforcement The PROJECT.md specifies parallel signing only ("any order"). Do not add sequencing logic.
WebSockets for real-time signing status Polling the agent dashboard every 30 seconds is sufficient for one agent monitoring a handful of documents. No WebSocket infrastructure needed.
Separate migration container A depends_on: condition: service_completed_successfully init container is architecturally cleaner but adds complexity. An entrypoint.sh in the same app container is simpler and sufficient at this scale.
HelloSign / DocuSign integration Explicitly out of scope per PROJECT.md. Custom e-signature is the intentional choice.
unpdf Already documented in v1.1 "What NOT to Add" — the existing pdfjs-dist legacy build is in use and working.

OpenSign Architecture Reference

OpenSign (React + Node.js + MongoDB) implements multi-recipient signing as a signers array embedded in each document record. Each signer object holds its own status, token reference, and completed-at timestamp. All signing links are sent simultaneously (parallel) by default. Their MongoDB document array maps directly to a PostgreSQL document_signers junction table in relational terms. The core insight confirmed by OpenSign's design: multi-signer needs no specialized packages — it is a data model and routing concern.


Sources (v1.2)


Last updated: 2026-04-03 — v1.2 multi-signer and Docker production additions; corrected unpdf status, env var names, and added missing packages to existing stack table