OCR

A scanned PDF or a photographed page contains no extractable text — only pixels. OCR reads the text back out. It is opt-in: pass the ocr option and @cognipeer/to-markdown runs the recognizer, otherwise nothing extra happens.

Enabling OCR

Pass ocr: true to accept the defaults (Tesseract, English, pdfMode: 'auto'):

import { convertToMarkdown } from '@cognipeer/to-markdown';

const markdown = await convertToMarkdown('./scan.pdf', { ocr: true });

Or pass an options object for more control:

const markdown = await convertToMarkdown('./receipt.png', {
  ocr: { provider: 'tesseract', lang: 'eng' }
});

Installing the Tesseract dependency

The default tesseract provider relies on a peer dependency. Install it alongside the package:

npm install tesseract.js

The vision-model providers use the native fetch API and require no additional install.

Providers

Provider	Engine	Requires
tesseract	Tesseract.js, runs locally (default)	tesseract.js peer dependency
openai-vlm	OpenAI vision model	vlm.model, vlm.apiKey
anthropic-vlm	Anthropic vision model	vlm.model, vlm.apiKey
ollama-vlm	Local Ollama vision model	vlm.model (apiBase defaults to http://localhost:11434)
azure-vision	Azure Computer Vision Read API	vlm.apiKey, vlm.apiBase (your Azure endpoint)
custom-vlm	Any OpenAI-compatible vision endpoint	vlm.model, vlm.apiBase

Options

interface OCROptions {
  provider?: OCRProvider;                  // default: 'tesseract'
  lang?: string;                           // default: 'eng' — tesseract only
  pdfMode?: 'auto' | 'always' | 'never';   // default: 'auto'
  vlm?: VLMOptions;                        // required for VLM providers
}

provider

Which recognizer to use. Defaults to tesseract, which runs entirely on your machine. The VLM providers send page images to a hosted or local vision model instead — better on messy layouts and handwriting, at the cost of a network call and per-token spend.

lang

Tesseract language code, defaulting to eng. Only applies to the tesseract provider.

pdfMode

Controls when a PDF gets OCR'd, which matters because rendering and recognizing every page is expensive:

• 'auto' (default) — try native text extraction first, and fall back to OCR only when the page yields no text. This is the right setting for a mixed corpus of digital and scanned PDFs.
• 'always' — OCR every page regardless of extractable text. Use when the embedded text layer is known to be wrong.
• 'never' — skip OCR for PDFs entirely, even when ocr is enabled. Images are still OCR'd.

vlm

Configuration for the vision-model providers:

interface VLMOptions {
  model: string;        // e.g. 'gpt-4o', 'claude-3-haiku-20240307', 'llava'
  apiKey?: string;      // required for openai-vlm, anthropic-vlm, azure-vision, custom-vlm
  apiBase?: string;     // Azure endpoint, Ollama host, or custom endpoint
  prompt?: string;      // custom extraction prompt; defaults to a generic OCR prompt
  maxTokens?: number;   // default: 4096
}

Examples

// 1) OCR a scanned PDF with the default provider (tesseract)
const scannedPdf = await convertToMarkdown('./scan.pdf', {
  ocr: true,
});

// 2) OCR an image with explicit tesseract options
const imageText = await convertToMarkdown('./receipt.png', {
  ocr: { provider: 'tesseract', lang: 'eng' },
});

// 3) OCR a PDF only when native text extraction comes back empty
const autoFallback = await convertToMarkdown('./document.pdf', {
  ocr: { pdfMode: 'auto' },
});

Using a vision model

const markdown = await convertToMarkdown('./handwritten-notes.jpg', {
  ocr: {
    provider: 'openai-vlm',
    vlm: {
      model: 'gpt-4o',
      apiKey: process.env.OPENAI_API_KEY,
      prompt: 'Transcribe this page as Markdown, preserving headings and tables.',
    },
  },
});

Using a local Ollama model

const markdown = await convertToMarkdown('./scan.pdf', {
  ocr: {
    provider: 'ollama-vlm',
    pdfMode: 'always',
    vlm: {
      model: 'llava',
      apiBase: 'http://localhost:11434',
    },
  },
});

See Also

• Converter Options
• Type Definitions
• Format Support