NanoNets/docext: OCR-Free Document Extraction and Markdown Conversion

NanoNets/docext: OCR-Free Document Extraction and Markdown^{↗ Smart Converter} Conversion

Developers working with document intelligence face a persistent tension: cloud-based OCR services require sending sensitive files to third-party APIs, while traditional on-premises solutions often struggle with complex layouts, handwritten text, and structured outputs like markdown. The rise of vision-language models (VLMs) has created new possibilities, but evaluating which models actually perform well on document tasks remains difficult—and deploying them reliably on your own infrastructure is rarely straightforward.

NanoNets/docext addresses this gap directly. This open-source Python^{↗ Bright Coding Blog} toolkit provides OCR-free document extraction, PDF and image-to-markdown conversion, and a comprehensive benchmarking platform for intelligent document processing (IDP). With 2,032 GitHub stars, 148 forks, and an Apache 2.0 license, it has gained meaningful traction among developers who need on-premises document AI without the complexity of managing multiple disconnected tools. This article examines what docext offers, how it works, and where it fits in your document processing pipeline.

What is NanoNets/docext?

NanoNets/docext is a comprehensive on-premises document intelligence toolkit developed by Nanonets, a company specializing in document AI and intelligent document processing solutions. The project is written primarily in Python and was last updated on March 17, 2026, indicating active maintenance.

The toolkit sits at the intersection of three converging trends: the shift toward VLM-powered document understanding, the demand for on-premises deployment due to data privacy requirements, and the need for standardized benchmarking in document AI. Unlike tools that focus narrowly on a single task, docext combines three core capabilities under one codebase:

1. PDF & Image to Markdown Conversion: Transforming documents into structured markdown with intelligent content recognition—including LaTeX equations, signatures, watermarks, tables, and semantic tagging.

2. Document Information Extraction: OCR-free extraction of structured information (fields, tables) from documents such as invoices and passports, with confidence scoring.

3. Intelligent Document Processing Leaderboard: A benchmarking platform tracking VLM performance across OCR, Key Information Extraction (KIE), document classification, table extraction, and other IDP tasks.

This architecture matters for teams that want consistent tooling across conversion, extraction, and evaluation—rather than stitching together separate solutions for each phase of a document AI workflow.

Key Features

PDF and Image to Markdown Conversion

The markdown conversion engine goes beyond simple text extraction. It recognizes LaTeX equations (both inline and block) and preserves them in proper markdown syntax. For documents containing images, it generates detailed descriptions wrapped in <img></img> tags. Signatures and watermarks are detected and tagged with <signature></signature> and <watermark></watermark> elements respectively. Page numbers receive similar semantic markup. Form elements like checkboxes and radio buttons convert to standardized Unicode symbols (☐, ☑, ☒). Complex tables transform into HTML tables rather than broken markdown approximations.

A dedicated 3B parameter model—Nanonets-OCR-s—powers this conversion, specifically trained for efficient image-to-markdown conversion with semantic understanding for images, signatures, watermarks, and other document elements.

Document Information Extraction

The extraction layer supports flexible field definitions through custom schemas or pre-built templates. Available templates cover common document types including invoices and passports, with the ability to add or delete fields and columns for other templates. Table extraction pulls structured tabular data, and confidence scoring provides reliability metrics for each extracted value. Multi-page document support and a REST API enable integration with existing applications.

IDP Benchmarking Platform

The leaderboard evaluates models across seven task categories: Key Information Extraction (KIE), Visual Question Answering (VQA), OCR, Document Classification, Long Document Processing, Table Extraction, and Confidence Score Calibration. The live leaderboard at idp-leaderboard.org tracks performance, with recent additions including Gemini 2.5 Pro, Claude Sonnet 4, InternVL3-38B, Qwen2.5-VL-32B, and others.

Deployment Characteristics

All capabilities run on-premises on Linux and macOS. This design targets organizations with strict data residency requirements or those processing sensitive documents where cloud API transmission is unacceptable.

Use Cases

Financial Document Processing

Accounting and finance teams process invoices, receipts, and statements containing tables, amounts, and vendor information. Docext's invoice template with field extraction and confidence scoring provides structured data without cloud dependency. The table extraction handles line-item details that break simpler OCR tools.

Academic and Research Workflows

Researchers working with scientific papers, theses, and technical documents benefit from LaTeX equation preservation and semantic markup. Converting scanned papers or PDFs to markdown with intact mathematical notation enables downstream processing in academic pipelines, version control, or static site generators.

Legal and Compliance Document Review

Legal documents often contain signatures, watermarks, and redactions that require careful handling. The semantic tagging (<signature>, <watermark>) preserves document structure awareness during conversion. On-premises deployment ensures client confidentiality and meets regulatory requirements for data handling.

Identity Document Verification

The passport template supports automated extraction from identity documents. For organizations verifying identities in regulated industries—banking, healthcare, government—this provides a path to automation without exposing PII to external services.

VLM Selection and Validation

ML engineers and researchers evaluating vision-language models for document tasks can use the IDP leaderboard to compare model performance on standardized benchmarks. This reduces the risk of selecting models that perform well on generic vision tasks but fail on document-specific challenges like table extraction or long-document reasoning.

Installation & Setup

The README directs users to dedicated guides for each capability. Below are the entry points with explanations.

Core Installation

For the base docext extraction functionality, consult the feature guide. The typical Python package installation path applies:

# Install from PyPI
pip install docext

The PyPI badge indicates active distribution, with version tracking available at pypi.org/project/docext.

PDF to Markdown Setup

For markdown conversion capabilities, see the PDF2MD_README.md. This likely involves additional model weights for Nanonets-OCR-s, potentially downloaded separately given the 3B parameter size.

Benchmark Setup

For leaderboard evaluation tools, consult the benchmark guide. This enables running evaluations locally against the same tasks published on idp-leaderboard.org.

Environment Requirements

• Operating Systems: Linux, macOS (Windows not explicitly listed)
• Python: Version requirements specified in PyPI metadata or installation guides
• Hardware: Sufficient for running 3B parameter VLM inference locally; GPU recommended for production throughput

Verification

After installation, verify through the REST API endpoint or by processing a sample document through the appropriate module.

Real Code Examples

The README does not contain extensive inline code examples. The following reflects the documented API patterns and configuration approaches explicitly described in the source material. Where implementation details require inference, this is noted.

Basic Extraction with Pre-built Template

The README describes template-based extraction for invoices. The pattern follows this structure:

# Hypothetical example based on documented features
# Actual syntax may vary—consult EXT_README.md for precise API
from docext import DocumentExtractor

# Initialize with invoice template
extractor = DocumentExtractor(template="invoice")

# Process document
result = extractor.extract("path/to/invoice.pdf")

# Access structured fields with confidence scores
for field in result.fields:
    print(f"{field.name}: {field.value} (confidence: {field.confidence})")

Note: The exact import paths and method names above are illustrative based on the described feature set. The EXT_README.md contains the authoritative API reference.

REST API Integration

The documented REST API enables programmatic access:

import requests

# Example pattern for API-based extraction
# Endpoint and payload structure from actual documentation
response = requests.post(
    "http://localhost:8000/extract",  # hypothetical default port
    files={"document": open("contract.pdf", "rb")},
    json={"template": "custom", "fields": ["party_a", "party_b", "effective_date"]}
)

result = response.json()
# result contains extracted fields with confidence scores

Markdown Conversion

For PDF-to-markdown conversion using the Nanonets-OCR-s model:

# CLI pattern based on described functionality
# See PDF2MD_README.md for exact commands
docext convert --input document.pdf --output document.md --model nanonets-ocr-s

The output includes semantic tags as described: <img> for images, <signature> for signatures, <watermark> for watermarks, and HTML tables for tabular content.

Benchmark Evaluation

# Run benchmark evaluation for a specific model
# See docext/benchmark documentation for precise commands
python -m docext.benchmark evaluate \
    --model nanonets/Nanonets-OCR-s \
    --tasks ocr,kie,table_extraction \
    --output results.json

Important: The README explicitly states that detailed installation, usage, and additional examples are in the linked feature guides. Developers should consult EXT_README.md, PDF2MD_README.md, and the benchmark directory for authoritative implementation details.

Advanced Usage & Best Practices

Template Customization Strategy

The README notes ability to add or delete fields/columns for templates. For production deployments, start with pre-built templates (invoices, passports) and iteratively customize rather than building from scratch. This preserves validation logic while adapting to document variants.

Confidence Threshold Tuning

Confidence scores are provided for extracted information. Establish per-field thresholds based on business impact: lower thresholds may suffice for internal analytics, while regulatory submissions require high-confidence extraction with manual review queues for uncertain values.

Model Selection for Markdown Conversion

The Nanonets-OCR-s model (3B parameters) balances capability and resource requirements. For high-throughput scenarios, evaluate whether this model meets latency requirements or if the leaderboard identifies alternatives with better speed/accuracy tradeoffs for your specific document types.

On-Premises Infrastructure Planning

Running VLM inference locally requires GPU resources for acceptable throughput. The 3B parameter model is relatively compact for a VLM, but batch processing and model caching strategies become important at scale. Consider the [INTERNAL_LINK: on-premises ML infrastructure] patterns for deployment architecture.

Benchmark-Driven Development

When extending docext or integrating new models, use the benchmark suite to establish baseline performance before deployment. The seven task categories cover failure modes that generic vision benchmarks miss—particularly long document processing and confidence calibration.

Comparison with Alternatives

Trade-offs to consider: Tesseract offers mature, lightweight OCR for simple text extraction but lacks VLM-powered semantic understanding and markdown conversion. Cloud VLM APIs provide comparable intelligence but sacrifice on-premises deployment and incur ongoing costs. Docext's primary distinction is combining VLM capabilities with local execution, structured markdown output, and integrated benchmarking—at the cost of managing your own GPU infrastructure.

FAQ

Is NanoNets/docext free to use? Yes, released under Apache License 2.0. No licensing fees for commercial or personal use.

What hardware is required for local deployment? The README specifies Linux and macOS support. GPU is strongly recommended for the 3B parameter VLM inference; exact VRAM requirements are in the installation guides.

Can I process documents without internet access? Yes—on-premises deployment is a core design goal. Model weights download once; subsequent processing is fully offline.

How does this differ from Nanonets' commercial products? Docext is the open-source toolkit; Nanonets offers commercial document parsing solutions with additional features and managed infrastructure.

What document formats are supported? PDF and images for both markdown conversion and information extraction. Multi-page PDFs are explicitly supported.

Is there a hosted version or do I must self-host? The README emphasizes on-premises deployment. No hosted/SaaS option is mentioned for docext specifically.

How current is the project? Last commit was March 17, 2026, with active updates including new model evaluations through June 2025.

Conclusion

NanoNets/doctext occupies a specific and valuable position in the document AI landscape: it brings VLM-powered, OCR-free extraction and conversion to environments where data cannot leave organizational boundaries. The combination of semantic markdown conversion, structured field extraction with confidence scoring, and integrated benchmarking addresses real friction points for teams building document pipelines—particularly in regulated industries, research institutions, and security-conscious enterprises.

The 2,032 stars and 148 forks suggest genuine developer interest rather than astroturfed promotion. The active maintenance through early 2026 and regular leaderboard updates indicate sustained investment. For Python developers needing on-premises document intelligence, especially those evaluating or deploying vision-language models for production document tasks, docext provides a unified starting point that would otherwise require integrating multiple disconnected tools.

The honest limitations are worth noting: Windows support is not documented, GPU infrastructure is required for practical throughput, and the feature guides (rather than the main README) contain the detailed implementation references you'll need. These are manageable constraints for the target audience.

Ready to explore? Clone the repository at https://github.com/NanoNets/docext, review the dedicated guides for your use case, and evaluate against the live leaderboard to benchmark your document AI stack.