dwzhu-pku/PaperBanana: Automate Academic Illustrations with Multi-Agent AI

Creating publication-quality diagrams and plots is a persistent bottleneck in academic research. AI scientists routinely spend hours translating complex methodologies into visual formats that meet journal standards—time diverted from actual research. PaperBanana, an open-source Python^{↗ Bright Coding Blog} framework maintained by Dawei Zhu and collaborators from Peking University and Google Research, addresses this directly through a reference-driven multi-agent pipeline that generates academic illustrations from raw scientific content.

With 6,780 GitHub stars and 509 forks as of its latest commit on June 25, 2026, PaperBanana has gained substantial traction in the research community. The project is licensed under Apache 2.0 and represents a community-driven evolution of the original Google Research project PaperVizAgent. This article breaks down how PaperBanana works, what it can realistically do for your research workflow, and how to get it running.

What is dwzhu-pku/PaperBanana?

PaperBanana is a reference-driven multi-agent framework for automated academic illustration generation. It operates as a coordinated system of specialized AI agents—Retriever, Planner, Stylist, Visualizer, and Critic—that transform scientific method descriptions and figure captions into publication-ready diagrams and plots.

The project originated from research conducted during Dawei Zhu's internship at Google, where the core methodology was developed and patented by Google. The original implementation was released as PaperVizAgent under Google Research. PaperBanana forks that codebase with explicit intent: to continue open-source evolution focused on broader academic illustration support, without commercial direction. The maintainers state clearly that their goal is community benefit, not monetization.

Key technical characteristics:

• Primary language: Python 3.12
• Package management: uv (modern Python package installer)
• Interface options: Gradio web app (recommended), Streamlit interactive demo, command-line interface
• Model support: Google Gemini, OpenRouter (unified access to OpenAI, Anthropic, and other providers)
• Task coverage: Conceptual diagrams and statistical plots
• Image resolution: Configurable up to 4K via API generation

The framework's architecture reflects current advances in multi-agent LLM systems. Each agent handles a distinct phase of the illustration pipeline, with the Critic agent enabling iterative refinement loops that progressively improve output quality. This design acknowledges a critical reality: single-pass image generation from text prompts rarely produces publication-quality academic visuals.

Key Features

Five-Agent Orchestration Pipeline

The core technical differentiation is the structured agent pipeline:

1. Retriever Agent: Performs generative retrieval from a curated reference collection (PaperBananaBench) to identify stylistically and semantically relevant examples for in-context learning
2. Planner Agent: Translates method content and communicative intent into comprehensive textual descriptions
3. Stylist Agent: Refines descriptions against automatically synthesized style guidelines to enforce academic aesthetic standards
4. Visualizer Agent: Executes image generation through state-of-the-art image generation APIs
5. Critic Agent: Forms a closed-loop refinement mechanism with the Visualizer through multi-round iterative improvements

Flexible Experiment Modes

PaperBanana exposes six pipeline configurations for different use cases:

Parallel Generation Infrastructure

The Gradio and Streamlit interfaces support generating up to 20 candidate diagrams simultaneously, with batch export as PNG or ZIP. This addresses a practical research need: comparing multiple visual interpretations before selecting the most effective communication.

High-Resolution Refinement

Post-generation upscaling to 2K or 4K resolution is available through the "Refine Image" workflow, using the same image generation APIs rather than traditional super-resolution techniques.

Extensible Agent Architecture

Each agent is independently configurable through modular Python classes in the agents/ directory. The framework supports custom model selection for both the main VLM (vision-language model) and image generation backend, including preset options and custom API endpoint input.

Use Cases

Neural Architecture Diagrams

Researchers describing novel transformer variants or GNN architectures can input method sections and receive structured diagrams showing layer connectivity, attention flows, or message-passing operations. The Retriever Agent identifies relevant reference diagrams from computer science literature to guide visual conventions.

Training Pipeline Visualizations

Complex multi-stage training procedures—common in reinforcement learning from human feedback (RLHF), adversarial training, or curriculum learning—can be rendered as flow diagrams. The Planner Agent's textual description phase ensures methodological accuracy before visualization begins.

Statistical and Experimental Result Plots

While plot generation code remains partially in development (noted in the TODO list), the framework supports matplotlib-based plot code generation through the legacy vanilla mode with task_name=plot. This suits researchers needing standard visualization types (line plots, bar charts, error-bar plots) from structured data.

Figure Refinement and Style Standardization

Existing diagrams can be uploaded for refinement against academic style guidelines, with explicit change requests (e.g., "increase contrast," "standardize font sizes," "convert to two-column format"). The Stylist Agent's automatically synthesized guidelines help enforce venue-specific visual standards.

Cross-Domain Method Illustration

The reference-driven approach enables adaptation to new domains as the reference set expands. Current coverage focuses on computer science; the maintainers explicitly note expansion to additional fields as active development priority.

Installation & Setup

PaperBanana requires Python 3.12 and uses uv for dependency management. Follow these exact steps from the repository documentation:

Step 1: Clone the repository

git clone https://github.com/dwzhu-pku/PaperBanana.git
cd PaperBanana

Step 2: Configure API access

Duplicate the template configuration file:

cp configs/model_config.template.yaml configs/model_config.yaml

Edit configs/model_config.yaml to specify:

• defaults.main_model_name: Your chosen VLM (e.g., Gemini, Claude, GPT-4V via OpenRouter)
• defaults.image_gen_model_name: Image generation backend
• At least one API key under api_keys: either google_api_key (Gemini) or openrouter_api_key (OpenRouter unified API)

Note: You do not need both keys. If both are configured, OpenRouter is preferred for routing when available. High-concurrency generation requires an API key supporting sufficient rate limits.

Step 3: Download reference dataset (optional but recommended)

# Create data directory and download PaperBananaBench from Hugging Face
# Place under data/PaperBananaBench/

The framework functions without the dataset by bypassing the Retriever Agent's few-shot learning. Original PDFs are available separately at PaperBananaDiagramPDFs.

Step 4: Install environment

# Install uv if not present
# See: https://docs.astral.sh/uv/getting-started/installation/

# Create virtual environment
uv venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install Python 3.12
uv python install 3.12

# Install dependencies
uv pip install -r requirements.txt

Real Code Examples

Example 1: Launch the recommended Gradio interface

python app.py

The Gradio app provides the most accessible entry point. The Figure Size setting maps physical dimensions to API resolution parameters: 1-3cm and 4-6cm map to 1k, 7-9cm and 10-13cm to 2k, and 14-17cm to 4k for Gemini and OpenRouter calls. OpenAI's gpt-image uses its fixed-size API path regardless of this setting.

Example 2: Full pipeline command-line execution

python main.py \
  --dataset_name "PaperBananaBench" \
  --task_name "diagram" \
  --split_name "test" \
  --exp_mode "dev_full" \
  --retrieval_setting "auto"

This runs the complete agent pipeline with automatic retrieval from the benchmark dataset. The --exp_mode "dev_full" activates all five agents; --retrieval_setting "auto" enables the Retriever Agent's example selection.

Example 3: Legacy plot generation without retrieval

python main.py \
  --dataset_name "PaperBananaBench" \
  --task_name "plot" \
  --split_name "test" \
  --exp_mode "vanilla" \
  --retrieval_setting "none"

This produces matplotlib code directly without planning or refinement stages—useful for baseline comparison or when working with structured tabular data rather than conceptual illustrations.

Example 4: Launch Streamlit interactive demo

streamlit run demo.py

The Streamlit interface exposes two tabs: "Generate Candidates" for parallel diagram generation with configurable pipeline modes, and "Refine Image" for post-hoc modifications and upscaling of existing visuals.

Example 5: Pipeline evolution visualization

streamlit run visualize/show_pipeline_evolution.py

This diagnostic tool traces how diagrams evolve through Planner → Stylist → Critic stages, useful for understanding failure modes and tuning agent behavior.

Advanced Usage & Best Practices

API Key Strategy: For research groups, consider OpenRouter as the primary configuration. It provides unified access to multiple providers without managing separate billing relationships, and PaperBanana's routing logic prefers it when available.

Retrieval Setting Selection: The auto retrieval setting works well for standard computer science methods. For novel architectures poorly represented in PaperBananaBench, manual or random may yield more diverse stylistic starting points—though this requires curating your own reference set.

Critic Round Trade-offs: Each additional critic round improves quality at linear latency and API cost increase. For exploratory drafts, dev_planner or dev_planner_stylist suffices. Reserve dev_full with multi-round critic for final submission figures.

Concurrent Generation Limits: The 20-candidate parallel generation requires API keys with sufficient rate limits. Google's Gemini and OpenRouter tiers vary significantly; test with small batches before scaling.

Community Forks: The maintainers actively reference community efforts including PaperBanana-Pro (Chinese-enhanced, stability-focused) and independent reproductions. Evaluate these if your use case aligns with their specific improvements.

Comparison with Alternatives

PaperBanana's distinctive advantage is the explicit multi-agent refinement loop and academic style guideline synthesis. However, tools emphasizing editable outputs may better serve workflows requiring frequent post-hoc adjustments in vector graphics editors.

FAQ

Q: Is PaperBanana free to use? A: The code is Apache 2.0 licensed and open-source. You pay only for API usage (Gemini, OpenRouter, etc.). The maintainers state no commercialization plans.

Q: Can I run it without API keys? A: No. At minimum, one API key for Google Gemini or OpenRouter is required for any generation.

Q: What Python version is required? A: Python 3.12, managed through uv.

Q: Does it work without the PaperBananaBench dataset? A: Yes, but the Retriever Agent's few-shot learning is bypassed, potentially reducing output quality.

Q: Are generated figures editable? A: The README does not explicitly mention vector output formats; generated images appear to be raster (PNG). Community forks like Edit-Banana focus on editability.

Q: Can I use this for non-computer-science fields? A: Currently the reference set is CS-focused. Expansion to other domains is on the TODO list.

Q: What about the Google patents mentioned? A: Patents cover the core workflows developed at Google. This restricts third-party commercial applications using similar logic, but does not affect open-source research use.

Conclusion

PaperBanana represents a pragmatic, engineering-focused approach to automating one of research's most time-consuming visual tasks. Its multi-agent architecture directly addresses the failure modes of single-pass text-to-image generation for academic content—namely, methodological inaccuracy and inconsistent adherence to publication style conventions.

The tool best serves AI/ML researchers who regularly produce architecture diagrams and training flowcharts, particularly those comfortable with Python environments and API-based workflows. It is not yet a turnkey solution for all scientific visualization needs—statistical plot generation remains partially in development, and cross-domain expansion is ongoing.

For researchers matching its current capabilities, PaperBanana offers genuine time savings through parallel candidate generation and iterative refinement that would be impractical to replicate manually. The active community ecosystem and explicit non-commercial orientation suggest sustainable open-source evolution.

Explore the repository, try the Hugging Face Spaces demo, or clone locally from https://github.com/dwzhu-pku/PaperBanana.

Related: [INTERNAL_LINK: multi-agent-llm-frameworks] for broader context on agent orchestration patterns in research tools.