Stop Guessing Your Mac's LLM Performance! Anubis Exposes the Truth

Stop Guessing Your Mac's LLM Performance! Anubis Exposes the Truth

Your Mac is screaming at you, and you can't even hear it.

Every time you run a local LLM—whether it's Llama 3.2, DeepSeek-R1, or Qwen3—your Apple Silicon chip is making thousands of micro-decisions per second. GPU cores firing. Memory bandwidth saturating. Thermal throttling kicking in when things get too hot. And what do you see? A blinking cursor and a vague sense that "this feels slow."

That's not engineering. That's superstition.

Here's the brutal truth: most developers running local LLMs on macOS are flying completely blind. They have zero visibility into whether their model is GPU-bound, memory-bound, or thermally throttled. They don't know if switching from Q4_K_M to Q8_0 quantization actually helped or hurt. They can't tell if Ollama is using the ANE (Apple Neural Engine) or just hammering the CPU. And when they try to compare two models? They're reduced to stopwatching token generation like it's 1999.

But what if you could see everything?

Enter Anubis, the open-source native macOS app that transforms local LLM benchmarking from guesswork into precise, repeatable science. Built with SwiftUI for Apple Silicon, Anubis doesn't just run benchmarks—it correlates real-time hardware telemetry with inference performance in ways no CLI tool or chat wrapper even attempts. GPU utilization, CPU power draw, ANE wattage, memory bandwidth, thermal states, GPU frequency scaling—all streaming live alongside your tokens per second.

The gap between "it works" and "it's optimized" just collapsed. And if you're serious about local AI on macOS, you need to understand why Anubis is becoming the secret weapon of developers who refuse to tolerate mystery meat performance.

What Is Anubis? The Native macOS LLM Benchmarking Powerhouse

Anubis is a native macOS application for benchmarking, comparing, and managing local large language models using any OpenAI-compatible endpoint. Created by uncSoft and released under GPL-3.0, it's specifically engineered for Apple Silicon—meaning M1, M2, M3, M4, and beyond. Intel Macs need not apply.

The project emerged from a clear void in the local LLM ecosystem. Chat wrappers like Ollama's native UI, LM Studio, and Jan prioritize conversation flow over systematic performance analysis. CLI monitors like asitop, macmon, and mactop expose hardware metrics but completely lack LLM context—you can see your GPU is at 80%, but you have no idea if that's from inference or your browser. Evaluation frameworks like promptfoo require YAML configuration and terminal expertise that many macOS developers simply don't have time for.

Anubis fuses these worlds. It's the only tool that correlates hardware telemetry—GPU/CPU/ANE/DRAM power consumption, thermal states, frequency scaling, per-core utilization—with actual inference metrics like time-to-first-token (TTFT), output tokens per second, prefill speed, and reasoning duration. All in real time. All with persistent history. All exportable.

The app has gained serious traction in the Apple Silicon AI community, with 375+ community-submitted benchmark runs analyzed in its open dataset and a growing community leaderboard that lets you compare your exact hardware configuration against others. Recent updates have added Apple Intelligence backend support (macOS 26+), reasoning-aware metrics for thinking models like DeepSeek-R1, hardware stress testing, and even a floating monitor HUD that stays visible across all Spaces.

This isn't a side project. It's becoming infrastructure for anyone who treats local LLM performance as a first-class concern.

Key Features That Separate Anubis from Everything Else

Anubis packs capabilities that sound like they should require three separate tools. Here's what makes it genuinely distinct:

Real-Time Hardware Telemetry During Inference Anubis captures Apple Silicon telemetry via IOReport and system APIs that most developers never touch directly. You get live GPU utilization percentage, CPU utilization across all cores, GPU/CPU/ANE/DRAM power consumption in watts, GPU frequency from P-state residency, process memory via phys_footprint (the same metric Activity Monitor uses, including Metal/GPU allocations), and system thermal state. These aren't sampled once per second—they stream continuously, synchronized with your inference run.

Reasoning-Aware Performance Metrics This is where Anubis shows deep domain expertise. For thinking models like DeepSeek-R1, Qwen3-thinking, GLM, and gpt-oss, Anubis excludes thinking time from output tokens per second. Previously, thinking tokens were erroneously counted as output, inflating numbers and hiding true performance. Now you get separate tracking: output tok/s (visible throughput only), reasoning tok/s, reasoning duration, and prefill tok/s as a first-class metric. The session detail view shows both metrics side by side.

Multi-Backend Flexibility Anubis speaks to Ollama (native support, auto-detected), Apple Intelligence (on-device Foundation Models, macOS 26+), LM Studio, mlx-lm, vLLM, LocalAI, Docker^{↗ Bright Coding Blog} ModelRunner, and any OpenAI-compatible server you configure. The backend abstraction uses a clean Swift protocol, so adding new inference engines is straightforward.

Arena Mode for A/B Comparison Side-by-side model comparison with the same prompt, supporting sequential mode (memory-safe, one at a time) or parallel mode (both simultaneously). Includes a voting system with persistence, per-panel stats grids with 9 metrics each, and comparison history.

System Monitor with Stress Testing Standalone hardware monitoring with CPU stress (yes processes per core, configurable for P-cores/E-cores/all/single), GPU stress (Metal compute shader rendering Mandelbrot fractals with four intensity levels), and memory bandwidth stress (memcpy-based saturation with measured GB/s). Includes safety mechanisms: 5-minute auto-timeout, thermal watchdog, GPU auto-downgrade if FPS drops below 5.

Floating Monitor HUD A compact, frameless, always-on-top overlay showing live CPU%, GPU%, memory, power, GPU frequency, and thermal state. Draggable, visible on all Spaces, with dark glass material.

Export and Reporting Direct export without screenshots: copy to clipboard, save as PNG (2x retina with watermark, respecting light/dark mode), or export raw data as Markdown^{↗ Smart Converter} or CSV. The Reports tab generates branded per-model summaries with hardware banners.

Privacy-First Leaderboard Submit results with one click—no account required, no response text ever uploaded. HMAC-signed submissions with server-side rate limiting. Filter by chip, model, quantization, and format for apples-to-apples comparison.

Use Cases: Where Anubis Transforms Your Workflow

1. Quantization Optimization for Production Deployment

You're deciding between Q4_K_M, Q5_K_M, and Q8_0 for a Llama 3.1 8B model serving internal tools. Anubis lets you benchmark each quantization with identical prompts, comparing not just tokens per second but GPU memory pressure, power draw, and thermal throttling points. You discover Q5_K_M hits a sweet spot: only 8% slower than Q8_0 but 40% less memory and significantly lower thermals under sustained load. That data goes straight into your deployment spec.

2. Reasoning Model Evaluation and Cost Projection

Your team wants to evaluate DeepSeek-R1 32B against Qwen3-30B-thinking for a reasoning-heavy application. Anubis's reasoning-aware metrics reveal that DeepSeek-R1 spends 60% of total time in thinking mode, with reasoning tok/s of 12 versus output tok/s of 28. Qwen3-30B-thinking is faster overall but produces lower-quality reasoning traces. You now have quantitative data for a decision that was previously based on vibes.

3. Hardware Procurement and Upgrade Justification

You need to convince your manager that upgrading from M3 Pro to M3 Max is worth it for local LLM inference. Anubis's leaderboard lets you filter by exact chip and model configuration. You show that M3 Max with 36GB achieves 2.3x higher output tok/s on your target models, with sustained performance that doesn't thermal-throttle like the Pro. The community data backs your business case.

4. Backend Performance Comparison and Migration Planning

You're currently on Ollama but considering mlx-lm for better Metal performance. Anubis's identical benchmark prompts across backends—with correlated hardware telemetry—reveals that mlx-lm achieves 15% higher GPU utilization but 20% higher memory footprint. For your memory-constrained deployment, Ollama remains optimal. Without Anubis, you'd be comparing incomparable numbers from different tools.

5. Thermal and Power Characterization for Edge Deployments

You're building a kiosk application that runs local LLM inference continuously. Anubis's stress testing and thermal monitoring reveal that sustained inference on M4 Pro triggers thermal throttling after 8 minutes in a sealed enclosure, dropping performance 35%. You redesign ventilation based on actual power draw data (peak 28W GPU+CPU) rather than manufacturer TDP estimates.

Step-by-Step Installation & Setup Guide

Prerequisites

Before starting, verify your system meets these requirements:

• macOS 15.0 (Sequoia) or later
• Apple Silicon (M1/M2/M3/M4/M5 or newer) — Intel Macs are not supported
• 8 GB unified memory minimum (16 GB+ strongly recommended for larger models)
• Xcode with Swift 5 toolchain for building from source
• At least one inference backend installed (Ollama recommended for easiest start)

Step 1: Install Ollama (Recommended Backend)

# Install Ollama via Homebrew
brew install ollama

# Start the Ollama server (keep this terminal running)
ollama serve

# In a new terminal, pull a model to benchmark
ollama pull llama3.2:3b

# Verify Ollama is responding
curl http://localhost:11434/api/tags

Step 2: Clone and Build Anubis from Source

# Clone the repository
git clone https://github.com/uncSoft/anubis-oss.git

# Navigate to the project directory
cd anubis-oss/anubis

# Open in Xcode (or use xcodebuild below)
open anubis.xcodeproj

In Xcode:

1. Select the project in the navigator
2. Go to Signing & Capabilities
3. Set your Development Team (required for code signing)
4. Select Product > Run (⌘R) or Product > Build (⌘B)

Alternative command-line build:

# Build debug configuration
xcodebuild -scheme anubis-oss -configuration Debug build

# Run tests
xcodebuild -scheme anubis-oss -configuration Debug test

Dependencies (GRDB.swift, Sparkle, Swift Charts) resolve automatically via Swift Package Manager on first build.

Step 3: Configure Backends

Anubis auto-detects Ollama on launch. For additional backends:

1. Open Settings (⌘,)
2. Click Add OpenAI-Compatible Server
3. Enter name, URL, and optional API key
4. Use quick presets for LM Studio (port 1234), mlx-lm (port 8080), vLLM (port 8000)

Smart URL handling: Anubis automatically strips /v1 suffixes to prevent double-pathing errors.

Step 4: Run Your First Benchmark

1. Select a model from the dropdown (auto-populated from detected backends)
2. Type a custom prompt or select from 15 preset prompts organized by category: Reasoning, Coding, Creative, Knowledge, Instruction
3. Click Run
4. Watch live metrics populate: Output Tok/s, GPU %, CPU %, TTFT with Prefill tok/s, Process Memory, Model Memory, Thermal State, GPU Frequency
5. Observe 7 live charts updating in real time

Step 5: Export or Submit Results

• Click Copy or Save for instant image export
• Access Session History for full replay and CSV/Markdown export
• Click Upload in the benchmark toolbar to submit to the community leaderboard

REAL Code Examples: Inside Anubis's Architecture

Anubis is open source, and its architecture reveals sophisticated engineering decisions. Let's examine actual patterns from the repository.

Example 1: Backend Abstraction Protocol

This Swift protocol is the foundation that lets Anubis support any inference backend uniformly:

protocol InferenceBackend {
    var id: String { get }
    var displayName: String { get }
    var isAvailable: Bool { get async }

    func listModels() async throws -> [ModelInfo]
    func generate(prompt: String, parameters: GenerationParameters)
        -> AsyncThrowingStream<InferenceChunk, Error>
}

Why this matters: The AsyncThrowingStream<InferenceChunk, Error> return type enables true streaming—tokens arrive incrementally rather than buffering the complete response. This is critical for real-time metrics: Anubis calculates tokens per second as they arrive, not after completion. The GenerationParameters struct encapsulates temperature, top-p, max tokens, and system prompts, ensuring consistent behavior across Ollama, MLX, Apple Intelligence, and custom servers. Adding a new backend means implementing this single protocol—no view changes required.

Example 2: Ollama Thinking Toggle (Version 3.2)

Anubis exposes Ollama's think parameter with careful backward compatibility:

// Tri-state enum for thinking control
enum ThinkingMode {
    case auto       // Omit field, use server-side default
    case on         // Force think: true
    case off        // Force think: false
}

// Applied when Ollama backend is selected
func applyThinkingMode(_ mode: ThinkingMode, to parameters: inout [String: Any]) {
    switch mode {
    case .auto:
        break // Intentionally omit the key
    case .on:
        parameters["think"] = true
    case .off:
        parameters["think"] = false
    }
}

Why this matters: The auto case omits the field entirely rather than sending think: null. This preserves compatibility with older Ollama versions and models that reject unknown parameters—a subtle but crucial detail for production reliability. The choice persists across launches via UserDefaults, respecting user preference without surprise resets.

Example 3: Process Memory Monitoring

Anubis uses proc_pid_rusage for accurate memory accounting that includes GPU allocations:

import Darwin

func getProcessMemory(pid: pid_t) -> UInt64? {
    var rusage = rusage_info_v4()
    let result = proc_pid_rusage(pid, RUSAGE_INFO_V4, &rusage)
    
    guard result == 0 else {
        return nil // Process may have exited
    }
    
    // phys_footprint includes Metal/GPU buffer allocations
    // This is the same metric Activity Monitor displays
    return rusage.ri_phys_footprint
}

Why this matters: Standard memory APIs often exclude GPU allocations, making MLX and other Metal-accelerated backends appear to use far less memory than reality. By using ri_phys_footprint, Anubis captures the true physical memory impact—including Metal buffers, IOSurfaces, and driver allocations. This explains why your 8GB Mac struggles despite "only" 6GB reported by simpler tools.

Example 4: Building from Source with xcodebuild

# Clone the complete repository
git clone https://github.com/uncSoft/anubis-oss.git
cd anubis-oss/anubis

# Command-line build for CI or automation
xcodebuild -scheme anubis-oss -configuration Debug build

# Run the full test suite
xcodebuild -scheme anubis-oss -configuration Debug test

# Or open in Xcode for interactive development
open anubis.xcodeproj

Why this matters: The scheme name anubis-oss matches the open-source target, distinct from any App Store version. The Debug configuration enables Swift Package Manager resolution without code signing requirements for local testing. For distribution, you'd switch to Release configuration with proper signing.

Example 5: Ollama Installation and Model Pull

# macOS - install Ollama via Homebrew
brew install ollama

# Start the server (required for Anubis auto-detection)
ollama serve

# Pull a model for benchmarking
ollama pull llama3.2:3b

# Verify the API is accessible
curl http://localhost:11434/api/tags

Why this matters: Anubis auto-detects Ollama by attempting connection to localhost:11434. The ollama serve command must be running before Anubis launches for automatic discovery. The api/tags endpoint confirms both server availability and lists pulled models. Anubis can even OLLAMA PULL models directly from its Vault interface, eliminating terminal context-switching.

Advanced Usage & Best Practices

Optimize Your Benchmarking Workflow

Use Sequential mode in Arena when memory-constrained—parallel mode loads both models simultaneously, which can trigger swap thrashing on 8GB systems. For thermal characterization, run 3+ consecutive benchmarks with identical prompts; the third run reveals steady-state thermal behavior after initial transient heating.

Leverage Stress Testing for Baseline Characterization

Before benchmarking LLMs, run the full stress test suite to establish your Mac's theoretical limits. Note the GPU frequency under sustained load—if your LLM benchmark shows GPU frequency 15% below stress test levels, you're not GPU-bound; investigate memory bandwidth or CPU bottlenecks instead.

Export Strategy for Team Collaboration

Use Markdown export for human-readable reports with hardware banners and fastest/most-efficient summaries. Use CSV export for spreadsheet analysis and regression tracking across Anubis versions. The PNG export includes 2x retina rendering with watermark—ideal for presentations and social sharing.

Leaderboard Optimization

For valid leaderboard submissions, ensure stable thermal conditions (avoid submitting immediately after heavy compilation or video export). The community dataset uses HMAC-signed submissions with rate limiting—quality data over quantity.

Backend Selection Heuristics

• Ollama: Best compatibility, broad model support, easiest setup
• mlx-lm: Highest GPU utilization for MLX-native models, but higher memory footprint
• Apple Intelligence: Zero configuration for supported hardware, but limited to Apple's Foundation Model
• vLLM: Best for concurrent request scenarios, more complex setup

Comparison with Alternatives

Why Anubis wins: No alternative combines real-time Apple Silicon telemetry with LLM-specific inference metrics in a native, open-source macOS application. CLI tools expose hardware without LLM context; chat wrappers expose LLMs without hardware context. Anubis is the intersection.

FAQ: Common Developer Concerns

Q: Does Anubis work on Intel Macs? No. Anubis requires Apple Silicon (M1/M2/M3/M4/M5 or newer) because it relies on IOReport APIs and performance counters specific to Apple's unified memory architecture. Intel Macs lack the integrated GPU/ANE/DRAM power telemetry that makes Anubis's correlation meaningful.

Q: Can I benchmark cloud APIs like OpenAI or Anthropic? Anubis is designed for local inference. While you can add any OpenAI-compatible server, benchmarking cloud APIs introduces network latency that obscures hardware performance. The tool's value is correlating local hardware telemetry with local inference speed.

Q: How accurate is the power consumption data? Anubis reads from IOReport Energy Model, the same source Apple's own tools use. GPU/CPU/ANE/DRAM power is reported in watts with granularity matching the chip generation. Note that total system power (display, SSD, etc.) is not captured—only compute subsystem power.

Q: Is my data private when submitting to the leaderboard? Yes. No account required, no response text ever uploaded. Only performance metrics, hardware info, and a display name are submitted. HMAC signing prevents tampering, and rate limiting prevents spam. You can verify exactly what's sent before confirming.

Q: Can I use Anubis without building from source? A sandboxed version with reduced features is available on the Mac App Store as part of The Architects Toolkit. For full features including stress testing and leaderboard submission, build from source or download releases from GitHub.

Q: How do I add a custom backend not in the presets? Go to Settings > Add OpenAI-Compatible Server, enter any name, URL, and optional API key. Anubis will probe the /v1/models and /v1/chat/completions endpoints. The smart URL handler automatically corrects common path errors.

Q: What causes "No GPU metrics" and how do I fix it? GPU metrics require IOKit access, which may be unavailable in VMs, with certain security configurations, or on macOS versions below 15.0. Anubis degrades gracefully— you'll still see inference-derived metrics (tokens/sec, TTFT). Check System Settings > Privacy & Security for any blocked extensions.

Conclusion: Stop Flying Blind, Start Engineering with Data

The local LLM revolution on Apple Silicon is here, but most participants are driving without a dashboard. They feel performance differences without understanding causes. They make optimization decisions based on forum anecdotes rather than measured data. They deploy to production with no quantitative characterization of thermal limits, memory pressure, or power consumption.

Anubis ends that era.

This isn't about vanity benchmarks or leaderboard bragging rights—though the community aspect is genuinely useful. It's about engineering discipline: the ability to make informed tradeoffs between quantization, model size, backend selection, and hardware configuration based on correlated data that no other tool provides.

Whether you're optimizing a production deployment, justifying hardware upgrades, evaluating reasoning models, or simply curious why your MacBook Pro sometimes feels sluggish with local AI, Anubis transforms mystery into measurement. The native SwiftUI interface, real-time telemetry correlation, reasoning-aware metrics, and open-source extensibility make it the definitive tool for serious local LLM work on macOS.

Your next step is simple: clone the repository, build it in Xcode, and run your first benchmark. Within five minutes, you'll see your Mac's true LLM performance profile—and you'll never want to go back to guessing again.

The gap between "it works" and "it's optimized" just got a lot smaller. Cross it.

Star the repo, submit to the leaderboard, and join the growing community of developers who refuse to tolerate mystery meat performance. Anubis is waiting.