HeapDumpStarDiver

This tool converts JVM heap dumps (HPROF binary format) into Apache Parquet files, enabling columnar analytical processing of heap objects. It parses every object instance, class definition, primitive array, and object reference from a heap dump and exports them as typed Parquet columns — one file per Java class.

It is also able to dump information classes on heap to your terminal, if you want a more high level/fast look at things.

This is an early stage project, and was made during a LinkedIn hackathon! Any improvements or PR's are welcome!

Dependencies

This project depends heavily on the work done by Marshall Pierce (thanks Marshall!).

We depend on a fork of the jvm-hprof crate located here: https://bitbucket.org/ZacAttack/jvm-hprof-rs-li-hackweek/src/master/

To build, you'll need to check out that repository at ../../bitbucket/jvm-hprof-rs-li-hackweek/ (relative to this project root), or update the path in Cargo.toml. (I'm thinking we'll at some point just vendor this dependency).

Build & Run

# Build
cargo build --release

# Run (requires an HPROF file)
./target/release/HeapDumpStarDiver -f <path-to-heap-dump>.hprof <command>

Commands

dump-objects-to-parquet

The primary command. Exports all heap objects to Parquet files in a parquet/ directory (created automatically). Produces one .parquet file per Java class, plus special files for primitive arrays, object arrays, static fields, and GC roots.

> ./target/release/HeapDumpStarDiver -f heap.hprof dump-objects-to-parquet

> ls parquet/
_gc_roots.parquet
_object_arrays.parquet
_primitive_arrays_byte.parquet
_primitive_arrays_int.parquet
_static_fields.parquet
java.lang.String.parquet
java.util.HashMap.parquet
java.util.HashMap.Node.parquet
...

Data is flushed to disk incrementally, so memory usage stays bounded even for very large heap dumps. You can tune the flush frequency with --flush-rows (default 500,000):

# Use less memory by flushing more often
./target/release/HeapDumpStarDiver -f heap.hprof dump-objects-to-parquet --flush-rows 100000

#### Robo Mode (--robo-mode)

Robo mode is an alternative output format optimized for speed at the expense of human readability. It's designed for LLM-assisted querying -- an LLM can easily follow bare object ID references and join across files, negating the readability loss while benefiting from the faster export.

./target/release/HeapDumpStarDiver -f heap.hprof dump-objects-to-parquet --robo-mode

How it differs from default mode:

Additional files produced in robo mode:

• _object_index_chunk{0-15}.parquet -- Maps every object ID to its type name. Schema: (obj_id: UInt64, type_name: Utf8). Use this to resolve the bare IDs in other files.
• _class_hierarchy.parquet -- Class inheritance tree. Schema: (class_obj_id: UInt64, class_name: Utf8, super_class_obj_id: UInt64?, super_class_name: Utf8?).

When to use robo mode: When you're querying the heap dump programmatically (e.g. via Python/DuckDB/an LLM) and want the fastest possible export. The chunked output is trivially queryable -- tools like DuckDB and PyArrow can glob parquet/ClassName_chunk*.parquet to read all chunks as one table.

dump-objects

Prints all heap objects to stdout in a human-readable format.

> ./target/release/HeapDumpStarDiver -f heap.hprof dump-objects

25789437384: byte[] = [0x35, 0x33, 0x39, 0x36, 0x34, ]

id 25789437408: java/lang/String
  - hashIsZero: boolean = false
  - hash: int = 0
  - coder: byte = 0
  - value = id 25789437384 (byte[])

count-records

Tallies the top-level HPROF record types.

> ./target/release/HeapDumpStarDiver -f heap.hprof count-records

Utf8: 48206
LoadClass: 2079
StackFrame: 48
HeapDumpSegment: 14
StackTrace: 7
HeapDumpEnd: 1

Configuring MCP For Agent Driven Analysis

HeapDumpStarDiver includes an MCP (Model Context Protocol) server that lets any compatible AI agent convert heap dumps, run SQL queries, and perform automated waste detection — no manual scripting required.

Install

# 1. Build the Rust binary (see Dependencies above)
cargo build --release

# 2. Install the MCP server
pip install heapdump-stardiver-mcp

Or install from source:

python3 -m venv .venv
.venv/bin/pip install -e .

Available Tools

The server also exposes reference resources (heapdump://guides/setup, heapdump://guides/sql-examples, heapdump://guides/waste-checks) that agents can read on demand for setup help, query examples, and waste check documentation.

Agent Configuration

Each agent needs a one-line config pointing at the heapdump-stardiver-mcp command. The format is the same across most agents — only the config file location differs.

Add the following to the appropriate config file:

{
  "mcpServers": {
    "heapdump-stardiver": {
      "command": "heapdump-stardiver-mcp"
    }
  }
}

If installing from source instead of pip, point at the venv Python:

{
  "mcpServers": {
    "heapdump-stardiver": {
      "command": ".venv/bin/python",
      "args": ["-m", "mcp_server.server"]
    }
  }
}

Typical Agent Workflow

1. Convert: Agent calls convert_heap_dump(hprof_path="/path/to/dump.hprof") — converts HPROF to Parquet and opens a session named after the file
2. Discover: Agent calls list_parquet_files() — sees available tables and their schemas
3. Analyze: Agent calls analyze_heap() — gets automated waste detection results
4. Query: Agent calls query_heap(sql="SELECT ...") — runs ad-hoc DuckDB queries
5. Clean up: Agent calls close_session(id) to keep files, or cleanup_session(id, confirm=True) to delete them

Multiple sessions can be open simultaneously for comparing heap dumps across services or time periods.

Agent Skills

A SKILL.md file is included to help agents discover and use the MCP tools. The canonical file lives at .claude/skills/heap-analysis/SKILL.md and is symlinked into the skill directories of other popular agents:

Windows note: Git symlinks require Developer Mode or elevated privileges on Windows. Without this, Git creates a plain text file containing the symlink target path instead of an actual symlink. If this is an issue, copy the file manually rather than relying on the symlink.