This post covers three things I get asked about when people compare Reddit data tools for research: whether collapsed branches really get expanded, what fields come back for each comment, and what 1,000 comments cost. I work on ProLevelMax, the data source used in the examples, so the numbers below are ours. The checklist at the end applies to any tool.

Why "nested comments" is not the same as "all comments"

Reddit does not send a whole comment tree in one response. Past a certain depth or count it returns a stub that says, in effect, "there are more replies here, ask again with this cursor".

A scraper can handle that stub in two ways:

1. Return it as a placeholder record with an ID and move on. The output looks nested and complete, but the deepest replies are missing.
2. Follow the stub, fetch the branch, and repeat for any stubs inside it.

Both can be described as "supports nested comments". Only the second gets you the full thread. When you test a tool, count how many records in its output are placeholders.

How the expansion works here

There are two per-call sources for a single thread:

A collapsed branch comes back with a more.cursor value. Send that to procli_reddit_h and you get the replies under it. If those replies contain stubs of their own, do the same again. An agent connected over MCP does this when you ask for "all comments, including collapsed replies". In your own code you loop until no cursor is left.

For bigger jobs across many posts there is also a bulk source (procli_reddit_a) that runs searches, subreddits or URL lists in one job, with a per-post comment cap.

What comes back for each comment

From a live procli_reddit_b result:

• id (for example t1_...), so you can de-duplicate and join
• createdAt (ISO timestamp)
• authorInfo (username and author details)
• content.markdown (the text)
• score and isScoreHidden
• permalink
• depth (nesting level)
• isRemoved, isAdminTakedown, isLocked, isStickied, isInitiallyCollapsed, initiallyCollapsedReason
• more.cursor on collapsed branches
• page.commentCount and page.hasNextPage on each page

Deleted and removed comments come back the way Reddit shows them publicly, usually without text, and flagged so you can filter them out before counting.

The cost of 1,000 comments

Per-call route: every call is $0.003. Across our recent calls the Post comments source returned a median of 10 comments per call, so:

1,000 comments ≈ 100 calls × $0.003 = $0.30

Each collapsed branch you expand is one more $0.003 call, so deep threads cost somewhat more than that.

Bulk route: $0.0108 per result plus $0.06 per run.

1,000 results in one run = 1,000 × $0.0108 + $0.06 = $10.86

The bulk route costs more per comment but covers many posts in one job. For reading a limited set of threads in full, the per-call route is cheaper. Every run is quoted before it starts and the charge never goes above the quote. Current prices are on the pricing page.

A script that walks the whole thread

This pages through a post's comments, then expands every collapsed branch it finds, and stops if anything would need a confirmation (quotes above $0.10). It does not assume the exact nesting of the response: it walks the JSON and picks up anything that looks like a comment (id starting with t1_) or a stub (more.cursor).

import json, sys, time, requests

API = "https://prolevelmax.com/v1"
H = {"Authorization": "Bearer sk_procli_..."}
POST = "t3_1abcde"                      # the post you want

def run(scraper, inp):
    q = requests.post(f"{API}/inspect", headers=H, json={"scraper": scraper, "input": inp}).json()
    if q["quote"]["needsConfirmation"]:
        sys.exit(f"Quote is {q['quote']['maxChargeDisplay']}, stopping for review")
    time.sleep(1.2)
    return requests.post(f"{API}/run", headers=H, json={"scraper": scraper, "input": inp}).json()

def walk(node, comments, cursors):
    if isinstance(node, dict):
        if str(node.get("id", "")).startswith("t1_"):
            comments[node["id"]] = node
        more = node.get("more")
        if isinstance(more, dict) and more.get("cursor"):
            cursors.append(more["cursor"])
        for v in node.values():
            walk(v, comments, cursors)
    elif isinstance(node, list):
        for v in node:
            walk(v, comments, cursors)

comments, cursors, calls = {}, [], 0

# 1. page through the top of the thread
inp = {"post_id": POST, "sort_type": "TOP"}
while calls < 50:
    r = run("procli_reddit_b", inp); calls += 1
    walk(r["items"], comments, cursors)
    page = r.get("page") or {}
    if not page.get("hasNextPage") or not page.get("after"):
        break
    inp["after"] = page["after"]

# 2. expand every collapsed branch, including branches inside branches
seen = set()
while cursors and calls < 200:
    c = cursors.pop()
    if c in seen:
        continue
    seen.add(c)
    r = run("procli_reddit_h", {"post_id": POST, "cursor": c}); calls += 1
    walk(r["items"], comments, cursors)

json.dump(list(comments.values()), open("thread.json", "w"), ensure_ascii=False, indent=2)
print(len(comments), "comments in", calls, "calls; balance", r["wallet"]["balanceDisplay"])

The two calls limits are a budget guard: 200 calls is $0.60 at most. Raise them once you have seen what a typical thread in your niche needs.

A test you can run on any Reddit tool

Before committing to one tool for a research project, pick three threads you can check by hand: one small, one with a few hundred comments, and one with long collapsed arguments. Then for each tool:

1. Count the comments with text in its output and compare with what you can read on Reddit. Reddit's displayed count includes deleted and removed comments, so expect fewer.
2. Count placeholder records (an ID with no text and a "more" marker). If there are many, the tool is not expanding branches.
3. Check that every comment has an ID, author, timestamp, score and permalink.
4. Divide what you were charged by the number of comments with text. That is your real cost per comment, which is the number worth comparing.

Limits

• Public data only. Private and quarantined communities cannot be read.
• Scores are a snapshot at request time.
• Up to 5 jobs run at once per account. Results are kept for 30 days.
• Using Reddit data commercially is subject to Reddit's terms. Collecting it does not by itself give you the right to use it.

Setup

In Claude, Claude Code, Codex or Cursor: create an API key in the console ("Generate setup message" on the Get started page) and send it to your AI. In ChatGPT: add https://prolevelmax.com/mcp as an app and sign in with OAuth. New accounts get $1 of credit after email verification, which covers the script above several times over.

The full reference for this, including the Chinese version, is in the guide How to get full Reddit comment threads, including collapsed replies, into your AI. Questions or corrections: toph@prolevelmax.com.

I work on ProLevelMax, the data source used in the examples below. The approach (discover a source, check the price, run it, page through results) is the part worth taking away.

Why scraping Reddit yourself gets old fast

Getting one thread is easy. Getting a few hundred threads reliably is where the time goes:

• Comment trees are nested and truncated. Long threads come back with collapsed branches and "more replies" stubs. To get the whole conversation you have to follow each stub, keep parent links straight, and stitch the tree back together.

• Pagination is per thread. Every post has its own cursor, so a research job across 30 threads is 30 separate paging loops, each of which can fail halfway.

• Access needs upkeep. You register an app, manage credentials, watch rate limits and handle the day the response format or the HTML changes.

• Agents make it worse. If you hand a raw HTTP tool to an LLM and say "read these threads", it will happily fetch page one and summarise a fraction of the discussion as if it were all of it.

What I wanted instead: the agent asks for "comments on this post", gets a price before anything runs, and receives clean records it can count and quote.

Two ways in: MCP or REST

ProLevelMax exposes the same data three ways:

• A remote MCP server at https://prolevelmax.com/mcp. Claude.ai, Claude Desktop and ChatGPT connect with OAuth (no key). Claude Code, Codex and Cursor can use an API key as a Bearer token.

• An Agent Skill at https://prolevelmax.com/SKILL.md. You create an API key in the dashboard, then send your AI the line set up https://prolevelmax.com/SKILL.md together with the key, and it configures itself.

• A REST API under https://prolevelmax.com/v1, described in an OpenAPI 3.1 file.

For Claude Code, the setup line from SKILL.md is:

claude mcp add --scope user --transport http prolevelmax https://prolevelmax.com/mcp --header "Authorization: Bearer sk_procli_..."

Once connected, the agent has eight tools. The four that matter for research are procli_discover, procli_inspect, procli_run and procli_runs_get. The REST endpoints mirror them, so I will use REST for the rest of this post because it is easier to show.

The request flow

1. Discover. You describe what you want in plain words (English or Chinese) and get candidate sources back.

curl https://prolevelmax.com/v1/discover \
  -H "Authorization: Bearer sk_procli_..." \
  -H "Content-Type: application/json" \
  -d '{"query":"reddit comments"}'

The response has recommended and items. Each source carries a scraper id, a title, bestFor, a verdict (recommended, ok or not_recommended), a why, its pricing and an estimate. For Reddit there are separate sources for search, subreddit feeds, a post with its comments, post comments page by page, and replies under one comment for deep threads.

2. Inspect. Before running anything, ask for the input fields and a quote.

curl https://prolevelmax.com/v1/inspect \
  -H "Authorization: Bearer sk_procli_..." \
  -H "Content-Type: application/json" \
  -d '{"scraper":"procli_reddit_b","input":{"post_id":"t3_1abcde"}}'

The response shape (values omitted):

{
  "scraper": "procli_reddit_b",
  "inputFields": [
    { "name": <string>, "type": <string>, "required": <bool>, "description": <string>, "paging": <bool> }
  ],
  "exampleInput": { ... },
  "pagingNote": <string>,
  "quote": {
    "maxCharge": <number, USD>,
    "maxChargeDisplay": <string>,
    "needsConfirmation": <bool>,
    "balanceTooLow": <bool>
  }
}

quote.maxCharge is the most this run can cost. needsConfirmation is true when the quote is above $0.10. Fields marked paging: true are the ones you fill from the previous result to get the next page.

3. Run.

curl https://prolevelmax.com/v1/run \
  -H "Authorization: Bearer sk_procli_..." \
  -H "Content-Type: application/json" \
  -d '{"scraper":"procli_reddit_b","input":{"post_id":"t3_1abcde"}}'

A finished run returns runId, items, count, a page object with cursor values for the next page, cost (with amountDisplay), quote, wallet (with balanceDisplay) and download links for json, csv and xlsx. For long jobs, send "async": true and you get a 202 with a runId to poll at GET /v1/runs/:id.

4. Download. GET /v1/runs/RUN_ID/download?format=csv (or xlsx, json). Nested fields become a.b columns in CSV.

A small working script

This pulls every page of comments for one Reddit post, refuses to run anything that needs confirmation, and writes the records to a JSON file. Replace the key and the post id.

import json, sys, requests

API = "https://prolevelmax.com/v1"
H = {"Authorization": "Bearer sk_procli_..."}
SOURCE = "procli_reddit_b"           # "Post comments", page by page
inp = {"post_id": "t3_1abcde"}       # the post you want
comments, spent = [], []

for _ in range(10):                  # hard stop at 10 pages
    info = requests.post(f"{API}/inspect", headers=H, json={"scraper": SOURCE, "input": inp}).json()
    if info["quote"]["needsConfirmation"]:
        sys.exit(f"Quote is {info['quote']['maxChargeDisplay']}, stopping for review")
    run = requests.post(f"{API}/run", headers=H, json={"scraper": SOURCE, "input": inp}).json()
    comments += run["items"]
    spent.append(run["cost"]["amountDisplay"])
    paging = [f["name"] for f in info["inputFields"] if f.get("paging")]
    page = run.get("page") or {}
    if not paging or not any(page.get(n) for n in paging):
        break                        # no next cursor: done
    inp.update({n: page[n] for n in paging if page.get(n)})

json.dump(comments, open("comments.json", "w"), ensure_ascii=False, indent=2)
print(len(comments), "comments, charges:", spent, "balance:", run["wallet"]["balanceDisplay"])

The script does not hard-code the cursor field name. It reads which input fields are marked paging from inspect and copies matching values from the result's page object, which is how the docs describe paging for per-call sources. Error handling is left out to keep it short; errors come back as JSON with a stable type (for example insufficient_balance, spend_limit, invalid_input, rate_limited) you can branch on.

From here, feed comments.json to whatever model you use and ask for themes with counts and verbatim quotes. Asking for quotes with links keeps the summary honest, because you can check each claim against the thread.

Keeping costs predictable

Running an agent against a paid API is only comfortable if spend has a ceiling. The pieces that give you one:

• A quote before every run. inspect returns maxCharge, and the charge never exceeds the quote. The Skill tells agents to state the price and wait for your OK when the quote is above $0.10.

• No overdraft. Per-result runs shrink to what your balance covers, per-call runs do not start without enough balance, and the account cannot go below zero.

• Spend limits per key or app. You can set daily or monthly limits in the dashboard. A run that hits one fails with spend_limit instead of charging.

• Caps on schedules. If you monitor threads on a schedule (POST /v1/schedules), each schedule takes a runCap and an optional monthlyCap, and dryRun: true previews the cost without saving anything.

• Failed runs. These are generally not charged; a partial charge, if any, stays within the quote.

Billing is pay per use: some sources charge per call, others per result plus a per-run fee with a minimum. Most Reddit sources are per call. Current rates are on the pricing page, and new accounts get $1 of credit after email verification, which is enough to try the flow above.

Limits worth knowing

• Public data only. Private and quarantined subreddits cannot be read, and anything that needs someone's login is not collected.

• Deleted content stays deleted. Removed or deleted posts and comments show only what Reddit still shows publicly, usually nothing.

• Scores are a snapshot. Upvotes and comment counts are the public numbers at the moment of the request.

• Concurrency. Up to 5 jobs run at once per account, across all keys and connections. Extra runs get 429 too_many_runs.

• Run size and time. Per-result sources return up to 100 results unless you set maxItems. A synchronous run waits at most 300 seconds (timeout), so use async for big jobs.

• Retention. Run results are kept for 30 days. Download what you want to keep.

Where to go next

• Docs: prolevelmax.com/docs

• OpenAPI: prolevelmax.com/openapi.json

• Agent Skill: prolevelmax.com/SKILL.md

• Client configs: github.com/tophmeee/prolevelmax-mcp

If something in the flow is unclear or broken, email toph@prolevelmax.com.