Rnote API Rnote API
Back to Blog

RedNote API Best Practices: Rate Limits, Retries, Pagination & Dedup

Rnote API Team · · 25 views · 中文
Xiaohongshu Data Best Practices Rate Limits

Taking RedNote (Xiaohongshu) data collection from "it runs" to "stable at scale" comes down to a few engineering details: rate limiting, retries, pagination, deduplication, and cost control. Here's a copy-paste-ready set of best practices for use with the Rnote API.

1. Rate limiting & concurrency

Cap concurrency and throttle your requests instead of maxing out immediately. On 429 (rate limited), back off and retry — a smooth request curve is more stable and less stressful than bursty floods.

2. Retries & backoff

Retry only on network errors and 5xx, using exponential backoff (1s, 2s, 4s…). The good news: the Rnote API bills only successful (HTTP 2xx) requests, so retries on failures cost nothing — add retry logic with confidence.

3. Pagination: cursors differ by endpoint

Endpoint Pagination
search/notes increment page
user/posted use the cursor from the previous page
note/comments cursor object cursor + index
topic/feed cursor_score + last_note_id + last_note_ct

Always carry the cursor fields from the previous response into the next request to keep pagination consistent.

4. Deduplication

Across pages, sort orders, and jobs you'll see duplicates. Deduplicate by note_id / user_id (a set or a unique DB index) — it saves credits and keeps your data clean.

5. Cost & budget control

  • Only successful requests are billed — failures are free.
  • Validate your logic and fields on a small batch first, then scale.
  • Set a budget cap per job so a misbehaving script can't run away.

A robust paginated collection skeleton (Python)

import time, requests

API = "https://rnote.dev/api/v2/crawler"
H = {"X-API-Key": "YOUR_API_KEY"}

def get(path, params, retries=3):
    for i in range(retries):
        r = requests.get(f"{API}/{path}", params=params, headers=H, timeout=30)
        if r.status_code == 200:
            return r.json()
        if r.status_code == 429:
            time.sleep(2 ** i)          # rate-limit backoff
        elif r.status_code >= 500:
            time.sleep(2 ** i)          # server-error backoff
        else:
            r.raise_for_status()        # don't retry 4xx
    raise RuntimeError("retries exhausted")

def search_all(keyword, max_pages=10):
    seen = set()
    for page in range(1, max_pages + 1):
        data = get("search/notes", {"keyword": keyword, "page": page})
        items = extract_items(data)     # pull the list per response shape
        if not items:
            break
        for it in items:
            nid = it["note_id"]
            if nid not in seen:         # dedup
                seen.add(nid)
                yield it

Get started

Drop these practices into your collection jobs and they'll run reliably at volume with little change. Sign up free for an API key, get going with the Python / Go tutorials, and see the API docs and pricing.

Back to Blog