This page explains how the benchmark works: how we choose what to test, how we decide which answer is right, and how the score is built.
Last updated: September 5, 2026
A continuous, multi-region benchmark of public Solana RPC providers: how the queries, scoring, and consensus checks work, and how to read the numbers.
This page explains how the benchmark works: how we choose what to test, how we decide which answer is right, and how the score is built.
0.25·Latency + 0.25·Win-rate + 0.25·Reliability + 0.20·Correctness + 0.05·Freshness. The Overview blends
that across the preset's regions and methods into one score; pick a preset
(Balanced / Trading / Apps) and tune the component and per-method weights
on the Overview./raw?challenge=<id> for any test to see the
pinned inputs, the revealed seed, every provider's response, and the verdict.Each test (we call it a "challenge") moves through these steps:
We never trust one provider to tell us the "right" answer. Every provider gets the same request, and we go with the answer most of them agree on. Match the majority and you're marked correct; be the odd one out and you're marked wrong.
We only score a test when the outcome is clear. Cold and warm requests are judged separately:
| What happened | Result |
|---|---|
| Fewer providers returned a usable answer than the method's panel needs (3) | Skipped — not enough to compare |
| No clear majority (e.g. a 2–2 tie) | Skipped — too close to call |
| A clear majority agrees | That group is correct; anyone who disagrees is wrong |
Who votes. The panel is Helius, Triton, Alchemy, Quicknode, and Chainstack, five providers on most methods. If a provider's plan doesn't offer a method, it isn't counted for or against on that method.
Some methods have fewer voters, because a panel provider either doesn't offer the method or returns it in a format we can't compare against the others:
simulateBundle — three voters (Helius, Triton, Alchemy). Neither Quicknode
nor Chainstack serve it: it's a Jito bundle-simulation extension, and neither
runs Jito-enabled infra on this tier.getTransactionsForAddress — three voters (Helius, Alchemy, Quicknode).
It's a custom indexer-backed method, not standard Solana JSON-RPC:
Chainstack (a standard core RPC node) doesn't serve it at all, and Triton —
which did serve it compatibly — dropped it in August 2026 and now returns
"Method not found" for it while every other method on the same endpoint stays
healthy. Quicknode ran the opposite course: its variant was non-comparable
(a different response envelope, and it ignored both the slot pin and the
detail-level option), and it became byte-comparable with the rest of the
panel in August 2026.getStakeMinimumDelegation — four voters (Helius, Triton, Quicknode,
Chainstack). Alchemy returns "unsupported method"; it's a standard method
everyone else on the panel serves.getTokenLargestAccounts — four voters (Helius, Triton, Alchemy,
Quicknode). Chainstack's shared tier restricts it to dedicated nodes only.On the two three-voter methods (simulateBundle and
getTransactionsForAddress), two providers agreeing is enough to settle the
answer (the third is then the odd one out and scored wrong). We still need all
three to answer for the test to count. There's no external tie-breaker, so two
independent providers agreeing is the entire correctness signal there. The two
four-voter methods need the usual strict majority (3 of 4) instead.
Each provider gets five sub-scores (0–100), combined into one number:
100 × avg(fastest p50 / your p50, fastest p95 / your p95)Blends “usually fast” (p50 median) with “tight tail” (p95), each scored against the panel’s best.
100 × your win rate / best win rateShare of challenges where this provider was the single fastest correct sample, normalized to the panel’s best winner.
Before comparing answers, we boil each response down to just the parts that should match (ignoring incidental fields like timestamps or list ordering), then compare them with the right rule for that method: an exact match, a similarity threshold, a slot tolerance, or a well-formedness check.
Immutable block (read at confirmed)
getTransactionsForAddress is a non-standard method — an indexer-backed
address-history API served comparably by Helius, Alchemy and Quicknode
(Chainstack doesn't serve it and Triton dropped it in August 2026; see the
consensus section). Two things differ from its standard sibling
getSignaturesForAddress:
slot ≤ tip − 5000
(~35 minutes back, deeply finalized), newest-first. That makes the answer
immutable — the tip drift that forces getSignaturesForAddress into fuzzy
matching is excluded — so both buckets match byte-for-byte, as consensus
requires. Trade-off: live-tip behavior isn't measured.signatures (limit 1000) hashes
{ signature, slot, err } per entry; full (limit 25) hashes the same
canonical slice as getTransaction
({ signature, slot, err, fee, preBalances, postBalances }). Both drop
provider-internal fields (paginationToken, blockTime, memo, …), and
params stay in the cross-provider common subset (no Helius-only filters, no
processed commitment, limit under Alchemy's cap).Test addresses are signers pulled from a block just below the pin (guaranteeing a transaction in range), restricted to non-high-activity addresses — programs and vote authorities index differently across providers, so excluding them is what makes byte-equal agreement possible.
Most tests draw from live, recent chain state (the last seconds to hours). The
methods whose answers are immutable history — getBlock, getTransaction,
getBlockTime, getBlocks, getBlocksWithLimit, getBlockCommitment, and
getSignaturesForAddress — also carry an archival bucket that samples a
uniform-random slot 182–365 epochs back (≈1–2 years). That depth sits well
past every provider's recent-ledger retention and warm storage, so archival
buckets measure real archive reads (cold deep-history lookups), not caches.
Every archival input is freshly drawn per test — nothing is reused, so
providers can't pre-warm the answers.
Two archival-specific rules:
getSignaturesForAddress
archival bucket pins its query strictly before a 1–2-year-old anchor
signature, making the expected result immutable. Consensus there is strict
byte-equality — any divergence is a real archive gap, so the similarity
tolerance used for tip-anchored windows doesn't apply.Methods whose deep history isn't reliably served by validators (leader
schedules: getSlotLeaders, getLeaderSchedule) have no archival bucket, and
account-state methods can't have one: Solana RPC has no point-in-time account
reads.
Freshness is measured on a separate connection so it never adds time to the latency we report.
Freshness is measured against a contemporaneous reference — the median of the providers' own tip slots observed during the same round — not the tip captured when the test was created (which is a few seconds stale by the time providers answer, so every provider looks "ahead" of it). This keeps the Freshness sub-score discriminating: a provider genuinely behind the live panel shows a positive lag; the fresh cluster sits near zero.
Every test has a public page at /raw?challenge=<id>: the inputs, commitment
hash, revealed seed, each provider's response, and the consensus result. To
confirm the inputs were fixed before anyone answered, recompute
commitment = SHA-256(seed ‖ canonical-JSON(params)) yourself (canonical-JSON
sorts object keys recursively — the same canonicalization used for projection
hashing).
For honeypots, the pre-seeded known answer's full response is shown for the first 6 hours, then trimmed to save storage; its hash is kept permanently, so the verdict stays verifiable indefinitely.
A provider that can recognize benchmark traffic could serve it from a fast path and look better than it is. The defenses, all verifiable in this repo:
Every provider is measured the same way: same challenges, same timeouts, same scoring. The one difference is cost. The operator runs its own provider for free, but anyone reproducing the benchmark needs a paid tier on every provider. Free tiers can't keep up with the benchmark's volume (the Helius free tier's ~1M credits/month, for instance, is far below it), so running on free tiers won't give the same or accurate results. This only changes what it costs to reproduce the benchmark, not the measurements. Anyone can run the same code against their own keys and recompute every score.
The sends board (/sends) is a parallel benchmark to the read board. Instead of
scoring RPC responses by cross-provider consensus, it broadcasts real
transactions through competing send paths and scores them against the chain.
It is versioned independently (SEND_METHODOLOGY_VERSION); read scoring is
untouched.
The 5 benchmarked read RPC providers — Helius, Alchemy, Triton,
Quicknode, Chainstack (send-target id === provider id, each once). We send
plain JSON-RPC sendTransaction to every provider's standard endpoint —
the identical call for all five, no tips, no relays, no premium/staked/Jito send
paths. The only per-tick lever is the shared adaptive priority fee (below),
applied uniformly, so it's a clean apples-to-apples landing comparison of each
provider's ordinary sendTransaction. (Providers may auto-route the standard call
through their own staked/SWQoS connections server-side; that's part of what's being
measured, and it requires nothing from us.)
Ground truth is the chain (a confirmed transaction status), not consensus or
honeypots. The generator's confirm poll queries getSignatureStatuses for in-flight sends —
no Yellowstone/gRPC stream. Every send is scored independently — no panel
majority vote. Four outcomes:
err ≠ null, e.g. swap
slippage). A successful landing, not a send-path miss — counted in the
landing-rate numerator. Because the payload is identical across targets in a
tick, reverts wash out across targets (published as a diagnostic).slot_latency = slot_landed − slot_sent — the primary metric, both ends on the
confirm poll's single slot clock (the RPC it queries). slot_sent is the
confirm head slot at the poll cycle when it the pending send
(not the wire send), so it lags true wire-send by a small, uniform-across-targets
amount; robust to per-host clock skew, but not "clock-independent."getTransactionsForAddress's panel changed twice in August 2026, and we'd
rather say so than quietly restate the current rule. Triton dropped the
method, leaving two voters — correctness there was a pairwise agreement
check, not a majority vote: both providers had to answer and match
byte-for-byte, a disagreement was thrown out (nothing can break a 1–1 tie), and
two providers agreeing on the same wrong answer was indistinguishable from both
being right. Quicknode's variant then became byte-comparable with the panel,
restoring a three voter panel with the ordinary 2-of-3 rule and the same caveat
as simulateBundle, no more. Both changes landed under methodology version 4,
so this method's correctness series changes shape twice within one version:
read a step there as a rule change, not a provider regression. Its latency,
reliability and freshness numbers are ordinary panel-wide measurements
throughout and carry no such asterisk.
These voter counts describe the full published panel and are fixed
regardless of which subset a given deployment actually configures. A
reproducer who runs the full generator/worker/DB stack (this repo's "Option
B") but deliberately configures fewer than all five provider env vars — fully
supported, see the README — still gets these fixed thresholds, not ones
recomputed from their smaller subset. Concretely: a reproducer running only
Helius+Triton+Quicknode for getStakeMinimumDelegation (omitting Chainstack)
only ever gets 3 actual votes, but the thresholds are still derived from the
full 4-voter structural panel, so the relaxed 2-of-3 rule never kicks in —
those 3 votes must agree unanimously, and a 2-1 split among them is rejected
as no_consensus rather than being decided in the majority's favor. Under the
pre-Chainstack four-provider panel that same subset (then the entire panel
for this method) would have gotten the two-of-three relaxation instead. This
is a known, accepted trade-off, not a bug — the alternative (deriving
thresholds from each run's actual configuration) would make the CLI's own
quickstart reporting unreliable, since CLI-mode providers never populate
these env vars in the first place. (Honeypots — spot-check tests where we already know the
correct answer, described under Anti-gaming below — don't cover these
methods;
they apply only to getBlock, getTransaction, and getSignaturesForAddress.)
100 × responses that succeeded / non-ambiguous samplesShare of non-ambiguous samples that responded. An HTTP 200 with incorrect data still counts as reliable, but not correct.
100 × correct / (correct + incorrect + stale)Share of validated samples that were correct. Timeouts hit R, not C, so a sample is never penalized twice.
100 × lowest tip-lag / your tip-lagTip-lag at p95 versus the panel’s freshest. A tiebreaker, not a primary axis.
Latency (L) and win rate (W) are kept separate: L rewards a tight, consistently-fast distribution, while W rewards being first on head-to-head requests. They can disagree. Timeouts hurt Reliability, not Correctness, so a flaky provider isn't punished twice for the same call.
By the same principle, two situations are no-fault — excluded from both Correctness and Reliability, never counted against a provider:
getBalance), if a
provider's answer differs from the majority but it read a strictly newer slot
(its response's own context.slot is ahead of the freshest agreeing provider),
the value legitimately moved — there's no ground truth at that newer slot, so the
sample is excluded rather than scored wrong. A provider reading an older slot is
still stale; a divergence at the same slot is still incorrect.That five-part formula produces a score per (method, region). The Overview headline then blends those up two more levels: across regions, then across the methods in the active preset.
The "Overall" view blends regions with these default weights:
| Region | Weight |
|---|---|
| NA East | 0.35 |
| EU Central | 0.35 |
| AP Northeast | 0.15 |
| NA West | 0.05 |
| EU West | 0.05 |
| AP Southeast | 0.05 |
For each provider the weights are re-normalized over only the regions where it qualifies, so a provider that isn't in a region isn't penalized for it. A preset may also use a subset of regions (e.g. Trading scores only NA-East, EU-Central, and AP-Northeast); the same re-normalization applies over that subset. These region weights (and the same re-normalization) drive the displayed win rate too, not just the score — so a provider that wins heavily in a low-weight region (say NA-West) no longer shows a headline win% out of line with where it ranks.
The scoring W above is global: for each challenge, the single fastest-correct
provider across the whole panel takes the win, so it splits wins across every
provider at once. It can't tell you whether one provider beat another
specifically.
The head-to-head win rate answers exactly that, for a chosen pair. Over the challenges both providers answered correctly (the contested set), the one with the lower latency wins that challenge — ties broken by the earlier request start, the same ordering as the global winner. The pair's win rate is its share of contested challenges. A challenge only counts when both answered correctly, so a method one provider doesn't support yields zero contested challenges (and a null rate) rather than a skewed one.
Like every "Overall" figure on the site, the overall head-to-head rate is a
fixed-weight region blend of the per-geo rates (the Region weights above),
not a traffic-weighted average — so a pairing isn't dominated by whichever geo
happens to run the most samples. Selecting a specific region shows that geo's rate
directly. This is a descriptive comparator exposed at GET /api/head-to-head; it
is not fed into the composite score.
The Overview ranks by a workload preset — a set of methods + per-method weights
| Preset | Focus | Methods | Regions |
|---|---|---|---|
| Balanced | Even, everything | all scored methods, equal weight | all 6 |
| Trading | Latency / win-rate | getLatestBlockhash, getSlot, getAccountInfo, getProgramAccounts | NA-East, EU-Central, AP-Northeast |
| Apps | Reliability / correctness | getTransaction, getSignaturesForAddress, getProgramAccounts, getTokenAccountsByOwner, getAccountInfo, getMultipleAccounts | all 6 |
The leaderboard score blends across methods, but latency percentiles (p50/p95) can't be pooled the same way — a true percentile needs the underlying distribution, which the rollups don't keep. So when a preset spans several methods, the Performance latency chart shows a sample-count-weighted average of each method's p50/p95 (weighted by call volume). It's a useful "how fast for this workload" summary but an approximation, not a real percentile — the chart labels it as such, and exact per-method percentiles live in each provider's per-method drill-down. The displayed win rate is blended the same way the score's win-rate component is — region-weighted, then method-weighted, re-normalized over the regions/methods where the provider qualifies — so the win% you read moves with the ranking instead of being a region-blind pooled average. (This is a display-aggregation choice, not a scoring-formula change: the score itself is unchanged. The blended win rate is not guaranteed identical to the win-rate sub-score, which additionally normalizes each region to its fastest winner before blending.) The remaining totals — calls, success, failure breakdown — are simple pooled sums across the blended cells.
sendTransaction is absent from every preset: it's a broadcast with no
replayable correct answer to validate against, so it isn't scored.
Because a preset blends several methods, a provider that qualifies on only one or two of them could otherwise top the board on a sliver of the workload. So a provider is ranked only if it qualifies in methods worth ≥60% of the preset's total method weight; below that it's shown as "insufficient method coverage" rather than ranked. The 60% bar is comfortably clear of every current provider (each qualifies on ~93%+ of the method universe) — it's a guard for sparse windows and future entrants.
To appear ranked, a provider needs enough data to be meaningful: currently a 4h window, at least 50 samples per (provider × method × region), ≥80% reliability, ≥80% correctness, and a ≥95% honeypot pass rate (Wilson lower bound). Below that, it shows with a "below thresholds" note. Under a preset, that gate is applied per (method, region) before the blend, and the coverage gate above is applied to the blended result.
wall_latency_ms — confirm-observed landing time minus the worker's sent_at;
depends on NTP-synced fleet clocks.submit_latency_ms = HTTP-ack − submit — recorded separately.slot_latency is slot-granular (~400ms), a coarse regional signal; the fine
per-region signal is in submit_latency_ms / wall_latency_ms.
Block position is not measured: it can only be derived from a full-block stream (Yellowstone), and status polling doesn't expose it. Dropped rather than approximated.
Two axes, best-normalized like the read scorer, with a max(1, …) zero-guard
(slot_latency=0, a same-slot land, is routine):
landing_rate = (landed + reverted) / (landed + reverted + not_landed + submit_error).slot_latency.total = 0.55·R_send + 0.45·L_send. Reliability-dominant because landing is the
point.
Every target in a tick gets the identical call: same sendTransaction, same
recent-blockhash value, same payload + CU limit, and the same priority fee —
no tips anywhere. They're fired together via a barrier; each signs from its
own per-target wallet, so all sends are independent and can all land (a true
landing rate) — not a shared-nonce race. One challenge fans to several
vantages, so each transaction folds a small per-vantage nonce into its
compute-unit limit (the tx-landing-canary technique): the per-target wallet
makes signatures distinct across targets, and the CU nonce makes them distinct
across vantages, giving every send a unique signature and clean attribution on
the shared blockhash — at zero extra compute (no memo). The priority fee is
adaptive per tick, tuned by a controller toward a ~50–70% aggregate landing
band (where providers separate), so R_send ranks near the contention margin —
not an absolute production landing rate. Cross-time comparisons should account for
the in-effect fee (stamped on every row), which moves with network conditions.
Because the call and fee are identical for all five, no provider gets special
treatment — differences are purely in how each provider's own infrastructure
handles the standard sendTransaction.
A send exposes its funded fee-payer pubkey on-chain the instant it broadcasts; there is no commit-reveal equivalent. Wallet rotation raises the cost of fingerprinting but does not close the gap. Sends are therefore a disclosed-wallet benchmark, a weaker anti-gaming guarantee than reads — stated outright rather than implied away.
Published: scenario definitions, fixed CU limits (transfer 1,000 / raydium_swap
80,000 / orca_swap 150,000, plus a per-vantage nonce ≤4,095 folded into the
limit for signature uniqueness), monitored pool addresses, and
the fact that every target is hit with the identical plain sendTransaction (no
tips). Anyone can recompute every score.
Swap trade size. Forward swaps trade a fixed 0.0002 SOL / equivalent
(swapAmountLamports). Reverse swaps trade balance ÷ (2 × K) of the
accumulated counter-token (K = the vantage fan-out VANTAGE_SAMPLE_SIZE = 3, so
÷6) — not the full balance: several vantages share one per-target wallet, so a
full-balance dump lets the first vantage drain it and the rest revert
(ZeroTradableAmount). The fractional slice keeps concurrent reverses independent
and bounds inventory to the ~3×–6× range of the per-swap forward output: with
directions alternating 1:1 it oscillates between the post-reverse trough
X·(D − K) ≈ 3× and the post-forward peak X·D ≈ 6× (D = 6, K = 3). A
reverse whose slice rounds to 0 (drained/empty balance) is
skipped rather than sent, so a target's forward/reverse sample mix can skew
slightly toward forward during warmup or if its forwards are failing —
negligible in steady state. Because reverses leave that counter-token as WSOL that
never returns to native, a generator-side harvest job periodically closes each
wallet's WSOL ATA (unwrapping it back to native in place) to close the inventory
loop and keep the master's funding outflow from growing unbounded (see
docs/operations.md § Harvest / SOL recovery).
Cost. The board's cost/tx = base_fee (5,000) + priority_fee (µlamports/CU) × CU_limit ÷ 1e6 lamports — Solana charges the priority fee on the compute-unit
limit (cu_requested), not units consumed. Computed per (scenario, region)
then averaged; no tips on the scored path, so this is the full per-tx cost.