Before you buy an RTX 3090 — run your workload first

Used RTX 3090 cards can excel at local AI, but runtime builds, VRAM headroom, power limits and seller testing…

The short version

The RTX 3090 remains compelling for local AI when its 24GB memory fits the intended workload.
Two PCIe-connected cards improved controlled vLLM throughput by only 16% to 35%, far short of doubling.
Buyers should test sustained performance, software versions, power limits, cooling, and VRAM headroom before purchasing.

Two RTX 3090 cards beat one by only 16% to 35% in a controlled vLLM test. So buy a tested card and flee any mystery listing whose medical history is one FurMark screenshot and “AI READY” in all caps.

The RTX 3090 has aged like weird Italian cheese: bulky, power-hungry and more desirable after years on the shelf. But GPU benchmarks depend on the runtime build, model format, context and job. Omit those details and you get fan fiction with decimals.

Loading the model is the easy part

The RTX 3090 shines when one GPU must hold a capable quantized model with room to work. Qwen3.8-27B W4A16 is a useful stress test because it fits close enough to the limit that sloppy planning hurts. Loading proves only that the startup weights fit. The runtime still needs space for activation peaks, buffers and speculative-decoding components. Prompt processing can cause the largest activation spike, forcing me to lower GPU-memory utilization despite comfortable steady-state decoding. The KV cache takes the remaining pool as requests grow. Then Chrome steals some VRAM and everything dies mid-coding session, usually when I’m late for dinner.

Bar chart comparing current figures against their baselines: dual-3090 DFlash2 greedy decode… 474 tok/s versus 432 tok/s, single-stream prose decode on the… 85 tok/s versus 98 tok/s, single-user speculative decode at long… 32 tok/s versus 68 tok/s, steady-state aggregate decode throughput… 1100 tok/s versus 46 tok/s.

Usable KV capacity matters more than a victorious loading screen. On the same dual-card host, the DFlash2 speed configuration exposed about 259,000 logical KV tokens; the lighter MTP context tier reached roughly 532,000. DFlash2 reserved more room for the first prefill, leaving around half the context capacity. Both worked because they served different jobs.

Concurrency worsens the arithmetic. A four-card DFlash2 setup with an FP8 KV cache could hold nearly four full native-context requests; the two-card version could not fit one. The larger machine paid during generation: prose ran at about 85 tokens per second across four cards versus roughly 98 on one NVLinked pair.

Buyers add VRAM totals as if GPUs were Lego. Tensor parallelism splits model work across cards, creating aggregate room for weights and cache. But each decoding step also exchanges intermediate results. The slowest link joins every token’s execution path: NVLink helps within a connected pair, while PCIe carries traffic across pairs. A long-context server may accept that bill because fitting the request matters more than latency. One person generating prose can buy twice the hardware for a slower answer. Molto premium.

Two RTX 3090 GPUs installed in an open workstation with NVLink, PCIe connections, power cables, and cooling paths visible.

The binary can kneecap the RTX 3090

The funniest RTX 3090 benchmark made the physical GPU look almost innocent. In lijialong1313’s Ollama report, the tester kept the host and model file, swapped the cards between runs and watched the slowdown follow the binary.

The comparison covered Ollama 0.32.13 and 0.33.2. Since swapping cards did not move the problem, a defective individual GPU became an unlikely suspect.

With Qwen3-VL fully resident in VRAM, the older build generated about 129 tokens per second. The newer one managed roughly 26 under fixed settings: an 80% software-path collapse. Nobody has established the root cause or confirmed which release fixes it. Any confident kernel-level explanation today is seasoning the air.

The causal chain matters. The runtime selects operators and turns them into GPU kernels, which determine memory movement and hardware precision paths. Change the binary and the same VRAM-resident model can run through slower code while clocks look normal. Scheduling controls how work reaches the device; caching can remove repeated computation. Swapping cards isolated one variable unusually well: performance followed the software. The test points upstream from the silicon but cannot identify the broken component. Honest debugging stops with the evidence, however painful that is for Reddit detectives.

Configuration flags can swing results too. In one WSL2 field report, enabling INT8_ACT improved prefill throughput by 59% over the stock path. It uses int8 activations for linear-layer computation, reducing that work. On a much longer prompt, adding int8 prefill attention gained another 6% over activations alone. Used alone, the attention option regressed. Optimizations are ingredients, not Pokémon; collecting every flag does not guarantee a stronger build. The report also reveals nothing about accuracy or long-run reliability on undisclosed workloads.

Power limits also belong in the benchmark header. One sustained test measured about 58 tokens per second at 200 watts and roughly 86 at 250 watts, a 50% gain on the same service. At 200 watts, the power-constrained workload reduced sustained clocks. Raising the limit restored throughput until temperature became the next constraint. Push further and throttling consumes the gain while your electricity funds a CUDA space heater.

Put the runtime version, precision path, context length, power limit and temperature beside every result. A peak screenshot taken before cooler saturation shows only the first few seconds—lovely if production ends before the fan wakes up.

A second card helps, and PCIe collects rent

A controlled vLLM comparison found two PCIe-connected RTX 3090 cards ran 16% to 35% faster than one across the tested concurrency sweep. The fixed host, model and harness isolate tensor parallelism as the source. Doubling the cards came nowhere near doubling greedy-decode throughput because every token added another communication round.

At the highest tested concurrency, DFlash2 reached 474 tokens per second, while MTP managed 432 under the same greedy-decoding conditions. That matters for a busy server but says little about one interactive request: aggregate throughput rewards batching; humans notice latency.

Here’s how vLLM creates the gap. During prompt processing, chunked prefill passes prompt tokens through one shared per-step budget. Concurrent prompts queue for slices rather than receiving independent prefill capacity, as the syv-ai documentation explains. Decode differs because the server can batch tokens from many active requests, occupying more of the GPU. In one benchmark, aggregate decode rose from about 46 tokens per second with one request to around 1,100 under heavy concurrency. The silicon did not change; the scheduler filled idle execution slots. Great for shared-service throughput. Deeply flattering for a machine serving one impatient founder. I’ve stared at a token counter as if anger improves CUDA utilization.

Speculative decoding targets another part of the loop. A drafter proposes candidate tokens; the target model verifies them. If it accepts several, the system emits multiple tokens in one verification round. The lookup-augmented variant finds a matching suffix in the request’s recent history and copies the following context as its proposal, skipping the drafter forward pass until the continuation diverges. Repetitive text works well because likely continuations already appear in the answer. A dual-card DFlash2 report found about twice the decode throughput on a “repeat this phrase” workload with context copying enabled. The current chain path supported only single-request batches, and nobody knows whether the gain survives normal production traffic with concurrent requests.

There is a genuine correctness dispute. The tonyd2wild repository authors explain that the target model retains its normal verification loop, which should preserve its output distribution. Yet a greedy SGLang comparison found deterministic token divergence with Qwen3.8 thinking mode enabled. Repeating the target-only run produced the same sequence; the no-thinking control matched exactly. This narrows the safe claim and leaves an implementation question around thinking modes. I’d test my prompts before promising bit-for-bit equivalence.

Long context adds another trade. On one card, KVarN compression expanded the KV pool from about 174,000 to 292,000 tokens. Compressing the cache created the room; generation paid for it.

On the same tasks, speculative decode fell from 68 tokens per second with an FP8 cache to 32 with KVarN. No free antipasto.

I test the job before the seller

I start with the workload the RTX 3090 must run for hours. I load the intended model at the intended context, let temperatures stabilize, watch sustained clocks and wait for errors. If I need concurrency, I replay that traffic instead of multiplying a single-stream result in Excel like a tiny management consultant. I reserve room for the first-prefill activation spike and every other process using the card. Then I price the power supply and cooling around the executor. A bargain GPU that forces a full rebuild has eaten the bargain.

The stupidest version is on my desk. My RTX 5060 Ti runs image generation, and ComfyUI leaves Ollama about 150 MB of VRAM. So a 20B language model runs entirely on the CPU. The card exists; its memory is spoken for.

Our baseline uses measurements taken on an M3 Max on August 25, 2026. Both gpt-oss models used MXFP4 and stayed fully resident in unified memory.

The smaller gpt-oss:20b generated about 74 tokens per second. The larger gpt-oss:120b managed roughly 51 on the same machine.

That’s a substantial parameter gap: around 21 billion versus 117 billion. Prompt processing fell from about 756 tokens per second on the smaller model to 215 on the larger. Time to first token rose from around four seconds to nearly six.

A used GPU must beat my existing computer on my workload or unlock a model it serves poorly. Otherwise I bought a loud metal rectangle because the internet gave me nostalgia.

The card makes sense when the seller will run my test, especially if memory fit blocks the job. Sealed old stock priced as a collectible can remain sealed. An untested marketplace card belongs in the repair-project budget.

My bet: working RTX 3090 cards will hold their value through 2027 as local AI keeps demanding this memory tier. The dangerous listings will be immaculate. A dusty card with a workload log has probably confessed its sins. A pristine box marked “never mined” is where the opera begins.

Frequently asked questions

Is the RTX 3090 still good for local AI?

The RTX 3090 remains compelling for local AI because 24GB of VRAM can hold capable quantized models with working room. Its value depends on the runtime build, model format, context length, power limit, cooling, and whether the intended workload fits without exhausting memory.

Does using two RTX 3090 cards double AI performance?

Two PCIe-connected RTX 3090 cards delivered only 16% to 35% more throughput than one in a controlled vLLM comparison. Tensor parallelism creates aggregate memory room, but each decoding step exchanges intermediate results across cards, so PCIe communication joins every token’s execution path and prevents performance from doubling.

How should a used RTX 3090 be tested before purchase?

A used RTX 3090 should run the intended model at the intended context for long enough to stabilize temperatures. Testing should monitor sustained clocks, errors, power limits, cooling, activation spikes, and available VRAM while reproducing the concurrency and software configuration expected in normal use.