OBS Studio: NVIDIA NVENC Encoding Settings for Streaming¶

If you have an NVIDIA GPU and you're still using x264 (CPU) encoding in OBS, you're leaving performance on the table. NVENC offloads encoding to a dedicated hardware block on the GPU, which means your CPU stays free for the game or application you're capturing, and your stream quality actually improves at the same bitrate. This article covers the settings that work for streaming to Twitch and YouTube on NVIDIA cards, why each setting matters, and the gotchas that will bite you if you don't know about them.

The Short Answer¶

Output Mode: Advanced
Encoder: NVIDIA NVENC H.264 (or AV1 if your card supports it)
Rate Control: CBR
Bitrate: 6000 Kbps (Twitch max) / 8000-15000 Kbps (YouTube)
Keyframe Interval: 2
Preset: P5 – Slow (Quality)
Tuning: High Quality
Multipass Mode: Two Passes (Quarter Resolution)
Profile: High
Look-ahead: Enabled
Psycho Visual Tuning: Enabled
Max B-frames: 2

Background¶

NVENC is the hardware encoder built into NVIDIA GPUs. It's been on cards since the Kepler generation (GTX 600 series), but the quality has improved dramatically with each generation. On Pascal (GTX 1000) and newer, NVENC quality is competitive with medium x264. On Turing (RTX 2000) and Ampere (RTX 3000) and newer, it's legitimately good — most viewers can't tell the difference at equivalent bitrates.

The tradeoff compared to x264: NVENC has less flexibility in quality scaling at very low bitrates, and there's only one NVENC encoder block per GPU (on most consumer cards), so if you're encoding multiple streams simultaneously you'll hit a limit. For normal single-stream use, none of that matters.

OBS added the new NVENC interface (using the NVIDIA SDK directly instead of FFmpeg) starting with OBS 28. The settings names changed — if you're on an older OBS or following an older guide, the options may look different. This guide covers OBS 28+.

Steps¶

1. Open OBS Output Settings¶

Go to Settings > Output and set Output Mode to Advanced. The Simple mode works but doesn't expose all the NVENC options that matter.

2. Select the NVENC Encoder¶

Under the Streaming tab:

Encoder: NVIDIA NVENC H.264

If you have an RTX 4000 series or newer and your streaming platform supports AV1 (YouTube does, Twitch added support in 2023), consider NVIDIA NVENC AV1 instead — better quality at the same bitrate. But H.264 is universal and safe for everything.

3. Set Rate Control and Bitrate¶

Rate Control: CBR (Constant Bitrate)

Always use CBR for streaming. VBR produces better quality for recording but streaming platforms expect a consistent bitrate. CBR keeps the encoder honest.

Bitrate by platform:

Platform	Max Bitrate	Recommended
Twitch (standard)	6000 Kbps	6000 Kbps
Twitch (Partner)	8000 Kbps	7500-8000 Kbps
YouTube Live	51000 Kbps (theoretical)	8000-15000 Kbps
Kick	8000 Kbps	6000-8000 Kbps

For 1080p60 on Twitch at 6000 Kbps with NVENC on an RTX 3000+ card, quality is solid. Don't go lower than 4500 Kbps for 1080p — you'll see compression artifacts in fast motion.

4. Set Keyframe Interval¶

Keyframe Interval: 2 seconds

Twitch and most platforms require a 2-second keyframe interval. Leave it at 2. Setting it to 0 (auto) will work but can cause issues with some platforms and VOD processing.

5. Configure Encoder Preset and Quality Settings¶

This is where the new NVENC SDK settings live in OBS 28+:

Preset: P5 – Slow (Quality)
Tuning: High Quality
Multipass Mode: Two Passes (Quarter Resolution)
Profile: High
Look-ahead: Enabled
Psycho Visual Tuning: Enabled
Max B-frames: 2

Why these settings:

P5 – Slow tells NVENC to use more of the encoder block's processing budget per frame. Despite the name, the performance impact on your GPU is minimal — NVENC runs on its own dedicated silicon, not the shader cores. P5 vs P7 is barely measurable in GPU load.

Two Passes (Quarter Resolution) does a fast first-pass analysis at reduced resolution to inform the full-resolution second pass. Better bit distribution than single-pass, especially for scenes with mixed motion and static areas.

Look-ahead lets the encoder analyze upcoming frames before encoding the current one, leading to better bitrate distribution across scene changes.

Psycho Visual Tuning optimizes for perceived visual quality rather than raw PSNR metrics. In practice this means better-looking textures and less "muddy" compression on complex scenes.

6. Set Video Resolution and Frame Rate¶

In Settings > Video:

Base (Canvas) Resolution: Your monitor resolution (e.g. 1920x1080)
Output (Scaled) Resolution: What you stream at
1920x1080 for 1080p
1280x720 for 720p (if bitrate is limited)
Common FPS Values: 60 for fast-paced games, 30 for slower content

For Twitch at 6000 Kbps: 1080p60 works well on RTX 3000+ cards. On older cards or at lower bitrates, 720p60 will look cleaner than 1080p60 because you're not spreading the bitrate as thin.

7. Test Before Going Live¶

Use OBS's built-in stream test or stream to a private/unlisted YouTube stream first. Watch back the VOD at 1080p and check for:

Compression artifacts in fast motion (lower bitrate or switch to 720p60)
Dropped frames in the OBS stats (encoder overload or network issue)
Audio sync drift over long sessions (set audio sample rate to 48kHz everywhere)

Check OBS stats while running: View > Stats. Encoder-side dropped frames means NVENC can't keep up. Render lag means your GPU is overloaded doing both rendering and encoding (rare with NVENC since it's on separate silicon, but possible on older cards).

Gotchas & Notes¶

Only one NVENC session on consumer cards. NVIDIA artificially limits consumer GPUs (GTX/RTX non-Ti non-Quadro) to a limited number of simultaneous NVENC sessions. In practice for streaming this doesn't matter — you're only running one stream. But if you're also recording with NVENC at the same time, you may hit this limit. The fix: record with a different encoder (x264 or the new AV1 if available) or use Shadowplay/Replay Buffer for recording instead.
OBS 27 vs OBS 28+ settings look completely different. The old NVENC interface used terms like "Max Quality" preset and had different option names. If you're following a pre-2022 guide or see "Max Quality" as a preset option, you're on the old interface. Update OBS.
Driver matters. NVENC quality and feature availability is partly driver-dependent. Keep NVIDIA drivers reasonably current. You don't need to be on the absolute latest, but being multiple major versions behind can mean missing NVENC improvements.
AV1 on RTX 4000+ is genuinely better. If you have a 4000-series card and you're streaming to YouTube, use AV1. Same settings as above, just swap the encoder. Quality difference at 6000 Kbps is noticeable — it handles fine detail and foliage much better than H.264.
CBR doesn't mean the bitrate is literally constant frame-to-frame. It means the encoder targets your specified average over time. You'll see small fluctuations in OBS stats — that's normal. What you don't want is sustained drops below your target, which usually means a network issue, not an encoder issue.
Audio settings: Set everything to 48kHz — OBS, your audio interface, your DAW if you use one. Mismatched sample rates cause drift over long streams that's impossible to fix in post.
On Linux, NVENC requires the proprietary NVIDIA driver. It does not work with Nouveau. If you're streaming on Linux and NVENC isn't showing up as an option in OBS, check that you're running the proprietary driver: nvidia-smi should return your GPU info. If it errors, you're on Nouveau.