[BUG 🪲] window capture getting pink color output in the recorded video

[yashwanth2804]

Describe the bug 🐛

use std::{
    any::Any,
    io::{self, Write},
    time::Instant,
};

use windows_capture::{
    capture::GraphicsCaptureApiHandler,
    encoder::{VideoEncoder, VideoEncoderQuality, VideoEncoderType},
    frame::Frame,
    graphics_capture_api::InternalCaptureControl,
    monitor::Monitor,
    settings::{ColorFormat, CursorCaptureSettings, DrawBorderSettings, Settings},
    window::Window,
};

// This struct will be used to handle the capture events.
struct Capture {
    // The video encoder that will be used to encode the frames.
    encoder: Option<VideoEncoder>,
    // To measure the time the capture has been running
    start: Instant,
}

impl GraphicsCaptureApiHandler for Capture {
    // The type of flags used to get the values from the settings.
    type Flags = String;

    // The type of error that can occur during capture, the error will be returned from `CaptureControl` and `start` functions.
    type Error = Box<dyn std::error::Error + Send + Sync>;

    // Function that will be called to create the struct. The flags can be passed from settings.
    fn new(message: Self::Flags) -> Result<Self, Self::Error> {
        println!("Got The Flag: {message}");

        let encoder = VideoEncoder::new(
            VideoEncoderType::Mp4,
            VideoEncoderQuality::HD1080p,
            1920,
            1080,
            "video.mp4",
        )?;

        Ok(Self {
            encoder: Some(encoder),
            start: Instant::now(),
        })
    }

    // Called every time a new frame is available.
    fn on_frame_arrived(
        &mut self,
        frame: &mut Frame,
        capture_control: InternalCaptureControl,
    ) -> Result<(), Self::Error> {
        print!(
            "\rRecording for: {} seconds",
            self.start.elapsed().as_secs()
        );
        io::stdout().flush()?;

        // Send the frame to the video encoder
        self.encoder.as_mut().unwrap().send_frame(frame)?;

        // Note: The frame has other uses too for example you can save a single for to a file like this:
        // frame.save_as_image("frame.png", ImageFormat::Png)?;
        // Or get the raw data like this so you have full control:
        // let data = frame.buffer()?;

        // Stop the capture after 6 seconds
        if self.start.elapsed().as_secs() >= 6 {
            // Finish the encoder and save the video.
            self.encoder.take().unwrap().finish()?;

            capture_control.stop();

            // Because there wasn't any new lines in previous prints
            println!();
        }

        Ok(())
    }

    // Optional handler called when the capture item (usually a window) closes.
    fn on_closed(&mut self) -> Result<(), Self::Error> {
        println!("Capture Session Closed");

        Ok(())
    }
}

fn main() {
    // Gets The Foreground Window, Checkout The Docs For Other Capture Items
    let primary_monitor = Monitor::primary().expect("There is no primary monitor");
    print!(
        "\rPrimary Monitor: {}",
        primary_monitor.name().unwrap_or_default()
    );

    let window_to_capture = Window::from_contains_name("Visual Studio Code")
        .expect("There is no window with that name");

    // print!("\rWindow to Capture: {}", window_to_capture.try_into().unwrap());

    let settings = Settings::new(
        // Item To Captue
        window_to_capture,
        // primary_monitor,
        // Capture Cursor Settings
        CursorCaptureSettings::Default,
        // Draw Borders Settings
        DrawBorderSettings::Default,
        // The desired color format for the captured frame.
        ColorFormat::Rgba8,
        // Additional flags for the capture settings that will be passed to user defined `new` function.
        "Yea This Works".to_string(),
    )
    .unwrap();

    // Starts the capture and takes control of the current thread.
    // The errors from handler trait will end up here
    Capture::start(settings).expect("Screen Capture Failed");
}

I changed this part

 let window_to_capture = Window::from_contains_name("Visual Studio Code")
        .expect("There is no window with that name");

but when i give primary monitor thing it works fine ,

Expected behavior 📝
LIke monitor it should work like normal

OS 🤖

Edition Windows 11 Pro
Version 23H2
Installed on ‎12-‎08-‎2023
OS build 22631.3447
Experience Windows Feature Experience Pack 1000.22688.1000.0

Additional context ➕
Add any other context about the problem here.

[NiiightmareXD]

This usually happens because of wrong resolution in VideoEncoder.

I'm looking for a way to make it work with all resolution but for now it should be the same as the source resolution.

[yashwanth2804]

Do we get window size from the window instance ,
Please share workable code if u got
Thank you

[NiiightmareXD]

What resolution is your monitor?

[yashwanth2804]

1920 1080 p 24 inch

[yashwanth2804]

use std::{
    any::Any,
    io::{self, Write},
    time::Instant,
};

use windows::Win32::{
    Foundation::{BOOL, HWND, LPARAM, RECT, TRUE},
    UI::WindowsAndMessaging::GetWindowRect,
};
use windows_capture::{
    capture::GraphicsCaptureApiHandler,
    encoder::{VideoEncoder, VideoEncoderQuality, VideoEncoderType},
    frame::Frame,
    graphics_capture_api::InternalCaptureControl,
    monitor::Monitor,
    settings::{ColorFormat, CursorCaptureSettings, DrawBorderSettings, Settings},
    window::Window,
};

// This struct will be used to handle the capture events.
struct Capture {
    // The video encoder that will be used to encode the frames.
    encoder: Option<VideoEncoder>,
    // To measure the time the capture has been running
    start: Instant,
}

impl GraphicsCaptureApiHandler for Capture {
    // The type of flags used to get the values from the settings.
    type Flags = String;

    // The type of error that can occur during capture, the error will be returned from `CaptureControl` and `start` functions.
    type Error = Box<dyn std::error::Error + Send + Sync>;

    // Function that will be called to create the struct. The flags can be passed from settings.
    fn new(message: Self::Flags) -> Result<Self, Self::Error> {
        println!("Got The Flag: {message}");
        let flags_array: Vec<&str> = message.split(',').collect();
        let mut w_width = flags_array[0].parse::<u32>().unwrap();
        let mut w_height = flags_array[1].parse::<u32>().unwrap();

        let encoder = VideoEncoder::new(
            VideoEncoderType::Mp4,
            VideoEncoderQuality::HD1080p,
            w_width,
            w_height,
            "Video_1.mp4",
        )?;

        Ok(Self {
            encoder: Some(encoder),
            start: Instant::now(),
        })
    }

    // Called every time a new frame is available.
    fn on_frame_arrived(
        &mut self,
        frame: &mut Frame,
        capture_control: InternalCaptureControl,
    ) -> Result<(), Self::Error> {
        print!(
            "\rRecording for: {} seconds",
            self.start.elapsed().as_secs()
        );
        io::stdout().flush()?;

        // Send the frame to the video encoder
        self.encoder.as_mut().unwrap().send_frame(frame)?;

        // Note: The frame has other uses too for example you can save a single for to a file like this:
        // frame.save_as_image("frame.png", ImageFormat::Png)?;
        // Or get the raw data like this so you have full control:
        // let data = frame.buffer()?;

        // Stop the capture after 6 seconds
        if self.start.elapsed().as_secs() >= 6 {
            // Finish the encoder and save the video.
            self.encoder.take().unwrap().finish()?;

            capture_control.stop();

            // Because there wasn't any new lines in previous prints
            println!();
        }

        Ok(())
    }

    // Optional handler called when the capture item (usually a window) closes.
    fn on_closed(&mut self) -> Result<(), Self::Error> {
        println!("Capture Session Closed");

        Ok(())
    }
}

pub fn get_window_size(hwnd: HWND) -> Result<(i32, i32), windows::core::Error> {
    let mut rect = RECT::default();
    unsafe {
        GetWindowRect(hwnd, &mut rect).ok().expect("s");
    }
    let width = rect.right - rect.left;
    let height = rect.bottom - rect.top;
    Ok((width, height))
}

fn main() {
    // Gets The Foreground Window, Checkout The Docs For Other Capture Items
    let primary_monitor = Monitor::primary().expect("There is no primary monitor");
    print!(
        "\rPrimary Monitor: {}",
        primary_monitor.name().unwrap_or_default()
    );

    let window_to_capture = Window::from_contains_name("Visual Studio Code")
        .expect("There is no window with that name");

    let size = get_window_size(window_to_capture.as_raw_hwnd()).unwrap();

    print!("\rWindow Size: {}x{}", size.0, size.1);

    let array = vec![size.0.to_string(), size.1.to_string()];
    // print!("\rWindow to Capture: {}", window_to_capture.try_into().unwrap());

    let settings = Settings::new(
        // Item To Captue
        window_to_capture,
        // primary_monitor,
        // Capture Cursor Settings
        CursorCaptureSettings::Default,
        // Draw Borders Settings
        DrawBorderSettings::Default,
        // The desired color format for the captured frame.
        ColorFormat::Rgba8,
        // Additional flags for the capture settings that will be passed to user defined `new` function.
        array.join(","),
    )
    .unwrap();

    // Starts the capture and takes control of the current thread.
    // The errors from handler trait will end up here
    Capture::start(settings).expect("Screen Capture Failed");
}

in this code i made it worked , when vs code is in windowed mode ,

if the vs code is in full size mode , getting the same issue in pink color

[NiiightmareXD]

check if this works:

use std::{
    io::{self, Write},
    time::Instant,
};

use windows_capture::{
    capture::GraphicsCaptureApiHandler,
    encoder::{VideoEncoder, VideoEncoderQuality, VideoEncoderType},
    frame::Frame,
    graphics_capture_api::InternalCaptureControl,
    monitor::Monitor,
    settings::{ColorFormat, CursorCaptureSettings, DrawBorderSettings, Settings},
    window::Window,
};

// This struct will be used to handle the capture events.
struct Capture {
    // The video encoder that will be used to encode the frames.
    encoder: Option<VideoEncoder>,
    // To measure the time the capture has been running
    start: Instant,
}

impl GraphicsCaptureApiHandler for Capture {
    // The type of flags used to get the values from the settings.
    type Flags = String;

    // The type of error that can occur during capture, the error will be returned from `CaptureControl` and `start` functions.
    type Error = Box<dyn std::error::Error + Send + Sync>;

    // Function that will be called to create the struct. The flags can be passed from settings.
    fn new(message: Self::Flags) -> Result<Self, Self::Error> {
        println!("Got The Flag: {message}");

        let encoder = VideoEncoder::new(
            VideoEncoderType::Mp4,
            VideoEncoderQuality::HD1080p,
            1920,
            1080,
            "video.mp4",
        )?;

        Ok(Self {
            encoder: Some(encoder),
            start: Instant::now(),
        })
    }

    // Called every time a new frame is available.
    fn on_frame_arrived(
        &mut self,
        frame: &mut Frame,
        capture_control: InternalCaptureControl,
    ) -> Result<(), Self::Error> {
        if frame.width() == 1920 && frame.height() == 1080 {
            print!(
                "\rRecording for: {} seconds",
                self.start.elapsed().as_secs()
            );
            io::stdout().flush()?;

            // Send the frame to the video encoder
            self.encoder.as_mut().unwrap().send_frame(frame)?;

            // Stop the capture after 6 seconds
            if self.start.elapsed().as_secs() >= 6 {
                // Finish the encoder and save the video.
                self.encoder.take().unwrap().finish()?;

                capture_control.stop();

                // Because there wasn't any new lines in previous prints
                println!();
            }
        } else {
            println!("Wrong resolution");
        }

        Ok(())
    }

    // Optional handler called when the capture item (usually a window) closes.
    fn on_closed(&mut self) -> Result<(), Self::Error> {
        println!("Capture Session Closed");

        Ok(())
    }
}

fn main() {
    // Gets The Foreground Window, Checkout The Docs For Other Capture Items
    let primary_monitor = Monitor::primary().expect("There is no primary monitor");
    print!(
        "Primary Monitor: {}",
        primary_monitor.name().unwrap_or_default()
    );

    let window_to_capture = Window::from_contains_name("Visual Studio Code")
        .expect("There is no window with that name");

    let settings = Settings::new(
        // Item To Captue
        window_to_capture,
        // Capture Cursor Settings
        CursorCaptureSettings::Default,
        // Draw Borders Settings
        DrawBorderSettings::Default,
        // The desired color format for the captured frame.
        ColorFormat::Rgba8,
        // Additional flags for the capture settings that will be passed to user defined `new` function.
        "Yea This Works".to_string(),
    )
    .unwrap();

    // Starts the capture and takes control of the current thread.
    // The errors from handler trait will end up here
    Capture::start(settings).expect("Screen Capture Failed");
}

I know fullscreen windows can sometimes be tricky, especially when they are larger than 1920x1080 resolution. This code checks for such cases.

[yashwanth2804]

Need a solution for this , when window is full sized unable to capture , getting wrong resolution .
so programmatically need to resize window and let it capture

please let me know if any possible solution here

Thank you

[NiiightmareXD]

The most straightforward solution to the problem is to record with the same source resolution. However, I'm interested in developing a resizing operation that allows users to record at any desired resolution and output it in their desired resolution. This feature is similar to what most recording software does, but it's a bit complicated, and I want to make it highly efficient. To achieve this, I plan to use DirectX with compute shader technology, which I'm unfamiliar with. Nonetheless, recording monitors are currently effortless.

I will leave this issue open until the problem is fully resolved.

If anyone is familiar with DirectX, please let me know. My current idea is to create a scene with the same size as the VideoEncoder resolution and overlay the image on top so that if the image is slightly larger, it will be cropped, and if it's smaller, there will be a black border around it, similar to how OBS does it.

[yashwanth2804]

Thank you for the detailed response, please let us know once added the new features regarding this.

I'm expecting this library as a simple plug and play type ,
And go to library for video recording

All the best