Stuck at "Reading Image"

[pillai-karthik]

Stuck at "Reading Image" when running the analyzer.

from omegaconf import OmegaConf
import cv2
from pathlib import Path
import os
from facetorch import FaceAnalyzer
from omegaconf import OmegaConf

path_config="backgroundworker/deepfake_detection/conf/merged.config.yaml"

cfg = OmegaConf.load(path_config)
analyzer = FaceAnalyzer(cfg.analyzer)

os.environ["HYDRA_FULL_ERROR"] = "1"

def process(frame):
    
    Path("./temp").mkdir(parents=True, exist_ok=True)
    cv2.imwrite("./temp/in.jpg", frame)
    
    response = analyzer.run(
        path_image="./temp/in.jpg",
        batch_size=cfg.batch_size,
        fix_img_size=cfg.fix_img_size,
        return_img_data=cfg.return_img_data,
        include_tensors=cfg.include_tensors,
        path_output="./temp/out.jpg",
    )

    print(response)

    output = {face.indx: face.preds["deepfake"].other["multi"] for face in response.faces}

    print(output)
    return(output)

[tomas-gajarsky]

Based on the issue you've described where the process gets stuck at 'Reading Image', it seems that the step of writing and reading the image from disk could be the bottleneck or point of failure. Fortunately, with the introduction of the UniversalReader in facetorch version 0.5.0, there is a more efficient way to handle image data without the need to save frames as temporary files.

The UniversalReader class can directly process image data in various formats, including torch tensors and bytes, which can significantly streamline your workflow and potentially resolve the 'Reading Image' issue. Here's how you can adapt your current code to utilize this feature:

1. Remove the File Writing and Reading Steps: You no longer need to save the frame to disk. Instead, convert the frame directly to a tensor or a bytes object as required by your workflow.
2. Update Your Analyzer Configuration: Ensure that your configuration uses the UniversalReader. This may involve updating the configuration file or adjusting your code to instantiate the FaceAnalyzer with the appropriate reader specified.
3. Directly Pass Image Data to the Analyzer: Modify your process function to directly pass the image data to the analyzer.run method, using the image_source parameter instead of path_image.

Here is a conceptual example of how you might modify your process function:

def process(frame):
    # Convert your frame to a tensor or bytes format as required
    # Example for a tensor conversion; adjust as necessary for your data and model
    tensor_image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    tensor_image = transforms.ToTensor()(tensor_image)

    # Directly pass the tensor to the analyzer
    response = analyzer.run(
        image_source=tensor_image,  # Use the tensor here
        batch_size=cfg.batch_size,
        fix_img_size=cfg.fix_img_size,
        return_img_data=cfg.return_img_data,
        include_tensors=cfg.include_tensors,
    )

    print(response)
    output = {face.index: face.preds["deepfake"].other["multi"] for face in response.faces}
    print(output)
    return(output)

Please adjust the tensor conversion process based on your specific requirements and the expected format of your model. This approach should eliminate the need for temporary files and might resolve the issue you're experiencing.

If you're still encountering problems or have any further questions, feel free to reach out!