Generic Rust Struct Exposed by Pyo3

[StuartHadfield]

Hello

I am trying to define a very generic struct, something like:

struct Foo<T> {
  data: Vec<T>
}

impl<T> Foo {
  def a(&self) {
    println!("calling a()!");
  }
}

Naturally, however, pyo3 prehibits using generic types.

I'm a bit at a loss for how I could get around this. Fundamentally my problem is, in Python, I will be loading data out of files and into lists, but I want rust to do the operating on those lists. The lists will contain different types - ints, strings, and possibly datetime values, so a generic type is really what I need. The type of the individual fields for the operations I will be conducting in rust doesn't really matter.

The only real progress I can make is using macros:

// main.rs
use pyo3::prelude::*;

pub mod core;
pub mod interface;

use interface::{FloatClass, IntClass, StrClass};

#[pymodule]
fn pyo3_test(_py: Python, m: &PyModule) -> PyResult<()> {
    m.add_class::<FloatClass>()?;
    m.add_class::<IntClass>()?;
    m.add_class::<StrClass>()?;
    Ok(())
}

// core.rs
pub struct Core<T> {
    pub data: Vec<T>,
}

impl<T> Core<T> {
    pub fn a(&self) {
        println!("running: a");
    }
}

// interface.rs
use pyo3::prelude::*;

use crate::core::Core;

macro_rules! create_interface {
    ($name: ident, $type: ident) => {
        #[pyclass]
        pub struct $name {
            pub inner: Core<$type>,
        }
        #[pymethods]
        impl $name {
            #[new]
            pub fn from_vector(data: Vec<$type>) -> Self {
                Self { inner: Core { data: data } }
            }
            pub fn a(&self) {
                self.inner.a()
            }
        }
    };
}

create_interface!(IntClass, i64);
create_interface!(FloatClass, f64);
create_interface!(StrClass, String);

This is sort of okay, but mostly it doesn't fit the bill. Namely because I now have several interfaces for wrapping a vector, each one able to ingest only a single type; but also because the code gets extremely bloated and hard to maintain very quickly.

Are there any better ways of doing this?

[adamreichold]

If the set of relevant types is closed, you could put an enum inside the #[pyclass], e.g.

#[pyclass]
pub struct Wrapper {
  inner: WrapperInner,
}

#[pymethods]
impl Wrapper {
  pub fn a(&self) {
    match self.inner {
      WrapperInner::Int(inner) => inner.a(),
      WrapperInner::Float(inner) => inner.a(),
      WrapperInner::Str(inner) => inner.a(),
    }
  }
}

enum WrapperInner {
  Int(Core<i64>),
  Float(Core<f64>),
  Str(Core<String>),
}

If the set is open, you can try to put a trait object within the #[pyclass], e.g.

#[pyclass]
pub struct Wrapper {
  inner: Box<dyn WrapperInner>,
}

#[pymethods]
impl Wrapper {
  pub fn a(&self) {
    self.inner.a();
  }
}

trait WrapperInner {
  fn a(&self);
}

impl<T> WrapperInner for Core<T> {
  fn a(&self) {
    Core::a(self);
  }
}

[StuartHadfield]

(Apologies if I'm being really slow)

[adamreichold]

Python is dynamically typed, so at some point you have to convert those dynamically typed values into statically typed values (or limit yourself to the Python API):

impl Wrapper {
  #[new]
  pub fn new(members: &PyAny) -> PyResult<Self> {
    let inner = if let Ok(ints) = Vec::<i64>::extract(members) {
      WrapperInner::Int(ints)
    } else if let Ok(floats) = Vec::<f64>::extract(members) {
      WrapperInner::Float(floats)
    } else if let Ok(strs) = Vec::<String>::extract(members) {
      WrapperInner::Str(strs)
    } else {
      return Err(...);
    };

    Self { inner }
  }
}

If the set of relevant types is open, you'll probably need a registry or something where extraction targets can be collected.

[StuartHadfield]

Thanks folks, I managed to get this working!

[mejrs]

You can probably use https://pyo3.rs/main/conversions/traits.html#deriving-frompyobject-for-enums to do this in a more ergonomic way.

[StuartHadfield]

Cheers @mejrs - I'm under pretty strict timelines for work so I'm planning on looking into ergonomics after I've just got something working first 😅 I'll have lots of questions then I'm sure!!