How to iterate over vector of pyclass objects in rust?
Question:
I am using maturin and I am trying to implement the method get_car() for my class.
But using the following code
use pyo3::prelude::*;
#[pyclass]
struct Car {
name: String,
}
#[pyclass]
struct Garage {
cars: Vec<Car>,
}
#[pymethods]
impl Garage {
fn get_car(&self, name: &str) -> Option<&Car> {
self.cars.iter().find(|car| car.name == name.to_string())
}
}
/// A Python module implemented in Rust.
#[pymodule]
fn pyo3_iter_issue(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Car>()?;
m.add_class::<Garage>()?;
Ok(())
}
I get this error message
the trait bound `Car: AsPyPointer` is not satisfied
the following other types implement trait `AsPyPointer`:
CancelledError
IncompleteReadError
InvalidStateError
LimitOverrunError
Option<T>
PanicException
Py<T>
PyAny
and 107 others
required for `&Car` to implement `IntoPy<Py<PyAny>>`
1 redundant requirement hidden
required for `Option<&Car>` to implement `IntoPy<Py<PyAny>>`
required for `Option<&Car>` to implement `OkWrap<Option<&Car>>`
I am still very new to rust and I do not understand the issue here?
Answers:
Rust does not have lifetimes, so any & references are not Python compatible.
The easiest way to fix this would be to use Car and .clone() instead:
use pyo3::prelude::*;
#[derive(Clone, Debug)]
#[pyclass]
struct Car {
name: String,
}
#[derive(Clone, Debug)]
#[pyclass]
struct Garage {
cars: Vec<Car>,
}
#[pymethods]
impl Car {
fn __str__(&self) -> String {
format!("{:?}", self)
}
}
#[pymethods]
impl Garage {
fn get_car(&self, name: &str) -> Option<Car> {
self.cars
.iter()
.find(|car| car.name == name.to_string())
.cloned()
}
#[new]
fn new() -> Self {
Self {
cars: vec![
Car {
name: "Ferrari".to_string(),
},
Car {
name: "Audi".to_string(),
},
],
}
}
}
/// A Python module implemented in Rust.
#[pymodule]
fn rust_python_playground(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Car>()?;
m.add_class::<Garage>()?;
Ok(())
}
#!/usr/bin/env python3
from rust_python_playground import Garage
garage = Garage()
ferrari = garage.get_car("Ferrari")
print(f"Ferrari: {ferrari}")
bugatti = garage.get_car("Bugatti")
print(f"Bugatti: {bugatti}")
Ferrari: Car { name: "Ferrari" }
Bugatti: None
Of course that would create copies of the object, which isn’t always desirable.
It’s a lot harder to reference something from Python code, though. Even the default get/set implementations that pyo3 can create clone the data, they do not reference.
For that, you would need a refcounter around your objects. In the standard library, this would be Rc, but Rc is managed by Rust and can therefore not be passed to Python.
The equivalent Python-managed reference counter is called Py:
use pyo3::prelude::*;
#[derive(Clone, Debug)]
#[pyclass]
struct Car {
name: String,
#[pyo3(get, set)]
horsepower: u32,
}
#[derive(Clone, Debug)]
#[pyclass]
struct Garage {
cars: Vec<Py<Car>>,
}
#[pymethods]
impl Car {
fn __str__(&self) -> String {
format!("{:?}", self)
}
}
#[pymethods]
impl Garage {
fn get_car(&self, name: &str) -> Option<Py<Car>> {
Python::with_gil(|py| {
self.cars
.iter()
.find(|car| car.as_ref(py).borrow().name == name.to_string())
.cloned()
})
}
#[new]
fn new() -> PyResult<Self> {
Python::with_gil(|py| {
Ok(Self {
cars: vec![
Py::new(
py,
Car {
name: "Ferrari".to_string(),
horsepower: 430,
},
)?,
Py::new(
py,
Car {
name: "Audi".to_string(),
horsepower: 250,
},
)?,
],
})
})
}
fn __str__(&self) -> String {
Python::with_gil(|py| {
let mut garage_str = "[n".to_string();
for car in &self.cars {
garage_str += &format!(" {:?}n", car.as_ref(py).borrow());
}
garage_str += "]";
garage_str
})
}
}
/// A Python module implemented in Rust.
#[pymodule]
fn rust_python_playground(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Car>()?;
m.add_class::<Garage>()?;
Ok(())
}
#!/usr/bin/env python3
from rust_python_playground import Garage
garage = Garage()
print(f"Garage: {garage}")
print()
ferrari = garage.get_car("Ferrari")
bugatti = garage.get_car("Bugatti")
print(f"Ferrari: {ferrari}")
print(f"Bugatti: {bugatti}")
print()
print("Changing Ferrari's horsepower to >9000 ...")
ferrari.horsepower = 9001
print()
print(f"Garage: {garage}")
Garage: [
Car { name: "Ferrari", horsepower: 430 }
Car { name: "Audi", horsepower: 250 }
]
Ferrari: Car { name: "Ferrari", horsepower: 430 }
Bugatti: None
Changing Ferrari's horsepower to >9000 ...
Garage: [
Car { name: "Ferrari", horsepower: 9001 }
Car { name: "Audi", horsepower: 250 }
]
This has the drawback that now every time you want to access the object, even just from within Rust code, you need a GIL lock.
The third option is to use Rc (or Arc, because of threadsafety), but don’t expose it to Python directly; instead, write a CarRef wrapper that carries it. But then you might have to also use Mutex because of internal mutability, and it all gets messy pretty quickly. Although it’s of course doable:
use std::sync::{Arc, Mutex};
use pyo3::prelude::*;
#[derive(Clone, Debug)]
#[pyclass]
struct Car {
name: String,
#[pyo3(get, set)]
horsepower: u32,
}
#[derive(Clone, Debug)]
#[pyclass]
struct CarRef {
car: Arc<Mutex<Car>>,
}
#[derive(Clone, Debug)]
#[pyclass]
struct Garage {
cars: Vec<Arc<Mutex<Car>>>,
}
#[pymethods]
impl Car {
fn __str__(&self) -> String {
format!("{:?}", self)
}
}
#[pymethods]
impl CarRef {
fn __str__(&self) -> String {
format!("{:?}", self.car.lock().unwrap())
}
#[getter]
fn get_horsepower(&self) -> PyResult<u32> {
Ok(self.car.lock().unwrap().horsepower)
}
#[setter]
fn set_horsepower(&mut self, value: u32) -> PyResult<()> {
self.car.lock().unwrap().horsepower = value;
Ok(())
}
}
#[pymethods]
impl Garage {
fn get_car(&self, name: &str) -> Option<CarRef> {
self.cars
.iter()
.find(|car| car.lock().unwrap().name == name.to_string())
.map(|car| CarRef {
car: Arc::clone(car),
})
}
#[new]
fn new() -> PyResult<Self> {
Ok(Self {
cars: vec![
Arc::new(Mutex::new(Car {
name: "Ferrari".to_string(),
horsepower: 430,
})),
Arc::new(Mutex::new(Car {
name: "Audi".to_string(),
horsepower: 250,
})),
],
})
}
fn __str__(&self) -> String {
let mut garage_str = "[n".to_string();
for car in &self.cars {
garage_str += &format!(" {:?}n", car.lock().unwrap());
}
garage_str += "]";
garage_str
}
}
/// A Python module implemented in Rust.
#[pymodule]
fn rust_python_playground(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Car>()?;
m.add_class::<Garage>()?;
Ok(())
}
#!/usr/bin/env python3
from rust_python_playground import Garage
garage = Garage()
print(f"Garage: {garage}")
print()
ferrari = garage.get_car("Ferrari")
bugatti = garage.get_car("Bugatti")
print(f"Ferrari: {ferrari}")
print(f"Bugatti: {bugatti}")
print()
print("Changing Ferrari's horsepower to >9000 ...")
ferrari.horsepower = 9001
print()
print(f"Garage: {garage}")
Garage: [
Car { name: "Ferrari", horsepower: 430 }
Car { name: "Audi", horsepower: 250 }
]
Ferrari: Car { name: "Ferrari", horsepower: 430 }
Bugatti: None
Changing Ferrari's horsepower to >9000 ...
Garage: [
Car { name: "Ferrari", horsepower: 9001 }
Car { name: "Audi", horsepower: 250 }
]
But as you can see, it’s not quite straight-forward. But now you at least avoided the GIL lock.
I am using maturin and I am trying to implement the method get_car() for my class.
But using the following code
use pyo3::prelude::*;
#[pyclass]
struct Car {
name: String,
}
#[pyclass]
struct Garage {
cars: Vec<Car>,
}
#[pymethods]
impl Garage {
fn get_car(&self, name: &str) -> Option<&Car> {
self.cars.iter().find(|car| car.name == name.to_string())
}
}
/// A Python module implemented in Rust.
#[pymodule]
fn pyo3_iter_issue(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Car>()?;
m.add_class::<Garage>()?;
Ok(())
}
I get this error message
the trait bound `Car: AsPyPointer` is not satisfied
the following other types implement trait `AsPyPointer`:
CancelledError
IncompleteReadError
InvalidStateError
LimitOverrunError
Option<T>
PanicException
Py<T>
PyAny
and 107 others
required for `&Car` to implement `IntoPy<Py<PyAny>>`
1 redundant requirement hidden
required for `Option<&Car>` to implement `IntoPy<Py<PyAny>>`
required for `Option<&Car>` to implement `OkWrap<Option<&Car>>`
I am still very new to rust and I do not understand the issue here?
Rust does not have lifetimes, so any & references are not Python compatible.
The easiest way to fix this would be to use Car and .clone() instead:
use pyo3::prelude::*;
#[derive(Clone, Debug)]
#[pyclass]
struct Car {
name: String,
}
#[derive(Clone, Debug)]
#[pyclass]
struct Garage {
cars: Vec<Car>,
}
#[pymethods]
impl Car {
fn __str__(&self) -> String {
format!("{:?}", self)
}
}
#[pymethods]
impl Garage {
fn get_car(&self, name: &str) -> Option<Car> {
self.cars
.iter()
.find(|car| car.name == name.to_string())
.cloned()
}
#[new]
fn new() -> Self {
Self {
cars: vec![
Car {
name: "Ferrari".to_string(),
},
Car {
name: "Audi".to_string(),
},
],
}
}
}
/// A Python module implemented in Rust.
#[pymodule]
fn rust_python_playground(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Car>()?;
m.add_class::<Garage>()?;
Ok(())
}
#!/usr/bin/env python3
from rust_python_playground import Garage
garage = Garage()
ferrari = garage.get_car("Ferrari")
print(f"Ferrari: {ferrari}")
bugatti = garage.get_car("Bugatti")
print(f"Bugatti: {bugatti}")
Ferrari: Car { name: "Ferrari" }
Bugatti: None
Of course that would create copies of the object, which isn’t always desirable.
It’s a lot harder to reference something from Python code, though. Even the default get/set implementations that pyo3 can create clone the data, they do not reference.
For that, you would need a refcounter around your objects. In the standard library, this would be Rc, but Rc is managed by Rust and can therefore not be passed to Python.
The equivalent Python-managed reference counter is called Py:
use pyo3::prelude::*;
#[derive(Clone, Debug)]
#[pyclass]
struct Car {
name: String,
#[pyo3(get, set)]
horsepower: u32,
}
#[derive(Clone, Debug)]
#[pyclass]
struct Garage {
cars: Vec<Py<Car>>,
}
#[pymethods]
impl Car {
fn __str__(&self) -> String {
format!("{:?}", self)
}
}
#[pymethods]
impl Garage {
fn get_car(&self, name: &str) -> Option<Py<Car>> {
Python::with_gil(|py| {
self.cars
.iter()
.find(|car| car.as_ref(py).borrow().name == name.to_string())
.cloned()
})
}
#[new]
fn new() -> PyResult<Self> {
Python::with_gil(|py| {
Ok(Self {
cars: vec![
Py::new(
py,
Car {
name: "Ferrari".to_string(),
horsepower: 430,
},
)?,
Py::new(
py,
Car {
name: "Audi".to_string(),
horsepower: 250,
},
)?,
],
})
})
}
fn __str__(&self) -> String {
Python::with_gil(|py| {
let mut garage_str = "[n".to_string();
for car in &self.cars {
garage_str += &format!(" {:?}n", car.as_ref(py).borrow());
}
garage_str += "]";
garage_str
})
}
}
/// A Python module implemented in Rust.
#[pymodule]
fn rust_python_playground(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Car>()?;
m.add_class::<Garage>()?;
Ok(())
}
#!/usr/bin/env python3
from rust_python_playground import Garage
garage = Garage()
print(f"Garage: {garage}")
print()
ferrari = garage.get_car("Ferrari")
bugatti = garage.get_car("Bugatti")
print(f"Ferrari: {ferrari}")
print(f"Bugatti: {bugatti}")
print()
print("Changing Ferrari's horsepower to >9000 ...")
ferrari.horsepower = 9001
print()
print(f"Garage: {garage}")
Garage: [
Car { name: "Ferrari", horsepower: 430 }
Car { name: "Audi", horsepower: 250 }
]
Ferrari: Car { name: "Ferrari", horsepower: 430 }
Bugatti: None
Changing Ferrari's horsepower to >9000 ...
Garage: [
Car { name: "Ferrari", horsepower: 9001 }
Car { name: "Audi", horsepower: 250 }
]
This has the drawback that now every time you want to access the object, even just from within Rust code, you need a GIL lock.
The third option is to use Rc (or Arc, because of threadsafety), but don’t expose it to Python directly; instead, write a CarRef wrapper that carries it. But then you might have to also use Mutex because of internal mutability, and it all gets messy pretty quickly. Although it’s of course doable:
use std::sync::{Arc, Mutex};
use pyo3::prelude::*;
#[derive(Clone, Debug)]
#[pyclass]
struct Car {
name: String,
#[pyo3(get, set)]
horsepower: u32,
}
#[derive(Clone, Debug)]
#[pyclass]
struct CarRef {
car: Arc<Mutex<Car>>,
}
#[derive(Clone, Debug)]
#[pyclass]
struct Garage {
cars: Vec<Arc<Mutex<Car>>>,
}
#[pymethods]
impl Car {
fn __str__(&self) -> String {
format!("{:?}", self)
}
}
#[pymethods]
impl CarRef {
fn __str__(&self) -> String {
format!("{:?}", self.car.lock().unwrap())
}
#[getter]
fn get_horsepower(&self) -> PyResult<u32> {
Ok(self.car.lock().unwrap().horsepower)
}
#[setter]
fn set_horsepower(&mut self, value: u32) -> PyResult<()> {
self.car.lock().unwrap().horsepower = value;
Ok(())
}
}
#[pymethods]
impl Garage {
fn get_car(&self, name: &str) -> Option<CarRef> {
self.cars
.iter()
.find(|car| car.lock().unwrap().name == name.to_string())
.map(|car| CarRef {
car: Arc::clone(car),
})
}
#[new]
fn new() -> PyResult<Self> {
Ok(Self {
cars: vec![
Arc::new(Mutex::new(Car {
name: "Ferrari".to_string(),
horsepower: 430,
})),
Arc::new(Mutex::new(Car {
name: "Audi".to_string(),
horsepower: 250,
})),
],
})
}
fn __str__(&self) -> String {
let mut garage_str = "[n".to_string();
for car in &self.cars {
garage_str += &format!(" {:?}n", car.lock().unwrap());
}
garage_str += "]";
garage_str
}
}
/// A Python module implemented in Rust.
#[pymodule]
fn rust_python_playground(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Car>()?;
m.add_class::<Garage>()?;
Ok(())
}
#!/usr/bin/env python3
from rust_python_playground import Garage
garage = Garage()
print(f"Garage: {garage}")
print()
ferrari = garage.get_car("Ferrari")
bugatti = garage.get_car("Bugatti")
print(f"Ferrari: {ferrari}")
print(f"Bugatti: {bugatti}")
print()
print("Changing Ferrari's horsepower to >9000 ...")
ferrari.horsepower = 9001
print()
print(f"Garage: {garage}")
Garage: [
Car { name: "Ferrari", horsepower: 430 }
Car { name: "Audi", horsepower: 250 }
]
Ferrari: Car { name: "Ferrari", horsepower: 430 }
Bugatti: None
Changing Ferrari's horsepower to >9000 ...
Garage: [
Car { name: "Ferrari", horsepower: 9001 }
Car { name: "Audi", horsepower: 250 }
]
But as you can see, it’s not quite straight-forward. But now you at least avoided the GIL lock.