Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an option for more aggressive conversion #355

Open
LilithHafner opened this issue Sep 12, 2023 · 2 comments
Open

Add an option for more aggressive conversion #355

LilithHafner opened this issue Sep 12, 2023 · 2 comments
Labels
enhancement New feature or request

Comments

@LilithHafner
Copy link
Contributor

Is your feature request related to a problem? Please describe.
I have users that are passing in python objects of unknown type and Julia code that expects native Julia objects. When trying to migrate from PyCall to PythonCall, I find the conversions defined here do not convert user provided objects into the most native equivalent Julia type.

Describe the solution you'd like
I would like an option to convert an arbitrary Python object to a Julia object with eager semantics. For example,

julia> eager_pyconvert(Any, pyeval("[1.0, 2.0, 3.0]", Main))
3-element Vector{Float64}:
 1.0
 2.0
 3.0

julia> pyconvert(Any, pyeval("[1.0, 2.0, 3.0]", Main)) # Current (if type is unknown)
3-element PyList{Any}:
 1.0
 2.0
 3.0

julia> pyconvert(Array, pyeval("[1.0, 2.0, 3.0]", Main)) # Current (if type is known)
3-element Vector{Float64}:
 1.0
 2.0
 3.0

julia> py"[1.0, 2.0, 3.0]" # PyCall
3-element Vector{Float64}:
 1.0
 2.0
 3.0

By "eager semantics" I mean it will convert to as specific a target type as possible. Or, equivalently, opt out of types defined by PythonCall.jl.

Describe alternatives you've considered
Coming up with a "magic type" that can be passed to the existing pyconvert function

types(m) = (x for x in (getglobal(m, n) for n in names(m)) if x isa Type && x !== Any)
BaseTypes = BaseTypes = Union{types(Core)..., types(Base)...}
pyconvert(BaseTypes, pyeval("[1.0, 2.0, 3.0]", Main))

Fails because PyList <: BaseTypes.

Restricting to concrete types segfaults julia

types(m) = (x for x in (getglobal(m, n) for n in names(m)) if x isa Type && !isabstracttype(x))
BaseTypes = Union{types(Core)..., types(Base)...}
pyconvert(BaseTypes, pyeval("[1.0, 2.0, 3.0]", Main))
# [68187] signal (11.2): Segmentation fault: 11
# in expression starting at REPL[41]:1
# ...

Home rolling conversion rules for specific types feels like reinventing the wheel and hard to maintain.

Additional context
Basically, I'd like to opt into the conversion behavior of PyCall.

@LilithHafner LilithHafner added the enhancement New feature or request label Sep 12, 2023
@cjdoris
Copy link
Collaborator

cjdoris commented Sep 13, 2023

I've had a similar request a few times, and agree this would be useful.

My current suggestion is you write your own conversion function to cover any cases you need, such as:

function eager_convert(x::Py)
  if pyisinstance(x, pybuiltins.list)
    [eager_convert(x) for x in x]
  elseif pyisinstance(x, pybuiltins.dict)
    ...
  else
     pyconvert(Any, x)
  end
end

There is a future plan to support keyword arguments in pyconvert to more finely control its behaviour. In particular, I'd like to have pyconvert(T, x; copy=true) only perform copying conversions (which rules out all the wrapper types). But this is not imminent.

@LilithHafner
Copy link
Contributor Author

Lovely. I'll plan to implement some form of eager conversion for https://github.com/SciML/diffeqpy and if folks like how it behaves I'll try to upstream it by integrating it into the existing PythonCall api.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants