[UI] Custom Blender Interfaces and Data Passthrough #481
Replies: 11 comments 1 reply
-
|
Some nice ideas here. 😀 In case you don't know, the plugin system already lets you register stuff on the Blender side: While it's not currently supported, ideally, it should be possible to register custom pipelines from plugins too. I haven't thought of this before, but I think a custom message system for the Bridge API would be a nice addition. This would allow implementing buttons, but also sending extra data to Malt.
There's already a shared dictionary (Bridge.shared_dict), with an entry used for printing performance stats in Blender. The trickiest part would be the custom UI stuff for BlenderMalt parameters. If you have specific performance or UX issues we can also try to solve them at the core, instead of through plugins, though. |
Beta Was this translation helpful? Give feedback.
-
Yeah, a button type would be very useful, and a simple message API would work pretty well. I don't see how to implement them exactly here though. I think my UX issues come into two camps:
I don't know if you have ideas for those ? That might be independent of this discussion or linked to it, and might be very specific to my needs (needing a centralized place to hold and bake data, and needing access to the GPU AND a UI that's flexible enough for experiments). |
Beta Was this translation helpful? Give feedback.
-
Something along these lines (I haven't tested it):
I'm not sure what you mean here. Are still you talking about the UI?
Yep, this has been on my to-do since forever. It shouldn't be too hard.
That sounds like something that would have to be done on the Blender side. |
Beta Was this translation helpful? Give feedback.
-
Oh I see, would be useful indeed!
Hmmm... I think it makes more sense to only have it being called once, since you only press the button once and there's a single Malt instance. If we have a method to force a refresh of the current viewports, I think it will cover most usecases. Passing the scene seems useful.
Yes, I meant literally showing the parameters is slowing down my Blender lol, the shader / pipeline is going fast, but scrolling the material list is unpractical and very laggy (bordering on a second between scroll refreshes).
What are the parts we should figure out ? UX for the programmers, or UX for the users ? Maybe the simplest solution is having each Parameter have a
Yep that probably is something way too specific / involved. The plugin system you mentioned should be able to help there from what I understand ? I feel like the discussion is moving along three points:
Not to rush anything along since we're still discussing, but I think structure is useful over text, and we might be able to "convert" them into actual tasks later. |
Beta Was this translation helpful? Give feedback.
-
The issue is that each Viewport has its own Scene and Pipeline instance, so if you want to do something scene-related you would have to do it per Viewport. I think we might be able to share the same scenes across viewports as long as they use the same Scene and ViewLayer on the Blender side (except for the Camera).
So you are pre-registering from your Pipeline 1000 Parameters of type Material? I guess in the World or Scene tab?
For users, the MaltProperties class already has a remove_property function.
You can think of blendermalt_register/unregister like the regular register/unregister functions all Blender addons use. So yep, you have full access to the bpy API.
I think pull requests would work just fine? |
Beta Was this translation helpful? Give feedback.
-
Oh I didn't know that. I guess you can also change the displayed objects depending on the exact viewport ? That could be problematic if you're doing something like writing to the disk, if two processes try the same file at the same time. Maybe we could add a flag to the first one called (or an ID to each) so that it's sure to not mix up between processes ?
Yep, from the world parameters. I'm not sure where else I can get something I can reliably get back on the Malt side. I don't necessarily need all of them at the same time, oftentimes just showing 16 would be enough, but I can't trigger the refresh effectively without relaunching the malt pipeline.
That could be a solution (albeit more finicky I think), but I'm a bit confused: you can get the objects by name ? Last time we talked about this iirc you said it would be something that's possible, but not fit for general use (since it was in the context of mesh loading and could very well be a different mesh).
I'll take a look at remove_property, but I'm not sure exactly that it should be something for the end-user.
They can be the same, but I think the UI manipulation features should stay in the creator part, since they are the ones knowledgeable about their tool. End users in general are less technical, and leaving the option open is gonna be a source of errors and make it more complex for them. Based on the experience I had working it's better to keep it simple for them. Linking UI display options to the parameters make sense imo, because:
I think for starters, we can add two display options that I think are useful and I can think of:
I think this should already cover a fair amount of ground to start out, while staying within the current codebase as I understand it and fitting with common use case patterns, so I think it's worth implementing. What do you think ?
Thanks! I'll check it out. How do you think would be best to have that link to the Malt server/pipeline ? Could work with some IPC, but I think modules registered this way can ignore the Blender / Malt separation by design of being modules requiring both Malt and Blender, so having a built-in way might be better ? I don't know python and blender enough to think of repercussions of just giving the open pipelines as a parameter or something.
Yeah, I meant we can probably separate them into concrete tasks with solutions to implement when we decide on them, so that we can then merge them as pull requests. :) |
Beta Was this translation helpful? Give feedback.
-
There's only one Malt process for each Blender instance, different viewports are rendered sequentially.
The object's names are not sent, but all the resources (meshes, textures, materials) are loaded through a proxy system and they are all stored in the scene in a dictionary, so you should be able to retrieve them with Maybe you could instead add some enum or string parameters to the material themselves, so you can search for the relevant ones?
All caps parameters and parameters with a leading underscore are already hidden in the UI. We already have "categories" that get grouped together, .ie
Through the Bridge interface, that's the only possible way. That's why I suggested the CUSTOM message type.
You can certainly import a python module from both processes, and you can import any Malt module from BlenderMalt.
Sure, what I mean is that it's ok to open a pull request for a specific feature even if the code is not ready to be merged, or even if there's no code yet at all but you want to discuss the implementation and design details. |
Beta Was this translation helpful? Give feedback.
-
Oh neat, that's one less problem to worry about. Then why would we want to call a function several times when pressing a button if there is only one instance ? I would think a single call is the simplest and expected behavior
Oh I see! Thanks for the tip
I didn't want to clog it with potentially unneeded info because it's a bit long haha, I'll try to keep it concise.
Here is a picture to help convey the idea, IDs are arbitrary:
So my main issue here is improving the UX for me, so the solutions I imagined / tried:
So that was one of the objectives when making the post, to see if we could add some kind of way to categorize parameters as it would improve that UX for my case, but also in plenty of other cases by grouping parameters. Hopefully that wasn't too horrifying of a pipeline lol, I'm self taught on Blender so I don't know the conventions, and by being able to program I have enough power to break stuff. If you think I missed something in the solutions, don't hesitate to let me know! (to note, this pipeline isn't the only thing I came about here, I was wondering about the UX and plugin communication in general, since I know that I'll need that info in the future, but this is the most "practical" part).
Yeah, I wasn't suggesting that. I think that default behavior is fine, maybe slightly better if you could get that control, but it's honestly not a big enough issue at the moment. There's some other subjects in a similar vein for better integration with non-technical artists, but I think we can leave them for later lol.
Okay I see, so basically, "just" adding the fold to that system would work well.
Okay, so I'll try to phrase how I understand it works, let me know if I have a mistake somewhere or if you have more detail
Some points I'm not sure about:
I probably missed a few implementation points, but how far am I of the mark ?
Okay then, that would be unsuitable for that usecase, since I would need the same instance otherwise it won't have access to the correct data.
Ah okay haha, I've usually seen the workflow of discussion -> issue -> pull request used. If I sum up the remaining issues how I see it at the moment, and what is needed before I/we start working:
Thanks for all the time you spend with me here btw :) |
Beta Was this translation helpful? Give feedback.
-
Because:
It may not be needed for your specific use case since I guess all your scenes share the same materials (?), but that's not the usual case.
I think another (more simple) solution for your issue (and that still makes sense for the general use case) could be to add an option to fold material properties specifically, since they are the only parameter type that can increase the number of UI items arbitrarily. We could still add the foldable categories too if you want, though.
Beware, remove_property actually removes the property, it's not a UI thing. I mentioned it in the context of Malt remembering the values for undeclared parameters.
Yep! It also works for uniforms.
There's no explicit reference, but since they're both part of the same process you can do something like:
BlenderMalt doesn't have direct access to Malt, only to Bridge. graph LR
BM(BlenderMalt)
C(Client_API.py)
classDef Blender stroke:#fca503;
class BM,C Blender;
S(Server.py)
M(Malt)
classDef Malt stroke:#03fc4e;
class S,M Malt;
subgraph Blender Process
subgraph P1 [ ]
subgraph Bridge
C
end
BM <--> C
end
end
C <--> S
subgraph Malt Process
subgraph P2 [ ]
subgraph Bridge
S
end
S <--> M
end
end
classDef Padding fill:none,stroke:none;
class P1,P2 Padding
It simply tries to import any folder in the plugin dir as a python module and tries to call PLUGIN.blendermalt_register.
Never, unless you explicitly disable/enable BlenderMalt or use the Blender Reload Scripts operator.
Malt can't have access to BlenderMalt, they're two separate processes. All the communications are done through the Bridge interface.
Yes to both.
Here's where the UI is drawn: Malt/BlenderMalt/MaltProperties.py Line 566 in 9182cb2 We should have a boolean property for each group that should be rendered in the UI along its label: Malt/BlenderMalt/MaltProperties.py Line 636 in 9182cb2 In the for key in keys iteration that bool should be checked to know if the parameter must be rendered or not.
I'd say, read Bridge.Client_API.py, that should give you a good idea of how it works.
👍 👍 👍 |
Beta Was this translation helpful? Give feedback.
-
Oh okay! I understand better now.
That's a nice idea! I think there's still value to the foldable categories, because even when empty the list is long, but both can work in tandem as you suggest.
Oh okay, thanks for the clarification!
Oooh I didn't realize it was the same process, I understand that better now!
Oh okay, so that's the same as a regular plugin if I understand correctly ?
Thanks !
Neat! I'll do that. I think I'll start with the folding, since that seems to have less ramifications and be more directly useful. I'll get a PR when it starts to look decent, or come back if I have more questions. |
Beta Was this translation helpful? Give feedback.
-
|
Hello, to follow up on this, I've finally been able to work on this and managed to make most of foldable categories work I'll do a PR as you suggest when I can, as there are some issues still in my implementation that I'll need help with before merging. |
Beta Was this translation helpful? Give feedback.
-
|
That looks nice! 😀 |
Beta Was this translation helpful? Give feedback.
-
Hello, I have a more technical discussion to start, for solutions to get more advanced custom tools in.
Problem Statement
So at the moment Malt allows you to define custom pipelines which can do quite a lot, which is cool and can enable more advanced tools. The current issue I see is that the automatic interface generation can be a bit limited at times, for instance when you have a lot of materials / parameters at once (which then becomes hard to navigate and can even slow down the software pretty hard when reaching higher numbers). It also makes it a bit technical at times, by having raw variables to tune, which can make it unsuitable for less-technical artists.
I think adding some way to be able to communicate both ways with that interface could help make more complex tools and pipelines.
Example Use cases
Potential Issues
Implementation / Design Ideas
The global solution ideas I may envision for that issue (first draft, probably missing a lot of constraints):
I'll annotate that I'm not an expert with Blender / Python / bpy itself so there might be quite a few elements that I missed. This is also not a "straight" up request, I'm looking for a way to implement it and contribute it back in a way that works with Malt's design, and need guidance for that.
Anyway, please let me know what you think, what do you think I missed, what could be improved, potentially other use cases that could affect design! I'm pinging @pragma37 also since he's the Malt expert and we talked about passing some additional data down in the past (but not to that extent).
Beta Was this translation helpful? Give feedback.
All reactions