Planning an introduction to Malt video

[aVersionOfReality]

I am starting to plan a video to try to introduce Malt to the wider community. I've wanted to do this for a long time, but I hadn't had a chance to work with it enough myself until recently. I hear lots of people express interest in Malt, but they think the barrier of entry is too high. But many of the things they think are misconceptions, like that they have to work in glsl. Some good content to help people get started would show a lot of people why they may want to be using Malt, that it isn't actually that bad to get into, and how to get started.

To do this, I will need to be able to present a lot of information. So I am looking for ideas on what I should include, what is essential, etc. The goal of the video will be to summarize what Malt can do that makes it different from Eevee, what features it is missing that Eevee has, and how to do a basic node based setup. Topics like adding your own nodes and code would not be the priority.

Here are some incomplete notes on things to talk about so far. Please suggest more, or let me know if I've got something wrong.

Features:

• Flexible render pipeline
• Many effects unlocked by PrePass + MainPass combo.
• Light groups
• Proper toon shaders
• High quality shadows
• Screen shaders in viewport
• Line art
• Flexible Color Ramps
• Vertex shader
• Material input overrides (avoids compile times)
• A lot more data is available as inputs
• More data types available (int, bool, not just Float)
• Easy to add more stuff via plugins

Missing features (compared to Eevee.)

• No Bump mapping node (I have the code for an eevee equivalent one actually, but need to dig it up.)
• No Object Coords (but can make yourself.)
• Does not render Instances properly
• Worse viewport performance
• Can only input Attributes via vertex colors (and only 4.)
• Fairly limited set of Screen/Post effects compared to the Compositor (but you can use both.)
• Node user experience isn't as clean as eevee due to data types, no node groups, nodes showing up in contexts they can't be used, no node wrangler, etc
• Viewport delay

[rosstheblenderanimator]

Node wrangler does work in my experience. Thalia Solari shader file has node groups

[aVersionOfReality]

Ah, there's a whole node group branch that I missed, cool. Looks like its not fully complete though.
https://github.com/bnpr/Malt/releases/tag/node-groups-latest

Some of node wrangler works, like aligning nodes. But a bunch doesn't, like Viewer, which is probably the most important feature.

[pragma37]

Kolupsy added built-in support for a "Preview" feature that I think is similar to the Node Wrangler Viewer.

• Shift + Alt + Click on a node to set it as preview node (click multiple times to select the socket).
• Shift + Crtl + Click on a node to connect it to the preview socket (click multiple times to select the socket).

He also added some other goodies, like being able to search by node drop-down names, not just the node name itself (so you can search for "multiply", for example). You can also press the first letter of the drop-down name when you add a node to cycle through the drop-downs that start with that letter.

[pragma37]

It would be really nice to have some tutorials from someone who knows how to make a tutorial.
(I'm certainly not that person 😅)

Other things that may be worth mentioning:

• Layer based order independent transparency (Opaque and transparent materials follow the same path and have the same features available).
• Displacement and depth offset.
• Property overrides (preview, viewport render, and final render).
• Texture sockets.
• Custom everything support (shader nodes, render nodes, post-processing, shading models...)
• No built-in screen-space reflections/refractions/gi, probes or world lighting.
• No hair or volumes support.

Light groups

And shadow groups, self-shadow detection, and light nodes.

Proper toon shaders

And you can build your own custom shading models too.

High quality shadows

But no built-in support for soft shadows.

No Object Coords (but can make yourself)

I actually have this implemented somewhere, I should just commit it.

Does not render Instances properly

It's way more specific than that.
Regular instances (using empties) work.
"Real" GeoNodes instances work too (Object Info instances).
What doesn't work are GeoNodes "non-object" instances.
The reason is that the is_instance property doesn't work (it's always true):
https://developer.blender.org/docs/release_notes/3.0/python_api/#geometry-instancing
I have some partial workarounds, but I should just report the issue in the Blender bug tracker.

Fairly limited set of Screen/Post effects compared to the Compositor (but you can use both.)

And build your own.
With very little coding knowledge you can even adapt stuff from ShaderToy.

Worse viewport performance

I think it's worth mentioning that this is almost "by design".
One of the reasons Malt can be so flexible is that almost everything is implemented in a very straightforward unoptimized way.

no node groups

There's a node group branch, but it's been in "experimental" mode since forever.

nodes showing up in contexts they can't be used

This shouldn't be the case, do you have an example?

Also, specific features aside, I think it's worth setting the correct expectations for new users.
My vision for Malt was always to build, first and foremost, a toolkit to build your own experimental render techniques and share them with others as plugins.
So it's meant to be a platform that could grow in a decentralized way, not a regular out-of-the-box engine.
However that vision never took off since, aside from Kolupsy, I've been pretty much the only contributor.
If users approach it expecting a regular out-of-the-box experience they'll likely be disappointed.

[aVersionOfReality]

Alright, that makes sense. Thanks for the breakdown!

[Dereuz24]

I actually have this implemented somewhere, I should just commit it.

Do you mean you have a node to obtain coordinates from other objects? Would you be able to commit this?

It would be really helpful. I've been struggling thinking on how to obtain the coordinates from empties to project textures but reading through old posts I see the vertex color method and it isn't too friendly if an user wanted to project a bunch of decals on an object without an UV map.

Currently the easiest way has been to go to eevee, bake a texture with a bunch of decals and come back to malt and project it based on the current object coordinates but it breaks the flow and it isn't as modular or convenient

[Dereuz24]

Could you give us some idea of what would be involved in adding some of these missing features?

It's hard to say. First, you would have to decide HOW to support them. With real-time/rasterization everything has its pros and cons, there's not just a single route for supporting certain features.

Hair shouldn't be too hard, but I wouldn't consider it super important since you can convert it to geometry with Geo Nodes. It has to be converted to geometry anyway, so doing it on the Malt side doesn't provide a huge advantage.

Volumes is probably the trickiest one since for now Malt assumes everything is geometry. Volumes would require their own object type, follow their own render path, have their own node-graph/shader type and their own separate step in Render Layer nodes.

World lighting should be easyish, I think it could re-use screen shader nodes. Then it's only a matter of rendering that to a cubemap and providing access to it from mesh nodes.

Probes in general would be roughly the same (just run the whole render graph, but rendering it to a cubemap instead) and expose that to mesh nodes. It would also need to add the concept of Probe objects to the Malt scene and BlenderMalt, but that shouldn't be super hard either.

Screen-space tracing is tricky to implement, but it's probably not hard to find an almost ready-to-use implementation online. The real question is, what texture do you use to trace against? And when do you do it? Given the super-customizable nature of Malt, questions like that are not easy to answer. So it's possible to implement screen tracing, but it would be up to the user to decide how to integrate it.

For soft shadows there are plenty of options, but all of them suck in their own way, so not sure about this one.

Instances as I mentioned are supported, it's only a matter of Blender not doing a good job at providing access to their data for certain GeoNodes instances.

Also, is there any news on removing the frame delay in the viewport? I think it was mentioned some time ago that it might be possible to fix that. Is that the case?

I did a test some time ago and it kind of worked, but there are 2 main problems:

• Right now Blender and Malt do their stuff in parallel so while it introduces delay, the render is faster (more FPS). Disabling this would remove the delay, but would make render slower. So not a great option either.
• Blender swaps over multiple OpenGL contexts internally, and you can't share most OpenGL data between contexts, so for Malt to work correctly, it should be guaranteed that the Python code always runs with the same context active. Otherwise, shit breaks and Malt has no control over it, that's why the current approach of running on a separate process was implemented.

👀 sorry to bother you again Miguel but pinning you to see if its possible for you to commit the object coordinates

[pragma37]

Do you mean you have a node to obtain coordinates from other objects? Would you be able to commit this?

No, the patch I made was to get the equivalent of the Generated coordinates from EEVEE/Cycles, but it only works for the object being rendered.
If you want to get coordinates from other object you'll have to use drivers.

[Dereuz24]

Do you mean you have a node to obtain coordinates from other objects? Would you be able to commit this?

No, the patch I made was to get the equivalent of the Generated coordinates from EEVEE/Cycles, but it only works for the object being rendered. If you want to get coordinates from other object you'll have to use drivers.

That's also very needed. Can you put that into a release? That's the way I always mapped gradients on eevee reliably and those are very important for NPR.

I read in an old discussions that this wouldn't take displacement under consideration but that's ok for most cases. The alternative would be to manually put a box around the objects which is just more work in that case

Is there a convenient way currently to obtain generated coordinates? inline code?

[Warrenayiss]

That would be great to have a more up to date tutorial, the most recent ones are from 3 years ago

[tomByrer]

I will need to be able to present a lot of information

May I suggest - don't.
Start with a quick (3 minutes max) demo of what Malt can do currently. This will be your "elevator pitch". Then maybe just record 'building a scene in Malt' for 1 hour, then chop it up/speed up to a 20 minute demo.
These 2 vids should get enough comments on where your next priorities in tutoring will go.

I've been researching 'stylized 3d rendering' for 20 hours, & just now ran across this project. PR IMHO would be more short-term helpful. When folks cry out "Where's the tutorial?", point them here.
Or maybe by then, someone else would have their own tutorial, which could save you the time.

[aVersionOfReality]

Having some showcases and quick demos would be nice, and I intend to make those too. But they are different from what i'm talking about here. A demo shows what can be built, but it doesn't tell you much about how. And the how is what matters here. Like there's plenty of stuff that is possible to do currently with Eevee + compositor that Malt can do more efficiently. But its going to look the same in the final output.
The purpose of the video I'm talking about here isn't to introduce people new to stylized rendering to Malt, it is to introduce malt to people who already know rendering. Those people don't need to see me build a scene. They want to know what makes Malt different from their current workflows, and the pros/cons. I may also make an 'elevator pitch'. But what I'm investigating here is what is the details that come after hooking somebody.

[tomByrer]

what makes Malt different from their current workflows, and the pros/cons

OK sounds like you are talking about 2 different things here.
When I hear "how", that makes me think tutorial.
When I hear "different, pros/cons" that sounds like elevator pitch / short demo, closer to my proposal, though a bit longer.

Either way, it would be better to get fresh input from folks on YouTube/Redit first to see what is important to them. A short vid can help with this, & would at as a teaser (since you haven't made a new vid in years). Then just respond to the comments via new vids.

All the time you spent on this thread you could have created & posted a new 1-2 minute video, & got more to hear about how Malt "lets 3d artists tweak & quickly render stylized scenes" ;)

[aVersionOfReality]

I get you're trying to be helpful, but I don't think you understand what you're talking about, or what the problem is. Getting input from YouTube/Reddit isn't going to be very useful because people who don't have a big picture won't know what to ask for. And they may not engage much at all.

The problem that needs to be solved is that there is nowhere that goes into proper detail on what Malt can do and how. There's been plenty of teasers in the past, but there couldn't be any follow through. There are lots of ppl who are already interested in Malt, but can't move forward because there's no detailed information for them to follow, and they don't have time to learn it all from scratch. I'm not trying to do mass appeal to YouTube and reddit. I'm trying to create the needed content for other power users to get off the ground, and enable them to start using Malt and making content. I already have plenty of input from professionals and advanced hobbyists about what they want to know about Malt and can't find anywhere.

Also, I have videos much newer than a few years old. And I could not have made any useful sort of video in the time spent on this thread.

[Dereuz24]

I get you're trying to be helpful, but I don't think you understand what you're talking about, or what the problem is. Getting input from YouTube/Reddit isn't going to be very useful because people who don't have a big picture won't know what to ask for. And they may not engage much at all.

The problem that needs to be solved is that there is nowhere that goes into proper detail on what Malt can do and how. There's been plenty of teasers in the past, but there couldn't be any follow through. There are lots of ppl who are already interested in Malt, but can't move forward because there's no detailed information for them to follow, and they don't have time to learn it all from scratch. I'm not trying to do mass appeal to YouTube and reddit. I'm trying to create the needed content for other power users to get off the ground, and enable them to start using Malt and making content. I already have plenty of input from professionals and advanced hobbyists about what they want to know about Malt and can't find anywhere.

Also, I have videos much newer than a few years old. And I could not have made any useful sort of video in the time spent on this thread.

Going in depth is the only way. Your 9 hour video series on normals was enough for me to not get lost in an intellectual cul-de-sac trying to find solutions to things that fundamentally weren't there and couldn't be done with the system in place

Not everyone is going to appreciate the attention to detail but the videos are for people that are voluntarily looking for this information and that will help a lot more in the long run. We have enough excess of content out there with limited information and non explanatory recipes.

You have my thumbs up for a long video/video series. You are going to save a lot of man hours with this

[BlenderNPR]

Light here.

I'm currently in the process of making short tutorials (a minute or less) about Malt. These will answer the quick and basic questions about how to work with Malt. Currently I have 10 videos in production. These are easier to make and can be very agile with users' questions.
These videos also showcase the features Malt has. So interest will grow.

This is not competing with @aVersionOfReality . We both have discussed about it. We agreed that we fill different interest level for different user group.

[aVersionOfReality]

Yes, it is good especially since I have a new job and so have no idea when I'll actually be able to work on this.