Rewrite/v2.0 General Discussion Thread

[squarefeet]

Following incredibly valid points from @Usnul and @cihadturhan, in order to get this library into the state that it deserves, a rewrite would appear to be necessary. This thread is for anyone to add their 2-cents/pennies/other-currency worth of thoughts on improvements, new features, etc. The more critical, the better (IMHO) - that's one of the major ways to ensure the new version is as easy-to-use and performant as possible.

I do ask that any requests/comments should follow the same format as below: one feature per bullet-point, and grouped into either Required or Nice-to-Have (or both, if you're feeling generous).

Thanks!

---

Required

• Sprite-sheet support for both animated textures and shared texture maps.
• < 1 particle per second.
• A much more intuitive API (at the moment, there are just far too many options. These need to be grouped, or...??)
• Dynamic, and efficient, creation/destruction of emitters.
  • Choice of immediate destruction vs. stopping the emitter and waiting for existing particles to die before being removed.
  • Removal of static emitters.
• Remove the 'Group' constructor, and have only one constructor for the entire library.
  • 'Group' constructor should be replaced by internal logic that groups emitters by texture.
  • A user shouldn't have to create a group manually.
• Runtime adjustment of all parameters.
• Efficient callback/event system that allows users access to individual particle parameters on a per-frame basis.
  • This could (read: probably will) have a negative performance impact, so will have to be thoroughly thought-out.
• "Spiral" emitter type
• An implementation of 'angleAlignVelocity' that actually works when particles are not starting from vec3(0,0,0) position.
• A more intuitive way of setting options. At the moment, it's pretty crappy.
• More efficient JS and GLSL code. It's currently not quite as performant as I'd like, and I know that there are places to squeeze!
• Improved documentation and examples. Maybe a few tutorials.
• It's own website to hold the documentation and examples. I'm not sure that gh-pages is public-facing enough.

Nice-to-haves

• Gravity-well or general "gravity" support.
• Vector-field support.
• Angular velocity and/or acceleration instead of/as well as the current angleStart/Middle/End properties.
• An incredibly difficult one, this: a way to generate fairly realistic smoke. Would require a very high particle count, and either vector-field support, particles interacting with other particles, or a dynamic 'wind' system. Maybe all three and yet more still...

[squarefeet]

@movitto, @alexeld, @AdrienMorgan, @delvarworld, @stemkoski - Just tagging you guys to let you know the existence of this thread. If you have anything to add, please do. Not expecting anything, but comments/criticism of the current version would be very handy.

Cheers.

[Usnul]

hey @squarefeet thanks for creating this thread. I read SuckerPunch presentation on their particle engine for their ps4 game "infamous". The slides are at:
http://www.suckerpunch.com/images/stories/SPP_Particles_Full.pptx
they are some 500MB thanks to high-res clips, so download at your own risk. Bill Rockenbeck names 180,000 as a high number of particles to manage (what they have as a limit it seems). I know for a fact that glsl can manage a lot more than that for simple systems. The relevance of that particular engine is their simulation of smoke, they use only a handful of particles to do that. For those particles to look nice you need quite a few things though:

• randomized (noisy) behaviour
• fluid motion

but this is only for the sake of behaviour of particles themselves, another thing is appearance and there things like reaction to lights/shadows and casting of shadows is apparently a lot more important, I'm quite certain it would be pretty hard to formulate without heavily coupling with some particular rendering pipeline, so I'm just leaving it out there.

Trouble with encoding things like "spiral" or "straight line" or whatever else is an interesting motion shape someone might want directly into the engine is lack of flexibility. I believe a great engine is:

• not one where each particle can be modified and inspected individually
• is aware of noise functions
• has "language" (configuration or model or whatever else is more comfortable to use as a term) to describe motion rules

if you could supply emitter with following position function:

pos = vec3(sine(time)*r, cos(time)*r);
r = log(time);

you could essentially give people freedom to define spirals, however it would give them a lot more than that. And maybe it is a good way, to just allow injection of a snippet of code into shader, where shader includes some useful function like noise inside of it already.

Another thing in terms of implementation that i'm quite certain of - it has to remove JS out of the loop almost entirely, best option for that right now seems to use 2 buffers and swap them between simulation steps, use one for lookup and other for writing new state. I believe there are better ways potentially, but they aren't supported by what's enabled in a browser by default (without toggling flags in chrome or equivalent elsewhere).

From my personal experience - keeping things as generic and as simple as possible usually pays out in the long run.

[cihadturhan]

I agree on what @Usnul said about injectible function. An injectable functions means fully generic system. Adding a gravity would be as simple as

pos = vec3(x0, y0 + g*time*time/2, z0);

and you can switch from a cylinder to a sphere easily by changing two parameters. If you've heard about superformula you'll definetely understand what I mean. This was also an issue I felt when I was coding Hydrogen atom probablity density viewer with SPE because I had a formula for probablity distribution and I had to inject it to place points to the related coordinates but the current library didn't help me about that and I had to change many parts.

This also may lead an issue about easiness of code because a developer should find out the right formula to generate desired shape of particle system. We should consider it.

[Usnul]

I don't really see this as an issue @cihadturhan, there is an option of creating a library of these parameterizable Emitters, something along the lines of:

var BasicPhysicsEmitter = function(options){
...
var velocity = options.velocity;
addUniform({ name: "velocity", value: velocity, type: "v3"});
...
"pos = pos+velocity*time" 
...
}

let people use these as much as they like, I'd imagine it would serve large portion of users.

PS:
That hydrogen looks awesome :)

[cihadturhan]

Yep, on a second thought I think that won't be a problem.

Thanks @Usnul

[squarefeet]

This is all great stuff. I'd never thought of having custom position calculations... I'm assuming these will either be written in GLSL and injected at runtime (or, if an emitter is already running and its pos calculation is changed, it's shader's rebuilt)?

For the sake of playing devil's advocate, I have some issues with it:

1. GLSL isn't as easy to get to grips with as JS (it's userbase is much smaller). @cihadturhan - you mentioned this as well.
2. Changing the pos function during runtime could cause quite a few stutters.
3. Assuming it would be GLSL code injected into the shaders, this code would probably have to be written as strings... that feels a little clunky.

To argue against myself:

1. @Usnul: Your suggestion of a library of parameterizable emitters is a good call. That makes the issue of GLSL not being as well-known as JS a non-issue. We'd have the base library, then a bunch of general-use, pre-defined 'shape' emitters that would have their pos calculations filled in?
2. The use of multiple buffers (maybe even more than two) could alleviate this stuttering. Since we'd have fewer CPU cycles being used to calculate new pos/vel/accel/etc. values, these cycles could be used to swap buffers and re-compile shaders.
3. I can't think of another way around this problem... Chalk it up to a limitation of the platform?

I'm liking the idea of moving the functions that currently reside in SPE.utils into the shader, though. This would free up a lot of CPU cycles for the buffer swapping (which I'll probably have to quiz you quite a bit on, @Usnul!). I know neither of you explicitly stated that the SPE.utils functions would be moved to the shaders, but that does seem to be a logical extension of what you've both suggested so far...

What I would absolutely love to be able to do is get values back out of the shaders. At the moment (and please do correct me if I'm wrong) but I'm pretty sure that we can only whack values into a shader, but not get the result back into JS. WebCL should solve this problem, but I believe that is quite a way off from being included in browsers as standard.
The reason I bring this up is that, in theory, we could do all of the setting up in JS, and then use multiple shaders to do different calculations (possibly even particle interactivity, eg. boids simulation). Oh, the possibilities!

Dreaming aside, though, this is a good step forward. @cihadturhan: That hydrogen density sim is really awesome - great work!

[squarefeet]

PS. Just done a v. quick lookup on the state of WebCL. The 1.0 spec was released in March this year, but Firefox, at least, isn't going to implement it. They seem to be erring on the side of OpenGL ES 3.1's compute shaders instead. I haven't yet found a timeline for ES 3.1's adoption.

[Usnul]

Hey @squarefeet, I like your reasoning. Let me try and address some issues:

• 2 buffers are to go around the issue of reading and writing at the same time into the same buffer, and the problem is - it causes glitches which were discussed at lengths throughout github on various particle-related projects. The clear consensus is to use one buffer for reading values and one for writing, thus eliminating race conditions that otherwise mess with your mojo.
• buffers store particle state, such as position, velocity and whatever else, I imagine there would need to be a provision for injecting arbitrary variables from users into the chain, as little beyond color and position are "required".
• since particle state is stored in a buffer and we know which one will be used for lookup and which for writing - we can extract current state for individual particle quite easily between render cycles and similarly inject new state information. I would imagine this pipeline completely in JS which would make manipulation of individual particles somewhat slow, but at the same time possible and wouldn't require much memory at all, since manipulations would be done directly on buffers.
• shader re-compilation. First of all, as weird as it sounds, graphics pipeline accepts strings, so it's not that strange to be working at that level. Second issue is frequency of recompilation and performance associated with that. I can not envision single emitter behaviour being manipulated too often, I'd imagine its code to stay he same, but maybe uniforms to change between frames, a usecase which would require recompiling of shader every frame is probably a bad fit for shaders in the first place. As for recompilation performance, there is a demo from 2010 by Evan Wallace on path tracing, where he does exactly that - recompile the shader at high frequency, and the results are rather encouraging showing no stutter whatsoever.
  http://madebyevan.com/webgl-path-tracing/

I could see every emitter class (not instance) use a separate shader, but at the same time - even hundreds wouldn't present much of a challenge, when thinking about games today that employ thousands of shaders at the same time.

[Usnul]

here's a promising thread from three.js on the similar concept of using double buffering:
mrdoob/three.js#1183

[cihadturhan]

A couple of things,

• I think, we need to implement gl_Vertex attribute to store the coordinates of each vertex in gpu. As we discussed on vector fields issue (particle vector fields #37), we had a problem because gpu doesn't remember the previous changed position of the vertices. The problem was, it jumps the previous position after it leaves vector field because it just knows the first value defined. If we use gl_Vertex, gpu will remember the position and make computation according to current coordinate and that won't be a problem. gl_Vertex is a defined attribute in OpenGL already, but not WebGL. Therefore, we will implement this if we believe this is utilized well on vector fields or some other stuff. Also, in lightgl library, they implemented gl_Vertex somehow and they are using it in their library as well.
• on shader re-compliation, there is an existing library which recomplies shader immediately. There is also a demo where you can change shader properties and recompile on the runtime quickly.

[Usnul]

@cihadturhan I'm a little confused on this point. if we use an FBO - then vertex shader is pretty much excluded, you have a quad and that's it, each fragment however represents a particle, and looks up position of that particle from a texture (FBO). Having UV (0.5,1) for instance, we'd go to texture and lookup pixel at at that location, then we'd unpack state from it, including position of the particle. In a way UV only serves as ID of that particle.
GLSL is quite new to me, and a lot of it i still don't understand, for instance I'm not sure how then this data can be used to place sprites or how it can be shared with things like THREE.js. On the other hand, if we have a vertex shader responsible for particle simulation, I'm not sure how FBO lookups would work, and how these vertices are going to be linked to specific texels.

[cihadturhan]

@Usnul I understand how you want to use FBO but I have no knowledge on how FBO works in deep. Will the coordinates change in the next frame like in gl_Vertex? This is what I don't know and I wonder much. If it always stores the new one then it's perfectly fine to use FBO.

[Usnul]

@cihadturhan here's a basic rundown:

set fbo1 as texture for lookup
set fbo2 as render target (where we draw to)
begin render
...
render done
swap fbo1 and fbo2 so that next time 1 is used for render target and 2 for lookup
rinse and repeat

okay, that's great, and here's what's happening inside fragment shader:

get pixel from texture based on U and V coordinates
do clever stuff with pixel data
set gl_FragColor with new pixel data so that it gets written to render target U V coordinates

the proviso here is that texture for lookup and render targets are identical in structure, this is why you can quite easily identify where to write, and why you can swap them too.

As you can see, particle state would correspond to a specific pixel, so it is preserved between render cycles. Inside the shader you have access to that state and also have the opportunity to use that "old" state when creating new one. A simple static shader, for instance, could just copy old state so that new state remains the same as the old one.

[cihadturhan]

@Usnul Thank you very much for explaining, I'm new to texture stuff. I see what you want to do and it's very clever 👍

If the number of particles are n * m, then we'll use all of the pixels :)

PS: we can assign m = 1 everytime so that's not a problem though.

[movitto]

@squarefeet @Usnul this all seems reasonable / cool. Am also relatively new to GLSL so can't comment too much but agree it would be awesome to parameterize the shader w/ custom algorithms (if it's feasible / practical). Also agree on addressing the performance implications, perhaps in addition to the solutions presented above, we could support multiple modes w/ different shaders optimized for different scenarios.

In general would be good to try to do this as a pluggable architecture, where at the base we present simple classes / interface able to be extended/used for custom scenarios. Would be glad to help where/when I can.

[squarefeet]

@Usnul The FBO stuff does certainly seem to overcome our problem with particle state between render cycles - great call to use that. From my extremely limited understanding, though, wouldn't we only have 4 values per pixel to describe a particle's state (R/G/B/A)? I think I'm just missing the connection between how I describe a particle's state in my mind (pos, color, velocity, acceleration, opacity, angle), and how the shader would describe the particle's state using an FBO. Though I guess we would only really need to store the position? Maybe, if necessary, we could use more than one texel to describe a vertex...

As far as linking a particle/vertex to a specific texel co-ordinate, could we not just have a FloatArray that stores UV co-ords? Obv. two entries per vertex ([u,v,u,v, etc.]). Have that as a uniform, and store the start position in the FloatArray for each vertex as an attribute:

uniforms: {
    texels: new Float32Array(...)
}

attributes: {
    coords: { type: 'f', value: [...]
}

I know I'm probably bringing the discussion backwards a little here, but I just want to be sure I fully understand your proposal :)

---

@movitto Great to have you on board. I think it's more than feasible to have custom shader algorithms - as @Usnul mentioned, shader re-compilation doesn't seem to pose as much of a problem as we first thought. That, and the changing of shader algorithm(s) during runtime would probably be a small use-case anyway.

The idea of different shaders for different scenarios is a good idea - I reckon for ease of development, though (and for DRY purposes), it would be prudent to make our shader code as modular as possible; broken up into small-ish chunks that we could just tie together to make new shaders as and when necessary ( similar to what THREE.js does ). That said, I think that's what you might've meant when you talked about the 'pluggable architecture'!

[squarefeet]

Okay, been doing a bit of reading into the FBO stuff, and @Usnul - I followed your link to the double-buffering thread in the THREE.js repo. It helped quite a lot, thanks.

About the 4 values p/pixel issue: looking at this example of GPGPU boids/flocking (which is just stunning, btw) from here, it appears that @zz85 is using multiple textures per "particle" (one for position, another for velocity, etc.) Following this pattern would solve the issue of only 4 values p/pixel.

His SimulationRenderer.js file would be a good inspiration for the FBO-swapping...

[squarefeet]

Just got a quick practicality question:

• Where would be best for this rewrite to be hosted? I'm not keen on it being in a separate branch of this repo as it wouldn't make too much sense, but that said, nor am I keen on it being in a repo of its own.

What would you guys prefer? At this stage, it'd just be sketches, tests, etc.

[Usnul]

I'd say a SPE mark.2 would make sense :)
As a possibility we could bind that engine to this one's API as well, though not as a primary API target I guess. But yeah - i see it as a clean slate which shares common goals with this project, but is, as you said - a re-write as opposed to an alteration.

[squarefeet]

So I've been getting my head around GPGPU/FBO/render-to-texture stuff, and I've re-written the THREE.js gpgpu birds example. It sounds a little strange to have done that but it gave me a good way to learn FBO stuff, as well as make a base "class" that we can use in the next version of the particle engine.

I've submitted this as a possible replacement for the existing SimulationRenderer for THREE.js here. Hopefully I'll get some useful feedback either here or on that thread and it can move forward into a useful little "class".

In short, this version of the Simulation Renderer takes care of generating data textures, swapping renderer targets / buffers, and automatically passing generated textures into shaders. It's fairly straightforward to use, so I haven't added many comments, but let me know if it's not as straightforward as I think it is and I'll comment the crap out of it :)

@Usnul: Once this SimulationRenderer is declared usable/useful, then I'll start up the SPE v2.0 repo.

[squarefeet]

I'm quite comfortable with FBO stuff now, so I'm starting to think about how we should organise what textures hold what data.

To start with, lets keep things simple and have the following per-particle attributes:

• Position (RGBA),
• Velocity (RGB),
• Acceleration (RGB),
• Opacity (?),
• Max age (?)

We could have one texture for each of these, but I'm wondering if we could pack some of them together. Like using the unused w component of the RGBA Position texture as the opacity value, etc. Or maybe even having the position and velocity textures combined, using an offset parameter to determine whether a pixel of the texture is a position value or a velocity value - Soulwire did something similar here.

We could also work around having one render-to-texture pass for each of the textures as well - maybe by using the velocity calculations from the existing SPE shaders that extrapolate velocity values based on particle age. These velocity calculations could be integrated into the position texture shader directly, so that's one less render pass to do. This may cause problems later on when we come to add vector fields and stuff like that, though.

So... Issues that need discussing:

• What data should textures hold?
• Can we combine two or more data sets into one texture?
• How much of a particle's current position can be extrapolated without using FBOs and without affecting the ability to add vector fields (etc.) at a later date?

[Usnul]

could make a mapping mechanism for variables, something along the lines of:
int -> ...
float -> ...
v3 -> rgb
v4 -> rgba
let the mapping mechanism pack/unpack several floats into a single rgb/rgba etc as you said. Keep track of what FBOs we have and what fields in those aren't used yet. I'd say making it generic from the start would solve a few problems in the future.

[squarefeet]

I've been playing around with a few things tonight and wanted to get feedback on a possible "core" API. I'm not going to paste any of the internals I have going at the moment, because they're an absolute mess, but as far as using the thing goes, this is what I have:

var myParticles = new Particles();
myParticles.addProperty( 'acceleration', new THREE.Vector3( 0, 1, 0 ) );
myParticles.addCalculation( 'velocity', 'velocity += acceleration * delta' );
myParticles.addCalculation( 'position', 'position += velocity * delta' );

• Where the addProperty() method takes the following arguments: property name, property value
• Where the addCalculation() method takes the following arguments: property name, glsl calculation string.

For each addCalculation() call made, an FBO object is set up (double-buffered).

Note that I haven't defined any initial values for velocity or position. These default to vec3(0,0,0) if not defined.

Also, for each addCalculation() and each addProperty() call, a uniform is added to each of the FBOs created, so any of the properties can be accessed from any of the shaders that get created.

Oh, and one more thing, I haven't done anything re. emitter groups. Just this core API work.

Thoughts? I'm thinking that whilst this gives us the most freedom we need (particularly taking care of custom position calculations), it does seem to possibly be at the cost of usability.

[Usnul]

that looks pretty good @squarefeet. Regarding groups, i suggest using a hash of sprite URLs:

var sprites = {};
function Particles(url){
  var texture;
  if(sprites.hasOwnProperty(url)){
   texture = sprites[url];
  }else{
   texture = THREE.TextureUtils.load(url);
   sprites[url] = texture;
  }
  ...
}

also, regarding syntax: i do think it's verbose, but there are ways of reducing that. One way would be to write a glsl parser which would be a small investment and would probably be done later in the project. Another way would be to do something like this:

myParticles.properties.add('acceleration',new THREE.Vector3(0,1,0));
myParticles.calculations.add('velocity += acceleration * delta')
     .add('position += velocity * delta');

parsing this would be relatively easy, as all you'd have to do is tokenize it and exclude reserved tokens like float, int etc. and only keep ID tokens ([a-zA-Z_][a-zA-Z0-9_]*)

[squarefeet]

I'm not sure why we'd need a GLSL parser?

EDIT: Ah, wait. I think I understand why now... you've removed the first argument for the calculation.add() function. We'd need to parse the only argument you have given in order to find the property name that needs calculating?

[Usnul]

yup :)
besides that, not all operations might end up as assignments, and some "variables" may only need to be temporary

[Usnul]

glsl parser project in JS:
https://github.com/chrisdickinson/glsl-parser

[squarefeet]

I've taken a look through this, and it's absolutely superb, @Usnul. Exactly what I was looking to do. I reckon it'll serve as a good base for sorting out FBOs and assignments :)

I want to have a good ol' tinker with it, and see if I can build up a double-buffered render chain from it. Once that's done, I'll start trying to work it into an emitter to see how it holds up. I'll open up a new repo for all of this v. soon.

Thanks!

[squarefeet]

Hey guys,

So... it's been a long, long while since the last message in this thread, so quite a lot to update you all on:

• Apologies for lack of updates between then and now. Too many things and not enough time...
• I've almost finished writing an alpha version of v2.0. It's up on BitBucket under a private repo at the moment ('cos I'm a cheapskate and haven't paid Github for private repos). I've kept it private for the moment because it's a bit of a mess, and I'm changing core things far too often.
• If you want to see the progress, here's a video I made. It isn't quite running at 60fps in that video because of the screen-recording software taking up GPU cycles. Runs fine when not recording.

About the alpha version:

• I've inadvertently seemed to have based it off UE4's Cascade (see the Modules section here)
• So far I have the following modules written:
  • Acceleration,
  • Attraction (Point gravity and particle-to-particle attraction)
  • Color
  • Size
  • Curl Noise (as you can see in that video)
  • Velocity
  • Position
  • Spawn
• I'm working on the following modules:
  • Kill box/sphere
  • Particle rotation
  • Emitter rotation
  • Collision with the depth buffer (this one is proving rather tricky, but should be attainable)
  • Vector fields
  • Sub-UV (sprite-sheets and sprite-sheet animation)
  • Custom calculations

I would love to try to get particle lights implemented but I fear I'd have to crack open Three.js's internals to do that, which would be beyond the scope of the project. Having the particles react to a scene's lights might be do-able, though... Must look into that. Talking of which, I'd like to get shadows implemented, but that brings with it a whole new issue of particle sorting, which is a very expensive operation. I've experimented with radix sort on the GPU, but it's not smooth enough for my tastes (it sorts them over multiple frames).

There are also multiple ways of setting most parameters. For example, you can set the velocity value to be a THREE.Vector3, or a set of THREE.Vector3s that will be linearly interpolated through over the age of the particle, or as a random value between minimum and maximum bounds. I'm working on getting parameters to support sets of values that will change over the lifetime of the emitter as well as the lifetime of a particle. These value options replace all of the *Start, *Middle and *End properties in the current version. It's so much nicer to work with (IMHO).

So yeah, it's a very modular system. Create an emitter, then create as many modules as you want (each with various parameters) and add them to the emitter. Pretty much it. There's currently no grouping, though, and I'm not sure whether this is a viable with the architecture I've made so far.

With all that said..!
I'm afraid I'm re-writing it again (only the 4th re-write!) The way it's set up at the moment is that almost all of the modules have their own textures, and shaders. The more modules one has, the more render passes and buffer swapping, which isn't good for performance. I'm still planning on keeping the modularity, but I'm heavily re-working the shader code. As it stands at the moment, changing parameters on-the-fly isn't really cost-effective.

At the moment, I'm running only four passes regardless of how many modules are enabled (spawn, forces [vel., accel., noise, basically anything that affects velocity], color, and position/draw), and things are speeding up rather nicely. It also helps that I'm pushing less data to/from the GPU.

As far as custom calculations go, that's going to be added quite near the end of this re-write but it's at the front of my mind whilst I'm writing the rest of it.

With everything running on the GPU and JS only really doing the buffer swapping, there's not really a way to get per-frame access to a particular particle's properties. Luckily, in this re-write most of the parameters are dynamic, with only the one texture holding both velocity and position data. So things can still be changed on a per-frame basis without repopulating the texture buffer (which is an exponentially expensive operation).

I think that pretty much covers most things... hit me with any questions. It might be hard without seeing any code, but if you want examples of it's current state, just let me know and I'll do some copypasta.

I'll whack it up here under a new repo once I'm happy with it and it's stable enough for people to poke around and experiment with.

PS. Thanks @Usnul for pointing me in the direction of textures/FBO/GP-GPU. It led me down a wild garden path of joyous discovery!

[squarefeet]

Just because pictures can say more than words, here's some screenshots of the progress so far:

[cihadturhan]

Wow, you've made huge amount of work. That's awesome. I'm looking forward to see the demos live. Is this blue one called curl noise? I'm on mobile now, didn't watch the video.

By the way, if you're studying you can request github students pack which includes free github subscription for a year.

[Usnul]

Hey @squarefeet , that's amazing progress. Would be interesting to have a look at the code. You mentioned "per-particle" access, i'm actually not too worries about that since you can always use canvas operations to from JS side to work with pixels in a buffer, or if it's a binary array - things are even simpler. Would be silly to manipulate all particles this way and defeat the purpose of GPU engine, but for some small pools of say 200 particles or so it could be a nice addition.

Regarding particle lighting, there's an interesting introductory set of slides which you probably have seen already:
http://www.roxlu.com/downloads/scholar/008.rendering.practical_particle_lighting.pdf

I've been reading around particle engines for a while now, and i'm quite convinced now that integration and dynamic properties of particles are more important than count overall. By that I mean aspects which make particles "feel" more like they are a part of the world, here are a few examples:

• receiving light
• casting shadow
• casting light
• integration with geometry (collision detection)
• sprite blending (blog.wolfire.com/2010/04/Soft-Particles)
• self-shadowing (AO)
• cube maps (this is really part of the the first point about receiving light)

I've seen many examples where only a handful of particles (>500) were used to achieve visually amazing effects. That's not to say that really high particle count is a great thing on its own :)

One more piece to consider is screen-space, some effects such as rain or snow are often done in screen space, because you can get higher density in 2d with same count than you can in 3d.

Lastly, regarding cracking open three.js - it's remarkably nicer than it was several months ago, shaders for all materials are now composed out of fragments (chunks/snippets), so you might be able to create a custom material by manipulating order of these chunks and introducing your own.

At the end of the day, i'd personally be more in favour of having strong support for Three.js, but no core integration, allowing particle engine to be used with different rendering engines. Babylon is getting a lot of traction, and there are many projects which go beyond stock three.js.

[squarefeet]

Hey guys,
Thanks for the kind words :)

@cihadturhan: Yep, that's curl noise. In both of the screenshots, actually. I'm past my studying days - I just take free things where available ;)

@Usnul: I'm totally with you re. appearance of particles more important than the count. Sure, higher counts are more impressive initially but really of little real-world value (unless of course, you're building a tech demo!)

I think once I've got the new shader up and running (I've called it BigDaddyShader... it does velocity, position, colour, and as-yet-undefined "other" all in one pass, yay) and functioning with the modules from the previous rewrite, I'm definitely going to start tackling the task of using depth buffers to do various things.
First of my list will probably be collision, as that will teach me how to use the depth buffer in quite an in-depth (no pun intended) fashion. I'll also start testing the limits of THREE.js, as it might be that shadows and/or lighting are already supported for particles. The former is quite possible, the latter not so, if my understanding of THREE's source is anything to go by.

I'm definitely keen on getting soft particles introduced. Maybe that should be my first port of call instead of collision...I never make things easy for myself!

You're right about THREE's internals, btw. I nicked the idea of using shader "chunks" to build shaders, and I daresay I'll be able to bastardise their lighting shaders to whack in my draw shader. Maybe even the shadow stuff.

I think when it comes down to this engine being used as a part of a bigger system (games, environments, etc., rather than nerdy simulations), the sorting issue won't be too much to worry about, as the particle count will be fairly low. At least I hope so.

About this library supporting not just THREE: Definitely in favour of that. I'll need to get my "native" WebGL chops up to scratch, but it's something I've definitely considered and researched already. I'm not too sure about Babylon, but I daresay if it's anything like THREE re. ease-of-use, then it shouldn't be a problem. The "native" WebGL version will be the trickiest IMHO.

[Usnul]

Using chunks is the most primitive way to generate code, nothing wrong with primitive, but there is a lot more to code generation. Next step is probably templating, beyond that is tree transform (aka parse-tree -> AST -> transform -> transform -> generate code from AST)

You could use point cloud from threejs, but you would be bound to a vertex buffer then, which i assume isn't a good fit for your engine. Three JS works with shadows for geometry, so i doubt that you'd be able to make it work without changes. Good news is that shadow maps are pretty easy to code - it's just a B/W projection.

Collisions. Bullet physics engine has recently been updated (about a year ago now actually) to do solving on GPU, but i'm not sure what kind of shaders they use - probably geometry or compute, still might be a useful source of info as it's open source. I wouldn't go beyond planes, cubes, spheres and maybe heightmaps for collision solving. I'm still certain that it would/will be a pain to do this with just the vertex and fragment shaders.

[squarefeet]

Ha, I don't know whether going down the AST would is completely necessary, here; pretty sure chunking it up will suffice!

I'm currently using the PointCloud... I'm not sure why being bound to a vertex buffer is a bad thing? Not sure of the alternative..? Unless you're thinking that shadows are more easily calculated/cast using faces?

I'll definitely dig into the bullet engine and see how they're doing the depth collision. I'm also digging into UE4's internals to discover how they do their depth collision... hopefully I can get enough pointers. Failing that, this page has a very good tutorial. Just missing the bit about projecting a particle's position into clip-space (damn).

Also, just because it's kinda pretty, here's a screenshot of the BigDaddyShader (I really need to rename that...) in action.
The bottom square is the spawn texture (particle age is in the x, or red, channel, so that's what's causing the fade. Inactive particles are blue, active ones are slowly fading from that same blue to white.
The square above it is the BigDaddyShader's texture. It's 4x the size of the spawn texture, as there are 4 pixels per particle (velocity, position, color, and other). The velocity is currently being affected by curl noise (I'm addicted to that stuff):

Next up:

• Make the messy new Emitter constructor pretty and more efficient
• Add the remaining modules from the previous rewrite
• Add "over-lifetime" and random values for parameters
• Consider breaking the BigDaddyShader (and its textures) into smaller chunks if total size of BigDaddyTexture > MAX_TEXTURE_SIZE
• Once this version has caught up with the previous rewrite's state, crack on with collision & depth stuff (soft particles, etc.)

[Usnul]

having vertex buffer is actually great, I think. The projection itself is somewhat easy, but for light you will need a normal as well as position, otherwise it's back to billboards.

MAX_TEXTURE_SIZE is important to consider
BigDaddyShader is a good name, as good as any other, i'd drop "Shader" part for brevity just "BigDaddy" - i'd be proud to use something with a name like that :)
Can't wait to have a look at the code, everything so far looks very enticing.

[eldog]

Very useful library, looking forward to the new features and flexibility.

Just want to vouch for updating the editor with these changes too, as it is a really valuable tool for getting the right look for an effect (and made me decide that this was the library to use).

[squarefeet]

@Usnul "BigDaddy" is just the internal filename for the shader... I don't think that's what I'm going to call the final version! Maybe as a nickname... ;)

@eldog Thanks, I wasn't aware anyone was using the Editor! That has reminded me to push it to the master branch... Completely forgot to do that. I think there are some issues around exporting and whatnot, though..? There definitely will be an Editor for v2, but it will come once v2 is stable. I'm also going to get a site up for v2 specifically (that's not gh-pages), with some tutorials, maybe some videos, full and proper documentation, and the editor. Lots to do!

[squarefeet]

Got stuck doing particle collision with the depth buffer... Posted a question on stack overflow and linking it here in case anyone has any ideas!

https://stackoverflow.com/questions/26576429/projecting-fbo-value-to-screen-space-to-read-from-depth-texture

[Usnul]

i have a suspicion that you're not taking clipping planes into account (far, near). I'm not sure what space you use to resolve collisions, but if you are using screen-space - you would have to first project position of a particle into screen space coordinate system, otherwise you would have to apply inverse of that transform to depth buffer. If you are using the depth buffer - you'll have to get normals for texels as well, some SSAO algorithms implement inference of normals from depth buffer and can help with that.

[squarefeet]

I've tried taking clipping planes into account, in fact I use that already in the draw shader: http://jsfiddle.net/1dchvbge/1/ (see the HTML section, and scroll down until you see the draw shader section)

The problem I have is that I can't seem to project the particle position into screen space correctly. I need to use the particle's screenspace xy as a UV coord for the depth texture... I've read quite a lot about projecting to screenspace, but nothing seems to work. So frustrating!

Once I manage to find out how to (correctly) project into screenspace without using gl_FragCoord, then the rest of it will be nice and easy... Famous last words, hey?

[squarefeet]

Ah-ha! Figured it out. I've added an answer to my own question (!) on my SO thread:
https://stackoverflow.com/questions/26576429/projecting-fbo-value-to-screen-space-to-read-from-depth-texture/26609097#26609097

New fiddle showing it in action is here:
http://jsfiddle.net/30rtw4b1/1/

[squarefeet]

I can't be bothered to put up a fiddle for this just yet, so here's a video I made of collisions with (probably buggy) collision response:

https://vimeo.com/110275317

Vimeo's processing it at the moment, so should be ready in about 30mins. Will post a fiddle link when I get around to it! Was recorded on my Macbook Air, so performance isn't great when capturing the screen.

[squarefeet]

Okay, here's a fiddle: http://jsfiddle.net/vhw6o7ef/1/
Let me know if it doesn't render properly for you (compare it to the video).

[Usnul]

yup, looks the same. Looks awesome actually :)

[Usnul]

have you seen this?
http://david.li/disintegration/
light/shadow related

[squarefeet]

I haven't... super impressive stuff. I came across his Flow stuff before, though. It's interesting how he does an incremental odd-even merge sort for the particles... Must look into that!

[Usnul]

There are also webworkers to assist with sorting, even mobile devices have more than 1 core. Allowing sorting to be incomplete could produce visual artifacts, but with large number of particles this may be an acceptable trade-off, considering you'd only need this when number of particles is large :)

[squarefeet]

Ah-ha, good call... I'll take a look at how best that could be implemented :)

[squarefeet]

Okie dokie, here's a quick update on the (latest) rewrite:

• I currently have 5 shader passes: spawn, velocity, color, position, and draw.
  • This is a slight step backwards from using the BigDaddy shader that does the first four of those passes in one, but for now I believe this is the way to go. It mitigates issues when having large particle counts that create a texture that's larger than gl.MAX_TEXTURE_SIZE.
  • I do have a plan to use the BigDaddy shader when particle counts are < sqrt( gl.MAX_TEXTURE_SIZE ), but this hasn't been written yet. It'll need to dynamically assign shaders and that bit isn't done yet.
  • If, for whatever reason, I find the BigDaddy shader isn't a good solution to the problem of multiple shader passes, I'll still use the same approach to combine, say, the position pass and the velocity pass. This might be dynamically done, but I'm not sure whether to expose this "combining" as an option or not. If I do, it might make things needlessly complicated for the sake of a customisation that doesn't really have much affect in the vast majority of cases.
• I've been experimenting with a few API ideas, and settled on one. The main one I didn't settle on was this:

// Create emitter
var emitter = new Emitter();

// Set acceleration value. Behind the scenes, this sets 
// define values in the shader to 'activate' the acceleration 
// module. 
emitter.acceleration = new THREE.Vector3( 1, 2, 3 );

• The biggest problem with this approach is that since I was using Object.defineProperty and it's get/set functions, it got complicated very quickly when a module (Curl Noise, for example) would require additional define values to be set. TL;DR: Pain to maintain.
• The API that I've settled on is very much like the one from the rewrite before this one:

// Create emitter
var emitter = new Emitter();

// Add a module.
// This style allows for much easier customisation. Custom emitters 
// with custom GLSL calculations are SO easy, now!
emitter.addModule( new AccelerationModule( {
    value: new THREE.Vector3( 1, 2, 3 )
} );

// Repeat adding various modules until emitter is complete.
// Unlike the rewrite before this, no need to call `emitter.prepare();` 
// after all modules have been added.

• This approach allows for hugely customisable shaders. All GLSL uniform strings are automatically created, and wrapped in #ifdef statements to allow modules to be easily turned on and off at will with very minor performance hits.

---

So, I'm pretty sure I've hit a little bit of a jackpot re. combining the best bits from previous rewrites. I've just got to stabilise the emitter code, then tackle a few important internal issues:

• How best to "group" emitters? The more I think about it, the less I'm keen on having grouping happen automatically. If it happened automatically, how would new THREE.PointCloud instances be added to the scene? One could pass in the scene as an argument to an Emitter instance, but that feels clunky and a bit silly... I'm more keen on having grouping happen manually. Much easier control for the end user, that way.
• How best to determine when shader "combining" should happen?
• Should I use a THREE.WebGLRenderTargetCube to render 6 different scene depths in order to do much better collision detection? Failing that, would a camera looking in the opposite direction to the scene's camera and translated on the inverse z-axis (rendering just the 'back' of the scene) be enough?
• Since there are issues using the spawn shader when either spawnRate or maxAge is < 1.0, I'm considering swapping to CPU spawn calculation for this. This swap will be automatic, but will require using attributes to determine whether a particle is alive or not within the other shaders. This will need to be written, but can easily be wrapped in another #ifdef to allow state-swapping to easily occur. Also considering having a user-controllable option to force CPU spawn calculation.

---

Once I've rewritten the remaining modules into the new format, I'll definitely whack this up on Github, I promise! As ever, any questions, comments, or criticisms welcome :)

[Usnul]

@squarefeet

emitter.addModule( new AccelerationModule( {
     value: new THREE.Vector3( 1, 2, 3 )
} );

I like this, "module" feels a bit off though. If you already have the whole shader compiled and only defs are being manipulated, maybe consider using a flag instead:

emitter.accelerationEnabled = false

just a thought though.

How best to "group" emitters? The more I think about it, the less I'm keen on having grouping happen automatically. If it happened automatically, how would new THREE.PointCloud instances be added to the scene? One could pass in the scene as an argument to an Emitter instance, but that feels clunky and a bit silly... I'm more keen on having grouping happen manually. Much easier control for the end user, that way.

• using a group for multiple emitters can allow for fewer draw calls (less instances of geometry), if we know span rate and we know maxAge - it's pretty easy to determine maximum pool size (i.e. how many vertices we need for geometry) Other than using grouping to get better performance for multiple emitters using same particle material - there's a possible place to group various different emitters, like say for fire effect you could have "smoke","flame" and "embers" emitters, being co-located. Just throwing ideas out there.

How best to determine when shader "combining" should happen?

• i'm not sure what this means. Could you elaborate?

Should I use a THREE.WebGLRenderTargetCube to render 6 different scene depths in order to do much better collision detection? Failing that, would a camera looking in the opposite direction to the scene's camera and translated on the inverse z-axis (rendering just the 'back' of the scene) be enough?

• i'd say "no", short of doing world-space simulation, you will have error always, using already existing depth buffer is 0 overhead, and thus a pure win, adding 6 more draws even just for depth is quite considerable. It could be a nice "addon", but you'd be making a very significant performance trade, and sometimes it can be justified.

Since there are issues using the spawn shader when either spawnRate or maxAge is < 1.0, I'm considering swapping to CPU spawn calculation for this. This swap will be automatic, but will require using attributes to determine whether a particle is alive or not within the other shaders. This will need to be written, but can easily be wrapped in another #ifdef to allow state-swapping to easily occur. Also considering having a user-controllable option to force CPU spawn calculation.

• what storage type is used for these? I think it's fair enough, but the problem is probably in floating point rounding error somewhere.

[squarefeet]

I like this, "module" feels a bit off though. If you already have the whole shader compiled and only defs are being manipulated, maybe consider using a flag instead.

I hear that, but with the module style, it will allow for custom modules to be created really easily... Once it's up on Github and you can see the Module code, I daresay you might agree :)
I do see the issue about enabling/disabling modules, though: I seem to have added that functionality without properly thinking about how it would be exposed. The flag idea seems to be a fairly good solution. I'll experiment with it and see how it feels.

...say for fire effect you could have "smoke","flame" and "embers" emitters, being co-located. Just throwing ideas out there.

Good shout. I'll think a bit more about grouping, as I daresay it will affect some of the internals quite a lot. Something to do as soon as the Module stuff is fully supported by the Emitter.

How best to determine when shader "combining" should happen?
By this, I was thinking aloud re. whether to do, and if so when, the shader 'combining' should happen, i.e. combining the position and velocity shaders, or using the BigDaddy shader. I was just wondering what emitter 'states' should trigger what shaders to be used (i.e. small particle counts: use BigDaddy; larger counts: use combined shaders, huge counts: separate shaders.)

About the spawn rate issue: It'll be a lot easier to explain once the code is publicly accessible. Quite hard to explain otherwise without a huge wall of text. Needless to say, it's not a floating-point precision issue!

[Usnul]

@squarefeet
I'be happy to read through ha wall of text, if it helps solve the problem :)

[squarefeet]

@Usnul Ha, braver than I! I'll try and put it up soon for you to peruse.

In the meantime, here's something else that would be good to get your opinion on:

Two constructors, both alike in dignity, in fair JavaScript where we set our scene... One constructor, the ShaderPass creates shader passes (i.e. for velocity, position, etc.), the other a Module that affects the behaviour of a particular ShaderPass (i.e. to add CurlNoise, or acceleration to a velocity pass).

A question about the implementation... which do you think is more logical?

var emitter = new Emitter();

emitter add( new ShaderPass( { 
    name: 'velocity',
    // etc...
} ) );

emitter.add( new Module( {
    name: 'acceleration',
    value: new THREE.Vector3( 1, 2, 3 ),
    affectsShaderPass: 'velocity' // Assign this module to the ShaderPass created above
} ) );

Or:

var emitter = new Emitter(),
    pass = new ShaderPass( { 
        name: 'Velocity', 
        // etc...
    } ),
    module = new Module( {
        name: 'acceleration',
        value: new THREE.Vector3( 1, 2, 3 )
    } );

pass.addModule( module );
emitter.addPass( pass );

EDIT: Quick clarification: As far as the end user goes, most people won't have to create their own ShaderPass instances or Modules. There will be predefined ones in the library (like 'VelocityPass', CurlNoise module, etc.). This is mainly for custom stuff, but these constructors will also be used to create the predefined modules/passes.

[Usnul]

@squarefeet Seems like a "has a" relationship, maybe i'm just not getting something. Module seems to need a pass or many passes, then module should be responsible for a pass, no? And thanks for edit, for common use-cases this does seems like a beast of an API :D

[squarefeet]

Yeah, that's what I was thinking as well - the 2nd option "feels" correct to me. Long story short, it goes like this:

An Emitter needs ShaderPass instances (creates the main shader code, renderTargets, dataTextures, etc.. Also can have a basic calculation string (e.g. value.xyz *= 0.8;) if necessary) in order to actually do something. Some of these ShaderPass instances will be automatically added to an Emitter instance, such as the spawn shader.

A ShaderPass can have custom calculations added to it by adding Module instances to a ShaderPass. Module instances can contain calculations to adjust a ShaderPass's value, such as a CurlNoise module, or a Drag module.

Also, if a ShaderPass or a Module instance needs to read from another ShaderPass's output (for example, a velocity pass might need to read from the position pass), uniforms and texture sampler GLSL code is automatically added.

And yes... it's a dirty ol' beast of an API, but that's the way I like it :D At least it'll make adding new modules/passes nice and easy in the future. Expansions made easy!

[squarefeet]

Just had another thought: using this API, you could theoretically create a ShaderPass that would render a diffuse texture that you could use as the texture for the particles... Oh god. What have I created!? :D