Standardize an HTTP Server as a Transform

[formula1]

This is a low priority enhancement. But I do believe this is important in general to set a standard. That being said, the http server spans about... 8 different files with a collective 2661 lines (120+145+498+624+198+228+572+276) (some of which is repeated such as http and https). So I completely understand that this idea is going to be taken lightly if not ignored completely. However, seeing from the last TC meeting, you guys really are paying attention. And it warms my icy cold heart.

The problem

Nodejs already set the standard for using stream.Transform in servers, don't.

It can only be found in Crypto and zlib within the source

However, the nodejs implementers guide suggests using the transform for protocols. I'm not here to complain about hipocracy as much as I'm here to complain about how because node didn't lead by example others didn't follow and understand why it wasn't implemented that way.

Consequences

• tutorials don't teach using transforms
• The WebSocket Server almost all other WebSocket based servers require does not implement it
• A Proof of concept Stun Server also does not implement it
• A common multipart data parser doesn't use a transform
• Inability to pass these types of socket wrappers and others between processes
• people not seeing Transforms useful

Other solutions

• List all transforms that would be nice to have. This will give a hobby for anyone interested in it especially since almost every protocol is specified by w3 (I'm actually slightly interested after my recent work) (Also the most realistic)
• Specify that transformations for protocols aren't necessary and hardly ever used including within yours/nodes own framework so people don't believe whats on nodejs.org (That is kindof a joke)
• Giving the ability to send circular logic through child_process (That is kind of a joke)

Some related Lines

• https://github.com/iojs/io.js/blob/v0.12/lib/_http_server.js#L241 - Where its obvious node creates a Server
• https://github.com/iojs/io.js/blob/v0.12/lib/child_process.js#L427 - Where its checking the type of handle to send between processes

[Mickael-van-der-Beek]

👍

I really like the approach Substack is using for libraries like Hyperquest.

[sonewman]

I'm not quite sure what you mean by this. Are you saying you would like more things to be transform streams?

Because it seems like you examples demonstrate more of a duplex stream model, which is how the core already net.Socket works. The writable end does not pass data through to the readable end. With regards to the different events, this would involve creating a whole bunch of streams, which may not even be used by the developer. Both req and res are streams at heart, it is only frameworks like https://github.com/strongloop/express which abstract this away by using body-parser to intercept the data into req.body otherwise you can just pipe the req and pipe into the res. If you need the raw request you can use net.createServer() and parse the raw request headers incl.

Extra streams for particular use cases should probably be in user-land. By all means experiment!

Forgive me if I have misinterpreted your post.

[formula1]

They are old style streams

• https://github.com/iojs/io.js/blob/v0.12/lib/stream.js#L37
• https://github.com/iojs/io.js/blob/v0.12/lib/_http_outgoing.js#L25
• https://github.com/iojs/io.js/blob/v0.12/lib/_http_outgoing.js#L95

I'm not talking about express. Express is Incredible at what it does alone, extremely easy to use and tons of things that interact with it. I'm talking about how within Node's and io.js's framework they don't practice what they preach, usually because they got a million things to do and some of it is still legacy. I'm not even saying I want a body parser, that is available from somebody else. What I'm saying is that if transforms are meant to be for protocols -> They should use them so that others follow.

I'll delete the psuedocode because it seems that it distracted you and may distract others

[sonewman]

Ah ok, sorry I know http.OutgoingMessage and http.ServerResponse are both inherit from stream rather than any of the newer stream instantiables. I think they would be more appropriate as a stream.Writable because when you write to it do you really want to stream the contents out the other side of the res object? it kind of goes against the Law of Demeter but i do see what you are saying. I know @tjfontaine did a lot of work to break the http module into smaller parts but perhaps there is still work to be done, and some valid reasons why this has not been moved over to use a Writable I haven't looked too much into the source in this area of the project.

My advice would be to fork io.js and give it a go, i'm sure the contribution for this would be welcome, unless there is a compelling reason why this should remain a vintage stream.

There is no harm in experimentation, it certainly gives a better understanding of the problem domain.

[formula1]

I'll probably give it a go it after I'm done with some other stuff still pending. Stream Writable would work. The most important part is to make sure the Stream implementers in the future don't feel overwhelmed by the http example. Instead of solving the problem in their own way, they have a good example of how the sockets they're using can be piped/unpiped with ease and still be sent over child_processes.

By the way, you seem to have a cool head about you. It's pretty calming. I'm pretty sure your a pleasure to work with. Thanks for the responses!

[sonewman]

Thanks, I appreciate the feedback! I just see the benefit of listening to other peoples ideas, collaboration is key.

I can't tell you the number of times I have just hacked up some alternative solution to a problem just to see how feasible it would be. Experimentation is how awesome things come about (Node/io.js would probably not exist otherwise). Other times things don't work out so well, but either way it's a learning experience (forgive me I'm probably preaching to the choir! 😃).

I guess that's why I would encourage others to do the same(and I would be interested to see how you get on, what sort of problems you run into and the solutions you find to overcome them).

There has also a lot of discussions about the current streams implementations (nodejs/node#89 & nodejs/node#188) recently, the advent of io.js seems to have opened the doors for people to express their opinions on all sorts of things and get involved in the evolution of the project, which is great IMO.

I have a lot of ideas of my own, but it is great to hear what other people think from a different perspective 😄