Handle partially parsed objects (w/ querystrings as keys)

[kainosnoema]

tl;dr

• { 'foo[bar]': { qux: 0 }, 'foo': { baz: 1 } } => { foo: { bar: { qux: 0 }, baz: 1 } }
• previously this would work in some cases, this fix handles most cases properly
• allows for handling of pre-parsed multipart forms from felixge/node-formidable
• only added a few lines plus tests, all tests pass

Why?

For multipart forms, particularly from node-formidable. This particular use case is important:

We can use mimetypes to partially parse data from node-formidable parts, before using qs to reassemble

// Raw POST body:
//
// --xxxxxxxx
// Content-Disposition: form-data; name="foo"
// Content-Type: application/json
// 
// {"bar": "baz"}
// --xxxxxxxx
// Content-Disposition: form-data; name="foo[qux]"
// Content-Type: image/jpeg
// 
// (... image data ...)
// --xxxxxxxx--

// Simplified example:

> console.log(qs.parse({
>   foo: JSON.parse("{\"bar\": \"baz\"}")
> , 'foo[qux]': [object Object] // JPEG data in Formidable.File object
> }));
>
{
  foo: {
    bar: "baz",
    qux: [object Object] // JPEG data in Formidable.File object
  }
}

Does we really need this?

Yes. The RestKit iOS framework (among others) can send multipart forms containing both application/json and attachment (ie. image/jpeg) data. Properly merging the parts back into a single params object is crucial. The ExpressJS body-parser got us most of the way there—although we did have to override handlePart() so we could parse the JSON based on the mimetype.

The final piece of the puzzle is getting qs to reassemble the parts from the querystring paths as expected. As I mentioned in the commit message, qs was doing this properly in some cases, but not all (depending on part order from node-formidable, which is apparently non-deterministic). This fix should handle most cases as expected.

[tj]

why do we need this? where's v coming from that we need typeof

[tj]

oh it's right above haha, we dont need typeof then

[kainosnoema]

Just consistency really, and because undefined can be redefined in Javascript, meaning the safest way to check for undefined is with typeof. It's a habit of mine...

EDIT: I knew this, but just tried it... blows my mind:

> typeof undefined
'undefined'
> {}['foo'] === undefined
true
> undefined = 'foo';
'foo'
> typeof undefined
'string'
> {}['foo'] === undefined
false

EDIT 2: Apparently this behavior was eliminated in newer versions of V8? The above was in Node v0.4.8, v0.6.2 doesn't allow it... thank goodness.

[tj]

ah gotcha, personally I avoid defensive programming whenever possible to delegate the issue. plus there are a ton of retarded things you can do in js, people just shouldn't do them haha

[kainosnoema]

Ok, so I've found a weird edge case that I'd like to get your feedback on before you do anything with this:

Current behavior:

> console.log(qs.parse({ 'foo[items]': [], foo: { items: ['bar'] } }))
{ "foo": { "items": [ [ "bar" ] ] } }

> console.log(qs.parse({ foo: { items: ['bar'] }, 'foo[items]': [] }))
{ "foo": { "items": [ "bar", [] ] } }

Oddly enough, we actually ran into this situation with RestKit (starting to hate it). We were expecting a consistent output, regardless of order, but unfortunately there isn't one when you use the "rules" of querystrings: that repeated keys are pushed together into arrays. The output above actually makes sense given the rules—its just not what we expected or wanted.

One possible solution would be to use concat() instead of push() under the set() function, since with non-Array objects, concat() behaves similarly to push() (so we keep existing behavior for normal querystrings—all tests still pass). At least the result is consistent regardless of order, but again, this isn't standard querystring behavior:

Using concat() instead of push():

> console.log(qs.parse({ 'foo[items]': [], foo: { items: ['bar'] } }))
{ "foo": { "items": [ "bar" ] } }

> console.log(qs.parse({ foo: { items: ['bar'] }, 'foo[items]': [] }))
{ "foo": { "items": [ "bar" ] } }

To be honest, I don't really like either option—this whole thing seems really weird.

Thoughts?

[tj]

my vote is on the second I think