reproducer.. wt=phps incompatibility with /admin/luke still in doubts. ..

[mkhludnev]

Expected :a:1:{s:3:"map";a:1:{s:10:"lucene doc";a:3:{s:2:"id";s:1:"2";s:4:"cost";s:1:"5";s:4:"name";a:1:{s:7:"foo bar";}}}}
Actual   :a:1:{s:3:"map";a:1:{s:10:"lucene doc";i:0;a:3:{s:2:"id";s:1:"2";s:4:"cost";s:1:"5";s:4:"name";a:1:{i:0;s:7:"foo bar";}}}}

[thomascorthals]

Looks like there's a small mistake in the expected value. If name is multivalued, the PHPS representation must contain an index and a value.

a:1:{s:3:"map";a:1:{s:10:"lucene doc";a:3:{s:2:"id";s:1:"2";s:4:"cost";s:1:"5";s:4:"name";a:1:{i:0;s:7:"foo bar";}}}}

You can run some PHP code online to check the output: https://onlinephp.io?s=tY5NCsIwEEb3hd4hzCYVXPRHESaCuCi4EOwNSkxHGmib0DYiine3gYAX0NkM8wa-7-0PtrVxFEekWsN4-bCkZmqQr1l1qurychZxdJdj3bjeJm6YaNSy009KuMQMXxMWCL20IMKZpQidUzQQa4zyuPA4R9ANiOWPkPu9QVBmmgPaBjTInkKUxnRhO4SbMewqRxDvZfhqJb66RzU72f1O1nf-RfgD&v=8.2.10

[mkhludnev]

@thomascorthals no-handle bars fix. Still barely follow.
https://onlinephp.io?s=TY7NCoMwEITvgu8Q9hKFHvxpKWwKPQk9FOobSJpsMaAmGC2lpe9eLTk4l9n52F3mdHati6M4ItVaxquXIzWRRr5j9aVuqttVxNFTjo2ee5fMg6fRyM68KeESc_x4LBF66UCEmGcI3axoIKatCthgth3KdbFAMBrEcoFQrL5HUNZPAR0CGmRPmy8ejwgPa9ldjiC-f_E0XUr-AA%2C%2C&v=8.2.10

[mkhludnev]

remove me. wrong way

[dsmiley]

A side: I recall some doubt as to whether Solr ought to even have a "php" writer in the first place. Maybe it should be deprecated. Surely JSON is the standard? WDYT
CC @epugh

[thomascorthals]

As someone on the PHP side of things, there's a good reason to look into PHPS with the /luke handler in particular.

curl 'http://localhost:8983/solr/techproducts/admin/luke?id=VDBDB1A16&json.nl=flat'

Snippet of the output:

      "cat":{
        "type":"string",
        "schema":"I-S-UM----OF-----l",
        "flags":"ITS-------OF------",
        "value":"electronics",
        "internal":"electronics",
        "docFreq":15},
      "cat":{
        "type":"string",
        "schema":"I-S-UM----OF-----l",
        "flags":"ITS-------OF------",
        "value":"memory",
        "internal":"memory",
        "docFreq":3},

The output for a SimpleOrderedMap can contain duplicate keys and this can't be controlled with json.nl. Which means PHP's json_decode() loses data. PHPS solves this by mangling the keys to cat, cat 1, cat 2 …

This is a (admittedly very rare) case where dealing with JSON output in PHP is cumbersome.

[epugh]

A side: I recall some doubt as to whether Solr ought to even have a "php" writer in the first place. Maybe it should be deprecated. Surely JSON is the standard? WDYT CC @epugh

I think it's a valid question.. That applies to all the language specific writer types... Especially if we start supporting code generated API libraries that some of these wrappers for Solr could think about adopting? @thomascorthals, maybe a conversation for mailing list, but I'd love you thoughts on if we generated a PHP library, would that make your life easier?

[epugh]

So... I think we are faced here with a different issue than how PHP is handling the output.... When I run the curl command that was shared, I also see the cat key come back mulitple times... And that is a problem with the JSON output.. indeed, running

curl 'http://localhost:8983/solr/techproducts/admin/luke?id=VDBDB1A16&json.nl=flat'  | jq .

and we drop the extra cat clauses from the original. It seems like we actually have a bug???

[epugh]

Okay, so I see that cat is duplicated because it's a multivalued field, so we get:

      "cat":{
        "type":"string",
        "schema":"I-S-UM----OF-----l",
        "flags":"ITS-------OF------",
        "value":"electronics",
        "internal":"electronics",
        "docFreq":12
      },
      "cat":{
        "type":"string",
        "schema":"I-S-UM----OF-----l",
        "flags":"ITS-------OF------",
        "value":"memory",
        "internal":"memory",
        "docFreq":3
      },

Shouldn't this instead be instead an array of values for the cat key? I know this changes things...:

      "cat":[{
        "type":"string",
        "schema":"I-S-UM----OF-----l",
        "flags":"ITS-------OF------",
        "value":"electronics",
        "internal":"electronics",
        "docFreq":12
      },
      {
        "type":"string",
        "schema":"I-S-UM----OF-----l",
        "flags":"ITS-------OF------",
        "value":"memory",
        "internal":"memory",
        "docFreq":3
      }],

[mkhludnev]

I hardly get common hesitation over duplicated JSON keys. Why it is a problem? Github come up with 480 json PHP parsers. Isn't there a library, which don't feel so unique about keys?

[dsmiley]

Remember json.nl parameter options, which exist to balance convenience of the response with rigor over retrieving duplicated keys. But that only applies to JSON. https://solr.apache.org/guide/solr/latest/query-guide/response-writers.html#json-nl

[epugh]

I guess my point is that the actual API response, in JSON, is outputting invalid JSON. That the fix is to change the way we handle multivalued fields to nest them as an array under the top level key....

[dsmiley]

I guess my point is that the actual API response, in JSON, is outputting invalid JSON. That the fix is to change the way we handle multivalued fields to nest them as an array under the top level key....

It's invalid but the user is in control of supplying a different value to json.nl which avoids the invalidity. Use json.nl=arrmap for example. No change in Solr is necessary.

But this work item is about php.

[thomascorthals]

@thomascorthals, maybe a conversation for mailing list, but I'd love you thoughts on if we generated a PHP library, would that make your life easier?

I had the same thought when I saw the generated Python client. Solarium aims at accurately modelling Solr by providing a fluent interface to build queries and parsing the results to PHP objects. The implementation for API requests is currently generic and just gets you the output from Solr as an array.

Isn't there a library, which don't feel so unique about keys?

There is, and I'm already using it to parse that particular piece of JSON. There are two things at play for me:

• Solarium supports JSON and PHPS output since before I was using it. I'm trying not to break that. (If Solr deprecates PHPS, that's a different story of course.)
• I was curious how parsing it would compare performance wise.

Remember json.nl parameter options

That is for NamedLists, the "offending" data structure is a SimpleOrderedMap and those aren't affected by json.nl.

the actual API response, in JSON, is outputting invalid JSON.

Technically, it isn't. ECMA-404 doesn't say anything about it. RFC 8259 says names SHOULD be unique, not MUST. But you can see how this becomes a nuisance when decoding to a language that has no internal data structure that allows duplicate keys.

[dsmiley]

Sorry; I wasn't paying enough attention. We shouldn't be using a SimpleOrderedMap in any scenario where keys can be duplicated.