Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Confused JSON mimetypes in blackbox Data Service plumbing #1104

Closed
jmakeig opened this issue May 20, 2019 · 5 comments
Closed

Confused JSON mimetypes in blackbox Data Service plumbing #1104

jmakeig opened this issue May 20, 2019 · 5 comments
Assignees
@georgeajit
Labels
Bug major ship
Milestone
java-client-api-5...

Comments

@jmakeig
Copy link
Contributor

jmakeig commented May 20, 2019

I’m stuck with my Data Service. I want it to return a sequence (potentially empty) of JSON docs, but I’m getting an error in the non-user-servicable plumbing around Returned document as application/vnd.marklogic-api+json instead of application/json. If I POST to the same endpoint I get a correct looking multipart, but the blackbox client seems to barf on this.

Version of MarkLogic Java Client API

implementation 'com.marklogic:marklogic-client-api:4.2.0'
//…
id 'com.marklogic.ml-development-tools' version '4.2.0'

Version of MarkLogic Server

9.0-20190519

Java version

openjdk version "12.0.1" 2019-04-16
OpenJDK Runtime Environment (build 12.0.1+12)
OpenJDK 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)

OS and version

ProductName:	Mac OS X
ProductVersion:	10.14.5
BuildVersion:	18F132

Input

apis.json

{
  "functionName": "apis",
  "params": [
    {
      "name": "service",
      "datatype": "string"
    },
    {
      "name": "api",
      "datatype": "string",
      "nullable": true
    }
  ],
  "return": {
    "datatype": "jsonDocument",
    "multiple": true
  }
}

apis.sjs

cts.doc('/helloWorld/helloWorld.api');

Test

class DataServicesManagerTest {

	@Test
	void test() {
		DatabaseClient client = DatabaseClientFactory.newClient("localhost", 8010,
				new DatabaseClientFactory.DigestAuthContext("user", "********"));

		assertEquals("Nothing yet…", DataServicesManager.on(client).apis("/helloWorld", null));

		client.release();
	}

}

Actual output

image

java.lang.RuntimeException: Returned document as application/vnd.marklogic-api+json instead of application/json
	at com.marklogic.client.impl.OkHttpServices.checkNull(OkHttpServices.java:6151)
	at com.marklogic.client.impl.OkHttpServices$MultipleCallResponseImpl.setMultipart(OkHttpServices.java:5989)
	at com.marklogic.client.impl.OkHttpServices$MultipleCallResponseImpl.setResponse(OkHttpServices.java:5981)
	at com.marklogic.client.impl.OkHttpServices$CallRequestImpl.executeRequest(OkHttpServices.java:5620)
	at com.marklogic.client.impl.OkHttpServices$CallRequestImpl.withMultipartMixedResponse(OkHttpServices.java:5542)
	at com.marklogic.client.impl.BaseProxy$DBFunctionRequest.responseMultiple(BaseProxy.java:673)
	at com.marklogic.community.DataServicesManager$1DataServicesManagerImpl.apis(DataServicesManager.java:87)
	at com.marklogic.community.DataServicesManagerTest.test(DataServicesManagerTest.java:17)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:628)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:117)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:184)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:180)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:127)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:137)
	at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.run(JUnit5TestReference.java:89)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:41)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:541)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:763)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:463)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:209)

image
@ehennum
Copy link
Contributor

ehennum commented May 20, 2019 •
edited
Loading

If both multiple and null returns are valid, set both nullable and multiple to true in the return property of the api declaration.

By default, parameters and returns are non-nullable.

@jmakeig
Copy link
Contributor Author

jmakeig commented May 20, 2019

Ahh, the issue is that there’s a mime-type registered for the .api extension and thus the app server is being too smart and setting the Content-Type header as application/vnd.marklogic-api+json rather than just application/json.

xdmp.uriContentType('/helloWorld/helloWorld.api'); // application/vnd.marklogic-api+json

This is kind of an oddball case because I’m actually working with the *.api files. However, this might come up for others.

Array.from(xdmp.mimetypes()).filter(mime => /\+json/.test(mime.mimetype));
[
  {
    "mimetype": "application/rdf+json",
    "extensions": "rj",
    "format": "json"
  },
  {
    "mimetype": "application/sparql-results+json",
    "extensions": "srj",
    "format": "json"
  },
  {
    "mimetype": "application/vnd.marklogic-tde+json",
    "extensions": "tdej",
    "format": "json"
  },
  {
    "mimetype": "application/vnd.marklogic-api+json",
    "extensions": "api",
    "format": "json"
  }
]

The workaround is to slap a .toObject() on the cts.doc() call in my E-node code so that the node “forgets” its mime-type during serialization. Should the Data Service client be more defensive and interpret anything with a +json as JSON? Again, this is kind of a corner case.

@ehennum
Copy link
Contributor

ehennum commented May 21, 2019

Apologies, I misinterpreted the issue (which the original filing stated clearly).

The client should certainly accept

application/(.++)?json as a jsonDocument, object, or array
application/(.++)?xml as an xmlDocument
text/* as a textDocument
anything other than an atomic as a binaryDocument

@ehennum ehennum self-assigned this May 21, 2019
ehennum pushed a commit that referenced this issue May 30, 2019
regex for mimetype extensions when determining format #1104
5ecc625
@ehennum ehennum added test and removed new labels May 30, 2019
@ehennum ehennum assigned georgeajit and unassigned ehennum May 30, 2019
@ehennum
Copy link
Contributor

ehennum commented May 30, 2019

I think the ml-development-tools/src/test/ml-modules/root/dbfunctiondef/positive/mimetype unit test is probably sufficient to verify the fix.

I also noticed a bug in the conversion of handles that are bufferable but not resendable -- the converted BytesHandle value wasn't used -- and fixed that as well.

@georgeajit georgeajit added ship and removed test labels Sep 3, 2019
@georgeajit
Copy link
Contributor

ml-development-tools have been running fine after this fix.

@ehennum ehennum added this to the java-client-api-5.0.1 milestone Oct 4, 2019
@ehennum ehennum closed this as completed Oct 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@georgeajit georgeajit

Labels
Bug major ship
Projects
None yet
Milestone
java-client-api-5.0.1
Development

No branches or pull requests
3 participants
@jmakeig @ehennum @georgeajit