-
Notifications
You must be signed in to change notification settings - Fork 3
error 500 on GET /collections/:lidvid:/products #17
Comments
This is working since NASA-PDS/pds-api#4 was completed on PR NASA-PDS/pds-api#11 and it has examples showing that this is working. Does this mean it just needs to be deployed? |
@al-niessner per @tloubrieu-jpl
re-assigning to @tloubrieu-jpl |
closing per #17 (comment) |
@jordanpadams I am not convinced this ticket should be closed yet, because of this comment #17 (comment) |
thanks @tloubrieu-jpl who should we assign this task to then? are you able to look or should we have someone else? |
@jordanpadams I am slow on that. My plan next plan is to create a stable release (0.2.0) with the latest development that Al made and deploy it on pds-gamma. Then I can investigate that bug. i am also having some other issues to work on for the API. I will do a full day (minus meetings) on the API tomorrow but I start with NASA-PDS/registry-api#467 and maybe #10 I need to document the deployment on pds-gamma so that @al-niessner can do it and debug there. That is going to be useful for the performance ticket as we mentionned earlier. |
ok thanks @tloubrieu-jpl |
I confirm the error is still there after I upgraded the pds-gamma server to the latest version (0.2.0) |
@jordanpadams @tloubrieu-jpl -- cannot duplicate Just built the registry-api-service master branch and ran a curl of the form
Tried a non-existent lidvid too and it still "works"
|
Hi Al,
You are right there is something wrong with the non existing lid/lidvid behavior.
I will look at that.
Thomas
From: al-niessner ***@***.***>
Reply-To: NASA-PDS/registry-api-service ***@***.***>
Date: Wednesday, May 26, 2021 at 10:58 AM
To: NASA-PDS/registry-api-service ***@***.***>
Cc: "Loubrieu, Thomas G (US 398F)" ***@***.***>, Mention ***@***.***>
Subject: [EXTERNAL] Re: [NASA-PDS/registry-api-service] error 500 on GET /collections/:lidvid:/products (#17)
Just built the registry-api-service master branch and ran a curl of the form /collections/lidvid/products?start&limit&fields&summary and it worked (see below for gory details). Upgrade issue or something else?
$ curl -silent --header 'Accept: application/json' "http://localhost:8080/collections/urn:nasa:pds:izenberg_pdart14_meap:data_eetable::1.0/products?limit=1&start=0&fields=ops%3AData_File_Info.ops%3Amd5_checksum&only-summary=false"
HTTP/1.1 200
Content-Type: application/json
Transfer-Encoding: chunked
Date: Wed, 26 May 2021 17:55:51 GMT
{"summary":{"start":0,"limit":1,"sort":[],"properties":["ops:Data_File_Info.ops:md5_checksum"]},"data":[{"id":"urn:nasa:pds:izenberg_pdart14_meap:data_eetable:ele_evt_12hr_orbit_2011-2012::1.0","type":"Product_Observational","title":"Mercury Energetic Electrons Events Table","investigations":[{"id":"urn:nasa:pds:context:investigation:mission.messenger","href":"http://localhost:8080/products/urn:nasa:pds:context:investigation:mission.messenger"}],"observing_system_components":[{"id":"urn:nasa:pds:context:instrument_host:spacecraft.mess","href":"http://localhost:8080/products/urn:nasa:pds:context:instrument_host:spacecraft.mess"},{"id":"urn:nasa:pds:context:instrument:ns.mess","href":"http://localhost:8080/products/urn:nasa:pds:context:instrument:ns.mess"}],"targets":[{"id":"urn:nasa:pds:context:target:planet.mercury","href":"http://localhost:8080/products/urn:nasa:pds:context:target:planet.mercury"}],"metadata":{"version":"1.0","label_url":"/var/local/harvest/archive/data_eetable/ele_evt_12hr_orbit_2011-2012.xml"},"properties":{"ops:Data_File_Info.ops:md5_checksum":"b97c2b729120caf4d48f1f1442181396"}}]}
Tried a non-existent lidvid too and it still "works"
$ curl -silent --header 'Accept: application/json' "http://localhost:8080/collections/urn:nasa:pds:izenberg_pdart14_meap:data_eetables::1.0/products?limit=1&start=0&fields=ops%3AData_File_Info.ops%3Amd5_checksum&only-summary=false"
HTTP/1.1 200
Content-Type: application/json
Transfer-Encoding: chunked
Date: Wed, 26 May 2021 17:57:06 GMT
{"summary":{"start":0,"limit":1,"sort":[],"properties":[]}}
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https:/github.com/NASA-PDS/registry-api-service/issues/17*issuecomment-848997096__;Iw!!PvBDto6Hs4WbVuu7!fDuuzMCcSww616gzCGiHxtHX_9jHktdxQU8V354Xi2ZSXTnn5aWxitVDngZqxYJNjxceKhKHhvU$>, or unsubscribe<https://urldefense.us/v3/__https:/github.com/notifications/unsubscribe-auth/AORLCUBRPWYLSTQP7JSKMG3TPUZEDANCNFSM43AKMXGQ__;!!PvBDto6Hs4WbVuu7!fDuuzMCcSww616gzCGiHxtHX_9jHktdxQU8V354Xi2ZSXTnn5aWxitVDngZqxYJNjxcek91BsCo$>.
|
Actually that is a behavior I removed on purpose as an interim solution, see comments in the code at line Line 96 in 3db2bb4
To make that work again I would make getProductChildren throw an exception (e.g. LidVidNotFoundException) and trigger the 404 error on when this exception is raised. |
@al-niessner i am working on the exception management for the not found lidvid now. |
Main point of the two curls above are that neither causes error 500. So, how to either replicate original error or why does pds-gamma after upgrade still produce a 500 error? |
@al-niessner I am pretty sure the error comes from the data we have in the registry on pds-gamma. I will debug that by adding log messages if needed on the pds-gamma deployment, so to identify the eleasticSearch object which makes the software fail. |
Ah, is it possible that it is trying to return so much data that it causes a MemoryError? If so, does this mean it will be fixed (potentially) by NASA-PDS/pds-api#13? |
Maybe but I don't think we have memory issues like that, or at least we did not in the past. For now I was more thinking one specific object is not formatted as expected by the product bean (https://github.com/NASA-PDS/registry-api-service/blob/master/src/main/java/gov/nasa/pds/api/engineering/elasticsearch/entities/EntityProduct.java). For example one field expected as String is an array or something similar. |
@tloubrieu-jpl could this then maybe be related to NASA-PDS/harvest#50 ? |
note: i have no idea. just string vs. array reminded me of that ticket that Al created |
@tdddblog @jordanpadams @al-niessner ,
Usually it is an array. I will update the code to support the string. |
@tloubrieu-jpl @tdddblog so this appears to be directly tied to NASA-PDS/harvest#50. so do we fix in the API and then eventually harvest? |
I am fixing that in the API now. . |
For test purpose I deployed the branch error_500_17 on pds-gamma and it works. |
🐛 Describe the bug
📜 To Reproduce
Steps to reproduce the behavior:
Run request:
curl --location --request GET 'https://pds-gamma.jpl.nasa.gov/api/collections/urn:nasa:pds:insight_documents:document_mission::1.1/products?start=0&limit=10&fields=ops:Data_File_Info.ops:md5_checksum&only-summary=false'
The request returns an error 500
🕵️ Expected behavior
The request should return the list of products belonging to the collection
urn:nasa:pds:insight_documents:document_mission::1.1
📚 Version of Software Used
version 0.1.0 of the API
🩺 Test Data / Additional context
Log message is:
Run request:
2021-04-15 14:56:03.689 DEBUG 3583235 --- [nio-8081-exec-1] lasticSearchRegistrySearchRequestBuilder : Request product reference documents from 0 for size 1
2021-04-15 14:56:03.768 DEBUG 3583235 --- [nio-8081-exec-1] lasticSearchRegistrySearchRequestBuilder : search product ref request :SearchRequest{searchType=QUERY_THEN_FETCH, indices=[registry-refs], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, expand_wildcards_hidden=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], types=[], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=null, allowPartialSearchResults=null, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, ccsMinimizeRoundtrips=true, source={"from":0,"size":1,"query":{"match":{"collection_lidvid":{"query":"urn:nasa:pds:insight_documents:document_mission::1.1","operator":"OR","prefix_length":0,"max_expansions":50,"fuzzy_transpositions":true,"lenient":false,"zero_terms_query":"NONE","auto_generate_synonyms_phrase_query":true,"boost":1.0}}}}}
2021-04-15 14:56:04.205 ERROR 3583235 --- [nio-8081-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.ClassCastException: java.lang.String cannot be cast to java.util.ArrayList] with root cause
java.lang.ClassCastException: java.lang.String cannot be cast to java.util.ArrayList
at gov.nasa.pds.api.engineering.elasticsearch.business.CollectionProductIterator.initProductIterator(CollectionProductIterator.java:143) ~[classes/:na]
at gov.nasa.pds.api.engineering.elasticsearch.business.CollectionProductIterator.(CollectionProductIterator.java:43) ~[classes/:na]
at gov.nasa.pds.api.engineering.elasticsearch.business.CollectionProductRelationships.iterator(CollectionProductRelationships.java:81) ~[classes/:na]
at gov.nasa.pds.api.engineering.controllers.MyCollectionsApiController.getProductChildren(MyCollectionsApiController.java:146) ~[classes/:na]
at gov.nasa.pds.api.engineering.controllers.MyCollectionsApiController.productsOfACollection(MyCollectionsApiController.java:93) ~[classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_271]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_271]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_271]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_271]
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190) ~[spring-web-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138) ~[spring-web-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105) ~[spring-webmvc-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879) ~[spring-webmvc-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793) ~[spring-webmvc-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040) ~[spring-webmvc-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943) ~[spring-webmvc-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006) ~[spring-webmvc-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898) ~[spring-webmvc-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634) ~[tomcat-embed-core-9.0.36.jar:9.0.36]
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883) ~[spring-webmvc-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) ~[tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) ~[tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) ~[tomcat-embed-websocket-9.0.36.jar:9.0.36]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.36.jar:9.0.36]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119) ~[spring-web-5.2.7.RELEASE.jar:5.2.7.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) ~[tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) ~[tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) ~[tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:373) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1590) [tomcat-embed-core-9.0.36.jar:9.0.36]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-9.0.36.jar:9.0.36]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_271]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_271]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-9.0.36.jar:9.0.36]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_271]
🏞Screenshots
🖥 System Info
** 🦄 Applicable requirements**
The text was updated successfully, but these errors were encountered: