Need better connection retry logic

[cmgrote]

Currently seems that pretty much any failure to connect to IGC via the REST API results in a retry by opening a new session. Should be more intelligent and see if the error is to do with a session timeout before blindly retrying new sessions, or we needlessly risk session overflows.

[cmgrote]

Simplest way to observe this occurring at a large scale is to have the proxy connector running and then do a complete removal of the OpenIGC OMRS bundle extension (dropping all stubs immediately) -- all subsequent stub lookups will then fail along these lines:

2020-03-11 17:02:31.719 DEBUG 92806 --- [       Thread-3] o.o.e.c.i.i.clientlibrary.IGCRestClient  : POSTing to https://infosvr:9446/ibm/iis/igc-rest/v1/search with: {"types":["$OMRS-Stub"],"properties":["$sourceRID","$sourceType","$payload"],"pageSize":100,"where":{"conditions":[{"property":"name","operator":"=","value":"database_b1c497ce.6e83759b.001n1ik3n.oj34p77.n67g8r.0h3vso3ihd0igccg76aqs"}],"operator":"and"}}
2020-03-11 17:02:31.749  WARN 92806 --- [       Thread-3] o.o.e.c.i.i.clientlibrary.IGCRestClient  : Request failed -- session may have expired, retrying...

org.springframework.web.client.HttpClientErrorException$BadRequest: 400 Bad Request: [no body]
	at org.springframework.web.client.HttpClientErrorException.create(HttpClientErrorException.java:101) ~[spring-web-5.2.4.RELEASE.jar!/:5.2.4.RELEASE]
	at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:170) ~[spring-web-5.2.4.RELEASE.jar!/:5.2.4.RELEASE]
	at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:112) ~[spring-web-5.2.4.RELEASE.jar!/:5.2.4.RELEASE]
	at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63) ~[spring-web-5.2.4.RELEASE.jar!/:5.2.4.RELEASE]
	at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:782) ~[spring-web-5.2.4.RELEASE.jar!/:5.2.4.RELEASE]
	at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:740) ~[spring-web-5.2.4.RELEASE.jar!/:5.2.4.RELEASE]
	at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:714) ~[spring-web-5.2.4.RELEASE.jar!/:5.2.4.RELEASE]
	at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:602) ~[spring-web-5.2.4.RELEASE.jar!/:5.2.4.RELEASE]
	at org.odpi.egeria.connectors.ibm.igc.clientlibrary.IGCRestClient.makeRequest(IGCRestClient.java:531) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.clientlibrary.IGCRestClient.openNewSessionWithRequest(IGCRestClient.java:276) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.clientlibrary.IGCRestClient.makeRequest(IGCRestClient.java:540) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.clientlibrary.IGCRestClient.makeRequest(IGCRestClient.java:563) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.clientlibrary.IGCRestClient.searchJson(IGCRestClient.java:806) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.clientlibrary.IGCRestClient.search(IGCRestClient.java:818) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.repositoryconnector.IGCRepositoryHelper.getOMRSStubForAsset(IGCRepositoryHelper.java:1604) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.eventmapper.IGCOMRSRepositoryEventMapper.sendPurgedEntity(IGCOMRSRepositoryEventMapper.java:1423) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.eventmapper.IGCOMRSRepositoryEventMapper.sendPurgedEntity(IGCOMRSRepositoryEventMapper.java:1407) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.eventmapper.IGCOMRSRepositoryEventMapper.processAsset(IGCOMRSRepositoryEventMapper.java:542) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.eventmapper.IGCOMRSRepositoryEventMapper.processAsset(IGCOMRSRepositoryEventMapper.java:513) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.eventmapper.IGCOMRSRepositoryEventMapper.processAssetEventV115(IGCOMRSRepositoryEventMapper.java:325) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.eventmapper.IGCOMRSRepositoryEventMapper.processEventV115(IGCOMRSRepositoryEventMapper.java:253) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.eventmapper.IGCOMRSRepositoryEventMapper.processEvent(IGCOMRSRepositoryEventMapper.java:192) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at org.odpi.egeria.connectors.ibm.igc.eventmapper.IGCOMRSRepositoryEventMapper$IGCKafkaConsumerThread.run(IGCOMRSRepositoryEventMapper.java:169) [egeria-connector-ibm-information-server-1.6-SNAPSHOT-jar-with-dependencies.jar:na]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_242]

2020-03-11 17:02:31.749 ERROR 92806 --- [       Thread-3] o.o.e.c.i.i.c.e.IGCConnectivityException : Details for connectivity issue 'Opening a new session already attempted without success -- giving up.': POST to https://infosvr:9446/ibm/iis/igc-rest/v1/search with: {"types":["$OMRS-Stub"],"properties":["$sourceRID","$sourceType","$payload"],"pageSize":100,"where":{"conditions":[{"property":"name","operator":"=","value":"database_b1c497ce.6e83759b.001n1ik3n.oj34p77.n67g8r.0h3vso3ihd0igccg76aqs"}],"operator":"and"}}