Retry / File State Storage #385

vaibhav-db · 2023-03-27T13:20:47Z

refactor retry handler

…daptive card Merge in SYMPHONYP/symphony-java-toolkit from feature/SYMPHONYP-939-ms-teams-spring-bot-retry-messages to spring-bot-master-db * commit '3f2fec632e13056b0fec79bffe62697fec0e4913': SYMPHONYP-940 MS Teams feature to submit the adaptive card

…:8082/scm/symphonyp/symphony-java-toolkit into feature/SYMPHONYP-939-ms-teams-spring-bot-retry-messages

Merge in SYMPHONYP/symphony-java-toolkit from feature/SYMPHONYP-939-ms-teams-spring-bot-retry-messages to spring-bot-master-db * commit '3e4b60cbfa4318e45122d113b408aec181c8bf98': review comments changes done. added testcases for Message Retry. SYMPHONYP-939 implements review comments SYMPHONYP-940 refactor retry handler fixing formatting Adding roadmap Update LICENSE setting copyright name

linux-foundation-easycla · 2023-03-27T13:20:54Z

The committers listed above are authorized under a signed CLA.

✅ login: vaibhav-db / name: Vaibhav Mankar (3f2fec6, 25a538f, a9f3699, 2d448ea, 80509b3, 1781441)
✅ login: abhishekdahiya-db / name: Abhishek (b9a31ff, 3e4b60c)

robmoffat · 2023-03-27T14:15:42Z

...ring-boot-starter/src/main/java/org/finos/springbot/teams/handlers/TeamsResponseHandler.java

@@ -62,7 +61,7 @@ public class TeamsResponseHandler implements ResponseHandler, ApplicationContext
 	protected ThymeleafTemplateProvider displayTemplater;
 	protected TeamsStateStorage teamsState;
 	protected TeamsConversations teamsConversations;
-	protected MessageRetryHandler messageRetryHandler;
+	protected MessageRetryHandler retryHandler;


can we not have retryHandler in a subclass? e.g. RetryingTeamsResponseHandler?

Changes done as per review comments

robmoffat · 2023-03-27T14:20:59Z

...ring-boot-starter/src/main/java/org/finos/springbot/teams/handlers/TeamsResponseHandler.java

@@ -142,7 +141,7 @@ protected TemplateType getTemplateType(WorkResponse wr) {
 		TemplateType tt;
 		if (displayTemplater.hasTemplate(wr)) {
 			tt = TemplateType.THYMELEAF;
-		} else if (workTemplater.hasTemplate(wr)) {
+		} else if (workTemplater.hasTemplate(wr) || AdaptiveCardPassthrough.isAdaptiveCard(wr)) {


it's odd that this has changed?

Hi Rob,

We have usecase where we wanted to send adaptive card as passthrough like user send Adaptive card json as string input parameter and bot will send this adaptive card to ms teams channel.

robmoffat · 2023-03-27T14:21:30Z

...ring-boot-starter/src/main/java/org/finos/springbot/teams/handlers/TeamsResponseHandler.java

@@ -274,7 +251,7 @@ public void retryMessage() {
 		int messageCount = 0;

 		Optional<MessageRetry> opt;
-		while ((opt = messageRetryHandler.get()).isPresent()) {
+		while ((opt = retryHandler.get()).isPresent()) {


please make sure we have code coverage of this.

Merge in SYMPHONYP/symphony-java-toolkit from bugfix/SYMPHONYP-946-History-Api-bug-fix to spring-bot-master-db * commit '18d766fd56c9bb561b2de3179c75cd57fae5e3eb': bug fix implemented.

…andler refactor code Merge in SYMPHONYP/symphony-java-toolkit from feature/SYMPHONYP-947-teams-spring-bot-retry-handler-refactor-code to spring-bot-master-db * commit '320488b2b314041fab0498f244763a1c2250973c': refactor retry message handler refactor retry message handler refactor retry message handler refactor retry message handler refactor retry message handler refactor retry message handler

robmoffat · 2023-04-13T14:46:22Z

...ing-boot-starter/src/test/java/org/finos/springbot/teams/state/AbstractStateStorageTest.java

@@ -72,6 +72,28 @@ public void testStoreWithTagDates() throws IOException {
 		Assertions.assertEquals(2, hoover(tss.retrieve(tagList4, false)).size());
 	}

+	@Test
+	public void testSlashStoreWithMultTags() throws IOException {


testFilenamesWithSlash

robmoffat · 2023-04-13T14:46:54Z

...ing-boot-starter/src/test/java/org/finos/springbot/teams/state/AbstractStateStorageTest.java

+		tss.store("thefile/thefile", tags1, somedata);
+		tss.store("thefile/theotherfile", tags2, somedata);
+
+		Assertions.assertEquals(1, hoover(tss.retrieve(tagList1, false)).size());	


shouldn't you also check you can retrieve the other file as well?

ok.. I will add other file also.

robmoffat · 2023-04-13T14:51:59Z

.../src/main/java/org/finos/springbot/teams/handlers/retry/InMemoryRetryingActivityHandler.java

+
+public class InMemoryRetryingActivityHandler extends AbstractRetryingActivityHandler  {
+
+	private Queue<MessageRetry> queue = new ConcurrentLinkedQueue<>();


Have you looked at this? https://stackoverflow.com/questions/40485398/retry-logic-with-completablefuture

I have already tried this, it was not working. As per our requirement if exception occurred we need to wait for Retry-After then try retry.

What didn't work about it? Is it the lack of delay? I am looking at this code from the SO page:

public CompletableFuture<Result> executeActionAsync() { CompletableFuture<Result> f=executeMycustomActionHere(); for(int i=0; i<MAX_RETRIES; i++) { f=f.thenApply(CompletableFuture::completedFuture) .exceptionally(t -> executeMycustomActionHere()) .thenCompose(Function.identity()); } return f; }

should not be too hard to add a delay on the .exceptionally() line, if that is the problem

No our requirement is to wait for Retry-Afer time, but we can't add the sleep or wait as it will wait the thread. we have implemented similar only inserted data into queue. and added cron job to resent the data.

we just don't want to used Thread.sleep(ra); in code. As it will keep the current thread on wait. :)

We can connect on Monday, I will explain you.
I am learning the CompletableFuture :)

argh you're right!! 😦

How about this: deutschebank#1

I actually wrote a test here - well, changed your InMemoryRetryingActivityHandlerTest to see if this actually worked.

Test passes but maybe you have more to add..?

Hi Rob,

Testcase only pass, if we have for (int i = 0; i < teamsRetryCount; i++) { for loop in code. But in our case, if exception occurred we simply insert that record in in-memory queue.
Here we have to check the how to write testcase around schedule also.

Hi, we can wait until Wednesday to talk about this or we can maybe have a call a bit sooner?

Hi Rob, we will connect tomorrow. Completely overload with teams.
I have tried CompletableFuture.delayedExecutor(ra, TimeUnit.MILLISECONDS) but it is in java 9, so we can't go for it.
Alternate solution i found as scheduler.
https://stackoverflow.com/questions/58707689/is-it-possible-to-schedule-a-completablefuture

robmoffat

looks good to me

vaibhav-db and others added 8 commits February 24, 2023 13:15

SYMPHONYP-940 MS Teams feature to submit the adaptive card

3f2fec6

SYMPHONYP-940 refactor retry handler

a9f3699

Merge branch 'spring-bot-master' of https://stash.gto.intranet.db.com…

2d448ea

…:8082/scm/symphonyp/symphony-java-toolkit into feature/SYMPHONYP-939-ms-teams-spring-bot-retry-messages

SYMPHONYP-939 implements review comments

80509b3

added testcases for Message Retry.

b9a31ff

review comments changes done.

3e4b60c

pankaj-a-khandelwal-db self-assigned this Mar 27, 2023

pankaj-a-khandelwal-db approved these changes Mar 27, 2023

View reviewed changes

robmoffat requested changes Mar 27, 2023

View reviewed changes

abhishekdahiya-db and others added 5 commits March 31, 2023 15:20

bug fix implemented.

18d766f

refactor retry message handler

add1a4d

refactor retry message handler

4b9e848

refactor retry message handler

5c475ca

Pull request finos#31: bug fix implemented.

f49836b

Merge in SYMPHONYP/symphony-java-toolkit from bugfix/SYMPHONYP-946-History-Api-bug-fix to spring-bot-master-db * commit '18d766fd56c9bb561b2de3179c75cd57fae5e3eb': bug fix implemented.

robmoffat mentioned this pull request Apr 3, 2023

File Storage not working properly #386

Closed

robmoffat changed the title ~~Spring bot master db~~ Retry / File State Storage Apr 3, 2023

vaibhav-db added 5 commits April 4, 2023 14:40

refactor retry message handler

a5496fe

refactor retry message handler

e2c1e8e

refactor retry message handler

320488b

Fixed state storage testcases

115d93e

pankaj-a-khandelwal-db requested a review from robmoffat April 7, 2023 10:02

pankaj-a-khandelwal-db approved these changes Apr 7, 2023

View reviewed changes

robmoffat requested changes Apr 13, 2023

View reviewed changes

vaibhav-db added 2 commits April 14, 2023 12:18

State storage testcases.

eb8de4a

refactor file stage storage and testcase

8f82361

vaibhav-db added 2 commits April 27, 2023 15:20

Teams retry InMemory handler

3bf0ea7

Teams retry InMemory handler

784b2c7

robmoffat approved these changes May 1, 2023

View reviewed changes

vaibhav-db added 2 commits May 2, 2023 12:36

Corrected InMemoryRetryingActivityHandler reference

7e21035

Error stream id in log

9007a16

robmoffat approved these changes May 9, 2023

View reviewed changes

robmoffat merged commit 4280e2f into finos:spring-bot-master May 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Retry / File State Storage #385

Retry / File State Storage #385

vaibhav-db commented Mar 27, 2023

linux-foundation-easycla bot commented Mar 27, 2023 •

edited

Loading

robmoffat Mar 27, 2023

pankaj-a-khandelwal-db Apr 7, 2023

robmoffat Mar 27, 2023

vaibhav-db Mar 28, 2023

robmoffat Mar 27, 2023

robmoffat Apr 13, 2023

robmoffat Apr 13, 2023

vaibhav-db Apr 14, 2023

robmoffat Apr 13, 2023

vaibhav-db Apr 14, 2023

robmoffat Apr 14, 2023

robmoffat Apr 14, 2023

vaibhav-db Apr 14, 2023

vaibhav-db Apr 14, 2023

robmoffat Apr 14, 2023

vaibhav-db Apr 15, 2023

robmoffat Apr 17, 2023

vaibhav-db Apr 18, 2023

robmoffat left a comment


		public class InMemoryRetryingActivityHandler extends AbstractRetryingActivityHandler {

		private Queue<MessageRetry> queue = new ConcurrentLinkedQueue<>();

Retry / File State Storage #385

Retry / File State Storage #385

Conversation

vaibhav-db commented Mar 27, 2023

linux-foundation-easycla bot commented Mar 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robmoffat left a comment

Choose a reason for hiding this comment

linux-foundation-easycla bot commented Mar 27, 2023 •

edited

Loading