Incorrect results intermittently when sorting

[naleeha]

Bug Report

Steps to Reproduce:

1. Run the code with the basket constituent sort fix Basket Constituent table does not sort #1027
2. If you sort by description more than once - sometimes the results are out of order, sometimes they are correct
3. If you had sort on before loading the VP, it always creates VP with correct sort

When it correctly sorts

When it sorts incorrectly

[naleeha]

• Confirmed that the UI sends CHANGE_VP correctly and the TABLE_ROW that is returned from server has incorrect data. Incorrect index for certain rows sometimes, sometimes multiple results for the same index
• Confirmed sort logic always returned the expected order
• Seems to have multiple thread publishing updates at the same time - which seems to be causing the confusion in results

Open questions

• How does the UI deals with it if duplicate record with same index in a table?

[naleeha]

Description of what is happening
Can see nioEventLoopGroup is publishing updates for each rows with the old sorted values
viewPortExecutorRunner does the sort
viewPortExecutorRunner also publish updates for each rows with new sorted values
TABLE_ROW update is generated as aggregates of these update row events but now has mix of two results

server logs.txt
websocket messages.txt

Relevant Logs
17:21:21.563 [nioEventLoopGroup-3-2] INFO org.finos.vuu.core.sort.GenericSort2 - Sorting by List(description) and List(A)
17:21:21.563 [nioEventLoopGroup-3-2] INFO o.f.vuu.viewport.ViewPortContainer - [VP] default else condition in change() call
17:21:21.564 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - changeStructure(..) onlySortOrFilterChange=false
17:21:21.564 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - Sending updates on 283 inrangeKeys
17:21:21.564 [viewPortExecutorRunner[4]5] INFO o.f.vuu.viewport.ViewPortContainer - refreshing view port to apply sortGenericSort2(SortSpec(List(SortDef(description,A))),List(SimpleColumn(description,8,class java.lang.String))) requestId VP(user:nha1,table:basketConstituent,size: 782,id:nha1-VP-00000000) @SESS-5be9ea2c-c5ca-41be-981f-d851c4c5a87b
17:21:21.564 [viewPortExecutorRunner[4]5] INFO o.f.vuu.viewport.ViewPortContainer - refreshing view port to apply filter org.finos.vuu.core.filter.NoFilter$@ba3121c requestId VP(user:nha1,table:basketConstituent,size: 782,id:nha1-VP-00000000) @SESS-5be9ea2c-c5ca-41be-981f-d851c4c5a87b
17:21:21.564 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.core.sort.GenericSort2 - Starting map
17:21:21.565 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[0] = III.L..FTSE100 for request id 4
17:21:21.565 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[1] = MMM..SP500 for request id 4
17:21:21.565 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.core.sort.GenericSort2 - Starting sort
....

17:21:21.569 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.core.sort.GenericSort2 - [SORT]: Table Size: 782 DataToArray: 1ms, Sort: 2ms, ImmutArr: 2ms
...

17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[171] = 01209.HK..HSI for request id 4
17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[172] = 00836.HK..HSI for request id 4
17:21:21.570 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[0] = ZS..NASDAQ100 for request id 4
17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[173] = 01088.HK..HSI for request id 4
17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[174] = 00762.HK..HSI for request id 4
17:21:21.570 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[1] = ZM..NASDAQ100 for request id 4
17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[175] = 01378.HK..HSI for request id 4
17:21:21.570 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[176] = CMG..SP500 for request id 4
17:21:21.570 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[2] = ZTS..SP500 for request id 4
17:21:21.571 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[177] = 01929.HK..HSI for request id 4
17:21:21.571 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[3] = ZION..SP500 for request id 4
17:21:21.571 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[178] = CB..SP500 for request id 4
17:21:21.571 [viewPortExecutorRunner[4]5] INFO org.finos.vuu.viewport.ViewPortImpl - publishing update @[4] = ZBH..SP500 for request id 4

[naleeha]

17:21:21.563 [nioEventLoopGroup-3-2] INFO org.finos.vuu.core.sort.GenericSort2 - Sorting by List(description) and List(A)
17:21:21.563 [nioEventLoopGroup-3-2] INFO o.f.vuu.viewport.ViewPortContainer - [VP] default else condition in change() call
17:21:21.564 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - changeStructure(..) onlySortOrFilterChange=false
17:21:21.564 [nioEventLoopGroup-3-2] INFO org.finos.vuu.viewport.ViewPortImpl - Sending updates on 283 inrangeKeys

The onlySortOrFilterChange is expected to be true in this case and if so, it won't be publishing updates for the keys on nioEventLoopGroup

Currently onlySortOrFilterChange returns false even when it only has sort because the columns being compared here (ViewPortColumns) are not case class and the equality check returns false currently.

  private def onlyFilterOrSortChanged(newStructuralFields: ViewPortStructuralFields, current: ViewPortStructuralFields): Boolean = {
    newStructuralFields.table.asTable.name == current.table.asTable.name && newStructuralFields.groupBy == current.groupBy && newStructuralFields.columns == current.columns
  }