Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

IndexOutOfBoundsException during running TPC-DS Q2 #211

Closed
zhztheplayer opened this issue Mar 29, 2021 · 0 comments · Fixed by #212
Closed

IndexOutOfBoundsException during running TPC-DS Q2 #211

zhztheplayer opened this issue Mar 29, 2021 · 0 comments · Fixed by #212
Labels
bug Something isn't working

Comments

@zhztheplayer
Copy link
Collaborator

java.lang.IndexOutOfBoundsException: index: 0, length: 63544 (expected: range(0, 15880))                                         
        at org.apache.arrow.memory.ArrowBuf.checkIndex(ArrowBuf.java:699)                                          
        at org.apache.arrow.memory.ArrowBuf.setBytes(ArrowBuf.java:952)                                            
        at org.apache.arrow.vector.BaseFixedWidthVector.reAlloc(BaseFixedWidthVector.java:445)                                                                                                                                               
        at org.apache.arrow.vector.BaseFixedWidthVector.setValueCount(BaseFixedWidthVector.java:725)             
        at org.apache.arrow.vector.VectorSchemaRoot.setRowCount(VectorSchemaRoot.java:244)             
        at org.apache.arrow.vector.VectorLoader.load(VectorLoader.java:63)                                         
        at com.intel.oap.vectorized.ArrowWritableColumnVector.loadColumns(ArrowWritableColumnVector.java:141)      
        at com.intel.oap.vectorized.ArrowWritableColumnVector.loadColumns(ArrowWritableColumnVector.java:131)    
        at com.intel.oap.expression.ColumnarConditionProjector$FieldOptimizedProjector.evaluate(ColumnarConditionProjector.scala:537)
        at com.intel.oap.expression.ColumnarConditionProjector$$anon$1.hasNext(ColumnarConditionProjector.scala:275)
        at com.intel.oap.vectorized.CloseableColumnBatchIterator.hasNext(CloseableColumnBatchIterator.scala:47)    
        at com.intel.oap.execution.ColumnarWholeStageCodegenExec$$anon$2.process(ColumnarWholeStageCodegenExec.scala:484)
        at com.intel.oap.execution.ColumnarWholeStageCodegenExec$$anon$2.hasNext(ColumnarWholeStageCodegenExec.scala:498)
        at com.intel.oap.vectorized.CloseableColumnBatchIterator.hasNext(CloseableColumnBatchIterator.scala:47)    
        at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:458)                                          
        at org.apache.spark.shuffle.ColumnarShuffleWriter.write(ColumnarShuffleWriter.scala:84)                  
        at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)                                                                                                                                              
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)                      
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)                            
        at org.apache.spark.scheduler.Task.run(Task.scala:127)                                                  
        at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:444)                  
        at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1377)                                       
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:447)                                   
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)                       
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
        at java.lang.Thread.run(Thread.java:748)
@zhouyuan zhouyuan added the bug Something isn't working label Mar 30, 2021
zhouyuan pushed a commit to zhouyuan/native-sql-engine that referenced this issue Mar 30, 2021
zhouyuan added a commit that referenced this issue Apr 2, 2021
* [NSE-185] Avoid unnecessary copying when simply projecting on fields (#187)

* [NSE-185] Avoid unnecessary copying when simply projecting on fields

* Avoid sharing buffers in output

* [NSE-211] IndexOutOfBoundsException during running TPC-DS Q2 (#212)

Closes #211

* fix missing decimal check (#219)

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

* [NSE-227]fix issues from codescan (#225)

* fix code scan

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

* fix code scan

* fix format

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

* fix wrong patch

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

* Klockwork issues fix

* fix format

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

* more fixes

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Co-authored-by: Rui Mo <rui.mo@intel.com>
Co-authored-by: Hongze Zhang <hongze.zhang@intel.com>

* allow to config batchsize in hashagg and wscg (#222)

Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>

Co-authored-by: Hongze Zhang <hongze.zhang@intel.com>
Co-authored-by: Rui Mo <rui.mo@intel.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants