Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](analyzer) fixed the NullPointerException #43269

Merged
merged 2 commits into from
Nov 16, 2024

Conversation

dtkavin
Copy link
Contributor

@dtkavin dtkavin commented Nov 5, 2024

stack info :

2024-11-01 14:04:24,570 INFO (replayer|118) [EditLog.loadJournal():249] Begin to unprotect add partition. db = 50402 table = 3355198 partitionName = p2024110115
2024-11-01 14:04:24,572 INFO (replayer|118) [Env.replayJournal():2795] replayed journal id is 91381595, replay to journal id is 91381596
2024-11-01 14:04:24,572 INFO (replayer|118) [EditLog.loadJournal():259] Begin to unprotect drop partition. db = 50402 table = 23712088 partitionName = p2024110112
2024-11-01 14:04:24,573 INFO (replayer|118) [CatalogRecycleBin.recyclePartition():197] recycle partition[30713663-p2024110112] of table [23712088-real_result]
2024-11-01 14:04:24,573 INFO (replayer|118) [OlapTable.updateVisibleVersionAndTime():2740] updateVisibleVersionAndTime, tableName: real_result, visibleVersion, 1653, visibleVersionTime: 1730441064570
2024-11-01 14:04:24,573 WARN (mysql-nio-pool-109891|1159519) [StmtExecutor.executeByLegacy():985] execute Exception. stmt[76210227, 9822ba51b0b14b33-b8c313b570b26832]
java.lang.NullPointerException: null
        at org.apache.doris.qe.cache.CacheAnalyzer.buildCacheTableForOlapScanNode(CacheAnalyzer.java:705) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.cache.CacheAnalyzer.buildCacheTableList(CacheAnalyzer.java:516) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.cache.CacheAnalyzer.innerCheckCacheMode(CacheAnalyzer.java:259) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.cache.CacheAnalyzer.getCacheData(CacheAnalyzer.java:528) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleCacheStmt(StmtExecutor.java:1615) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleQueryStmt(StmtExecutor.java:1721) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleQueryWithRetry(StmtExecutor.java:810) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:903) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:597) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:523) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:328) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:260) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:288) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:341) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_272]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_272]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_272]
2024-11-01 14:04:24,578 INFO (replayer|118) [Env.replayJournal():2795] replayed journal id is 91381596, replay to journal id is 91381597
2024-11-01 14:04:24,578 INFO (replayer|118) [EditLog.loadJournal():249] Begin to unprotect add partition. db = 50402 table = 23712088 partitionName = p2024110115
2024-11-01 14:04:24,580 INFO (replayer|118) [Env.replayJournal():2795] replayed journal id is 91381597, replay to journal id is 91381598
2024-11-01 14:04:24,580 INFO (replayer|118) [EditLog.loadJournal():259] Begin to unprotect drop partition. db = 50402 table = 67267 partitionName = p2024110111
# partition get null here 
Partition partition = olapTable.getPartition(partitionId);

# Unable to obtain the specified partition in idToPartition and tempPartitions through partitionId
public Partition getPartition(long partitionId) {
        Partition partition = idToPartition.get(partitionId);
        if (partition == null) {
            partition = tempPartitions.getPartition(partitionId);
        }
        return partition;
}

Because there is reading and writing, the partition should have already been deleted when reading again

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@dtkavin
Copy link
Contributor Author

dtkavin commented Nov 5, 2024

run buildall

@dtkavin
Copy link
Contributor Author

dtkavin commented Nov 6, 2024

run buildall

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Nov 16, 2024
Copy link
Contributor

PR approved by anyone and no changes requested.

@924060929
Copy link
Contributor

Thanks @dtkavin, can you cherry pick this pr to branch master/branch-3.0/branch-2.0?

@yiguolei yiguolei merged commit 44ed5d6 into apache:branch-2.1 Nov 16, 2024
21 of 22 checks passed
dtkavin added a commit to dtkavin/doris that referenced this pull request Nov 18, 2024
dtkavin added a commit to dtkavin/doris that referenced this pull request Nov 18, 2024
dataroaring pushed a commit that referenced this pull request Nov 18, 2024
924060929 pushed a commit that referenced this pull request Nov 18, 2024
924060929 pushed a commit that referenced this pull request Nov 18, 2024
cherry-pick: #43269

stack info :
```
2024-11-01 14:04:24,570 INFO (replayer|118) [EditLog.loadJournal():249] Begin to unprotect add partition. db = 50402 table = 3355198 partitionName = p2024110115
2024-11-01 14:04:24,572 INFO (replayer|118) [Env.replayJournal():2795] replayed journal id is 91381595, replay to journal id is 91381596
2024-11-01 14:04:24,572 INFO (replayer|118) [EditLog.loadJournal():259] Begin to unprotect drop partition. db = 50402 table = 23712088 partitionName = p2024110112
2024-11-01 14:04:24,573 INFO (replayer|118) [CatalogRecycleBin.recyclePartition():197] recycle partition[30713663-p2024110112] of table [23712088-real_result]
2024-11-01 14:04:24,573 INFO (replayer|118) [OlapTable.updateVisibleVersionAndTime():2740] updateVisibleVersionAndTime, tableName: real_result, visibleVersion, 1653, visibleVersionTime: 1730441064570
2024-11-01 14:04:24,573 WARN (mysql-nio-pool-109891|1159519) [StmtExecutor.executeByLegacy():985] execute Exception. stmt[76210227, 9822ba51b0b14b33-b8c313b570b26832]
java.lang.NullPointerException: null
        at org.apache.doris.qe.cache.CacheAnalyzer.buildCacheTableForOlapScanNode(CacheAnalyzer.java:705) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.cache.CacheAnalyzer.buildCacheTableList(CacheAnalyzer.java:516) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.cache.CacheAnalyzer.innerCheckCacheMode(CacheAnalyzer.java:259) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.cache.CacheAnalyzer.getCacheData(CacheAnalyzer.java:528) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleCacheStmt(StmtExecutor.java:1615) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleQueryStmt(StmtExecutor.java:1721) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleQueryWithRetry(StmtExecutor.java:810) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:903) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:597) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:523) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:328) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:260) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:288) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:341) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_272]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_272]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_272]
2024-11-01 14:04:24,578 INFO (replayer|118) [Env.replayJournal():2795] replayed journal id is 91381596, replay to journal id is 91381597
2024-11-01 14:04:24,578 INFO (replayer|118) [EditLog.loadJournal():249] Begin to unprotect add partition. db = 50402 table = 23712088 partitionName = p2024110115
2024-11-01 14:04:24,580 INFO (replayer|118) [Env.replayJournal():2795] replayed journal id is 91381597, replay to journal id is 91381598
2024-11-01 14:04:24,580 INFO (replayer|118) [EditLog.loadJournal():259] Begin to unprotect drop partition. db = 50402 table = 67267 partitionName = p2024110111
```

```
# partition get null here 
Partition partition = olapTable.getPartition(partitionId);

# Unable to obtain the specified partition in idToPartition and tempPartitions through partitionId
public Partition getPartition(long partitionId) {
        Partition partition = idToPartition.get(partitionId);
        if (partition == null) {
            partition = tempPartitions.getPartition(partitionId);
        }
        return partition;
}
```

Because there is reading and writing, the partition should have already been deleted when reading again
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.x dev/2.1.8-merged dev/3.0.3-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants