Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor TestMemoryManager.testOutOfMemoryKiller #11818

Merged
merged 5 commits into from
Apr 6, 2022
Merged

Refactor TestMemoryManager.testOutOfMemoryKiller #11818

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bac55a5
Replace @Test.expectedExceptions with localized check
findepi Apr 6, 2022
6b7b0c2
Wait for a killed query in waitForQueryToBeKilled
findepi Apr 6, 2022
7959752
Remove redundant assertion
findepi Apr 6, 2022
e11d9fe
Slow down test query in testOutOfMemoryKiller
findepi Apr 6, 2022
8eca03d
Pick failing query faster in testOutOfMemoryKiller
findepi Apr 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions testing/trino-tests/src/test/java/io/trino/memory/TestMemoryManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@

import com.google.common.collect.ImmutableMap;
import io.trino.Session;
import io.trino.plugin.blackhole.BlackHolePlugin;
import io.trino.server.BasicQueryInfo;
import io.trino.server.BasicQueryStats;
import io.trino.server.testing.TestingTrinoServer;
Expand Down Expand Up @@ -115,6 +116,12 @@ public void testOutOfMemoryKiller()
.buildOrThrow();

try (DistributedQueryRunner queryRunner = createQueryRunner(TINY_SESSION, properties)) {
queryRunner.installPlugin(new BlackHolePlugin());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be because OOM killer has 5s minimal delay
This will be dropped (as does not play well with task-level retries. (part of #11800).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will the test be updated to use smaller delay? This could make the test run shorter.

Still, the change here should remain. If query can succeed in low number of seconds, this feels racy.

queryRunner.createCatalog("blackhole", "blackhole");
queryRunner.execute("" +
"CREATE TABLE blackhole.default.take_30s(dummy varchar(10)) " +
"WITH (split_count=1, pages_per_split=30, rows_per_page=1, page_processing_delay='1s')");

// Reserve all the memory
QueryId fakeQueryId = new QueryId("fake");
for (TestingTrinoServer server : queryRunner.getServers()) {
Expand All @@ -124,11 +131,14 @@ public void testOutOfMemoryKiller()

List<Future<?>> queryFutures = new ArrayList<>();
for (int i = 0; i < 2; i++) {
queryFutures.add(executor.submit(() -> queryRunner.execute("SELECT COUNT(*), clerk FROM orders GROUP BY clerk")));
queryFutures.add(executor.submit(() -> queryRunner.execute("" +
"SELECT COUNT(*), clerk " +
"FROM (SELECT clerk FROM orders UNION ALL SELECT dummy FROM blackhole.default.take_30s)" +
"GROUP BY clerk")));
}

// Wait for queries to start
assertEventually(() -> assertThat(queryRunner.getCoordinator().getQueryManager().getQueries()).hasSize(queryFutures.size()));
assertEventually(() -> assertThat(queryRunner.getCoordinator().getQueryManager().getQueries()).hasSize(1 + queryFutures.size()));

// Wait for one of the queries to die
waitForQueryToBeKilled(queryRunner);
Expand Down