Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expire PageFunction and CursorProcessor caches by time #11434

Closed
wants to merge 1 commit into from

Conversation

guyco33
Copy link
Member

@guyco33 guyco33 commented Mar 11, 2022

Description

Set cache expiration time for generated PageFunction classes and CursorProcessor classes.

Related issues, pull requests, and links

Relates to #11234 and #11358

Documentation

(x) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.

Release notes

() No release notes entries required.
(x) Release notes entries required with the following suggested text:

# Section
* Set cache expiration time for generated PageFunction classes and CursorProcessor classes. ([{issue}`11234`]

@cla-bot cla-bot bot added the cla-signed label Mar 11, 2022
@guyco33
Copy link
Member Author

guyco33 commented Mar 13, 2022

@dain
Adding expiration time for PageFunctionCompiler cache and for ExpressionCompiler cache didn't help.
Generated classes still retained and not collected after 1 hour.
That's the retained classes found now on cluster running queries from 2 days ago.

$ jcmd trino VM.class_hierarchy  | egrep 'io[.]trino[.][$]gen[.].+[0-9]{8,8}[_][0-9]{6,6}[_][0-9]+[/]' | awk -F_ '{arr[$1]++}END{for (a in arr) if (arr[a] > 100) {print a, arr[a]}}' | sort
|--io.trino.$gen.CursorProcessor 3256
|--io.trino.$gen.PageFilter 2548
|--io.trino.$gen.PageProjectionWork 5928


$ jcmd trino VM.class_hierarchy  | egrep 'io[.]trino[.][$]gen[.]CursorProcessor' | sort | more
|--io.trino.$gen.CursorProcessor_20220311_210106_21/0x00007f897407d0f0
|--io.trino.$gen.CursorProcessor_20220311_210106_22/0x00007f89681d7e50
|--io.trino.$gen.CursorProcessor_20220311_210107_34/0x00007f898c07d000
...
...
|--io.trino.$gen.CursorProcessor_20220312_000455_11883/0x00007f89485bd570
|--io.trino.$gen.CursorProcessor_20220312_000455_11886/0x00007f89743f5270
|--io.trino.$gen.CursorProcessor_20220312_000456_11888/0x00007f89702ede80


$ jcmd trino VM.class_hierarchy  | egrep 'io[.]trino[.][$]gen[.]PageFilter' | sort | more
|--io.trino.$gen.PageFilter_20220311_210107_36/0x00007f898c083d80
|--io.trino.$gen.PageFilter_20220311_210107_37/0x00007f896847a490
|--io.trino.$gen.PageFilter_20220311_210109_55/0x00007f896c0ce150
...
...
|--io.trino.$gen.PageFilter_20220312_000455_11882/0x00007f89589025e0
|--io.trino.$gen.PageFilter_20220312_000455_11887/0x00007f89743f73d0
|--io.trino.$gen.PageFilter_20220312_000456_11889/0x00007f8970303ad0

@bitsondatadev
Copy link
Member

👋 @guyco33 - this PR has become inactive. If you're still interested in working on it, please let us know, and we can try to get reviewers to help with that.

We're working on closing out old and inactive PRs, so if you're too busy or this has too many merge conflicts to be worth picking back up, we'll be making another pass to close it out in a few weeks.

@guyco33 guyco33 closed this Jan 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

2 participants