Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache files library and CLI tool #266

Open
wants to merge 259 commits into
base: master
Choose a base branch
from
Open

Cache files library and CLI tool #266

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
259 commits
Select commit Hold shift + click to select a range
d73c699
Remove old jsonschema directory and move tests
warnio Feb 26, 2024
4dfcbbd
Add test cache files
warnio Feb 26, 2024
49ab95d
Make cache paths glob more general
warnio Feb 26, 2024
c51b155
Remove version folders
warnio Feb 26, 2024
717b100
Update paths for test caches
warnio Feb 26, 2024
f61bb4c
Correct valid cache example
warnio Feb 26, 2024
045d266
Unignore JSON schema files
warnio Feb 26, 2024
29afd28
Improve cache file glob matching
warnio Feb 27, 2024
2044e78
Reorder a few functions
warnio Feb 27, 2024
ed877ac
Reorder a few lines
warnio Feb 27, 2024
30eb173
Loading Cachefiles and added test.py script
rmgerritse Feb 28, 2024
b5e555d
Combined read and write into KTLibrary.
rkcash Feb 29, 2024
caae7b3
Do not ignore json schema files
TobiasHofstra Mar 1, 2024
f21fb32
Basic example cache file conversion
TobiasHofstra Mar 1, 2024
7239e43
Added testing files, using prefixes
cesarg1337 Mar 4, 2024
2d0a6ca
Removed unneccessary files
cesarg1337 Mar 4, 2024
45080e1
Rename cache files and remove redundant ones
warnio Mar 5, 2024
76f7429
Ensure timestamp is a datetime
warnio Mar 5, 2024
63c0918
Added support for default conversions; need to implement default conv…
Mar 6, 2024
d9de155
Added a default conversion function and default values
Mar 6, 2024
22889a0
Update .gitignore and a couple lines changed in convert_cache_file()
Mar 6, 2024
d62b57c
Don't sort json output in default conversion by key
Mar 6, 2024
338566c
Add typehints
warnio Mar 7, 2024
f3d12ef
Improve schema
warnio Mar 7, 2024
d98ba80
Simplify tests
warnio Mar 7, 2024
dc1e06e
Make test names more descriptive
warnio Mar 7, 2024
0a49dfc
Improve path resolution
warnio Mar 7, 2024
848ac31
Remove unnecessary type hint
warnio Mar 7, 2024
4f3d07f
Improvetest and rename CacheEntryJSON to CacheLineJSON
warnio Mar 7, 2024
5b923da
Updated convert.py so that cache conversion functions use a cache dic…
Mar 7, 2024
34564a6
conver cache test now works
TobiasHofstra Mar 8, 2024
b691419
Add version property to JSON-schema
tdejong00 Mar 9, 2024
7dcf52a
Add test for schema version
tdejong00 Mar 9, 2024
ba5e6ba
Merge pull request #1 from tdejong00/jsonschema
tdejong00 Mar 9, 2024
36db603
A couple changes, almost ready to merge
Mar 11, 2024
642dc61
Changed .gitignore ahead of pull request
TobiasHofstra Mar 12, 2024
fd537fb
Final version before merge
Mar 13, 2024
4dcdaf2
Merge branch 'master' into convertcache
TobiasHofstra Mar 13, 2024
a4006be
Created an json encoder for the custom format we use.
rkcash Mar 13, 2024
310638c
Remove old test and prepare for new test writing
rmgerritse Mar 15, 2024
a62b667
Test for load and store cachefile
rmgerritse Mar 15, 2024
154c479
Merge pull request #2 from tdejong00/jsonschema-versioning
tdejong00 Mar 18, 2024
5cb80b2
Add tests for additional properties
warnio Mar 23, 2024
4165bf2
Processed comments and requested changes from Pull Request
Mar 24, 2024
81268b3
Added some comments.
rkcash Mar 25, 2024
fa9342d
Added multiple new tests, added test support for default_convert, cha…
Mar 26, 2024
97a0b59
Merge pull request #3 from tdejong00/convertcache
TobiasHofstra Mar 26, 2024
91b2e5c
PR request fixes. Renames and remove catching exceptions (up to the u…
rkcash Mar 28, 2024
7e09f8b
Paths file now relative, deleted test file and rename
rmgerritse Apr 2, 2024
30d2b34
Added 'cachefile' directory in 'kernel_tuner' and utilized 'tmp_path'…
rmgerritse Apr 3, 2024
0159ad1
KTLibrary no longer a class, renamed cachefile folder to cache and ed…
rakesh37187 Apr 4, 2024
901afa6
Added comments and fixed test file
rmgerritse Apr 3, 2024
2ec9321
Merge pull request #4 from tdejong00/jsonschema-test-edge-cases
tdejong00 Apr 5, 2024
535c026
Update README.MD
rmgerritse Apr 5, 2024
23850ad
Move SampleCacheFiles to test folder
rmgerritse Apr 5, 2024
40e313e
Merge branch 'cachefiles' of https://github.com/tdejong00/kernel_tune…
rmgerritse Apr 5, 2024
a420a7f
Merge pull request #5 from tdejong00/cachefiles
rmgerritse Apr 8, 2024
eb1ded2
Add conversion from cache file to T4 format
tdejong00 Apr 8, 2024
db999d9
Move read/write function to a different location
warnio Apr 11, 2024
2d5db60
Rename file
warnio Apr 11, 2024
2450618
Remove KTLibrary folder
warnio Apr 11, 2024
8aed003
Ignore docstring requirement in tests
warnio Apr 11, 2024
ec67b59
Make code conform with Ruff project config
warnio Apr 11, 2024
c868de8
Create global variable for sample cache file
warnio Apr 15, 2024
982a266
Test reading open cache files
warnio Apr 15, 2024
ecbaef8
Fix cache file reading
warnio Apr 15, 2024
db3a3d5
Rename variable
warnio Apr 15, 2024
3bbb040
Remove newline at EOF
warnio Apr 15, 2024
a8213e2
Add close cache file functionality
warnio Apr 15, 2024
413571c
Add functionality for opening a cache file
warnio Apr 15, 2024
07ce347
Rename function
warnio Apr 15, 2024
7244dc4
Implement appending cache lines to cache files
warnio Apr 16, 2024
8f689be
Ensure open or close cache files in read_cache_file via keyword args
warnio Apr 16, 2024
8e130e2
Simplify calls marked with parametrize
warnio Apr 16, 2024
1b86ca2
Improve function names and documentation
warnio Apr 16, 2024
4f0d580
Create separate functions for ensuring open and closed cache files
warnio Apr 16, 2024
6b1a9bd
Assert read_cache_file does not change the file being read by default
warnio Apr 16, 2024
e4e24fb
Don't allow ensure_open and ensure_closed to be simultaneously set
warnio Apr 16, 2024
78476b3
Simplify cache file function names
warnio Apr 16, 2024
8e357c1
Fix closing improperly opened cache files
warnio Apr 16, 2024
bc0dbb5
Add comments
warnio Apr 16, 2024
504431f
Cover additional case where cache is invalid
warnio Apr 16, 2024
148c013
Add custom error type
warnio Apr 16, 2024
10acd86
Rename tests to be more descriptive
warnio Apr 16, 2024
ac1eac5
Cleanup unneeded file
warnio Apr 16, 2024
ea5afd4
Invoke super
warnio Apr 16, 2024
5aa707a
Fix ErrorConfig bug
warnio Apr 16, 2024
1747022
Add tests and encode branch for empty list
warnio Apr 16, 2024
8af5249
Add support for encoding non-json objects
warnio Apr 16, 2024
8550037
Minor changes to T4 conversion according to client meeting
tdejong00 Apr 18, 2024
46c4870
Test encode cache last
warnio Apr 18, 2024
214a697
Add test for appending cache lines to empty cache
warnio Apr 18, 2024
06e8e47
Simplify reading and writing cache files
warnio Apr 18, 2024
f785227
Restructure code and simplify append_cache_line function signature
warnio Apr 18, 2024
67dd388
Add important documentation to function append_cache_line
warnio Apr 18, 2024
9c513d4
Added unversioned conversion and test for it
Apr 20, 2024
46733d3
Rename 'mock' to 'real' in test_convert_real
Apr 21, 2024
6fd8caf
Reformatted the docstrings in convert.py
Apr 21, 2024
645a4af
Indentation fix
Apr 21, 2024
5d9f994
Add TODO for using our custom encoder
Apr 21, 2024
8354d0e
Add cache creation functionality
warnio Apr 22, 2024
94f856f
Implement reading existing cache files
warnio Apr 22, 2024
9cf9beb
Improve test
warnio Apr 22, 2024
bf79ec3
Add extra test for reading and writing cache files
warnio Apr 22, 2024
f2350eb
Return native Python objects when reading cache
warnio Apr 23, 2024
5df79e2
Implement appending cache lines and fix bug
warnio Apr 23, 2024
36e69af
Add filepath property to Cache
warnio Apr 23, 2024
5e55703
Reserve a subset of tune_param_keys
warnio Apr 23, 2024
7dde4d5
Raise error when tunable param has invalid value
warnio Apr 23, 2024
29411ef
Add test cases for getting a default from Cache.Lines
warnio Apr 23, 2024
0f26a75
Remove print statement
warnio Apr 23, 2024
28b4b5c
Add type verification for appending lines
warnio Apr 23, 2024
c0d6f60
Remove frozendict and use of tuple in properties
warnio Apr 23, 2024
79ce240
Improve readability
warnio Apr 23, 2024
7981159
Rename CacheLinePosition
warnio Apr 23, 2024
a81b283
Slightly alter type signature to only accept lists
warnio Apr 23, 2024
1050219
Add JSON validation
warnio Apr 23, 2024
c37a63b
Extract modules for readability
warnio Apr 23, 2024
da2b7b5
Remove automatic conversion of cache files in cache library
warnio Apr 23, 2024
ba98da5
Remove comments
warnio Apr 23, 2024
67b16c4
Remove unused import
warnio Apr 25, 2024
573891f
Improve documentation
warnio Apr 25, 2024
bbdb2fd
Improve code readability
warnio Apr 25, 2024
fe91990
Return json serializable objects from __getitem__
warnio Apr 25, 2024
9c39284
Replace printf by print
warnio Apr 26, 2024
7b68a1a
Make validation more strict
warnio Apr 26, 2024
2c3f5ea
Improve documentation and naming
warnio Apr 26, 2024
bbdf428
Merge pull request #6 from tdejong00/kt-library
warnio Apr 26, 2024
f065e53
Merge branch 'master' into t4-conversion
tdejong00 May 1, 2024
d82c347
Add and use classes representing the T4 file.
tdejong00 May 1, 2024
ec536f1
Added unversioned conversion and test for it
Apr 20, 2024
05585c8
Rename 'mock' to 'real' in test_convert_real
Apr 21, 2024
54c6924
Reformatted the docstrings in convert.py
Apr 21, 2024
8b14156
Indentation fix
Apr 21, 2024
8ca0f41
Add TODO for using our custom encoder
Apr 21, 2024
ca2a9de
Merge branch 'unversioned-convert' of github.com:tdejong00/kernel_tun…
TobiasHofstra May 2, 2024
0967528
Remove empty return statements
TobiasHofstra May 2, 2024
12d6625
Conversion now uses VERSIONS from cache.version
May 2, 2024
b1a13b2
Conversion now uses CACHE_SCHEMAS_DIR from cache.paths
May 2, 2024
0d083c8
Conversion now uses read_cache and write_cache from cache.file
May 2, 2024
3a17722
Fix problem size being a required property
tdejong00 May 7, 2024
1c44802
Fix incorrect schema types for some time properties
tdejong00 May 7, 2024
4c32df4
Remove unnecessary raw string literal
tdejong00 May 7, 2024
b87aaea
Refactor cache util functions to make use of cache library
tdejong00 May 7, 2024
cb8ecd8
removed key regex and list made for input paramters
rmgerritse May 10, 2024
8b97ec5
Added staticmethod above some functions which are static
rakesh37187 May 12, 2024
b11ae8e
Bind functions in _CONVERT_FUNCTIONS through the __func__ attribute
May 12, 2024
7bf8a5a
Merge pull request #7 from tdejong00/unversioned-convert
TobiasHofstra May 12, 2024
9e39a16
Update: added basic cli functionality. Now merging seems to work, and…
cesarg1337 May 13, 2024
a02b598
Update: improved imports and changed merging function
cesarg1337 May 13, 2024
c484f13
Add description to cache file JSON schema and added Allowed enum valu…
rmgerritse May 13, 2024
98d02ad
Validator of jsonschema and timestamp
rmgerritse May 14, 2024
bc82d70
Schema version 1.1.0
rmgerritse May 14, 2024
8809a64
test files to validate cachefiles
rmgerritse May 14, 2024
d49633f
added removing and improved functionality
cesarg1337 May 16, 2024
492d837
Added test for validate function
rmgerritse May 16, 2024
d6805b1
Added comments and changed comments to English
rmgerritse May 16, 2024
9ec455b
Merge branch 'master' into t4-conversion
tdejong00 May 16, 2024
99058b7
Remove unnecessary return statement
tdejong00 May 16, 2024
4d12c4c
Merge pull request #8 from tdejong00/t4-conversion
tdejong00 May 16, 2024
9ef3336
Restore file
rmgerritse May 17, 2024
5393ece
Comments processed
rmgerritse May 17, 2024
15ab06d
Big update to cli; now basic functionalities; merge, delete-line, get…
cesarg1337 May 18, 2024
d610590
Updated docstrings and fixed minor bug
cesarg1337 May 18, 2024
93b9f6a
Fix for python 3.8
May 18, 2024
adb6775
Added working conversion to cli
May 18, 2024
c3523a3
Merge branch 'master' into cli
May 18, 2024
7daecfc
Merge branch 'master' into integrate-library
tdejong00 May 19, 2024
ed60385
Added target_version functionality to convert_cache_file
May 19, 2024
bc87072
Added target_version functionality to the CLI; some bug fixes and che…
May 19, 2024
ead04d2
Fix incorrect type for problem size property
tdejong00 May 19, 2024
291dec0
Make problem size property required
tdejong00 May 19, 2024
f62cb26
Add schema description
tdejong00 May 19, 2024
d69ca0b
Allow string type for problem size property
tdejong00 May 19, 2024
e133b89
Remove unnecessary None check for required property
tdejong00 May 19, 2024
60f6186
Added T4 conversion to the CLI
May 19, 2024
aaf2e55
Refactored cli code in preparation for adding tests
May 19, 2024
82f883a
Added cli test file to define test structure for the CLI
May 19, 2024
4e98077
Fixed type checking in Cache.Lines.append for verification_time and s…
rakesh37187 May 20, 2024
224b353
Merge branch 'integrate-library' of https://github.com/tdejong00/kern…
rakesh37187 May 20, 2024
b9ef7ca
Changed 1.0.0 to 1.0.1 since patch should be bumped. Restored 1.0.0 t…
rakesh37187 May 20, 2024
4a61193
Bool is an instance of int hence we need to verify whether it is a bo…
rakesh37187 May 20, 2024
4068264
Changed description json schema
rmgerritse May 21, 2024
2e86246
Update to cli: improved docstrings, implemented Dylan's comments, upd…
cesarg1337 May 21, 2024
f89b142
Removed useless import.
cesarg1337 May 21, 2024
2f8eab6
Merge pull request #9 from tdejong00/jsonschema1.1.0
rmgerritse May 22, 2024
2314950
Minor update: added merging check for multiple equivalent keys for en…
cesarg1337 May 23, 2024
fb2d7cb
Changed argparse parameter names, and changed deletion function, impr…
cesarg1337 May 23, 2024
4eb79a7
Merge branch 'master' into cli
May 28, 2024
a11a3fd
Reformed cli/cli_functionality. Improved usage, added more tests, fix…
cesarg1337 May 28, 2024
c030ba4
Removed (now) useless cli_fct.py
cesarg1337 May 28, 2024
c214dd3
Updated cli: delete-line now only works with latest version, added mo…
cesarg1337 May 29, 2024
731c9c1
Updated docstrings and removed (now useless) cli.py file in cache/cli…
cesarg1337 May 30, 2024
703e96b
Fix incorrect version string for 1.0.1 schema
tdejong00 May 30, 2024
25a6387
Merge branch 'master' into integrate-library
tdejong00 May 30, 2024
03b291a
Fix incorrect schema definitions
tdejong00 May 30, 2024
9a10df2
Merge pull request #10 from tdejong00/integrate-library
tdejong00 May 31, 2024
880b939
Reformatted some code to be more consistent with itself and the rest …
May 31, 2024
7f7fca1
Fixes to test_cli.py, and a couple exception name changes
Jun 2, 2024
cc41caf
Added the remaining tests, fixed some small errors
Jun 2, 2024
5ffe6a7
Changed to Dylan's requirements except for pip install fix, merge cha…
cesarg1337 Jun 3, 2024
e813dca
Changed function name to convert_files_to_their_least_common_version …
cesarg1337 Jun 3, 2024
ce6c97d
Ensure script works
warnio Jun 3, 2024
82f1680
Fix undefined `cache` in Python 3.8
warnio Jun 3, 2024
fc89275
Format file and fix error by addin typing
warnio Jun 3, 2024
4a6b1f9
Use more general key format
warnio Jun 3, 2024
cb0dc52
Raise a key error in case a key exists
warnio Jun 3, 2024
64eccaf
Reformat test_cli and fix import
warnio Jun 3, 2024
cb4788a
Fix Cache.Lines not returning Cache.Line and add type hints
warnio Jun 3, 2024
4833d72
Move cli test script
warnio Jun 3, 2024
8214a30
Simplify merging
warnio Jun 3, 2024
6a7ab1e
Import undefined type hint
warnio Jun 3, 2024
442e741
Rename read to load
warnio Jun 3, 2024
50be895
Add readonly variant of Cache
warnio Jun 3, 2024
503ebfe
Enable reading cache files of older versions
warnio Jun 4, 2024
81350fb
Improve delete line in CLI
warnio Jun 4, 2024
8f13aab
Improve validation
warnio Jun 4, 2024
fccb2db
Added unversioned functionality to conversion and updated epilog and …
cesarg1337 Jun 4, 2024
415b571
Removed example usages from docstrings.
cesarg1337 Jun 4, 2024
32df788
Changed help message of parser.
cesarg1337 Jun 4, 2024
a317f34
Rename `Cache.load` to `Cache.open`
warnio Jun 4, 2024
6f01d0d
Merge branch 'master' into cli
warnio Jun 6, 2024
77007c5
Fix warnings/errors in imports
warnio Jun 6, 2024
f938225
Fix bug where code tried to mutate an immutable Cache.Line object
warnio Jun 6, 2024
1f8a782
Merge pull request #11 from tdejong00/cli
tdejong00 Jun 10, 2024
9eacf02
Make latest schema version the new 1.0.0
tdejong00 Jun 10, 2024
308cc34
make unversioned convert a bit more permissive
benvanwerkhoven Jun 18, 2024
d284c35
merge master into cache_files
benvanwerkhoven Jun 19, 2024
d1d99b4
sonarcube does not like hack to check for NaN
benvanwerkhoven Jun 19, 2024
ccc580c
fix various issues reported by sonarcube
benvanwerkhoven Jun 19, 2024
3617b06
NumPy 2.0 compliant np.inf
benvanwerkhoven Jun 19, 2024
2735ff8
NumPy 2.0 compliant np.inf
benvanwerkhoven Jun 19, 2024
e9f7c34
support NumPy 2.0 at some later stage
benvanwerkhoven Jun 19, 2024
82e2c30
fix typo
benvanwerkhoven Jun 19, 2024
5e4351a
make sure test cache has cache field
benvanwerkhoven Jun 19, 2024
c2836c2
more informative error message
benvanwerkhoven Jun 20, 2024
3c49578
fix compatibility issues between sequential runner and new cache library
benvanwerkhoven Jun 20, 2024
9468339
change behavior wrt read-only caches
benvanwerkhoven Jun 20, 2024
dbc811e
update test read-only cache
benvanwerkhoven Jun 20, 2024
f9a55f4
remove test with very large cache file
benvanwerkhoven Jun 20, 2024
4be4a04
rewrote cache json encoder
benvanwerkhoven Jun 24, 2024
b91d954
fix sonarcube issues wrt f-string usage
benvanwerkhoven Jun 24, 2024
c049903
simplified code
benvanwerkhoven Jun 24, 2024
1c39812
fix for python 3.9
benvanwerkhoven Jun 24, 2024
6a7581d
simplify code, delegate checks to jsonschema.validate
benvanwerkhoven Jun 28, 2024
63ce10c
fix minor codestyle issues
benvanwerkhoven Jun 28, 2024
4e599ec
clean up code
benvanwerkhoven Jun 28, 2024
2458357
removed unused key argument in append_cache_line
benvanwerkhoven Jun 28, 2024
d0999ed
fix sonarcube issues
benvanwerkhoven Jun 28, 2024
6ef20cf
handle invalid configurations when converting to T4
benvanwerkhoven Oct 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ deploy_key
temp_*.*
.python-version
.nox
.venv

### Visual Studio Code ###
!.vscode/settings.json
Expand All @@ -37,4 +38,11 @@ temp_*.*
.LSOverride

.vscode
.idea
.idea

### JSON Schemas ###
!kernel_tuner/schema/**/*.json

### Tests ###
!test/test_cache_files/**/*.json
!test/test_convert_files/**/*.json
Empty file added kernel_tuner/cache/__init__.py
View file
Open in desktop
Empty file.
Loading