Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove NumPy argument support for dpex kernels #866

Closed
wants to merge 55 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
f4d916c
Move passbuilder into core.
diptorupd Oct 22, 2022
7c626b5
Add a compiler module into core.
diptorupd Oct 22, 2022
077ca33
Adds an arg_pack_unpack module to kernel_interface
diptorupd Oct 22, 2022
d8f81a6
Change exception behavior
diptorupd Oct 26, 2022
3ab688f
Refactored kernel dispatch API.
diptorupd Oct 18, 2022
8e3e18a
Temp commit to add a driver.py to run refactored code base.
diptorupd Oct 24, 2022
0c7374f
Improve exceptions.
diptorupd Oct 28, 2022
80fa3f7
Update the temporary driver.
diptorupd Nov 4, 2022
b362b76
Added global range checks, kernel now uses dispatcher.
diptorupd Nov 4, 2022
b69aa50
Fix the pack/repack of Numpy arrays
diptorupd Nov 8, 2022
72ec9cc
Update existing compute follows data unit test.
diptorupd Nov 8, 2022
59df7b9
Fix failing test_sycl_usm_array_iface_interop tests.
diptorupd Nov 8, 2022
b4ab99f
Rewrite test to use refactored API.
diptorupd Nov 8, 2022
f503061
Update tests to check DI tag generation.
diptorupd Nov 9, 2022
0eafa3f
Fix to address failing unit test for strided numpy array support.
diptorupd Nov 9, 2022
038e593
move pass pipelines into compiler.py
diptorupd Nov 20, 2022
8bc0c0c
Incorporate review comments.
diptorupd Nov 20, 2022
5dd94c8
Port func decorator to new API.
diptorupd Nov 21, 2022
973f41b
Migrate functionality in numba_dpex.compiler module to new compiler.
diptorupd Dec 7, 2022
d732b49
Mark the caching tests as xfail.
diptorupd Dec 7, 2022
f1bcac3
A new LRU cache backed by numba's pickling mechanism
chudur-budur Dec 8, 2022
a54c23a
Added rst for caching
chudur-budur Dec 13, 2022
667d081
Fix type import.
diptorupd Dec 19, 2022
28cff46
Bring back support for kernel specialization.
diptorupd Dec 27, 2022
1ddbab2
Update driver.py with specialization.
diptorupd Jan 5, 2023
8726ad7
Fix driver
diptorupd Jan 6, 2023
2468b97
Fix formatting issues.
diptorupd Jan 6, 2023
99f51e7
Improvements to USMNdArrayType.
diptorupd Jan 6, 2023
7e1c102
Remove the function signature from the cache composite key.
diptorupd Jan 6, 2023
be7fe42
Add unit tests for kernel specialization.
diptorupd Jan 6, 2023
fd7c370
Mark all vectorize tests as xfail.
diptorupd Jan 6, 2023
292c57a
Improve support for SUAI arrays as kernel arguments.
diptorupd Jan 7, 2023
f5fa531
Fix failing test_barrier tests and add checks to disallow specializat…
diptorupd Jan 7, 2023
cb15e0a
Fix test_kernel_has_return_value_error test case.
diptorupd Jan 7, 2023
6ef0637
Address review comments.
diptorupd Jan 8, 2023
5e15920
Use new compiler to compiler parfors.
diptorupd Jan 10, 2023
d183c7a
Fully remove numba_dpex.compiler module.
diptorupd Jan 7, 2023
c7d1270
Remove the temporary driver.py file.
diptorupd Jan 10, 2023
1d65949
Added ndarray setup check.
mingjie-intel Jan 9, 2023
1c9dd0f
Added tests for ndrange exceptions.
mingjie-intel Jan 10, 2023
479197b
Add an example for aot kernel specialization.
diptorupd Jan 10, 2023
1bc8cdd
Fix deprecation warnings.
diptorupd Jan 10, 2023
3dd71c8
Switched to dpctl.tensor in test_ndrange_exceptions.py.
mingjie-intel Jan 10, 2023
814762b
Clean up numpy array.
mingjie-intel Jan 11, 2023
db37ef1
Clean up numpy array.
mingjie-intel Jan 11, 2023
dd47913
Add docsstrings.
diptorupd Jan 11, 2023
057f470
Switched to dpctl.tensor in test_ndrange_exceptions.py.
mingjie-intel Jan 10, 2023
0a70dd9
Clean up more NumPy array.
mingjie-intel Jan 11, 2023
ed74dd5
Clean up NumPy array.
mingjie-intel Jan 12, 2023
7d4bee9
Changes to compute-follows-data implementation and specialization.
diptorupd Jan 11, 2023
63acaee
Rename AOT to eager compilation.
diptorupd Jan 13, 2023
ac496ac
Merge branch 'refactor/kernel_interfaces' of https://github.com/Intel…
mingjie-intel Jan 13, 2023
7e077d0
WIP changes for range and ndrange.
diptorupd Jan 16, 2023
ce00025
Merge branch 'refactor/kernel_interfaces' of https://github.com/Intel…
mingjie-intel Jan 17, 2023
3cdd4f6
Merge branch 'refactor/kernel_interfaces' into numpy_removal
mingjie-intel Jan 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/developer_guides/caching.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
.. _caching:

Caching Mechanism in Numba-dpex
================================

Caching is done by saving the compiled kernel code, the ELF object of the executable code. By using the kernel code, cached kernels have minimal overhead because no compilation is needed.

Unlike Numba, we do not perform file-based caching, instead we use an Least Recently Used (LRU) caching mechanism. However when a kernel needs to be evicted, we utilize numba's file-based caching mechanism described `here <https://numba.pydata.org/numba-doc/latest/developer/caching.html>`_.

Algorithm
==========

The caching mechanism for Numba-dpex works as follows: The cache is an LRU cache backed by an ordered dictionary mapped onto a doubly linked list. The tail of the list contains the most recently used (MRU) kernel and the head of the list contains the least recently used (LRU) kernel. The list has a fixed size. If a new kernel arrives to be cached and if the size is already on the maximum limit, the algorithm evicts the LRU kernel to make room for the MRU kernel. The evicted item will be serialized and pickled into a file using Numba's caching mechanism.

Everytime whenever a kernel needs to be retrieved from the cache, the mechanism will look for the kernel in the cache and will be loaded if it's already present. However, if the program is seeking for a kernel that has been evicted, the algorithm will load it from the file and enqueue in the cache.

Settings
========

Therefore, we employ similar environment variables as used in Numba, i.e. ``NUMBA_CACHE_DIR`` etc. However we add three more environment variables to control the caching mechanism.

- In order to specify cache capacity, one can use ``NUMBA_DPEX_CACHE_SIZE``. By default, it's set to 10.
- ``NUMBA_DPEX_ENABLE_CACHE`` can be used to enable/disable the caching mechanism. By default it's enabled, i.e. set to 1.
- In order to enable the debugging messages related to caching, one can set ``NUMBA_DPEX_DEBUG_CACHE`` to 1. All environment variables are defined in :file:`numba_dpex/config.py`.
3 changes: 3 additions & 0 deletions numba_dpex/codegen.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ def _init(self, llvm_module):
assert list(llvm_module.global_variables) == [], "Module isn't empty"
self._data_layout = SPIR_DATA_LAYOUT[utils.MACHINE_BITS]
self._target_data = ll.create_target_data(self._data_layout)
self._tm_features = (
"" # We need this for chaching, not sure about this value for now
)

def _create_empty_module(self, name):
ir_module = lc.Module(name)
Expand Down
Loading