From 1901b7361891fc7235b3bb8e80c89130cfe2c91d Mon Sep 17 00:00:00 2001 From: Chris Sidebottom Date: Mon, 9 May 2022 15:29:00 +0100 Subject: [PATCH 1/4] Add Target Pre-processing RFC --- rfcs/0070-target-preprocessing.md | 208 ++++++++++++++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 rfcs/0070-target-preprocessing.md diff --git a/rfcs/0070-target-preprocessing.md b/rfcs/0070-target-preprocessing.md new file mode 100644 index 00000000..796b11b8 --- /dev/null +++ b/rfcs/0070-target-preprocessing.md @@ -0,0 +1,208 @@ +- Feature Name: target-architecture-preprocessor +- Start Date: 2022-04-04 +- RFC PR: [apache/tvm-rfcs#0070](https://github.com/apache/tvm-rfcs/pull/0000) +- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000) + +# Summary +[summary]: #summary +Provide a standard and easily testable way to inspect architecture extensions and provide them to the various parts of TVM which utilise that information. + +# Motivation +[motivation]: #motivation +TVM has multiple ways to define a `Target`s architectural features for use in deciding on schedules or other calculations, here's a few different ways we do this: + +* CPU to Feature Mapping: https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/python/tvm/target/arm_isa.py#L22-L39 +* Inspecting `Target` in utility functions: https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/python/tvm/topi/arm_cpu/arm_utils.py#L24-L70 +* Inspecting `Target` in utility functions inside legalization code: https://github.com/apache/tvm/blob/02fbaf0ed9120a8f95155e63de42459f230584aa/python/tvm/relay/qnn/op/legalizations.py#L350-L359 +* Inspecting `Target` inside the definition a strategy: https://github.com/apache/tvm/blob/b542724873140bb051492530d97a78b9b7b7983d/python/tvm/relay/op/strategy/arm_cpu.py#L232 +* Processing bespoke Compiler arguments: https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/src/relay/backend/contrib/cmsisnn/compiler_attrs.cc#L47-L70 +* Registered as a `PackedFunc` (https://github.com/apache/tvm/blob/24e5498021cecca2fe7d44149ce90efe28b6d930/python/tvm/topi/x86/utils.py#L21-L34) and then used as part of `Op` processing: https://github.com/apache/tvm/blob/24e5498021cecca2fe7d44149ce90efe28b6d930/src/relay/qnn/op/requantize_config.h#L58-L73 + +This RFC aims to standardise the way in which we convert `Target` attributes into architectural features by processing them ahead of time. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +Two additional pre-processors can be added to the `Target`, for users to preprocess architectural information when the `Target` is created: +* Architecture Pre-processing - maps `Target` `attrs` to a new `arch` object +* Keys Pre-processing - maps `Target` `attrs` and `keys` to a new set of `keys` + +These new preprocessors will be illustrated using examples targeting TVM for Arm(R) Cortex(R)-M4. + +## Architecture Pre-processing +```c++ +Target("c") + .set_arch_preprocessor(MyArchPreprocessor) +``` + +This takes the `attrs` from `Target` and converts them into an object representing the architectural features of the `Target`, which can then be accessed using the `GetArch` method similar to `GetAttr`: + +```c++ +Target my_target("c -mcpu=cortex-m4"); +my_target->GetArch("is_aarch64", false); // false +my_target->GetArch("has_dsp", false); // true +``` + +```python +my_target = Target("c -mcpu=cortex-m4") +my_target.arch.is_aarch64 // false +my_target.arch.has_dsp // true +``` + +## Keys Pre-processing + +```c++ +Target("c") + .set_keys_preprocessor(MyKeysPreprocessor) +``` + +This takes the `attrs` from `Target` and maps them to relevant `keys` for use when selecting schedules: + +```c++ +Target my_target("c -mcpu=cortex-m4"); +my_target->keys; // ["arm_cpu", "cpu"] <-- "cpu" is taken from default keys and merged by the pre-preprocessor +``` + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +Currently, there is a single `preprocessor` which takes an input of `attrs` and expects the same `attrs` returned with pre-processing applied: + +https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/src/target/target.cc#L810-L814 + +In extension to this, a series of new pre-processors will be defined: + +```c++ +using TargetAttrs = Map; +using TargetArch = Map; +using TargetKeys = Array; + +using FTVMAttrPreprocessor = runtime::TypedPackedFunc; +using FTVMArchPreprocessor = runtime::TypedPackedFunc; +using FTVMKeysPreprocessor = runtime::TypedPackedFunc; +``` + +These implementations can be stored under `src/target/preprocessors/.{cc.h}` to allow them to be composed together such as: + +* src/target/preprocessors/aarch64.cc +* src/target/preprocessors/cpu.cc + +Where the `cpu` pre-processor can utilise the `aarch64` pre-processor if detected. + +## Rename Attr Preprocessor +To help avoid confusion between the existing `attrs` `preprocessor` and the new pre-processors, the `attrs` pre-processor will be renamed from `preprocessor` to `attr_preprocessor`: + +```c++ +class TargetKind { + ... + FTVMAttrPreprocessor attr_preprocessor; + + ... +} +``` + +## Architecture Preprocessor +The first new pre-processor, which processes `attrs` in to an `arch` object, is registered as a new field is added to `TargetKind`: + +```c++ +class TargetKind { + ... + FTVMArchPreprocessor arch_preprocessor; + + ... +} +``` + +This pre-processes `Target` attributes into a new field on `Target` called `arch`: +```c++ +class Target { + ... + DictAttrs arch; + + ... +} +``` + +Which will have similar helper methods to those seen in `IRModule` for `DictAttrs` but with reference to `Arch` rather than `Attr`: + +```c++ +template +Optional GetArch( + const std::string& attr_key, + Optional default_value = Optional(nullptr)) const { +return attrs.GetAttr(attr_key, default_value); +} +template +Optional GetArch(const std::string& attr_key, TObjectRef default_value) const { +return GetArch(attr_key, Optional(default_value)); +} +``` + +As well as a Python class to represent this and allow simple access: +```python +class TargetArch { + def __init__(self, target): + self._target = target + + def __getattr__(self, name): + return _ffi_api.TargetGetArch(self._target, name) +} +``` + +## Key Preprocessor +The second new pre-processor will populate the `keys` fields from the initial `Target` `attrs` and existing `keys`, it simply requires an additional field on `TargetKind`: + +```c++ +class TargetKind { + ... + FTVMKeysPreprocessor keys_preprocessor; + + ... +} +``` + +As the signature of the pre-processor passes the existing keys into the `keys_preprocessor` it is responsible for merging them or removing them if necessary. + +# Drawbacks +[drawbacks]: #drawbacks + +By adding these new pre-processing options to `Target` we increase the amount of work incurred when instantiating a `Target`, it was ultimately considered that this one-time cost would be similar to repeatedly querying the `Target` attributes. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +## Re-use Target Attributes +Instead of creating a new field (`arch`), instead extend `Target` attributes with preprocessed results such that you can do: + +```python +my_target = Target("c -mcpu=cortex-m4") +my_target.is_aarch64 # Extra attribute in `attrs` +``` + +It was felt this would become confusing to work with alongside the documented `Target` attributes in `target_kind.cc` or `target_kind.cc` would need to be bloated with every potential architecture field. The approach of overlapping with `Target` attributes would also increase testing overhead rather than having a straight forward `attrs` to `arch` mapping to test. + +## Extend Utility Functions +Using a standalone function or class across the various areas of the codebase, such as: + +``` +TargetArch my_target_arch(target) +my_target_arch->is_aarch64; // false +``` + +This means re-processing `Target` whenever a specific attribute is required but would provide a single source of truth for doing so. + +# Prior art +[prior-art]: #prior-art + +* This follows the original Target Specification RFC: https://discuss.tvm.apache.org/t/rfc-tvm-target-specification/6844 +* Pre-processor definitions follow the pattern set out in Target Hooks: https://github.com/apache/tvm-rfcs/blob/main/rfcs/0010-target-registered-compiler-flow-customisation.md +* LLVM applies similar processing steps, using defined tables, for example: https://github.com/llvm/llvm-project/blob/7dddf12f448d7ed7e2e35a4de69b53bd140f12c0/llvm/lib/Target/ARM/ARM.td +* GCC applies similar processing steps, using defined tables, for example: https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/gcc/config/arm/arm-cpus.in + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +# Future possibilities +[future-possibilities]: #future-possibilities + +Similar to LLVM and GCC, we may be able to use a custom file format to describe `Target`s more effectively in future which can be added using the same hooks, allowing for easier contributions. From 1c30007d94ad28560d0c9b2d65f1f2d6fb1050df Mon Sep 17 00:00:00 2001 From: Chris Sidebottom Date: Mon, 16 May 2022 10:53:00 +0100 Subject: [PATCH 2/4] Properly reference TargetKind construction and add more Prior Art --- rfcs/0070-target-preprocessing.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/rfcs/0070-target-preprocessing.md b/rfcs/0070-target-preprocessing.md index 796b11b8..624beedb 100644 --- a/rfcs/0070-target-preprocessing.md +++ b/rfcs/0070-target-preprocessing.md @@ -31,7 +31,7 @@ These new preprocessors will be illustrated using examples targeting TVM for Arm ## Architecture Pre-processing ```c++ -Target("c") +TVM_REGISTER_TARGET_KIND("c", kDLCPU) .set_arch_preprocessor(MyArchPreprocessor) ``` @@ -52,7 +52,7 @@ my_target.arch.has_dsp // true ## Keys Pre-processing ```c++ -Target("c") +TVM_REGISTER_TARGET_KIND("c", kDLCPU) .set_keys_preprocessor(MyKeysPreprocessor) ``` @@ -194,10 +194,19 @@ This means re-processing `Target` whenever a specific attribute is required but # Prior art [prior-art]: #prior-art +Taking the example of LLVM, it follows a similar methodology, resulting in a `Features` vector: +* `clang` uses `mtriple` to determine the correct parser to use for the various other options: https://github.com/llvm/llvm-project/blob/2f04e703bff3d9858f53225fa7c780b240c3e247/clang/lib/Driver/ToolChains/Clang.cpp#L324 +* `clang` uses the LLVM parsers to determine available features for a given set of `Target` parameters such as `mcpu` and `mtune`: https://github.com/llvm/llvm-project/blob/43d758b142bbdf94a1c55dc0950637ae74f825b9/clang/lib/Driver/ToolChains/Arch/AArch64.cpp +* LLVM implements the `Features` parsers: https://github.com/llvm/llvm-project/blob/09c2b7c35af8c4bad39f03e9f60df8bd07323028/llvm/lib/Support/AArch64TargetParser.cpp +* The parser is tested in insolation: https://github.com/llvm/llvm-project/blob/09c2b7c35af8c4bad39f03e9f60df8bd07323028/llvm/unittests/Support/TargetParserTest.cpp + +You can see similar definitions within GCC: +* Pre-processes the CLI arguments to add more specific flags: https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/gcc/config/aarch64/driver-aarch64.c#L246 +* Extensions are defined here: https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/gcc/config/aarch64/aarch64-option-extensions.def + +This RFC builds upon the following existing TVM RFCs: * This follows the original Target Specification RFC: https://discuss.tvm.apache.org/t/rfc-tvm-target-specification/6844 * Pre-processor definitions follow the pattern set out in Target Hooks: https://github.com/apache/tvm-rfcs/blob/main/rfcs/0010-target-registered-compiler-flow-customisation.md -* LLVM applies similar processing steps, using defined tables, for example: https://github.com/llvm/llvm-project/blob/7dddf12f448d7ed7e2e35a4de69b53bd140f12c0/llvm/lib/Target/ARM/ARM.td -* GCC applies similar processing steps, using defined tables, for example: https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/gcc/config/arm/arm-cpus.in # Unresolved questions [unresolved-questions]: #unresolved-questions From 2b0552d4140a98dab9a06a61da9a0df76cdb733e Mon Sep 17 00:00:00 2001 From: Chris Sidebottom Date: Thu, 9 Jun 2022 15:24:36 +0100 Subject: [PATCH 3/4] Reformulate RFC to simply adding the target_parser --- rfcs/0070-target-preprocessing.md | 217 ------------------------------ rfcs/0071-target-json-parser.md | 172 +++++++++++++++++++++++ 2 files changed, 172 insertions(+), 217 deletions(-) delete mode 100644 rfcs/0070-target-preprocessing.md create mode 100644 rfcs/0071-target-json-parser.md diff --git a/rfcs/0070-target-preprocessing.md b/rfcs/0070-target-preprocessing.md deleted file mode 100644 index 624beedb..00000000 --- a/rfcs/0070-target-preprocessing.md +++ /dev/null @@ -1,217 +0,0 @@ -- Feature Name: target-architecture-preprocessor -- Start Date: 2022-04-04 -- RFC PR: [apache/tvm-rfcs#0070](https://github.com/apache/tvm-rfcs/pull/0000) -- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000) - -# Summary -[summary]: #summary -Provide a standard and easily testable way to inspect architecture extensions and provide them to the various parts of TVM which utilise that information. - -# Motivation -[motivation]: #motivation -TVM has multiple ways to define a `Target`s architectural features for use in deciding on schedules or other calculations, here's a few different ways we do this: - -* CPU to Feature Mapping: https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/python/tvm/target/arm_isa.py#L22-L39 -* Inspecting `Target` in utility functions: https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/python/tvm/topi/arm_cpu/arm_utils.py#L24-L70 -* Inspecting `Target` in utility functions inside legalization code: https://github.com/apache/tvm/blob/02fbaf0ed9120a8f95155e63de42459f230584aa/python/tvm/relay/qnn/op/legalizations.py#L350-L359 -* Inspecting `Target` inside the definition a strategy: https://github.com/apache/tvm/blob/b542724873140bb051492530d97a78b9b7b7983d/python/tvm/relay/op/strategy/arm_cpu.py#L232 -* Processing bespoke Compiler arguments: https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/src/relay/backend/contrib/cmsisnn/compiler_attrs.cc#L47-L70 -* Registered as a `PackedFunc` (https://github.com/apache/tvm/blob/24e5498021cecca2fe7d44149ce90efe28b6d930/python/tvm/topi/x86/utils.py#L21-L34) and then used as part of `Op` processing: https://github.com/apache/tvm/blob/24e5498021cecca2fe7d44149ce90efe28b6d930/src/relay/qnn/op/requantize_config.h#L58-L73 - -This RFC aims to standardise the way in which we convert `Target` attributes into architectural features by processing them ahead of time. - -# Guide-level explanation -[guide-level-explanation]: #guide-level-explanation - -Two additional pre-processors can be added to the `Target`, for users to preprocess architectural information when the `Target` is created: -* Architecture Pre-processing - maps `Target` `attrs` to a new `arch` object -* Keys Pre-processing - maps `Target` `attrs` and `keys` to a new set of `keys` - -These new preprocessors will be illustrated using examples targeting TVM for Arm(R) Cortex(R)-M4. - -## Architecture Pre-processing -```c++ -TVM_REGISTER_TARGET_KIND("c", kDLCPU) - .set_arch_preprocessor(MyArchPreprocessor) -``` - -This takes the `attrs` from `Target` and converts them into an object representing the architectural features of the `Target`, which can then be accessed using the `GetArch` method similar to `GetAttr`: - -```c++ -Target my_target("c -mcpu=cortex-m4"); -my_target->GetArch("is_aarch64", false); // false -my_target->GetArch("has_dsp", false); // true -``` - -```python -my_target = Target("c -mcpu=cortex-m4") -my_target.arch.is_aarch64 // false -my_target.arch.has_dsp // true -``` - -## Keys Pre-processing - -```c++ -TVM_REGISTER_TARGET_KIND("c", kDLCPU) - .set_keys_preprocessor(MyKeysPreprocessor) -``` - -This takes the `attrs` from `Target` and maps them to relevant `keys` for use when selecting schedules: - -```c++ -Target my_target("c -mcpu=cortex-m4"); -my_target->keys; // ["arm_cpu", "cpu"] <-- "cpu" is taken from default keys and merged by the pre-preprocessor -``` - -# Reference-level explanation -[reference-level-explanation]: #reference-level-explanation - -Currently, there is a single `preprocessor` which takes an input of `attrs` and expects the same `attrs` returned with pre-processing applied: - -https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/src/target/target.cc#L810-L814 - -In extension to this, a series of new pre-processors will be defined: - -```c++ -using TargetAttrs = Map; -using TargetArch = Map; -using TargetKeys = Array; - -using FTVMAttrPreprocessor = runtime::TypedPackedFunc; -using FTVMArchPreprocessor = runtime::TypedPackedFunc; -using FTVMKeysPreprocessor = runtime::TypedPackedFunc; -``` - -These implementations can be stored under `src/target/preprocessors/.{cc.h}` to allow them to be composed together such as: - -* src/target/preprocessors/aarch64.cc -* src/target/preprocessors/cpu.cc - -Where the `cpu` pre-processor can utilise the `aarch64` pre-processor if detected. - -## Rename Attr Preprocessor -To help avoid confusion between the existing `attrs` `preprocessor` and the new pre-processors, the `attrs` pre-processor will be renamed from `preprocessor` to `attr_preprocessor`: - -```c++ -class TargetKind { - ... - FTVMAttrPreprocessor attr_preprocessor; - - ... -} -``` - -## Architecture Preprocessor -The first new pre-processor, which processes `attrs` in to an `arch` object, is registered as a new field is added to `TargetKind`: - -```c++ -class TargetKind { - ... - FTVMArchPreprocessor arch_preprocessor; - - ... -} -``` - -This pre-processes `Target` attributes into a new field on `Target` called `arch`: -```c++ -class Target { - ... - DictAttrs arch; - - ... -} -``` - -Which will have similar helper methods to those seen in `IRModule` for `DictAttrs` but with reference to `Arch` rather than `Attr`: - -```c++ -template -Optional GetArch( - const std::string& attr_key, - Optional default_value = Optional(nullptr)) const { -return attrs.GetAttr(attr_key, default_value); -} -template -Optional GetArch(const std::string& attr_key, TObjectRef default_value) const { -return GetArch(attr_key, Optional(default_value)); -} -``` - -As well as a Python class to represent this and allow simple access: -```python -class TargetArch { - def __init__(self, target): - self._target = target - - def __getattr__(self, name): - return _ffi_api.TargetGetArch(self._target, name) -} -``` - -## Key Preprocessor -The second new pre-processor will populate the `keys` fields from the initial `Target` `attrs` and existing `keys`, it simply requires an additional field on `TargetKind`: - -```c++ -class TargetKind { - ... - FTVMKeysPreprocessor keys_preprocessor; - - ... -} -``` - -As the signature of the pre-processor passes the existing keys into the `keys_preprocessor` it is responsible for merging them or removing them if necessary. - -# Drawbacks -[drawbacks]: #drawbacks - -By adding these new pre-processing options to `Target` we increase the amount of work incurred when instantiating a `Target`, it was ultimately considered that this one-time cost would be similar to repeatedly querying the `Target` attributes. - -# Rationale and alternatives -[rationale-and-alternatives]: #rationale-and-alternatives - -## Re-use Target Attributes -Instead of creating a new field (`arch`), instead extend `Target` attributes with preprocessed results such that you can do: - -```python -my_target = Target("c -mcpu=cortex-m4") -my_target.is_aarch64 # Extra attribute in `attrs` -``` - -It was felt this would become confusing to work with alongside the documented `Target` attributes in `target_kind.cc` or `target_kind.cc` would need to be bloated with every potential architecture field. The approach of overlapping with `Target` attributes would also increase testing overhead rather than having a straight forward `attrs` to `arch` mapping to test. - -## Extend Utility Functions -Using a standalone function or class across the various areas of the codebase, such as: - -``` -TargetArch my_target_arch(target) -my_target_arch->is_aarch64; // false -``` - -This means re-processing `Target` whenever a specific attribute is required but would provide a single source of truth for doing so. - -# Prior art -[prior-art]: #prior-art - -Taking the example of LLVM, it follows a similar methodology, resulting in a `Features` vector: -* `clang` uses `mtriple` to determine the correct parser to use for the various other options: https://github.com/llvm/llvm-project/blob/2f04e703bff3d9858f53225fa7c780b240c3e247/clang/lib/Driver/ToolChains/Clang.cpp#L324 -* `clang` uses the LLVM parsers to determine available features for a given set of `Target` parameters such as `mcpu` and `mtune`: https://github.com/llvm/llvm-project/blob/43d758b142bbdf94a1c55dc0950637ae74f825b9/clang/lib/Driver/ToolChains/Arch/AArch64.cpp -* LLVM implements the `Features` parsers: https://github.com/llvm/llvm-project/blob/09c2b7c35af8c4bad39f03e9f60df8bd07323028/llvm/lib/Support/AArch64TargetParser.cpp -* The parser is tested in insolation: https://github.com/llvm/llvm-project/blob/09c2b7c35af8c4bad39f03e9f60df8bd07323028/llvm/unittests/Support/TargetParserTest.cpp - -You can see similar definitions within GCC: -* Pre-processes the CLI arguments to add more specific flags: https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/gcc/config/aarch64/driver-aarch64.c#L246 -* Extensions are defined here: https://github.com/gcc-mirror/gcc/blob/16e2427f50c208dfe07d07f18009969502c25dc8/gcc/config/aarch64/aarch64-option-extensions.def - -This RFC builds upon the following existing TVM RFCs: -* This follows the original Target Specification RFC: https://discuss.tvm.apache.org/t/rfc-tvm-target-specification/6844 -* Pre-processor definitions follow the pattern set out in Target Hooks: https://github.com/apache/tvm-rfcs/blob/main/rfcs/0010-target-registered-compiler-flow-customisation.md - -# Unresolved questions -[unresolved-questions]: #unresolved-questions - -# Future possibilities -[future-possibilities]: #future-possibilities - -Similar to LLVM and GCC, we may be able to use a custom file format to describe `Target`s more effectively in future which can be added using the same hooks, allowing for easier contributions. diff --git a/rfcs/0071-target-json-parser.md b/rfcs/0071-target-json-parser.md new file mode 100644 index 00000000..bdd83b74 --- /dev/null +++ b/rfcs/0071-target-json-parser.md @@ -0,0 +1,172 @@ +- Feature Name: target-json-preprocessor +- Start Date: 2022-04-04 +- RFC PR: [apache/tvm-rfcs#0071](https://github.com/apache/tvm-rfcs/pull/71) +- GitHub Issue: [apache/tvm#0000](https://github.com/apache/tvm/issues/0000) + +# Summary +[summary]: #summary +Extend the existing `TargetKind` `preprocessor` to allow preprocessing of the entire `Target` JSON representation rather than just `attrs`. + +# Motivation +[motivation]: #motivation + +Taking an example `Target` in JSON form: + +```js +{ + "id": "cuda", + "tag": "nvidia/tx2-cudnn", + "keys": ["cuda", "gpu"], + "libs": ["cudnn"], + "target_host": { + "id": "llvm", + "system_lib": True, + "mtriple": "aarch64-linux-gnu", + "mattr": "+neon" + } +} +``` + +We can see that there are additional fields which are of interest to TVM, note-ably `keys` and `libs` which we currently do not apply parsing to on `Target` instantiation. Extending the `TargetKind` `preprocessor` beyond `attrs` enables to customise parsing of the entire `Target`, enabling the values passed by the user to be used to infer other properties used during compilation. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +Alongside the existing `set_attrs_preprocessor` method on `TargetKind`, there will be an alternative `set_target_parser` method, this new parser will take precedence over the `attrs` preprocessor if present: + +```c++ +TVM_REGISTER_TARGET_KIND("target", kDLCPU) + .set_target_parser(TargetParser); +``` + +The canonical JSON form of `Target` will be passed to the new `Target` parser and the parser will return the transformed variant in JSON form for further steps: + +```c++ +using TargetJSON = Map; +TargetJSON TargetParser(TargetJSON target) { + // ... transforms ... + return target; +} +``` + +The parser will have to be capable of handling the diversity of types of `Target` in TVM, therefore the underlying mechanism of the parser is left as an implementation detail. Using the example of pre-processing the `keys` attribute (used for detecting appropriate schedules), it can be seen how this can apply to various `Target`s. + +## TVM Target's Directly Mapping to a Backend's Target +Take the example of pre-processing `keys` (in this case using the `cuda` `Target`): +```c++ +using TargetJSON = Map; + +TargetJSON CUDAParser(TargetJSON target) { + if (IsSuper(target)) { + target["keys"].push_back("super_cuda"); + } +} + +TVM_REGISTER_TARGET_KIND("cuda", kDLGPU) + .set_target_parser(CUDAParser); +``` + +This takes the `attrs` from `Target` and maps them to relevant `keys` for use when selecting schedules: + +```c++ +Target my_target("cuda -msuper"); +my_target->keys; // ["cuda", "gpu", "super_cuda"] <-- "cpu" and "cuda" are taken from default keys - "super_cuda" is added +``` + +## TVM Target's Mapping to a Backend with Multiple Target's +The previous example would work for `Target`s which map to a specific architecture, such as `cuda`. To parse a `Target` which has a number of its own targets, such as `llvm`, the parser can be broken down within the parent parser: + +```c++ +using TargetJSON = Map; + +TargetJSON AArch64TargetParser(TargetJSON target) { + target["keys"].push_back("arm_cpu"); + return target; +} + +TargetJSON x86TargetParser(TargetJSON target) { + target["keys"].push_back("x86_64"); + return target; +} + +TargetJSON CPUTargetParser(TargetJSON target) { + if (IsAArch64Target(target)) { + return AArch64TargetParser(target); + } + if (IsX86Target(target)) { + return x86TargetParser(target); + } + return target; +} + +TVM_REGISTER_TARGET_KIND("llvm", kDLCPU) + .set_target_parser(CPUTargetParser); +``` + +This has the additional advantage that if there are standard arguments, such as `mcpu`, `mattr` and `march`, the parser can be re-used in both `Target`s - for example the `c` `Target` can re-use the above `llvm` `Target` parser: + +```c++ +TVM_REGISTER_TARGET_KIND("c", kDLCPU) + .set_target_parser(CPUTargetParser); +``` + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +Currently, there is a single `preprocessor` which takes an input of `attrs` and expects the same `attrs` returned with pre-processing applied: + +https://github.com/apache/tvm/blob/d2db9cb0d839e32778f461b77e59f6418282a511/src/target/target.cc#L810-L814 + +The new `Target` parser will live in addition to the `preprocessor` until such a time as the `preprocessor` can be fully removed. This extends `TargetKind` to support both `preprocessor` and `target_parser`: + +```c++ +using TargetJSON = Map; +using FTVMTargetParser = TypedPackedFunc; + +class TargetKind { + ... + PackedFunc preprocessor; + FTVMTargetParser target_parser; + + ... +} +``` + +Implementations for `Target` parsers will be stored under `src/target/parsers/.{cc.h}`, allowing them to be composed together (as shown above), such as: + +* src/target/parsers/cuda.cc +* src/target/parsers/aarch64.cc +* src/target/parsers/cpu.cc + +Where the `cpu` pre-processor can utilise the `aarch64` pre-processor if detected and `cuda` is an independent parser specific to that `Target`. + +# Drawbacks +[drawbacks]: #drawbacks + +By adding these new pre-processing options to `Target` we increase the amount of work incurred when instantiating a `Target`, it was ultimately considered that this one-time cost would be similar or less than repeatedly querying the `Target` attributes. + +Providing the ability to completely change a `Target` on parsing could allow an extensive mutation of the input `Target`. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +Instead of providing a single parser entrypoint, we can instead use several parsers for each attribute - this clearly separates the responsibility of each parser but also means maintaining many entrypoints to `Target` parsing. + +# Prior art +[prior-art]: #prior-art + +## Other Compilers +Taking the example of LLVM, it follows a similar methodology, resulting in a `Features` vector: +* `clang` uses the LLVM parsers to determine available features for a given set of `Target` parameters such as `mcpu` and `mtune`: https://github.com/llvm/llvm-project/blob/43d758b142bbdf94a1c55dc0950637ae74f825b9/clang/lib/Driver/ToolChains/Arch/AArch64.cpp +* LLVM implements the `Features` parsers: https://github.com/llvm/llvm-project/blob/09c2b7c35af8c4bad39f03e9f60df8bd07323028/llvm/lib/Support/AArch64TargetParser.cpp +* The parser is tested in insolation: https://github.com/llvm/llvm-project/blob/09c2b7c35af8c4bad39f03e9f60df8bd07323028/llvm/unittests/Support/TargetParserTest.cpp + +## Existing TVM RFCs +This RFC builds upon the following existing TVM RFCs: +* This follows the original Target Specification RFC: https://discuss.tvm.apache.org/t/rfc-tvm-target-specification/6844 + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +# Future possibilities +[future-possibilities]: #future-possibilities From 5ba1e458c4906b583d3aee46f03c600ea02207fd Mon Sep 17 00:00:00 2001 From: Chris Sidebottom Date: Tue, 2 Aug 2022 16:20:51 +0100 Subject: [PATCH 4/4] Minor text fixes --- rfcs/0071-target-json-parser.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/0071-target-json-parser.md b/rfcs/0071-target-json-parser.md index bdd83b74..27d362af 100644 --- a/rfcs/0071-target-json-parser.md +++ b/rfcs/0071-target-json-parser.md @@ -32,7 +32,7 @@ We can see that there are additional fields which are of interest to TVM, note-a # Guide-level explanation [guide-level-explanation]: #guide-level-explanation -Alongside the existing `set_attrs_preprocessor` method on `TargetKind`, there will be an alternative `set_target_parser` method, this new parser will take precedence over the `attrs` preprocessor if present: +Alongside the existing `set_attrs_preprocessor` method on `TargetKind`, there will be an alternative `set_target_parser` method to bind a `FTVMTargetParser` to the `TargetKind`. The new `FTVMTargetParser` will take precedence over the `attrs` preprocessor if present: ```c++ TVM_REGISTER_TARGET_KIND("target", kDLCPU)