Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tuner][Codegen] Make default tuning be available per-SKU instead of per-arch #19720

Closed
kuhar opened this issue Jan 16, 2025 · 1 comment
Closed
Assignees
Labels

Comments

@kuhar
Copy link
Member

kuhar commented Jan 16, 2025

We have multiple GPUs that share the same ISA architecture, for example gfx940 is implemented by MI300X, MI300A, and MI308X. These targets have different number of workgroup processors and different power caps. Because of this, they require dedicated tuning specs.

We should update our tuning specs to be per-SKU first, and then per-architecture as a fallback. We can start with MI300X and MI308X.

The default tuning specs are stored in https://github.com/iree-org/iree/tree/main/compiler/plugins/target/ROCM/builtins/tuning while the logic to decide which file to pick up is in

// Try to look up the default tuning spec for this architecture, if any.
StringRef arch = gpuTarget.getArch();
std::string defaultTuningSpecName =
llvm::formatv("iree_default_tuning_spec_{}.mlir", arch);
std::optional<StringRef> defaultTuningSpecSource;
EmbeddedDataDirectory::withGlobal([&](EmbeddedDataDirectory &dir) {
defaultTuningSpecSource = dir.getFile(defaultTuningSpecName);
});
.

To test this, we can modify the existing test to contain these RUN lines: https://github.com/iree-org/iree/blob/main/compiler/plugins/target/ROCM/builtins/tuning/test/spec_gfx942.mlir

  • one for --iree-gpu-test-target=mi300x
  • one for --iree-gpu-test-target=mi300a -- check that without a spec for this SKU, we load the spec for gfx942
  • one for --iree-gpu-test-target=gfx942 -- we can still support sku-agnostic targets like we do today
@kuhar
Copy link
Member Author

kuhar commented Jan 16, 2025

cc: @MaheshRavishankar

@kuhar kuhar added the tuner label Jan 16, 2025
bangtianliu added a commit that referenced this issue Jan 24, 2025
This PR implements support for per-sku tuning specs, and nd then
per-architecture as a fallback, which is relevant to task outlined in
#19720.

---------

Signed-off-by: Bangtian Liu <liubangtian@gmail.com>
kuhar added a commit to kuhar/iree that referenced this issue Jan 25, 2025
Add the product name (SKU) naming scheme and explain the difference
compared to the architecture scheme. This scheme has been supported for
a while but not documented. We expect it to become more relevant after
the work related to SKU-specific default tuning specs:
iree-org#19762.

Issue: iree-org#19720
Signed-off-by: Jakub Kuderski <jakub@nod-labs.com>
kuhar added a commit that referenced this issue Jan 25, 2025
This flag is intended for tests only. Rename it and mark as it as
hidden.

Requested by @qedawkins in
#19762 (comment).

Issue: #19720

Signed-off-by: Jakub Kuderski <jakub@nod-labs.com>
kuhar added a commit that referenced this issue Jan 25, 2025
Add the product name (SKU) naming scheme and explain the difference
compared to the architecture scheme. This scheme has been supported for
a while but not documented. We expect it to become more relevant after
the work related to SKU-specific default tuning specs:
#19762.

Issue: #19720

Signed-off-by: Jakub Kuderski <jakub@nod-labs.com>
@kuhar kuhar closed this as completed Jan 26, 2025
ita9naiwa pushed a commit to ita9naiwa/iree that referenced this issue Feb 4, 2025
This PR implements support for per-sku tuning specs, and nd then
per-architecture as a fallback, which is relevant to task outlined in
iree-org#19720.

---------

Signed-off-by: Bangtian Liu <liubangtian@gmail.com>
Signed-off-by: Hyunsung Lee <ita9naiwa@gmail.com>
ita9naiwa pushed a commit to ita9naiwa/iree that referenced this issue Feb 4, 2025
This flag is intended for tests only. Rename it and mark as it as
hidden.

Requested by @qedawkins in
iree-org#19762 (comment).

Issue: iree-org#19720

Signed-off-by: Jakub Kuderski <jakub@nod-labs.com>
Signed-off-by: Hyunsung Lee <ita9naiwa@gmail.com>
ita9naiwa pushed a commit to ita9naiwa/iree that referenced this issue Feb 4, 2025
Add the product name (SKU) naming scheme and explain the difference
compared to the architecture scheme. This scheme has been supported for
a while but not documented. We expect it to become more relevant after
the work related to SKU-specific default tuning specs:
iree-org#19762.

Issue: iree-org#19720

Signed-off-by: Jakub Kuderski <jakub@nod-labs.com>
Signed-off-by: Hyunsung Lee <ita9naiwa@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants