Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SubDevice Implementation Master Issue #13655

Open
10 of 17 tasks
tt-aho opened this issue Oct 9, 2024 · 2 comments
Open
10 of 17 tasks

SubDevice Implementation Master Issue #13655

tt-aho opened this issue Oct 9, 2024 · 2 comments
Assignees
@tt-aho
Labels
feature P1

Comments

@tt-aho
Copy link
Contributor

tt-aho commented Oct 9, 2024 •
edited
Loading

  • SubDevice object

    • Vec of CoreRangeSets by CoreType

  • SubDeviceManager Object

    • Allocator changes to support reduced size global L1 allocator, and local SubDevice allocators
    • State required to be tracked per manager
    • APIs for Create/Load/Clear of CoreMeshManager
    • Event/finish APIs need to take in SubDevices to block on
    • Program spanning multiple sub-devices (not needed for llama?)
      • CB validation needs to be across used sub-devices
      • Syncing/levelling of ring buffers for semaphores

Potential dependencies for feature completeness of SubDevice

  • Refactor of program to separate api and state
    • Potentially needed since we only finalize programs once, but recompiling for a different device may have different commands, so we need to track state by device. May not be immediately needed since op infra does not recompile a program across multiple devices
  • API changes for accessing rtas/updating ptrs (ETA ?)
    • Needed for generality of running the same program with different SubDeviceManagers loaded. May not be immediately needed for Llama deadline

Potentially needed features that depend on SubCoreMesh:
@tt-aho tt-aho self-assigned this Oct 9, 2024
tt-aho added a commit that referenced this issue Oct 29, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
970a69b 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE will be updated to take in a list of sub devices for blocking on
- Trace will currently track all sub devices. Potential to track specific sub devices (could be automatic) in the future
- EP is currently hardcoded to sub device 0. This will be updated to determine the used sub devices in the future
@tt-aho tt-aho mentioned this issue Oct 29, 2024
Merged
5 tasks
tt-aho added a commit that referenced this issue Oct 29, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
1b0640f 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE will be updated to take in a list of sub devices for blocking on in the future. Currently will sync all sub_devices
- Trace will currently track all sub devices. Potential to track specific sub devices (could be automatic) in the future
- EP is currently hardcoded to sub device 0. This will be updated to determine the used sub devices in the future
tt-aho added a commit that referenced this issue Oct 30, 2024
@tt-aho
#13655: Remove compile time mcast grid and unicast cores from dispatc…
0a14709 
…h kernels

CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Oct 30, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
bf74f4f 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE will be updated to take in a list of sub devices for blocking on in the future. Currently will sync all sub_devices
- Trace will currently track all sub devices. Potential to track specific sub devices (could be automatic) in the future
- EP is currently hardcoded to sub device 0. This will be updated to determine the used sub devices in the future
tt-aho added a commit that referenced this issue Oct 30, 2024
@tt-aho
#13655: Remove compile time mcast grid and unicast cores from dispatc…
d385b5d 
…h kernels

CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Oct 30, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
0879d66 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE will be updated to take in a list of sub devices for blocking on in the future. Currently will sync all sub_devices
- Trace will currently track all sub devices. Potential to track specific sub devices (could be automatic) in the future
- EP is currently hardcoded to sub device 0. This will be updated to determine the used sub devices in the future
tt-aho added a commit that referenced this issue Oct 30, 2024
@tt-aho
#13655: Remove compile time mcast grid and unicast cores from dispatc…
bcf2e19 
…h kernels

CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Oct 30, 2024
@tt-aho
#13655: Remove compile time mcast grid and unicast cores from dispatc…
aced07a 
…h kernels

CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Oct 30, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
d4f6af7 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE will be updated to take in a list of sub devices for blocking on in the future. Currently will sync all sub_devices
- Trace will currently track all sub devices. Potential to track specific sub devices (could be automatic) in the future
- EP is currently hardcoded to sub device 0. This will be updated to determine the used sub devices in the future
tt-aho added a commit that referenced this issue Oct 30, 2024
@tt-aho
#13655: Remove compile time mcast grid and unicast cores from dispatc…
c5b52f0 
…h kernels

CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Oct 31, 2024
@tt-aho
#13655: Add limited ability to shrink allocator size from the bottom
9ce9a20
@tt-aho tt-aho mentioned this issue Oct 31, 2024
Merged
5 tasks
tt-aho added a commit that referenced this issue Nov 1, 2024
@tt-aho
#13655: Add limited ability to shrink allocator size from the bottom
d0c1de1
tt-aho added a commit that referenced this issue Nov 1, 2024
@tt-aho
#13655: Add limited ability to shrink allocator size from the bottom
21b9a59
tt-aho added a commit that referenced this issue Nov 1, 2024
@tt-aho
#13655: Add limited ability to shrink allocator size from the bottom
a440d45
tt-aho added a commit that referenced this issue Nov 1, 2024
@tt-aho
#13655: Remove compile time mcast grid and unicast cores from dispatc…
b9eb472 
…h kernels

CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Nov 1, 2024
@tt-aho
#13655: Add limited ability to shrink allocator size from the bottom
fa459fa
tt-aho added a commit that referenced this issue Nov 1, 2024
@tt-aho
#13655: Add limited ability to shrink allocator size from the bottom
1739075
tt-aho added a commit that referenced this issue Nov 1, 2024
@tt-aho
#13655: Add limited ability to shrink allocator size from the bottom
d52a958
@tt-aho tt-aho changed the title SubCoreMesh Implementation Master Issue SubDevice Implementation Master Issue Nov 1, 2024
o2buzzle pushed a commit that referenced this issue Nov 4, 2024
@tt-aho @o2buzzle
#13655: Add limited ability to shrink allocator size from the bottom
d933707
tt-aho added a commit that referenced this issue Nov 4, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
20d26cc 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE will be updated to take in a list of sub devices for blocking on in the future. Currently will sync all sub_devices
- Trace will currently track all sub devices. Potential to track specific sub devices (could be automatic) in the future
- EP is currently hardcoded to sub device 0. This will be updated to determine the used sub devices in the future
tt-aho added a commit that referenced this issue Nov 4, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
8c6e45d 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE will be updated to take in a list of sub devices for blocking on in the future. Currently will sync all sub_devices
- Trace will currently track all sub devices. Potential to track specific sub devices (could be automatic) in the future
- EP is currently hardcoded to sub device 0. This will be updated to determine the used sub devices in the future
tt-aho added a commit that referenced this issue Nov 4, 2024
@tt-aho
#13655: Remove compile time mcast grid and unicast cores from dispatc…
2f1dbc7 
…h kernels

CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Nov 5, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
54dddb6 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE will be updated to take in a list of sub devices for blocking on in the future. Currently will sync all sub_devices
- Trace will currently track all sub devices. Potential to track specific sub devices (could be automatic) in the future
- EP is currently hardcoded to sub device 0. This will be updated to determine the used sub devices in the future
tt-aho added a commit that referenced this issue Nov 5, 2024
@tt-aho
#13655: Remove compile time mcast grid and unicast cores from dispatc…
2f84796 
…h kernels

CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Nov 12, 2024
@tt-aho
#13655: Add SubDevice and SubDeviceManager implementations
8c38296 
Add support for splitting a device into multiple SubDevices, as well and maintaining different SubDeviceManager configurations, owned by device
Add basic tests to validate sub-device support
tt-aho added a commit that referenced this issue Nov 12, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
4e45777 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE takes in a list of sub devices for blocking/issuing waits on. Will wait on all sub-devices if none are provided
- Trace will track only specific sub devices used
- EP currently only supports one sub-device
- Remove compile time mcast grid and unicast cores from dispatch kernels
  CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Nov 12, 2024
@tt-aho
#13655: Update allocator to support taking in a CoreRangeSet for bank…
9666070 
…s, instead of assuming a rectangular grid

Update device allocator related apis to take in a sub-device parameter
tt-aho added a commit that referenced this issue Nov 12, 2024
@tt-aho
#13655: Add SubDevice and SubDeviceManager implementations
c12b69e 
Add support for splitting a device into multiple SubDevices, as well and maintaining different SubDeviceManager configurations, owned by device
Add basic tests to validate sub-device support
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Add SubDevice and SubDeviceManager implementations
f1495dd 
Add support for splitting a device into multiple SubDevices, as well and maintaining different SubDeviceManager configurations, owned by device
Add basic tests to validate sub-device support
Update device apis to overload rather than take in optional sub_device ids
Make SubDeviceId, SubDeviceManagerId strong types
Refactor Device/SubDeviceManager state so that the default state is also encapsulated in a SubDeviceManager, and access the active SubDeviceManager through a pointer instead of map lookup
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Refactor dispatching of go signal to not send noc txn data af…
52c352f 
…ter go signal command

Instead, we populate a static array on dispatcher when we change sub-device configurations with all noc txn data, and read from it using an offset passed in the go signal command
Remove dynamic allocation of sub-devices/expected workers pairs, and pass them as separate spans
Fix cmd in sweep_pgm_dispatch
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
68cde49 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE takes in a list of sub devices for blocking/issuing waits on. Will wait on all sub-devices if none are provided
- Trace will track only specific sub devices used
- EP currently only supports one sub-device
- Remove compile time mcast grid and unicast cores from dispatch kernels
  CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Update allocator to support taking in a CoreRangeSet for bank…
69355de 
…s, instead of assuming a rectangular grid

Update device allocator related apis to take in a sub-device parameter
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Add SubDevice and SubDeviceManager implementations
d311edd 
Add support for splitting a device into multiple SubDevices, as well and maintaining different SubDeviceManager configurations, owned by device
Add basic tests to validate sub-device support
Update device apis to overload rather than take in optional sub_device ids
Make SubDeviceId, SubDeviceManagerId strong types
Refactor Device/SubDeviceManager state so that the default state is also encapsulated in a SubDeviceManager, and access the active SubDeviceManager through a pointer instead of map lookup
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Refactor dispatching of go signal to not send noc txn data af…
d092ba7 
…ter go signal command

Instead, we populate a static array on dispatcher when we change sub-device configurations with all noc txn data, and read from it using an offset passed in the go signal command
Remove dynamic allocation of sub-devices/expected workers pairs, and pass them as separate spans
Fix cmd in sweep_pgm_dispatch
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
044f18a 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE takes in a list of sub devices for blocking/issuing waits on. Will wait on all sub-devices if none are provided
- Trace will track only specific sub devices used
- EP currently only supports one sub-device
- Remove compile time mcast grid and unicast cores from dispatch kernels
  CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Update allocator to support taking in a CoreRangeSet for bank…
097ab84 
…s, instead of assuming a rectangular grid

Update device allocator related apis to take in a sub-device parameter
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Add SubDevice and SubDeviceManager implementations
1ea3cb2 
Add support for splitting a device into multiple SubDevices, as well and maintaining different SubDeviceManager configurations, owned by device
Add basic tests to validate sub-device support
Update device apis to overload rather than take in optional sub_device ids
Make SubDeviceId, SubDeviceManagerId strong types
Refactor Device/SubDeviceManager state so that the default state is also encapsulated in a SubDeviceManager, and access the active SubDeviceManager through a pointer instead of map lookup
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Refactor dispatching of go signal to not send noc txn data af…
c0b0645 
…ter go signal command

Instead, we populate a static array on dispatcher when we change sub-device configurations with all noc txn data, and read from it using an offset passed in the go signal command
Remove dynamic allocation of sub-devices/expected workers pairs, and pass them as separate spans
Fix cmd in sweep_pgm_dispatch
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Refactor dispatching of go signal to not send noc txn data af…
3fb1a0e 
…ter go signal command

Instead, we populate a static array on dispatcher when we change sub-device configurations with all noc txn data, and read from it using an offset passed in the go signal command
Remove dynamic allocation of sub-devices/expected workers pairs, and pass them as separate spans
Fix cmd in sweep_pgm_dispatch
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
af24696 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE takes in a list of sub devices for blocking/issuing waits on. Will wait on all sub-devices if none are provided
- Trace will track only specific sub devices used
- EP currently only supports one sub-device
- Remove compile time mcast grid and unicast cores from dispatch kernels
  CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Update allocator to support taking in a CoreRangeSet for bank…
9ff3971 
…s, instead of assuming a rectangular grid

Update device allocator related apis to take in a sub-device parameter
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Add SubDevice and SubDeviceManager implementations
92bdf49 
Add support for splitting a device into multiple SubDevices, as well and maintaining different SubDeviceManager configurations, owned by device
Add basic tests to validate sub-device support
Update device apis to overload rather than take in optional sub_device ids
Make SubDeviceId, SubDeviceManagerId strong types
Refactor Device/SubDeviceManager state so that the default state is also encapsulated in a SubDeviceManager, and access the active SubDeviceManager through a pointer instead of map lookup
tt-aho added a commit that referenced this issue Nov 13, 2024
@tt-aho
#13655: Refactor dispatching of go signal to not send noc txn data af…
89331b6 
…ter go signal command

Instead, we populate a static array on dispatcher when we change sub-device configurations with all noc txn data, and read from it using an offset passed in the go signal command
Remove dynamic allocation of sub-devices/expected workers pairs, and pass them as separate spans
Fix cmd in sweep_pgm_dispatch
tt-aho added a commit that referenced this issue Nov 14, 2024
@tt-aho
#13655: Initial FD refactor to support sub devices
ec673d3 
Support multiple dispatch entries for worker->dispatch sync
Update dispatch d/s to have a semaphore per dispatch entry to enable syncing on specific worker counts
Update LaunchMessageRingBufferState and WorkerConfigBufferMgr to be tracked per sub_device
Update various FD commands to support syncing on multiple sub devices:
- ERB, EWB, ERE takes in a list of sub devices for blocking/issuing waits on. Will wait on all sub-devices if none are provided
- Trace will track only specific sub devices used
- EP currently only supports one sub-device
- Remove compile time mcast grid and unicast cores from dispatch kernels
  CQDispatchGoSignalMcastCmd now expects noc txn data to follow the cmd for sending go signal to cores
tt-aho added a commit that referenced this issue Nov 14, 2024
@tt-aho
#13655: Update allocator to support taking in a CoreRangeSet for bank…
5536fe4 
…s, instead of assuming a rectangular grid

Update device allocator related apis to take in a sub-device parameter
tt-aho added a commit that referenced this issue Nov 14, 2024
@tt-aho
#13655: Add SubDevice and SubDeviceManager implementations
5d470b7 
Add support for splitting a device into multiple SubDevices, as well and maintaining different SubDeviceManager configurations, owned by device
Add basic tests to validate sub-device support
Update device apis to overload rather than take in optional sub_device ids
Make SubDeviceId, SubDeviceManagerId strong types
Refactor Device/SubDeviceManager state so that the default state is also encapsulated in a SubDeviceManager, and access the active SubDeviceManager through a pointer instead of map lookup
tt-aho added a commit that referenced this issue Nov 14, 2024
@tt-aho
#13655: Refactor dispatching of go signal to not send noc txn data af…
14a6f6f 
…ter go signal command

Instead, we populate a static array on dispatcher when we change sub-device configurations with all noc txn data, and read from it using an offset passed in the go signal command
Remove dynamic allocation of sub-devices/expected workers pairs, and pass them as separate spans
Fix cmd in sweep_pgm_dispatch
tt-aho added a commit that referenced this issue Nov 14, 2024
@tt-aho
#13655: Add pybindings to create sub-devices, and creating/configurin…
7b54081 
…g sub-device managers

Rename clear_loaded_sub_device_manager->reset_active_sub_device_manager
Remove device->allocator_ reacharound in ttnn
tt-aho added a commit that referenced this issue Nov 14, 2024
@tt-aho
#13655: Add pybindings to create sub-devices, and creating/configurin…
37ba0d6 
…g sub-device managers

Rename clear_loaded_sub_device_manager->reset_active_sub_device_manager
Remove device->allocator_ reacharound in ttnn
tt-aho added a commit that referenced this issue Nov 14, 2024
@tt-aho
#13655: Fix sub-device tests for BH
3a81f96 
Was not accounting for alignment when calculating expected addresses
tt-aho added a commit that referenced this issue Nov 14, 2024
@tt-aho
#13655: Fix sub-device tests for BH
03f4539 
Was not accounting for alignment when calculating expected addresses
tt-aho added a commit that referenced this issue Nov 14, 2024
@tt-aho
#13655: Fix sub-device tests for BH
544165e 
Was not accounting for alignment when calculating expected addresses
@pgkeller
Copy link
Contributor

@tt-aho is this still in flight or resolved at this point?

@tt-aho
Copy link
Contributor Author

tt-aho commented Jan 30, 2025

There are a few outstanding features we specced out to support, but wasn't immediately needed for llama support. I can just spin those as separate issues and close this one?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tt-aho tt-aho

Labels
feature P1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tt-aho @pgkeller