Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why have the options to define the single and double blocks_to_swap been removed? #1923

Open
Deathawaits4 opened this issue Feb 7, 2025 · 1 comment

Comments

@Deathawaits4
Copy link

Deathawaits4 commented Feb 7, 2025

Any model with a different block count can only inefficiently make use of this option:

--double_blocks_to_swap and --single_blocks_to_swap are deprecated. These options is still available, but they will be removed in the future. Please use --blocks_to_swap instead. These options are equivalent to specifying double_blocks_to_swap + single_blocks_to_swap // 2 in --blocks_to_swap.

Cannot swap more than 6 double blocks and 36 single blocks. Requested 18 double blocks and 36 single blocks.

i manually edited the flux_models.py so i can actually swap the blocks, but it would still be nice if we could define single and double blocks manually.

@rockerBOO
Copy link
Contributor

rockerBOO commented Feb 11, 2025

I do not know the original reason it was modified but I had a similar approach too. For flex training it's only 8 blocks in the double and 36 for the single so it limits it to 12 blocks by default max for Flex. I added a single_blocks_to_swap value in my local code but maybe a better approach could be done here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants