feat(pt): support fine-tuning from random fitting #3914
Conversation
WalkthroughWalkthroughThe recent changes introduce new functionality for handling the initialization of model branches in a deep learning model, specifically when fine-tuning with randomly chosen branches. The updates also ensure correct initialization of the fitting net based on predefined rules, and provide clearer documentation for these processes. Tests are updated accordingly to ensure these new functionalities are correctly implemented and verified. Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Trainer
participant Model
participant FittingNet
User->>Trainer: Configure fine-tuning with `--model-branch RANDOM`
Trainer->>Model: Initialize
Model->>FittingNet: Check `model_branch_from`
alt model_branch_from is RANDOM
FittingNet-->>FittingNet: Initialize with random weights
else model_branch_from is pre-trained
FittingNet-->>FittingNet: Initialize with pre-trained weights
end
Model->>Trainer: Initialization complete
Trainer->>User: Fine-tuning setup complete
Recent review detailsConfiguration used: CodeRabbit UI Files selected for processing (2)
Files skipped from review as they are similar to previous changes (1)
Additional context usedRuff
Additional comments not posted (2)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## devel #3914 +/- ##
==========================================
- Coverage 82.87% 82.87% -0.01%
==========================================
Files 519 519
Lines 50666 50670 +4
Branches 3015 3015
==========================================
+ Hits 41990 41993 +3
- Misses 7739 7740 +1
Partials 937 937 ☔ View full report in Codecov by Sentry. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Outside diff range and nitpick comments (2)
deepmd/pt/entrypoints/main.py (2)
Line range hint
117-117: Remove unused variablef.The variable
fis declared but not used, which could lead to confusion and is not efficient.- with h5py.File(stat_file_path_single, "w") as f: + with h5py.File(stat_file_path_single, "w") as _:
Line range hint
382-385: Simplify argument parsing using a ternary operator.Replacing the
if-elseblock with a ternary operator simplifies the code and enhances readability.- if not isinstance(args, argparse.Namespace): - FLAGS = parse_args(args=args) - else: - FLAGS = args + FLAGS = parse_args(args=args) if not isinstance(args, argparse.Namespace) else args
Support fine-tuning from random fitting in single-from-single fine-tuning. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced the ability to randomly initialize fitting nets for fine-tuning by setting the model branch to "RANDOM". - Added support for "RANDOM" model branches in multitask pre-trained model scenarios. - **Documentation** - Updated fine-tuning documentation to include details on handling fitting net weights and using the `--model-branch RANDOM` parameter. - **Tests** - Added tests to verify the random initialization of fitting nets and parameter closeness checks during fine-tuning. - Updated test cases to include "RANDOM" model branches in the output. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Support fine-tuning from random fitting in single-from-single fine-tuning.
Summary by CodeRabbit
New Features
Documentation
--model-branch RANDOMparameter.Tests