Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(pt): fix lammps nlist sort with large sel #3993

Merged
merged 13 commits into from
Jul 24, 2024

Conversation

iProzd
Copy link
Collaborator

@iProzd iProzd commented Jul 18, 2024

nlist was not sorted and passed into build_multiple_neighbor_list when using dpa2 and large sel (larger than lammps max nnei).

Bug fixed on several error systems but a UT needed.

  • UT for special sel case in dpa2.

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Introduced a new parameter to enhance sorting flexibility in computation methods.
    • Added methods to check if sorted neighbor lists are required, providing improved control over model behavior.
  • Bug Fixes

    • Enhanced functionality related to neighbor list formatting for improved performance across models.
  • Documentation

    • Updated method signatures to reflect new parameters and functionalities, clarifying usage for end-users.

Copy link
Contributor

coderabbitai bot commented Jul 18, 2024

Walkthrough

Walkthrough

The updates enhance the configurability of deep potential molecular dynamics simulations within LAMMPS by introducing the extra_nlist_sort parameter across several methods, thereby improving control over neighbor list sorting. Additionally, the need_sorted_nlist_for_lower method has been implemented in various classes to clarify when a sorted neighbor list is necessary. These modifications aim to optimize performance and accuracy in diverse simulation scenarios.

Changes

File Path Change Summary
deepmd/pt/model/model/ener_model.py Added extra_nlist_sort parameter to forward_lower for enhanced neighbor list sorting.
deepmd/pt/model/model/make_model.py Introduced extra_nlist_sort in call_lower, format_nlist, and _format_nlist for improved neighbor list handling.
deepmd/dpmodel/atomic_model/*.py Added need_sorted_nlist_for_lower method to various atomic model classes for sorted neighbor list indication.
deepmd/pt/model/model/*.py Modified forward_lower in dipole, dos, polar, and spin models to incorporate extra_nlist_sort.
deepmd/dpmodel/descriptor/*.py Added need_sorted_nlist_for_lower method to descriptor classes, indicating sorted neighbor list requirements.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant EnerModel
    participant MakeModel
    participant LAMMPS

    User->>EnerModel: Call forward_lower(extra_nlist_sort=True)
    EnerModel->>MakeModel: Call forward_common_lower(extra_nlist_sort=True)
    MakeModel->>MakeModel: Process with extra_nlist_sort
    MakeModel->>EnerModel: Return results
    EnerModel->>User: Return results

    User->>LAMMPS: Run test suite
    LAMMPS->>LAMMPS: Setup simulations with various configurations
    LAMMPS->>LAMMPS: Compute energies, forces, and virials
    LAMMPS->>User: Validate results with pytest assertions
Loading

Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 11dbada and b62ebc3.

Files selected for processing (15)
  • deepmd/dpmodel/atomic_model/linear_atomic_model.py (1 hunks)
  • deepmd/dpmodel/atomic_model/pairtab_atomic_model.py (1 hunks)
  • deepmd/dpmodel/descriptor/dpa1.py (2 hunks)
  • deepmd/dpmodel/descriptor/repformers.py (1 hunks)
  • deepmd/dpmodel/descriptor/se_r.py (1 hunks)
  • deepmd/pt/model/atomic_model/linear_atomic_model.py (1 hunks)
  • deepmd/pt/model/atomic_model/pairtab_atomic_model.py (1 hunks)
  • deepmd/pt/model/descriptor/descriptor.py (1 hunks)
  • deepmd/pt/model/descriptor/repformers.py (1 hunks)
  • deepmd/pt/model/descriptor/se_a.py (2 hunks)
  • deepmd/pt/model/descriptor/se_atten.py (1 hunks)
  • deepmd/pt/model/descriptor/se_r.py (1 hunks)
  • deepmd/pt/model/descriptor/se_t.py (2 hunks)
  • deepmd/pt/model/model/spin_model.py (4 hunks)
  • source/tests/universal/common/cases/model/utils.py (1 hunks)
Files skipped from review due to trivial changes (1)
  • deepmd/dpmodel/descriptor/se_r.py
Files skipped from review as they are similar to previous changes (13)
  • deepmd/dpmodel/atomic_model/pairtab_atomic_model.py
  • deepmd/dpmodel/descriptor/dpa1.py
  • deepmd/dpmodel/descriptor/repformers.py
  • deepmd/pt/model/atomic_model/linear_atomic_model.py
  • deepmd/pt/model/atomic_model/pairtab_atomic_model.py
  • deepmd/pt/model/descriptor/descriptor.py
  • deepmd/pt/model/descriptor/repformers.py
  • deepmd/pt/model/descriptor/se_a.py
  • deepmd/pt/model/descriptor/se_atten.py
  • deepmd/pt/model/descriptor/se_r.py
  • deepmd/pt/model/descriptor/se_t.py
  • deepmd/pt/model/model/spin_model.py
  • source/tests/universal/common/cases/model/utils.py
Additional comments not posted (1)
deepmd/dpmodel/atomic_model/linear_atomic_model.py (1)

99-101: LGTM! The new method need_sorted_nlist_for_lower is clear and serves its purpose.

This method explicitly indicates that a sorted neighbor list is required for the forward_lower functionality.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

deepmd/pt/model/model/make_model.py Fixed Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (3)
deepmd/pt/model/model/make_model.py (3)

243-244: Document the new parameter extra_nlist_sort.

The new parameter extra_nlist_sort should be documented in the method's docstring for clarity.

+            extra_nlist_sort
+                whether to forcibly sort the nlist.

383-384: Document the new parameter extra_nlist_sort.

The new parameter extra_nlist_sort should be documented in the method's docstring for clarity.

+            extra_nlist_sort
+                whether to forcibly sort the nlist.

408-408: Issues found with the new parameter extra_nlist_sort:

  • The parameter extra_nlist_sort is added to the _format_nlist function but is not utilized within the function.
  • There are no test cases covering the new parameter extra_nlist_sort.

Please ensure that the parameter is correctly handled within the _format_nlist function and add appropriate test cases to verify its behavior.

Analysis chain

Verify the impact of the new parameter extra_nlist_sort.

The new parameter extra_nlist_sort is added with a default value of False. Verify if this change impacts the behavior of the method and ensure it is tested appropriately.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the impact of the new parameter `extra_nlist_sort`.

# Test: Search for the function usage. Expect: Ensure the new parameter is handled correctly.
rg --type python -A 5 $'_format_nlist'

Length of output: 68


Script:

#!/bin/bash
# Description: Verify the impact of the new parameter `extra_nlist_sort`.

# Test: Search for the function usage. Expect: Ensure the new parameter is handled correctly.
rg --type py -A 5 '_format_nlist'

Length of output: 1610


Script:

#!/bin/bash
# Description: Verify the handling of the new parameter `extra_nlist_sort` within the `_format_nlist` function and check for related test cases.

# Test: Search for the usage of `extra_nlist_sort` within the `_format_nlist` function.
rg --type py 'extra_nlist_sort' -A 10 'def _format_nlist'

# Test: Search for any test cases related to `_format_nlist` or `extra_nlist_sort`.
rg --type py '_format_nlist|extra_nlist_sort' --glob '*test*'

Length of output: 179


Script:

#!/bin/bash
# Description: Verify the handling of the new parameter `extra_nlist_sort` within the `_format_nlist` function and check for related test cases.

# Test: Search for the usage of `extra_nlist_sort` within the `_format_nlist` function.
rg --type py -A 10 'def _format_nlist' | rg 'extra_nlist_sort'

# Test: Search for any test cases related to `_format_nlist` or `extra_nlist_sort`.
rg --type py '_format_nlist|extra_nlist_sort' --glob '*test*'

Length of output: 206

Copy link

codecov bot commented Jul 18, 2024

Codecov Report

Attention: Patch coverage is 90.54054% with 7 lines in your changes missing coverage. Please review.

Project coverage is 82.84%. Comparing base (269ed3e) to head (b62ebc3).
Report is 103 commits behind head on devel.

Files with missing lines Patch % Lines
deepmd/dpmodel/atomic_model/linear_atomic_model.py 50.00% 1 Missing ⚠️
...eepmd/dpmodel/atomic_model/pairtab_atomic_model.py 50.00% 1 Missing ⚠️
deepmd/dpmodel/descriptor/repformers.py 50.00% 1 Missing ⚠️
...epmd/pt/model/atomic_model/pairtab_atomic_model.py 50.00% 1 Missing ⚠️
deepmd/pt/model/descriptor/repformers.py 50.00% 1 Missing ⚠️
deepmd/pt/model/model/frozen.py 50.00% 1 Missing ⚠️
deepmd/pt/model/model/spin_model.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            devel    #3993   +/-   ##
=======================================
  Coverage   82.83%   82.84%           
=======================================
  Files         522      522           
  Lines       50936    51004   +68     
  Branches     3015     3015           
=======================================
+ Hits        42195    42253   +58     
- Misses       7804     7813    +9     
- Partials      937      938    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@njzjz
Copy link
Member

njzjz commented Jul 18, 2024

when using dpa2 and large sel

Do you mean it only affects dpa2 but not other descriptors?

Co-authored-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Signed-off-by: Duo <50307526+iProzd@users.noreply.github.com>
@iProzd
Copy link
Collaborator Author

iProzd commented Jul 18, 2024

when using dpa2 and large sel

Do you mean it only affects dpa2 but not other descriptors?

yes, since only dpa2 (and linear_atomic_model) uses build_multiple_neighbor_list, which requires nlist to be sorted: https://github.com/deepmodeling/deepmd-kit/blob/devel/deepmd/pt/utils/nlist.py#L220-L222

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (16)
source/lmp/tests/test_lammps_dpa_sel_pt.py (16)

22-34: Consider adding comments to explain variable purposes.

Adding comments to explain the purpose of each variable can improve code readability and maintainability.


308-317: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


320-340: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


343-367: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


369-404: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


407-433: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


436-466: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


469-478: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


481-492: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


495-519: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


522-549: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


552-593: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


596-626: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


629-663: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


666-675: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.


678-724: Consider adding comments to explain test steps.

Adding comments to explain each step of the test can improve code readability and maintainability.

source/lmp/tests/test_lammps_dpa_sel_pt.py Show resolved Hide resolved
@njzjz
Copy link
Member

njzjz commented Jul 18, 2024

when using dpa2 and large sel

Do you mean it only affects dpa2 but not other descriptors?

yes, since only dpa2 (and linear_atomic_model) uses build_multiple_neighbor_list, which requires nlist to be sorted: https://github.com/deepmodeling/deepmd-kit/blob/devel/deepmd/pt/utils/nlist.py#L220-L222

In this case, why do you only modify the energy model?

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (2)
deepmd/dpmodel/model/make_model.py (2)

Ensure correct usage of extra_nlist_sort.

The extra_nlist_sort parameter is not consistently provided in all calls to format_nlist. Ensure that all calls to format_nlist include the extra_nlist_sort parameter.

  • Missing extra_nlist_sort parameter:

    • source/tests/pt/model/test_dp_model.py (multiple instances)
    • source/tests/common/dpmodel/test_nlist.py (multiple instances)
  • Correct usage:

    • deepmd/dpmodel/model/make_model.py (lines 342-349 and other instances)

Please update the test files to include the extra_nlist_sort parameter in all calls to format_nlist.

Analysis chain

Line range hint 319-346:
Ensure correct usage of extra_nlist_sort.

The extra_nlist_sort parameter is correctly integrated into the format_nlist method. Ensure that all calls to format_nlist provide the appropriate value for extra_nlist_sort.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all calls to `format_nlist` provide the appropriate value for `extra_nlist_sort`.

# Test: Search for the function usage. Expect: Only occurances of the new signature.
rg --type python -A 5 $'format_nlist'

Length of output: 67


Script:

#!/bin/bash
# Description: Verify all calls to `format_nlist` provide the appropriate value for `extra_nlist_sort`.

# Test: Search for the function usage. Expect: Only occurrences of the new signature.
rg --type py -A 5 $'format_nlist'

Length of output: 6211


217-222: Ensure correct usage of extra_nlist_sort.

The extra_nlist_sort parameter is correctly integrated into the call_lower method, but not all calls to call_lower provide the appropriate value for extra_nlist_sort. The following instances need to be updated to include this parameter:

  • source/tests/common/dpmodel/test_dp_model.py
  • source/tests/pt/model/test_dp_model.py
  • source/tests/pt/model/test_ener_spin_model.py

Please update these calls to include the extra_nlist_sort parameter to ensure consistency and proper functionality.

Analysis chain

Ensure correct usage of extra_nlist_sort.

The extra_nlist_sort parameter is correctly integrated into the call_lower method. Ensure that all calls to call_lower provide the appropriate value for extra_nlist_sort.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all calls to `call_lower` provide the appropriate value for `extra_nlist_sort`.

# Test: Search for the function usage. Expect: Only occurances of the new signature.
rg --type python -A 5 $'call_lower'

Length of output: 65


Script:

#!/bin/bash
# Description: Verify all calls to `call_lower` provide the appropriate value for `extra_nlist_sort`.

# Test: Search for the function usage. Expect: Only occurrences of the new signature.
rg --type py -A 5 $'call_lower'

Length of output: 5318

@iProzd iProzd requested review from njzjz and wanghan-iapcm July 22, 2024 07:30
source/lmp/tests/test_lammps_dpa_sel_pt.py Outdated Show resolved Hide resolved
deepmd/pt/model/model/ener_model.py Outdated Show resolved Hide resolved
deepmd/dpmodel/descriptor/dpa2.py Outdated Show resolved Hide resolved
deepmd/dpmodel/descriptor/repformers.py Outdated Show resolved Hide resolved
@iProzd iProzd requested a review from wanghan-iapcm July 23, 2024 04:09
@iProzd iProzd requested a review from njzjz July 23, 2024 04:24
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

deepmd/dpmodel/descriptor/hybrid.py Show resolved Hide resolved
@wanghan-iapcm wanghan-iapcm enabled auto-merge July 23, 2024 23:59
@wanghan-iapcm wanghan-iapcm added this pull request to the merge queue Jul 24, 2024
Merged via the queue into deepmodeling:devel with commit a708c7a Jul 24, 2024
60 checks passed
iProzd added a commit to iProzd/deepmd-kit that referenced this pull request Jul 29, 2024
iProzd added a commit that referenced this pull request Aug 1, 2024
mtaillefumier pushed a commit to mtaillefumier/deepmd-kit that referenced this pull request Sep 18, 2024
`nlist` was not sorted and passed into `build_multiple_neighbor_list`
when using dpa2 and large sel (larger than lammps max nnei).

Bug fixed on several error systems but a UT needed.

- [x] UT for special sel case in dpa2.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Summary by CodeRabbit

- **New Features**
- Introduced a new parameter to enhance sorting flexibility in
computation methods.
- Added methods to check if sorted neighbor lists are required,
providing improved control over model behavior.

- **Bug Fixes**
- Enhanced functionality related to neighbor list formatting for
improved performance across models.

- **Documentation**
- Updated method signatures to reflect new parameters and
functionalities, clarifying usage for end-users.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Duo <50307526+iProzd@users.noreply.github.com>
Co-authored-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants