Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sweep: Auto Summarize Working Context (βœ“ Sandbox Passed) #24

Closed
wants to merge 16 commits into from

Conversation

sweep-ai[bot]
Copy link
Contributor

@sweep-ai sweep-ai bot commented Dec 29, 2023

PR Feedback (click)

  • πŸ‘ Sweep Did Well
  • πŸ‘Ž Sweep Needs Improvement

Description

This pull request introduces changes to the codebase that implement auto summarization of the working context in the coding agent. It also includes tests to verify the functionality of the new features.

Summary

  • Added a new condition in the query method of coding_agent.py to print the summarized context if the turn counter is 0.
  • Imported the summarize function from gensim.summarize in memory_manager.py.
  • Added a new method summarize_context in memory_manager.py to summarize the context using the imported function.
  • Modified the update_context method in memory_manager.py to update the context and reset the turn counter to 0 after every 5th update.
  • Added two new tests in test_memory_manager.py to verify the functionality of summarize_context and the turn counter reset.

Fixes #23.


πŸŽ‰ Latest improvements to Sweep:

  • We just released a dashboard to track Sweep's progress on your issue in real-time, showing every stage of the process – from search to planning and coding.
  • Sweep uses OpenAI's latest Assistant API to plan code changes and modify code! This is 3x faster and significantly more reliable as it allows Sweep to edit code and validate the changes in tight iterations, the same way as a human would.
  • Try using the GitHub issues extension to create Sweep issues directly from your editor! GitHub Issues and Pull Requests.

πŸ’‘ To get Sweep to edit this pull request, you can:

  • Comment below, and Sweep can edit the entire PR
  • Comment on a file, Sweep will only modify the commented file
  • Edit the original issue to get Sweep to recreate the PR from scratch

Copy link
Contributor Author

sweep-ai bot commented Dec 29, 2023

Sandbox Executions

  • Running GitHub Actions for backend/memory/memory_manager.py βœ“
Check backend/memory/memory_manager.py with contents:

Ran GitHub Actions for b11cd11459272b993c369412c13f17cf0877806a:

  • Running GitHub Actions for backend/tests/test_memory_manager.py βœ“
Check backend/tests/test_memory_manager.py with contents:

Ran GitHub Actions for 459d4cf4b6896fb17bc275efaa5e919603af55ad:

  • Running GitHub Actions for backend/agent/coding_agent.py βœ“
Check backend/agent/coding_agent.py with contents:

Ran GitHub Actions for 1e0a34a20233a2c8b5293f950db716e9dea2a755:

Copy link
Contributor Author

sweep-ai bot commented Dec 29, 2023

Rollback Files For Sweep

  • Rollback changes to True

Copy link
Contributor Author

sweep-ai bot commented Dec 29, 2023

Apply Sweep Rules to your PR?

  • Apply: Leftover TODOs in the code should be handled.
  • Apply: All new business logic should have corresponding unit tests in the tests/ directory.
  • Apply: Any clearly inefficient or repeated code should be optimized or refactored.
  • Apply: Update the README.md file to reflect the changes in this pull request.
  • Apply: Add a changelog entry to CHANGELOG.md.

@sweep-ai sweep-ai bot added the sweep label Dec 29, 2023
@sweep-ai sweep-ai bot mentioned this pull request Dec 29, 2023
6 tasks
Copy link
Contributor

PR Analysis

  • 🎯 Main theme: This PR introduces auto summarization of the working context in the coding agent and includes tests to verify the new features.
  • πŸ“ PR summary: The PR modifies the query method in coding_agent.py to print the summarized context if the turn counter is 0. It also adds a summarize_context method in memory_manager.py to summarize the context and modifies the update_context method to reset the turn counter after every 5th update. Two new tests are added to verify these functionalities.
  • πŸ“Œ Type of PR: Enhancement
  • πŸ§ͺ Relevant tests added: Yes
  • ⏱️ Estimated effort to review [1-5]: 3, because the PR involves changes in multiple files and introduces new functionalities which need to be thoroughly reviewed.
  • πŸ”’ Security concerns: No security concerns found

PR Feedback

πŸ’‘ General suggestions: The PR is well-structured and includes relevant tests for the new functionalities. However, it would be beneficial to handle potential exceptions that might occur during the summarization process. Also, consider using a more descriptive variable name than turn_counter.

πŸ€– Code feedback:
relevant filebackend/memory/memory_manager.py
suggestion Β Β Β Β Β 

Consider handling potential exceptions that might occur during the summarization process. This could be done by wrapping the summarize function call inside a try/except block. [important]

relevant linereturn summarize(self.context)

relevant filebackend/memory/memory_manager.py
suggestion Β Β Β Β Β 

Consider using a more descriptive variable name than turn_counter. A name like context_update_counter might be more indicative of its purpose. [medium]

relevant lineself.turn_counter += 1

✨ Usage tips:

To invoke the PR-Agent, add a comment using one of the following commands:

  • /review: Request a review of your Pull Request.
  • /describe: Update the PR title and description based on the contents of the PR.
  • /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
  • /ask <QUESTION>: Ask a question about the PR.
  • /update_changelog: Update the changelog based on the PR's contents.
  • /add_docs πŸ’Ž: Generate docstring for new components introduced in the PR.
  • /generate_labels πŸ’Ž: Generate labels for the PR based on the PR's contents.
  • /analyze πŸ’Ž: Automatically analyzes the PR, and presents changes walkthrough for each component.

See the tools guide for more details.
To edit any configuration parameter from the configuration.toml, add --config_path=new_value.
For example: /review --pr_reviewer.extra_instructions="focus on the file: ..."
To list the possible configuration parameters, add a /config comment.

@blazickjp
Copy link
Owner

Add gensim to requirements.txt

Copy link
Contributor Author

sweep-ai bot commented Dec 30, 2023

πŸš€ Wrote Changes

Done.

@blazickjp
Copy link
Owner

Lets use OpenAI for the summary.

Copy link
Contributor Author

sweep-ai bot commented Dec 30, 2023

πŸš€ Wrote Changes

Done.

@blazickjp
Copy link
Owner

/improve

Copy link
Contributor Author

sweep-ai bot commented Dec 30, 2023

πŸš€ Wrote Changes

Done.

Comment on lines +118 to +119
if self.memory_manager.working_context.turn_counter == 0:
print("Summarized Context: ", self.memory_manager.working_context.context)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Instead of directly printing the summarized context, consider returning it as part of the response from the query method. This would make the output more testable and flexible for different interfaces. [enhancement]

Suggested change
if self.memory_manager.working_context.turn_counter == 0:
print("Summarized Context: ", self.memory_manager.working_context.context)
if self.memory_manager.working_context.turn_counter == 0:
summarized_context = self.memory_manager.working_context.context
return {"Summarized Context": summarized_context}

Comment on lines 98 to 101
def summarize_context(self):
# Call OpenAI API to get the summary
response = self.client.create_summary(self.context)
return response.choices[0].text.strip()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The summarize_context method currently returns the summary directly. It would be better to update the context attribute of the WorkingContext object within this method. This would ensure that the summarized context is used in subsequent operations. [enhancement]

Suggested change
def summarize_context(self):
# Call OpenAI API to get the summary
response = self.client.create_summary(self.context)
return response.choices[0].text.strip()
def summarize_context(self):
# Call OpenAI API to get the summary
response = self.client.create_summary(self.context)
self.context = response.choices[0].text.strip()

Comment on lines 331 to 334
self.turn_counter += 1
if self.turn_counter == 5:
self.context = self.summarize_context()
self.turn_counter = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The update_context method currently increments the turn_counter and resets it after 5 turns. However, it does not handle the case where the turn_counter exceeds 5. Consider adding a condition to reset the turn_counter when it exceeds 5. [bug]

Suggested change
self.turn_counter += 1
if self.turn_counter == 5:
self.context = self.summarize_context()
self.turn_counter = 0
self.turn_counter = (self.turn_counter + 1) % 6
if self.turn_counter == 0:
self.context = self.summarize_context()

Comment on lines 81 to 87
# Arrange: Reset the turn counter to 0
self.memory_manager.working_context.turn_counter = 0
# Act: Call the update_context method 5 times
for _ in range(5):
asyncio.run(self.memory_manager.update_context())
# Assert: Check whether the turn_counter is reset to 0
assert self.memory_manager.working_context.turn_counter == 0, 'Turn counter was not reset after 5th update'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The test test_turn_counter_reset_after_5th_update currently checks if the turn_counter is reset to 0 after 5 updates. However, it does not check if the context is summarized. Consider adding an assertion to check if the context is summarized after 5 updates. [enhancement]

Suggested change
# Arrange: Reset the turn counter to 0
self.memory_manager.working_context.turn_counter = 0
# Act: Call the update_context method 5 times
for _ in range(5):
asyncio.run(self.memory_manager.update_context())
# Assert: Check whether the turn_counter is reset to 0
assert self.memory_manager.working_context.turn_counter == 0, 'Turn counter was not reset after 5th update'
def test_turn_counter_reset_after_5th_update(self):
# Arrange: Reset the turn counter to 0
self.memory_manager.working_context.turn_counter = 0
original_context = self.memory_manager.working_context.context
# Act: Call the update_context method 5 times
for _ in range(5):
asyncio.run(self.memory_manager.update_context())
# Assert: Check whether the turn_counter is reset to 0 and context is summarized
assert self.memory_manager.working_context.turn_counter == 0, 'Turn counter was not reset after 5th update'
assert self.memory_manager.working_context.context != original_context, 'Context was not summarized after 5th update'

@blazickjp
Copy link
Owner

Says it failed the pytest check

Copy link
Contributor Author

sweep-ai bot commented Dec 30, 2023

πŸš€ Wrote Changes

Done.

@blazickjp blazickjp closed this Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sweep: Auto Summarize Working Context
1 participant