Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving Recommendations #55

Open
0x4007 opened this issue Dec 4, 2024 · 3 comments
Open

Improving Recommendations #55

0x4007 opened this issue Dec 4, 2024 · 3 comments

Comments

@0x4007
Copy link
Member

0x4007 commented Dec 4, 2024

Cool to see it always recommends at least one person but it is very much not relevant.

Whats the timeline looking like to populate the database by scraping the existing closed issues and their assignees @sshivaditya2019?

repo/org/global Context Matching

I think we should optimize our logic. Tasks that may look similar but posted in two different repositories may in reality be quite different. Normally I would prefer to matchmake against those who completed similar tasks within the same repository, but we can "fallback" to org context (try to matchmake against any repository's task in the org) and then a final "fallback" to global context.

Perhaps the simplest way to calculate this could be to subtract relevance "points"

Example:

0% adjustment: Same repository matching percentage 
-25% adjustment: Same organization
-50%: Global

Meaning for an identical task posted in three different locations:

same repo = 100% match
same org, different repo = 75% match
different org = 50% match

Just an idea but open to discussing to make this more accurate and simple to implement.

Recent Code Changes Fallback

Another idea to always try and match somebody is to check how much of the codebase was authored by whom. If nobody hits a relevance of greater than 25% we can fallback to assigning whoever wrote most of the code in the repository. Surely we can find the stats somewhere.

Future Considerations

Because we want the system to always be assigning the right contributor at the right time in the future, we should consider the output of the /start command as well, with all its checks. If UbiquityOS is able to run /start on behalf of others this can solve most of the problems I can think of.

Specifically, imagine the scenario where the same top contributor is always recommended to work on the newly posted tasks. With a concurrent task limit of 2, they will quickly be saturated with work. We should be able to detect this and fallback/recommend/assign the next best.

Original Context

Note

The following contributors may be suitable for this task:

gentlementlegen

1% Match ubiquity-os/ubiquity-os-kernel#125

Originally posted by @ubiquity-os-beta[bot] in ubiquity-os-marketplace/daemon-disqualifier#57 (comment)

@shiv810
Copy link
Collaborator

shiv810 commented Dec 5, 2024

The recommendation of 1% or 0% is due to the alwaysRecommend flag. A straightforward solution would be to populate the database with completed entries, which would give the system more context to work with. I suggest we begin by scraping and adding the completed tasks to the database, and then, we can explore incorporating relevance points.

@0x4007
Copy link
Member Author

0x4007 commented Dec 6, 2024

I set up another task for that so keep me posted there for scraping.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants