Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relevance Augmentations #215

Open
0x4007 opened this issue Dec 13, 2024 · 18 comments · May be fixed by #227
Open

Relevance Augmentations #215

0x4007 opened this issue Dec 13, 2024 · 18 comments · May be fixed by #227

Comments

@0x4007
Copy link
Member

0x4007 commented Dec 13, 2024

There's a few high impact elements to determine the relevance of a comment. Which is why in the default formatting scoring we have high ($5) credit per instance.

image

Relevance currently doesn't understand image contents.

We can insert the image description inside of its "alt" attribute.

<img width="106" alt="@0x4007 profile pic" src="https://github.com/user-attachments/assets/a4109ba7-fdeb-49e0-b492-039cea79bcc7" />

link

It could also be really interesting to pull the text contents from links to also score their relevance.

We can insert a summary inside of its title attribute.

<a href="https://example.com" title="A summary of the link's contents">Example</a>

code

It would be interesting to discuss this in a separate proposal because this seems like it could be really complex to implement well.

Original context ubiquity-os-marketplace/command-wallet#28 (comment)

Copy link

ubiquity-os-beta bot commented Dec 13, 2024

Note

The following contributors may be suitable for this task:

sshivaditya2019

0% Match ubiquity-os-marketplace/text-vector-embeddings#36

@0x4007
Copy link
Member Author

0x4007 commented Dec 13, 2024

Saving the standardized metadata for the images can be tricky given that our metadata parser in the SDK seems unstable. @whilefoo can you add input on the state of this?

I think @sshivaditya should add input on what's best for the vector embeddings to understand image contents.

The problem I'm trying to solve here is that I don't want to ask LLMs to describe an image both in the conversation rewards and vector embeddings plugins. It's redundant work. So we can solve for this with my HTML comment metadata proposal above.

@sura1-0-1
Copy link

/help

Copy link

Available Commands

Command Description Example
/help List all available commands. /help
/allow Allows the user to modify the given label type. /allow @user1 label
/ask Ask any question about the repository, issue or pull request /ask
/query Returns the user's wallet, access, and multiplier information. /query @UbiquityOS
/start Assign yourself and/or others to the issue/task. /start
/stop Unassign yourself from the issue/task. /stop
/wallet Register your wallet address for payments. Use '/wallet unset' to unlink your wallet. /wallet ubq.eth

Copy link

+ Successfully set wallet

@sura1-0-1
Copy link

/wallet 0xB13260bfEe08DcA208F2ECc735171B21763EaaF6

Copy link

! Failed to register wallet because it is already associated with another user.

@sura1-0-1
Copy link

/wallet 0x331D1C984A43087427BBC224Cb4aD9f019336e75

Copy link

+ Successfully set wallet

@sura1-0-1
Copy link

/start

Copy link

Deadline Sat, Dec 21, 9:51 AM UTC
Beneficiary 0x331D1C984A43087427BBC224Cb4aD9f019336e75

Tip

  • Use /wallet 0x0000...0000 if you want to update your registered payment wallet address.
  • Be sure to open a draft pull request as soon as possible to communicate updates on your progress.
  • Be sure to provide timely updates to us when requested, or you will be automatically unassigned from the task.

@sura1-0-1
Copy link

@0x4007 Can you provide more details on how the relevance score should be calculated? Are there specific weights or factors to consider for text, images, and links?

@0x4007
Copy link
Member Author

0x4007 commented Dec 15, 2024

@0x4007 Can you provide more details on how the relevance score should be calculated? Are there specific weights or factors to consider for text, images, and links?

That's already built and out of scope for this task. Refer to the existing prompts for inspiration on new prompts specific for these.

Copy link

Passed the disqualification threshold and no activity is detected, removing assignees: @sura1-0-1.

@3scava1i3r
Copy link

/start

Copy link

! You do not have the adequate role to start this task (your role is: member). Allowed roles are: collaborator, admin.

@3scava1i3r
Copy link

I finished my work should i PR then?

@3scava1i3r 3scava1i3r linked a pull request Dec 27, 2024 that will close this issue
@3scava1i3r
Copy link

There seems to be this issue that if i have description and title the amount of words are increasing leading to threshhold reach for task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants