Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GenAI] Roberta model training #14

Open
head-iie-vnr opened this issue Jun 30, 2024 · 2 comments
Open

[GenAI] Roberta model training #14

head-iie-vnr opened this issue Jun 30, 2024 · 2 comments

Comments

@head-iie-vnr
Copy link
Contributor

No description provided.

@head-iie-vnr
Copy link
Contributor Author

When I gave shorter context the confidence was higher.

Test data

test_data = [
    {
     "context": "Mohandas Karamchand Gandhi was born on October 2, 1869, in Porbandar, a small town in present-day Gujarat, India.", 
     "question": "When was Mahatma Gandhi born?"},
    {
     "context": "Mohandas Karamchand Gandhi was born on October 2, 1869, in Porbandar, a small town in present-day Gujarat, India.", 
     "question": "Where was Gandhi born?"},
    # Add more test examples
]

Output

question answer score
When was Mahatma Gandhi born? October 2, 1869 0.957528
Where was Gandhi born? Porbandar 0.731315

@head-iie-vnr
Copy link
Contributor Author

when I gave larger context for the same questions & changed name the answer was same but confidence decreased

question answer score
When was Mahatma Gandhi born? October 2, 1869 0.906785
Where was Mahatma Gandhi born? Porbandar 0.389754
Where was Gandhi born? Porbandar 0.396170
Where was Mohandas Karamchand Gandhi born? Porbandar 0.416316
Where was he born? Porbandar 0.384240

head-iie-vnr added a commit that referenced this issue Jun 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant