Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arena hard elad2 #1026

Merged
merged 110 commits into from
Jul 22, 2024
Merged

Arena hard elad2 #1026

merged 110 commits into from
Jul 22, 2024

Commits on Jul 17, 2024

  1. bug fixes in PairwiseChoiceTemplate

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    fd17810 View commit details
    Browse the repository at this point in the history
  2. add arena hard regex parser operator

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    0e2d7ce View commit details
    Browse the repository at this point in the history
  3. update mt bench card common

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    95ec7fe View commit details
    Browse the repository at this point in the history
  4. update mt bench card common

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    44f398a View commit details
    Browse the repository at this point in the history
  5. add reward bench

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    a5ce257 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    c019b6e View commit details
    Browse the repository at this point in the history
  7. arena hard tasks and cards

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    d9c3960 View commit details
    Browse the repository at this point in the history
  8. update mt bench template

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c5bbade View commit details
    Browse the repository at this point in the history
  9. add duplicate stream operator

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    1f0e2da View commit details
    Browse the repository at this point in the history
  10. add PairwiseComparativeRatingTemplate

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    5c868bc View commit details
    Browse the repository at this point in the history
  11. add card

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    416d9e9 View commit details
    Browse the repository at this point in the history
  12. add card

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    3c89849 View commit details
    Browse the repository at this point in the history
  13. add template

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    186faa5 View commit details
    Browse the repository at this point in the history
  14. add winrate metrics

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    b5bf6a2 View commit details
    Browse the repository at this point in the history
  15. add comparative rating task

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    a02e23c View commit details
    Browse the repository at this point in the history
  16. add ExtractArenaHardNumericalJudgment

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    8893eee View commit details
    Browse the repository at this point in the history
  17. add arena hard cards

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    ddcb726 View commit details
    Browse the repository at this point in the history
  18. add arena hard template

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    a361172 View commit details
    Browse the repository at this point in the history
  19. add weighted winrate metrics

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    e144985 View commit details
    Browse the repository at this point in the history
  20. delete file

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    f5c2ccf View commit details
    Browse the repository at this point in the history
  21. Configuration menu
    Copy the full SHA
    d53de7c View commit details
    Browse the repository at this point in the history
  22. add metric

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    f9123d8 View commit details
    Browse the repository at this point in the history
  23. add metric

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9c97815 View commit details
    Browse the repository at this point in the history
  24. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    111c6a6 View commit details
    Browse the repository at this point in the history
  25. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    35bac84 View commit details
    Browse the repository at this point in the history
  26. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    a56e8f8 View commit details
    Browse the repository at this point in the history
  27. fix template bug

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    7a4ecb0 View commit details
    Browse the repository at this point in the history
  28. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    4b461af View commit details
    Browse the repository at this point in the history
  29. llama 3 update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c3278a6 View commit details
    Browse the repository at this point in the history
  30. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    5ed6fb4 View commit details
    Browse the repository at this point in the history
  31. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    5f336da View commit details
    Browse the repository at this point in the history
  32. update jsons

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    b86c810 View commit details
    Browse the repository at this point in the history
  33. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    44aa2e8 View commit details
    Browse the repository at this point in the history
  34. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    585ae9a View commit details
    Browse the repository at this point in the history
  35. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    fd200ba View commit details
    Browse the repository at this point in the history
  36. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    519c645 View commit details
    Browse the repository at this point in the history
  37. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    85ead7f View commit details
    Browse the repository at this point in the history
  38. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    34f164a View commit details
    Browse the repository at this point in the history
  39. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    0a670b6 View commit details
    Browse the repository at this point in the history
  40. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    506fd49 View commit details
    Browse the repository at this point in the history
  41. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    6741061 View commit details
    Browse the repository at this point in the history
  42. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    dfd84a6 View commit details
    Browse the repository at this point in the history
  43. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    932a0d7 View commit details
    Browse the repository at this point in the history
  44. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    c61ddd2 View commit details
    Browse the repository at this point in the history
  45. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    3d73655 View commit details
    Browse the repository at this point in the history
  46. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    f46f7ae View commit details
    Browse the repository at this point in the history
  47. fix

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    29f9f06 View commit details
    Browse the repository at this point in the history
  48. fix

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    77aafee View commit details
    Browse the repository at this point in the history
  49. fix

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    2c7d1e0 View commit details
    Browse the repository at this point in the history
  50. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    5377f7a View commit details
    Browse the repository at this point in the history
  51. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    a79096b View commit details
    Browse the repository at this point in the history
  52. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    05a2d4f View commit details
    Browse the repository at this point in the history
  53. bluebench related changes

    perlitz authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    71bb7c1 View commit details
    Browse the repository at this point in the history
  54. fix type issue

    Signed-off-by: Yotam Perlitz <yotam.perlitz@ibm.com>
    perlitz authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    74cb737 View commit details
    Browse the repository at this point in the history
  55. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    b742025 View commit details
    Browse the repository at this point in the history
  56. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    2ba6766 View commit details
    Browse the repository at this point in the history
  57. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    e4f446a View commit details
    Browse the repository at this point in the history
  58. prometheus1

    gitMichal authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    ceb11ae View commit details
    Browse the repository at this point in the history
  59. update

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    9cb00cc View commit details
    Browse the repository at this point in the history
  60. fix

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    5519221 View commit details
    Browse the repository at this point in the history
  61. fix

    OfirArviv authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    61ff2aa View commit details
    Browse the repository at this point in the history
  62. merge with arena_branch

    Signed-off-by: Yotam Perlitz <yotam.perlitz@ibm.com>
    perlitz authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    873138c View commit details
    Browse the repository at this point in the history
  63. rebuild catalog

    Signed-off-by: Yotam Perlitz <yotam.perlitz@ibm.com>
    perlitz authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    ff9cbf4 View commit details
    Browse the repository at this point in the history
  64. add debugging to clapnq

    perlitz authored and eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    0a6da25 View commit details
    Browse the repository at this point in the history
  65. Reproduce all artifacts

    eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    eca8649 View commit details
    Browse the repository at this point in the history
  66. Configuration menu
    Copy the full SHA
    258c954 View commit details
    Browse the repository at this point in the history
  67. Add secrets baseline

    Signed-off-by: Elad Venezian <eladv@il.ibm.com>
    eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    432b5e6 View commit details
    Browse the repository at this point in the history
  68. Configuration menu
    Copy the full SHA
    0b3ebc5 View commit details
    Browse the repository at this point in the history
  69. Configuration menu
    Copy the full SHA
    adb421f View commit details
    Browse the repository at this point in the history
  70. Configuration menu
    Copy the full SHA
    dd9a298 View commit details
    Browse the repository at this point in the history
  71. Configuration menu
    Copy the full SHA
    3310968 View commit details
    Browse the repository at this point in the history
  72. Fix test failures

    eladven committed Jul 17, 2024
    Configuration menu
    Copy the full SHA
    6e8c986 View commit details
    Browse the repository at this point in the history
  73. Configuration menu
    Copy the full SHA
    4b1b718 View commit details
    Browse the repository at this point in the history
  74. Configuration menu
    Copy the full SHA
    92b632d View commit details
    Browse the repository at this point in the history
  75. Update docs/docs/examples.rst

    Co-authored-by: Yoav Katz <68273864+yoavkatz@users.noreply.github.com>
    OfirArviv and yoavkatz authored Jul 17, 2024
    Configuration menu
    Copy the full SHA
    e68e042 View commit details
    Browse the repository at this point in the history
  76. Configuration menu
    Copy the full SHA
    82bf199 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2024

  1. Configuration menu
    Copy the full SHA
    157be76 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    53c6ce5 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    16dcec6 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    c0eb6bf View commit details
    Browse the repository at this point in the history
  5. Update docs/docs/examples.rst

    Co-authored-by: Yoav Katz <68273864+yoavkatz@users.noreply.github.com>
    OfirArviv and yoavkatz authored Jul 21, 2024
    Configuration menu
    Copy the full SHA
    64edd8f View commit details
    Browse the repository at this point in the history
  6. bug fix in LoadFromHFSpace

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    ff7fee3 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    2a00756 View commit details
    Browse the repository at this point in the history
  8. revert

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    e1d606e View commit details
    Browse the repository at this point in the history
  9. revert

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    8522030 View commit details
    Browse the repository at this point in the history
  10. update examples

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    6a05fdc View commit details
    Browse the repository at this point in the history
  11. add coment to expain change

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    4042e7b View commit details
    Browse the repository at this point in the history
  12. update to new params usage

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    5955dfa View commit details
    Browse the repository at this point in the history
  13. pr fixes

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    7800998 View commit details
    Browse the repository at this point in the history
  14. pr fixes

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    5a078cb View commit details
    Browse the repository at this point in the history
  15. update

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    29994a3 View commit details
    Browse the repository at this point in the history
  16. update

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    656cd7c View commit details
    Browse the repository at this point in the history
  17. update

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    a42c811 View commit details
    Browse the repository at this point in the history
  18. update

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    e321825 View commit details
    Browse the repository at this point in the history
  19. update

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    e33b475 View commit details
    Browse the repository at this point in the history
  20. Configuration menu
    Copy the full SHA
    d59600c View commit details
    Browse the repository at this point in the history
  21. update

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    80088bf View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    ad32171 View commit details
    Browse the repository at this point in the history
  23. Update prepare/templates/rag/response_generation.py

    Co-authored-by: Yotam Perlitz <perlitz@gmail.com>
    OfirArviv and perlitz authored Jul 21, 2024
    Configuration menu
    Copy the full SHA
    af5ad98 View commit details
    Browse the repository at this point in the history
  24. Update prepare/templates/rag/response_generation.py

    Co-authored-by: Yotam Perlitz <perlitz@gmail.com>
    OfirArviv and perlitz authored Jul 21, 2024
    Configuration menu
    Copy the full SHA
    3ec947f View commit details
    Browse the repository at this point in the history
  25. update

    OfirArviv committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    a2c428e View commit details
    Browse the repository at this point in the history
  26. Configuration menu
    Copy the full SHA
    ab50245 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Configuration menu
    Copy the full SHA
    ec0c0fd View commit details
    Browse the repository at this point in the history
  2. cr fixes

    OfirArviv committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    37e5ad2 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4f70ed8 View commit details
    Browse the repository at this point in the history
  4. llmaj format fix

    OfirArviv committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    b92b2a0 View commit details
    Browse the repository at this point in the history
  5. llmaj format fix

    OfirArviv committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    e69c772 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    c426040 View commit details
    Browse the repository at this point in the history
  7. merge

    OfirArviv committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    aa78fdd View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    c7a48bc View commit details
    Browse the repository at this point in the history