Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: automatic evaluation functionalities #2058

Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
e4c09b9
Merge branch 'main' of https://github.com/Agenta-AI/agenta into enhan…
ashrafchowdury Sep 3, 2024
0a7e551
ui(frontend): automatic eval funcational table
ashrafchowdury Sep 3, 2024
71c36e2
Merge branch 'AGE-587/-implement-evaluation-main-page' of https://git…
ashrafchowdury Sep 3, 2024
1fe2bab
fix(frontend): search issue with numbers
ashrafchowdury Sep 3, 2024
32ea255
Merge branch 'AGE-587/-implement-evaluation-main-page' of https://git…
ashrafchowdury Sep 4, 2024
cacad57
ui(frontend): added table results column
ashrafchowdury Sep 4, 2024
a1cbbd6
refactor(frontend): removed unsed code
ashrafchowdury Sep 4, 2024
c7f1afd
fix(frontend): failing cypress test due to layout change
ashrafchowdury Sep 5, 2024
827680e
enhance(frontend): improved structure
ashrafchowdury Sep 5, 2024
d8bca04
fix(frontend): prettier format
ashrafchowdury Sep 5, 2024
baf3e68
fix(frontend): lint error
ashrafchowdury Sep 5, 2024
af8728e
Merge branch 'AGE-587/-implement-evaluation-main-page' into enhance/a…
bekossy Sep 5, 2024
cc8cf94
Merge branch 'AGE-587/-implement-evaluation-main-page' of https://git…
ashrafchowdury Sep 5, 2024
d04631c
Merge branch 'enhance/automatic-tab-functionalitis' of https://github…
ashrafchowdury Sep 5, 2024
c43e6c1
fix(frontend): added EvaluationErrorPopover component and improve eva…
bekossy Sep 5, 2024
11bcc90
Merge branch 'enhance/automatic-tab-functionalitis' of https://github…
ashrafchowdury Sep 6, 2024
9cf40f7
enhance(frontend): edit columns and short columns
ashrafchowdury Sep 6, 2024
bbaf1e9
Merge branch 'AGE-587/-implement-evaluation-main-page' of https://git…
ashrafchowdury Sep 6, 2024
2ab3da3
fix(frontend): fixed status update issue
ashrafchowdury Sep 6, 2024
2d13427
Merge branch 'AGE-587/-implement-evaluation-main-page' of https://git…
ashrafchowdury Sep 8, 2024
0b8dbe7
Merge branch 'AGE-587/-implement-evaluation-main-page' into enhance/a…
bekossy Sep 8, 2024
10098ab
fix(frontend): removed bad code
bekossy Sep 8, 2024
74c32ea
Merge branch 'enhance/automatic-tab-functionalitis' of https://github…
ashrafchowdury Sep 9, 2024
b8480d3
test(frontend): fixed evaluator tests
ashrafchowdury Sep 9, 2024
3d856ee
Merge branch 'AGE-587/-implement-evaluation-main-page' into enhance/a…
bekossy Sep 9, 2024
5ebeb6c
fix(frontend): improved StatusRenderer logic to update status count
bekossy Sep 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 9 additions & 13 deletions agenta-web/cypress/e2e/eval.comparison.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,20 +39,16 @@ describe("Evaluation Comparison Test", function () {

context("Executing Evaluation Comparison Workflow", () => {
beforeEach(() => {
cy.visit(`/apps/${app_id}/evaluations/results`)
cy.location("pathname").should("include", "/evaluations/results")
cy.visit(`/apps/${app_id}/evaluations`)
cy.location("pathname").should("include", "/evaluations")
})

it("Should create 2 new Evaluations", () => {
cy.request({
url: `${Cypress.env().baseApiURL}/evaluations/?app_id=${app_id}`,
method: "GET",
}).then((resp) => {
if (resp.body.length) {
cy.get('[data-cy="new-evaluation-button"]').click()
} else {
cy.get('[data-cy="new-evaluation-button__no_variants"]').click()
}
cy.get('[data-cy="new-evaluation-button"]').click()
})
cy.get(".ant-modal-content").should("exist")

Expand All @@ -73,19 +69,19 @@ describe("Evaluation Comparison Test", function () {
})

it("Should verify that there are completed evaluations in the list", () => {
cy.get('.ag-row[row-index="0"]').should("exist")
cy.get('.ag-row[row-index="1"]').should("exist")
cy.get('.ag-cell[col-id="status"]', {timeout: 60000})
cy.get(".ant-table-row").eq(0).should("exist")
cy.get(".ant-table-row").eq(1).should("exist")
cy.get('[data-cy="evaluation-status-cell"]', {timeout: 60000})
.eq(0)
.should("contain.text", "Completed")
cy.get('.ag-cell[col-id="status"]', {timeout: 60000})
cy.get('[data-cy="evaluation-status-cell"]', {timeout: 60000})
.eq(1)
.should("contain.text", "Completed")
})

it("Should select 2 evaluations, click on the compare button, and successfully navigate to the comparison page", () => {
cy.get("div.ag-selection-checkbox input").eq(0).check()
cy.get("div.ag-selection-checkbox input").eq(1).check()
cy.get(".ant-checkbox-input").eq(0).check()

cy.get('[data-cy="evaluation-results-compare-button"]').should("not.be.disabled")
cy.get('[data-cy="evaluation-results-compare-button"]').click()
cy.location("pathname").should("include", "/evaluations/results/compare")
Expand Down
18 changes: 10 additions & 8 deletions agenta-web/cypress/e2e/eval.evaluations.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ describe("Evaluations CRUD Operations Test", function () {

context("Executing Evaluations CRUD operations", () => {
beforeEach(() => {
cy.visit(`/apps/${app_id}/evaluations/results`)
cy.location("pathname").should("include", "/evaluations/results")
cy.visit(`/apps/${app_id}/evaluations`)
cy.location("pathname").should("include", "/evaluations")
})

it("Should successfully create an Evaluation", () => {
Expand All @@ -26,15 +26,17 @@ describe("Evaluations CRUD Operations Test", function () {
})

it("Should verify the successful creation and completion of the evaluation", () => {
cy.get('.ag-row[row-index="0"]').should("exist")
cy.get('.ag-cell[col-id="status"]').should("contain.text", "Completed")
cy.get(".ant-table-row").eq(0).should("exist")
cy.get('[data-cy="evaluation-status-cell"]').should("contain.text", "Completed")
})

it("Should select evaluation and successfully delete it", () => {
cy.get(".ag-root-wrapper").should("exist")
cy.get("div.ag-selection-checkbox input").eq(0).check()
cy.get(":nth-child(1) > .ant-btn > .ant-btn-icon > .anticon > svg").click()
cy.get(".ant-modal-confirm-btns > :nth-child(2) > span").click()
cy.get(".ant-checkbox-wrapper").should("exist")
cy.get(".ant-checkbox-input").eq(0).check()
cy.get('[data-cy="delete-evaluation-button"]').click()

cy.get(".ant-modal-content").should("exist")
cy.get(".ant-modal-footer > .ant-btn-primary").click()
})
})

Expand Down
13 changes: 6 additions & 7 deletions agenta-web/cypress/e2e/eval.scenarios.cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,23 @@ describe("Evaluation Scenarios Test", function () {

context("Executing Evaluation Scenarios Workflow", () => {
beforeEach(() => {
cy.visit(`/apps/${app_id}/evaluations/results`)
cy.location("pathname").should("include", "/evaluations/results")
cy.visit(`/apps/${app_id}/evaluations`)
cy.location("pathname").should("include", "/evaluations")
})

it("Should successfully create an Evaluation", () => {
cy.createNewEvaluation()
})

it("Should verify that evalaution was created and completed successfully", () => {
cy.get('.ag-row[row-index="0"]').should("exist")
cy.get('.ag-cell[col-id="status"]').should("contain.text", "Completed")
cy.get(".ant-table-row").eq(0).should("exist")
cy.get('[data-cy="evaluation-status-cell"]').should("contain.text", "Completed")
})

it("Should double click on the Evaluation and successfully navigate to the evalaution results page", () => {
cy.get(".ag-root-wrapper").should("exist")
cy.get('.ag-row-first > [col-id="aggregated_results"]').click()
cy.get(".ant-table-row").eq(0).should("exist")
cy.get(".ant-table-row").click()
cy.wait(1000)
cy.get(".ag-cell-focus").dblclick()
cy.contains(/Evaluation Results/i)
cy.get('[data-cy="evalaution-scenarios-table"]').should("exist")
})
Expand Down
6 changes: 1 addition & 5 deletions agenta-web/cypress/support/commands/evaluations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,7 @@ Cypress.Commands.add("createNewEvaluation", () => {
url: `${Cypress.env().baseApiURL}/evaluations/?app_id=${app_id}`,
method: "GET",
}).then((resp) => {
if (resp.body.length) {
cy.get('[data-cy="new-evaluation-button"]').click()
} else {
cy.get('[data-cy="new-evaluation-button__no_variants"]').click()
}
cy.get('[data-cy="new-evaluation-button"]').click()
})
cy.get(".ant-modal-content").should("exist")

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,10 @@ const AbTestingEvaluation = ({viewType}: {viewType: "evaluation" | "overview"})
<div className="flex items-center justify-between">
<Space>
<Title>A/B Testing Evaluations</Title>
<Button size="small" href={`/apps/${appId}/evaluations/human_a_b_testing`}>
<Button
size="small"
href={`/apps/${appId}/evaluations?selectedEvaluation=ab_testing_evaluation`}
>
View all
</Button>
</Space>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,10 @@ const SingleModelEvaluation = ({viewType}: {viewType: "evaluation" | "overview"}
<Space>
<Title>Single Model Evaluations</Title>

<Button size="small" href={`/apps/${appId}/evaluations/single_model_test`}>
<Button
size="small"
href={`/apps/${appId}/evaluations?selectedEvaluation=single_model_evaluation`}
>
View all
</Button>
</Space>
Expand Down
Loading
Loading