Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: remove flaky test #30395

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
+60 −51
Draft

chore: remove flaky test #30395

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
28e6f8f
remove flaky test
eschutho Sep 25, 2024
0eda181
skip test
eschutho Sep 25, 2024
d6c8cf6
use cypress docker container
eschutho Sep 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/workflows/bashlib.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,17 @@ cypress-run-all() {
USE_DASHBOARD_FLAG='--use-dashboard'
fi

python ../../scripts/cypress_run.py --parallelism $PARALLELISM --parallelism-id $PARALLEL_ID $USE_DASHBOARD_FLAG
docker pull cypress/browsers:node-20.16.0-chrome-127.0.6533.119-1

docker run --rm \
-e CYPRESS_BASE_URL \
-e PARALLELISM \
-e PARALLELISM_ID \
-e USE_DASHBOARD=${USE_DASHBOARD} \
-v "$GITHUB_WORKSPACE/superset-frontend/cypress-base:/e2e" \
-w /e2e \
cypress/browsers:node-20.16.0-chrome-127.0.6533.119-1 \
npx cypress run --browser chrome $USE_DASHBOARD_FLAG
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that mean means no parallelism, guessing you're not meaning to merge this change


# After job is done, print out Flask log for debugging
echo "::group::Flask log for default run"
Expand Down
18 changes: 9 additions & 9 deletions superset-frontend/cypress-base/cypress/e2e/dashboard/horizontalFilterBar.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ describe('Horizontal FilterBar', () => {
setFilterBarOrientation('vertical');
});

it('should show all default actions in horizontal mode', () => {
it.skip('should show all default actions in horizontal mode', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of doing all the skips here and below, you can just do it.only and it'll only run that test above. You can probably put a note after that test to list out why you're skipping all these other tests

visitDashboard();
openVerticalFilterBar();
setFilterBarOrientation('horizontal');
Expand All @@ -92,15 +92,15 @@ describe('Horizontal FilterBar', () => {
cy.getBySel('filterbar-action-buttons').should('exist');
});

it('should stay in horizontal mode when reloading', () => {
it.skip('should stay in horizontal mode when reloading', () => {
visitDashboard();
openVerticalFilterBar();
setFilterBarOrientation('horizontal');
cy.reload();
cy.getBySel('dashboard-filters-panel').should('not.exist');
});

it('should show all filters in available space on load', () => {
it.skip('should show all filters in available space on load', () => {
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
{ name: 'test_2', column: 'country_code', datasetId: 2 },
Expand All @@ -110,7 +110,7 @@ describe('Horizontal FilterBar', () => {
cy.get('.filter-item-wrapper').should('have.length', 3);
});

it('should show "more filters" on window resizing up and down', () => {
it.skip('should show "more filters" on window resizing up and down', () => {
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
{ name: 'test_2', column: 'country_code', datasetId: 2 },
Expand All @@ -135,7 +135,7 @@ describe('Horizontal FilterBar', () => {
cy.getBySel('dropdown-container-btn').should('not.exist');
});

it('should show "more filters" and scroll', () => {
it.skip('should show "more filters" and scroll', () => {
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
{ name: 'test_2', column: 'country_code', datasetId: 2 },
Expand All @@ -162,7 +162,7 @@ describe('Horizontal FilterBar', () => {
cy.getBySel('filter-control-name').contains('test_12').should('be.visible');
});

it('should display newly added filter', () => {
it.skip('should display newly added filter', () => {
visitDashboard();
openVerticalFilterBar();
setFilterBarOrientation('horizontal');
Expand All @@ -173,7 +173,7 @@ describe('Horizontal FilterBar', () => {
validateFilterNameOnDashboard(testItems.topTenChart.filterColumn);
});

it('should spot changes in "more filters" and apply their values', () => {
it.skip('should spot changes in "more filters" and apply their values', () => {
cy.intercept(`/api/v1/chart/data?form_data=**`).as('chart');
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
Expand Down Expand Up @@ -201,7 +201,7 @@ describe('Horizontal FilterBar', () => {
);
});

it('should focus filter and open "more filters" programmatically', () => {
it.skip('should focus filter and open "more filters" programmatically', () => {
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
{ name: 'test_2', column: 'country_code', datasetId: 2 },
Expand All @@ -228,7 +228,7 @@ describe('Horizontal FilterBar', () => {
cy.get('.ant-select-focused').should('be.visible');
});

it('should show tag count and one plain tag on focus and only count on blur in select ', () => {
it.skip('should show tag count and one plain tag on focus and only count on blur in select ', () => {
prepareDashboardFilters([
{ name: 'test_1', column: 'country_name', datasetId: 2 },
]);
Expand Down
20 changes: 10 additions & 10 deletions superset-frontend/cypress-base/cypress/e2e/dashboard/nativeFilters.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -312,15 +312,15 @@ describe('Native filters', () => {
});

describe('Nativefilters initial state not required', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of skipping each test, i believe if you did a describe.skip() you'll skip all tests in the describe block

it("User can check 'Filter has default value'", () => {
it.skip("User can check 'Filter has default value'", () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
enterNativeFilterEditModal();
inputNativeFilterDefaultValue(testItems.filterDefaultValue);
});

it('User can add a new native filter', () => {
it.skip('User can add a new native filter', () => {
prepareDashboardFilters([]);

let filterKey: string;
Expand All @@ -345,7 +345,7 @@ describe('Native filters', () => {
cy.get(nativeFilters.modal.container).should('not.exist');
});

it('User can restore a deleted native filter', () => {
it.skip('User can restore a deleted native filter', () => {
prepareDashboardFilters([
{ name: 'country_code', column: 'country_code', datasetId: 2 },
]);
Expand All @@ -363,7 +363,7 @@ describe('Native filters', () => {
);
});

it('User can create a time grain filter', () => {
it.skip('User can create a time grain filter', () => {
prepareDashboardFilters([]);
enterNativeFilterEditModal();
fillNativeFilterForm(
Expand Down Expand Up @@ -424,7 +424,7 @@ describe('Native filters', () => {
validateFilterContentOnDashboard(testItems.topTenChart.filterColumnYear);
});

it('User can create a numerical range filter', () => {
it.skip('User can create a numerical range filter', () => {
visitDashboard();
enterNativeFilterEditModal(false);
fillNativeFilterForm(
Expand Down Expand Up @@ -454,7 +454,7 @@ describe('Native filters', () => {
});
});

it('User can undo deleting a native filter', () => {
it.skip('User can undo deleting a native filter', () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
Expand All @@ -465,7 +465,7 @@ describe('Native filters', () => {
.should('have.attr', 'value', testItems.topTenChart.filterColumn);
});

it('User can cancel changes in native filter', () => {
it.skip('User can cancel changes in native filter', () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
Expand All @@ -479,7 +479,7 @@ describe('Native filters', () => {
cy.contains('You have removed this filter.').should('be.visible');
});

it('User can create a value filter', () => {
it.skip('User can create a value filter', () => {
visitDashboard();
enterNativeFilterEditModal(false);
addCountryNameFilter();
Expand All @@ -490,7 +490,7 @@ describe('Native filters', () => {
validateFilterNameOnDashboard(testItems.topTenChart.filterColumn);
});

it('User can apply value filter with selected values', () => {
it.skip('User can apply value filter with selected values', () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
Expand All @@ -502,7 +502,7 @@ describe('Native filters', () => {
});
});

it('User can stop filtering when filter is removed', () => {
it.skip('User can stop filtering when filter is removed', () => {
prepareDashboardFilters([
{ name: 'country_name', column: 'country_name', datasetId: 2 },
]);
Expand Down
61 changes: 30 additions & 31 deletions superset-frontend/cypress-base/cypress/e2e/explore/chart.test.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
*/
import { CHART_LIST } from 'cypress/utils/urls';
import { interceptGet as interceptDashboardGet } from 'cypress/e2e/dashboard/utils';
import { FORM_DATA_DEFAULTS, NUM_METRIC } from './visualizations/shared.helper';
import {
interceptFiltering,
saveChartToDashboard,
Expand Down Expand Up @@ -98,7 +97,7 @@ describe('Cross-referenced dashboards', () => {
cy.wait('@filtering');
});

it('should show the cross-referenced dashboards', () => {
it.skip('should show the cross-referenced dashboards', () => {
visitSampleChartFromList('1 - Sample chart');

cy.getBySel('metadata-bar').contains('Not added to any dashboard');
Expand All @@ -122,32 +121,32 @@ describe('Cross-referenced dashboards', () => {
});
});

describe('No Results', () => {
beforeEach(() => {
cy.intercept('POST', '/superset/explore_json/**').as('getJson');
});

it('No results message shows up', () => {
const formData = {
...FORM_DATA_DEFAULTS,
metrics: [NUM_METRIC],
viz_type: 'line',
adhoc_filters: [
{
expressionType: 'SIMPLE',
subject: 'state',
operator: 'IN',
comparator: ['Fake State'],
clause: 'WHERE',
sqlExpression: null,
},
],
};

cy.visitChartByParams(formData);
cy.wait('@getJson').its('response.statusCode').should('eq', 200);
cy.get('div.chart-container').contains(
'No results were returned for this query',
);
});
});
// describe('No Results', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can do a describe.skip here to skip all these tests

// beforeEach(() => {
// cy.intercept('POST', '/superset/explore_json/**').as('getJson');
// });

// it('No results message shows up', () => {
// const formData = {
// ...FORM_DATA_DEFAULTS,
// metrics: [NUM_METRIC],
// viz_type: 'line',
// adhoc_filters: [
// {
// expressionType: 'SIMPLE',
// subject: 'state',
// operator: 'IN',
// comparator: ['Fake State'],
// clause: 'WHERE',
// sqlExpression: null,
// },
// ],
// };

// cy.visitChartByParams(formData);
// cy.wait('@getJson').its('response.statusCode').should('eq', 200);
// cy.get('div.chart-container').contains(
// 'No results were returned for this query',
// );
// });
// });
Loading