Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test/Integration/Tracking] implement generic test framework for integration test #2647

Closed
1 task done
Tracked by #3428
ZhiHanZ opened this issue Nov 4, 2021 · 8 comments
Closed
1 task done
Tracked by #3428
Assignees
Labels

Comments

@ZhiHanZ
Copy link
Collaborator

ZhiHanZ commented Nov 4, 2021

Summary
Similar to integration test in kubernetes and kubebuilder https://book.kubebuilder.io/cronjob-tutorial/writing-tests.html
For local test:
1, we use cargobin to ensure binaries on desired local and bootstrap cluster through code similar to bendctl basic test.
2. For stateless and stateful test we could choose desired number of query instance to run and deploy test them uniformly on test_env.
The test framwork would be

├── integration
│   ├── Cargo.toml
│   └── src
│       ├── framework
│       │   └── local
│       ├── lib.rs
│       └── sql

For cluster test:
we need to ensure some third party kubernetes dependencies and do the same thing

@PsiACE
Copy link
Member

PsiACE commented Nov 4, 2021

├── integration
│   ├── Cargo.toml
│   └── src
│       ├── framework
│       │   └── local
│       ├── lib.rs
│       └── sql
├── integration
│   ├── Cargo.toml
|   ├── sql # If it is large, we can keep only a subset
│   └── src
│       ├── framework
│       │   └── local
│       └── lib.rs

@BohuTANG BohuTANG added the C-testing Category: testing label Nov 6, 2021
@PsiACE PsiACE added the v0.7 label Dec 13, 2021
@ZhiHanZ ZhiHanZ changed the title [Test/Integration] implement generic test framework for integration test [Test/Integration/Tracking] implement generic test framework for integration test Jan 12, 2022
@Xuanwo
Copy link
Member

Xuanwo commented Jan 18, 2022

/assignme

@Xuanwo
Copy link
Member

Xuanwo commented Jan 18, 2022

Come on, Let's rock! I expect that we can remove all python scripts after this issue implemented.

@Xuanwo
Copy link
Member

Xuanwo commented Mar 27, 2022

I'm not interested in this issue any more. Feel free to take it 🚀

@Xuanwo Xuanwo removed their assignment Mar 27, 2022
@PsiACE
Copy link
Member

PsiACE commented Apr 20, 2022

/assignme

@leiysky
Copy link
Contributor

leiysky commented Apr 26, 2022

Is it possible to support error check in this framework?

The current integration tests relies on hint to specify the error code, which is not a good method.

@ZhiHanZ
Copy link
Collaborator Author

ZhiHanZ commented Apr 26, 2022

Is it possible to support error check in this framework?

The current integration tests relies on hint to specify the error code, which is not a good method.

PTAL at the new framework, #5048
It should support error regex matching . Retry and result sorting in the new framework

@PsiACE
Copy link
Member

PsiACE commented Jun 14, 2022

Now we have a new sql logic test framework

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants