-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Add POC of a testing framework #22
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good start, I will just do Apify Actor of it.
Plus will be goo to run it in parallel on platform,
@@ -0,0 +1,14 @@ | |||
export const TEST_CASES: { | |||
name:string, | |||
expectedOutput: Record<string, unknown>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add model as well?
@@ -0,0 +1,25 @@ | |||
import 'zx/globals'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is good start, I would make this Actor, which can be push to Apify.
The tricky part will be to distinguish between local run and run in platform.
Localy it will be basically same as you have now and you need to add some ifs, in case you run it on platform
- Use Actor.call instead npm run start:dev
- Input from Actor.getInput not local file.
- Save results to dataset maybe.
Let's draft this one until we can back to agent, @jirimoravcik |
You can run it via
OPENAI_API_KEY=***** npm run e2e
.Let's discuss how to proceed with this.