Koala scenario

[YianZhang]

The test dataset used by the Koala team to evaluate instruction-following models.

https://bair.berkeley.edu/blog/2023/04/03/koala/

Command

To run GPT-4 on the scenario, use: venv/bin/helm-run -r koala:model=openai/gpt-4-0314 -m <max_eval_instances> -t <num_trials> --suite <suite_id>

Results

Some example scenario_states:

Example 1:
Prompt: "In order to launch a token for a Defi project, I want to sell some tokens privately to early investors and then add it to a launchpad. let me know what can I do with my smart contract? Should I create a presale contract separate from main contract?"
GPT-4 Response: "Yes, it's a good idea to create a separate presale contract for your private sale before launching your token on a launchpad. This allows you to manage the private sale and allocate tokens to early investors in a more controlled manner. Here's a suggested approach:\n\n1. Develop your main token contract: Create an ERC20 token contract (or any other relevant standard) with the desired tokenomics for your DeFi project.\n\n2. Create a presale contract: Develop a separate contract for your private sale. This contract should include functions to accept payments from investors, allocate tokens to them, and set a specific time frame for the sale. You might also want to include functions for whitelisting early investors or establishing a vesting schedule for the tokens, if desired.\n\n3. Deploy the contracts: Deploy both the main token contract and the presale contract on the Ethereum network or any other compatible blockchain platform.\n\n4. Allocate tokens for the private sale: Send a portion of your total token supply from the main token contract to the presale contract. This will allow you to sell these tokens to early investors during the private sale.\n\n5. Conduct the private sale: Invite your early investors to participate in the private sale by sending them the presale contract address. Ensure they are aware of the terms and conditions of the sale, including any vesting schedules or whitelisting requirements.\n\n6. Finalize the private sale: Once the private sale is over, execute any necessary functions in the presale contract to finalize the sale, distribute tokens to investors, and, if applicable, enable token transfers.\n\n7. Launch your token on a launchpad: After completing the private sale, you can proceed with listing your token on a launchpad or decentralized exchange (DEX) to make it available for public trading.\n\nRemember to thoroughly audit and test your smart contracts before deploying them on the mainnet to ensure they function as intended and to avoid potential security risks. Additionally, consider seeking legal advice to ensure your token sale complies with relevant regulations."

Example 2:
Prompt: "Message to client for accepting offer for summary video"
GPT-4 Response: "Subject: Offer Accepted: Summary Video Project\n\nDear [Client's Name],\n\nI hope this message finds you well. I am writing to inform you that I have accepted your offer for the summary video project. I am truly excited to start working on this creative venture and deliver a compelling and engaging video that meets your expectations.\n\nAs we discussed previously, the project requirements include a [duration] summary video, summarizing the key points of [subject matter], and incorporating your desired visual and audio elements.\n\nTo ensure a smooth workflow and timely completion of the project, kindly provide me with any specific instructions, preferences, or materials that you would like me to incorporate into the video. If possible, please share this information by [date], so I can commence work without delay.\n\nThe agreed-upon deadline for the project is [deadline date]. Rest assured, I will keep you updated on my progress and share the completed video with you on or before this date.\n\nPlease feel free to reach out to me if you have any questions or concerns during the course of the project. Your satisfaction is my top priority, and I am confident that we will create a captivating summary video that effectively conveys your message.\n\nThank you once again for entrusting me with this project, and I look forward to a successful collaboration.\n\nBest regards,\n\n[Your Name]"