Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support using transcripts to generate tests #2181

Open
chrimc62 opened this issue Aug 24, 2020 · 2 comments
Open

Support using transcripts to generate tests #2181

chrimc62 opened this issue Aug 24, 2020 · 2 comments
Assignees
Labels
Enhancement New feature or request to evolve an additional feature

Comments

@chrimc62
Copy link
Contributor

Problem

A convenient way to generate a test is to generate a transcript using the emulator and then turn that transcript into a test script. The issue is that randomness in bots makes it impossible to replay a transcript. In order to fix this, the SDK has:

  1. Defined middle ware in Adaptive.Testing to pick up a SetTestOptions custom event and copy the payload into conversation.testOptions. We made the payload open ended.
  2. Defined randomSeed to seed a random number generator or randomValue to return a constant value from all calls to random and adjusted all users of random to make use of that.
  3. Defined a bf plugin that converts a transcript into a test script that supports the history and mocking.
    What is missing is a way to emit SetTestOptions in the emulator. We need to emit SetTestOptions before the conversationUpdate event with the same conversation id in order to set the test options before the transcript starts. While this is a specific ask, it is also related to Be able to send arbitrary event/invoke with a JSON Payload #2174 which is the general requirement to emit any custom event.

Solution

For this specific event it could be a an additional drop down that allows restarting a conversation with test options. Ideally randomSeed/randomValue would be explicit together with open ended values. This could also be a framework for the more general requirements of #2174 where you can send a custom event with/without conversationUpdate.
[enhancement]

@chrimc62 chrimc62 added the Enhancement New feature or request to evolve an additional feature label Aug 24, 2020
@tonyanziano tonyanziano self-assigned this Sep 22, 2020
@tonyanziano tonyanziano added the R11 Release 11 - November 15th, 2020 label Sep 22, 2020
@cwhitten cwhitten added this to the R11 milestone Sep 28, 2020
@cwhitten cwhitten removed the R11 Release 11 - November 15th, 2020 label Sep 28, 2020
@cwhitten cwhitten modified the milestones: R11, R12 Oct 26, 2020
@tonyanziano
Copy link
Contributor

tonyanziano commented Dec 1, 2020

P0:

  • Add UI to "Open bot dialog" to allow user to additionally specify values for randomSeed & randomValue properties
  • Before conversation updates are sent to a conversation, send a custom event activity with the name SetTestOptions with an eventValue property containing the randomSeed & randomValue properties if the above properties were set

P1:

@tonyanziano
Copy link
Contributor

P0 requirements have been met in #2209. Expect to see these changes in tonight's nightly build 👍

@cwhitten cwhitten removed this from the R12 milestone Feb 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request to evolve an additional feature
Projects
None yet
Development

No branches or pull requests

3 participants