Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[translation] add tests #17041

Closed
kristapratico opened this issue Mar 3, 2021 · 2 comments
Closed

[translation] add tests #17041

kristapratico opened this issue Mar 3, 2021 · 2 comments

Comments

@kristapratico
Copy link
Member

kristapratico commented Mar 3, 2021

Test classes should inherit from this base class: https://github.com/Azure/azure-sdk-for-python/blob/b972486d549a66ae6b33d3f95486687c1b1ff45e/sdk/documenttranslation/azure-ai-documenttranslation/tests/testcase.py

  • Add tests for sync client
  • Add tests for async client
  • Generate test recordings that we can use to run tests in playback mode (not live)

We should ensure that any sensitive information (like SAS tokens) are scrubbed from the test recordings. This is done automatically with the testcase/preparers.

See table where we are tracking testing scenarios across languages.

@kristapratico
Copy link
Member Author

kristapratico commented Mar 18, 2021

@mohamedshabanofficial

Running tests

After we write some tests, just wanted to note how to run them locally here.

Two modes: recorded and live

  • Live runs against real azure resources and produces a recording of the network traffic in the tests/recordings folder
  • Recorded runs against recorded network traffic and don't actually make any calls to the service. To create a recording for a particular test, just run it live first.

To toggle between the two modes, create a file called "testsettings_local.cfg" in \azure-sdk-for-python\tools\azure-sdk-tools\devtools_testutils\testsettings_local.cfg
Add 1 line:

live-mode: true

If you want to run in recorded mode, change to false.

Set environment variables:

set TRANSLATION_DOCUMENT_TEST_ENDPOINT=https://{your-resource-name}.cognitiveservices.azure.com/
set TRANSLATION_DOCUMENT_TEST_API_KEY={your document translation API key}
set TRANSLATION_DOCUMENT_NAME={your-resource-name}
set TRANSLATION_DOCUMENT_STORAGE_NAME={your-storage-blob-resource-name}
set TRANSLATION_DOCUMENT_STORAGE_KEY={your-storage-blob-api-key}
set TRANSLATION_DOCUMENT_CLIENT_SECRET=""
set TRANSLATION_DOCUMENT_CLIENT_ID=""
set TRANSLATION_CLIENT_SECRET=""
set TRANSLATION_CLIENT_ID=""
set TRANSLATION_TENANT_ID=""
set TRANSLATION_DOCUMENT_TENANT_ID=""

The test env is looking for the few variables that are set to empty string so we need them there, but they aren't actually used. Note: you will need a storage account to run these tests.

After these env vars are set, you should be able to simply run tests from commandline like:

pytest test_service.py

or run a particular test

pytest test_service.py -k test_translate

Running live tests in the CI pipelines
Comment /azp run python - translation - tests on your PR

@kristapratico
Copy link
Member Author

closed with #17511

openapi-sdkautomation bot pushed a commit to AzureSDKAutomation/azure-sdk-for-python that referenced this issue Jan 19, 2022
Compute 2021-11-01 Release (Azure#17120)

* compute folder with pre-population

* fix examples folder

* fix an example file

* update compute to match last version

* run prettier fix examples

* change capitalization

* Compute Swagger changes to include cross-region Restore Points scenarios (Azure#16570)

* RRP changes in swagger

* Fix conflicts

* fix validations

* Fix instance view

* Revert "Compute Swagger changes to include cross-region Restore Points scenarios (Azure#16570)" (Azure#16663)

This reverts commit 70a8729e86b30440cdd3c239272e31dff7f9627b.

* sync with last version

* add missing examples

* suppress bodyTopLevelProperties

* Add VMSS filter to List VMs (Azure#16813)

* change1 for change file

* change description

* change description

* add suppression for required properties

* Revert "Add VMSS filter to List VMs (Azure#16813)" (Azure#16956)

This reverts commit 231fd2260cb9e63ba16d8b15d405f3134fe612c1.

* Remove impossible state from example (Azure#16544)

* Downmerging change from Azure:main (Azure#16654)

Co-authored-by: Avinash Akka <avakka@microsoft.com>

* Added CVM settings for version 2021-11-01 (Azure#16622)

* Added CVM settings

* added examples

* Add CMK examples

* fixed typo

* fixed file name

* prettier check

* removed a br

* changed to VMDiskSecurityProfile

* type as object

* Added vm size properties for vm scale set (Azure#16723)

* Added vm size properties

* Added get examples

* Added example for vm size properties

* Added example for vm size properties

Co-authored-by: Theodore Chang <thchan@microsoft.com>

* Spec for ProtectedSettingsFromKeyVault and AllowExtensionOperations (Azure#16590)

* KV changes

* adding allowExtensionOperation

* fixing examples

* Add repairAction to auto repairs feature and update grace period to PT10M (Azure#16535)

* Add timeCreated to properties for VM, VMSS, CR, DH resources (Azure#16539)

* add creationTime to properties for VM, VMSS, CR, DH resources + remove
required location for Resources

* rename creationTime to timeCreated

* examples for timeCreated

* add minimum api-version to descriptions for timeCreated

* add GetVirtualMachineScaleSet example

* reformat example

Co-authored-by: Chase Van Buskirk <cvanbuskirk@microsoft.com>

* DedicatedHost Reboot Feature (Azure#16737)

* added new feature to compute.json

* added example for my feature

* prettier check on the reboot example

* fixed names of parameters in example file:

* changed name to DedicatedHosts_reboot and moved change to where the other DH APIs are

* added cloud error to the reboot

* fixed default error, it was in the wrong spot

* moved reboot to end of specs

* moved older swagger files to 2021-11-01

* reverted previous commit

* renamed reboot to restart as per sameers comment

* updated description as per sameers comment

* updated api version to 2021-11-01 per sameers comment

* Adding the new paramaters zone/placementGroupId to forceRecoveryServiceFabricPlatformUpdateDomainWalk VMSS API (Azure#17041)

* save (Azure#17091)

Co-authored-by: Theodore Chang <thchan@microsoft.com>

* Update compute.json (Azure#16482)

When a customer tries to scale VMSS using Terraform, since the current definitions do not have publicIpPrefix property, the new VM will be assigned a random IP address which is outside the range of public Ip prefix. Customer has to resolve the issue by deleting the Vmss and recreating it. This PR tries to resolve the issue by adding the publicIpPrefix property in the JSON definition which is used to generate a request for VMSS update.

More details about the issue can be found here - Azure/azure-rest-api-specs#10190

* add vmss filter to list  (Azure#16957)

* change1 for change file

* change description

* change description

Co-authored-by: LexieXie <lexiexie@microsoft.com>

* move Kashif's change to 2021-11-01

* Revert "Spec for ProtectedSettingsFromKeyVault and AllowExtensionOperations (Azure#16590)" (Azure#17121)

This reverts commit 220cfd0638942c04275d69fd485ceb2da02a96d3.

* fix CI failures, and run prettier on added examples

* for credscan. change password example

* Update readme.python.md

* Compute Swagger changes to include cross-region Restore Points scenarios (Azure#16682)

* RRP changes in swagger

* Fix conflicts

* fix validations

* Fix instance view

* change instance view

* Fix example

* Fix prettier

* Fix and modify description

* Review comments

* make new api call long-running-operation

* compute folder with pre-population

* fix examples folder

* fix an example file

* update compute to match last version

* run prettier fix examples

* change capitalization

* Compute Swagger changes to include cross-region Restore Points scenarios (Azure#16570)

* RRP changes in swagger

* Fix conflicts

* fix validations

* Fix instance view

* Revert "Compute Swagger changes to include cross-region Restore Points scenarios (Azure#16570)" (Azure#16663)

This reverts commit 70a8729e86b30440cdd3c239272e31dff7f9627b.

* sync with last version

* add missing examples

* Add VMSS filter to List VMs (Azure#16813)

* change1 for change file

* change description

* change description

* Revert "Add VMSS filter to List VMs (Azure#16813)" (Azure#16956)

This reverts commit 231fd2260cb9e63ba16d8b15d405f3134fe612c1.

* Remove impossible state from example (Azure#16544)

* Added CVM settings for version 2021-11-01 (Azure#16622)

* Added CVM settings

* added examples

* Add CMK examples

* fixed typo

* fixed file name

* prettier check

* removed a br

* changed to VMDiskSecurityProfile

* type as object

* Added vm size properties for vm scale set (Azure#16723)

* Added vm size properties

* Added get examples

* Added example for vm size properties

* Added example for vm size properties

Co-authored-by: Theodore Chang <thchan@microsoft.com>

* Spec for ProtectedSettingsFromKeyVault and AllowExtensionOperations (Azure#16590)

* KV changes

* adding allowExtensionOperation

* fixing examples

* Add repairAction to auto repairs feature and update grace period to PT10M (Azure#16535)

* Add timeCreated to properties for VM, VMSS, CR, DH resources (Azure#16539)

* add creationTime to properties for VM, VMSS, CR, DH resources + remove
required location for Resources

* rename creationTime to timeCreated

* examples for timeCreated

* add minimum api-version to descriptions for timeCreated

* add GetVirtualMachineScaleSet example

* reformat example

Co-authored-by: Chase Van Buskirk <cvanbuskirk@microsoft.com>

* DedicatedHost Reboot Feature (Azure#16737)

* added new feature to compute.json

* added example for my feature

* prettier check on the reboot example

* fixed names of parameters in example file:

* changed name to DedicatedHosts_reboot and moved change to where the other DH APIs are

* added cloud error to the reboot

* fixed default error, it was in the wrong spot

* moved reboot to end of specs

* moved older swagger files to 2021-11-01

* reverted previous commit

* renamed reboot to restart as per sameers comment

* updated description as per sameers comment

* updated api version to 2021-11-01 per sameers comment

* Adding the new paramaters zone/placementGroupId to forceRecoveryServiceFabricPlatformUpdateDomainWalk VMSS API (Azure#17041)

* save (Azure#17091)

Co-authored-by: Theodore Chang <thchan@microsoft.com>

* Update compute.json (Azure#16482)

When a customer tries to scale VMSS using Terraform, since the current definitions do not have publicIpPrefix property, the new VM will be assigned a random IP address which is outside the range of public Ip prefix. Customer has to resolve the issue by deleting the Vmss and recreating it. This PR tries to resolve the issue by adding the publicIpPrefix property in the JSON definition which is used to generate a request for VMSS update.

More details about the issue can be found here - Azure/azure-rest-api-specs#10190

* add vmss filter to list  (Azure#16957)

* change1 for change file

* change description

* change description

Co-authored-by: LexieXie <lexiexie@microsoft.com>

* move Kashif's change to 2021-11-01

* fix CI failures, and run prettier on added examples

* Revert "Spec for ProtectedSettingsFromKeyVault and AllowExtensionOperations (Azure#16590)" (Azure#17121)

This reverts commit 220cfd0638942c04275d69fd485ceb2da02a96d3.

* for credscan. change password example

* make new api call long-running-operation

* Update readme.python.md

* Compute Swagger changes to include cross-region Restore Points scenarios (Azure#16682)

* RRP changes in swagger

* Fix conflicts

* fix validations

* Fix instance view

* change instance view

* Fix example

* Fix prettier

* Fix and modify description

* Review comments

* update Repair action to enum and update readme

* rebase to main since 2021-08-01 merged. update readme

* update x-ms-enum name for RepairType

* add default response to operations

* update

* put back 'required' tag for Resource.Location property
and use a new object for VM_LIST return object

* fix json format

* update examples

* run prettier on updated examples

* update example for credScan

* add VirtualMachineResource for toplevel property suppression.

* lint diff errors

* lint diff fix update

* remove change for VMextensions.location bug

* return readme file and examples before vm.vmextension.location change

Co-authored-by: sukodava <78733210+sukodava@users.noreply.github.com>
Co-authored-by: Dapeng Zhang <dapzhang@microsoft.com>
Co-authored-by: xielexie <94083876+xielexie@users.noreply.github.com>
Co-authored-by: Mike Richmond <mirichmo@microsoft.com>
Co-authored-by: Avinash <akka@usc.edu>
Co-authored-by: Avinash Akka <avakka@microsoft.com>
Co-authored-by: ms-saypaul <67093296+ms-saypaul@users.noreply.github.com>
Co-authored-by: Raktima Das <raktdas@microsoft.com>
Co-authored-by: kamusta-msft <56413142+kamusta-msft@users.noreply.github.com>
Co-authored-by: frank-pang-msft <92764154+frank-pang-msft@users.noreply.github.com>
Co-authored-by: Chase VanBuskirk <chasevanb@gmail.com>
Co-authored-by: Chase Van Buskirk <cvanbuskirk@microsoft.com>
Co-authored-by: vbhasker-msft <93355049+vbhasker-msft@users.noreply.github.com>
Co-authored-by: avjai <34567499+avjai@users.noreply.github.com>
Co-authored-by: kangsun-ctrl <69279251+kangsun-ctrl@users.noreply.github.com>
Co-authored-by: karthikka4820 <88366202+karthikka4820@users.noreply.github.com>
Co-authored-by: LexieXie <lexiexie@microsoft.com>
Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com>
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

2 participants