Strategy Test Interface to omit Real node interaction and API calls.

[thehapax]

from #464 "Lots of functions requires API calls and bitshares node. Real node interaction can be omitted by using a fixtures and mocks (simulate API calls and responses)."

this task to create dummy interface for testing strategies and omit real calls

[thehapax]

Decide on what frameworks for unit testing. Here are 3 frameworks for python testing. we should pick one as a guide for uniformity across all future tests.

1. PyUnit, also known as unittest in the Python modules.
2. Pytest – a mature full-featured Python testing tool that helps you write better programs.
3. Nose – a nicer testing for Python? It’s more like a unit test loader since it supports all the above for loading.

[thehapax]

Other strategy unit tests should base their unit test code off of this interface

[bitphage]

I did some research on topic, basically we have several options:

• Create a custom bitshares classes which aren't interacting with real node, but loads data fixtures from local yaml files. This approach is used by python-graphenelib https://github.com/xeroc/python-graphenelib/blob/master/tests/fixtures.py , but has huge limitations: it can serve some fake data, but if you want to process transactions and maintain some state to be able to do more complex tests, you need to reimplement bitshares-core logic inside
• Develop a local mock server which implements some of the basic API calls of the bitshares node. Maybe a bit cleaner solution than custom BitShares classes. Can be started from implementing only basic API calls and extended as needed. Basic example described here https://realpython.com/testing-third-party-apis-with-mock-servers/
• Implement auto-deploy of a bitshares node locally inside a VM or docker container to spawn a standalone testnet + maybe bitshares-ui instance. This approach seems most clean as API will be 100% identical to real API. Before running a test module, a specific fixtures may be applied to setup testing environment like this:
  • create test assets with needed settings like market fees, precisions, etc
  • create test accounts with pre-defined keys
  • cancel all orders on test accounts
  • ensure accounts have exact balances like 1000 TEST1 and 1000 TEST2

The 3rd approach seems most natural. Exact tools to implement this needs to be picked.

• Pytest fixtures can spawn docker container with bitshares-core using docker-py. After spawn, next fixture can prepare accounts and assets.
• Alternatively, bitshares-core instance may be spawned and provisioned using vagrant and ansible.

[thehapax]

I support your 3rd approach. This would be useful in the event that we also write tests querying the nodes using websocket only code in addition to bitshares-instance

[bitphage]

Remark from the dev chat.

Running all tests on clean fresh environment gives us 100% reproducible results. Relying on public testnet is bad, because:

• test may fail because of node connectivity issues
• test results may depend on testnet state
• we have to worry about distributing credentials for test accounts
• we cannot run several test suites in parallel using same accounts
• tests run time will be much slower because of 3 sec blocks and network overhead