Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strategy Test Interface to omit Real node interaction and API calls. #515

Closed
thehapax opened this issue Mar 18, 2019 · 5 comments
Closed
Assignees

Comments

@thehapax
Copy link
Collaborator

thehapax commented Mar 18, 2019

from #464 "Lots of functions requires API calls and bitshares node. Real node interaction can be omitted by using a fixtures and mocks (simulate API calls and responses)."

this task to create dummy interface for testing strategies and omit real calls

@thehapax
Copy link
Collaborator Author

Decide on what frameworks for unit testing. Here are 3 frameworks for python testing. we should pick one as a guide for uniformity across all future tests.

  1. PyUnit, also known as unittest in the Python modules.
  2. Pytest – a mature full-featured Python testing tool that helps you write better programs.
  3. Nose – a nicer testing for Python? It’s more like a unit test loader since it supports all the above for loading.

@thehapax thehapax changed the title Test Interface to omit Real node interaction and API calls. Strategy Test Interface to omit Real node interaction and API calls. Mar 18, 2019
@thehapax
Copy link
Collaborator Author

Other strategy unit tests should base their unit test code off of this interface

@bitphage
Copy link
Collaborator

I did some research on topic, basically we have several options:

  • Create a custom bitshares classes which aren't interacting with real node, but loads data fixtures from local yaml files. This approach is used by python-graphenelib https://github.com/xeroc/python-graphenelib/blob/master/tests/fixtures.py , but has huge limitations: it can serve some fake data, but if you want to process transactions and maintain some state to be able to do more complex tests, you need to reimplement bitshares-core logic inside
  • Develop a local mock server which implements some of the basic API calls of the bitshares node. Maybe a bit cleaner solution than custom BitShares classes. Can be started from implementing only basic API calls and extended as needed. Basic example described here https://realpython.com/testing-third-party-apis-with-mock-servers/
  • Implement auto-deploy of a bitshares node locally inside a VM or docker container to spawn a standalone testnet + maybe bitshares-ui instance. This approach seems most clean as API will be 100% identical to real API. Before running a test module, a specific fixtures may be applied to setup testing environment like this:
    • create test assets with needed settings like market fees, precisions, etc
    • create test accounts with pre-defined keys
    • cancel all orders on test accounts
    • ensure accounts have exact balances like 1000 TEST1 and 1000 TEST2

The 3rd approach seems most natural. Exact tools to implement this needs to be picked.

  • Pytest fixtures can spawn docker container with bitshares-core using docker-py. After spawn, next fixture can prepare accounts and assets.
  • Alternatively, bitshares-core instance may be spawned and provisioned using vagrant and ansible.

@thehapax
Copy link
Collaborator Author

I support your 3rd approach. This would be useful in the event that we also write tests querying the nodes using websocket only code in addition to bitshares-instance

@bitphage
Copy link
Collaborator

bitphage commented Apr 26, 2019

Remark from the dev chat.

Running all tests on clean fresh environment gives us 100% reproducible results. Relying on public testnet is bad, because:

  • test may fail because of node connectivity issues
  • test results may depend on testnet state
  • we have to worry about distributing credentials for test accounts
  • we cannot run several test suites in parallel using same accounts
  • tests run time will be much slower because of 3 sec blocks and network overhead

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants