-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mock s3 test #231
Merged
Merged
Mock s3 test #231
Changes from all commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
9fc335e
mock server test
TingDaoK b22af9a
add it to CO
TingDaoK 569ab8e
fix the test
TingDaoK 4dd231c
Merge branch 'main' into mock-s3-test
TingDaoK 2dab4cc
format error
TingDaoK cc0a8cf
remove non-used import
TingDaoK ce35d84
missing -D
TingDaoK 16062ac
update the windows one
TingDaoK 12d2d18
why
TingDaoK 9c305c5
accident
TingDaoK 2a1bf09
extra line copied
TingDaoK d8fcccc
doesn't make sense
TingDaoK 5f78137
what about the latest?
TingDaoK 4c47712
handle different path
TingDaoK 273149d
fix the path and add default
TingDaoK 898fd7e
report the internal error case
TingDaoK 6736f3c
get it ready for review
TingDaoK b1ff3ff
more test
TingDaoK f9f6e21
add readme and stuff
TingDaoK da67c11
I don't know
TingDaoK 42ecec8
try to see if it's because the ubuntu version
TingDaoK 7275878
try 18.04
TingDaoK 7084d69
what about using container
TingDaoK ea32977
18
TingDaoK 9cf7b8b
It probably not matters
TingDaoK 577e358
remove not used line
TingDaoK d0fbc08
Merge branch 'main' into mock-s3-test
TingDaoK d3f076e
seems working
TingDaoK 05d25fa
ops..
TingDaoK ba68502
exit if install failed
TingDaoK 22ce005
exit when failed
TingDaoK c6fa19e
skip mock server test sliently?
TingDaoK 1df2327
remove special CI
TingDaoK 28fe75c
remove copy/pasted comment
TingDaoK 3f5e699
Merge branch 'main' into mock-s3-test
TingDaoK 2f70eb2
use {python}?
TingDaoK c537d24
Merge branch 'mock-s3-test' of github.com:awslabs/aws-c-s3 into mock-…
TingDaoK 1d6f882
KEYS?
TingDaoK b2e1f38
data.keys?
TingDaoK c553c2a
env.config["python"]
TingDaoK 94c3322
finally
TingDaoK a233c25
return op_success instead
TingDaoK c01875f
comment addresed
TingDaoK File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
""" | ||
Setup local mock server for tests | ||
""" | ||
|
||
import Builder | ||
|
||
import os | ||
import sys | ||
import subprocess | ||
import atexit | ||
|
||
|
||
class MockServerSetup(Builder.Action): | ||
""" | ||
Set up this machine for running the mock server test | ||
|
||
This action should be run in the 'pre_build_steps' or 'build_steps' stage. | ||
""" | ||
|
||
def run(self, env): | ||
self.env = env | ||
python_path = sys.executable | ||
# install dependency for mock server | ||
self.env.shell.exec(python_path, | ||
'-m', 'pip', 'install', 'h11', 'trio', check=True) | ||
# check the deps can be import correctly | ||
self.env.shell.exec(python_path, | ||
'-c', 'import h11, trio', check=True) | ||
|
||
# set cmake flag so mock server tests are enabled | ||
env.project.config['cmake_args'].append( | ||
'-DENABLE_MOCK_SERVER_TESTS=ON') | ||
|
||
base_dir = os.path.dirname(os.path.realpath(__file__)) | ||
dir = os.path.join(base_dir, "..", "..", "tests", "mock_s3_server") | ||
os.chdir(dir) | ||
|
||
p = subprocess.Popen([python_path, "mock_s3_server.py"]) | ||
|
||
@atexit.register | ||
def close_mock_server(): | ||
p.terminate() | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,5 +16,6 @@ | |
], | ||
"test_steps": [ | ||
"test" | ||
] | ||
], | ||
"pre_build_steps": ["mock-server-setup"] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
{ | ||
"status": 204, | ||
"headers": {"Connection": "keep-alive"}, | ||
"body": [ | ||
] | ||
} |
14 changes: 14 additions & 0 deletions
14
tests/mock_s3_server/CompleteMultipartUpload/async_access_denied_error.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
{ | ||
"status": 200, | ||
"headers": {"Connection": "close"}, | ||
"body": [ | ||
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>", | ||
"", | ||
"<Error>", | ||
"<Code>AccessDenied</Code>", | ||
"<Message>Access denied.</Message>", | ||
"<RequestId>656c76696e6727732072657175657374</RequestId>", | ||
"<HostId>Uuag1LuByRx9e6j5Onimru9pO4ZVKnJ2Qz7/C1NPcfTWAtRPfTaOFg==</HostId>", | ||
"</Error>" | ||
] | ||
} |
14 changes: 14 additions & 0 deletions
14
tests/mock_s3_server/CompleteMultipartUpload/async_internal_error.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
{ | ||
"status": 200, | ||
"headers": {"Connection": "close"}, | ||
"body": [ | ||
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>", | ||
"", | ||
"<Error>", | ||
"<Code>InternalError</Code>", | ||
"<Message>We encountered an internal error. Please try again.</Message>", | ||
"<RequestId>656c76696e6727732072657175657374</RequestId>", | ||
"<HostId>Uuag1LuByRx9e6j5Onimru9pO4ZVKnJ2Qz7/C1NPcfTWAtRPfTaOFg==</HostId>", | ||
"</Error>" | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
{ | ||
"status": 200, | ||
"headers": {"Connection": "close"}, | ||
"body": [ | ||
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>", | ||
"<CompleteMultipartUploadResult xmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\">", | ||
"<Location>http://default.s3.us-west-2.amazonaws.com/default</Location>", | ||
"<Bucket>default</Bucket>", | ||
"<Key>default</Key>", | ||
"<ETag>\"3858f62230ac3c915f300c664312c11f-9\"</ETag>", | ||
"</CompleteMultipartUploadResult>" | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
{ | ||
"status": 200, | ||
"headers": {}, | ||
"body": [ | ||
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>", | ||
"<InitiateMultipartUploadResult>", | ||
"<Bucket>default</Bucket>", | ||
"<Key>default</Key>", | ||
"<UploadId>defaultID</UploadId>", | ||
"</InitiateMultipartUploadResult>" | ||
] | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
# Mock S3 server | ||
|
||
A **NON-TLS** mock S3 server based on [python-hyper/h11](https://github.com/python-hyper/h11) and [trio](http://trio.readthedocs.io/en/latest/index.html). The server code implementation is based on the trio-server example from python-hyper/h11 [here](https://github.com/python-hyper/h11/blob/master/examples/trio-server.py). Only supports very basic mock response for request received. | ||
|
||
## How to run the server | ||
|
||
Python 3.5+ required. | ||
|
||
- Install hyper/h11 and trio python module. `python3 -m pip install h11 trio` | ||
- Run python. `python3 ./mock_s3_server.py`. | ||
|
||
### Supported Operations | ||
|
||
- CreateMultipartUpload | ||
- CompleteMultipartUpload | ||
- UploadPart | ||
- AbortMultipartUpload | ||
|
||
### Defined response | ||
|
||
The server will read from ./{OperationName}/{Key}.json. The json file is formatted as following: | ||
|
||
```json | ||
{ | ||
"status": 200, | ||
"headers": {"Connection": "close"}, | ||
"body": [ | ||
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>", | ||
"", | ||
"<Error>", | ||
"<Code>InternalError</Code>", | ||
"<Message>We encountered an internal error. Please try again.</Message>", | ||
"<RequestId>656c76696e6727732072657175657374</RequestId>", | ||
"<HostId>Uuag1LuByRx9e6j5Onimru9pO4ZVKnJ2Qz7/C1NPcfTWAtRPfTaOFg==</HostId>", | ||
"</Error>" | ||
] | ||
} | ||
``` | ||
|
||
Where you can define the expected response status, header and response body. If the {Key}.json is not found from file system, it will load the `default.json`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
{ | ||
"status": 200, | ||
"headers": {"ETag": "b54357faf0632cce46e942fa68356b38", "Connection": "keep-alive"}, | ||
"body": [ | ||
] | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
neato