Skip to content

Commit

Permalink
merge list_objects_v2() into list_objects() method
Browse files Browse the repository at this point in the history
  • Loading branch information
@balamurugana
balamurugana committed Aug 24, 2020
1 parent ec60a37 commit c43362b
Show file tree
Hide file tree
Showing 11 changed files with 298 additions and 436 deletions.
3 changes: 0 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,6 @@ The full API Reference is available here.
* [`bucket_exists`](https://docs.min.io/docs/python-client-api-reference#bucket_exists)
* [`remove_bucket`](https://docs.min.io/docs/python-client-api-reference#remove_bucket)
* [`list_objects`](https://docs.min.io/docs/python-client-api-reference#list_objects)
* [`list_objects_v2`](https://docs.min.io/docs/python-client-api-reference#list_objects_v2)
* [`list_incomplete_uploads`](https://docs.min.io/docs/python-client-api-reference#list_incomplete_uploads)

### API Reference : Bucket policy Operations
Expand Down Expand Up @@ -144,7 +143,6 @@ The full API Reference is available here.
* [`put_object`](https://docs.min.io/docs/python-client-api-reference#put_object)
* [`stat_object`](https://docs.min.io/docs/python-client-api-reference#stat_object)
* [`copy_object`](https://docs.min.io/docs/python-client-api-reference#copy_object)
* [`get_partial_object`](https://docs.min.io/docs/python-client-api-reference#get_partial_object)
* [`remove_object`](https://docs.min.io/docs/python-client-api-reference#remove_object)
* [`remove_objects`](https://docs.min.io/docs/python-client-api-reference#remove_objects)
* [`remove_incomplete_upload`](https://docs.min.io/docs/python-client-api-reference#remove_incomplete_upload)
Expand Down Expand Up @@ -195,7 +193,6 @@ The full API Reference is available here.
* [put_object.py](https://github.com/minio/minio-py/blob/master/examples/put_object.py)
* [stat_object.py](https://github.com/minio/minio-py/blob/master/examples/stat_object.py)
* [copy_object.py](https://github.com/minio/minio-py/blob/master/examples/copy_object.py)
* [get_partial_object.py](https://github.com/minio/minio-py/blob/master/examples/get_partial_object.py)
* [remove_object.py](https://github.com/minio/minio-py/blob/master/examples/remove_object.py)
* [remove_objects.py](https://github.com/minio/minio-py/blob/master/examples/remove_objects.py)
* [remove_incomplete_upload.py](https://github.com/minio/minio-py/blob/master/examples/remove_incomplete_upload.py)
Expand Down
68 changes: 5 additions & 63 deletions docs/API.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,10 @@ s3Client = Minio(
| [`bucket_exists`](#bucket_exists) | [`copy_object`](#copy_object) | [`presigned_post_policy`](#presigned_post_policy) | [`delete_bucket_policy`](#delete_bucket_policy) |
| [`remove_bucket`](#remove_bucket) | [`stat_object`](#stat_object) | | [`get_bucket_notification`](#get_bucket_notification) |
| [`list_objects`](#list_objects) | [`remove_object`](#remove_object) | | [`set_bucket_notification`](#set_bucket_notification) |
| [`list_objects_v2`](#list_objects_v2) | [`remove_objects`](#remove_objects) | | [`remove_all_bucket_notification`](#remove_all_bucket_notification) |
| [`list_incomplete_uploads`](#list_incomplete_uploads) | [`remove_incomplete_upload`](#remove_incomplete_upload) | | [`listen_bucket_notification`](#listen_bucket_notification) |
| [`enable_bucket_versioning`](#enable_bucket_versioning) | [`fput_object`](#fput_object) | | [`get_bucket_encryption`](#get_bucket_encryption) |
| [`disable_bucket_versioning`](#disable_bucket_versioning) | [`fget_object`](#fget_object) | | [`put_bucket_encryption`](#put_bucket_encryption) |
| [`list_incomplete_uploads`](#list_incomplete_uploads) | [`remove_objects`](#remove_objects) | | [`remove_all_bucket_notification`](#remove_all_bucket_notification) |
| [`enable_bucket_versioning`](#enable_bucket_versioning) | [`remove_incomplete_upload`](#remove_incomplete_upload) | | [`listen_bucket_notification`](#listen_bucket_notification) |
| [`disable_bucket_versioning`](#disable_bucket_versioning) | [`fput_object`](#fput_object) | | [`get_bucket_encryption`](#get_bucket_encryption) |
| | [`fget_object`](#fget_object) | | [`put_bucket_encryption`](#put_bucket_encryption) |
| | [`select_object_content`](#select_object_content) | | [`delete_bucket_encryption`](#delete_bucket_encryption) |

## 1. Constructor
Expand Down Expand Up @@ -256,64 +256,6 @@ for object in objects:
print(object)
```

<a name="list_objects_v2"></a>

### list_objects_v2(bucket_name, prefix=None, recursive=False, start_after=None, include_user_meta=False, include_version=False)

Lists object information of a bucket using S3 API version 2, optionally for prefix recursively.

__Parameters__

| Param | Type | Description |
|:--------------------|:-------|:---------------------------------------------------------|
| `bucket_name` | _str_ | Name of the bucket. |
| `prefix` | _str_ | Object name starts with prefix. |
| `recursive` | _bool_ | List recursively than directory structure emulation. |
| `start_after` | _str_ | List objects after this key name. |
| `include_user_meta` | _bool_ | MinIO specific flag to control to include user metadata. |
| `include_version` | _bool_ | Flag to control whether include object versions. |

__Return Value__

| Return |
|:----------------------------------------------------------|
| An iterator contains object information as _minio.Object_ |

__Example__

```py
# List objects information.
objects = minio.list_objects_v2('foo')
for object in objects:
print(object)

# List objects information those names starts with 'hello/'.
objects = minio.list_objects_v2('foo', prefix='hello/')
for object in objects:
print(object)

# List objects information recursively.
objects = minio.list_objects_v2('foo', recursive=True)
for object in objects:
print(object)

# List objects information recursively those names starts with
# 'hello/'.
objects = minio.list_objects_v2(
'foo', prefix='hello/', recursive=True,
)
for object in objects:
print(object)

# List objects information recursively after object name
# 'hello/world/1'.
objects = minio.list_objects_v2(
'foo', recursive=True, start_after='hello/world/1',
)
for object in objects:
print(object)
```

<a name="list_incomplete_uploads"></a>

### list_incomplete_uploads(bucket_name, prefix='', recursive=False)
Expand Down Expand Up @@ -752,7 +694,7 @@ finally:

// Get object data for offset/length.
try:
response = minio.get_partial_object('foo', 'bar', 2, 4)
response = minio.get_object('foo', 'bar', 2, 4)
// Read data from response.
finally:
response.close()
Expand Down
34 changes: 0 additions & 34 deletions examples/get_partial_object.py
View file

This file was deleted.

4 changes: 2 additions & 2 deletions examples/list_objects.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@

# List all object paths in bucket that begin with my-prefixname using
# V2 listing API.
objects = client.list_objects_v2('my-bucketname', prefix='my-prefixname',
recursive=True)
objects = client.list_objects('my-bucketname', prefix='my-prefixname',
recursive=True)
for obj in objects:
print(obj.bucket_name, obj.object_name.encode('utf-8'), obj.last_modified,
obj.etag, obj.size, obj.content_type)
2 changes: 1 addition & 1 deletion examples/remove_objects.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
try:
names = map(
lambda x: x.object_name,
client.list_objects_v2('my-bucketname', 'my-prefix', recursive=True)
client.list_objects('my-bucketname', 'my-prefix', recursive=True)
)
for err in client.remove_objects('my-bucketname', names):
print("Deletion Error: {}".format(err))
Expand Down
2 changes: 1 addition & 1 deletion minio/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@

__title__ = 'minio-py'
__author__ = 'MinIO, Inc.'
__version__ = '6.0.1'
__version__ = '7.0.0'
__license__ = 'Apache 2.0'
__copyright__ = 'Copyright 2015, 2016, 2017, 2018, 2019, 2020 MinIO, Inc.'

Expand Down
73 changes: 15 additions & 58 deletions minio/api.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -871,7 +871,7 @@ def get_object(self, bucket_name, object_name, offset=0, length=0,
// Get object data for offset/length.
try:
response = minio.get_partial_object('foo', 'bar', 2, 4)
response = minio.get_object('foo', 'bar', 2, 4)
// Read data from response.
finally:
response.close()
Expand Down Expand Up @@ -1039,53 +1039,8 @@ def put_object(self, bucket_name, object_name, data, length,
progress=progress)

def list_objects(self, bucket_name, prefix=None, recursive=False,
include_version=False):
"""
Lists object information of a bucket using S3 API version 1, optionally
for prefix recursively.
:param bucket_name: Name of the bucket.
:param prefix: Object name starts with prefix.
:param recursive: List recursively than directory structure emulation.
:param include_version: Flag to control whether include object
versions.
:return: An iterator contains object information.
Example::
# List objects information.
objects = minio.list_objects('foo')
for object in objects:
print(object)
# List objects information those names starts with 'hello/'.
objects = minio.list_objects('foo', prefix='hello/')
for object in objects:
print(object)
# List objects information recursively.
objects = minio.list_objects('foo', recursive=True)
for object in objects:
print(object)
# List objects information recursively those names starts with
# 'hello/'.
objects = minio.list_objects(
'foo', prefix='hello/', recursive=True,
)
for object in objects:
print(object)
"""
return self._list_objects(
bucket_name,
delimiter=None if recursive else "/",
prefix=prefix,
use_version1=True,
include_version=include_version,
)

def list_objects_v2(self, bucket_name, prefix=None, recursive=False,
start_after=None, include_user_meta=False,
include_version=False):
start_after=None, include_user_meta=False,
include_version=False, use_api_v1=False):
"""
Lists object information of a bucket using S3 API version 2, optionally
for prefix recursively.
Expand All @@ -1098,35 +1053,36 @@ def list_objects_v2(self, bucket_name, prefix=None, recursive=False,
user metadata.
:param include_version: Flag to control whether include object
versions.
:param use_api_v1: Flag to control to use ListObjectV1 S3 API or not.
:return: An iterator contains object information.
Example::
# List objects information.
objects = minio.list_objects_v2('foo')
objects = minio.list_objects('foo')
for object in objects:
print(object)
# List objects information those names starts with 'hello/'.
objects = minio.list_objects_v2('foo', prefix='hello/')
objects = minio.list_objects('foo', prefix='hello/')
for object in objects:
print(object)
# List objects information recursively.
objects = minio.list_objects_v2('foo', recursive=True)
objects = minio.list_objects('foo', recursive=True)
for object in objects:
print(object)
# List objects information recursively those names starts with
# 'hello/'.
objects = minio.list_objects_v2(
objects = minio.list_objects(
'foo', prefix='hello/', recursive=True,
)
for object in objects:
print(object)
# List objects information recursively after object name
# 'hello/world/1'.
objects = minio.list_objects_v2(
objects = minio.list_objects(
'foo', recursive=True, start_after='hello/world/1',
)
for object in objects:
Expand All @@ -1138,6 +1094,7 @@ def list_objects_v2(self, bucket_name, prefix=None, recursive=False,
include_user_meta=include_user_meta,
prefix=prefix,
start_after=start_after,
use_api_v1=use_api_v1,
include_version=include_version,
)

Expand Down Expand Up @@ -2151,7 +2108,7 @@ def _list_objects( # pylint: disable=too-many-arguments,too-many-branches
prefix=None, # all
start_after=None, # all: v1:marker, versioned:key_marker
version_id_marker=None, # versioned
use_version1=False,
use_api_v1=False,
include_version=False,
):
"""
Expand All @@ -2172,10 +2129,10 @@ def _list_objects( # pylint: disable=too-many-arguments,too-many-branches
query = {}
if include_version:
query["versions"] = ""
elif not use_version1:
elif not use_api_v1:
query["list-type"] = "2"

if not include_version and not use_version1:
if not include_version and not use_api_v1:
if continuation_token:
query["continuation-token"] = continuation_token
if fetch_owner:
Expand All @@ -2190,7 +2147,7 @@ def _list_objects( # pylint: disable=too-many-arguments,too-many-branches
if start_after:
if include_version:
query["key-marker"] = start_after
elif use_version1:
elif use_api_v1:
query["marker"] = start_after
else:
query["start-after"] = start_after
Expand All @@ -2207,7 +2164,7 @@ def _list_objects( # pylint: disable=too-many-arguments,too-many-branches
objects, is_truncated, start_after, version_id_marker = (
parse_list_object_versions(response.data, bucket_name)
)
elif use_version1:
elif use_api_v1:
objects, is_truncated, start_after = parse_list_objects(
response.data,
bucket_name,
Expand Down
Loading

0 comments on commit c43362b

Please sign in to comment.