Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge list_objects_v2() into list_objects() method #953

Merged
merged 2 commits into from
Sep 1, 2020
Merged

merge list_objects_v2() into list_objects() method #953

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3a9b22d
merge list_objects_v2() into list_objects() method
balamurugana Aug 6, 2020
28931bd
fix examples/list_objects.py
balamurugana Aug 31, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,6 @@ The full API Reference is available here.
* [`bucket_exists`](https://docs.min.io/docs/python-client-api-reference#bucket_exists)
* [`remove_bucket`](https://docs.min.io/docs/python-client-api-reference#remove_bucket)
* [`list_objects`](https://docs.min.io/docs/python-client-api-reference#list_objects)
* [`list_objects_v2`](https://docs.min.io/docs/python-client-api-reference#list_objects_v2)
* [`list_incomplete_uploads`](https://docs.min.io/docs/python-client-api-reference#list_incomplete_uploads)

### API Reference : Bucket policy Operations
Expand Down Expand Up @@ -144,7 +143,6 @@ The full API Reference is available here.
* [`put_object`](https://docs.min.io/docs/python-client-api-reference#put_object)
* [`stat_object`](https://docs.min.io/docs/python-client-api-reference#stat_object)
* [`copy_object`](https://docs.min.io/docs/python-client-api-reference#copy_object)
* [`get_partial_object`](https://docs.min.io/docs/python-client-api-reference#get_partial_object)
* [`remove_object`](https://docs.min.io/docs/python-client-api-reference#remove_object)
* [`remove_objects`](https://docs.min.io/docs/python-client-api-reference#remove_objects)
* [`remove_incomplete_upload`](https://docs.min.io/docs/python-client-api-reference#remove_incomplete_upload)
Expand Down Expand Up @@ -195,7 +193,6 @@ The full API Reference is available here.
* [put_object.py](https://github.com/minio/minio-py/blob/master/examples/put_object.py)
* [stat_object.py](https://github.com/minio/minio-py/blob/master/examples/stat_object.py)
* [copy_object.py](https://github.com/minio/minio-py/blob/master/examples/copy_object.py)
* [get_partial_object.py](https://github.com/minio/minio-py/blob/master/examples/get_partial_object.py)
* [remove_object.py](https://github.com/minio/minio-py/blob/master/examples/remove_object.py)
* [remove_objects.py](https://github.com/minio/minio-py/blob/master/examples/remove_objects.py)
* [remove_incomplete_upload.py](https://github.com/minio/minio-py/blob/master/examples/remove_incomplete_upload.py)
Expand Down
68 changes: 5 additions & 63 deletions docs/API.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,10 @@ s3Client = Minio(
| [`bucket_exists`](#bucket_exists) | [`copy_object`](#copy_object) | [`presigned_post_policy`](#presigned_post_policy) | [`delete_bucket_policy`](#delete_bucket_policy) |
| [`remove_bucket`](#remove_bucket) | [`stat_object`](#stat_object) | | [`get_bucket_notification`](#get_bucket_notification) |
| [`list_objects`](#list_objects) | [`remove_object`](#remove_object) | | [`set_bucket_notification`](#set_bucket_notification) |
| [`list_objects_v2`](#list_objects_v2) | [`remove_objects`](#remove_objects) | | [`remove_all_bucket_notification`](#remove_all_bucket_notification) |
| [`list_incomplete_uploads`](#list_incomplete_uploads) | [`remove_incomplete_upload`](#remove_incomplete_upload) | | [`listen_bucket_notification`](#listen_bucket_notification) |
| [`enable_bucket_versioning`](#enable_bucket_versioning) | [`fput_object`](#fput_object) | | [`get_bucket_encryption`](#get_bucket_encryption) |
| [`disable_bucket_versioning`](#disable_bucket_versioning) | [`fget_object`](#fget_object) | | [`put_bucket_encryption`](#put_bucket_encryption) |
| [`list_incomplete_uploads`](#list_incomplete_uploads) | [`remove_objects`](#remove_objects) | | [`remove_all_bucket_notification`](#remove_all_bucket_notification) |
| [`enable_bucket_versioning`](#enable_bucket_versioning) | [`remove_incomplete_upload`](#remove_incomplete_upload) | | [`listen_bucket_notification`](#listen_bucket_notification) |
| [`disable_bucket_versioning`](#disable_bucket_versioning) | [`fput_object`](#fput_object) | | [`get_bucket_encryption`](#get_bucket_encryption) |
| | [`fget_object`](#fget_object) | | [`put_bucket_encryption`](#put_bucket_encryption) |
| | [`select_object_content`](#select_object_content) | | [`delete_bucket_encryption`](#delete_bucket_encryption) |

## 1. Constructor
Expand Down Expand Up @@ -256,64 +256,6 @@ for object in objects:
print(object)
```

<a name="list_objects_v2"></a>

### list_objects_v2(bucket_name, prefix=None, recursive=False, start_after=None, include_user_meta=False, include_version=False)

Lists object information of a bucket using S3 API version 2, optionally for prefix recursively.

__Parameters__

| Param | Type | Description |
|:--------------------|:-------|:---------------------------------------------------------|
| `bucket_name` | _str_ | Name of the bucket. |
| `prefix` | _str_ | Object name starts with prefix. |
| `recursive` | _bool_ | List recursively than directory structure emulation. |
| `start_after` | _str_ | List objects after this key name. |
| `include_user_meta` | _bool_ | MinIO specific flag to control to include user metadata. |
| `include_version` | _bool_ | Flag to control whether include object versions. |

__Return Value__

| Return |
|:----------------------------------------------------------|
| An iterator contains object information as _minio.Object_ |

__Example__

```py
# List objects information.
objects = minio.list_objects_v2('foo')
for object in objects:
print(object)

# List objects information those names starts with 'hello/'.
objects = minio.list_objects_v2('foo', prefix='hello/')
for object in objects:
print(object)

# List objects information recursively.
objects = minio.list_objects_v2('foo', recursive=True)
for object in objects:
print(object)

# List objects information recursively those names starts with
# 'hello/'.
objects = minio.list_objects_v2(
'foo', prefix='hello/', recursive=True,
)
for object in objects:
print(object)

# List objects information recursively after object name
# 'hello/world/1'.
objects = minio.list_objects_v2(
'foo', recursive=True, start_after='hello/world/1',
)
for object in objects:
print(object)
```

<a name="list_incomplete_uploads"></a>

### list_incomplete_uploads(bucket_name, prefix='', recursive=False)
Expand Down Expand Up @@ -752,7 +694,7 @@ finally:

// Get object data for offset/length.
try:
response = minio.get_partial_object('foo', 'bar', 2, 4)
response = minio.get_object('foo', 'bar', 2, 4)
// Read data from response.
finally:
response.close()
Expand Down
34 changes: 0 additions & 34 deletions examples/get_partial_object.py
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions examples/list_objects.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@

# List all object paths in bucket that begin with my-prefixname using
# V2 listing API.
objects = client.list_objects_v2('my-bucketname', prefix='my-prefixname',
recursive=True)
objects = client.list_objects('my-bucketname', prefix='my-prefixname',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example has duplicated code after your change, I guess you need to use api v1 in the first line and update some wording

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

recursive=True)
for obj in objects:
print(obj.bucket_name, obj.object_name.encode('utf-8'), obj.last_modified,
obj.etag, obj.size, obj.content_type)
2 changes: 1 addition & 1 deletion examples/remove_objects.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
try:
names = map(
lambda x: x.object_name,
client.list_objects_v2('my-bucketname', 'my-prefix', recursive=True)
client.list_objects('my-bucketname', 'my-prefix', recursive=True)
)
for err in client.remove_objects('my-bucketname', names):
print("Deletion Error: {}".format(err))
Expand Down
2 changes: 1 addition & 1 deletion minio/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@

__title__ = 'minio-py'
__author__ = 'MinIO, Inc.'
__version__ = '6.0.1'
__version__ = '7.0.0'
__license__ = 'Apache 2.0'
__copyright__ = 'Copyright 2015, 2016, 2017, 2018, 2019, 2020 MinIO, Inc.'

Expand Down
73 changes: 15 additions & 58 deletions minio/api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -871,7 +871,7 @@ def get_object(self, bucket_name, object_name, offset=0, length=0,

// Get object data for offset/length.
try:
response = minio.get_partial_object('foo', 'bar', 2, 4)
response = minio.get_object('foo', 'bar', 2, 4)
// Read data from response.
finally:
response.close()
Expand Down Expand Up @@ -1039,53 +1039,8 @@ def put_object(self, bucket_name, object_name, data, length,
progress=progress)

def list_objects(self, bucket_name, prefix=None, recursive=False,
include_version=False):
"""
Lists object information of a bucket using S3 API version 1, optionally
for prefix recursively.

:param bucket_name: Name of the bucket.
:param prefix: Object name starts with prefix.
:param recursive: List recursively than directory structure emulation.
:param include_version: Flag to control whether include object
versions.
:return: An iterator contains object information.

Example::
# List objects information.
objects = minio.list_objects('foo')
for object in objects:
print(object)

# List objects information those names starts with 'hello/'.
objects = minio.list_objects('foo', prefix='hello/')
for object in objects:
print(object)

# List objects information recursively.
objects = minio.list_objects('foo', recursive=True)
for object in objects:
print(object)

# List objects information recursively those names starts with
# 'hello/'.
objects = minio.list_objects(
'foo', prefix='hello/', recursive=True,
)
for object in objects:
print(object)
"""
return self._list_objects(
bucket_name,
delimiter=None if recursive else "/",
prefix=prefix,
use_version1=True,
include_version=include_version,
)

def list_objects_v2(self, bucket_name, prefix=None, recursive=False,
start_after=None, include_user_meta=False,
include_version=False):
start_after=None, include_user_meta=False,
include_version=False, use_api_v1=False):
"""
Lists object information of a bucket using S3 API version 2, optionally
for prefix recursively.
Expand All @@ -1098,35 +1053,36 @@ def list_objects_v2(self, bucket_name, prefix=None, recursive=False,
user metadata.
:param include_version: Flag to control whether include object
versions.
:param use_api_v1: Flag to control to use ListObjectV1 S3 API or not.
:return: An iterator contains object information.

Example::
# List objects information.
objects = minio.list_objects_v2('foo')
objects = minio.list_objects('foo')
for object in objects:
print(object)

# List objects information those names starts with 'hello/'.
objects = minio.list_objects_v2('foo', prefix='hello/')
objects = minio.list_objects('foo', prefix='hello/')
for object in objects:
print(object)

# List objects information recursively.
objects = minio.list_objects_v2('foo', recursive=True)
objects = minio.list_objects('foo', recursive=True)
for object in objects:
print(object)

# List objects information recursively those names starts with
# 'hello/'.
objects = minio.list_objects_v2(
objects = minio.list_objects(
'foo', prefix='hello/', recursive=True,
)
for object in objects:
print(object)

# List objects information recursively after object name
# 'hello/world/1'.
objects = minio.list_objects_v2(
objects = minio.list_objects(
'foo', recursive=True, start_after='hello/world/1',
)
for object in objects:
Expand All @@ -1138,6 +1094,7 @@ def list_objects_v2(self, bucket_name, prefix=None, recursive=False,
include_user_meta=include_user_meta,
prefix=prefix,
start_after=start_after,
use_api_v1=use_api_v1,
include_version=include_version,
)

Expand Down Expand Up @@ -2151,7 +2108,7 @@ def _list_objects( # pylint: disable=too-many-arguments,too-many-branches
prefix=None, # all
start_after=None, # all: v1:marker, versioned:key_marker
version_id_marker=None, # versioned
use_version1=False,
use_api_v1=False,
include_version=False,
):
"""
Expand All @@ -2172,10 +2129,10 @@ def _list_objects( # pylint: disable=too-many-arguments,too-many-branches
query = {}
if include_version:
query["versions"] = ""
elif not use_version1:
elif not use_api_v1:
query["list-type"] = "2"

if not include_version and not use_version1:
if not include_version and not use_api_v1:
if continuation_token:
query["continuation-token"] = continuation_token
if fetch_owner:
Expand All @@ -2190,7 +2147,7 @@ def _list_objects( # pylint: disable=too-many-arguments,too-many-branches
if start_after:
if include_version:
query["key-marker"] = start_after
elif use_version1:
elif use_api_v1:
query["marker"] = start_after
else:
query["start-after"] = start_after
Expand All @@ -2207,7 +2164,7 @@ def _list_objects( # pylint: disable=too-many-arguments,too-many-branches
objects, is_truncated, start_after, version_id_marker = (
parse_list_object_versions(response.data, bucket_name)
)
elif use_version1:
elif use_api_v1:
objects, is_truncated, start_after = parse_list_objects(
response.data,
bucket_name,
Expand Down
Loading