Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Files API docs to the SDK Documentation #556

Merged
merged 1 commit into from
Feb 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/gen-client-docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ def class_methods(self, inst) -> list[MethodDoc]:

def service_docs(self, client_inst) -> list[ServiceDoc]:
client_prefix = 'w' if isinstance(client_inst, WorkspaceClient) else 'a'
ignore_client_fields = ('config', 'dbutils', 'api_client', 'files', 'get_workspace_client', 'get_workspace_id')
ignore_client_fields = ('config', 'dbutils', 'api_client', 'get_workspace_client', 'get_workspace_id')
all = []
for service_name, service_inst in inspect.getmembers(client_inst):
if service_name.startswith('_'):
Expand Down
150 changes: 150 additions & 0 deletions docs/workspace/files/files.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
``w.files``: Files
==================
.. currentmodule:: databricks.sdk.service.files

.. py:class:: FilesAPI

The Files API allows you to read, write, list, and delete files and directories. We support Unity Catalog
volumes with paths starting with "/Volumes/<catalog>/<schema>/<volume>".

The Files API is designed like a standard HTTP API, rather than as a JSON RPC API. This is intended to
make it easier and more efficient to work with file contents as raw bytes.

Because the Files API is a standard HTTP API, the URI path is used to specify the file or directory to
operate on. The path is always absolute.

The Files API has separate endpoints for working with files, `/fs/files`, and working with directories,
`/fs/directories`. The standard HTTP methods `GET`, `HEAD`, `PUT`, and `DELETE` work as expected on these
endpoints.

.. py:method:: create_directory(directory_path: str)

Create a directory.

Creates an empty directory. If necessary, also creates any parent directories of the new, empty
directory (like the shell command `mkdir -p`). If called on an existing directory, returns a success
response; this method is idempotent.

:param directory_path: str
The absolute path of a directory.




.. py:method:: delete(file_path: str)

Delete a file.

Deletes a file. If the request is successful, there is no response body.

:param file_path: str
The absolute path of the file.




.. py:method:: delete_directory(directory_path: str)

Delete a directory.

Deletes an empty directory.

To delete a non-empty directory, first delete all of its contents. This can be done by listing the
directory contents and deleting each file and subdirectory recursively.

:param directory_path: str
The absolute path of a directory.




.. py:method:: download(file_path: str) -> DownloadResponse

Download a file.

Downloads a file of up to 5 GiB. The file contents are the response body. This is a standard HTTP file
download, not a JSON RPC.

:param file_path: str
The absolute path of the file.

:returns: :class:`DownloadResponse`


.. py:method:: get_directory_metadata(directory_path: str)

Get directory metadata.

Get the metadata of a directory. The response HTTP headers contain the metadata. There is no response
body.

This method is useful to check if a directory exists and the caller has access to it.

If you wish to ensure the directory exists, you can instead use `PUT`, which will create the directory
if it does not exist, and is idempotent (it will succeed if the directory already exists).

:param directory_path: str
The absolute path of a directory.




.. py:method:: get_metadata(file_path: str) -> GetMetadataResponse

Get file metadata.

Get the metadata of a file. The response HTTP headers contain the metadata. There is no response body.

:param file_path: str
The absolute path of the file.

:returns: :class:`GetMetadataResponse`


.. py:method:: list_directory_contents(directory_path: str [, page_size: Optional[int], page_token: Optional[str]]) -> Iterator[DirectoryEntry]

List directory contents.

Returns the contents of a directory. If there is no directory at the specified path, the API returns a
HTTP 404 error.

:param directory_path: str
The absolute path of a directory.
:param page_size: int (optional)
The maximum number of directory entries to return. The response may contain fewer entries. If the
response contains a `next_page_token`, there may be more entries, even if fewer than `page_size`
entries are in the response.

We recommend not to set this value unless you are intentionally listing less than the complete
directory contents.

If unspecified, at most 1000 directory entries will be returned. The maximum value is 1000. Values
above 1000 will be coerced to 1000.
:param page_token: str (optional)
An opaque page token which was the `next_page_token` in the response of the previous request to list
the contents of this directory. Provide this token to retrieve the next page of directory entries.
When providing a `page_token`, all other parameters provided to the request must match the previous
request. To list all of the entries in a directory, it is necessary to continue requesting pages of
entries until the response contains no `next_page_token`. Note that the number of entries returned
must not be used to determine when the listing is complete.

:returns: Iterator over :class:`DirectoryEntry`


.. py:method:: upload(file_path: str, contents: BinaryIO [, overwrite: Optional[bool]])

Upload a file.

Uploads a file of up to 5 GiB. The file contents should be sent as the request body as raw bytes (an
octet stream); do not encode or otherwise modify the bytes before sending. The contents of the
resulting file will be exactly the bytes sent in the request body. If the request is successful, there
is no response body.

:param file_path: str
The absolute path of the file.
:param contents: BinaryIO
:param overwrite: bool (optional)
If true, an existing file will be overwritten.



3 changes: 2 additions & 1 deletion docs/workspace/files/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ Manage files on Databricks in a filesystem-like interface
.. toctree::
:maxdepth: 1

dbfs
dbfs
files
Loading