Skip to content

The many flavors of PUT

Tomas Celaya edited this page Dec 29, 2017 · 2 revisions

The put methods of MantaClient which deal with objects specifically supply overloads for a variety of types:

  • String, with overloads for character sets
  • byte[]
  • File
  • InputStream
  • OutputStream through the putAsOutputStream methods

Note that the OutputStream variants behave differently and have implications on resource consumption different from the other variants. Creating objects using an OutputStream is covered in a later section of this guide.

In addition to creating objects by providing their content directly, we can also create "links" to existing objects with new names without having to actually copy any data around. This is achieved by manta's SnapLink feature.

Directories are also created using a PUT operation achieved by the putDirectory methods. The guide on getting started with directories provides more information about the various ways one can create, list, and delete directories.

Files

Strings aren't terribly interesting so let's upload a local image file residing at /tmp/flower.jpg:

String path = "/you/stor/flower.jpg"
File flower = new File("/tmp/flower.jpg");

MantaObjectResponse videoPutResponse = client.put(path, content);

System.out.println(strPutResponse);

/*

  com.joyent.manta.client.MantaObjectResponse{
    path='/yout/stor/flower.jpg',
    contentLength=null,

    contentType='image/jpeg',

    etag='c21d8844-3725-ce91-baad-a7247d7b233f',
    mtime='Fri, 29 Dec 2017 20:37:09 GMT',
    type='null',
    requestId='01421b28-ecd8-11e7-88af-634b5f995047',
    httpHeaders=MantaHttpHeaders{
      wrappedHeaders={
        connection=keep-alive,
        x-response-time=16020,
        x-server-name=56564894-a7c2-470e-a218-3d859e7e1687,
        computed-md5=0arIhVBgY8p2vraxN1ymkQ==,
        content-type=image/jpeg,
        x-request-id=01421b28-ecd8-11e7-88af-634b5f995047,
        x-load-balancer=72.2.114.72,
        server=Manta,
        date=Fri, 29 Dec 2017 20:37:09 GMT,
        last-modified=Fri, 29 Dec 2017 20:37:09 GMT,
        etag=c21d8844-3725-ce91-baad-a7247d7b233f
      }
    },
    directory=false
  }

In this case, contentType has been automatically detected. We can override the content-type by passing a MantaHttpHeaders object we construct ourselves. This may be necessary if the object extension does not indicate a filetype and automatic filetype detection fails.

byte[]

In the rare case that you have a raw array of bytes to upload into manta, the byte[] variants of put can be used to upload those bytes.

InputStream

It's possible to minimize memory usage at the expense of the ability to automatically retry requests. This can be beneficial if a large object is being uploaded which cannot be saved to disk and is larger than available memory. In this case, passing an InputStream as the request body will allow the client to stream an indefinitely large object into manta.

Let's assume we've been handed an InputStream implementation from another API (some object implementing a getInputStream method in this case). We can upload that data using the following code:

String path = "/you/stor/massive-object"
InputStream generatedData = randomDataSource.getInputStream();

MantaObjectResponse isPutResponse = client.put(path, generatedData);

If the size of the InputStream is known ahead of time and is smaller than the manta.upload_buffer_size setting, the stream will be read into memory and sent as a byte buffer. Specifying a content-length will also avoid the usage of chunked transfer-encoding and can improve performance.

OutputStream

Similar to InputStreams, it is possible upload data through an OutputStream.

MantaObjectOutputStream out = mantaClient.putAsOutputStream(path);

try {
    out.write("small data stream".getBytes(StandardCharsets.UTF_8));
} finally {
    out.close();
}

MantaObjectResponse uploaded = out.getObjectResponse();

See the relevant JavaDoc for an explanation of the resource requirements for writing data through an OutputStream.

A note on headers and metadata

Most of the put* methods also include overloads for providing MantaHttpHeaders and/or MantaMetadata objects in addition to the object content. Since metadata is implemented in the form of m- and e- prefixed headers for plain and encrypted metadata, it is possible to perform the following conversion:

MantaMetadata metadata = new MantaMetadata();

metadata.put("m-motto", "the way of the future");
// more metadata updates...

client.put("/you/stor/profile-data-with-meta", profileData, new MantaHttpHeaders(metadata));