Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cloud-storage-contrib-nio: Add support for newFileChannel #4718

Merged
merged 11 commits into from
Apr 12, 2019
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.nio.channels.FileChannel;
import java.nio.channels.SeekableByteChannel;
import java.nio.file.AccessMode;
import java.nio.file.AtomicMoveNotSupportedException;
Expand Down Expand Up @@ -300,6 +301,48 @@ public SeekableByteChannel newByteChannel(
}
}

/**
* Open a file for reading OR writing. The {@link FileChannel} that is returned will only allow
* reads or writes depending on the {@link OpenOption}s that are specified. If any of the
* following have been specified, the {@link FileChannel} will be write-only: {@link
* StandardOpenOption#CREATE}
*
* <ul>
* <li>{@link StandardOpenOption#CREATE}
* <li>{@link StandardOpenOption#CREATE_NEW}
* <li>{@link StandardOpenOption#WRITE}
* <li>{@link StandardOpenOption#TRUNCATE_EXISTING}
* </ul>
*
* In all other cases the {@link FileChannel} will be read-only.
*
* @param path The path to the file to open or create
* @param options The options specifying how the file should be opened, and whether the {@link
* FileChannel} should be read-only or write-only.
* @param attrs (not supported, the values will be ignored)
Copy link
Contributor

@JesseLovelace JesseLovelace Apr 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the null check for attrs below if this is the case? attrs is optional in File.createFile

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I chose to do so to keep it consistent with the other methods in CloudStorageFileSystemProvider that accept an array of FileAtributes as an input parameter. These do the exact same check, even though the input value is ignored.

(See newByteChannel(...)``createDirectory(...))

Furthermore, newFileChannel method can also be used to read an existing file, so it does not always call File.createFile.

* @throws IOException
*/
@Override
public FileChannel newFileChannel(
Path path, Set<? extends OpenOption> options, FileAttribute<?>... attrs) throws IOException {
checkNotNull(path);
initStorage();
CloudStorageUtil.checkNotNullArray(attrs);
if (options.contains(StandardOpenOption.CREATE_NEW)) {
Files.createFile(path, attrs);
} else if (options.contains(StandardOpenOption.CREATE) && !Files.exists(path)) {
Files.createFile(path, attrs);
}
if (options.contains(StandardOpenOption.WRITE)
|| options.contains(StandardOpenOption.CREATE)
|| options.contains(StandardOpenOption.CREATE_NEW)
|| options.contains(StandardOpenOption.TRUNCATE_EXISTING)) {
return new CloudStorageWriteFileChannel(newWriteChannel(path, options));
} else {
return new CloudStorageReadFileChannel(newReadChannel(path, options));
}
}

private SeekableByteChannel newReadChannel(Path path, Set<? extends OpenOption> options)
throws IOException {
initStorage();
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
/*
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.google.cloud.storage.contrib.nio;

import com.google.common.base.Preconditions;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;
import java.nio.channels.ReadableByteChannel;
import java.nio.channels.SeekableByteChannel;
import java.nio.channels.WritableByteChannel;

class CloudStorageReadFileChannel extends FileChannel {
private static final String READ_ONLY = "This FileChannel is read-only";
private final SeekableByteChannel readChannel;

CloudStorageReadFileChannel(SeekableByteChannel readChannel) {
Preconditions.checkNotNull(readChannel);
this.readChannel = readChannel;
}

@Override
public int read(ByteBuffer dst) throws IOException {
return readChannel.read(dst);
}

@Override
public long read(ByteBuffer[] dsts, int offset, int length) throws IOException {
long res = 0L;
synchronized (this) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might as well jut put synchronized in the signature.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

for (int i = offset; i < offset + length; i++) {
res += readChannel.read(dsts[i]);
}
}
return res;
}

@Override
public int write(ByteBuffer src) throws IOException {
throw new UnsupportedOperationException(READ_ONLY);
}

@Override
public long write(ByteBuffer[] srcs, int offset, int length) throws IOException {
throw new UnsupportedOperationException(READ_ONLY);
}

@Override
public long position() throws IOException {
return readChannel.position();
}

@Override
public FileChannel position(long newPosition) throws IOException {
readChannel.position(newPosition);
return this;
}

@Override
public long size() throws IOException {
return readChannel.size();
}

@Override
public FileChannel truncate(long size) throws IOException {
throw new UnsupportedOperationException(READ_ONLY);
}

@Override
public void force(boolean metaData) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public long transferTo(long position, long count, WritableByteChannel target) throws IOException {
long res = 0L;
synchronized (this) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

synchronized in the signature might be clearer.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

long originalPosition = position();
position(position);
int blockSize = (int) Math.min(count, 0xfffffL);
int bytesRead = 0;
ByteBuffer buffer = ByteBuffer.allocate(blockSize);
while (res < count && bytesRead >= 0) {
buffer.position(0);
bytesRead = read(buffer);
if (bytesRead > 0) {
buffer.position(0);
buffer.limit(bytesRead);
target.write(buffer);
res += bytesRead;
}
}
position(originalPosition);
}
return res;
}

@Override
public long transferFrom(ReadableByteChannel src, long position, long count) throws IOException {
throw new UnsupportedOperationException(READ_ONLY);
}

@Override
public int read(ByteBuffer dst, long position) throws IOException {
synchronized (this) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

long originalPosition = position();
position(position);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this redundant?

      long originalPosition = position();
      position(position);

Do we actually need position(position);?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we need to position the channel where the read should begin. I've changed the name of the parameter to make it clearer.

int res = readChannel.read(dst);
position(originalPosition);
return res;
}
}

@Override
public int write(ByteBuffer src, long position) throws IOException {
throw new UnsupportedOperationException(READ_ONLY);
}

@Override
public MappedByteBuffer map(MapMode mode, long position, long size) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public FileLock lock(long position, long size, boolean shared) throws IOException {
throw new UnsupportedOperationException();
}

@Override
public FileLock tryLock(long position, long size, boolean shared) throws IOException {
throw new UnsupportedOperationException();
}

@Override
protected void implCloseChannel() throws IOException {
readChannel.close();
}
}
Loading