Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

expose method to create parser from direct memory #435

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

expose method to create parser from direct memory #435

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions msgpack-core/src/main/java/org/msgpack/core/buffer/ByteBufferInput.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,16 @@ public void close()
{
// Nothing to do
}

/**
* Create a ByteBufferInput on off heap memory
* @param address the address
* @param offset the offset
* @param length the length
* @return a new ByteBufferInput on the specified address
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This static method returns ByteBuffer not ByteBufferInput

*/
public static ByteBuffer directBuffer(long address, int offset, int length)
{
return DirectBufferAccess.newByteBuffer(address, offset, length, null);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem to me that this method is related to ByteBufferInput.

@xerial Do you think this is the right place to put the method in?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make DirectBufferAccess a public class instead of adding this method.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this location is a bit weird.

}
}
28 changes: 28 additions & 0 deletions msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,15 @@
import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.io.IOContext;
import org.msgpack.core.MessagePack;
import org.msgpack.core.buffer.ByteBufferInput;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.Writer;
import java.nio.ByteBuffer;
import java.util.Arrays;

public class MessagePackFactory
Expand Down Expand Up @@ -98,6 +100,22 @@ public JsonParser createParser(byte[] data)
return _createParser(data, 0, data.length, ioContext);
}

@Override
public JsonParser createParser(byte[] data, int offset, int length)
throws IOException, JsonParseException
{
IOContext ioContext = _createContext(data, false);
return _createParser(data, offset, length, ioContext);
}

public JsonParser createParser(long memoryAddress, int offset, int length)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you might notice since this method doesn't have @Override annotation, Jackson doesn't support this kind of API, I think.

How do you use this method via Jackson's API? It would be great, if you add some unit tests to make it clear 😺

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use the MessagePackFactory directly, currently using reflection.

throws IOException, JsonParseException
{
ByteBuffer byteBuffer = ByteBufferInput.directBuffer(memoryAddress, offset, length);
IOContext ioContext = _createContext(byteBuffer, false);
return _createParser(byteBuffer, ioContext);
}

@Override
public JsonParser createParser(InputStream in)
throws IOException, JsonParseException
Expand All @@ -117,6 +135,16 @@ protected MessagePackParser _createParser(InputStream in, IOContext ctxt)
return parser;
}

protected MessagePackParser _createParser(ByteBuffer in, IOContext ctxt)
throws IOException
{
MessagePackParser parser = new MessagePackParser(ctxt, _parserFeatures, _objectCodec, in, reuseResourceInParser);
if (extTypeCustomDesers != null) {
parser.setExtensionTypeCustomDeserializers(extTypeCustomDesers);
}
return parser;
}

@Override
protected JsonParser _createParser(byte[] data, int offset, int len, IOContext ctxt)
throws IOException, JsonParseException
Expand Down
13 changes: 13 additions & 0 deletions msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
import org.msgpack.core.MessagePack;
import org.msgpack.core.MessageUnpacker;
import org.msgpack.core.buffer.ArrayBufferInput;
import org.msgpack.core.buffer.ByteBufferInput;
import org.msgpack.core.buffer.InputStreamBufferInput;
import org.msgpack.core.buffer.MessageBufferInput;
import org.msgpack.value.ValueType;
Expand All @@ -41,6 +42,7 @@
import java.io.InputStream;
import java.math.BigDecimal;
import java.math.BigInteger;
import java.nio.ByteBuffer;
import java.util.LinkedList;

public class MessagePackParser
Expand Down Expand Up @@ -149,6 +151,17 @@ public MessagePackParser(
this(ctxt, features, new ArrayBufferInput(bytes), objectCodec, bytes, reuseResourceInParser);
}

public MessagePackParser(
IOContext ctxt,
int features,
ObjectCodec objectCodec,
ByteBuffer byteBuffer,
boolean reuseResourceInParser)
throws IOException
{
this(ctxt, features, new ByteBufferInput(byteBuffer), objectCodec, byteBuffer, reuseResourceInParser);
}

private MessagePackParser(IOContext ctxt,
int features,
MessageBufferInput input,
Expand Down