Question: numeric value conversion ambiguity #453
Comments
|
Sparkplug Metrics contain both the value and the value type. The type
allows the Metric's value to be restored on the Host implementation.
As for the value conversion, look at SparkplugBPayloadEncoder.java
<https://github.com/eclipse/tahu/blob/v1.0.3/java/lib/core/src/main/java/org/eclipse/tahu/message/SparkplugBPayloadEncoder.java>
(method
setMetricValue()), SparkplugBPayloadDecoder.java
<https://github.com/eclipse/tahu/blob/v1.0.3/java/lib/core/src/main/java/org/eclipse/tahu/message/SparkplugBPayloadDecoder.java>
(method
getMetricValue()), and a Java topic
<https://docs.oracle.com/javase/specs/jls/se10/html/jls-5.html> on
"Narrowing Primitive Conversions."
The gist is on the encoder side, values are promoted to the respective data
type without losing resolution. So in your case, int8 is just casted to be
an int.
On the Host Side, the decoder, knowing the MetricDataType of the Metric,
then casts the protobuf value to the appropriate environment data type
value by casting. On the decoder, there are data types that are down casted
(e.g. int to int8).
…On Wed, May 10, 2023 at 12:58 AM malessainray ***@***.***> wrote:
What do you want to know?
I'm struggling with transmitting values of types other than the directly
used protobuf types.
For example, to send an int8 I was unable to find any hint in the
specification how I should convert the possibly negative int8 value to a
uint32 for encoding through protobuf. Could you please point me towards the
proper definition of how values must be converted to ensure cross
implementation compatibility? As it's ambiguous how values must be
converted between datatypes and the specification requires implementations
to convert numeric values for transmission I beliebe that the specification
shall provide at the very least guidance, or at best specify how values
must be converted. Otherwise, cross implementation compatibility cannot be
guaranteed, even if all implementations perfectly adhere to the
specification.
Is this related to a Sparkplug Listing request? If so, link the issue from
https://github.com/eclipse-sparkplug/sparkplug.listings here.
*No response*
Version
None
Accept EFTL Terms
- I agree to the terms of EFTL
—
Reply to this email directly, view it on GitHub
<#453>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEF6AZRZYUAKSJODBRO4Z3LXFNDB5ANCNFSM6AAAAAAX4LFTWY>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
Thanks for your reply! What you describe is what I initially assumed and was successful with for initial tests. |
|
I brought up your issue during our most recent specification meeting.
We are pressing forward with the 4.0 specification and don't want to delay
this update.
One of the members mentioned his issue
<#62> to utilize more
of the protobuf's data types. This will be implemented in 4.0 and should
provide more clarity into the handling of value. Though this doesn't
immediately help you.
In addition, we discussed and agreed about the addition of some kind of
documentation to outline how types are handled.
Getting back to your C# situation, when encoding the protobuf stream, there
are certain data promotions to fit the protobuf's implemented data types.
Using the Sparkplug 3.0 DataTypes listed in 6.4.16, here is how types are
converted.
Int8, UInt8, Int16, UInt16, and Int32 are converted to an UInt32 (C# uint).
Int64 and UInt64 are converted to an UInt64 (C# ulong).
UInt32 can be converted to either a UInt32 or a UInt64 (C# uint or ulong).
The 3.0 decoder handles both.
DateTime is converted to a UInt64 that represents the milliseconds since
Midnight, January 1, 1970, UTC.
String, Text, and UUID are strings.
Bytes (byte arrays) are byte arrays.
For the encoding of the array types Int8Array, Int16Array, Int32Array,
Int64Array, UInt8Array, UInt16Array, UInt32Array, UInt64Array, FloatArray,
and DoubleArray, these are converter to little endian ordered byte arrays.
As you experienced, casting of specific values (e.g. negative values) can
cause issues, especially when the represented value cannot be properly
represented by either a uint or a ulong.
I have used a couple of .Net hacks to bit transfer the value and circumvent
the bounds checking. Note, there may be more elegant solutions.
// from Int32 to UInt32
uint u32 = BitConverter.ToUInt32(BitConverter.GetBytes(Int32), 0);
// from Int64 to UInt64
ulong u64 = BitConverter.ToUInt64(BitConverter.GetBytes(Int64), 0);
…On Fri, May 12, 2023 at 8:02 AM malessainray ***@***.***> wrote:
Thanks for your reply! What you describe is what I initially assumed and
was successful with for initial tests.
But when using C#, simply casting a negative value to an unsigned integer
will result in an exception, and since the Sparkplug spec specifies that
signed integers must be converted to unsigned protobuf values for the
transport layer, and the spec doesn't declare how the conversion must be
performed both from a signed value to an unsigned one and vice versa, it is
at best a guess to implement a conversion. One might conclude that the spec
is missing a section when relying on all implementations behaving
identically when casting a negative value to an unsigned integer and back.
I suggest that the next iteration of the spec includes such section on
value conversion to ensure compatibility across applications. I'd
appreciate it if it were possible to get a draft of the above section.
—
Reply to this email directly, view it on GitHub
<#453 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEF6AZUAXLXGX5KQ5OSS45TXFZGIDANCNFSM6AAAAAAX4LFTWY>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
@bryce-nakatani Is there and ETA when v4 will be available? |
|
Based on time and resources, perhaps late 2024. We could always use the
help.
…On Thu, Dec 7, 2023 at 8:28 AM HansM ***@***.***> wrote:
@bryce-nakatani <https://github.com/bryce-nakatani> Is there and ETA when
v4 will be available?
—
Reply to this email directly, view it on GitHub
<#453 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AEF6AZRBF667ER2J6VP3NWLYIHVCXAVCNFSM6AAAAAAX4LFTW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBVGY2TINRYGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Well yeah... I'm already quite busy with keeping my Sparkplug library for C# up-to-date at the moment :D |
What do you want to know?
I'm struggling with transmitting values of types other than the directly used protobuf types.
For example, to send an int8 I was unable to find any hint in the specification how I should convert the possibly negative int8 value to a uint32 for encoding through protobuf. Could you please point me towards the proper definition of how values must be converted to ensure cross implementation compatibility? As it's ambiguous how values must be converted between datatypes and the specification requires implementations to convert numeric values for transmission I beliebe that the specification shall provide at the very least guidance, or at best specify how values must be converted. Otherwise, cross implementation compatibility cannot be guaranteed, even if all implementations perfectly adhere to the specification.
Is this related to a Sparkplug Listing request? If so, link the issue from https://github.com/eclipse-sparkplug/sparkplug.listings here.
No response
Version
None
Accept EFTL Terms
The text was updated successfully, but these errors were encountered: