Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parity scale codec #547

Open
wants to merge 39 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
27c62bc
Add parity scale codec
Gauthamastro Sep 6, 2022
394d8f8
Add scale-info too
Gauthamastro Sep 6, 2022
204d127
Add scale-info too
Gauthamastro Sep 6, 2022
610096f
Make from_parts_raw
Gauthamastro Sep 14, 2022
b896337
Merge branch 'master' into master
Gauthamastro Jun 22, 2023
25f0849
Make deps in alphabetical order
Gauthamastro Jun 22, 2023
031d203
Clippy/fix ruleset (#598)
mkatychev Jul 16, 2023
deeb242
Fixes issue with truncating implicitly rounding in some cases (#600)
paupino Jul 17, 2023
091d715
added new rkyv (#597)
mkatychev Jul 17, 2023
67efda9
Version 1.31.0 (#601)
paupino Jul 29, 2023
29d3e2e
Specify version on macro dependency
paupino Jul 29, 2023
a6d5caf
Drop unnecessary byteorder dependency (#603)
psychon Aug 18, 2023
951512d
Fixes issue with is_integer failing on decimal bounds (#605)
paupino Aug 23, 2023
03ce442
Version 1.32 (#606)
paupino Aug 23, 2023
f18e57c
Introduce breaking change check (#608)
paupino Sep 9, 2023
aacdefd
Remove semver checks from build and put into makefile (#609)
paupino Sep 9, 2023
1686b69
Fixes issue whereby scale 29 is required however is optimized away (#…
paupino Nov 10, 2023
ede308d
Upgrade Borsh to resolve vulnerability (#621)
paupino Nov 13, 2023
6cfccf3
deserializes empty string into None Option (#607)
gai6948 Nov 14, 2023
1c80137
Version 1.33.0 (#622)
paupino Nov 14, 2023
c3802db
Fixes precision issue with to_f64 with some numbers without fractions…
paupino Nov 16, 2023
c4d4d3e
Version 1.33.1 (#626)
paupino Nov 16, 2023
829355a
Introduces `macros` feature and cleans up serde documentation a littl…
paupino Dec 1, 2023
f02f5ac
Update readme to add comparisons (#630)
paupino Dec 22, 2023
d323dd4
Remove cargo-edit reference from README (#633)
robjtede Jan 9, 2024
10ee2ee
Auto-document feature guarded items (#634)
robjtede Jan 9, 2024
80e9f08
Reimplement pow function for integer exponent. (#638)
schungx Jan 13, 2024
aa2c3d3
Fixes error handling when second decimal place at tail position (#636)
paupino Jan 13, 2024
bacc60b
Version 1.34 (#640)
paupino Jan 31, 2024
999bb02
Force version number during macro resolution (#642)
paupino Feb 1, 2024
179039a
Rollback macros feature flag due to dependency resolution issues (#643)
paupino Feb 1, 2024
d72e787
Bump macros version
paupino Feb 1, 2024
4666555
Add parity scale codec
Gauthamastro Sep 6, 2022
65ec77a
Add scale-info too
Gauthamastro Sep 6, 2022
e7511dc
Add scale-info too
Gauthamastro Sep 6, 2022
62cf912
Make from_parts_raw
Gauthamastro Sep 14, 2022
d778de7
Make deps in alphabetical order
Gauthamastro Jun 22, 2023
bbcc77a
Merge remote-tracking branch 'origin/master'
Gauthamastro Feb 8, 2024
e1b54cd
Update hashbrown
Gauthamastro Feb 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .buildnumber
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
1
30
0
34
2
11 changes: 11 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[target.'cfg(all())']
rustflags = [
# BEGIN - Embark standard lints v0.4
# do not change or add/remove here, but one can add exceptions after this section
# for more info see: <https://github.com/EmbarkStudios/rust-ecosystem/issues/59>
"-Aclippy::inconsistent_digit_grouping",
"-Aclippy::large_digit_groups",
"-Aclippy::excessive_precision",
"-Aclippy::zero_prefixed_literal",
]

1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@ Cargo.lock
artifacts
corpus
target
.vscode/settings.json
83 changes: 83 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,88 @@
# Version History

## 1.34.2

### Fixed

* Deprecate `macros` feature until circular dependency issue is resolved properly.

## 1.34.1

### Fixed

* Hotfix for circular dependency when using the `macros` feature.

## 1.34.0

### Added

* `rust_decimal_macros` can now be utilized using the `macros` feature flag. ([#628](https://github.com/paupino/rust-decimal/pull/628))

### Fixed

* Reimplement `pow` function to more effectively handle larger exponents ([#638](https://github.com/paupino/rust-decimal/pull/638))
* Fixes error handling when parsing a `Decimal` from a string when a decimal point or placeholder is at the rounding position ([#636](https://github.com/paupino/rust-decimal/pull/636))

### Changed

* Added documentation for serde features as well as a few examples. ([#630](https://github.com/paupino/rust-decimal/pull/630))
* Documentation and rustdoc generation improvements ([#633](https://github.com/paupino/rust-decimal/pull/633), [#634](https://github.com/paupino/rust-decimal/pull/634))

### Credit

Thank you to [@robjtede](https://github.com/robjtede) and [@schungx](https://github.com/schungx) for your contributions this release.

Last but not least, a special thank you to [@Tony-Samuels](https://github.com/Tony-Samuels) for your help with managing `rust_decimal`!

## 1.33.1

### Fixed

* Fixes an issue when converting from a `Decimal` to a float whereby the float would be inaccurate after rounding. ([#625](https://github.com/paupino/rust-decimal/pull/625))

## 1.33.0

### Fixed

* Fixes an issue when adding/subtracting two `Decimal`s when one has a scale of 29. ([#619](https://github.com/paupino/rust-decimal/pull/619))
* An empty string will be parsed as `None` during JSON deserialization instead of panicking. ([#607](https://github.com/paupino/rust-decimal/pull/607))

### Changed

* Upgrades `borsh` to version `1.1` as a result of a [security advisory](https://rustsec.org/advisories/RUSTSEC-2023-0033.html). ([#621](https://github.com/paupino/rust-decimal/pull/621))

### Credit

Thank you to [@gai6948](https://github.com/gai6948) for their contribution! Also thank you to all of those that pushed for the security advisory changes.

## 1.32.0

### Fixed

* Fixes an issue with `is_integer` returning incorrect results for mantissa's 10^n where n >= 10. ([#605](https://github.com/paupino/rust-decimal/pull/605))

### Changed

* `byteorder` is no longer required as a dependency for the postgres feature. ([#603](https://github.com/paupino/rust-decimal/pull/603))

### Credit

Thank you to [@psychon](https://github.com/psychon) for your contribution!

## 1.31.0

### Fixed

* Fixes an issue with `trunc_with_scale` implicitly rounding in some scenarios ([#600](https://github.com/paupino/rust-decimal/pull/600))

### Changed

* Various dependency features were updated.

### Credit

Thank you to [@mkatychev](https://github.com/mkatychev) for your contribution this release.

## 1.30.0

As the minor releases for Rust Decimal are getting smaller, I'll be looking at formally starting version 2 of the
Expand Down
48 changes: 29 additions & 19 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
[[bench]]
harness = false
name = "comparison"
path = "benches/comparison.rs"

[package]
authors = ["Paul Mason <paul@form1.co.nz>"]
build = "build.rs"
Expand All @@ -17,46 +12,51 @@ name = "rust_decimal"
readme = "./README.md"
repository = "https://github.com/paupino/rust-decimal"
rust-version = "1.60"
version = "1.30.0"
version = "1.34.2"

[package.metadata.docs.rs]
all-features = true
rustdoc-args = ["--cfg", "docsrs"]

[dependencies]
arbitrary = { default-features = false, optional = true, version = "1.0" }
arrayvec = { default-features = false, version = "0.7" }
borsh = { default-features = false, optional = true, version = "0.10.0" }
bytecheck = { default-features= false, optional = true, version = "0.6" }
byteorder = { default-features = false, optional = true, version = "1.0" }
borsh = { default-features = false, features = ["derive", "unstable__schema"], optional = true, version = "1.1.1" }
bytes = { default-features = false, optional = true, version = "1.0" }
diesel1 = { default-features = false, optional = true, package = "diesel", version = "1.0" }
diesel2 = { default-features = false, optional = true, package = "diesel", version = "2.0" }
diesel2 = { default-features = false, optional = true, package = "diesel", version = "2.1" }
ndarray = { default-features = false, optional = true, version = "0.15.6" }
num-traits = { default-features = false, features = ["i128"], version = "0.2" }
parity-scale-codec = { optional = true, version = "3.1.5", features = ["max-encoded-len"], default-features = false}
parity-scale-codec-derive = { optional = true, version = "3.1.3",default-features = false}
postgres = { default-features = false, optional = true, version = "0.19" }
proptest = { default-features = false, optional = true, features = ["std"], version = "1.0" }
rand = { default-features = false, optional = true, version = "0.8" }
rkyv = { default-features = false, features = ["size_32", "std"], optional = true, version = "0.7" }
rocket = { default-features = false, optional = true, version = "0.5.0-rc.1" }
rkyv = { default-features = false, features = ["size_32", "std"], optional = true, version = "0.7.44" }
rocket = { default-features = false, optional = true, version = "0.5.0-rc.3" }
#rust_decimal_macros = { default-features = false, optional = true, version = "1.34" } # This needs to a published version
scale-info = {optional=true, version = "2.1.2", features = ["derive"], default-features = false}
serde = { default-features = false, optional = true, version = "1.0" }
serde_json = { default-features = false, optional = true, version = "1.0" }
tokio-postgres = { default-features = false, optional = true, version = "0.7" }

[dev-dependencies]
bincode = { default-features = false, version = "1.0" }
bytes = { default-features = false, version = "1.0" }
criterion = { default-features = false, version = "0.4.0" }
criterion = { default-features = false, version = "0.5" }
csv = "1"
futures = { default-features = false, version = "0.3" }
rand = { default-features = false, features = ["getrandom"], version = "0.8" }
rust_decimal_macros = { path = "macros" } # This should be ok since it's just for tests
rust_decimal_macros = { default-features = false, version = "1.33" }
serde = { default-features = false, features = ["derive"], version = "1.0" }
serde_json = "1.0"
tokio = { default-features = false, features = ["macros", "rt-multi-thread", "test-util"], version = "1.0" }
version-sync = { default-features = false, features = ["html_root_url_updated", "markdown_deps_updated"], version = "0.9" }

[features]
default = ["serde", "std"]
# Removed in 1.34.2 due to an issue during version resolution
#macros = ["dep:rust_decimal_macros"]

borsh = ["dep:borsh", "std"]
c-repr = [] # Force Decimal to be repr(C)
Expand All @@ -66,18 +66,19 @@ db-diesel1-mysql = ["diesel1/mysql", "std"]
db-diesel1-postgres = ["diesel1/postgres", "std"]
db-diesel2-mysql = ["diesel2/mysql", "std"]
db-diesel2-postgres = ["diesel2/postgres", "std"]
db-postgres = ["dep:byteorder", "dep:bytes", "dep:postgres", "std"]
db-tokio-postgres = ["dep:byteorder", "dep:bytes", "dep:postgres", "std", "dep:tokio-postgres"]
db-postgres = ["dep:bytes", "dep:postgres", "std"]
db-tokio-postgres = ["dep:bytes", "dep:postgres", "std", "dep:tokio-postgres"]
legacy-ops = []
maths = []
maths-nopanic = ["maths"]
ndarray = ["dep:ndarray"]
proptest = ["dep:proptest"]
rand = ["dep:rand"]
rkyv = ["dep:rkyv"]
rkyv-safe = ["dep:bytecheck", "rkyv/validation"]
rkyv-safe = ["rkyv/validation"]
rocket-traits = ["dep:rocket"]
rust-fuzz = ["dep:arbitrary"]
scale-codec = ["parity-scale-codec-derive","parity-scale-codec","scale-info"]
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're starting to use the updated format for feature dependencies:

Suggested change
scale-codec = ["parity-scale-codec-derive","parity-scale-codec","scale-info"]
scale-codec = ["dep:parity-scale-codec-derive","dep:parity-scale-codec","dep:scale-info"]

serde = ["dep:serde"]
serde-arbitrary-precision = ["serde-with-arbitrary-precision"]
serde-bincode = ["serde-str"] # Backwards compatability
Expand All @@ -86,8 +87,17 @@ serde-str = ["serde-with-str"]
serde-with-arbitrary-precision = ["serde", "serde_json/arbitrary_precision", "serde_json/std"]
serde-with-float = ["serde"]
serde-with-str = ["serde"]
std = ["arrayvec/std", "borsh?/std", "bytecheck?/std", "byteorder?/std", "bytes?/std", "rand?/std", "rkyv?/std", "serde?/std", "serde_json?/std"]
std = ["arrayvec/std", "borsh?/std", "bytes?/std", "rand?/std", "rkyv?/std", "serde?/std", "serde_json?/std", "parity-scale-codec?/std","scale-info?/std"]
tokio-pg = ["db-tokio-postgres"] # Backwards compatability

[[bench]]
harness = false
name = "comparison"
path = "benches/comparison.rs"

[workspace]
members = [".", "./macros"]
members = [
".",
"./macros"
]
resolver = "2"
86 changes: 71 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,6 @@ The binary representation consists of a 96 bit integer number, a scaling factor

## Installing

Using [`cargo-edit`](https://crates.io/crates/cargo-edit):

```sh
$ cargo add rust_decimal
```
Expand All @@ -29,16 +27,16 @@ Alternatively, you can edit your `Cargo.toml` directly and run `cargo update`:

```toml
[dependencies]
rust_decimal = "1.30"
rust_decimal_macros = "1.30"
rust_decimal = "1.34"
rust_decimal_macros = "1.34"
```

## Usage

Decimal numbers can be created in a few distinct ways. The easiest and most efficient method of creating a Decimal is to use the procedural macro within the `rust_decimal_macros` crate:
Decimal numbers can be created in a few distinct ways. The easiest and most efficient method of creating a Decimal is to use the procedural macro that can be enabled using the `macros` feature:

```rust
// Procedural macros need importing directly
// Import the `rust_decimal_macros` crate and use the macro directly from there.
use rust_decimal_macros::dec;

let number = dec!(-1.23) + dec!(3.45);
Expand Down Expand Up @@ -198,8 +196,8 @@ Enable `rust-fuzz` support by implementing the `Arbitrary` trait.

### `serde-float`

**Note:** it is recommended to use the `serde-with-*` features for greater control. This allows configurability at the data
level.
> **Note:** This feature applies float serialization/deserialization rules as the default method for handling `Decimal` numbers.
See also the `serde-with-*` features for greater flexibility.

Enable this so that JSON serialization of `Decimal` types are sent as a float instead of a string (default).

Expand All @@ -212,8 +210,8 @@ e.g. with this turned on, JSON serialization would output:

### `serde-str`

**Note:** it is recommended to use the `serde-with-*` features for greater control. This allows configurability at the data
level.
> **Note:** This feature applies string serialization/deserialization rules as the default method for handling `Decimal` numbers.
See also the `serde-with-*` features for greater flexibility.

This is typically useful for `bincode` or `csv` like implementations.

Expand All @@ -227,17 +225,20 @@ converting to `f64` _loses_ precision, it's highly recommended that you do NOT e

### `serde-arbitrary-precision`

**Note:** it is recommended to use the `serde-with-*` features for greater control. This allows configurability at the data
level.
> **Note:** This feature applies arbitrary serialization/deserialization rules as the default method for handling `Decimal` numbers.
See also the `serde-with-*` features for greater flexibility.

This is used primarily with `serde_json` and consequently adds it as a "weak dependency". This supports the
`arbitrary_precision` feature inside `serde_json` when parsing decimals.

This is recommended when parsing "float" looking data as it will prevent data loss.

Please note, this currently serializes numbers in a float like format by default, which can be an unexpected consequence. For greater
control over the serialization format, please use the `serde-with-arbitrary-precision` feature.

### `serde-with-float`

Enable this to access the module for serializing `Decimal` types to a float. This can be use in `struct` definitions like so:
Enable this to access the module for serializing `Decimal` types to a float. This can be used in `struct` definitions like so:

```rust
#[derive(Serialize, Deserialize)]
Expand All @@ -254,9 +255,18 @@ pub struct OptionFloatExample {
}
```

Alternatively, if only the serialization feature is desired (e.g. to keep flexibility while deserialization):
```rust
#[derive(Serialize, Deserialize)]
pub struct FloatExample {
#[serde(serialize_with = "rust_decimal::serde::float::serialize")]
value: Decimal,
}
```

### `serde-with-str`

Enable this to access the module for serializing `Decimal` types to a `String`. This can be use in `struct` definitions like so:
Enable this to access the module for serializing `Decimal` types to a `String`. This can be used in `struct` definitions like so:

```rust
#[derive(Serialize, Deserialize)]
Expand All @@ -273,9 +283,19 @@ pub struct OptionStrExample {
}
```

This feature isn't typically required for serialization however can be useful for deserialization purposes since it does not require
a type hint. Consequently, you can force this for just deserialization by:
```rust
#[derive(Serialize, Deserialize)]
pub struct StrExample {
#[serde(deserialize_with = "rust_decimal::serde::str::deserialize")]
value: Decimal,
}
```

### `serde-with-arbitrary-precision`

Enable this to access the module for serializing `Decimal` types to a `String`. This can be use in `struct` definitions like so:
Enable this to access the module for deserializing `Decimal` types using the `serde_json/arbitrary_precision` feature. This can be used in `struct` definitions like so:

```rust
#[derive(Serialize, Deserialize)]
Expand All @@ -292,6 +312,20 @@ pub struct OptionArbitraryExample {
}
```

An unexpected consequence of this feature is that it will serialize as a float like number. To prevent this, you can
target the struct to only deserialize with the `arbitrary_precision` feature:
```rust
#[derive(Serialize, Deserialize)]
pub struct ArbitraryExample {
#[serde(deserialize_with = "rust_decimal::serde::arbitrary_precision::deserialize")]
value: Decimal,
}
```

This will ensure that serialization still occurs as a string.

Please see the `examples` directory for more information regarding `serde_json` scenarios.

### `std`

Enable `std` library support. This is enabled by default, however in the future will be opt in. For now, to support `no_std`
Expand All @@ -309,3 +343,25 @@ which was released on `2022-04-07`.
This library maintains support for rust compiler versions that are 4 minor versions away from the current stable rust compiler version.
For example, if the current stable compiler version is `1.50.0` then we will guarantee support up to and including `1.46.0`.
Of note, we will only update the minimum supported version if and when required.

## Comparison to other Decimal implementations

During the development of this library, there were various design decisions made to ensure that decimal calculations would
be quick, accurate and efficient. Some decisions, however, put limitations on what this library can do and ultimately what
it is suitable for. One such decision was the structure of the internal decimal representation.

This library uses a mantissa of 96 bits made up of three 32-bit unsigned integers with a fourth 32-bit unsigned integer to represent the scale/sign
(similar to the C and .NET Decimal implementations).
This structure allows us to make use of algorithmic optimizations to implement basic arithmetic; ultimately this gives us the ability
to squeeze out performance and make it one of the fastest implementations available. The downside of this approach however is that
the maximum number of significant digits that can be represented is roughly 28 base-10 digits (29 in some cases).

While this constraint is not an issue for many applications (e.g. when dealing with money), some applications may require a higher number of significant digits to be represented. Fortunately,
there are alternative implementations that may be worth investigating, such as:

* [bigdecimal](https://crates.io/crates/bigdecimal)
* [decimal-rs](https://crates.io/crates/decimal-rs)

If you have further questions about the suitability of this library for your project, then feel free to either start a
[discussion](https://github.com/paupino/rust-decimal/discussions) or open an [issue](https://github.com/paupino/rust-decimal/issues) and we'll
do our best to help.
Loading