On macOS arm64, quanta::Instant::elapsed method tends to return a shorter duration than std::time::Instant::elapsed does

[tatsuya6502]

• quanta versions: v0.12.1, v0.11.1
• Platform: macOS arm64

On macOS arm64, quanta::Instant::elapsed method tends to return a shorter duration than std::time::Instant::elapsed does.

Steps to reproduce

Run the following program. It measures the duration of 1 send thread sleep using std::time::Instant and quanta::Instant.

// Cargo.toml
//
// [dependencies]
// quanta = "0.12.1"

use std::{thread, time::Duration};

fn main() {
    const N: usize = 20;
    measure_elapsed("std", N, elapsed_std);
    measure_elapsed("quanta", N, elapsed_quanta);
}

fn measure_elapsed<F>(ty: &str, repeat: usize, f: F)
where
    F: Fn() -> Duration,
{
    let mut elapsed_sum = 0;

    for i in 1..=repeat {
        let elapsed = f().as_nanos();
        println!("{ty}: {i:02} - {elapsed} ns");
        elapsed_sum += elapsed;
    }

    let elapsed_avg = elapsed_sum / repeat as u128;
    println!("{ty} - avg: {elapsed_avg} ns");
}

fn elapsed_std() -> Duration {
    let start = std::time::Instant::now();
    thread::sleep(Duration::from_secs(1));
    start.elapsed()
}

fn elapsed_quanta() -> Duration {
    let start = quanta::Instant::now();
    thread::sleep(Duration::from_secs(1));
    start.elapsed()
}

Expected result

Both std::time::Instant and quanta::Instant should return durations slightly longer than 1 second, the sleep duration.

Actual results

Rust 1.75.0.

• On macOS AArch64 (arm64), quanta::Instant::elapsed method returned durations slightly shorter then 1 second.
• On Linux AArch64 and x86_64, quanta::Instant::elapsed method returned durations slightly longer then 1 second.

OS	CPU arch	Target triple	std::time::Instant	quanta::Instant v0.12.1	quanta::Instant v0.11.1
macOS Sonoma 14.2.1	arm64 (Apple M1)	aarch64-apple-darwin	1,003,161,220 ns	987,545,939 ns	986,960,251 ns
Ubuntu 22.04	AArch64 (Amazon Graviton)	aarch64-unknown-linux-gnu	1,000,113,459 ns	1,000,125,420 ns	1,000,129,295 ns
Ubuntu 22.04	x86_64	x86_64-unknown-linux-gnu	1,000,147,215 ns	1,000,150,965 ns	1,000,145,745 ns

[tobz]

Thanks for filing this issue and providing an MCVE. 🙇🏻

What version of macOS are you testing this on?

When I run this locally on an older macOS platform (Intel-based MBP, macOS Ventura 13.4), I don't observe a difference between the two variants... so it seems like either the difference is coming from the macOS version, or potentially from the hardware architecture.

[tatsuya6502]

What version of macOS are you testing this on?

Ah, sorry, I forgot to mention. It is macOS Sonoma 14.2.1. (Mac mini M1)

[tobz]

Alright, good to know. I'm not ready to update locally to Sonoma yet (😅) but I think we can continue to do some debugging here.

Firstly, here's a slightly updated version of your MVCE which uses hdrhistogram just to get more information about the spread of the measurements: https://gist.github.com/tobz/79f207c7af13f792af42f4845af853a4. It'd be good if you could run that and report back the results.

Secondly, I created a branch -- tobz/debug-macos-measurement-error -- that changes how we measure the monotonic time on macOS/iOS to be the same as other Unix-based platforms. It'd be good if you could try this with either your MVCE or the one I linked above.

This change is based a hunch that the way we measure time on macOS currently -- mach_absolute_time -- is being affected by better/more granular device sleep on your M1-based system, whereas on my older system, the processor isn't being put to sleep and so time isn't being affected. The monotonic logic for Unix should theoretically measure time in a way that isn't affected by anything else... but we'll see.

[tatsuya6502]

Thanks for working on this so quickly!

The monotonic logic for Unix should theoretically measure time in a way that isn't affected by anything else... but we'll see.

I believe it is working! Here are the results.

Quanta v0.12.1

$ cargo tree -i quanta
quanta v0.12.1
└── quanta-stdlib-deviation v0.1.0 (...)

$ cargo run --release
running test for std...
.................... done!

std statistics:
  min: 1000300544ns
  p50: 1004347391ns
  p99: 1005293567ns
  max: 1005293567ns
  range: 4993023ns (100.03005% to 100.52936% of target)

running test for quanta...
.................... done!

quanta statistics:
  min: 984543232ns
  p50: 988971007ns
  p99: 988979199ns
  max: 988979199ns
  range: 4435967ns (98.45432% to 98.89792% of target)

Quanta tobz/debug-macos-measurement-error branch

$ cargo tree -i quanta
quanta v0.12.1 (https://github.com/metrics-rs/quanta?branch=tobz/debug-macos-measurement-error#55d077b1)
└── quanta-stdlib-deviation v0.1.0 (...)

$ cargo run --release
running test for std...
.................... done!

std statistics:
  min: 1000251392ns
  p50: 1005043711ns
  p99: 1005060095ns
  max: 1005060095ns
  range: 4808703ns (100.02514% to 100.50601% of target)

running test for quanta...
.................... done!

quanta statistics:
  min: 1000505344ns
  p50: 1003974655ns
  p99: 1005060095ns
  max: 1005060095ns
  range: 4554751ns (100.05053% to 100.50601% of target)

Environment

• macOS Sonoma 14.2.1
• arm64 (M1 chip)

$ sw_vers
ProductName:		macOS
ProductVersion:		14.2.1
BuildVersion:		23C71

$ rustc -Vv
rustc 1.75.0 (82e1608df 2023-12-21)
binary: rustc
commit-hash: 82e1608dfa6e0b5569232559e3d385fea5a93112
commit-date: 2023-12-21
host: aarch64-apple-darwin
release: 1.75.0
LLVM version: 17.0.6

[tobz]

Awesome!

I'll get the change merged in shortly. 👍🏻

[tobz]

Released the fix as quanta@v0.12.2.

Thanks again for the report and providing the MVCE. 🙇🏻