chore: schedule chains

[romange]

Use intrusive queue that allows batching of scheduling calls instead of handling each call separately. This optimizations improves latency and throughput by 3-5%

[dranikpg]

Questionable 🤔🤔🤔 I don't understand the technical motivation behind it. We have the same amount of inter-thread coordination (if not more), it's just a mpsc queue embedded in a mpsc queue.

We should ask why it is costly to use the shard set queue? Dynamic dispatch? Heavy functor objects? Object access times? What if we use variant<ScheduleContext, Functor> in shard set?

[dranikpg]

🤓

alignas(64) MPSC queue;
alignas(64) atomic_bool armed;

[BorysTheDev]

but isntead of 64 exist better option std::hardware_destructive_interference_size

[romange]

yes, but this is just an awful name that translated to 64 anyways

[BorysTheDev]

in the future it can be 128 so awful name is better than const

[romange]

you are right, my answer was emotional 😺

[dranikpg]

Please comment on what it does and what the motiviation is, we'll be puzzling in a few months

[BorysTheDev]

unique_ptr

[romange]

does not provide any value since we must delete manually anyways.

[romange]

1. Batching of code - allows locality. I have not yet measured metrics but the loop may process 10 items and more in a batch
2. More importantly - it will allow us to try and prefetch keys memory in Dashtable when processing operations. So before that it was not possible to prefetch memory for multiple ops because all of them were executed individually.

You are asking a good question - why using intrusive queue and not a regular queue. I do not know what's more efficient but using a variant will require factoring our FiberQueue from helio, so it's more work.

[romange]

FiberQueue uses mpmc_bounded_queue - which notifies a consumer. mpsc is fully lock free - does not allow blocking

[dranikpg]

Batching of code - allows locality. I have not yet measured metrics but the loop may process 10 items and more in a batch

What you call batching and locality are purely code driven concepts. Technically nothing has changed - you executed 10 times the task sequentially before, just each time calling a functor, not the function itself directly. Instead of checking on shard queue atomics you check on your new queue atomics, that's it.

pop one job on shard set {
  for i = 1, 10 {
     Run()
  }
}

and

for i = 1, 10 {
  pop job from shard set {
    Run()
  }
}

More importantly - it will allow us to try and prefetch keys memory in Dashtable when processing operations. So before that it was not possible to prefetch memory for multiple ops because all of them were executed individually.

So we can plan it first on a higher level. Let's revise what we use the shard set queues for. Structured information is always better than generalized (functors) and our execution flow is always better off in a structured way than hidden. So I assume to parallelize memory access one day you intend to

vector<ScheduleCntx> batch = next_ten_readonly();
vector<vector<it>> keys;
for (cntx: batch):
  keys += fetch_iterators()
for (cntx: batch)
  run(cntx, keys[i])

[romange]

Yes, this is the plan. In any case I am not in hurry to submit this Pr. it can stay as food for thought

[dranikpg]

In any case I am not in hurry to submit this Pr

Let's merge, it's just that you reveal your plans step by step whereas we're curious for the final plan

[romange]

hardware_destructive_interference_size does not exist in the gcc versions we use so I can not use it. I added a named const for that. PTAL

[BorysTheDev]

hardware_destructive_interference_size does not exist in the gcc versions we use so I can not use it. I added a named const for that. PTAL

It's really strange because it's c++17

[kostasrim]

hardware_destructive_interference_size does not exist in the gcc versions we use so I can not use it. I added a named const for that. PTAL

It's really strange because it's c++17

the support for C++17 changes from compiler version to version. Same for all C++ releases. For example, you got modules in C++20. Are they properly supported in recent versions of gcc and clang ? I don't think so. Basically, for every new release they just add support of the missing features so even if you compiler something with C++17 flag it doesn't necessarily mean that the feature exists

[dranikpg]

hardware_destructive_interference_size does not exist in the gcc versions we use so I can not use it. I added a named const for that. PTAL

We use absl::hardware_something in a few places 🙂

---

Let's merge and see what we make out of it later