[clickhouse] can't show progress for some query

[BohuTANG]

Summary

If we run this query in ClickHouse client:

SELECT number FROM numbers_mt(10000000000) ORDER BY number DESC LIMIT 100;

It can not show the progress, but the group by query is ok:

datafuse :) SELECT max(number), sum(number) FROM numbers_mt(1000000000) GROUP BY number % 3, number % 4, number % 5;

It seems that only the group by query working.

[compasses]

BTW, just kindly remind, I did some performance test against clickhouse follow the guide from https://datafuse.rs/overview/performance/, all use the newest master branch, but i have a very different result with that, some cases clickhouse is better :)

[BohuTANG]

BTW, just kindly remind, I did some performance test against clickhouse follow the guide from https://datafuse.rs/overview/performance/, all use the newest master branch, but i have a very different result with that, mostly clickhouse is better :)

Hi!
Could you give the env and results of your test?

[compasses]

Hi,
Just some case, before i make some mis config, you can just check the two case appeared in this issue.

Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz 48 core:

[BohuTANG]

Hmm, how do you build the datafuse?
The performance binary build with make build and test on Hardware: AMD Ryzen 7 PRO 4750U, 8 CPU Cores, 16 Threads

[compasses]

Hmm, how do you build the datafuse?

Just run with: cargo run --bin fuse-query --release

instead of make build, seems no difference.

[BohuTANG]

You could try

RUSTFLAGS="-C target-cpu=native" cargo build --bin=fuse-query --features allocator --release

and test again.

[BohuTANG]

On my local, I have set set max_threads=16 for clickhouse, but it seems slower than the default(max_threads=8).
I will re-measure the performance against the latest version ClickHouse, there is already a task here: #1129

Thanks for your report @compasses

[compasses]

Yes, that's meet my test to

RUSTFLAGS="-C target-cpu=native" cargo build --bin=fuse-query --features allocator --release

That's not work on my environment:

.cargo/registry/src/git.luolix.top-1ecc6299db9ec823/snmalloc-sys-0.2.27/snmalloc/src/override/../mem/../mem/../pal/pal_posix.h:294:23: error: ‘getentropy’ was not declared in this scope; did you mean ‘get_entropy64’?
    294 |         if (getentropy(&result, sizeof(result)) != 0)
        |             ~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
        |             get_entropy64
  make[3]: *** [CMakeFiles/snmallocshim-1mib-rust.dir/src/override/rust.cc.o] Error 1
  make[2]: *** [CMakeFiles/snmallocshim-1mib-rust.dir/all] Error 2
  make[1]: *** [CMakeFiles/snmallocshim-1mib-rust.dir/rule] Error 2
  make: *** [snmallocshim-1mib-rust] Error 2
  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 2

BTW, where I can find the newest progress of this project? Currently I cannot create a local table and do some insert & query test right ?

[BohuTANG]

@compasses

1. snmalloc has few problems here, what's your os version?
2. We have roadmap to track the progress Roadmap 2021 #746
3. local table insert is working, there's an example for the memory engine https://datafuse.rs/sqlstatement/data-manipulation-language-dml/dml-insert/
   and, the remote table is working too but has not documents yet, I think it will done soon

[compasses]

cool！big thanks, the os version: SMP Debian 4.14.81.