v1.16.0

Dragonfly v1.16.0

Our spring release. We are getting closer to 2.0 with some very exciting features ahead. Stay tuned!

Some prominent changes include:

• Improved memory accounting of client connections (#2710 #2755 and #2692 )
• FT.AGGREGATE call (#2413)
• Properly handle and replicate Memcache flags (#2787 #2807)
• Intoduce BF.AGGREGATE BD.(M)ADD and BF.(M)EXISTS methods (#2801). Note, that it does not work with snapshots and replication yet.
• Dragonfly builds natively on MacOS. We would love some help with extending the release pipeline to create a proper macos binary.
• Following the requests from the Edge developers community, we added a basic HTTP API support! Try running Dragonfly with:
  --expose_http_api flag and then call curl -X POST -d '["ping"]' localhost:6379/api. We will follow up with more extensive docs later this month.
• Lots of stability fixes, especially around Sidekiq and BullMQ workloads.

What's Changed

• chore: make usan asan optional and enable them on CI by @kostasrim in #2631
• fix: missing manual trigger for daily sanitizers by @kostasrim in #2682
• bug(tiering): fix overflow in page offset calculation and wrong hash offset calculation by @theyueli in #2683
• Chore: Fixed Docker Health Check by @manojks1999 in #2659
• chore: Increase disk space in the coverage runs by @romange in #2684
• fix(flushall): Decommit memory after releasing tables. by @chakaz in #2691
• feat(server): Account for serializer's temporary buffer size by @chakaz in #2689
• refactor: remove FULL-SYNC-CUT cmd #2687 by @BorysTheDev in #2688
• chore: add malloc-based stats and decommit by @romange in #2692
• feat(cluster): automatic slot migration finalization #2697 by @BorysTheDev in #2698
• Basic FT.AGGREGATE by @dranikpg in #2413
• chore: little transaction cleanup by @dranikpg in #2608
• fix(channel store): add acquire/release pair in fast update path by @dranikpg in #2704
• chore: add ubuntu22 devcontainer by @romange in #2700
• feat(cluster): Add --cluster_id flag by @chakaz in #2695
• feat(server): Use mimalloc in SSL calls by @chakaz in #2710
• chore: Pull helio with new BlockingCounter by @dranikpg in #2711
• chore(transaction): Simplify PollExecution by @dranikpg in #2712
• chore(transaction): Don't call GetLocalMask from blocking controller by @dranikpg in #2715
• chore: improve compatibility of EXPIRE functions with Redis by @romange in #2696
• chore: disable flaky fuzzy migration test by @kostasrim in #2716
• chore: update sanitizers workflow by @kostasrim in #2686
• chore: Use c-ares for resolving hosts in ProtocolClient by @chakaz in #2719
• Remove check-fail in ExpireIfNeeded and introduce DFLY LOAD by @romange in #2699
• chore: Record cmd stat from invoke by @dranikpg in #2720
• fix(transaction): nullptr access on non transactional commands by @kostasrim in #2724
• fix(BgSave): async from sync by @kostasrim in #2702
• chore: remove core/fibers by @romange in #2723
• fix(transaction): Replace with armed sync point by @dranikpg in #2708
• feat(json): Deserialize ReJSON format by @chakaz in #2725
• feat: add flag masteruser by @kostasrim in #2693
• refactor: block list refactoring #2580 by @BorysTheDev in #2732
• chore: fix DeduceExecMode by @dranikpg in #2733
• fix(cluster): Reply with correct \n / \r\n from CLUSTER sub cmd by @chakaz in #2731
• chore: Introduce fiber stack allocator by @romange in #2730
• fix(cluster): Save replica ID per replica by @chakaz in #2735
• fix(ssl): Proper cleanup by @chakaz in #2742
• chore: add skeleton files for flat_dfs code by @romange in #2738
• chore: better error reporting when connecting to tls with plain socket by @romange in #2740
• chore: Support json paths without root selector by @dranikpg in #2747
• chore: journal cleanup by @romange in #2749
• refactor: remove start-slot-migration cmd #2727 by @BorysTheDev in #2728
• feat(server): Add TLS usage to /metrics and INFO MEMORY by @chakaz in #2755
• chore: preparations for adding flat json support by @romange in #2752
• chore(transaction): Introduce RunCallback by @dranikpg in #2760
• feat(replication): Do not auto replicate different master by @chakaz in #2753
• Improve Helm chart to be rendered locally and on machines where is not the application target by @fafg in #2706
• chore: preparation for basic http api by @romange in #2764
• feat(server): Add metric for RDB save duration. by @chakaz in #2768
• chore: fix flat_dfs read tests. by @romange in #2772
• fix(ci): do not overwrite last_log_file among tests by @kostasrim in #2759
• feat(server): support cluster replication by @adiholden in #2748
• fix: fiber preempts on read path and OnCbFinish() clears fetched_items_ by @kostasrim in #2763
• chore(ci): open last_log_file in append mode by @kostasrim in #2776
• doc(README): fix outdated expiry ranges description by @enjoy-binbin in #2779
• Benchmark runner by @adiholden in #2780
• chore(replication-tests): add cache_mode on test replication all by @kostasrim in #2685
• feat(tiering): DiskStorage by @dranikpg in #2770
• chore: add a boilerplate for bloom filter family by @romange in #2782
• chore: introduce conversion routines between JsonType and FlatJson by @romange in #2785
• chore: Fix memcached flags not updated by @dranikpg in #2787
• chore: remove duplicate code from dash and simplify by @kostasrim in #2765
• chore: disable test_cluster_slot_migration by @kostasrim in #2788
• fix: new[] delete[] missmatch in disk_storage by @kostasrim in #2792
• fix: sanitizers clang build and clean up some warnings by @kostasrim in #2793
• chore: add bloom filter class by @romange in #2791
• chore: add SBF data structure by @romange in #2795
• chore(tiering): Disable compilation for MacOs by @dranikpg in #2799
• chore: fix daily build by @romange in #2798
• chore: expose SBF via compact_object by @romange in #2797
• fix(ci): malloc trim on sanitizers workflow by @kostasrim in #2794
• fix(cluster): Don't miss updates in FLUSHSLOTS by @chakaz in #2783
• feat: add bf.(m)add and bf.(m)exists commands by @romange in #2801
• fix: SBF memory leaks by @kostasrim in #2803
• chore: refactor StringFamily::Set to use CmdArgParser by @romange in #2800
• fix: propagate memcached flags to replica by @romange in #2807
• DFLYMIGRATE ACK refactoring by @BorysTheDev in #2790
• feat: add master lsn and journal_executed dcheck in replica via ping by @kostasrim in #277...

v1.15.1

Dragonfly v1.15.1

This is a patch release.

What's Changed

fix: Workaround for a possible crash during BLPOP calls.

Full Changelog: v1.15.0...v1.15.1

v1.15.0

Dragonfly v1.15.0

After more than a month, we are releasing a new version with even more efficient transactional algorithms, improved lua compatibility, better memory management and lots of stability fixes. For Mac enthusiasts - the project now builds natively on MacOs 💻

Some prominent changes include:

• Lua redis.log(..) is recognized (#2476)
• Lua can optionally return doubles as longs when replying to RESP2 clients using --lua_resp2_legacy_float flag (#2672)
• Added backlog option (#2456)

What's Changed

• chore: improve benchmark for bpptree and zsl tree by @romange in #2450
• feat(transaction): Independent out of order execution by @dranikpg in #2426
• fix(RestoreStreamer): Fix a few glitches by @chakaz in #2452
• feat(cluster): add tx execution in cluster_shard_migration by @BorysTheDev in #2385
• feat(server): Add backlog flag for listen() by @chakaz in #2456
• feat: allow reading offloaded strings without loading to the store by @romange in #2449
• refactor: return OpResult in DbSlice::AddOrFind instead of throwing std::bad_alloc by @kostasrim in #2427
• fix(transaction): Improve ACTIVE flags management by @dranikpg in #2458
• test(cluster): Enable seeder to work against a Dragonfly cluster by @chakaz in #2462
• Update README.md to include dragonfly cloud link by @worldsoup in #2469
• fix: resolves the crash with MGET and duplicate keys by @romange in #2471
• fix: fixes for v1.14.0 by @dranikpg in #2473
• feat(lua): add no-op redis.log command by @kostasrim in #2476
• fix: reduce test_pipeline_batching_while_migrating flakiness by @dranikpg in #2475
• chore: switch json object to pmr allocator by @romange in #2483
• chore(transaction): Simplify PollExecution() by @dranikpg in #2457
• chore: fix clang warnings by @romange in #2485
• chore(transaction): Launder copied keys in multi transactions by @dranikpg in #2478
• chore: lock keys when going through fast-path execution by @romange in #2491
• fix(server): mget crash on same key get by @adiholden in #2474
• chore: Introduce LockKey for LockTable by @romange in #2463
• fix(cluster): fix slot filtration to RestoreStreamer by @BorysTheDev in #2477
• feat(lua): add missing no-op redis.replicate_commands() by @kostasrim in #2472
• chore: remove batch reply statistics by @romange in #2490
• Fix redis->DF replication tests by @romange in #2495
• chore: introduce TaskQueue for executing shard local tasks by @romange in #2498
• fix(cluster): fix incorrect version checking and resource double free by @BorysTheDev in #2499
• fix(server): Fix crash in RENAME by @chakaz in #2503
• test: Disable flaky test test_policy_based_eviction_propagation by @chakaz in #2505
• feat(server): Memory tracker by @chakaz in #2501
• fix(server): update post updater iterator in tiering by @adiholden in #2497
• chore(transaction): Use PhasedBarrier for easier synchronization by @dranikpg in #2455
• feat(server): Implement SCRIPT FLUSH command by @highpon in #2493
• fix: do not crash with inconsistent watch queue by @romange in #2515
• Fix: awakened_keys inconcistency by @romange in #2517
• feat(cluster): add migration finalization by @BorysTheDev in #2507
• fix: fix BLOCKING/REVERSE_MAPPING flags for some commands by @romange in #2516
• fix: handle null length strings in stored commands by @dranikpg in #2518
• feat(server): Add MEMORY TRACK ADDRESS command by @chakaz in #2513
• feat(server): Implement CONFIG HELP command by @highpon in #2510
• fix(transaction): Add special barrier for blocking tx by @dranikpg in #2512
• fix(zset): ignore cancelled op status in BZPopMinMax by @andydunstall in #2527
• fix: fix script replication by @dranikpg in #2531
• chore: remove redis sorted set implementation by @romange in #2522
• fix: add emoji when regression tests timeout by @kostasrim in #2543
• fix(cluster): fix #2532 tests redis client close errors by @BorysTheDev in #2542
• bug: DFLY_PASSWORD environment variable deprecation by @mprimeaux in #2539
• fix: wrongly printing timedout emoji on test failures by @kostasrim in #2546
• feat: move lint test chart to seperate action and trigger it on reg tests by @kostasrim in #2536
• Update helio by @romange in #2538
• fix(server): Do not yield in journal if not allowed by @chakaz in #2540
• fix: fix interpreter acquisition with MULTI by @dranikpg in #2549
• fix(tests): increase interpreter test load, lower notice limit, include 2553 by @dranikpg in #2554
• feat: introduce a skeleton for jsonpath parser by @romange in #2521
• fix(pytest): Disable flaky interpreter test monitoring by @dranikpg in #2558
• feat(cluster): add slots blocking when we finalize migration by @BorysTheDev in #2484
• fix(server): Init tx time for all multi/lua transactions by @chakaz in #2562
• chore: enable ContendExpire by @romange in #2563
• fix(server): do not print warning on replica stop/ takeover by @adiholden in #2560
• chore: parse a simple jsonpath by @romange in #2566
• feat(pytest): Gen2 seeder, part 1 by @dranikpg in #2556
• fix: make sure SCRIPT FLUSH concludes by @romange in #2565
• chore: Remove manual memory management from stats by @dranikpg in #2550
• fix: asan errors on unit tests by @kostasrim in #2564
• chore(transaction): Simplify armed state by @dranikpg in #2508
• test(memory): Unaccounted memory test + add DEBUG POPULATE TYPE <type> by @chakaz in #2561
• chore: Recognize exit status in regression tests by @romange in #2571
• chore(server): replica not accumulate multi commands untill exec by @adiholden in #2557
• chore: use helio MPSCIntrusiveQueue queue by @romange in #2573
• chore: provide plumbing for jsonpath error propagation by @romange in #2567
• test(cluster-migration): Fix some bugs and add cluster migration fuzzy tests by @chakaz in #2572
• fix: do not migrate during connection close by @romange in #2570
• chore: fix fiber types in the codebase by @romange in #2574
• feat(release): Also bundle dragonfly-debug builds by @Pothulapati in #2424
• feat: jsonpath supports index/wildcard expressions by @romange in #2578
• fix(tiering tests): introduce wait until tiering entries num EQ/GT by @adiholden in #2559
• chore(pytest): Refactor snapshot test by @dranikpg in #2583
• fix(test): Apply pytest filter to regression tests as well by @chakaz in #2589
• feat: add descent operator to jsonpath parser by @romange in #2588
• test(cluster): Fix and un...

v1.14.7

Dragonfly v1.14.7

This is a patch release.

What's Changed

• chore: add oom stats to /metrics (#2680)
• feat(server): introduce table_growth_margin flag (#2678)

Full Changelog: v1.14.6...v1.14.7

v1.14.6

Dragonfly v1.14.6

This is a patch release.

What's Changed

• fix: data race during snapshotting with INFO calls (#2637)
• fix: avoid crashing during the connection traversal when TLS is enabled (#2651)

Full Changelog: v1.14.5...v1.14.6

v1.14.5

Dragonfly v1.14.5

This is a patch release.

What's Changed

• fix(bug): server crash info memory while saving (#2637)

Full Changelog: v1.14.4...v1.14.5

v1.14.4

Dragonfly v1.14.4

This is a patch release.

What's Changed

• LOADING state error type to be compatible with redis (#2629)
• fix crash in helio library (#2599)
• crash during concurrent save and replica sync (#2597)

Full Changelog: v1.14.3...v1.14.4

v1.14.3

What's Changed

• fix upgrade flow deadlock from earlier version to 1.14.x by @adiholden in #2551
• fix a possible deadlock when running stress workloads ivolving lua and multi/exec combinations

Full Changelog: v1.14.2...v1.14.3

v1.14.2

Dragonfly v1.14.2

This is a patch release.
Fixes stability issues around the replication, memory management and other issues that have been recently identified.

What's Changed

• fix(server): auto updater find key again if needed by @adiholden in #2533
• fix: fix replication v1.14 by @dranikpg in #2537
• fix: switch json object to pmr allocator (#2483)
• fix(server): Fix crash in RENAME (#2503)
• fix: handle null length strings in stored commands (#2518)

Full Changelog: v1.14.1...v1.14.2

v1.14.1

Dragonfly v1.14.1

This is the patch release.
Fixes possible memory leaks during the replication, as well as possible crash in MGET with cache_mode enabled.

What's Changed

• fix: fixes for v1.14.0 by @dranikpg in #2470
• f69f2ec

Full Changelog: v1.14.0...v1.14.1