nightly

Automated nightly release, will be updated every day at 20:00 UTC

1.2.0-rc0

What's Changed

• refactor(log): introduce async close by @DaniilRoman in #1503
• feat(s3stream): Modify the memory usage detection interval in ByteBufAlloc through environment variables. by @llzcx in #1557
• fix(s3stream): fix npe when controller not config wal path by @lifepuzzlefun in #1560
• feat(s3stream): add random jitter delay time for StreamObjectsCompaction by @lifepuzzlefun in #1561
• chore: adjust readme file by @KaimingWan in #1549
• feat(e2e): add timeout for ConsumerPerformanceService. by @llzcx in #1563
• feat(e2e): kafka.py JVM OPT uses parameter passing instead of fixed values. by @llzcx in #1562
• test(core): fix broken test by @SCNieh in #1565
• feat(e2e): Memory occupancy E2E test. by @llzcx in #1564
• refactor(s3stream): rename newMultiPartWriter to newLargeObjectWriter and change visibility to public by @warr99 in #1559
• feat: object storage enhancement by @Chillax-0v0 in #1570
• refactor(s3stream): change NetworkBandwidthLimiter attribute access modifiers by @warr99 in #1569
• feat: Downgrade argparse4j to 0.7.0 by @Amitbhave in #1566
• feat(metadata): remove useless s3object fields by @superhx in #1571
• feat(metadata): remove useless fields from s3streamobject by @superhx in #1573
• feat(s3stream): introduce fast retry mechanism in ObjectStorage by @ShadowySpirits in #1574
• feat(s3stream): batch delete in AbstractObjectStorage by @lifepuzzlefun in #1572
• feat(metadata): use delta list for so & sso by @superhx in #1575
• feat(metadata): replace deltamap to timelinehashmap by @superhx in #1580
• fix: use Properties#store to save properties by @Chillax-0v0 in #1581
• fix(test): fix small timeout cause testHighTrafficBatchDelete fail by @lifepuzzlefun in #1583
• chore: change the copyright to AutoMQ HK Limited by @daniel-y in #1586
• feat(core): optimize indexing performance by using sparse index cache by @SCNieh in #1585
• feat(metadata): clean up sparse index map when empty by @SCNieh in #1587
• feat(s3stream): build read options when recover s3 wal by @ShadowySpirits in #1589
• feat(metadata): add sso in-memory index cache by @SCNieh in #1588
• feat(s3stream): fix buffer leak by @ShadowySpirits in #1591
• feat(metadata): add version control to sparse index serialization by @SCNieh in #1590
• feat(wal): graceful shutdown optimization #1578 by @CLFutureX in #1579
• fix(s3stream): add temp method to get bucketId by @SCNieh in #1594
• feat(s3stream): limit s3 wal object size by @ShadowySpirits in #1593
• refactor(s3stream): remove minPartSize parameter from newLargeObjectWriter method by @warr99 in #1598
• test(s3stream): add waiting for InflightReadaheadCf by @llzcx in #1596
• feat(s3stream): refactor s3 client by @ShadowySpirits in #1600
• fix(perf): fix options in "automq-perf-test.sh" by @Chillax-0v0 in #1605
• feat(s3stream): support config MAJOR_V1 skip compact small object by @lifepuzzlefun in #1584
• fix(log): fix consumer offsets load missing (#1606) by @superhx in #1607
• feat(metadata): optimize object lifecycle check by @superhx in #1603
• fix(metadata): fix buffer leak by @SCNieh in #1609
• feat(s3stream): change LocalStreamRangeIndexCache to normal instance by @SCNieh in #1597
• chore: use gif and update badges by @KaimingWan in #1592
• fix(auto_balancer): fix potential memory leak by @SCNieh in #1612
• feat(issues1608): clean up useless range metadata by @superhx in #1610
• fix(metadata): remove duplicated buffer release by @SCNieh in #1614
• fix(s3stream): metric record zero value when read composite index by @lifepuzzlefun in #1601
• feat(s3stream): extract uri util methods by @SCNieh in #1615
• fix(s3stream): parse with raw query string to prevent double decode by @SCNieh in #1617
• fix(s3stream): fix get the v0 automq version when recover by @superhx in #1618
• feat(api): add automq node api by @superhx in #1616
• fix(s3stream): release buffer on exception by @SCNieh in #1622
• fix(s3stream): add compaction delay metrics by @SCNieh in #1623
• feat(e2e): jmx.py supports obtaining the maximum average value of all windows by @llzcx in #1595
• test(e2e): add quota test by @llzcx in #1604
• feat(controller): add node manager by @superhx in #1627
• fix(issue1620): fix flaky test by @SCNieh in #1621
• feat(metadata): use lru cache to control the overall size of node sparse index cache by @SCNieh in #1599
• fix(s3stream): fix version get npe by change the s3stream load timing by @superhx in #1632
• feat(core): optimize LoadAwarePartitionLeaderSelector by @SCNieh in #1633
• fix: update docker-release.yml to fetch tags before determining… by @Gezi-lzq in #1631
• feat(e2e): JMX tool supports obtaining metrics for each broker by @llzcx in #1626
• fix(s3stream): fix index out of bound exception by @SCNieh in #1636
• feat(core): add quota metrics into jmx metrics mapping by @ShadowySpirits in #1637
• fix(api): fix node metadata by @superhx in #1638
• fix(s3stream): fix prepared object deletion by @superhx in #1639
• feat(s3stream): report delta mean values for selected jmx metrics by @SCNieh in #1629
• feat(metadata): add sanity check of getObjects result by @SCNieh in #1634
• feat(core): refactor request quota limitation by @ShadowySpirits in #1635
• feat(admin): add get nodes admin by @superhx in #1641
• test(s3stream): disable unstable test by @SCNieh in #1640
• fix(issues1642): fix bucket not match by @superhx in #1643
• fix(s3stream): release byteBuf before complete readTask by @warr99 in #1644
• fix(ab): fix partition status update by @SCNieh in #1647
• feat(s3stream): add new wal config by @superhx in #1646
• feat(s3stream): IdURI support path substitude by @superhx in #1651
• chore: get extensions in IdURI by @Chillax-0v0 in #1654
• fix(e2e): add init.py to ensure proper module import by @llzcx in #1645
• test(e2e): add autobalancer test by @llzcx in #1653
• feat(s3stream): id uri encode by @superhx in #1656
• fix(metadata): avoid throwing exception to crash process by @SCNieh in #1649
• docs: Correct spelling error in README.md by @jasminaaa20 in #1652
• chore(s3stream): make failover more extensible by @superhx in #1658
• feat(s3stream): add configurable retry delay for objectStorage by @warr99 in #1659
• feat(metrics): use uri to simplify metrics configuration by @SCNieh in #1657
• feat: convert IdURI to BucketURI by @Chillax-0v0 in #1660
• perf(s3stream): increase the object overhead of StreamRecordBatch by @Chillax-0v0 in #1661
• refactor(s3stream): move component initialization to start by @superhx in #1664
• fix(s3stream): fix startup NPE by @superhx in #1667
• feat(s3stream): support burst upload for upload WAL task to reduce failover time. by @lifepuzzlefun in #1662
• feat(metrics): optimize s3 operation metrics interface by @SCNieh in https://github.co...

1.1.3-rc0

What's Changed

• fix(issues1999): fix Processor.channelContexts memory leak by @superhx in #2001
• fix(issue2004): fix AutoBalancerMetricsReporter cannot process with t… by @SCNieh in #2007
• fix(core): release all records before delayed fetch by @ShadowySpirits in #2010
• feat(metrics): switch to stable jvm semconv by @SCNieh in #2011
• perf(DelayedFetch): only try to fast read on complete a delayed fetch by @Chillax-0v0 in #2012
• feat(version): bump version to 1.1.3-rc0 by @superhx in #2015

Full Changelog: 1.1.2...1.1.3-rc0

1.1.2

What's Changed

• fix(log): use the same view to calculate trim offsets by @Chillax-0v0 in #1971
• fix(metadata): limit single kv record size by @SCNieh in #1976
• feat(core): default configuration optimization by @SCNieh in #1981
• chore(version): release 1.1.2 by @SCNieh in #1983

Full Changelog: 1.1.2-rc1...1.1.2

1.1.2-rc1

What's Changed

• fix: update docker-release.yml to fetch tags before determining… by @Gezi-lzq in #1630
• fix(ab): fix partition status update by @SCNieh in #1648
• feat(auto_balancer): optimize broker schedule threshold by @SCNieh in #1748
• feat(auto_balancer): preserve out-of-sync broker and partition in cluster model snapshot by @SCNieh in #1762
• feat(auto_balancer): allow change broker rack on register by @SCNieh in #1771
• feat(auto_balancer): assign partitions within rack on broker fence by @SCNieh in #1778
• feat(metrics): add topic partition distribution metric by @SCNieh in #1782
• fix(metrics): topic partition count metrics should be reported by all brokers by @SCNieh in #1803
• fix(issues1798): force expired data block after create timestamp exceed FORCE_EXPIRED_DELTA_MILLS by @superhx in #1804
• fix(auto_balancer): fix load balancer threshold by @SCNieh in #1806
• feat(s3stream): run s3stream e2e manually by @ShadowySpirits in #1807
• chore(workflow): e2e worflow keep consistent with main by @superhx in #1930
• chore(version): release 1.1.2-rc1 by @SCNieh in #1954

Full Changelog: 1.1.2-rc0...1.1.2-rc1

1.0.6

What's Changed

• fix(metadata): fix mark destoryed object queue won't be cleanup in fo… by @superhx in #1474
• feat(metadata): cleanup object wait last round cleanup record replay … by @superhx in #1476
• fix(log): fix consumer offsets load missing by @superhx in #1602
• chore(version): release 1.0.6 by @SCNieh in #1953

Full Changelog: 1.0.6-rc2...1.0.6

1.1.2-rc0

What's Changed

• fix(log): fix consumer offsets load missing by @superhx in #1606
• fix(auto_balancer): fix potential memory leak (#1612) by @SCNieh in #1613
• fix(build): fix version check for e2e test by @ShadowySpirits in #1624
• fix(build): update docker-compose.yaml by @ShadowySpirits in #1628

Full Changelog: 1.1.1...1.1.2-rc0

1.1.1

What's Changed

• fix(auto_balancer): fix metrics compatibility for version V0 by @SCNieh in #1431
• fix(config): fix default config for auto balancer listener name by @SCNieh in #1436
• fix(metadata): fix mark destoryed object queue won't be cleanup in follower by @superhx in #1473
• feat(metadata): cleanup object wait last round cleanup record replay … by @superhx in #1475
• fix(auto_balancer): fix broker status by @SCNieh in #1479
• fix(auto_balancer): remove partition from cluster model when it has n… by @SCNieh in #1483
• fix(auto_balancer): fix npe when remove partition from cluster by @SCNieh in #1485
• fix(api): fix channel isn't switch when encounter NOT_CONTROLLER by @superhx in #1490
• fix(s3shell): fix object cleanup by @ShadowySpirits in #1492
• feat: bump version to 1.1.1 by @superhx in #1568

Full Changelog: 1.1.0...1.1.1

1.2.0-beta0

What's Changed

• docs(metrics): refine telemetry README and remove unused components by @SCNieh in #1333
• fix(issues1334): fix stream reader endless looping by @superhx in #1335
• fix(perf): increase request timeout of the admin client by @Chillax-0v0 in #1336
• feat(core): forward stream apis by @ShadowySpirits in #1338
• chore: set code owners for the whole repo by @Chillax-0v0 in #1340
• feat(core): add command config by @ShadowySpirits in #1342
• fix(auto_balancer): fix broker status change by @SCNieh in #1339
• feat(metadata): add missing tags for the old stream by @superhx in #1346
• fix(s3stream): refine network rate limiter for better precision by @SCNieh in #1344
• fix(s3stream): fix variable typo by @yx9o in #1351
• docs(telemetry): simplify grafana dashboard templates by @SCNieh in #1354
• feat(shell): remove automq cli by @ShadowySpirits in #1355
• feat(core): log the block operation in controller event loop by @ShadowySpirits in #1352
• fix(s3stream): temporarily disable percentile metrics for performance issue by @SCNieh in #1353
• perf(s3stream/allocator): increase trunk size of the netty allocator by @Chillax-0v0 in #1364
• fix(issues1357): fix missing deletes when load from snapshot by @superhx in #1358
• chore(log): fix log by @superhx in #1365
• fix(s3stream): fix mismatched placeholders by @yx9o in #1369
• feat(shell): add basic command for automq cli by @ShadowySpirits in #1370
• fix(issues1367): stream reader prevent adding invalid blocks by @superhx in #1372
• chore(s3stream): add toString methods to fix logs by @Chillax-0v0 in #1381
• fix(cache): fix readhead fail when user read fast than readahead by @superhx in #1382
• fix(core): add tolerance delay when append with non-zero seq number with empty producer snapshot (#1377) by @SCNieh in #1384
• refactor(s3stream): remove DefaultS3Client's dependency on KafkaConfig by @Chillax-0v0 in #1388
• fix(issues604): fix response size not match by @superhx in #1392
• feat(tools/perf): adjust send rate during warmup and catchup by @Chillax-0v0 in #1395
• feat(s3stream): add new object storage abstract by @superhx in #1396
• fix(action): fix release action (#1398) by @superhx in #1399
• feat(core): optimize the block operation log in controller event loop by @ShadowySpirits in #1403
• fix(auto_balancer): remove unnecessary map copy by @SCNieh in #1404
• feat(core): make controller builder extensible by @SCNieh in #1406
• perf(s3stream): reduce lock granularity in StreamMetadataManager by @lifepuzzlefun in #1411
• fix(TimeIndex): wait for the last append to finish before fetching the stream by @Chillax-0v0 in #1407
• feat(auto_balancer): opt cluster model interface by @SCNieh in #1414
• chore(EventQueue): increase EVENT_PROCESS_TIME_THRESHOLD_MICROSECOND to 5ms by @Chillax-0v0 in #1412
• fix(cache): fix cache reuse for re-created topic by @superhx in #1417
• perf(metadata): use Timeline for objects image by @superhx in #1419
• feat(s3stream): add aws s3 rangeRead implementation for object storage by @warr99 in #1416
• fix(metadata): fix metadata access outdated image by @superhx in #1420
• fix(core): fix unexpected truncation of file cache by @ShadowySpirits in #1421
• feat(core): report s3 object number only on active controller by @SCNieh in #1424
• perf(FileCache): avoid evicting too many blocks at once by @Chillax-0v0 in #1425
• feat: merge from apache kafka 3.8 0971924 by @superhx in #1427
• fix(docs): fix contributing guide by @jitokim in #1428
• chore(examples): remove hardcoded value in TransactionProducer by @jitokim in #1426
• fix(config): fix default config for auto balancer listener name by @SCNieh in #1429
• fix(tools/perf): discard invalid latency values by @Chillax-0v0 in #1430
• test(s3stream): add objectStorage rangeRead test by @warr99 in #1423
• feat(s3stream): add aws s3 write implementation for object storage by @warr99 in #1434
• fix(auto_balancer): fix metrics compatibility for version V0 by @SCNieh in #1433
• fix(auto_balancer): match all endpoints when listener name is not spe… by @SCNieh in #1435
• feat(s3stream): composite object reader&writer by @superhx in #1432
• feat(s3stream): add s3 multipart implementation for object storage by @warr99 in #1438
• feat(s3stream): composite object add object attributes by @superhx in #1437
• fix(s3stream/wal): check the block size before start by @Chillax-0v0 in #1439
• feat(tools): fix StorageTool by @ShadowySpirits in #1442
• feat(s3stream): compact to composite object by @superhx in #1443
• feat(s3stream): add composite object deletion by @superhx in #1444
• feat(s3stream): add s3 delete implementation for object storage by @warr99 in #1441
• perf(s3stream): Avoid non-reuse TimerUtil object creation on write path by @lifepuzzlefun in #1447
• feat(s3stream): object storage writer by @warr99 in #1445
• feat(s3stream): add stream object compaction v1 trigger by @superhx in #1448
• feat(issue1134): prevent trivial scheduling by @SCNieh in #1449
• feat(auto_balancer): move internal topic partition only when necessary by @SCNieh in #1450
• feat(s3stream): adapt to new object storage abstraction by @superhx in #1451
• feat(core): implement broker and cluster level quota by @ShadowySpirits in #1440
• fix: stream e2e by @superhx in #1453
• feat: add composite object test by @superhx in #1454
• feat(s3stream): integrate composite object by @superhx in #1456
• fix(test): fix e2e by @superhx in #1457
• test(s3stream): add test case to verify end-of-object read behavior by @warr99 in #1455
• feat(auto_balancer): refactor cluster load snapshot for better extens… by @SCNieh in #1459
• chore: update README.md by @wensongz in #1460
• refactor(license): organize license files together by @KaimingWan in #1458
• chore: add a readme doc for s3stream by @daniel-y in #1461
• feat(issue1462): Optimize the Readme. #1462 by @vintagewang in #1463
• feat(s3stream): implement a prototype for the S3 Write-Ahead Log by @ShadowySpirits in #1465
• feat(s3stream): remove dependency on stream range offset by @SCNieh in #1466
• feat(metadata): optimize streamsetobject to support huge cluster by @superhx in #1464
• fix(test): fix connect e2e task timeout by @superhx in #1467
• fix(test): skip unsupported test by @superhx in #1468
• fix(s3stream): ensure wal header shutdown type UNGRACEFULLY when startup by @lifepuzzlefun in #1469
• chore(all): adapt to java21 by @superhx in #1471
• feat(s3stream): implement AcknowledgmentService by @ShadowySpirits in #1470
• feat(core): add extension point for TelemetryManager by @SCNieh in #1472
• fix(auto_balancer): fix broker status by @SCNieh in #1480
• fix(auto_balancer): remove partition from cluster model when it has n… ...

1.1.1-rc0

What's Changed

• fix(auto_balancer): fix metrics compatibility for version V0 by @SCNieh in #1431
• fix(config): fix default config for auto balancer listener name by @SCNieh in #1436
• fix(metadata): fix mark destoryed object queue won't be cleanup in follower by @superhx in #1473
• feat(metadata): cleanup object wait last round cleanup record replay … by @superhx in #1475
• fix(auto_balancer): fix broker status by @SCNieh in #1479
• fix(auto_balancer): remove partition from cluster model when it has n… by @SCNieh in #1483
• fix(auto_balancer): fix npe when remove partition from cluster by @SCNieh in #1485
• fix(api): fix channel isn't switch when encounter NOT_CONTROLLER by @superhx in #1490
• fix(s3shell): fix object cleanup by @ShadowySpirits in #1492

Full Changelog: 1.1.0...1.1.1-rc0