Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Increase the size limit of the log volume #354

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,13 @@ All notable changes to this project will be documented in this file.
- Set explicit resources on all container ([#345], [#347])
- Support podOverrides ([#352])

### Fixed

- Increase the size limit of the log volume ([#354]).

### Changed

- Operator-rs: `0.40.2` -> `0.41.0` ([#336]).
- Operator-rs: `0.40.2` -> `0.44.0` ([#336], [#354]).
- Use 0.0.0-dev product images for testing ([#337])
- Use testing-tools 0.2.0 ([#337])
- Added kuttl test suites ([#348])
Expand All @@ -26,6 +30,7 @@ All notable changes to this project will be documented in this file.
[#347]: https://github.com/stackabletech/hive-operator/pull/347
[#348]: https://github.com/stackabletech/hive-operator/pull/348
[#352]: https://github.com/stackabletech/hive-operator/pull/352
[#354]: https://github.com/stackabletech/hive-operator/pull/354

## [23.4.0] - 2023-04-17

Expand Down
8 changes: 4 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion rust/crd/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ indoc = "1.0.8"
serde = "1.0"
serde_json = "1.0"
snafu = "0.7"
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "0.42.2" }
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "0.44.0" }
strum = { version = "0.24", features = ["derive"] }
tracing = "0.1"

Expand Down
4 changes: 2 additions & 2 deletions rust/operator-binary/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,12 @@ serde = "1.0"
serde_json = "1.0"
snafu = "0.7"
stackable-hive-crd = { path = "../crd" }
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "0.42.2" }
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "0.44.0" }
strum = { version = "0.24", features = ["derive"] }
tokio = { version = "1.23", features = ["full"] }
tracing = "0.1"

[build-dependencies]
built = { version = "0.5", features = ["chrono", "git2"] }
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "0.42.2" }
stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "0.44.0" }
stackable-hive-crd = { path = "../crd" }
24 changes: 11 additions & 13 deletions rust/operator-binary/src/controller.rs
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,10 @@ pub const HIVE_UID: i64 = 1000;
pub const HIVE_CONTROLLER_NAME: &str = "hivecluster";
const DOCKER_IMAGE_BASE_NAME: &str = "hive";

pub const MAX_HIVE_LOG_FILES_SIZE_IN_MIB: u32 = 10;

const OVERFLOW_BUFFER_ON_LOG_VOLUME_IN_MIB: u32 = 1;
const LOG_VOLUME_SIZE_IN_MIB: u32 =
MAX_HIVE_LOG_FILES_SIZE_IN_MIB + OVERFLOW_BUFFER_ON_LOG_VOLUME_IN_MIB;
pub const MAX_HIVE_LOG_FILES_SIZE: MemoryQuantity = MemoryQuantity {
value: 10.0,
unit: BinaryMultiple::Mebi,
};

pub struct Ctx {
pub client: stackable_operator::client::Client,
Expand Down Expand Up @@ -625,6 +624,7 @@ fn build_rolegroup_service(
///
/// The [`Pod`](`stackable_operator::k8s_openapi::api::core::v1::Pod`)s are accessible through the
/// corresponding [`Service`] (from [`build_rolegroup_service`]).
#[allow(clippy::too_many_arguments)]
fn build_metastore_rolegroup_statefulset(
hive: &HiveCluster,
hive_role: &HiveRole,
Expand Down Expand Up @@ -803,14 +803,12 @@ fn build_metastore_rolegroup_statefulset(
}),
..Default::default()
})
.add_volume(Volume {
name: STACKABLE_LOG_DIR_NAME.to_string(),
empty_dir: Some(EmptyDirVolumeSource {
medium: None,
size_limit: Some(Quantity(format!("{LOG_VOLUME_SIZE_IN_MIB}Mi"))),
}),
..Volume::default()
})
.add_empty_dir_volume(
STACKABLE_LOG_DIR_NAME,
Some(product_logging::framework::calculate_log_volume_size_limit(
&[MAX_HIVE_LOG_FILES_SIZE],
)),
)
.affinity(&merged_config.affinity)
.service_account_name(sa_name)
.security_context(
Expand Down
8 changes: 6 additions & 2 deletions rust/operator-binary/src/product_logging.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use crate::controller::MAX_HIVE_LOG_FILES_SIZE_IN_MIB;
use crate::controller::MAX_HIVE_LOG_FILES_SIZE;

use snafu::{OptionExt, ResultExt, Snafu};
use stackable_hive_crd::{Container, HiveCluster, HIVE_LOG4J2_PROPERTIES, STACKABLE_LOG_DIR};
Expand All @@ -7,6 +7,7 @@ use stackable_operator::{
client::Client,
k8s_openapi::api::core::v1::ConfigMap,
kube::ResourceExt,
memory::BinaryMultiple,
product_logging::{
self,
spec::{ContainerLogConfig, ContainerLogConfigChoice, Logging},
Expand Down Expand Up @@ -94,7 +95,10 @@ pub fn extend_role_group_config_map(
container = Container::Hive
),
HIVE_LOG_FILE,
MAX_HIVE_LOG_FILES_SIZE_IN_MIB,
MAX_HIVE_LOG_FILES_SIZE
.scale_to(BinaryMultiple::Mebi)
.floor()
.value as u32,
CONSOLE_CONVERSION_PATTERN,
log_config,
),
Expand Down