[1.x] Add Configurable Trim Duration for Pulse Data Storage

[tharlei]

Description

This pull request introduces a configurable option to define how long Pulse data should be retained in the database. Previously, the retention period was hardcoded to 7 days. With this update, developers can now specify the desired duration through configuration.

PULSE_INGEST_KEEP="1 week" #example

Benefits to End Users

1. Flexibility: Allows developers to adjust the data retention period based on project needs.
2. Compatibily: Pulse already automatically clears out old data.
3. Scalability: Helps optimize database usage by tailoring retention settings to different environments (e.g., production vs. development).

Key Points

• Default retention period remains unchanged unless explicitly configured.
• Fully backward-compatible, ensuring no breakage in existing implementations.

Let me know if any adjustments are required.

[timacdonald]

We already have an option to trim the ingest length. Trimming happens automatically.

We also already trim the storage, e.g., database, however it is hard coded to 7 days of data.

If we wanted to support this feature, think I would prefer we match the existing pattern in Pulse and introduce a configurable trim duration for storage.

Then you don't need to run this command. Pulse automatically clear things out after they become too old.

[tharlei]

We already have an option to trim the ingest length. Trimming happens automatically.

We also already trim the storage, e.g., database, however it is hard coded to 7 days of data.

If we wanted to support this feature, think I would prefer we match the existing pattern in Pulse and introduce a configurable trim duration for storage.

Then you don't need to run this command. Pulse automatically clear things out after they become too old.

Great @timacdonald! Now I see that the trim() method is already being applied to each inserted data.

Taking the opportunity, I applied your suggestion to remove the hardcoded 7-day value from the code and implement a configuration for it. I used the same configuration that is already set in Redis.

Follow commit

And thanks for the support 😄