Old records are not being removed #1056

kasparsd · 2020-03-19T18:05:05Z

From https://wordpress.org/support/topic/old-records-are-not-being-removed/

I have a few sites with the Stream plugin installed. I just noticed that none of them are removing old records. I have the “Keep Records for” setting set to 30, but with records going back more than a year. They aren’t being removed. Any idea why that would happen and how to resolve it?

Duplicate reports:

https://wordpress.org/support/topic/records-not-automatically-deleting/

lkraav · 2020-04-22T10:45:59Z

I can confirm, this is happening on one of our client site as well. Oldest records available are now from 2016. This site was not intended to have an eternal memory.

kopepasah · 2020-06-30T22:11:11Z

I am starting review of this today and will carve out some time to investigate/fix this week.

kopepasah · 2020-07-07T19:29:23Z

@kasparsd did you do any research into how the cron schedules run for deleting the logs. WordPress Cron is notorious for missing events without a dedicated native cron to trigger WordPress Cron. I have a suspicion this may be related to these records not being removed and posted a question in the WordPress support forum topic related to this issue.

kasparsd · 2020-07-08T07:54:40Z

@kopepasah The method used for deleting the older logs is defined here:

stream/classes/class-admin.php

Lines 735 to 779 in d465e35

    
           	public function purge_scheduled_action() { 
        
           		global $wpdb; 
        
           		// Don't purge when in Network Admin unless Stream is network activated. 
        
           		if ( 
        
           			is_multisite() 
        
           			&& 
        
           			is_network_admin() 
        
           			&& 
        
           			! $this->plugin->is_network_activated() 
        
           		) { 
        
           			return; 
        
           		} 
        
           		if ( is_multisite() && $this->plugin->is_network_activated() ) { 
        
           			$options = (array) get_site_option( 'wp_stream_network', array() ); 
        
           		} else { 
        
           			$options = (array) get_option( 'wp_stream', array() ); 
        
           		} 
        
           		if ( ! empty( $options['general_keep_records_indefinitely'] ) || ! isset( $options['general_records_ttl'] ) ) { 
        
           			return; 
        
           		} 
        
           		$days     = $options['general_records_ttl']; 
        
           		$timezone = new DateTimeZone( 'UTC' ); 
        
           		$date     = new DateTime( 'now', $timezone ); 
        
           		$date->sub( DateInterval::createFromDateString( "$days days" ) ); 
        
           		$where = $wpdb->prepare( ' AND `stream`.`created` < %s', $date->format( 'Y-m-d H:i:s' ) ); 
        
           		// Multisite but NOT network activated, only purge the current blog. 
        
           		if ( is_multisite() && ! $this->plugin->is_network_activated() ) { 
        
           			$where .= $wpdb->prepare( ' AND `blog_id` = %d', get_current_blog_id() ); 
        
           		} 
        
           		$wpdb->query( 
        
           			"DELETE `stream`, `meta` 
        
           			FROM {$wpdb->stream} AS `stream` 
        
           			LEFT JOIN {$wpdb->streammeta} AS `meta` 
        
           			ON `meta`.`record_id` = `stream`.`ID` 
        
           			WHERE 1=1 {$where};" // @codingStandardsIgnoreLine $where already prepared 
        
           		); 
        
           	}

Notice that it has several conditionals which can stop the deletion from happening. I haven't verified if all of them work as expected.

The wp_stream_auto_purge action is scheduled here to run twice per day:

stream/classes/class-admin.php

Lines 724 to 728 in d465e35

    
           public function purge_schedule_setup() { 
        
           	if ( ! wp_next_scheduled( 'wp_stream_auto_purge' ) ) { 
        
           		wp_schedule_event( time(), 'twicedaily', 'wp_stream_auto_purge' ); 
        
           	} 
        
           }

There is also a test case for this method and it appears to be passing:

stream/tests/tests/test-class-admin.php

Lines 270 to 307 in 4c60b9c

    
           public function test_purge_scheduled_action() { 
        
           	// Set the TTL to one day 
        
           	if ( is_multisite() && is_plugin_active_for_network( $this->plugin->locations['plugin'] ) ) { 
        
           		$options = (array) get_site_option( 'wp_stream_network', array() ); 
        
           		$options['general_records_ttl'] = '1'; 
        
           		update_site_option( 'wp_stream_network', $options ); 
        
           	} else { 
        
           		$options = (array) get_option( 'wp_stream', array() ); 
        
           		$options['general_records_ttl'] = '1'; 
        
           		update_option( 'wp_stream', $options ); 
        
           	} 
        
           	global $wpdb; 
        
           	// Create (two day old) dummy records 
        
           	$stream_data = $this->dummy_stream_data(); 
        
           	$stream_data['created'] = gmdate( 'Y-m-d h:i:s', strtotime( '2 days ago' ) ); 
        
           	$wpdb->insert( $wpdb->stream, $stream_data ); 
        
           	$stream_id = $wpdb->insert_id; 
        
           	$this->assertNotFalse( $stream_id ); 
        
           	// Create dummy meta 
        
           	$meta_data = $this->dummy_meta_data( $stream_id ); 
        
           	$wpdb->insert( $wpdb->streammeta, $meta_data ); 
        
           	$meta_id = $wpdb->insert_id; 
        
           	$this->assertNotFalse( $meta_id ); 
        
           	// Purge old records and meta 
        
           	$this->admin->purge_scheduled_action(); 
        
           	// Check if the old records have been cleared 
        
           	$stream_results = $wpdb->get_row( "SELECT * FROM {$wpdb->stream} WHERE ID = $stream_id" ); 
        
           	$this->assertEmpty( $stream_results ); 
        
           	// Check if the old meta has been cleared 
        
           	$meta_results = $wpdb->get_row( "SELECT * FROM {$wpdb->streammeta} WHERE meta_id = $meta_id" ); 
        
           	$this->assertEmpty( $meta_results ); 
        
           }

codepuncher · 2020-08-06T08:12:51Z

@kasparsd @kopepasah I have done some testing and I can see that the plugin options are empty on my environments.

stream/classes/class-admin.php

Line 752 in d465e35

$options = (array) get_option( 'wp_stream', array() );

This is an empty array as the wp_stream option is not saved to the database.

After changing the values in the plugin settings and saving multiple times, it does update them and I can then run wp cron event run wp_stream_auto_purge with succession.

Whilst the event could run to completion on some environments and entries are removed, it does not work for a site that has 5,883,474 entries (294,174 pages) as it times out.

user@hostname:~/public/current$ wp cron event run wp_stream_auto_purge
WordPress database error Lock wait timeout exceeded; try restarting transaction for query DELETE `stream`, `meta`
                        FROM wp_stream AS `stream`
                        LEFT JOIN wp_stream_meta AS `meta`
                        ON `meta`.`record_id` = `stream`.`ID`
                        WHERE 1=1  AND `stream`.`created` < '2020-07-06 17:01:33'; made by include('phar:///usr/local/bin/wp/php/boot-phar.php'), include('phar:///usr/local/bin/wp/vendor/wp-cli/wp-cli/php/wp-cli.php'), WP_CLI\bootstrap, WP_CLI\Bootstrap\LaunchRunner->process, WP_CLI\Runner->start, WP_CLI\Runner->run_command_and_exit, WP_CLI\Runner->run_command, WP_CLI\Dispatcher\Subcommand->invoke, call_user_func, WP_CLI\Dispatcher\CommandFactory::WP_CLI\Dispatcher\{closure}, call_user_func, Cron_Event_Command->run, Cron_Event_Command::run_event, do_action_ref_array('wp_stream_auto_purge'), WP_Hook->do_action, WP_Hook->apply_filters, WP_Stream\Admin->purge_scheduled_action
Executed the cron event 'wp_stream_auto_purge' in 51.741s.

kasparsd · 2021-02-12T06:20:16Z

Reopening since #1159 has been reverted.

kasparsd added the bug label Mar 19, 2020

This was referenced Mar 25, 2020

Fix: Remove old records #1075

Closed

Fix: Remove old records #1076

Closed

kasparsd mentioned this issue Apr 16, 2020

3.4.3: prevent DB_Driver_WPDB::insert_record() from inserting empty meta values #1080

Closed

kopepasah self-assigned this Jun 15, 2020

kidunot89 mentioned this issue Aug 20, 2020

Default options provided to "wp_stream_auto_purge" callbacks. #1159

Merged

kidunot89 closed this as completed in #1159 Jan 5, 2021

kasparsd reopened this Feb 12, 2021

kidunot89 linked a pull request Feb 22, 2021 that will close this issue

Settings page defaults repatched #1236

Open

timbutler mentioned this issue Sep 13, 2022

Options defaults fix #1365

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Old records are not being removed #1056

Old records are not being removed #1056

kasparsd commented Mar 19, 2020 •

edited

Loading

lkraav commented Apr 22, 2020

kopepasah commented Jun 30, 2020

kopepasah commented Jul 7, 2020

kasparsd commented Jul 8, 2020

codepuncher commented Aug 6, 2020 •

edited

Loading

kasparsd commented Feb 12, 2021

Old records are not being removed #1056

Old records are not being removed #1056

Comments

kasparsd commented Mar 19, 2020 • edited Loading

lkraav commented Apr 22, 2020

kopepasah commented Jun 30, 2020

kopepasah commented Jul 7, 2020

kasparsd commented Jul 8, 2020

codepuncher commented Aug 6, 2020 • edited Loading

kasparsd commented Feb 12, 2021

kasparsd commented Mar 19, 2020 •

edited

Loading

codepuncher commented Aug 6, 2020 •

edited

Loading