what's difference between retention and forget

[jee-r]

Hello,

I'm wondering what is the difference between :

• forget https://creativeprojects.github.io/resticprofile/configuration/reference/index.html#section-profileforget
  AND
• retention https://creativeprojects.github.io/resticprofile/configuration/reference/index.html#section-profileretention

They look very similar the only notable difference is that retention can't be scheduled.

Which one should i use ?

Thanks

[jkellerer]

Hi @jee-r, retention can be applied automatically after backup while forget is a separate command that can be used independently. Technically, both sections use restic forget.

In the day to day backup you may want to configure retention for your general lifecycle management and use resticprofile forget to manually adjust the general lifecycle, e.g. to remove snapshots if you accidentally saved something in the backup that you want to remove. So usually you would leave forget unconfigured and only configure retention.

However it also depends on your use case. restic forget (used by sections retention & forget) is a repository wide action, thus not every backup needs to run it if it creates snapshots of the same kind on the same repository.

[jee-r]

Ok, that makes sense, thank you

[fdw]

retention with a schedule is deprecated, it seems. What can I use instead if I want to automatically backup and forget (and maybe even check) at about the same time, but not at once 😉?

[jkellerer]

You can schedule forget but you need to take care to pass correct settings so that the command operates on the snapshots you expect. If in doubt, add tag=true, path=true and host=true to copy backup sources, backup tags and the hostname as conditions for matching snapshots.

[nicfab]

Hi!
I am testing resticprofile, and I find it very powerful.

Below is an excerpt of my configuration (I removed some sensitive info).

version: "1"

global:
  # run 'snapshots' when no command is specified when invoking resticprofile
  default-command: snapshots
  # initialize a repository if none exist at location
  initialize: false
  # priority is using priority class on windows, and "nice" on unixes
  priority: low
  # resticprofile won't start a profile if there's less than 100MB of RAM available
  min-memory: 100
  
# a group is a profile that will call all profiles one by one
groups:
  # when starting a backup on profile "full-backup",
  # it will run the "root" and "src" backup profiles
  full-backup:
    - docs
    - dp
    - xxx
    - sdf

# Default profile when not specified on the command line (-n or --name)
# There's no default inheritance from the 'default' profile,
# you can use the 'inherit' flag if needed
default:
  # add environment variables
  env:
    # AWS S3
    AWS_ACCESS_KEY_ID: "keyid"
    AWS_SECRET_ACCESS_KEY: "accesskey"
    TMPDIR: /tmp
  password-file: "restickey.txt"
  repository: "s3:s3.amazonaws.com/bucketname"

# New profile named 'docs'
docs:
  inherit: default
  initialize: true
  # LOCAL lockfile so you cannot run the same profile more than once at a time
  # (it's totally independent of the restic locks on the repository)
  lock: "/tmp/resticprofile-profile-docs.lock"
  force-inactive-lock: true
  
  backup:
    verbose: true
    exclude-caches: true
    # files with no path are relative to the configuration file
    source:
      - /folder
    tag:
      - docs 
    
    # ignore restic warnings when files cannot be read
    no-error-on-warning: true

    # run every day at 00:15
    schedule: "*-*-* 00:15:00"
    schedule-permission: user
    schedule-priority: background
    schedule-log: "docs-backup.log"
    schedule-lock-wait: 1m
    
  
  # check every month at 2:00
  check:
    read-data: true
    # check the repository the first day of each month at 3am
    schedule: "*-*-01 02:00:00"
    
  forget:
    keep-last: 1
    prune: true
    # forget the repository the first day of each month at 2.50
    schedule: "*-*-* *:50,55,00,05,10,15,20,25,30,35,40,45"
    schedule-permission: user
    schedule-lock-wait: 1m

  # prune every month at 2:30 
  prune:
    schedule: "*-*-01 02:30:00"
    schedule-permission: user
    schedule-lock-wait: 1m

Is there something I miss or wrong?

[creativeprojects]

Sorry for the late answer 😞

A few comment on your configuration:

• you don't need the groups section, it doesn't look correct as it is
• you might want to change your scheduling slightly because at 00:15 you're going to start a backup at the same time as a forget. Only one of them will run (randomly)
• you'll have the same problem the 1st of the month at 2:30 where you're going to try running a forget and prune at the same time.

[nicfab]

Sorry for the late answer 😞

A few comment on your configuration:

* you don't need the `groups` section, it doesn't look correct as it is

* you might want to change your scheduling slightly because at 00:15 you're going to start a `backup` at the same time as a `forget`. Only one of them will run (randomly)

* you'll have the same problem the 1st of the month at 2:30 where you're going to try running a `forget` and `prune` at the same time.

Thank you for your reply.

1. groups
   Why don't I need the groups section? I underlined that the configuration I posted is an excerpt and didn't report all the backups. Can you explain this point?
2. schedule
   I understand that it's not possible to schedule all the backups, forget, and prune at the same time; is it so?
   Is the same valid for check?

[creativeprojects]

1. Sorry, I noticed that the group section contained some backup targets not in the configuration so I wrongly assumed it was a leftover. It's all fine 👍🏻
2. The first command to run will lock the repository and any other command trying to run at the same time will be delayed and eventually cancelled if the other command takes too long to finish. Nothing would break, but you cannot be sure each command will run.