Not all resources are destroyed after build

[EugenKon]

Overview of the Issue

During AMI build, probably, when build process was interrupted, not all resources were destroyed in AWS.

Reproduction Steps

N/A. The build was 3 week ago. We just noticed this.

Packer version

packer --version
1.9.4

Operating system and Environment details

Darwin Eugens-MBP 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:09:52 PDT 2024; root:xnu-10063.121.3~5/RELEASE_X86_64 x86_64 i386 Darwin

Log Fragments and crash.log files

N/A

[lbajolet-hashicorp]

Hi @EugenKon,

For context when you say the build was interrupted, did Packer have a chance to cleanup the resources, or was it SIGKILLed? If the latter, there's not a lot we can do, as the plugin is supposed to cleanup those resources, but if the process get immediately killed, it doesn't have time to do it, and the resources are left dangling as a result.
Otherwise, if it gets SIGTERMed, it goes through the cleanup phases and removes every temporary resource before ending.

In any case I'll transfer this issue over to the AWS plugin, as it is the component impacted by this, but before we can classify this as a bug to fix, I'd need more information on the context in which the process got terminated, if possible.

Thanks!

[EugenKon]

@lbajolet-hashicorp Unfortunately, I do not know how process was stopped/killed. We just noticed, those lurking resources. It would be nice to store "in progress" resources to some state. Thus, even if packer was terminated, it will warn at the next invocations about some unfinished operations and cleanup all resources.

1. Write info that resource will be created
2. Possible termination of packer
3. Create resource
4. Write info that resource was created
5. Possible termination of packer
6. Build finished, cleanup resources
7. Start packer second time
   8a. If packer was terminated on step 2 it will not see the info about created resource. Resource could be checked if it is created or not and cleaned up.
   8b. If packer was terminated on step 5 it will see that a resource was created, but building was not finished. Cleanup the created resources.

[lbajolet-hashicorp]

Hi @EugenKon,

While I understand the reasoning/need for state, Packer is stateless as of now, and as far as I know there's no plan to change this, as this would have side-effects we'd need to consider before introducing such a feature.

Besides this, a lot of the real-world usage of Packer is in CIs, which are generally running with a temporary file system (typically in containers), which won't have this information in between runs, so even if Packer became stateful, those cases won't see an improvement on this front.

I'll keep the issue open for now, but this leans into wontfix territory unfortunately, sorry to not bring better news.

cc @nywilken for insights