[WIP]Add packer support for building AWS AMI #441

shub-kris · 2024-01-25T15:44:37Z

This PR aims to fix issue #433.

philschmid

Left some comments!

Do you know if we also can add a "print" when the instance is launched with some instructions, e.g. where to find examples, docs and how to run something. Like a welcome messages.

.github/workflows/build-ami.yml

aws-ami-creation/README.md

aws-ami-creation/hcl2-files/sources.pkr.hcl

aws-ami-creation/hcl2-files/variables.pkr.hcl

aws-ami-creation/scripts/install_huggingface_libraries.sh

philschmid · 2024-01-26T08:14:36Z

aws-ami-creation/scripts/install_huggingface_libraries.sh

+echo "Step: install-hugging-face-libraries"
+
+echo "Activating the virtual-env"
+source /opt/aws_neuron_venv_pytorch/bin/activate


This would be a big breaking change to the existing AMI. Where you can straight up use "python". Can we either activate it by default or not use a virtual env?

The base AMI let's us use the neuron package and other pre-installed packages only if we activate this environment. If we plan to not use a virtual env then we need to install everything just like the earlier repo.
Let me check if I can still access the neuron stuff atleast without activating the venv

And in that case we should use https://aws.amazon.com/releasenotes/aws-deep-learning-ami-base-neuron-ubuntu-20-04/ as mentioned in the documentation that

The Base AMI comes with a foundational platform of GPU drivers and acceleration libraries to deploy your own customized deep learning environment.

If those are only installed in venv then we should defaulty activate those on instance start and ssh connection.

okay, will add it in bashrc which should solve it in the final AMI

aws-ami-creation/hcl2-files/build.pkr.hcl

aws-ami-creation/hcl2-files/variables.pkr.hcl

shub-kris · 2024-01-29T09:49:50Z

Left some comments!

Do you know if we also can add a "print" when the instance is launched with some instructions, e.g. where to find examples, docs and how to run something. Like a welcome messages.

Hi @philschmid regarding this, I tried making a file called 99-custom-message in /etc/update-motd.d, it doesn't work when ssshing for the first time but works in the second time and I have no clue why?

I also tried doing something like this

  provisioner "shell" {
    inline = [
      "sudo mv /tmp/welcome.sh /etc/ssh/welcome.sh",
      "sudo chmod +x /etc/ssh/welcome.sh",
      "sudo sed -i 's/#Banner/Banner/' /etc/ssh/sshd_config",
      "sudo sed -i '/^Banner/c\\Banner /etc/ssh/welcome.sh' /etc/ssh/sshd_config",
      "sudo systemctl restart sshd"
    ]
  }

This works but the script is not seen as a script but it rather prints out everything.

So if my /tmp/welcome.sh was like this

#!/bin/bash

"Welcome to your instance!"
"You can find examples and documentation at <link>"
"To run something, use <command>"

Then everything is printed as it is

philschmid · 2024-01-29T11:34:10Z

Left some comments!
Do you know if we also can add a "print" when the instance is launched with some instructions, e.g. where to find examples, docs and how to run something. Like a welcome messages.

Hi @philschmid regarding this, I tried making a file called 99-custom-message in /etc/update-motd.d, it doesn't work when ssshing for the first time but works in the second time and I have no clue why?

I also tried doing something like this
  provisioner "shell" {
    inline = [
      "sudo mv /tmp/welcome.sh /etc/ssh/welcome.sh",
      "sudo chmod +x /etc/ssh/welcome.sh",
      "sudo sed -i 's/#Banner/Banner/' /etc/ssh/sshd_config",
      "sudo sed -i '/^Banner/c\\Banner /etc/ssh/welcome.sh' /etc/ssh/sshd_config",
      "sudo systemctl restart sshd"
    ]
  }
This works but the script is not seen as a script but it rather prints out everything.

So if my /tmp/welcome.sh was like this
#!/bin/bash

"Welcome to your instance!"
"You can find examples and documentation at <link>"
"To run something, use <command>"
Then everything is printed as it is

Please try look into what it is not shown. Having a message for the second ssh is not helpful. Have you tried searching for it?, e.g. https://askubuntu.com/questions/1394600/motd-not-showing-up-on-ubuntu-21-10
Or does the base AMI already have something which can lead to issues etc. It should work since i used other ubuntu based ami in the past which had this.

philschmid

I did another review. Please look at my comments there are a lot of things unclear to me. Also we want to use the existing Neuron AMI. We don't want to install any neuronx dependencies.

.github/workflows/build-ami.yml

infrastructure/ami/scripts/setup-neuron.sh

shub-kris · 2024-01-30T10:43:01Z

.github/workflows/build-ami.yml

+          env:
+            SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
+            SLACK_CHANNEL: ${{ secrets.SLACK_CHANNEL }}


I need to create a channel and an app in order to set this up.
I was thinking of naming it optimum-neuron-ci-daily
@philschmid

can you check with the infra team. They have some conventions for alerts, and they can create a channel

philschmid

Left two minor comments, but overall looks good!

philschmid · 2024-01-30T15:06:53Z

infrastructure/ami/hcl2-files/build.pkr.hcl

+    script = "scripts/install-huggingface-libraries.sh"
+    environment_vars = [
+      "TRANSFORMERS_VERSION=${var.transformers_version}",
+      "OPTIMUM_NEURON_TAG=${var.optimum_neuron_tag}",


Lets unify this an call it OPTIMUM_VERSION

philschmid · 2024-01-30T15:07:25Z

infrastructure/ami/hcl2-files/variables.pkr.hcl

+}
+
+variable "source_ami" {
+  default     = "ami-0fbea04d7389bcd4e"


can we add a comment where/how to get new ami ids?

infrastructure/ami/scripts/welcome-msg.sh

philschmid

LGTM!

dacorvo · 2024-02-01T13:40:00Z

infrastructure/ami/README.md

+## Folder Structure
+
+- [hcl2-files](./hcl2-files/) - Includes different files which are used by a Packer pipeline to build an AMI. The files are:
+  - [build.pkr.hcl](./hcl2-files/build.pkr.hcl): contains the [build](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build) defines what builders are started, how to provision them using [provisioner](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build/provisioner) and if necessary what to do with their artifacts using `post-process`.


Suggested change

- [build.pkr.hcl](./hcl2-files/build.pkr.hcl): contains the [build](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build) defines what builders are started, how to provision them using [provisioner](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build/provisioner) and if necessary what to do with their artifacts using `post-process`.

- [build.pkr.hcl](./hcl2-files/build.pkr.hcl): contains the [build](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build), defines what builders are started, how to provision them using [provisioner](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build/provisioner) and if necessary what to do with their artifacts using `post-process`.

dacorvo · 2024-02-01T13:44:37Z

infrastructure/ami/hcl2-files/variables.pkr.hcl

+}
+
+variable "source_ami" {
+  default     = "ami-0fbea04d7389bcd4e" # To get latest AMI use: aws ec2 describe-images --region us-east-1 --owners amazon --filters 'Name=name,Values=Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) ????????' 'Name=state,Values=available' --query 'reverse(sort_by(Images, &CreationDate))[:1].ImageId' --output text


Consider splitting this comment on multiple lines: not all editors wrap lines automatically (mine doesn't)

dacorvo · 2024-02-01T13:45:19Z

infrastructure/ami/hcl2-files/variables.pkr.hcl

+
+variable "ami_users" {
+  default     = ["754289655784", "558105141721"]
+  description = "AWS accounts to share AMI with"


Are we sure we want to share this on a public repository ?

Umm, then we add it in secrets somewhere I guess.
@philschmid wdyt?

Yes AWS Account IDs are not sensitive.

While account IDs, like any identifying information, should be used and shared carefully, they are not considered secret, sensitive, or confidential information.

https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-identifiers.html#:~:text=While%20account%20IDs%2C%20like%20any,Canonical%20user%20ID

dacorvo · 2024-02-01T13:46:45Z

infrastructure/ami/scripts/install-huggingface-libraries.sh

+echo "OPTIMUM_VERSION: $OPTIMUM_VERSION"
+
+pip install --upgrade --no-cache-dir \
+    "transformers[sklearn,sentencepiece,vision]==$TRANSFORMERS_VERSION" \


Why not letting optimum-neuron pull the right version ?

Would this install the extras? i don't think thats possible.

I don't think it will install extras, https://github.com/huggingface/optimum-neuron/blob/main/setup.py#L16

We require the transformers extras with sklearn,sentencepiece,vision to make sure training for all model works correctly, e.g. models needing sentence piece or pillow.

dacorvo

LGTM, thanks for the pull-request !

dacorvo · 2024-02-01T15:01:20Z

The CI is failing: is this expected ?

shub-kris · 2024-02-01T15:25:15Z

The CI is failing: is this expected ?

@dacorvo fails because of not having the aws account credentials yet in the repo.

philschmid

Lets align the ref and then we are good.

philschmid · 2024-02-02T17:28:24Z

.github/workflows/build-ami.yml

@@ -56,13 +56,13 @@ jobs:

          - name: Packer Validate
            id: validate
-            run: packer validate -var "optimum_version=${{ github.event.inputs.tag }}" -var "region=${{ env.AWS_REGION }}" hcl2-files 
+            run: packer validate -var "optimum_version=${{ github.event.inputs.tag || github.event.repository.default_branch }}" -var "region=${{ env.AWS_REGION }}" hcl2-files 


Why is it here github.event.repository.default_branch and not github.ref? Wouldn't this mean that when we open a PR with changes, we would still install the main branch?

Lets align this. Otherwise i can be confusing for others when the need to makes changes for the ami and the main branch.

Yes, it would then install the main branch.

The workflow either gets triggered when someone makes changes to infrastructure/ami/**' then in that case the optimum_version would be main and if it is triggered automatically using scheduler or manually through cli then it would either be main or the tag passed.

If we want to build this image at any particular change in ref repo, which makes sense then I need to remove the path: infrastructure/ami/**

okay let's keep it but add a comment that others understand why.

@philschmid Added the comments

So shall I go ahead and merge it?

Yes feel free to merge

Add packer support for bulding AWS AMI

044bb6d

shub-kris linked an issue Jan 25, 2024 that may be closed by this pull request

Migrate Hugging Face DLAMI into optimum neuron #433

Closed

6 tasks

shub-kris requested review from dacorvo, philschmid and JingyaHuang January 25, 2024 15:50

shub-kris changed the title ~~Add packer support for building AWS AMI~~ [WIP]Add packer support for building AWS AMI Jan 25, 2024

Add initial workflow, format hcl2 files

c476536

philschmid reviewed Jan 26, 2024

View reviewed changes

dacorvo reviewed Jan 26, 2024

View reviewed changes

aws-ami-creation/hcl2-files/build.pkr.hcl Outdated Show resolved Hide resolved

aws-ami-creation/hcl2-files/variables.pkr.hcl Outdated Show resolved Hide resolved

shub-kris added 3 commits January 26, 2024 17:11

Remove virtual-env, change folder name

7503a14

Update README

6e1791b

Add tagging, update README

4e0de2a

shub-kris requested review from philschmid, glegendre01 and dacorvo January 29, 2024 09:42

Add tag in packer sources

b9cde61

philschmid requested changes Jan 29, 2024

View reviewed changes

shub-kris added 3 commits January 29, 2024 18:28

Remove files, add tag as parameter

7122c49

Remove default value for optimum_neuron_tag

37a05c8

Add welcome message

fb9d397

shub-kris requested a review from philschmid January 30, 2024 09:58

Add slack notification on failure

7c1a513

shub-kris commented Jan 30, 2024

View reviewed changes

Format file, change ami-naming to match existing AMI

14d1af5

philschmid approved these changes Jan 30, 2024

View reviewed changes

Update Slack notification, change variable name, update README

9005e6f

philschmid approved these changes Jan 30, 2024

View reviewed changes

Update workflow trigger path

8b16ad6

shub-kris added 2 commits January 30, 2024 17:46

Fix yaml linting

cdab1d1

Fix yaml

874b1b3

dacorvo reviewed Feb 1, 2024

View reviewed changes

Update README, convert comment to multiline

4f27875

dacorvo approved these changes Feb 1, 2024

View reviewed changes

shub-kris added 4 commits February 2, 2024 17:17

Update name of environment variable

3fcb276

Update name of secrets

d37e445

Update workflow

8c11192

Modify workflow to get tag input

cc1b007

philschmid reviewed Feb 2, 2024

View reviewed changes

Add comment to explain workflow

5ac7fde

shub-kris merged commit 7c7d508 into main Feb 3, 2024
1 check passed

shub-kris deleted the feature/aws-neuron-internal-migrate branch February 3, 2024 12:23

	- [build.pkr.hcl](./hcl2-files/build.pkr.hcl): contains the [build](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build) defines what builders are started, how to provision them using [provisioner](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build/provisioner) and if necessary what to do with their artifacts using `post-process`.
	- [build.pkr.hcl](./hcl2-files/build.pkr.hcl): contains the [build](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build), defines what builders are started, how to provision them using [provisioner](https://developer.hashicorp.com/packer/docs/templates/hcl_templates/blocks/build/provisioner) and if necessary what to do with their artifacts using `post-process`.

[WIP]Add packer support for building AWS AMI #441

[WIP]Add packer support for building AWS AMI #441

Conversation

shub-kris commented Jan 25, 2024 • edited Loading

philschmid left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shub-kris Jan 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shub-kris Jan 29, 2024 • edited Loading

Choose a reason for hiding this comment

shub-kris commented Jan 29, 2024

philschmid commented Jan 29, 2024

philschmid left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philschmid Jan 30, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philschmid left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

philschmid left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shub-kris Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dacorvo left a comment

Choose a reason for hiding this comment

dacorvo commented Feb 1, 2024

shub-kris commented Feb 1, 2024

philschmid left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shub-kris Feb 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shub-kris commented Jan 25, 2024 •

edited

Loading

shub-kris Jan 26, 2024 •

edited

Loading

shub-kris Jan 29, 2024 •

edited

Loading

philschmid Jan 30, 2024 •

edited

Loading

shub-kris Feb 1, 2024 •

edited

Loading

shub-kris Feb 2, 2024 •

edited

Loading