Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automate blobfuse2 setup for new VM #1575

Merged
merged 16 commits into from
Jan 24, 2025
Merged

Conversation

ashruti-msft
Copy link
Collaborator

@ashruti-msft ashruti-msft commented Nov 22, 2024

✅ What

Created a script automating blobfuse2 setup required in a fresh VM. Also added a script to install azsecpack on new UBN VMs to avoid SFI alerts.

🤔 Why

Helps save time settign up blobfuse2 on a new VM especially while perf testing on multiple VMs

👩‍🔬 How to validate if applicable

run the script ./setup/setupUBN.sh and ./setup/vmSetupAzSecPack.sh

setup.sh Outdated Show resolved Hide resolved
setupUBN.sh Outdated Show resolved Hide resolved
setupUBN.sh Outdated Show resolved Hide resolved
@vibhansa-msft vibhansa-msft added this to the v2-2.4.1 milestone Nov 25, 2024
@vibhansa-msft vibhansa-msft changed the base branch from main to blobfuse/2.4.1 November 29, 2024 07:55
@jainakanksha-msft
Copy link
Collaborator

@ashruti-msft
proper working secpack installation steps are these

curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
      sudo apt-get update
      sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release -y
sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release -y
sudo mkdir -p /etc/apt/keyrings
curl -sLS https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | sudo tee /etc/apt/keyrings/microsoft.gpg > /dev/null
sudo chmod go+r /etc/apt/keyrings/microsoft.gpg
AZ_DIST=$(lsb_release -cs)
echo "Types: deb
URIs: https://packages.microsoft.com/repos/azure-cli/
Suites: ${AZ_DIST}
Components: main
Architectures: $(dpkg --print-architecture)
Signed-by: /etc/apt/keyrings/microsoft.gpg" | sudo tee /etc/apt/sources.list.d/azure-cli.sources
sudo apt-get install azure-cli -y
sudo apt-get update
sudo apt-get install azure-cli -y
sudo apt autoremove
az upgrade

az login --tenant 72f988bf-86f1-41af-91ab-2d7cd011db47
az vm extension set -n AzureMonitorLinuxAgent --publisher Microsoft.Azure.Monitor --version 1.0 --vm-name --resource-group --enable-auto-upgrade true --settings '{"GCS_AUTO_CONFIG": true}'
az vm extension set -n AzureSecurityLinuxAgent --publisher Microsoft.Azure.Security.Monitoring --version 2.0 --vm-name --resource-group --enable-auto-upgrade true --settings '{"enableGenevaUpload":true,"enableAutoConfig":true}'
sudo /usr/local/bin/azsecd status

az vm assess-patches --resource-group --name
az vm install-patches --resource-group --name --maximum-duration PT2H --reboot-setting IfRequired --classifications-to-include-linux Critical Security

setup/vmSetupAzSecPack.sh Outdated Show resolved Hide resolved
setup/vmSetupAzSecPack.sh Show resolved Hide resolved
Ubuntu and others added 2 commits January 23, 2025 07:06
@ashruti-msft ashruti-msft merged commit c576f39 into blobfuse/2.4.1 Jan 24, 2025
8 checks passed
@ashruti-msft ashruti-msft deleted the ashruti/setupBlobfuse branch January 24, 2025 06:56
vibhansa-msft added a commit that referenced this pull request Feb 3, 2025
* feat: support workload identity token (#1556)

* feat: support workload identity token

* Create block pool only once in child process (#1581)

* create block pool in child only

* Update golang.org/x/crypto to v0.31.0 (#1594)

* Update golang.org/x/crypto to v0.31.0

* sync with main (#1603)

* updated year in copyright message (#1601)

* Use ListBlob for hns accounts  (#1555)

* Optimize HNS listing

* Added statfs for block-cache (#1470)

* Added statfs for block_cache

* Add strong consistency check for data on disk (#1604)

* Add strong consistency check for data on disk

* bug in block cache open call (#1580)

* current implementation of open file when opened in O_WRONLY will truncate the file to zero. This is incorrect behaviour.
We don't see it in the normal scenario as write-back cache is on by default. Hence all the open calls with O_WRONLY will be redirected O_RDWR.
To simulate this turn of the write-back cache and then open file in O_WRONLY.

* Feature: Blob filter (#1595)

* Integrating blob filter in azstorage

* Serve getAttr call for destination file after the Copy finishes from the cache

* Cleanup on start shall be set to cleanup temp cache (#1613)

* Add Tests

* Refactor the code and refresh the cache after copying the attributes

* Automate blobfuse2 setup for new VM (#1575)

added script for blobfuse setup and azsecpack setup in VM

* * Update the Unit tests.
* Refactor the Code

* Update Changelog

* do go fmt on src

* Downgrade go version to 1.22.7 due to memory issues in 1.23 (#1619)

* Enable ETAG based validation on every block download to provide higher consistency (#1608)

* Make etag validation a defualt option

* BUG#31069208:  Fixed Prefix filtering from File Path (#1618)

* Fixed the logic to filter out folder prefix from path
* Added/Updated/Removed test case

---------

Co-authored-by: weizhi <weizhichen@microsoft.com>
Co-authored-by: Sourav Gupta <98318303+souravgupta-msft@users.noreply.github.com>
Co-authored-by: Jan Jagusch <77677602+JanJaguschQC@users.noreply.github.com>
Co-authored-by: ashruti-msft <137055338+ashruti-msft@users.noreply.github.com>
Co-authored-by: syeleti-msft <syeleti@microsoft.com>
Co-authored-by: jainakanksha-msft <jainakanksha@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants