Leverage custom script extension post deployment tasks in Azure

The custom script extension is an extension you can apply on your virtual machine or virtual machine scale set that can download and execute scripts inside your operating system. This is beneficial if you have some post deployment tasks that you want achieved once the VM is deployed.
For example you may want a specific version of apache or nginx installed or maybe you want some software removed due to safety concerns. All of this is possible with the extension.
Some guidelines working with the extension and running scripts
There are many reasons for why script execution might fail and one of the most common ones are that the script is syntactically incorrect. It is a good idea to run the script locally first to ensure it actually works to save yourself time troubleshooting later. Some general rules to have mind mind follow here:
- Ensure that no user-input is required. This is an environment where the script will be executed and you will have no way of supplying input. If input is required the script will wait and eventually time out.
- Ensure no reboots happen. If you must use reboots with your script use cron jobs or other tools like DSC, Puppet or Chef to manage your post-deployment tasks.
- There is a max time limit of 90 minutes so ensure your script does not run for longer than that.
The custom script extension will download the script you wish to execute. It can be hosted in Azure Storage, Github or any other place where your VM or VMSS endpoint can access it. You need to ensure that the machine can route towards the location where the script is stored. Be sure to check if any NSG or Firewall blocks these attempts to download the script.
Extension status in the Portal
One way to tell if the script is executing or not is to review the extension status in the Azure Portal. If it has the status of transitioning it means the script is executing.
You will also see this extension status reported using Azure CLI.
Custom script extension property values
There are several settings which you can send to your VM via the Custom Script Exntesion.
Public settings are pretty much what it says. Settings you define here are sent in clear text to the virtual machine or scale set that you are deploying your extension to.
Protected settings are decrypted with a key only known to Azure and your VM or VMSS. The information is decrypted first when at the operating system of your machines.
Some rules regarding the property settings
The following switches are allowed to be configured in both Public & Protected settings of the custom script extension, but not in both places at the same time:
commandToExecute
script
fileUris
You can find all the settings here: https://learn.microsoft.com/en-us/azure/virtual-machines/extensions/custom-script-linux#property-values
Managed identities with version 2.1
Version 2.1 of the extension allows you to configure and use a managed identity to authenticate to for example your storage account. An example configuration might look like this:
{
"fileUris": ["https://mystorage.blob.core.windows.net/privatecontainer/script1.sh"],
"commandToExecute": "sh script1.sh",
"managedIdentity" : { "clientId": "31b403aa-c364-4240-a7ff-d85fb6cd7232" }
}
This is beneficial as you do not need to configure any SAS tokens but you can assign permissions for the managed identity instead.
The extension can be deployed in multiple ways
You can deploy the extension using Powershell or the Azure CLI which is an easy method of installing your scripts quickly.
It is however recommended for automation and documentation purposes to try and use ARM templates, Bicep or Terraform, essentially any IaC tooling you can think of.
The benefit of this is you can document exactly which script and configuration the extension should use and you can deploy & assign the managed identity all the access it needs in one simple deployment. Making your deployments faster and more efficient as you do not need to do so many manual steps.
Here is a simple example of deploying the extension to an existing virtual machine instructing it to install apache2 on the machine by running the command apt-get -y update && apt-get install -y apache2
which is embedded in the "commandToExecute":
setting.
az vm extension set --resource-group tim0329vmRG --vm-name tim0329vm --name customScript --publisher Microsoft.Azure.Extensions --settings '{"commandToExecute": "apt-get -y update && apt-get install -y apache2"}'
Using the custom script extension with Azure Scale Sets
You can deploy the extension with Azure Virtual Machine Scale Sets (VMSS).
Something you need to have in mind though if you have your script stored in Azure Storage and you use a SAS token to access the container is that once the SAS token expires and VMSS performs scale out and deploys more compute units it will fail to fetch the script. This is only a problem if you are using the Azure Portal to deploy the extension as you get no control over the SAS token.
The solution here is to deploy with ARM template, Powershell or CLI and use a managed identity instead.
Troubleshooting options for the custom script extension
There are some troubleshooting steps we can take if we are having problems with the extension. First make sure the extension even ran at all by running the following command against your machine unit: sudo cat /var/log/waagent.log
Once this is verified the next step is to check the script output by running the following command: sudo cat /var/log/azure/custom-script/handler.log
We can also see the execution state of the extension in Azure with: az vm extension list -g myResourceGroup --vm-name myVM
Demo
I have setup a repository with some basic code to deploy a virtual machine into a new network that you can find HERE
Clone the repository and cd terraform
to place yourself in the correct folder.
terraform init
terraform apply -var-file variables/prod.tfvars -auto-approve
(Enter your PIP prompt)
(Enter password prompt)
SSH into the machine using ssh adminuser@public_ip
and enter the password you created.
We can verify that powershell is installed once connected to the machine:

References

About me
