How to upload a file to an Azure Storage Blob PowerShell

Intro

We are configuring an Azure Automation environment. This is all done through Azure DevOps CI/CD.

We won’t use a static files server for files like html content or zip files, but we will upload them to an Azure Storage Blob instead.

We been working on uploading files to the Azure Storage Blob in a creative way since Azure DevOps won’t get access to the Storage Container. Our StorageContainer is limited by network & Azure DevOps is not on the trusted services list yet.

I’ve been working on a module for the Azure Rest API for some time now. This makes it easier for me to deal with throttling, batching, refreshing the AccessToken and so on.
The module is the easiest to use, but I understand that people don’t always want a third party module on their company servers.

I have written a tutorial for uploading zip files with the module and manually.

I always opt for the minimum use of modules because then you become dependent. If you need one file, I wouldn’t go for the Optimized.Aza module.
Do you have a series of files to upload from an Azure Storage Blob? Then I would look at the module to make sure throttling & RefreshToken won’t be an issue.


The easy way: Optimized.Aza

For more about Optimized.Aza:
PowerShellGallery.com
Github.com


Let’s start uploading?

If you haven’t yet, install Optimized.Aza via:

Install-Module Optimized.Aza -Scope CurrentUser

Logon to the Azure Rest API:

$ConnectAzaSplatting = @{
    ClientSecret =  'XXXXXXX'
    Resource = 'https://XXXXXXX.blob.core.windows.net/'
    ApplicationId = 'XXXXXXXX'
    Tenant = 'XXXXXXX'
}
Connect-Aza @ConnectAzaSplatting

Get the bytes content from your file & create a CustomHeader.
Do not forget to change the path in [System.IO.File]::ReadAllBytes.

$Bytes = [System.IO.File]::ReadAllBytes('C:\temp\temp.zip') 
$CustomHeader = @{
    'x-ms-version' = '2020-10-02'
    'x-ms-blob-type' = 'BlockBlob'
    'x-ms-type' = 'file'
    'Content-Type' = 'application/octet-stream'
}

Now run the cmdlet with the correct Url & filename (in the Url):

$PutAzaSplatting = @{
    CustomHeader = $CustomHeader
    InputObject = $Bytes
    KeepFormat = $true
    Url = 'https://baswijdenes2019.blob.core.windows.net/moduleblob/temp.zip' 
}
Put-Aza @PutAzaSplatting

The file should now be uploaded into the Azure Storage Container. Check if the file has been uploaded correctly.


I’d like to upload files without a third party module

Understandable. There are many companies that would rather not use a third party module because you simply don’t know what it does unless you go through the source code.
The disadvantage of doing this manually is that you will have to build in certain things yourself, such as: automatically refreshing the AccessToken, throttling issues, etcetera.


We need an OauthToken before we can continue

In short: we need an Oauth Token to identify us in header of the request that we invoke to Azure Rest API.

There are several ways to obtain the Access Token, but the most common are the following Identities: User, Service Principal, and Managed Identity.

For more about the Oauth protocol:
OAUTH 2.0 authentication with Azure Active Directory | Microsoft Docs


As an User or Service Principal identity

You can use Get-AzAccessToken to get the OauthToken from your Connect-AzAccount login.

Connect-AzAccount
$TokenCache = (Get-AzAccessToken).Token
$Headers = @{Authorization = "Bearer $TokenCache"}

Unfortunately this often doesn’t work because you’re missing permissions to actually download a file from the blob.

What you can try as well is the below, but you will need a Service Principal for this with the permission set for the Storage Blob endpoint.

$ApplicationID = '3aa17b12-ea2f-4c99-817c-XXXXXXX'
$loginURI = "https://login.microsoft.com"
$resource = 'https://STORAGEACCOUNT.blob.core.windows.net/'
$tenant = 'XXXXXXXX.onmicrosoft.com'
$Username = 'XXXXXXXXXXXXXXXXXXX'
$Password = 'XXXXXXXXXXXXXXXXXXX'

$Body = @{
    grant_type = 'password'
    resource   = $($Resource)
    username   = $Username
    password   = $Password
    client_id  = $ApplicationID
}
$TokenCache = Invoke-RestMethod -Method Post -Uri $loginURI/$Tenant/oauth2/token?api-version=1.0 -Body $Body -UseBasicParsing

The downside of doing it this way is that your OauthToken won’t be refreshed automatically unless you use a Az cmdlet again.

For more about Get-AzAccessToken please check:
Get-AzAccessToken (Az.Accounts) | Microsoft Docs


As a Managed Identity

As a managed Identity you can use a function I created for a Customer of us
You can get the function from my Github: New-managedIdentityAccessToken.ps1

And then run this:

$Headers = New-ManagedIdentityAccessToken -Resource 'https://management.azure.com/'

Now that we’ve got our headers, let’s upload a file from Azure Storage Blob

Before we can start uploading to the Azure storage Container we will need to modify the headers a bit.

$Headers = @{
    Authorization = "Bearer $($TokenCache.access_token)"
    'x-ms-version' = '2020-10-02'
    'x-ms-blob-type' = 'BlockBlob'
    'x-ms-type' = 'file'
    'Content-Type' = 'application/octet-stream'
}

We can now create the Body by Reading the file bytes.
Do not forget to change the path in [System.IO.File]::ReadAllBytes.

$Bytes = [System.IO.File]::ReadAllBytes('C:\temp\temp.zip')

Last but not least we can invoke our request to Azure.
Do not forget to change the Uri StorageAccount & file

$InvokeRestMethodSplatting = @{
    method = 'Put'
    Uri = 'https://XXXXXXXXX.blob.core.windows.net/moduleblob/XXXXX.ext'
    Headers = $Headers
    Body = $Bytes
}
Invoke-RestMethod @InvokeRestMethodSplatting

The file should now be uploaded into the Azure Storage Container. Check if the file has been uploaded correctly.


Where do you get your information from?

I use the official Azure Rest API documentation to compose the URL and headers.

The official docs for getting Blobs:
Put Blob (REST API) – Azure Storage | Microsoft Docs

Published by

Bas Wijdenes

My name is Bas Wijdenes and I work as a PowerShell DevOps Engineer. In my spare time I write about interesting stuff that I encounter during my work.

Leave a Reply

Your email address will not be published.