How to download a file from an Azure Storage Blob PowerShell

Intro

We are setting up a completely new Azure Automation environment for a new customer. This is all done through Azure DevOps CI/CD.

Static files such as an html or zip file are no longer on an on-premise server, but from now on in an Azure Storage Blob.

We have all documented the script parts to download files from an Azure Storage Blob with different encoding types. Now that it’s finally working, I’m happy to share this with you.

I’ve been working on a module for the Azure Rest API for some time now. This makes it easier for me to deal with throttling, batching, refreshing the AccessToken and so on.
The module is the easiest to use, but I understand that people don’t always want a third party module on their company servers.

I have written a tutorial for downloading zip files with the module and manually.

I always opt for the minimum use of modules because then you become dependent. If you need one file, I wouldn’t go for the Optimized.Aza module.
Do you have a series of files to download from an Azure Storage Blob? Then I would look at the module to make sure throttling & RefreshToken won’t be an issue.


The easy way: Optimized.Aza

For more about Optimized.Aza:
PowerShellGallery.com
Github.com


Let’s start downloading?

If you haven’t yet, install Optimized.Aza via:

Install-Module Optimized.Aza -Scope CurrentUser

Logon to the Azure Rest API.
Change the XXXXX’s to the correct value.

$ConnectAzaSplatting = @{
    ClientSecret =  'XXXXXXX'
    Resource = 'https://XXXXXXX.blob.core.windows.net/'
    ApplicationId = 'XXXXXXX'
    Tenant = 'XXXXXXX'
}
Connect-Aza @ConnectAzaSplatting

Create a CustomHeader

$CustomHeader = @{
    'x-ms-version' = '2020-10-02'
    'Accept' = 'Application/octet-stream'
}

Now run the cmdlet with the correct Uri , filename & CustomHeader.
Change temp.zip to whatever your filename is & don’t forget to change the XXXXXX in the URL to the correct StorageAccount.
Keep in mind that all this is case sensitive.

$Result = Get-Aza 'https://XXXXXXXX.blob.core.windows.net/moduleblob/temp.zip' -CustomHeader $CustomHeader -verbose

The results are returned in Bytes when it cannot recognize the Content-Type as for example with zip files.
With Set-Content you can write the Bytes to a file.
Do not forget to change the Set-Content path.

$Result | Set-Content C:\temp\Connect-Aza_download.zip -Encoding Byte

I’d like to download files without a third party module

Understandable. There are many companies that would rather not use a third party module because you simply don’t know what it does unless you go through the source code.
The disadvantage of doing this manually is that you will have to build in certain things yourself, such as: automatically refreshing the AccessToken, throttling issues, etcetera.


We need an OauthToken before we can continue

In short: we need an Oauth Token to identify us in header of the request that we invoke to Azure Rest API.

There are several ways to obtain the Access Token, but the most common are the following Identities: User, Service Principal, and Managed Identity.

For more about the Oauth protocol:
OAUTH 2.0 authentication with Azure Active Directory | Microsoft Docs


As an User or Service Principal identity

You can use Get-AzAccessToken to get the OauthToken from your Connect-AzAccount login.

Connect-AzAccount
$TokenCache = (Get-AzAccessToken).Token
$Headers = @{Authorization = "Bearer $TokenCache"}

Unfortunately this often doesn’t work because you’re missing permissions to actually download a file from the blob.

What you can try as well is the below, but you will need a Service Principal for this with the permission set for the Storage Blob endpoint.

Do not forget to change the:

  • ApplicationId
  • Resource
  • Tenant
  • UserName
  • Password
$ApplicationID = '3aa17b12-ea2f-4c99-817c-XXXXXXX'
$loginURI = "https://login.microsoft.com"
$resource = 'https://STORAGEACCOUNT.blob.core.windows.net/'
$tenant = 'XXXXXXXX.onmicrosoft.com'
$Username = 'XXXXXXXXXXXXXXXXXXX'
$Password = 'XXXXXXXXXXXXXXXXXXX'

$Body = @{
    grant_type = 'password'
    resource   = $($Resource)
    username   = $Username
    password   = $Password
    client_id  = $ApplicationID
}
$TokenCache = Invoke-RestMethod -Method Post -Uri $loginURI/$Tenant/oauth2/token?api-version=1.0 -Body $Body -UseBasicParsing
$Headers = @{Authorization = "Bearer $($TokenCache.access_token)" }
$Headers.add('x-ms-version','2020-10-02')

The downside of doing it this way is that your OauthToken won’t be refreshed automatically unless you use a Az cmdlet again.

For more about Get-AzAccessToken please check:
Get-AzAccessToken (Az.Accounts) | Microsoft Docs


As a Managed Identity

As a managed Identity you can use a function I created for a Customer of us
You can get the function from my Github: New-managedIdentityAccessToken.ps1

And then run this:

$Headers = New-ManagedIdentityAccessToken -Resource 'https://management.azure.com/'
$Headers.add('x-ms-version','2020-10-02')

Now that we’ve got our headers, let’s download a file from Azure Storage Blob

If you’re here for zip files or other encoded files, please see the next header.

I’ve also created a PowerShell function that can automatically download files for you. Github: Get-StaticFiles.ps1

You can retrieve this from Github and edit it according to your own wishes.

See the below as an example.
The Filename is case sensitive & do not forget to change the XXXXXXX’s.

$Content = Get-StaticFiles -headers $Headers -Storage 'XXXXXXX' -Container 'XXXXXXX' -filename 'Filename'

You can now use Out-File to create a file.
Do not forget to change the Out-File path.

$Content | Out-File C:\temp\FILENAME.extension

For zip files or any other encoded file it’s a bit different….

This does not work for zip files because they’re encoded differently. The contentType for zip files is application/x-zip-compressed. This means that it is encoded differently than a normal text file.

I also created a function specifically for zip files. You can get it from Github and edit according to your own wishes. Github: Get-AzureBlobZipFiles.ps1

Add this to your header:

$headers.add('Accept','Application/octet-stream')

We have to add this, so that Azure understands in which format we need the content.

There are two options you can use in this function, one is downloading the zip files and the other is unpacking the zip file.

Download the zip files.
Don’t forget the XXXXXXX’s, BaseDir & FileName

Get-AzureBlobZipFiles -headers $Headers -Storage 'XXXXXXX' -Container 'XXXXXXX' -BaseDir 'C:\temp' -FileName 'temp.zip'

Download and unpack the zip file.
Don’t forget the XXXXXXX’s, BaseDir & FileName

Get-AzureBlobZipFiles -headers $Headers -Storage 'XXXXX' -Container 'XXXXX' -BaseDir 'C:\temp' -FileName 'temp.zip' -Unpack

Where do you get your information from?

I use the official Azure Rest API documentation to compose the URL and headers.

The official docs for getting Blobs:
Get Blob (REST API) – Azure Storage | Microsoft Docs

Published by

Bas Wijdenes

My name is Bas Wijdenes and I work as a PowerShell DevOps Engineer. In my spare time I write about interesting stuff that I encounter during my work.

Leave a Reply

Your email address will not be published.