On Thursday December 3, 2015 I spoke to an engaged crowed at Andy Novick’s New England Microsoft Developers’ Group. The topic was a high-level look at the Microsoft Azure cloud platform with the perspective of a toolbox for developers.
Here is the deck used: 2015-12-03-microsoft-azure-cloud-platform-toolbox-overview-nemicrosoftdevelopers
At this past Monday’s Boston Azure meeting, I spoke about using Azure Automation Runbooks to automate your automation. I have a short overview to define the need, then mostly gave demos.
Here is the deck:
Add-AzureAccount : The data is invalid.
At line:1 char:1
+ Add-AzureAccount + ~~~~~~~~~~~~~~~~ + CategoryInfo : CloseError: (:) [Add-AzureAccount], AadAuthenticationFailedException + FullyQualifiedErrorId : Microsoft.WindowsAzure.Commands.Profile.AddAzureAccount
PS> (Get-Module Azure).Version Major Minor Build Revision ----- ----- ----- -------- 0 8 11 -1
-Debug and -Verbose command line options, which are often useful. But no difference in the output. So this was failing pretty early!gci "$env:APPDATA\Windows Azure Powershell\TokenCache.dat" ri "$env:APPDATA\Windows Azure Powershell\TokenCache.dat"
RunDll32.exe InetCpl.cpl,ClearMyTracksByProcess 2
A couple of nights ago, I gave a guest lecture at Dino Konstantopoulos’ BU MET CS755 Cloud Computing class to a small group whose ability to stay awake and alert until 9:00 PM was impressive. My deck is attached.
For any of that class (or anyone else reading this), if interested in more Azure goodness, check out Boston Azure – www.bostonazure.org – a local user group that has now been meeting regularly (mostly at NERD) for around 4 1/2 years to learn about Azure, the cloud, architecting applications for the cloud, and more.
My book is available on Amazon: Cloud Architecture Patterns.
Slides from the talk: 2014-04-17 – April 17 – Building Cloud-Native Applications – Bill Wilder (blog.codingoutloud.com) – BU MET CS755
If you are trying to Embrace SSL During Development when authenticating with Azure Active Directory, you may run into a little glitch if you do so on one of those handy MSDN Dev/Test VMs in Azure.
The glitch is that when running SSL on the MSDN VM the digital certificate for the SSL cert isn’t quite right. Here is a description of what you might see, followed by a workaround (until fixed at the source in the VM image).
Visual Studio 2013 uses IIS Express by default and offers a very simple experience for HTTPS locally:
Using MSDN Visual Studio Azure VMs, this developer experience does not quite work out of the box. SSL Enabled is true automatically when creating an ASP.NET app that uses Azure Active Directory for org authentication. If you create a new web app, then simply click Change Authentication and select Organizational Accounts, set one up, and then proceed as normal, then hit F5. When your app runs, it will try to authenticate over HTTPS, and it fails as in the scenario above if running on one of these MSDN Visual Studio Azure VMs.
Follow these steps:
The above code will work in the default state of these VMs at this time which assumes only a single certificate with Subject of ‘CN=localhost’ is present in the certificate store.
[This is part of a series of posts on #StupidAzureTricks, explained here.]
A professional reality is looming: fear of obsolescence. Like so many of you, I worry about becoming obsolete (I work in cloud software which moves VERY FAST and getting faster), so I figured it was time to do something for a small (and shrinking) set of people supporting my profession. For these people, their job has groan increasingly difficult: I refer, of course, to the fine people who work inside of cloud data centers. The job is so difficult primarily because it is boring.
This is not because cloud data centers are not advancing – in fact that is the root of the problem – there are so few data center workers because of the growing efficiencies in cloud data centers simply means fewer people are needed to run them.
You may be wondering how you can help. It is very simple – remotely connect into a server on the data center of your choice, and just talk to the nice people in that data center through the speaker on the computer your’ve remoted into. But first you’ll need to enable the audio service on the operating system, since it is off by default in the cloud. But turning it on is fairly simple. This short video shows you how.
What will YOU say to the lonely cloud data center people?
Today I spoke at VirtG Boston’s annual Deep Dive Day. The title of my talk, Meet Windows Azure, Your Next Data Center, is probably descriptive enough to get the gist of it.
My slide deck follows.
2014-03-12 – Meet Windows Azure, Your Next Data Center – VirtG Virtualization Deep Dive Day
[Ugh – editing 2nd week in a row after accidental early publishing.]
The original programming model for Windows Azure applications was to use Cloud Services (originally known as Hosted Services, but still the same thing). Of particular note, Cloud Services run on VMs with disks that are not-persistent – you can write data locally (some pointers here), but any locally stored data is not guaranteed to stick around. This is a powerful model for some scenarios, especially highly scalable applications. Another feature of Cloud Services has always been that it comes with an emulator you can run locally – on your laptop at 30,000 feet was a common way to hammer home the point. (Remember, Cloud Services were announced in 2008 – a long time before we had wifi on airplanes!) There are actually two emulators: Compute – which emulates the Cloud Service model by supporting Web Role and Worker Role abstractions, and Storage – which emulates Blob, Table, and [Storage] Queue Services. The rest of this post will focus specifically on the Storage Emulator.
Since their announcement in 2012, Windows Azure Web Sites and Virtual Machines have been taking on many of the common workloads that used to require Cloud Services. This diagram at least conceptually should capture the sense that the when to use which model decision has become blurred over time. This is good – with more choice comes the freedom to get started more simply – often a Virtual Machine is an easier onramp for existing apps, and a Web Site can be a great onramp for a website that adheres to some of the well-known programming stacks running on PHP, ASP.NET, Python, or Node.js. If you are a big success, consider upgrading to Cloud Services.
Notably absent from the diagram is the Storage Emulator. It should be in the middle of the diagram because while the local storage emulator is still useful for Cloud Services, you can also use it locally when developing applications targeting Windows Azure Web Sites or Virtual Machines.
This is awesome – of course, it will be popular to create applications destined for Windows Azure Web Sites or Virtual Machines that take advantage of the various Storage Services.
So that’s the trick – be sure to take advantage of the Storage Emulator, even when you are not targeting a Cloud Service. You need to know two things: how to turn it on, and how to address it.
If you create a regular old Web Site and run that in Visual Studio, the Storage Emulator is not turned on. Visual Studio only turns on the Storage Emulator for you when you debug using a Cloud Service, but this is not convenient.
It is easy to turn on. I have a whole post that explains how to start the storage emulator from a shortcut, but the keys are:
Here’s PowerShell script that will turn it on:
| & 'C:\Program Files\Microsoft SDKs\Windows Azure\Emulator\csrun.exe' /devstore:start |
The other part is knowing how to set up your Storage Connection String so that it accesses local storage emulator instead of the cloud.
Here are the values to use to make it look like any other Storage Account, while still addressing local emulated storage rather than in the cloud:
Emulator Storage Account Name: devstoreaccount1 Emulator Storage Account Key: Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
The latest version of the Windows Azure Storage Emulator (v2.2.1) is in Preview. This release has support for “2013-08-15” version of Storage which adds CORS and JSON and still has all those features from years gone by…
A comparison of emulated and cloud storage services is also available. There are some differences.
[This is part of a series of posts on #StupidAzureTricks, explained here.]
Two more Windows Azure data center regions have moved into production – in Japan this time.
There was a press release, but in Japanese. You can either learn Japanese or read the Bing Translator or Google Translate versions.
I updated the JSON meta data in the Azure Map project to indicate these data center regions into the “Production” mode, then re-ran my script to regenerate and upload the GeoJSON and TopoJSON maps. All data is in GitHub. For more info, see these two posts:
[Edit: I originally accidentally published an old draft. The draft went out to all email subscribers and was public for around 90 minutes. Fixed now.]
In the most recent Stupid Azure Trick installment, I explained how one could host a 1000 visitor-per-day web site for one penny per month. Since then I also explained my choice to use CORS in that same application. Here I will dig into specifically using CORS with Windows Azure.
I also show how the curl command line tool can be helpful to examine CORS properties in HTTP headers for a blob service.
I also will briefly describe a simple tool I built that could quickly turn CORS on or off for a specified Blob service – the CORS Toggler. The CORS Toggler (in its current simple form) was useful to me because of two constraints that were true for my scenario:
These two simplifications mean that the toggler knew what it meant to enable CORS (open up for reading to all comers) and to disable. (Though it is worth noting that opening up CORS to any caller is probably a common scenario. Also worth noting that tool could easily extended to support a whitelist for allowed domains or other features.)
First, here’s the code for the toggler – there are three files here:
| @echo off | |
| D:\dev\path\to\ToggleCors.exe azuremap 123abcYourStorageKeyIsHere987zyx== %1 | |
| echo. | |
| echo FOR COMPARISON QUERY THE nocors SERVICE (which never has CORS set) | |
| echo. | |
| D:\dev\path\to\ToggleCors.exe nocors 123abcYourStorageKeyIsHere987zyx== -q | |
| echo. | |
| PowerShell -Command Get-Date |
| using System; | |
| using Microsoft.WindowsAzure.Storage.Auth; | |
| using Microsoft.WindowsAzure.Storage.Blob; | |
| using Microsoft.WindowsAzure.Storage.Shared.Protocol; | |
| using Newtonsoft.Json; | |
| namespace BlobCorsToggle | |
| { | |
| public class SimpleAzureBlobCorsSetter | |
| { | |
| public CloudBlobClient CloudBlobClient { get; private set; } | |
| public SimpleAzureBlobCorsSetter(Uri blobServiceUri, StorageCredentials storageCredentials) | |
| { | |
| this.CloudBlobClient = new CloudBlobClient(blobServiceUri, storageCredentials); | |
| } | |
| public SimpleAzureBlobCorsSetter(CloudBlobClient blobClient) | |
| { | |
| this.CloudBlobClient = blobClient; | |
| } | |
| /// <summary> | |
| /// Set Blob Service CORS settings for specified Windows Azure Storage Account. | |
| /// Either sets to a hard-coded set of values (see below) or clears of all CORS settings. | |
| /// | |
| /// Does not check for any existing CORS settings, but clobbers with the CORS settings | |
| /// to allow HTTP GET access from any origin. Non-CORS settings are left intact. | |
| /// | |
| /// Most useful for scenarios where a file is published in Blob Storage for read-access | |
| /// by any client. | |
| /// | |
| /// Can also be useful in conjunction with Valet Key Pattern-style limited access, as | |
| /// might be useful with a mobile application. | |
| /// </summary> | |
| /// <param name="clear">if true, clears all CORS setting, else allows GET from any origin</param> | |
| public void SetBlobCorsGetAnyOrigin(bool clear) | |
| { | |
| // http://msdn.microsoft.com/en-us/library/windowsazure/dn535601.aspx | |
| var corsGetAnyOriginRule = new CorsRule(); | |
| corsGetAnyOriginRule.AllowedOrigins.Add("*"); // allow access to any client | |
| corsGetAnyOriginRule.AllowedMethods = CorsHttpMethods.Get; // only CORS-enable http GET | |
| corsGetAnyOriginRule.ExposedHeaders.Add("*"); // let client see any header we've configured | |
| corsGetAnyOriginRule.AllowedHeaders.Add("*"); // let clients request any header they can think of | |
| corsGetAnyOriginRule.MaxAgeInSeconds = (int)TimeSpan.FromHours(10).TotalSeconds; // clients are safe to cache CORS config for up to this long | |
| var blobServiceProperties = this.CloudBlobClient.GetServiceProperties(); | |
| if (clear) | |
| { | |
| blobServiceProperties.Cors.CorsRules.Clear(); | |
| } | |
| else | |
| { | |
| blobServiceProperties.Cors.CorsRules.Clear(); // replace current property set | |
| blobServiceProperties.Cors.CorsRules.Add(corsGetAnyOriginRule); | |
| } | |
| this.CloudBlobClient.SetServiceProperties(blobServiceProperties); | |
| } | |
| public void DumpCurrentProperties() | |
| { | |
| var blobServiceProperties = this.CloudBlobClient.GetServiceProperties(); | |
| var blobPropertiesStringified = StringifyProperties(blobServiceProperties); | |
| Console.WriteLine("Current Properties:\n{0}", blobPropertiesStringified); | |
| } | |
| internal string StringifyProperties(ServiceProperties serviceProperties) | |
| { | |
| // JsonConvert.SerializeObject(serviceProperties) for whole object graph or | |
| // JsonConvert.SerializeObject(serviceProperties.Cors) for just CORS | |
| return Newtonsoft.Json.JsonConvert.SerializeObject(serviceProperties, Formatting.Indented); | |
| } | |
| } | |
| } | |
| using System; | |
| using System.Diagnostics; | |
| using System.IO; | |
| using Microsoft.WindowsAzure.Storage.Auth; | |
| namespace BlobCorsToggle | |
| { | |
| class Program | |
| { | |
| static string clearFlag = "-clear"; | |
| static string queryFlag = "-q"; | |
| static void Main(string[] args) | |
| { | |
| if (args.Length == 0 || !ValidCommandArguments(args)) | |
| { | |
| #region Show Correct Usage | |
| if (args.Length != 0 && !ValidCommandArguments(args)) | |
| { | |
| var saveForegroundColor = Console.ForegroundColor; | |
| Console.ForegroundColor = ConsoleColor.Red; | |
| Console.Write("\nINVALID COMMAND LINE OR UNKNOWN FLAGS: "); | |
| foreach (var arg in args) Console.Write("{0} ", arg); | |
| Console.WriteLine(); | |
| Console.ForegroundColor = saveForegroundColor; | |
| } | |
| Console.WriteLine("usage:\n{0} <storage acct name> <storage acct key> [{1}]", | |
| Path.GetFileNameWithoutExtension(Environment.GetCommandLineArgs()[0]), | |
| clearFlag); | |
| Console.WriteLine(); | |
| Console.WriteLine("example setting:\n{0} {1} {2}", | |
| Path.GetFileNameWithoutExtension(Environment.GetCommandLineArgs()[0]), | |
| "mystorageacct", | |
| "lala+aou812SomERAndOmStrINglOlLolFroMmYStORaG3akounT314159265358979323ilIkEpi=="); | |
| Console.WriteLine(); | |
| Console.WriteLine("example clearing:\n{0} {1} {2} {3}", | |
| Path.GetFileNameWithoutExtension(Environment.GetCommandLineArgs()[0]), | |
| "mystorageacct", | |
| "lala+aou812SomERAndOmStrINglOlLolFroMmYStORaG3akounT314159265358979323ilIkEpi==", | |
| clearFlag); | |
| if (Debugger.IsAttached) | |
| { | |
| Console.WriteLine("\nPress any key to exit."); | |
| Console.ReadKey(); | |
| } | |
| #endregion | |
| } | |
| else | |
| { | |
| var storageAccountName = args[0]; | |
| var storageKey = args[1]; | |
| var blobServiceUri = new Uri(String.Format("https://{0}.blob.core.windows.net", storageAccountName)); | |
| var storageCredentials = new StorageCredentials(storageAccountName, storageKey); | |
| var blobConfig = new SimpleAzureBlobCorsSetter(blobServiceUri, storageCredentials); | |
| bool query = (args.Length == 3 && args[2] == queryFlag); | |
| if (query) | |
| { | |
| blobConfig.DumpCurrentProperties(); | |
| return; | |
| } | |
| blobConfig.DumpCurrentProperties(); | |
| Console.WriteLine(); | |
| bool clear = (args.Length == 3 && args[2] == clearFlag); | |
| blobConfig.SetBlobCorsGetAnyOrigin(clear); | |
| Console.WriteLine("CORS Blob Properties for Storage Account {0} have been {1}.", | |
| storageAccountName, clear ? "cleared" : "set"); | |
| Console.WriteLine(); | |
| blobConfig.DumpCurrentProperties(); | |
| } | |
| } | |
| private static bool ValidCommandArguments(string[] args) | |
| { | |
| if (args.Length == 2) return true; | |
| if (args.Length == 3 && (args[2] == clearFlag || args[2] == queryFlag)) return true; | |
| return false; | |
| } | |
| } | |
| } |
One simple point to highlight – CORS properties are simply available on the Blob service object (and would be same for Table or Queue service within Storage):
Yes, this is a very simple API.
For those interested in the contents of these objects, here are a few ways to show content of properties (in code) before turning on CORS and after. (The object views are created using the technique I described my post on using JSON.NET as an object dumper that’s Good Enough™.)
DUMPING OBJECT BEFORE CORS ENABLED (just CORS properties):
{“Logging”:{“Version”:”1.0″,”LoggingOperations”:0,”RetentionDays”:null},”Metrics”:{“Version”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”HourMetrics”:{“Version”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”Cors”:{“CorsRules”:[]},”MinuteMetrics”:{“Version”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”DefaultServiceVersion”:null}
DUMPING OBJECT AFTER CORS ENABLED:
{“Logging”:{“Version”:”1.0″,”LoggingOperations”:0,”RetentionDays”:null},”Metrics”:{“Version”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”HourMetrics”:{“Version”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”Cors”:{“CorsRules”:[{“AllowedOrigins”:[“*”],”ExposedHeaders”:[“*”],”AllowedHeaders”:[“*”],”AllowedMethods”:1,”MaxAgeInSeconds”:36000}]},”MinuteMetrics”:{“Version”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”DefaultServiceVersion”:null}
DUMPING OBJECT BEFORE CORS ENABLED (but including ALL properties):
Current Properties:
{“Logging”:{“Version”:”1.0″,”LoggingOperations”:0,”RetentionDays”:null},”Metrics
“:{“Version”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”HourMetrics”:{“Versio
n”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”Cors”:{“CorsRules”:[]},”MinuteM
etrics”:{“Version”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”DefaultServiceV
ersion”:null}
DUMPING OBJECT AFTER CORS ENABLED (but including ALL properties):
Current Properties:
{“Logging”:{“Version”:”1.0″,”LoggingOperations”:0,”RetentionDays”:null},”Metrics
“:{“Version”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”HourMetrics”:{“Versio
n”:”1.0″,”MetricsLevel”:0,”RetentionDays”:null},”Cors”:{“CorsRules”:[{“AllowedOr
igins”:[“*”],”ExposedHeaders”:[“*”],”AllowedHeaders”:[“*”],”AllowedMethods”:1,”M
axAgeInSeconds”:36000}]},”MinuteMetrics”:{“Version”:”1.0″,”MetricsLevel”:0,”Rete
ntionDays”:null},”DefaultServiceVersion”:null}
| curl -H "Origin: http://example.com" -H "Access-Control-Request-Method: GET" -H "Access-Control-Request-Headers: X-Requested-With" -X OPTIONS --verbose http://azuremap.blob.core.windows.net/maps/azuremap.geojson |
CURL OUTPUT BEFORE CORS ENABLED:
D:\dev\github>curl -H “Origin: http://example.com” -H “Access-Control-Request-Method: GET” -H “Access-Control-Request-Headers: X-Requested-With” -X OPTIONS –verbose http://azuremap.blob.core.windows.net/maps/azuremap.geojson
* Adding handle: conn: 0x805fa8
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* – Conn 0 (0x805fa8) send_pipe: 1, recv_pipe: 0
* About to connect() to azuremap.blob.core.windows.net port 80 (#0)
* Trying 168.62.32.206…
* Connected to azuremap.blob.core.windows.net (168.62.32.206) port 80 (#0)
> OPTIONS /maps/azuremap.geojson HTTP/1.1
> User-Agent: curl/7.31.0
> Host: azuremap.blob.core.windows.net
> Accept: */*
> Origin: http://example.com
> Access-Control-Request-Method: GET
> Access-Control-Request-Headers: X-Requested-With
>
< HTTP/1.1 403 CORS not enabled or no matching rule found for this request.
< Content-Length: 316
< Content-Type: application/xml
* Server Blob Service Version 1.0 Microsoft-HTTPAPI/2.0 is not blacklisted
< Server: Blob Service Version 1.0 Microsoft-HTTPAPI/2.0
< x-ms-request-id: 04402242-d4a7-4d0c-bedc-ff553a1bc982
< Date: Sun, 26 Jan 2014 15:08:11 GMT
<
<?xml version=”1.0″ encoding=”utf-8″?><Error><Code>CorsPreflightFailure</Code><Message>CORS not enabled or no matching rule found for this request.
RequestId:04402242-d4a7-4d0c-bedc-ff553a1bc982
Time:2014-01-26T15:08:12.0193649Z</Message><MessageDetails>No CORS rules matches this request</MessageDetails></Error>*
Connection #0 to host azuremap.blob.core.windows.net left intact
CURL OUTPUT AFTER CORS ENABLED:
D:\dev\github>curl -H “Origin: http://example.com” -H “Access-Control-Request-Method: GET” -H “Access-Control-Request-Headers: X-Requested-With” -X OPTIONS –verbose http://azuremap.blob.core.windows.net/maps/azuremap.geojson
* Adding handle: conn: 0x1f55fa8
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* – Conn 0 (0x1f55fa8) send_pipe: 1, recv_pipe: 0
* About to connect() to azuremap.blob.core.windows.net port 80 (#0)
* Trying 168.62.32.206…
* Connected to azuremap.blob.core.windows.net (168.62.32.206) port 80 (#0)
> OPTIONS /maps/azuremap.geojson HTTP/1.1
> User-Agent: curl/7.31.0
> Host: azuremap.blob.core.windows.net
> Accept: */*
> Origin: http://example.com
> Access-Control-Request-Method: GET
> Access-Control-Request-Headers: X-Requested-With
>
< HTTP/1.1 200 OK
< Transfer-Encoding: chunked
* Server Blob Service Version 1.0 Microsoft-HTTPAPI/2.0 is not blacklisted
< Server: Blob Service Version 1.0 Microsoft-HTTPAPI/2.0
< x-ms-request-id: d4df8953-f8ae-441b-89fe-b69232579aa4
< Access-Control-Allow-Origin: http://example.com
< Access-Control-Allow-Methods: GET
< Access-Control-Allow-Headers: X-Requested-With
< Access-Control-Max-Age: 36000
< Access-Control-Allow-Credentials: true
< Date: Sun, 26 Jan 2014 16:02:25 GMT
<
* Connection #0 to host azuremap.blob.core.windows.net left intact
A new version of the Windows Azure Storage Emulator (v2.2.1) is now in Preview. This release has support for “2013-08-15” version of Storage which includes CORS (and JSON and other) support.
Overall description of Azure Storage’s CORS Support:
http://msdn.microsoft.com/en-us/library/windowsazure/dn535601.aspx
REST API doc (usually the canonical doc for any feature, though in code it is easily accessed with the Windows Azure SDK for .NET)
http://msdn.microsoft.com/en-us/library/windowsazure/hh452235.aspx
A couple of excellent posts from the community on CORS support in Windows Azure Storage:
[This is part of a series of posts on #StupidAzureTricks, explained here.]
Coding Out Loud 