Assuming you are using Visual Studio and .NET 4.5, the simple solution is to add the following to one of your Visual Studio Watch windows:

System.Threading.Thread.CurrentPrincipal

[If you happen to be debugging ASP.NET code, you could save a little typing and instead add User to your Watch window. User should have the same value as the CurrentPrincipal in the context of ASP.NET. For ASP.NET WebForms User is a property of the Page class (Page.User), while for ASP.NET MVC User is a property of both the Controller class (Controller.User) and the HttpContext class (HttpContext.User).]

Drill in, and you will see something like the following:

If you then right-click on the Results View entry under Claims (the one that says “Expanding the Results View will enumerate the IEnumerable”) and, uhh, click on that entry to expand the results view, you will see all the claims.

In my case, some claims were flowing through Windows Azure Access Control Service (ACS), and these list the ACS namespace as the Issuer. Other claims were added at runtime by my code using a ClaimsAuthenticationManager module, and these list LOCAL AUTHORITY as the Issuer.

Alternatively, you can add the more complex direct expression to your Watch window – using the cast to coerce the right values:

((System.Security.Claims.ClaimsPrincipal)(System.Threading.Thread.CurrentPrincipal))

This will also do the job – with a little less drilling.

This was two easy steps:

1. Create a simple program to exercise the code I cared about. This was a Console app I called TestRealmLib.exe.

2. Figure out everything that TestRealmLib.exe really depended upon.

Step one was easy since I knew the functionality I wanted to exercise.

Step two was easy once it dawned on me to use the right tool for the job: Visual Studio Ultimate 2012’s Generate Dependency Graph tool found under the ARCHITECTURE menu option.

NOTE: My understanding is that this feature is only fully available with Visual Studio Ultimate 2012, partially available in Premium (view but not create), and not available in lower editions – compare what’s included in which Visual Studio editions here: http://www.microsoft.com/visualstudio/eng/products/compare. I only have personally used it with Visual Studio Ultimate 2012 on solutions with C# projects (though I understand C++ is also supported, and assume any .NET language would work as well).

Running this for the whole solution, and rearranging the nodes with my mouse, quickly let me see where all the dependencies were.

I was quickly able to see that the modules I dragged to the right-hand side of the graph were not necessary, and those few on the left were the key. That was the answer I needed, but there are other interesting features. For example…

The thicker dependency lines represent more dependencies than the thinner lines. To drill in, right-click on a dependency line. Here’s what you get when you choose Advanced > Show Contributing Links on New Code Map:

Also note the “red dot” in the Code Map – this is indicating that I currently have a breakpoint set in there – yes, this is debugger-aware.

There are a bunch of other nifty features with this that you can check out here. These include dependency analysis, “Show Contributing Links” (on the dependency link to show which parts of module you are calling), and more. The UI gestures you’d expect all do something reasonable – like double-click, right-click, whether on nodes or lines. You can double-click a class or method name to jump to your C# code. And don’t forget to turn on the Legend (from toolbar at top of graph, not shown in my screen grabs) and explore the other features.

There is also a nice video demonstration in Channel 9: http://channel9.msdn.com/Series/Visual-Studio-2012-Premium-and-Ultimate-Overview/Visual-Studio-Ultimate-2012-Understand-your-code-dependencies-through-visualization

We agreed I would come back after the summer for an architecture-focused session; the session presented was more feature & technology-oriented.

A few followups:

• Get started with Windows Azure info and downloads from here: http://www.windowsazure.com
• Get a 30 Day FREE TRIAL by starting here: http://aka.ms/iaas
• You may also ALREADY HAVE pre-paid Windows Azure services if you have MSDN: http://www.windowsazure.com/en-us/pricing/member-offers/msdn-benefits/
• Learn more about the hosted Team Foundation Service (TFS) here: http://www.microsoft.com/visualstudio/eng/team-foundation-service
• Get familiar with compliance status through the Windows Azure Trust Center: http://www.windowsazure.com/en-us/support/trust-center/
• The Boston Azure Cloud User Group can be found at bostonazure.org

The deck I used is pretty short, but here in case you are interested:

• What’s new in Windows Azure — New England Microsoft Dev — 06-June-2013 — Bill Wilder (blog.codingoutloud.com)

My book, if you are interested, is described here (note: my next talk to the group will cover material more closely associated with the book, which is more focused on patterns and architecture in the context of designing effective cloud applications).

A few followups:

• Azure VM pricing: the $0.013/hour pricing mentioned for Extra Small instances of the Infrastructure as a Service (IaaS) Virtual Machine is shown here to be a promotional price, with the regular price of $0.02/hour (two cents per hour) kicking in on June 1. The architectures we spoke of in the talk used Platform as a Service (PaaS) Virtual Machines and the pricing for those is very similar, though slightly lower, and is shown here.
• How many customers does Azure have: here is the 10,000 number that Udai shared, which is from was about three years ago when most of the tech world had not yet even heard of Azure. More recently,  it was mentioned there are 200,000 Azure customers and it has passed $1 billion in revenue. So, according to those numbers, it appears to have grown 20x in a little less than three years. Additional interesting numbers mentioned here and here.
• We focused on use of Cloud Services last night, but we also mentioned Virtual Machines (part of what Microsoft is calling Infrastructure Services, like IaaS) and Web Sites, noting all use different approaches. You can read more about all of them here where you’ll see write-ups for each specific area.
• I mentioned that Blob Storage is also being used to support the persistent disks on the Infrastructure Services Virtual Machines, in part-enabled by new high performance network architecture. I wrote about some of this before in a blog post titled Azure Cloud Storage Improvements Hit the Target.

The deck I used follows.

Architecting for the Cloud — NH Azure — 15-Mar-2013 — Bill Wilder (blog.codingoutloud.com)

My book, if you are interested, is described here. And the Boston Azure Cloud User Group can be found here.

While I will not attempt to reproduce the discussion (sorry!, though there are a few photos), I do want to follow up on a few questions that I offered to provide references on. If you have further questions, please feel free to put a comment on this post. Also, at the end of this post, you will find a link to the short “Azure in 3 minutes or less” deck we used to introduce the Windows Azure Cloud Platform at the very beginning (per the ground rules of the panel – we limited the intro to 3 minutes).

• In response to the question about scalability of Windows Azure Blobs, here is the write-up I referenced on Windows Azure Storage Scalability Targets. Here is an additional (more comparative) discussion (follow links) you may find helpful: Azure Cloud Storage Improvements Hit the Target.
• In response to the question about pricing, check out the Windows Azure pricing calculator. Note that for the Microsoft Server products (e.g. Windows Server, or SQL Server on Windows Azure SQL Database (offered as a service) or on a Virtual Machine (that you manage)), the cost of the license is baked into the hourly rental cost.
• In response to the question about the ability to support different types of apps (whether new ones from startups, existing ones from big company, etc.), see the spectrum of offerings described here: https://www.windowsazure.com/en-us/develop/net/fundamentals/compute/. In a nutshell, Web Sites is for hosting (with a free Tier) for basic, low-scale sites, but these can scale very nicely too), Cloud Services is for building Cloud-Native applications using PaaS (which my book focuses on), Virtual Machines (parallel to what OpenStack offers in terms of managed VMs) is more useful for applications you want to run in the cloud with minimal change, and Virtual Networking allows many options for connecting your data center with a secure private network on Windows Azure among other options.
• In response to the question about openness, any programming language or platform can access the Windows Azure services through REST APIs, but here is the list of those with first-class SDKs: http://www.windowsazure.com/en-us/downloads/
• For any further follow-up questions feel free leave a COMMENT below and I will update this post.

Windows Azure is not the only full-service, rock-solid cloud platform out there, but I hope you got an appreciation for how it might help you and why you might wish to choose it for your applications and services. If you are interested in learning more about Windows Azure, you may wish to check out the Boston Azure User Group, which has been meeting regularly at NERD since October 2009. Our next meeting is in just a few days: Tuesday May 9.

The SLIDE DECK we used for the 3 minute intro is here:

• Clash-of-the-Clouds–Tech-in-Motion–Mark Eisenberg and Bill Wilder–Windows Azure in 3 Minutes or Less
• The eye-chart slide containing the Windows Azure Feature Map was stolen borrowed from here

The theme of my talk was Architecting to be Cloud Native – On Windows Azure or Otherwise. The slide deck I used is included below.

• Building Cloud-Native Applications – 21-Mar-2013 – Bill Wilder (blog.codingoutloud.com) – BU MET CS755
• Much of the content draws from my Cloud Architecture Patterns book.

Night class is tough. Thanks for a warm reception – so congratulations and many thanks to those of you able to stay awake until 9:00 PM (!).

I hope to see all of you at future Boston Azure events – to get announcements, Join our Meetup Group. We are also the world’s first/oldest Azure User Group. Here are a couple of upcoming events:

• Wednesday March 27 Boston Azure meeting with two talks: (a) Managing Applications in the Cloud, featuring an APM expert, and (b) expecting shortly to add an opening speaker by an Azure team member in from Redmond.
• Saturday April 27 Boston Azure is participating in a Global Windows Azure Bootcamp – an all-day, hands-on session where you get practical, hands-on-the-keyboard experience working with Windows Azure, combined with explanatory talks. Food is provided so you can stay focused on Azure!

Feel free to reach out with any questions (twitter (@codingoutloud) or  email (codingoutloud at gmail)) — especially if it will be “on the midterm” – and good luck in the cloud!

Bill Wilder

• Azure Best Practices – How to Successfully Architect Windows Azure Apps for the Cloud @ 1pm ET on 13-March-2013
• VIEW RECORDING HERE: http://bit.ly/ZzQDDW 

Abstract:

Discover how you can successfully architect Windows Azure-based applications to avoid and mitigate performance and reliability issues with our live webinar
Microsoft’s Windows Azure cloud offerings provide you with the ability to build and deliver a powerful cloud-based application in a fraction of the time and cost of traditional on-premise approaches.  So what’s the problem? Tried-and-true traditional architectural concepts don’t apply when it comes to cloud-native applications. Building cloud-based applications must factor in answers to such questions as:

• How to scale?
• How to overcome failure?
• How to build a manageable system?
• How to minimize monthly bills from cloud vendors?

During this webinar, we will examine why cloud-based applications must be architected differently from that of traditional applications, and break down key architectural patterns that truly unlock cloud benefits. Items of discussion include:

• Architecting for success in the cloud
• Getting the right architecture and scalability
• Auto-scaling in Azure and other cloud architecture patterns

If you want to avoid long nights, help-desk calls, frustrated business owners and end-users, then don’t miss this webinar or your chance to learn how to deliver highly-scalable, high-performance cloud applications.

Deck:

•  Azure Best Practices – How to Successfully Architect Windows-Azure based Apps for the Cloud – 13-Mar-2013 — Bill Wilder (blog.codingoutloud.com) — DELL WEBINAR

Book:

The core ideas were drawn from my Cloud Architecture Patterns (O’Reilly Media, 2012) book:

Hosted by Dell:

Brad Calder delivering SOSP talk

Since its initial release, Windows Azure has offered a storage service known as Windows Azure Storage (WAS). According to the SOSP paper and related talk published by the team (led by Brad Calder), WAS is architected to be a “Highly Available Cloud Storage Service with Strong Consistency.” Part of being highly availably is keeping your data safe and accessible. The SOSP paper mentions that the WAS service retains three copies of every stored byte, and (announced a few months before the SOSP paper) another asynchronously geo-replicated trio of copies in another data center hundreds of miles away in the same geo-political region. Six copies in total.

WAS is a broad service, offering not only blob (file) storage, but also a NoSQL store and a reliable queue.

Further, all of these WAS storage offerings are strongly consistent (as opposed to other storage approaches which are sometimes eventually consistent). Again citing the SOSP paper: “Many customers want strong consistency: especially enterprise customers moving their line of business applications to the cloud.” This is because traditional data stores are strongly consistent and code needs to be specially crafted in order to handle an eventually consistent model. This simplifies moving existing code into the cloud.

The points made so far are just to establish some basic properties of this system before jumping into the real purpose of this article: performance at scale. The particular points mentioned (highly available, storage in triplicate and then geo-replicated, strong consistency, and supporting also a NoSQL database and reliable queuing features) were highlighted since they may be considered disadvantages – rich capabilities that may be considered to hamper scalability and performance. Except that they don’t hamper scalability and performance at all. Read on for details.

Performance at Scale

A couple of years ago, Nasuni benchmarked the most important public cloud vendors on how their services performed on cloud file storage at scale (using workloads modeled after those observed from real world business scenarios). Among the public clouds tested were Windows Azure Storage (though only the blob/file storage aspect was considered), Amazon S3 (an eventually consistent file store), and a couple of others.

In the first published result in 2011, Nasuni declared Amazon S3 the overall winner, prevailing over Windows Azure Storage and others, though WAS fininshed ahead of Amazon in some of the tests. At the time of these tests, WAS was running on its first-generation network architecture and supported capacity as described in the team’s published scalability targets from mid-2010.

In 2012, Microsoft network engineers were busy implementing a new data center network design they are calling Quantum 10 (or Q10 for short). The original network design was hierarchical, but the Q10 design is flat (and uses other improvements like SSD for journaling). The end result of this dramatic redesign is that WAS-based network storage is much faster, more scalable, and as robust as ever. The corresponding Q10 scalability targets were published in November 2012 and show substantial advances.

Q10 was implemented during 2012 and apparently was in place before Nasuni ran its updated benchmarks between November 2012 and January 2013. With its fancy new network design in place, WAS really shined. While the results in 2011 were close, with Amazon S3 being the overall winner, in 2012 the results were a blowout, with Windows Azure Storage being declared the winner, sweeping all other contenders across the three categories.

“This year, our tests revealed that Microsoft Azure Blob Storage has taken a significant step ahead of last year’s leader, Amazon S3, to take the top spot. Across three primary tests (performance, scalability and stability), Microsoft emerged as a top performer in every category.” -Nusani Report

The Nasuni report goes on to mention that “the technology [Microsoft] are providing to the market is second to none.”

Reliability

One aspect of the report I found very interesting was in the error rates. For several of the vendors (including Amazon, Google, and Azure), Nasuni reported not a single error was detected during 100 million write attempts. And Microsoft stood alone for the read tests: “During read attempts, only Microsoft resulted in no errors.” In my book, I write about the Busy Signal Pattern which is needed whenever transient failures result during attempts to access a cloud service. The scenario described in the book showed the number of retries needed when I uploaded about four million files. Of course, the Busy Signal Pattern will still be needed for storage access and other services – not all transient failures can be eliminated from multitenant cloud services running on commodity hardware served over the public internet – and while this is not a guarantee there won’t be any, it does bode well for improvements in throughput and user experience.

And while it’s always been the case you can trust WAS for HA, these days it is very hard to find any reason – certainly not peformance or scalability – to not consider Windows Azure Storage. Further, WAS, S3, and Google Storage all have similar pricing (already low – and trending towards even lower prices) – and Azure, Google, and Amazon have the same SLAs for storage.

References

Note that the Nasuni report was published February 19, 2013 on the Nasuni blog and is available from their web site, though is gated, requiring that you fill out a contact form for access. The link is here: http://www.nasuni.com/blog/193-comparing_cloud_storage_providers_in

Other related articles of interest:

1. Windows Azure beats the competition in cloud speed test – Oct 7, 2011 – http://yossidahan.wordpress.com/2011/10/07/windows-azure-beats-the-competition-in-cloud-speed-test/
2. Amazon bests Microsoft, all other contenders in cloud storage test – Dec 12, 2011 -
3. Only Six Cloud Storage Providers Pass Nasuni Stress Tests for Performance, Stability, Availability and Scalability – Dec 11, 2011 – http://www.nasuni.com/news/press_releases/46-only_six_cloud_storage_providers_pass_nasuni_stress
4. Dec 3, 2012 – http://www.networkworld.com/news/2012/120312-argument-cloud-264454.html - Cloud computing showdown: Amazon vs. Rackspace (OpenStack) vs. Microsoft vs. Google
5. http://www.networkworld.com/news/2013/021913-azure-aws-266831.html?hpg1=bn - Feb 19, 2013 – Microsoft Azure overtakes Amazon’s cloud in performance test

• ABSTRACT FROM CODE CAMP SITE: https://www.bostoncodecamp.com/CC19/Sessions/Details/126
• THE SLIDE DECK: How to Architect for the Cloud — Boston Code Camp 19 — 09-Mar-2013 — Bill Wilder and Joan Wortman

I also hope to see many of the attendees at future Boston Azure meetings (held at same location as the Boston Code Camp – NERD in Cambridge, MA). Also feel free to post follow-up questions to this post or email me (codingoutloud on gmail) or ask me on twitter where I am @codingoutloud.

Here are a couple of questions that came up in the talk:

1. How much does the cloud cost? As I mentioned, this is a question that deserves some discussion since it is not as simple as looking at the pricing calculator (which can be found here). Sometimes it will be less costly, sometimes more costly. (I did point out there is a free tier for Windows Azure Web Sites.) One major factor is the cost of resources (which is trending down over time). Another major factor is the impact of reducing resource usage when it is not needed; for example, consider a Line of Business application which is used only during business hours in North America and can be turned off completely (accruing no VM usage charges) during non-business hours/weekends/holidays; as another example consider that you don’t need to own resource for the “spike” at the Superbowl (like Shazam scenario described by Joan) since you can “give it all back” (stop paying) once the rush is over. There are also other considerations when you get into DR and HA and geo-distribution. (I wrote about RPO and RTO terms in the context Engineering for DR in the Cloud recently.) And still another factor is understanding what you are paying for — don’t forget the Iceberg idea — so do not compare pricing with those of traditional hosting (unless that’s what you really want) since hosting is not cloud computing!
2. Why can I only access 32 messages at a time from the Windows Azure Storage Queue? This is the same limit when we talk about “peeking” (looking at what’s on the queue without removing it) and retrieving messages for exclusive access. I don’t know why this particular limit was chosen (why not 20? why not 100?) so could only speculate on that. The bottom line is that all messages can be accessed – sometimes requiring more than one call. I wish I had time to probe into the application scenario that would benefit from grabbing so many messages at once, but due to time constraints did not do that. I will answer the question further if I get a follow-up question.
3. Where can I find the mail app that Joan mentioned? The Mailbox app is for iOS and can be found in your app store or directly on iTunes here: https://itunes.apple.com/us/app/mailbox/id576502633?mt=8 (and there’s a lot of press – such as this story here).
4. OTHER QUESTIONS? Send ‘em along!

Hope to see you at Boston Azure:

Much of the material for the talk also appears in my book:

Here is the abstract:

If my application runs on cloud infrastructure, am I done? Not if you wish to truly take advantage of the cloud. The architecture of a cloud-native application is different than the architecture of a traditional application and this talk will explain why. How to scale? How do I overcome failure? How do I build a system that I can manage? And how can I do all this without a huge monthly bill from my cloud vendor? We will examine key architectural patterns that truly unlock cloud benefits. By the end of the talk you should appreciate how cloud architecture differs from what most of use have become accustomed to with traditional applications. You should also understand how to approach building self-healing distributed applications that automatically overcome hardware failures without downtime (really!), scale like crazy, and allow for flexible cost-optimization.

Here are the slides:

How is Architecting for the Cloud Different — DevBoston — 06-Feb-2013 — Bill Wilder (blog.codingoutloud.com)

Here is the book we gave away copies of (and from which some of the material was drawn):

Ready to learn more about Windows Azure? Come join us at the Boston Azure Cloud User Group!

As technology professionals we need to be careful about how we spend our time. Unless we want short careers, we find time to keep us with at least some new technologies, but there isn’t time in anyone’s day to keep up with every technology. We have to make choices.

For the IT Pro looking at cloud technologies, the IaaS capabilities are a far more obvious area on which to spend time than PaaS capabilities. In this post, we’ll take a peek into PaaS. The goal is to clarify the difference between IaaS and PaaS, understand what PaaS is uniquely good for, and offer some reasons why a busy IT Pro might want to invest some time learning about PaaS.

While the concepts in this point can apply generally to many platforms – including public and private clouds, Microsoft technologies and competing solutions – this post focuses on IaaS and PaaS capabilities within the Windows Azure Cloud Platform. Virtual machines and SQL databases are highlighted since these are likely of greatest interest to the IT Pro.

The Options – From Ten Thousand Feet

The NIST Definition of Cloud Computing (SP800-145) defines some terms that are widely used in the industry for classifying cloud computing approaches. One set of definitions delineates Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). You can read the NIST definitions for more details, but the gist is this:

Generally speaking, as we move from SaaS through PaaS to IaaS, we gain more control and flexibility at the expense of more cost and expertise needed due to added complexity. There are always exceptions (perhaps a SaaS solution that requires complex integration with an on-premises solution), but this is good enough to set the stage. Now let’s look at the core differences between PaaS and IaaS as they relate to the IT Pro.

Not All VMs are Created Equal

Even though Windows Azure has vastly more to offer (more on that later), the most obvious front-and-center offering is the humble VM. This is true both for PaaS and IaaS. So what distinguishes the two approaches?

The VMs for PaaS and IaaS behave very differently. The PaaS VM has a couple of behaviors that may surprise you, while the IaaS VM behavior is more familiar. Let’s start with the most far-reaching difference: On a PaaS VM, local storage is not durable.

This has significant implications. Suppose you install software (perhaps a database) on a PaaS VM and it stores some data locally. This will work fine… at least for a short while. At some point, Azure will migrate your application from one node to another… and it will not bring local data with it. Your locally-stored database data, not to mention any custom system tuning you did during installation, are gone. And this is by design. (For list of scenarios where PaaS VM drive data is destroyed, see the bottom of this document.)

How can this possibly be useful: a VM that doesn’t hold on to its local data…

You might wonder how this can possibly be useful: a VM that doesn’t hold on to its data. The fact of the matter is that it is not very useful for many applications written with conventional (pre-cloud) assumptions (such as guarantees around the durability of data). [PaaS may not be good at running certain applications, but is great at running others. So please keep reading!]

PaaS VM Local Storage

The PaaS VM drives use conventional server hard drives. These can fail, of course, and they are not RAID or high-end drives; this is commodity hardware optimized for high value for the money. And even if drives don’t outright fail, there are scenarios where the Azure operating environment does not guarantee durability of locally stored data (as referenced earlier).

IaaS VM Local Storage

On the other hand, IaaS VMs do have persistent/durable local drives. This is what makes them so much more convenient to use – and why they have a more familiar feel to IT Pros (and developers). But these drives are not the local server hard drives (other than the D: drive which is expected to be used only for temporary caching), they use a high capacity, highly scalable data storage service known as the Windows Azure Blob service (“blobs” for short, where each blob is roughly equivalent to a file, and each drive referenced by the VM is a VHD stored as one of these files). Data stored in blobs is safe from hardware failure: it is stored in triplicate by the blob service (each copy on a different physical node), and is even then geo-replicated in the background to a data center in another region, resulting (after a few minutes of latency) in an additional three copies.

IaaS VMs have persistent/durable local storage backed by blobs… this makes them so much more convenient to use – and more familiar to IT Pros

Storing redundant copies of your data offers a RAID-like feel, though is more cost-efficient at the scale of a data center.

Since blobs transparently handle storage for IaaS VMs (operating system drive, one or more data drives) and is external to any particular VM instance, in addition to being a familiar model, it is extremely robust and convenient.

Summarizing Some Key Differences

SLA details for the IaaS VM are preliminary since the service is still in preview as of this writing.

SQL Database Options: PaaS vs IaaS

Windows Azure offers a PaaS database option, formerly called SQL Azure, and today known simply as SQL Database. This is really SQL Server behind the scenes, though it is not exactly the same as SQL Server 2012 (“Denali”).

SQL Database is offered as a service. This means with a few mouse clicks (or a few lines of PowerShell) you can have a database connection string that’s ready to go. Connecting to this database will actually connect you to a 3-node SQL Server cluster behind the scenes, but this is not visible to you; it appears to you to simply be a single-node instance. Three copies of your data are maintained by the cluster (each on different hardware).

Consider the three copies of every byte to be great for High Availability (HA), but offers no defense against Human Error (HE). If someone drops the CUSTOMER table, that drop will be immediately replicated to all three copies of your data. You still need a backup strategy.

One big benefit of the SQL Database service is that the server is completely managed by Windows Azure… with the flip side of that coin being that an IT Pro simply cannot make any adjustments to the configuration. Note that SQL tuning and database schema design skills have not gone anywhere; this is all just as demanding in the cloud as outside the cloud.

SQL Database Service has a 150 GB Limit

SQL Database has some limitations. The most obvious is that you cannot store more than 150 GB in a single instance. What happens when you have 151 GB? This brings to light another PaaS/IaaS divergence: the IaaS approach is to grow the database (“scale up” or “vertical scaling”) while the PaaS approach is to add additional databases (“scale out” or “horizontal scaling”). For the SQL Database service in Windows Azure, only the “horizontal scaling” approach is supported – it becomes up to the application to distribute its data across more than one physical database, an approach known commonly as sharding, where each shard represents one physical database server. This can be a big change for an application to support since the database schema needs to be compatible, which usually means it needs to have been originally designed with sharding in mind. Further, the application needs to be built to handle finding and connecting to the correct shard.

For PaaS applications that wish to support sharding, the Federations in SQL Database feature provides robust support for handling most of the routine tasks. Without the kind of support offered by Federations, building a sharding layer can be far more daunting. Federations simplifies connection string management, has smart caching, and offers management features that allow you to repartition your data across SQL Database nodes without experiencing downtime.

The alternative to SQL Database is for you to simply use an IaaS VM to host your own copy of SQL Server. You have full control (you can configure, tune, and manage your own database, unlike with the SQL Database service where these functions are all handled and controlled by Windows Azure). You can grow it beyond 150 GB. It is all yours.

But realize that in the cloud, there are still limitations. All public cloud vendors offer a fixed menu of virtual machine sizes, so you will need to ensure that your self-managed IaaS SQL Server will have enough resources (e.g., RAM) for your largest database.

Any database can outgrow its hardware, whether on the cloud or not.

It is worth pointing out that any database can outgrow its hardware. And the higher end the hardware, the more expensive it becomes from a “capabilities for the money” point of view. And at some point you can reach the point where (a) you can’t afford sufficiently large hardware, or (b) the needed hardware is so high end that it is not commercially available. This will drive you towards a either a sharding architecture, or some other approach to make your very large database smaller so that it will fit in available hardware.

SQL Database Service is Multitenant

Another significant difference between the SQL Database service and a self-hosted IaaS SQL Server is that the SQL Database service is multitenant: your data sits alongside the data of other customers. This is secure – one customer cannot access another customer’s data – but it does present challenges when one customer’s queries are very heavy, and another customer [potentially] experiences variability in performance as a result. For this reason, the SQL Database service protects itself and other customers by not letting any one customer dominate resources – this is accomplished with “throttling” and can manifest in multiple ways, from a delay in execution to dropping a connection (which the calling application is responsible for reestablishing).

Don’t underestimate the importance of properly handling of throttling. Applications need to be written to handle these scenarios in order to function correctly. Throttling can happen even if your application is doing nothing wrong.

Handling the throttling should not be underestimated. Proper throttling handling requires that application code handle certain types of transient failures and retry. Most existing application code does not do this. Blindly pointing an existing application at a SQL Database instance might seem to work, but will also potentially experience odd errors occasionally that may be hard to track down or diagnose if the application was written (and tested) in an environment where interactions with SQL Server always succeeded.

The self-managed IaaS database does not suffer this unpredictability since you presumably control which application can connect and can manage resources more directly.

SQL Database Service has Additional Services

The SQL Database service has some easy to enable features that may make your like easier. One example is the database sync service that can be enabled in the Windows Azure Portal. You can easily configure a SQL Database instance to be replicated with one or more other instances in the same or different databases. This can help with an offsite-backup strategy, mirroring globally to reduce latency, and is one area where PaaS shines.

SQL Database Service is SQL Server

Windows Azure today offers the SQL Database service based on SQL Server 2012. If your application (for some reason) needs an older version SQL Server (perhaps it is a vendor product and you don’t control this), then your hands are tied.

Or perhaps you want another database besides SQL Server. Windows Azure has a partner offering MySQL, and other vendor products will likely be offered over time. NoSQL Databases are also becoming more popular. Windows Azure natively offers the NoSQL Windows Azure Table service, and a few examples of other third-party ones include MongoDB, Couchbase, RavenDB, and Riak. Unless (or until) these are offered as PaaS services through the Windows Azure Store, your only option will be to run them yourself in an IaaS VM.

WazOps Features and Limitations

The main thrust of PaaS is to make operations efficient for applications designed to align with the PaaS approach. For example, applications that can deal with throttling, or can deal with a PaaS VM being migrated and losing all locally stored data. This is all doable – and without degrading user experience – it just so happens that most applications that exist today (and will still exist tomorrow) don’t work this way.

The PaaS approach can be used to horizontally scale an application very efficiently (whether computational resources running on VMs or database resources sharded with Federations for SQL Database), overcome disruptions due to commodity hardware failures, gracefully handle throttling (whether from SQL Database or other Azure services not discussed), and do so with minimal human interaction. But getting to this point is not automatic.

WazOps – DevOps, Windows Azure style! – is the role that will build out this reality. There are auto-scaling tools, both external services, and some that we can run ourselves — like the awesome WASABi auto-scaling application block from Microsoft’s Patterns & Practices group – that can be configured to scale an application on a schedule or based on environmental signals (like the CPU is spiking in a certain VM).

There is also the mundane. How to script a managed deployment so our application can be upgraded without downtime? Windows Azure PaaS services have features for this, such as the in-place update and the VIP Swap. But we still need to understand them and create a strategy to use them appropriately.

Further, there are at least some of the same-old-details. For example, it is easy to deploy an SSL certificate to my PaaS VM that is being deployed to IIS… but it still will expire in a year and someone still needs to know this – and know what to do about it before it results in someone being called at 2:00 AM on a Sunday.

Should IT Pros Pass on PaaS?

Clearly there are some drawbacks to running PaaS since most existing applications will not run successfully without some non-trivial rework, but will work just fine if deployed to IaaS VMs.

However, that does not mean that PaaS is not useful. It turns out that some of the most reliable, scalable, cost-efficient applications in the world are architected for this sort of PaaS environment. The Bing services behind bing.com take this approach, as only one example. The key here is that these applications are architected assuming a PaaS environment. I don’t use the term “architected” lightly, since architecture dictates the most fundamental assumptions about how an application is put together. Most applications that exist today are not architected with PaaS-compatible assumption. However, as we move forward, and developer skills catch up with the cloud offerings, we will see more and more applications designed from the outset to be cloud-native; these will be deployed using these PaaS facilities.

A stateless web-tier (with no session affinity in the load balancer) is a good example today of an application that could run successfully in a PaaS environment – though I’ll be quick to note that other tiers of that application may not run so well in PaaS. Which bring up an obvious path going forward: hybrid applications that mix PaaS and IaaS. This will be a popular mix in the coming years.

Hybrid Applications

Consider a 3-tier application with a web tier running in IIS, a service tier, and a SQL Server back-end database. If built with conventional approaches, not considering the PaaS cloud, none of these three tiers would be ready for a PaaS environment. So we could deploy all three tiers using IaaS VMs.

As a software maintenance step, it would be reasonable to upgrade the web site (perhaps written in PHP or ASP.NET) to be stateless and not need session affinity (Windows Azure PaaS Cloud Services do not support session affinity from the load balancer). These types of changes may be enough to allow the web tier to run more efficiently using PaaS VMs, while still interacting with a service tier and database running on IaaS VMs.

A future step could upgrade the service tier to handle SQL Database throttling correctly, allowing the SQL Server instance running on an IaaS VM to be migrated to the SQL Database service. This will reduce the number of Windows servers and SQL Servers being managed by the organization (shifting these to Windows Azure), and may also simplify some other tasks (like replicating that data using the Data Sync Service). Each services and VM also has its own direct costs (our monthly bill to Microsoft for the Windows Azure Cloud services we consume), which are detailed in the pricing section of the Windows Azure Portal.

Still another future step could be to migrate the middle tier to be stateless – but maybe not. All of these decisions are business decisions; perhaps the cost-benefit is not there. It depends on you application and your business and the skills and preferences of the IT Pros and developers in the organization.

Conclusions

I’ll summarize here with some of the key take-aways for the IT Pro who is new to PaaS services:

1. Be aware of the challenges in migrating existing applications onto either PaaS VMs or SQL Database. If the application is not architected with the right assumptions (stateless VMs, SQL operations that may be throttled, 150 GB limit), it will not work correctly – even though it might seem to work at first. IaaS VMs will often present a better option.
2. SQL Database does not support all of the features that SQL Server 2012 support. Though it does have some special ones of its own: always runs as a three-node cluster for HA, and has Federation support.
3. PaaS is increasingly the right choice for new applications that can be built from the outset. Assumes that team understands PaaS and has learned the needed skills! (I wrote a book – Cloud Architecture Patterns – to illuminate these new skills.)
4. Pure IaaS and pure PaaS are not the only approaches. Hybrid approaches will be productive.
5. PaaS will gain momentum long-term due to the economic benefits since they can be cheaper to run and maintain. There are direct costs which are easy to measure (since you get a detailed bill) and indirect/people costs which are more challenging to measure.
6. WazOps (DevOps with an Azure spin) will be the role to deliver on the promise of PaaS going forward. Not only with the well-informed WazOps professional help avoid issues of going too fast (see earlier points which speak to not all applications being PaaS-ready), but also understand the business drivers and economics of investing to move faster where appropriate for your business.

Feedback always welcome and appreciated. Good luck in your cloud journey!

[This post is part 21 of the 31 Days of Server (VMs) in the Cloud Series - please return to the series index by clicking here]

Windows Azure Web Sites also support .NET 4.5, but what is the easiest way to deploy a .NET 4.5 application to Windows Azure as a Cloud Service? This post shows how easy this is.

Assumption

This post assumes you have updated to the most recent Windows Azure Tools for Visual Studio and the latest SDK for .NET.

For any update to a new operating system or new SDK, consult the Windows Azure Guest OS Releases and SDK Compatibility Matrix to understand which versions of operating systems and Azure SDKs are intended to work together.

You can do this with the Web Platform Installer by installing Windows Azure SDK for .NET (VS 2012) – Latest (best option) – or directly here (2nd option since this link will become out-of-date eventually).

Also pay close attention to the release notes, and don’t forget to Right-Click on your Cloud Service, hit Properties, and take advantage of some of the tooling support for the upgrade:

Creating New ASP.NET Web Role for .NET 4.5

Assuming you have up-to-date bits, a File | New from Visual Studio 2012 will look something like this:

Select a Cloud project template, and (the only current choice) a Windows Azure Cloud Service, and be sure to specify .NET Framework 4.5. Then proceed as normal.

Updating Existing ASP.NET Web Role for .NET 4.5

If you wish to update an existing Web Role (or Worker Role), you need to make a couple of changes in your project.

First, update the Windows Azure Operating System version use Windows Server 2012. This is done by opening your Cloud project (pageofphotos in the screen shot) and opening ServiceConfiguration.Cloud.cscfg.

Change the osFamily setting to be “3” to indicate Windows Server 2012.

   osFamily=”3″

As of this writing. the other allowed values for osFamily are “1” and “2” to indicate Windows Server 2008 SP2 and Windows Server 2008 R2 (or R2 SP1) respectively. The up-to-date settings are here.

Now you are set for your operating system to include .NET 4.5, but none of your Visual Studio projects have yet been updated to take advantage of this. For each project that you intend to update to use .NET 4.5, you need to update the project settings accordingly.

First, select the project in the Solution Explorer, right-click on it, and choose Properties from the pop-up menu. That will display the screen shown. Now simply select .NET Framework 4.5 from the available list of Target framework options.

If you open an older solution with the newer Azure tools for Visual Studio, you might see a message something like the following. If that happens, just follow the instructions.

That’s it!

Now when you deploy your Cloud Service to Windows Azure, your code can take advantage of .NET 4.5 features.

Troubleshooting

Be sure you get all the dependencies correct across projects. In one project I migrated, I realized the following came up because I had a mix of projects that needed to stay on .NET 4.0, but those aspects deployed to the Windows Azure Cloud could be on 4.5. If you don’t get this quite right, you may get a compiler warning like the following:

Warning  The referenced project ‘CapsConfig’ is targeting a higher framework version (4.5) than this project’s current target framework version (4.0). This may lead to build failures if types from assemblies outside this project’s target framework are used by any project in the dependency chain.    SomeOtherProjectThatReferencesThisProject

The warning text is self-explanatory: the solution is to not migrate that particular project to .NET 4.5 from .NET 4.0. In my case, I was trying to take advantage of the new WIF features, and this project did not have anything to do with Identity, so there was no problem.

The backstory is that Windows Azure uses certificates in a few different ways, and understanding the different types of certificate uses is key to understanding why these different ways of using and deploying certificates are the way they are.

The slide deck is here:

Sorting Out Digital Certificates – 13-Dec-2012 – Bill Wilder – Boston Azure

Clouds are not always inviting! Be prepared for storm clouds.

Damage from one of these events might be temporary: a prolonged power outage that is eventually restored. Damage might be permanent: servers immersed in water are unlikely to work after drying out.

Whether a one-person shop with all the customer data on a single laptop, or a large multi-national with its own data centers, any business that uses computers to manage data important to that business needs to consider DR.

The remainder of this article focuses on some useful DR approaches for avoiding loss of business data when engineering applications for the cloud. The detailed examples are specific to the Windows Azure Cloud Platform, but the concepts apply more broadly, such as with Amazon Web Services and other cloud platforms. Notable this post does not discuss DR approaches as they apply to other parts of infrastructure, such as web server nodes or DNS routing.

Minimize Exposure

Your first line of defense is to minimize exposure. Consider a cloud application with business logic running on many compute nodes.

Terminology note: I will use the definition of node from page 2 of my Cloud Architecture Patterns book (and occasionally in other places in this post I will reference patterns and primers from the book where they add more information):

An application runs on multiple nodes, which have hardware resources. Application logic runs on compute nodes and data is stored on data nodes. There are other types of nodes, but these are the primary ones. A node might be part of a physical server (usually a virtual machine), a physical server, or even a cluster of servers, but the generic term node is useful when the underlying resource doesn’t matter. Usually it doesn’t matter.

In cloud-native Windows Azure applications, these compute nodes are Web Roles and Worker Roles. The thing to realize is that local storage on Web Roles and Worker Roles is not a safe place to keep important data long term. Well before getting to an event significant enough to be characterized as needing DR, small events such as a hard-disk failure can result in the loss of such data.

While not a DR issue per se due to the small scope, these applications should nevertheless apply the Node Failure Pattern (Chapter 10) to deal with this.

But the real solution is to not use local storage on compute nodes to store important business data. This is part of an overall strategy of using stateless nodes to enable your application to scale horizontally, which comes with many important benefits beyond just resilience to failure. Further details are described in the Horizontally Scaling Compute Pattern (Chapter 2).

Leverage Platform Services

In the United States, there are television commercials featuring “The Most Interesting Man in the World” who lives an amazing, fantastical life, and doesn’t always drink beer, but when he does he drinks DOS EQUIS.

In the cloud, our compute nodes do not always need to persist data long-term, but when they do, they use cloud platform services.

And the “DOS” in “DOS EQUIS” stands for neither Disk Operating System nor Denial of Service here, but rather is the number two in Spanish. But cloud platform services for data storage do better than dos, they have tres – as in three copies.

Windows Azure Storage and Windows Azure SQL Database both write three copies of each byte onto three independent servers on three independent disks. The hardware is commodity hardware – chosen for high value, not strictly for high availability – so it is expected to fail, and the failures are overcome by keeping multiple copies of every byte. If the one of the three instances fails, a new third instance is created by making copies from the other two. The goal state is to continually have three copies of every byte.

Windows Azure Storage is always accessed through a REST interface, either directly, or via specific SDK which uses the REST interface under the hood. For any REST API call that modifies data, the API does not return until all three copies of the bytes are successfully stored.

Windows Azure SQL Database is always accessed through TDS, which is the same TCP protocol as SQL Server. While your application is provided a single connection string, and you create a single TDS connection, behind the scenes there is a three-node cluster. For any operation that modifies data, the operation does not return until at least two copies of the update have been successfully applied on two of the nodes in this cluster; the third node is updated asynchronously.

So if you have a Web Role or Worker Role in Windows Azure, and that node has to save data, it should use one of the persistent storage mechanisms just mentioned.

What about Windows Azure Virtual Machines?

Windows Azure also has a Virtual Machine node that you can deploy (Windows or Linux flavored), and the hard disks attached to those nodes are persistent, but how can that be? It turns out they are backed by Windows Azure Blob storage, so that doesn’t break the model: they also have some storage that is truly local and can use it for caching sorts of functions, but any long-term data is persisted to blob storage, even though it is indistinguishable from a local disk drive from the point of view of any code running on the virtual machine.

But wait, there’s more!

In addition to this, Windows Azure Storage asynchronously geo-replicates blobs and tables to a sister data center. There are eight Azure data centers, and they are paired as follows: East US-West US, North Central US-South Central US, North Europe-West Europe, and East Asia-Southeast Asia. Note that the pairs are chosen to be in the same geo-political region to simplify regulatory compliance in many cases. So if you save data to a blob in East US, three copies will be synchronously written in East US, then three more copies will be asynchronously written to West US.

It is easy to overlook the immense value of having data stored in triplicate and transparently geo-replicated. While the feature comes across rather matter-of-factly, you get incredibly rich DR features without lifting a finger. Don’t let the ease of use mask the great value of this powerful feature.

All of the local and geo-replication mentioned so far happens for free: it is included as part of the listed at-rest storage costs, and no action needed on your part to enable this capability (though you can turn it off).

Enable More as Needed

All the replication listed above will help DR. If a hardware failure takes out one of your three local copies, the system self-heals – you will never even know most types of failures happen. If a natural disaster takes out a whole data center, Microsoft decides when to reroute DNS traffic for Windows Azure Storage away from the disabled data center and over to its sister data center which has the geo-replicated copies.

Note that the geo-replication is only out-of-the-box today for Windows Azure Storage (and not for queues – just for blobs and tables) and not for SQL Database. However, this can be enabled using the sync service available today – you decide how many copies and to which data centers and at what frequency.

Note that there are additional costs associated with using the sync service for SQL Database, for the sync service itself and for data center egress bandwidth.

Regardless of the mechanism, there is always a time-lag in asynchronous geo-replication, so if a primary data center was lost suddenly, the last few minutes worth of updates may not have been fully replicated. Of course, you could choose to write synchronously to two data centers for super-extra safety, but please consult the Network Latency Primer (Chapter 11) before doing so.

This is all part of the overall Multisite Deployment Pattern (Chapter 15), though servicing a geo-distributed user base is another feature of this architecture pattern, beyond the DR features.

Where’s the Engineering?

The title of this blog post is “Engineering for Disaster Recovery in the Cloud” but where did all the engineering happen?

Much of what you need for DR is handled for you by cloud platform services, but not all of it. From time-to-time we alluded to some design patterns that your applications need to adhere to in order for these platform services to make sense. As one example, if your application is written to assume it is safe to use local storage on your web server as a good long-term home for business data, well… the awesomeness built into cloud platform services isn’t going to help you.

There is an important assumption here if you want to leverage the full set of services available in the cloud: you need to build cloud-native applications. These are cloud application that are architected to align with the architecture of the cloud.

I wrote an entire book explaining what it means to architect a cloud-native application and detailing specific cloud architecture patterns to enable that, so I won’t attempt to cover it in a blog post, except to point out that many of the architectural approaches of traditional software will not be optimal for applications deployed to the cloud.

Distinguish HE from DR

Finally, we need to distinguish DR from HE – Disaster Recover from Human Error.

Consider how the DR features built into the cloud will not help with many classes of HE. If you modify or delete data, your changes will dutifully be replicated throughout the system. There is no magic “undo” in the cloud. This is why you usually will still want to take control of making back-ups of certain data.

So backups are still desirable. There are cloud platform services to help you with backups, and some great third-party tools as well. Details on which to choose warrant an entire blog post of their own, but hopefully this post at least clarifies the different needs driven by DR vs. HE.

Is This Enough?

Maybe. It depends on your business needs. If your application is one of those rare applications that needs to be responsive 24×7 without exception, not even for a natural disaster, then no, this is not enough. If your application is a line-of-business application (even an important one), often it can withstand a rare outage under unusual circumstances, so this approach might be fine. Most applications are somewhere in between and you will need to exercise judgement in weighing the business value against the engineering investment and operational cost of a more resilient solution.

And while this post talked about how the combination of following some specific cloud architecture patterns to design cloud-native applications provides a great deal of out-of-the-box resilience in DR situations, it did not cover ongoing continuity, such as with computation, or immediate access to data from multiple data centers. If you rely entirely on the cloud platform to preserve your data, you may not have access to it for a while since (as mentioned earlier, and emphasized nicely in Neil’s comment) you don’t control all the failover mechanisms; you will need to wait until Microsoft decides to failover the DNS for Windows Azure Storage, for example. And remember that background geo-replication does not guarantee zero data loss: some changes may be lost due to the additional latency needed in moving data across data centers, and not all data is geo-replicated (such as queued messages and some other data not discussed).

The ITIL term for “how much data can I stand to lose” is known as the recovery point objective (RPO). The ITIL term for “how long can I be down” is known as the recovery time objective (RTO). The RPO and RTO are useful concepts for modeling DR.

So the DR capabilities built into cloud platform services are powerful, but somewhat short of all-encompassing. However, they do offer a toolbox providing you with unprecedented flexibility in making this happen.

Is This Specific to the Cloud?

The underlying need to understand RPO and RTO and use them to model for DR is not specific to the cloud. These are very real issues in on-premises systems as well. The approaches to addressing them may vary, however.

Generally speaking, while the cloud does not excuse you from thinking about these important characteristics, it does provide some handy capabilities that make it easier to overcome some of the more challenging data-loss threats. Hopefully this allows you to sleep better at night.

—-

Bill Wilder is the author of the book Cloud Architecture Patterns – Develop Cloud-Native Applications from O’Reilly. This post complements the content in the book. Feel free to connect with Bill on twitter (@codingoutloud) or leave a comment on this post. (He’s also warming up to Google Plus.)

—-

The certificate for the given thumbprint could not be loaded from the Current User/Personal certificate store. Please install the certificate.

While I was able to resolve the error, I cannot reproduce it (and gave up trying), but if you face the same problem, hopefully this will help you.

Background

If Visual Studio was looking for a certificate, where was it looking? It turns out that the Windows Azure Tools for Visual Studio store some certificate related references in a file called Windows Azure Connections.xml in your personal settings area on Windows. This file is created on your behalf once you’ve created any Publish Profiles by Publishing Cloud Services to Windows Azure from Visual Studio.

The file lives here:

%UserProfile%\Documents\Visual Studio 2012\Settings\Windows Azure Connections.xml

On my Windows 8 development machine, this is:

C:\Users\billdev\My Documents\Visual Studio 2012\Settings\Windows Azure Connections.xml

The file contains the credentials you’d previously supplied during publishing and will look something like the following:

<?xml version=”1.0″?>
<NamedCredentials xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema”>
<Items>
<NamedCredential>
    <SubscriptionId>12345678-abcd-ae10-abba-01812e1e1000</SubscriptionId>
<IsImported>false</IsImported>
<ServiceEndpoint>https://management.core.windows.net/</ServiceEndpoint>
<CertificateThumbprint>A123B123C12333EC954471ED75C37D59003681F7</CertificateThumbprint>
<Name>Page of Photos</Name>
</NamedCredential>
</Items>
<LastUsedName>Page of Photos</LastUsedName>
</NamedCredentials>

Note that the NamedCredential XML element item may be repeated.

Problem

The problem turns out to be that one of the certificates referenced (identified by thumbprint via CertificateThumbprint XML element) either it is not installed properly locally, or not installed in the associated Windows Azure Subscription (identified by SubscriptionId XML element).

Solution

For each certificate referenced by a CertificateThumbprint element (there could be more than one, unlike the simple example shown above):

1. Make sure the certificate is installed in your Local Certificate store and contains a Private Key – which usually can be found in the Personal (or “My”) store name under the Current User certificates by using the Certificates Snap-in with Microsoft Management Console. (You can also use certmgr.exe or write your own code to dump certificate info). (If the certificate exists in your local certificate store then it is probably fine. It is not likely it is missing a Private Key. But it is possible.)
2. Make sure the certificate has been uploaded to the Windows Azure Portal for the SubscriptionId  referenced within Windows Azure Connections.xml.

That’s it. Should work. Worst case you can delete each element of your Windows Azure Connections.xml profile and start over.

Specific Scenarios

These are the two specific scenarios where I saw the problem in case you are interested.

Scenario #1

This scenario failed whether or not a project was open.

1. Open the Server Explorer window in Visual Studio
2. Right-click on Windows Azure Storage, choose “Add New Storage Account…“, and the error dialog appears:
   “The certificate for the given thumbprint could not be loaded from the Current User/Personal certificate store. Please install the certificate.”
3. This message is extra confusing since I don’t think there ought to be any certificates involved here. And no project/solution is open.

Scenario #2.

This scenario requires an open Azure project.

1. Open the UI tool for editing Azure configuration by opening your Cloud Project in Solution Explorer, drilling into Roles, and double-clicking on a Web Role or Worker Role project. The Role configuration editor window opens in Visual Studio.
2. Choose Settings, then Add Setting (which creates Setting1 of Type=String), change Setting1‘s Type to Connection String, and the click the “…” button at far right (to pop up the connection string edit window), and an error dialog appears:
   “The certificate for the given thumbprint could not be loaded from the Current User/Personal certificate store. Please install the certificate.”

Here are the screen shots for the two error dialogs (slightly different).

The assembly needed in Andy’s instructions is hard to find these days (it lives in the GAC) and is undocumented. And you only want to do this in debug code running in your local Cloud Simulation environment anyway. So explicitly referencing the needed assembly feels a little dirty since you’d never want it to be deployed accidentally to the cloud.

The Solution

I’ve taken these considerations and created a very simple to use method that you can easily call from ASP.NET code — probably from Application_Start in Global.asax.cs — and not worry about it polluting your production code or causing other ills. The code uses reflection to load the needed assembly to avoid the need for an explicit reference, and the dynamic loading is only done under the proper circumstances; loading the assembly would never be attempted in a cloud deployment.

The Code

Bill is the author of the book Cloud Architecture Patterns, recently published by O’Reilly. Find Bill on twitter @codingoutloud or contact him for Windows Azure consulting.

And just for fun, here is a dump of the certificates running on a Windows Azure Web Role (I did not install any add’l certificates on this instance):

• Bob Goodearl – Among his many contributions, Bob built the Code Camp web site using Windows Azure – and will come to Boston Azure on Thursday November 15 to tell us about his experience in doing so.
• Ben Day – @benday – ALM / TFS Expert and TDD Nerd
• Chris Pels – Co-Leader of DevBoston with Ben Day, Arch/Dev
• Chris Bowen - @chrisbowen - Microsoft Developer Evangelist
• Jim O’Neil - @jimoneil - Microsoft Developer Evangelist
• John Zablocki - @johnzablocki - Couchbase Developer Advocate
• Patrick Hynds - @patrickhynds - Security Guru
• Talbott Crowell – @talbott - SharePoint Expert, F# MVP

And of course nothing would happen without support from the Sponsors:

• Full list here: https://www.bostoncodecamp.com/CC18/Sponsors
• I am particularly intrigued by the ApprendaCloud.com (“Built for Devs” and “Free to use”) from sponsor . .. Apprenda offers a private cloud PaaS solution focused on .NET applications.

Here is the abstract for my talk:

Just because we get an application to run on cloud infrastructure does ensure that it runs well. To truly take advantage of the cloud we need to build cloud-native applications. The architecture of a cloud-native application is different than the architecture of a traditional application. A cloud-native application is architected for cost-efficiency, availability, and scalability. We will examine several key architecture patterns that help unlock cloud-native benefits, spanning computation, database, and resource-focused patterns. By the end of the talk you should appreciate how cloud architecture is more demanding than you might be accustomed to in some areas, but with high payoff such as handling failure without downtime, scaling arbitrarily, and allowing aggressive cost-optimization.

All the concepts and patterns I spoke about are also discussed in my recently released book, Cloud Architecture Patterns:

More info on the book is here:

• www.cloudarchitecturepatterns.com
• If you do read the book and find it of value, I’d very much appreciate you considering a short review on Amazon, O’Reilly, or Barnes & Noble.

Got Azure or Cloud questions? Feedback on the book? Just want to stay in touch? Please feel free to reach out.

• I can be reached via twitter (@codingoutloud)
• I can also be reached via email (which is same as my twitter handle at gmail).
• You can also find me through my blog (blog.codingoutloud.com)

The slide deck I used is here:

Architecture Patterns for Building Cloud-Native Applications — Boston Code Camp 18 — 20-Oct-2012 — Bill Wilder (blog.codingoutloud.com) – WITH SOME SLIDES HIDDEN

This was the talk abstract:

Just because we get an application to run on cloud infrastructure does not ensure that it runs well. To truly take advantage of the cloud we need to build cloud-native applications. The architecture of a cloud-native application is different than the architecture of a traditional application. A cloud-native application is architected for cost-efficiency, availability, and scalability. We will examine several key architecture patterns that help unlock cloud-native benefits, spanning computation, database, and resource-focused patterns. By the end of the talk you should appreciate how cloud architecture is more demanding than you might be accustomed to in some areas, but with high payoff such as handling failure without downtime, scaling arbitrarily, and allowing aggressive cost-optimization.

All the concepts and patterns I spoke about are also discussed in my recently released book, Cloud Architecture Patterns:

More info on the book is here:

www.cloudarchitecturepatterns.com

If you do read the book, I’d very much appreciate a short review on Amazon.

Also, please stay in touch via twitter (@codingoutloud) or email (my twitter handle at gmail). Got Azure or Cloud questions? Feedback on the book? Please reach out.

And the slide deck I used is attached here:

Architecture Patterns for Building Cloud-Native Applications — CT.NET — 09-Oct-2012 — Bill Wilder (blog.codingoutloud.com)

Just because we get software to run on cloud infrastructure does not make it a good idea. To truly take advantage of the cloud we need to build cloud-native applications. The architecture of a cloud-native application is different than the architecture of a traditional application, an idea we will explore since it can impact costs, availability, and scalability. We will examine several key architecture patterns that help unlock cloud-native benefits, spanning computation, database, and resource-focused patterns. By the end of the talk you should appreciate how cloud architecture is more demanding than you might be accustomed to, but with high payoff such as handling failure without downtime, scaling arbitrarily, and allowing aggressive cost-optimization.

The slide deck is here:

Architecture Patterns for Building Cloud-Native Applications — NYC Code Camp 7 — 15-Sept-2012 — Bill Wilder (blog.codingoutloud.com)

You can Meet #WindowsAzure in a live stream featuring keynote speaker Scott Guthrie (@ScottGu) along with other Azure/cloud experts. Event is June 7 at 4:00 PM Boston time (UTC-7 hours).

I will be watching and you can find discussions on the Twitters…. I am @codingoutloud, the event hashtag is #MeetAzure, and be sure to check out the Lanyard page that Magnus set up.

Also if you are an Azure fan in the Boston area, please check out the Boston Azure cloud user group (www.bostonazure.org). The group meets monthly, with occasional special events, such as the 2-day bootcamp later this month. The group events are usually at NERD in Cambridge, MA.

References:

1. www.meetwindowsazure.com
2. Registration page: http://go.microsoft.com/?linkid=9809415
3. Live Stream page: http://go.microsoft.com/?linkid=9809426
4. Boston Azure cloud user group: www.bostonazure.org

MEET Windows Azure Blog Relay:

• Roger Jennings (@rogerjenn): Social meet up on Twitter for Meet Windows Azure on June 7th
• Anton Staykov (@astaykov): MEET Windows Azure on June the 7th
• Patriek van Dorp (@pvandorp): Social Meet Up for ‘MEET Windows Azure’ on June 7th
• Marcel Meijer (@MarcelMeijer): MEET Windows Azure on June the 7th
• Nuno Godinho (@NunoGodinho): Social Meet Up for ‘MEET Windows Azure’ on June 7th
• Shaun Xu (@shaunxu) Let’s MEET Windows Azure
• Maarten Balliauw (@maartenballiauw): Social meet up on Twitter for MEET Windows Azure on June 7th
• Brent Stineman (@brentcodemonkey): Meet Windows Azure (aka Learn Windows Azure v2)
• Herve Roggero (@hroggero): Social Meet up on Twitter for Meet Windows Azure on June 7th
• Paras Doshi (@paras_doshi): Get started on Windows Azure: Attend “Meet Windows Azure” event Online
• Simran Jindal (@SimranJindal): Meet Windows Azure – an online and in person event, social meetup #MeetAzure (+ Beer for Beer lovers) on June 7th 2012
• Michael Wood (@mikewo): Learn about Windows Azure and Chat with Experts, June 7th
• Shiju Varghese (@shijucv): Social meet up on Twitter for MEET Windows Azure on June the 7th
• Jeremie Devillard (@jeremiedev): Meet the Cloud–Windows Azure Event 7th June
• Kris van der Mast (@KvdM): Get ready to meet Windows Azure
• Mike Martin (@TechMike2KX): Don’t miss the online Windows Azure event of the year : MEET Windows Azure on June 7th 
• Bill Wilder (@codingoutloud): Get ready to “Meet #WindowsAzure” in a live streamed event June 7 at 4:00 PM Boston time
• Eric Boyd   (@EricDBoyd): Meet Windows Azure – Unveiling the Latest Platform
• Magnus Mårtensson (@noopman): Social meet up on Twitter for MEET Windows Azure on June 7th

Jason Haley was the featured speaker and provided a fantastic overview of the Windows Azure SDK and related platform features.

My talk was the opening act. I extended on my Hadoop on Azure talk from February and climbed up the stack to talk about HIVE and Pig which generate MapReduce under the hood. I only used a few slides in the talk, but they have some useful references, and are available here: Hadoop-BostonAzure-29-Mar-2012.

These items were also mentioned at the meeting:

• Boston Azure is on Twitter as @bostonazure
• New England Give Camp 2012 (A handful of Boston Azure members are signed up – including (off the top of my head) me, Jason, Maura, Joan, and Gladys): http://newenglandgivecamp.org/
• UX and Cloud studies were discussed and the vast majority of the attendees indicated they would prefer offers/opportunities be sent to/shared with the Boston Azure community – I will post on the Meetup.com distribution

Upcoming Boston Azure events include:

• Tuesday April 24 – Michael Cummings from Magenic on building mobile games with a Windows Azure back-end (signup here)
• Wednesday May 30 – SDK again – but this time will be hands-on – BYO laptop! (Tables and power strips will be provided.) (signup here)
• Fri-Sat June 22-23 – TWO FULL DAYS of Bootcamp! (signup here (Fri) and here (Sat))
• Thu June 28 – (Awesome topic TBD) – Michael Collier from Neudesic (signup here)
• Thu July 26 – maybe CQRS? (signup here)

There was interest in incorporating some topics into future Boston Azure meetings:

• Tools – what tools are useful in building/managing/monitoring cloud applications, especially Azure ones (PowerShell, cspack, Cerebrata, Storage Explorer from CodePlex, CloudBerry Explorer for Azure Blob Storage, monitoring tools, IntelliTrace from Visual Studio Ultimate, other Visual Studio tips/tricks, …)
• Comparisons of Azure with other clouds, e.g., OpenStack, AWS, Eucalyptus (note: AWS and Eucalptus are now partners), Rackspace, others…
• More big data via Hadoop on Azure (maybe a whole dedicated meeting is in order)

The slide deck from the talk is here:

florida–cloud-architecture-patterns-three-big-ideas–bill-wilder–28-march-2012.

The abstract for the talk reads as follows:

We can run pre-cloud software on cloud infrastructure, but to truly take advantage of the cloud we need to build cloud-native applications. The architecture of a cloud-native application is different than the architecture of a traditional pre-cloud application, and in this talk we will examine several big ideas in software architecture you need to ‘grok’ if you want to truly leverage the cloud for cost savings, higher availability, and better scalability. We will examine several key architecture patterns that help unlock those cloud-native benefits, spanning computation, database, and resource-focused patterns. By the end of the talk you should appreciate how cloud architecture is more of a partnership with your hardware than it was with pre-cloud applications (fail-retry anyone?), that the cloud may be infinite (but not all at once), and how the cloud enables cost optimization (and is “green”).

Feel free to reach out to me – my email address is in the slide deck, and my twitter handle is @codingoutloud. Also – if you are in the neighborhood – check out the Boston Azure User Group and our planned Boston Azure Bootcamp in June 2012! You may want to follow the adventures of Mr. SQL Azure Federations

In the featured talk, Michael Stiefel gave an insightful, thought-provoking talk on Architecting for Failure: Why Cloud Architecture is Different. Michael will post has posted his slides. The slides are listed under Cloud Computing Presentations called Architecting For Failure, Cloud Architecture is Different! (note this is not same as Michael’s blog).

As a warmup, I gave a short talk describing the challenges in making sense of Big Data, what (in the computer science sense) Map and Reduce are, and how the Hadoop infrastructure makes building MapReduce processes so easy. Ended with a bit of a peak at the CTP of “Hadoop as a Service” – the Microsoft Windows Azure service that is in CTP – at www.hadooponazure.com. The talk focused on Hadoop – and a simple Hadoop example at that – only mentioning that was a broader Hadoop ecosystem: the official Apache Hadoop project, some subprojects (HIVE, Pig (which has a Pig Latin language, not to be confused with this one), Mahout, ZooKeeper, HBASE, and others), some other related efforts (Cascading.org), and some commercial companies dedicated to Hadoop (Cloudera, Hortonworks, and others – they are roughly the Hadoop equivalents of Red Hat in the Linux world; Microsoft is working with Hortonworks on their Hadoop on Azure and Hadoop on Windows Server effort).

My Hadoop slides are attached here: Hadoop-BostonAzure-07-Feb-2012.

I also discussed some upcoming Azurey events of interest to the Boston Azure community. That deck is here: Upcoming Events of Interest to Boston Azure Community.

O’Reilly Radar has a concise roundup of some of these technologies here (which I noticed on a tweet here). And this excerpt from the official Apache Hadoop project lists some related technologies:

The project includes these subprojects:

• Hadoop Common: The common utilities that support the other Hadoop subprojects.
• Hadoop Distributed File System (HDFS™): A distributed file system that provides high-throughput access to application data.
• Hadoop MapReduce: A software framework for distributed processing of large data sets on compute clusters.

Other Hadoop-related projects at Apache include:

• Avro™: A data serialization system.
• Cassandra™: A scalable multi-master database with no single points of failure.
• Chukwa™: A data collection system for managing large distributed systems.
• HBase™: A scalable, distributed database that supports structured data storage for large tables.
• Hive™: A data warehouse infrastructure that provides data summarization and ad hoc querying.
• Mahout™: A Scalable machine learning and data mining library.
• Pig™: A high-level data-flow language and execution framework for parallel computation.
• ZooKeeper™: A high-performance coordination service for distributed applications.

Since this summary page is – by necessity – a point-in-time SNAPSHOT of what I see is going on, it will not necessarily be updated when event details change. So please always double-check with official event information!

I’ve attempted to list events of interest to the local Azure community – not just topics specific to the Windows Azure Cloud Platform. Know of any more cloud events of interest? Have any more information or corrections on the events listed? Please let everyone know about them by adding a comment.

Events are listed in the order in which they will occur.

January 2012 Events

1. Cloud Mixer + talk featuring Satya Nadella

• when: Thu 26-Jan-2012
  5:00 – 6:15 – Networking with MassTLC’s Cloud Community
  6:15 – 7:15 – Presentation and Q&A with Satya Nadella, President, Server and Tools Business, Microsoft
• where: NERD
• wifi: (usually)
• food: (perhaps during the networking segment?)
• cost: FREE,
• what:First a networking social – get to know other area folks interested in cloud, then the following:The transition to cloud computing has been talked about as one of the most profound shifts occurring in technology in decades. With the huge growth and broad range of computing devices increasingly available, we see a shift in “design point” to a world of connected devices and continuous services. In this talk, Satya Nadella, President, Server and Tools Business at Microsoft, will share what he learned running a global online service, Bing, and how these lessons are informing the direction of Microsoft’s cloud strategy.Nadella is one of very few people in the world who can speak first-hand about running an extremely large-scale cloud computing business. Having previously led R&D for Microsoft’s Online Services Division, which includes Bing and MSN, Nadella has practical and deep experience with cloud both from a technical and business perspective. Today, Nadella brings these experiences to bear in his current role, which includes accountability for the overall business and technical vision, strategy, operations, engineering and marketing for Microsoft’s $17+ billion Server and Tools Business.Join us to hear how Microsoft is focused on building a platform that spans public and private clouds to enable businesses to take advantage of this new design point.
• more info: http://126cloudmixer.eventbrite.com/
• register: http://126cloudmixer.eventbrite.com/
• twitter: @MassTLC and #CloudMixer

February 2012 Events

2. Boston Azure User Group meeting: Architecting for Failure: Why Cloud Architecture is Different & Hadoop

• when: Tuesday February 7, 2012, 6:00 – 8:30 PM
• where: Hosted at NERD Center, 1 Memorial Drive, Cambridge, MA (directions)
• wifi: Wireless Internet access will be available
• food: Pizza and drinks expected
• cost: FREE
• what: Architecting for Failure: Why Cloud Architecture is different – a talk by Michael Stiefel + a shorter intro talk on Hadoop on Windows Azure (currently in CTP) by Bill Wilder
• more info: See the Boston Azure cloud user group site for more info
• register: http://www.meetup.com/bostonazure/events/38945872/
• twitter: #bostonazure

3. Mass TLC Summit: The Big Data Disruption

• when: Wednesday, February 15, 2012 from 8:00 AM to noon
• where: Hosted at NERD Center, 1 Memorial Drive, Cambridge, MA (directions)
• wifi: Wireless Internet access expected
• food: Pizza and drinks expected
• cost: Costs money – see registration for details
• what: (quoted from registration site): “Join us as we take a deep dive into the future outlook of big data and how it is going to impact the tech industry down the road.
  

  Keynote Speaker:  Living Smarter

Session 1 – Crossing the Big Data Chasm
Moderated by: Mike Stonebraker, Professor, MIT
Panelists to date:
Fritz Knabe, Distinguished Engineer, Netezza
Christopher Ahlberg, Chief Executive Officer/Co-Founder, Recorded Future
This session will explore the hype versus reality of Big Data. Discussion will include the transformational opportunity it represents for the technology industry and offer a true perspective on why One Size Doesn’t Fit All. Panelists will discuss the different tools and approaches for a variety of outcomes.

Session 2 – From Hype to Reality
Moderated by:  Andy Palmer, Startup Specialist
Panelists to date:
Puneet Batra, Chief Data Scientist, Kyruus
This panel session will take a deep dive into the challenges, successes, failures and lessons learned of the early adopters of big data initiatives. The panelists will give insight using their own case studies as evidence of what has and has not worked.

• more info: http://21512bigdata.eventbrite.com/
• register: http://21512bigdata.eventbrite.com/
• twitter: @MassTLC #BigData

4. Boston Azure User Group meeting: Solving Access Control in the Cloud: from WIF to ACS

• when: Thu February 23, 2012, 6:00 – 8:30 PM
• where: Hosted at NERD Center, 1 Memorial Drive, Cambridge, MA (directions)
• wifi: Wireless Internet access will be available
• food: Pizza and drinks expected
• cost: FREE
• what: Solving Access Control in the Cloud: from WIF to ACS featuring Brock Allen – a deep dive into Claims, Windows Identity Foundation (WIF), and the Access Control Service (ACS) from Windows Azure
• more info: See the Boston Azure cloud user group site for more info
• register: RSVP here
• twitter: #bostonazure

March 2012 Events

5. Windows Azure: Focus on your application. Not the infrastructure.

• when: Thu March 8, 2012, 8:30 AM – noon
• where: 201 Jones Road, Waltham, MA (directions)
• wifi: not needed
• food: coffee!
• cost: FREE
• what: (quoted from registration site)
  “Microsoft Windows Azure and Microsoft SQL Azure enable you to build, host and scale applications in Microsoft datacenters. They require no up-front expenses, no long term commitment, and enable you to pay only for the resources you use. Windows Azure is about flexibility and adaptability; whether your application can or should run 100% in the cloud, or leverage components on-premise in a hybrid cloud, Windows Azure is the most comprehensive cloud platform.The #1 question we hear, from our customers, is “How can my business benefit from Windows Azure?”.Come join us in Waltham, MA on Thursday, March 8, 2012 to learn the business benefits of Windows Azure and SQL Azure and hear real world customer examples of how Windows Azure improves your business Agility while directly Reducing your Costs.

  The target audience for this event includes BDMs, TDMs, Architects, and Development leads. The sessions are targeted at the 100-200 level with a mix of business focused information as well as technical information.”
  

• more info: See the registration page for more info
• register: RSVP here
• twitter:

—–

Omissions? Corrections? Comments? Please leave a comment or reply on Twitter.
Share this post on Twitter

Jim introduced cloud and presented specifics on Blob and Table storage services and SQL Azure. I had the opportunity to present one of the sections – mine was a combination of Windows Azure Compute services + the Windows Azure Queue service with some basics around using these services to assemble “cloud native” applications. The official slides for the Windows Azure DevCamp series appear to be here, though my slides were a little different and are also available (WindowsAzureDeveloperCamp-FarmingtonCT-07Dec2011-BillWilder). At the end, Jim also ran through the creation of a RockPaperAzure “bot” and it was (literally!) game on as attendees raced to create competitive entries.

I took a few photos at the event – some of Jim presenting, some showing participants at the end coming to claim their prizes from the RockPaperAzure challenge – and none from the middle!

I spoke about Cloud Architecture Patterns like sharding, NoSQL, queue-based compute separation for scalability and reliability – with specific examples from the Windows Azure Platform such as SQL Azure Federations, Azure Table Storage, and Web Role + Queue + Worker Role patterns. The slides from my talk are here: nsnug-big-ideas-in-software-architecture-bill-wilder-14-dec-2011. (UPDATE: Note that I don’t seem to have the exact deck I used for the talk. As Ryan CrawCour pointed out, the deck I posted claims that SQL Azure is limited to 50 GB and Federations has not yet shipped, but at the talk I am certain I presented the 150 GB limit and a recently released Federations. I think I made the changes on the train en route to the event and somehow didn’t save them. Sorry! We’ll need to live with this small skew. Post a comment here if there are questions…)

There was definitely some good discussion and many questions. In fact, the following question came up, and I didn’t have a great response, but turns out there’s a timely response from Mr. SQL Azure Federations himself: http://blogs.msdn.com/b/cbiyikoglu/archive/2011/12/15/so-isn-t-the-root-database-a-bottleneck-for-federations-in-sql-azure.aspx

Also hope to see some nsnug folks at future Boston Azure User Group meetings and our planned Boston Azure Bootcamp in June 2012!

Since this summary page is – by necessity – a point-in-time SNAPSHOT of what I see is going on, it will not necessarily be updated when event details change. So please always double-check with official event information!

I’ve attempted to list events of interest to the local Azure community – not just topics specific to the Windows Azure Cloud Platform. Know of any more cloud events of interest? Have any more information or corrections on the events listed? Please let everyone know about them by adding a comment.

Events are listed in the order in which they will occur.

November 2011 Events

1. Cloudy Mondays

• when: Mon 14-Nov-2011, 5:00 – ?:?? PM
• where: Small Business Development Center (note: NOT Venture Development Center!), 100 Morrissey Boulevard, Boston, MA
  wifi: (not sure)
• food: (not sure, though food and spirits were provided last time)
• cost: FREE
• what: Discuss Amazon’s cloud: launching projects on AWS, comparing AWS to other public clouds, etc. One speaker is Jason Haruska, Chief Architect at Backupify. Vikram Kumar, CTO and Founder of OfficeDrop.com, will also be discussing how OfficeDrop.com launched using AWS.
• more info: http://www.meetup.com/Cloudy-Mondays/events/39091772/
• register: http://www.meetup.com/Cloudy-Mondays/events/39091772/
• twitter: (not sure)

2. Cloud Roundtable – Business in the Cloud -

• when: Tue 15-Nov-2011, 3:00 - 5:30 PM (updated: start time is officially 3:00 PM)
• where: Boston College Club, 100 Federal St., Boston, MA
• wifi: (not sure)
• food: (looks like cocktails and hors d’oeuvres towards the end)
• cost: FREE
• what: From the description on the web site: “The cloud is a fundamental paradigm shift from our current or past thinking about scalable architecture and there are security tradeoffs:  less control of data, new vulnerability classes, and compliance challenges.  However, if managed properly, these risks can be mitigated. This interactive seminar will discuss the challenges of cloud computing, demonstrate how to build a secure and redundant system, and touch upon real-world examples of cloud computing gone bad.”
• more info: http://web.securityinnovation.com/boston
• register: http://web.securityinnovation.com/boston
• twitter: (not sure)

3. Boston Azure User Group meeting: Introduction to Cloud, Windows Azure, Azure Dev Tools

• when: Thu 17-Nov-2011, 6:00 – 8:30 PM
• where: Hosted at NERD Center, 1 Memorial Drive, Cambridge, MA (directions)
• wifi: Wireless Internet access will be available
• food: Pizza and drinks will be provided
• cost: FREE
• what: Get ramped up on Cloud and Windows Azure – what is this cloud thing all about? How does the Windows Azure Cloud Platform fit in? And how can you get started using Visual Studio and the Windows Azure tools? Get all these questions answered in one night!
• more info: See the Boston Azure cloud user group site for more info
• register: http://www.meetup.com/bostonazure/events/38944852/
• twitter: #bostonazure

December – 2011 Events

4. Azure is on the Road! December Road Trip…

At a variety of locations (far from Boston, but at least regional):

• Check out Jim O’Neil’s blog post on the road show: http://blogs.msdn.com/b/jimoneil/archive/2011/11/04/taking-the-cloud-on-the-road.aspx

5. Testing in the Cloud with Boston Web Performance Group

• when: Tue 06-Dec-2011: 6:30-7:15 Eat pizza and meet other members, 7:15-8:15 Presentation, 8:15-9:00 Q&A, general socializing and more pizza eating
• where: Wayfair Offices – 177 Huntington Ave, 177 Huntington Ave, Boston, MA (map) Come up to the 24th floor, call 802-522-6294 with issues
• wifi: Wireless Internet access will be available
• food: Pizza and drinks will be provided
• cost: FREE
• what: Speaker: Dan Bartow on Testing in the Cloud – description from event site: “Achieving real performance on the web begins with realistic testing and an understanding of the application and it’s infrastructure.  Testing realistically and for real results across web & mobile applications means leveraging the
  cloud.  Agile development processes, complex multi-tier architectures, and the potential for massive (and sudden) load all require a different approach than
  historical apps.  Come hear from Soasta, the makers of the CloudTest platform and CloudTest Lite, about their experiences and why companies need to change their expectations about what “testing” means. Dan Bartow is Vice President and CloudTest Evangelist at SOASTA, the leader
  in performance testing from the cloud. Prior to joining SOASTA he was Senior Manager of Engineering at Intuit, where his team was responsible for the speed and stability of TurboTax Online, the #1 rated, best-selling online tax software. Over the past decade he has been responsible for the speed and
  scalability of websites for such well-known brands as American Eagle Outfitters, AT&T, Best Buy, Finish Line, J.Crew, Neiman Marcus and Sony Online
  Entertainment, among others. Dan has set multiple industry precedents including launching the worlds largest stateful JBoss cluster and using over 2000 cloud computing cores to generate load against a live web site.  Dan is a frequent industry presenter and has spoken at leading testing and cloud computing
  conferences such as Software Test & Performance (STP), O’Reilly’s Web2.0 Expo, Amazon Web Services Road Show, and SYS-CON’s Cloud Computing Expo.” Visit the event web site to view the Cloud Testing Bill of Rights.
• more info: See http://www.meetup.com/Web-Performance-Boston/events/38442872/ for more info
• register: http://www.meetup.com/Web-Performance-Boston/events/38442872/
• twitter: @JonathanKlein

6. Boston Azure User Group December

• when: Tue 13-Dec-2011, 6:00 – 8:30 PM
• where: Hosted at NERD Center, 1 Memorial Drive, Cambridge, MA (directions)
• wifi: Yes
• food: Pizza and drinks will be provided
• cost: FREE
• what: Azure-related topic(s) to be announced.
• more info: See the Boston Azure cloud user group site for more info
• register: http://www.meetup.com/bostonazure/events/38945582/
• twitter: #bostonazure

7. Cloud Architecture Patterns at North Shore .NET User Group

• when: Wed 14-Dec-2011, 6:00 – 8:30 PM
• where: EBSCO Publishing, 10 Estes Street, Ipswich, MA (map)
• wifi: (not sure)
• food: (Food is either available … or provided .. not sure)
• cost: FREE
• what: Cloud Architecture Patterns (generally), then see how the Windows Azure Platform helps you realize these patterns. Speaker is Bill Wilder, Boston Azure user group leader, Windows Azure MVP, and an Independent Azure Consultant.
• more info: http://www.meetup.com/North-Shore-NET-User-Group/events/40679722/?a=ea1.2_lnm&rv=ea1.2
• register: http://www.meetup.com/North-Shore-NET-User-Group/events/40679722/?a=ea1.2_lnm&rv=ea1.2
• twitter:

8. Using Windows Azure to Build Cloud Enabled Windows Phone Apps at Microsoft DevBoston

• when: Wed 14-Dec-2011, 6:00 – 8:30 (?) PM
• where: Microsoft Corporation 201 Jones Road, Waltham, MA Hosted at NERD Center, 1 Memorial Drive, Cambridge, MA (directions)
• wifi: I don’t think so (given the location)
  food: (I am guessing there will be pizza, but not 100% sure, please check with the group organizers)
• cost: FREE
• what: “Speaker: John Garland This presentation will discuss how Windows Phone applications can be enriched by leveraging the power of the Cloud that is made available by Windows Azure. The Windows Azure Toolkit for Windows Phone will be explored to show how to quickly tap into resources in the cloud for computation, storage, identity, and communication from within a Windows Phone application.”
• more info: http://www.meetup.com/DevBoston/events/39064112/
• register: http://www.meetup.com/DevBoston/events/39064112/
• twitter: (not sure about the event, but John Garland can be found at https://twitter.com/#!/search/dotnetgator)

—–

Omissions? Corrections? Comments? Please leave a comment or reply on Twitter.
Share this post on Twitter

Image courtesy of http://www.flickr.com/photos/thewazir/4232029536/sizes/s/in/photostream/

I believe we are the oldest Windows Azure user group in the world, still going strong. Our first meeting was held in October 2009, less than a year after Windows Azure was announced at PDC 2008, and a few months before it went RTW for real (which was, I believe, January 2010).

Now, back to our Oct meeting. Here are links to a few topics mentioned:

• The Enterprise Library Integration Pack for Windows Azure includes two new blocks: Autoscaling Application Block (“WASABi”) and Transient Fault Handling Application Block. Beta bits are available. http://entlib.codeplex.com/wikipage?title=EntLib5Azure, http://blogs.msdn.com/b/avkashchauhan/archive/2011/10/14/windows-azure-autoscaling-application-block-wasabi-beta-is-ready-for-you-to-try.aspx
• http://blogs.msdn.com/b/windowsazure/archive/2011/10/27/microsoft-lowers-storage-cost-for-blob-and-table-storage-in-windows-azure.aspx
• http://blogs.msdn.com/b/windowsazure/archive/2011/10/19/announcing-improved-in-place-updates.aspx
• http://www.cumulux.com/fall-2011-and-build-azure-related-new-features/
• http://www.davidaiken.com/2011/10/10/implementing-windows-azure-retry-logic/

Also, here is the slide deck from the main presentation – given by yours truly (that’s me, Bill Wilder) – called Big Ideas in Software Architecture (Cloud and Otherwise):

• PPTX file: boston-azure-big-ideas-in-software-architecture-cloud-and-otherwise-bill-wilder-27-oct-2011

At the meeting we also discussed some topics for future meetings. Here is that list (okay, I actually can’t find the list – may have forgotten to save it – sorry – so going from memory here) — note this list is in no particular order:

1. Introduction to Cloud, Azure, and developing for Windows Azure
2. More hands-on with the platform
3. Using languages, libraries, and software other than .NET/Microsoft – e.g., Java, Python, Node.js, NoSQL (MongoDB), …
4. Idempotency – look at a more challenging case than a simple thumbnailer
5. Security in the Cloud
6. Comparing Cloud platforms
7. Azure AppFabric topics: Service Bus, Caching, Access Control Service (can Curt come back?), …
8. More architecture patterns

WHAT ARE OTHER TOPICS YOU WOULD LIKE TO SEE? Please leave a comment, email me, or suggest via twitter (me: @codingoutloud or to the community: @bostonazure).

Like last year, the class was engaged, asking tough and interesting questions… which is all the more impressive since this class meets on FRIDAY NIGHT. Must be a Harvard thing… Anyhow, we went from around 5:30 – 8:00… ON FRIDAY NIGHT.

Below are the resources I mentioned at the end of my talk, and the slide deck I used is here: Harvard-WhyAzureIsAwesome-Bill-Wilder-04-Nov-2011

Also, hope to see all of you at Boston Azure user group meetings! Feel free to contact me with any follow-up questions.

The slide deck Jim O’Neil used is here, plus here are a few action shots of Jim doing his thang:

I also attended some cool talks – Brock Allen spoke about WIF, David Padbury on node.js, and Dominic Denicola on various Async approaches like Promises. Good time as usual! No after-event celebrating – everyone is running for cover due to the Nor’easter!

If you are interested in learning more about the Windows Azure Platform, please come join us at a Boston Azure cloud user group meeting. Details at www.bostonazure.org. We meet every month to learn about Azure. Sometimes we learn through prepared talks, sometimes we hold training events, and sometimes coding/hackathons. We are the oldest such user group in the world, turning two years old this month. Hope to see you!

Our next meeting is Thursday November 17 (the Thursday before Thanksgiving), featuring a very Azurey talk by Chris Rolon of Neudesic.

Got Azure Question? I am also a Windows Azure MVP for Windows Azure and know a thing or two about the platform. I am happy to answer questions you may have. Feel free to contact me on twitter (@codingoutloud) or by email (which is my twitter handle at gmail.com).

Azure On!

For my part, I attended a number of interesting sessions (especially the frighteningly entertaining talk by Francis Brown on using Google and Bing to hack (or protect) web properties). Due to scheduling challenges, I missed Andrew Wilson‘s talk on Reversing Web Applications, which I wanted to check out.

For my part, I offered a Birds-of-a-Feather session on Securing Applications in the Cloud (with examples drawn from Windows Azure Platform). In this session, I reviewed both pros and cons of cloud deployments from a security point of view, and attempted to make the case that, ultimately, either your applications will simply be safer in the cloud, or at least if you want them to be sufficiently safe, it will be more cost-effective to let the specialists at Microsoft (or some other trusted cloud vendor) handle much of the dirty work.

This session was interesting for me to put together and then go through with an intimate crowd (due, at least in part I suppose, to (me) changing the scheduled time slot after the conference schedule went to the printer… D’oh! … that combined with the seeming invisibility of the BoF sessions generally). Anyhow, it was still fun to discuss, and here is the slide deck I used: OWASP Boston – BoF – Securely Running Applications in Cloud (examples drawn from Windows Azure Platform) – Bill Wilder – 08-Oct-2011.

Here are a few links that folks attending might have been told about, plus a couple of answers I offered to gather offline.

Where can I get the materials used in the Bootcamp?

• The materials live here: http://www.azurebootcamp.com/materials
• However, as I explained at the bootcamp, the actual materials used at our sessions were a mix of what is posted on the web and some slide decks that had been updated (mostly for the Azure SDK 1.5, but also other changes in some cases). So you can pull the materials as linked to above and you’ll be pretty close, but the updated ones are not yet publicly posted.

How can I see what’s in Windows Azure Storage?

• In addition to the tools we showed (such as Visual Studio and Cerebrata Cloud Storage Studio), there’s a useful, free storage browser on the web: https://www.myazurestorage.com/Containers, and an Explorer-like browser as Open Source on CodePlex here: http://azurestorageexplorer.codeplex.com/

How can I track changes/upgrades to Windows Azure Guest OS?

• (There might be an RSS feed, but I can’t find it if there is..) A lower-tech approach is to use a service like https://www.changedetection.com/ for monitoring the Windows Azure Guest OS page at http://msdn.microsoft.com/en-us/library/ff729422.aspx

Does Azure use Hyper-threading?

• No

Where can I learn more about the Windows Azure Platform?

• Start coming to Boston Azure User Group meetings! Check out http://www.bostonazure.org - meetings are monthly at NERD – schedule now managed at http://www.meetup.com/bostonazure

Where can I read more?

• Insight from the big brains at Microsoft:
• http://www.microsoft.com/windowsazure/whitepapers/
• http://blogs.msdn.com/windowsazurestorage/

Who should I thank for this event?

• You can thank our TWO MAJOR SPONSORS: This event was provided free to you because our Gold Sponsor SNI TECHNOLOGY generously sponsored the food, and Microsoft NERD donated the space. Many thanks to these major sponsors!
• Without these sponsors this event would simply not have happened.
• You can thank our swag sponsors: O’Reilly (books), Pluralsight (training), Cerebrata (licenses), Packt Publishing (books), and Microsoft (books and licenses).
• And you can thank the Boston Azure Bootcamp team which included (in alphabetical order): Andy Novick (who led the SQL Azure segment), Arra Derderian (helped during labs), George Babey (“swag guy” – and helped during labs), Jim O’Neil (lab-time tech support, lecture-time answer-man), Joan Wortman (ran the registration), Maura Wilder (who led the Azure Table Storage segment – and helped during labs), Nazik Huq (“twitter guy” – plus made sure there was food – and helped during labs), and William Wilder (yes, that’s me; you can call me “Bill” but wanted to be listed last…). Also, many thanks to Martha O’Neil for baking us a cloudy cake.

We are planning another Boston Azure Bootcamp in 2012. Stay tuned!

Update 22-Oct-2011: Here is contact info for our Gold sponsors at SNI TECHNOLOGY:

Since this summary page is – by necessity – a point-in-time SNAPSHOT of what I see is going on, it will not necessarily be updated when event details change. So please always double-check with official event information!

updates: Added Tech Boston event + more details on Boston Azure

Know of any more cloud events of interest to the Windows Azure community? Have any more information or corrections on the events listed? Please let us know in the comments.

Events are listed in the order in which they will occur.

October Events

1. Mongo Boston

• when: Mon 03-Sep-2011, 9:00-5:00 PM
• where: Hosted at NERD Center
• wifi: Wireless Internet access will be available
• food: Provided
• cost: $30
• what: The main Azure-related content is a talk by Jim O’Neilon using Mongo with the Windows Azure Platform – from the published program description: “MongoDB in the Cloud, Jim O’Neil - Developer Evangelist, Microsoft: MongoDB is synonomous with scale and performance, and, hey, so is cloud computing! It’s peanut butter and chocolate all over again as we take a look at why you might consider running MongoDB in the cloud in general and also look at the alpha release of MongoDB on Azure, a collaboration from 10gen and Microsoft.”
• more info: http://www.10gen.com/events/mongo-boston-2011
• register: http://www.10gen.com/events/mongo-boston-2011
• twitter: @mongodb

2. Cloud Camp

• when: Thu 06-Oct-2011, 5:15 – 8:30 PM (then after-party)
• where: CloudCamp Boston #5 is Co-located with the OpenStack Design Summit. Intercontinental Hotel, 510 Atlantic Ave, Salon A (between Congress St & Fort Hill Wharf), Boston, MA 02210
• wifi: (not sure)
• food: (not sure, though food and beer were offered last time)
• cost:
• what: (from the event description on cloudcamp.org) “CloudCamp is an unconference where early adopters of Cloud Computing technologies exchange ideas.  With the rapid change occurring in the industry, we need a place where we can meet to share our experiences, challenges and solutions.  At CloudCamp, you are encouraged to share your thoughts in several open discussions, as we strive for the advancement of Cloud Computing.  End users, IT professionals and vendors are all encouraged to participate.”
• more info: http://www.cloudcamp.org/boston
• register: here
• twitter: (not sure)

3. Boston Tech: What Can Cloud Do for YOU?

• when: Mon 24-Oct-2011, 6:30 – 8:30 PM
• where: Hosted at NERD Center
• wifi: Wireless Internet access will be available
• food: Pizza and drinks will be provided
• cost: FREE
• what: “You’ve probably seen the “to the cloud” commercial and are aware of the hype
  that makes cloud computing sound like the next best thing since sliced bread,
  but do you really know what cloud computing is? And what it’s not? When does it
  make sense? And when doesn’t it? What does it mean to us as software developers,
  startup entrepreneurs, and end-users? And how do you sort through all of the
  vendors and offerings to determine whose cloud portfolio offers the most value
  to you? We’ll look at all of these questions and more as we spend the evening
  navigating through the cloudscape.” (text taken from the Meetup listing)
• more info: http://www.meetup.com/BostonTech/events/33357482/
• register: http://www.meetup.com/BostonTech/events/33357482/
• twitter:

4. Boston Azure User Group meeting: Cloud Architecture Patterns

• when: Thu 27-Oct-2011, 6:00 – 8:30 PM
• where: Hosted at NERD Center
• wifi: Wireless Internet access will be available
• food: Pizza and drinks will be provided
• cost: FREE
• what: Featured talk: “There are some big ideas in software architecture that are particularly relevant for cloud platforms. In this talk we will introduce a few of these big ideas – eventual consistency, scale out, and design for failure – and discuss the implications of these big ideas on cloud application architecture generally, with specific examples of useful patterns and services drawn from the Windows Azure Platform.” There will also be a shorter opening topic.
• more info: See our (new) Boston Azure Meetup.com site for more info
• register: http://www.meetup.com/bostonazure/events/35904052/
• twitter: #bostonazure

5. New England Code Camp #16

While not strictly an Azure-only event, there will be Azure content at this community-driven event. Hope to see you there!

• when: Saturday, October 29, 2011 9am–6pm
• where: Microsoft Office on Jones Road in Waltham
• wifi: (usually just for speakers)
• food: (usually pizza, sometimes donuts)
• cost: FREE
• what: It’s a Code Camp!
• more info: http://blogs.msdn.com/b/cbowen/archive/2011/08/24/new-england-code-camp-16-october-29th-save-the-date.aspx
• register (for speakers and all attendees): http://codecampboston.eventbrite.com/
• twitter:

Omissions? Corrections? Comments? Please leave a comment or reply on the Twitters!

I presented a talk in the morning called Cloud Architecture Patterns for Mere Mortals (with Examples in Windows Azure). If you are interested, my slide deck is attached: Vermont Code Camp III – Cloud Architecture Patterns for Mere Mortals – Bill Wilder – 10-Sept-2011 (also available on Slideshare)

Also, you are all invited to the (free) Boston Azure Bootcamp to be held in the Boston area (Cambridge, MA) on Friday September 30 and Saturday October 1. Sign up here, and please help spread the word. Hope to see some Vermont Code Camp friends there! Let me know if you have a strong desire to “couch surf”, especially on the middle night, and I’ll see if I can help out. Tickets won’t last forever, so I encourage you to sign up sooner than later.

Thank you to all the Vermont Code Camp III organizers, volunteers, and sponsors - like last year, this was an inspired event and I’m glad I made the trip. Find them on Twitter at @VTCodeCamp.

A handful of Vermont Code Camp photos follow… (and a couple from Sunday night on Church Street in Burlington)

A. This basic question comes up occassionally, usually when there is concern around correlating timestamps across instances, such as for log files or business events.  Over time, like mechanical clocks, computer clocks can drift, with virtual machines (especially when sharing cores) effected even more. (This is not specific to Microsoft technologies; for example, it is apparently an annoying issue on Linux VMs.)

I can’t find any official stats on how much drift happens generally (though some data is out there), but the question at hand is what to do to minimize it. Specifically, on Windows Azure Virtual Machines (VMs) – including Web Role, Worker Role, and VM Role - how is this handled?

According to this Word document - which specifies the “MICROSOFT ONLINE SERVICES USE RIGHTS SUPPLEMENTAL LICENSE TERMS, MICROSOFT WINDOWS SERVER 2008 R2 (FOR USE WITH WINDOWS AZURE)” – the answer is once a week. (Note: the title above includes “Windows Server 2008 R2″ – I don’t know for sure if the exact same policies apply to the older Windows Server 2008 SP2, but would guess that they do.)

Here is the full quote, in the context of which services you can expect will be running on your VM in Windows Azure:

Windows Time Service. This service synchronizes with time.windows.com once a week to provide your computer with the correct time. You can turn this feature off or choose your preferred time source within the Date and Time Control Panel applet. The connection uses standard NTP protocol.

So Windows Azure roles use the time service at time.windows.com to keep their local clocks up to snuff.  This service uses the venerable Network Time Protocol (NTP), described most recently in RFC 5905.

UDP Challenges

The documentation around NTP indicates it is based on User Datagram Protocol (UDP). While Windows Azure roles do not currently support you building network services that require UDP endpoints (though you can vote up the feature request here!), the opposite is not true: Windows Azure roles are able to communicate with non-Azure services using UDP, but only within the Azure Data Center. This is how some of the key internet plumbing based on UDP still works, such as the ability to do Domain Name System (DNS) lookups, and – of course – time synchronization via NTP.

This may lead to some confusion since UDP support is currently limited, while NTP being already provided.

The document cited above mentions you can “choose your preferred time source” if you don’t want to use time.windows.com. There are other sources from which you can update the time of a computing using NTP, such as free options from National Institute for Standards and Technology (NIST).

Here are the current NTP Server offerings as seen in the Control Panel on a running Windows Azure Role VM (logged in using Remote Desktop Connection). The list includes time.windows.com and four options from NIST:

Interestingly, when I manually tried changing the time on my Azure role using a Remote Desktop session, any time changes I made were immediately corrected whenever I tried to make changes. Not sure if it was doing an automatic NTP correction after any time change, but my guess is something else was going on since the advertised next time it would sync via NTP did not change based on this.

When choosing a different NTP Server, it did not always succeed (sometimes timing out), but also I did see it succeed, as in the following:

The interesting part of seeing any successful sync with time.nist.gov is that it implies UDP traffic leaving and re-entering the Windows Azure data center. This, in general, is just not allowed – all UDP traffic leaving or entering the data center is blocked (unless you use a VM Role with Windows Azure Connect). To prove this for yourself another way, configure your Azure role VM to use a DNS server which is outside of the Azure data center; all subsequent DNS resolution will fail.

If “weekly” is Not Enough

If the weekly synchronization frequency is somehow inadequate, you could write a Startup Task to adjust the frequency to, say, daily. This can be done via the Windows Registry (full details here including all the registry settings and some tools, plus there is a very focused summary here giving you just the one registry entry to tweak for most cases).

How frequently is too much? Not sure about time.windows.com, but time.nist.gov warns:

All users should ensure that their software NEVER queries a server more frequently than once every 4 seconds. Systems that exceed this rate will be refused service. In extreme cases, systems that exceed this limit may be considered as attempting a denial-of-service attack.

Of further interest, check out the NIST Time Server Status descriptions:

They recommend against using any of the servers, at least at the moment I grabbed these Status values from their web site.  I find this amusing since – other than the default time.windows.com – these are the only four servers offered as alternatives in the User Interface of the Control Panel applet. As I mentioned above, sometimes these servers timed out on an on-demand NTP sync request I issued through the applet user interface; this may explain why.

It may be possible to use a commercial NTP service, but I don’t know if the Windows Server 2008 R2 configuration supports it (at least I did not see it in the user interface), and if there was a way to specify it (such as in the registry), I am not sure that the Windows Azure data center will allow the UDP traffic to that third-party host. (They may – I just don’t know. They do appear to allow UDP requests/responses to NIST servers. Not sure if this is a firewall/proxy rule, and if so, is it for NTP, or just NTP to NIST?)

And – for the (good kind of) hacker in you – if you want to play around with accessing an NTP service from code, check out this open source C# code.

Is this useful? Did I leave out something interesting or get something wrong? Please let me know in the comments! Think other people might be interested? Spread the word!

I recently did some work with Windows Services, and since it had been rather a long while since I’d done so, I had to recall a couple of tips and tricks from the depths of my memory in order to get my “edit, run, test” cycle to be efficient. The singular challenge for me was quickly getting into a debuggable state with the service. How I did this is described below.

Does Windows Azure support Windows Services?

First, a trivia question…

Trivia Question: Does Windows Azure allow you to deploy your Windows Services as part of your application or cloud-hosted service?

Short Answer: Windows Azure is more than happy to run your Windows Services! While a more native approach is to use a Worker Role, a Windows Service can surely be deployed as well, and there are some very good use cases to recommend them.

More Detailed Answer: One good use case for deploying a Windows Service: you have legacy services and want to use the same binary on-prem and on-azure. Maybe you are doing something fancy with Azure VM Roles. These are valid examples. In general – for something only targetting Azure – a Worker Role will be easier to build and debug. If you are trying to share code across a legacy Windows Service and a shiny new Windows Azure Worker Role, consider following the following good software engineering practice (something you may want to do anyway): factor out the “business logic” into its own class(es) and invoke it with just a few lines of code from either host (or a console app, a Web Service, a unit test (ahem), etc.).

Windows Services != Web Services

Most readers will already understand and realize this, but just to be clear, a Windows Service is not the same as a Web Service. This post is not about Web Services. However, Windows Azure is a full-service platform, so of course has great support for not only Windows Services but also Web Services. Windows Communication Foundation (WCF) is a popular choice for implementing Web Services on Windows Azure, though other libraries work fine too – including in non-.NET languages and platforms like Java.

Now, on to the main topic at hand…

Why is Developing with Windows Services Slower?

Developing with Windows Services is slower than some other types of applications for a couple of reasons:

• It is harder to stop in the Debugger from Visual Studio. This is because a Windows Service does not want to be started by Visual Studio, but rather by the Service Control Manager (the “scm” for short – pronounced “the scum”). This is an external program.
• Before being started, Windows Services need to be installed.
• Before being installed, Windows Services need to be uninstalled (if already installed).

Tip 1: Add Services applet as a shortcut

I find myself using the Services applet frequently to see which Windows Services are running, and to start/stop and other functions. So create a shortcut to it. The name of the Microsoft Management Console snapin is services.msc and you can expect to find it in Windows/System32, such as here: C:\Windows\System32\services.msc

A good use of the Services applet is to find out the Service name of a Windows Service. This is not the same as the Windows Services’s Display name you seen shown in the Name column. For example, see the Windows Time service properties – note that W32Time is the real name of the service:

Tip 2: Use Pre-Build Event in Visual Studio

Visual Studio projects have the ability to run commands for you before and after the regular compilation steps. These are known as Build Events and there are two types: Pre-build events and Post-build events. These Build Events can be accessed from your Project’s properties page, on the Build Events side-tab. Let’s start with the Pre-build event.

Use this event to make sure there are no traces of the Windows Service installed on your computer. Depending on where you install your services from (see Tip 3), you may find that you can’t even recompile your service until you’ve at least stopped it; this smooths out that situation, and goes beyond it to make the usual steps happen faster than you can type.

One way to do this is to write a command file –  undeploy-service.cmd – and invoke it as a Pre-build event as follows:

You will need to make sure undeploy-service.cmd is in your path, of course, or else you could invoke it with the path, as in c:\tools\undeploy-service.cmd.

The contents of undeploy-service.cmd can be hard-coded to undeploy the service(s) you are building every time, or you can pass parameters to modularize it. Here, I hard-code for simplicity (and since this is the more common case).

1. Set a reusable variable to the name of my service (set ServiceName=NameOfMyService)
2. Stop it, if it is running (net stop)
3. Uninstall it (installutil.exe /u)
4. If the service is still around at this point, ask the SCM to nuke it (sc delete)
5. Return from this .cmd file with a  success status so that Visual Studio won’t think the Pre-Build event ended with an error (exit /b 0 => that’s a zero on the end)

The S3 service runs “out there” (in the cloud) and provides a scalable repository for applications to store and manage data files. The service can support files of any size, as well as any quantity. So you can put as much stuff up there as you want – and since it is a pay-as-you-go service, you pay for what you use. The S3 service is very popular. An example of a well-known customer, according to Wikipedia, is SmugMug:

Photo hosting service SmugMug has used S3 since April 2006. They experienced a number of initial outages and slowdowns, but after one year they described it as being “considerably more reliable than our own internal storage” and claimed to have saved almost $1 million in storage costs.

Good stuff.

Of course, Amazon isn’t the only cloud vendor with such an offering. Google offers Google Storage, and Microsoft offers Windows Azure Blob Storage; both offer features and capabilities very similar to those of S3. While Amazon was the first to market, all three services are now mature, and all three companies are experts at building internet-scale systems and high-volume data storage platforms.

As I mentioned above, S3 came up during a talk I attended. The speaker – CTO of a company built entirely on Amazon services – twice touted S3’s incredibly strong Service Level Agreement (SLA). He said this was both a competitive differentiator for his company, and also a competitive differentiator for Amazon versus other cloud vendors.

Pause and think for a moment – any idea? – What is the SLA for S3? How about Google Storage? How about Windows Azure Blob Storage?

Before I give away the answer, let me remind you that a Service Level Agreement (SLA) is a written policy offered by the service provider (Amazon, Google, and Microsoft in this case) that describes the level of service being offered, how it is measured, and consequences if it is not met. Usually, the “level of service” part relates to uptime and is measured in “nines” as in 99.9% (“three nines”) and so forth. More nines is better, in general – and wikipedia offers a handy chart translating the number of nines into aggregate downtime/unavailability. (More generally, an SLA also deals with other factors – like refunds to customers if expectations are not met, what speed to expect, limitations, and more. I will focus only on the “nines” here.)

So… back to the question… For S3 and equivalent services from other vendors, how many nines are in the Amazon, Google, and Microsoft SLAs? The speaker at the talk said that S3 had an uptime SLA with 11 9s. Let me say that again – eleven nines – or 99.999999999% uptime. If you attempt to look this up in the chart mentioned above, you will find this number is literally “off the chart” – the chart doesn’t go past six nines! But my back-of-the-envelope calculation says it amounts to – on average – less than 32 milliseconds of downtime per year. This is about half what “a blink of your eye” would take – yes, a mere half of an eye-blink. (Which ends with your eyes closed. )

This is an impressive number! If only it was true. It turns out the real SLA for Amazon S3 has exactly as many nines as the SLA for Windows Azure Blob Storage and the SLA for Google Storage: they are all 99.9%.

Storage SLAs for Amazon, Google, and Microsoft all have exactly the same number of nines: they are all 99.9%. That’s three nines.

I am not picking on the CTO I heard gushing about the (non-existant) eleven-nines SLA. (In fact, his or her identity is irrelevent to the overall discussion here.) The more interesting part to me is the impressive reality distortion field around Amazon and its platform’s capabilities. The CTO I heard speak got it wrong, but this is not the first time it was misinterpreted as an SLA, and it won’t be the last.

I tracked down the origin of the eleven nines. Amazon CTO Werners Vogels mentions in a blog post that the S3 service is “design[ed]” for “99.999999999% durability” – choosing his words carefully. Consistent with Vogels’ language is the following Amazon FAQ on the same topic:

Q: How durable is Amazon S3? Amazon S3 is designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000,000 years. In addition, Amazon S3 is designed to sustain the concurrent loss of data in two facilities.

First of all, these mentions are a comment on a blog and an item in an FAQ page; neither is from a company SLA. And second, they both speak to durability of objects – not uptime or availability. And third, also critically, they say “designed” for all those nines – but guarantee nothing of the sort. Even still, it is a bold statement. And good marketing.

It is nice that Amazon can have so much confidence in their S3 design. I did not find a comparable statement about confidence in the design of their compute infrastructure… Reality is that [cloud] services are about more than design and architecture – also about implementation, operations, management, and more. To have any hope, architecture and design need to be solid, of course, but alone they cannot prevent a general service outage which could take your site down with it (and even still lose data occasionally). Some others on the interwebs are skeptical as I am, not just of Amazon, but anyone claiming too many nines.

How about the actual 99.9% “three-nines” SLA? Be careful in your expectations. As a wise man once told me, there’s a reason they are called Service Level Agreements, rather than Service Level Guarantees. There are no guarantees here.

This isn’t to pick on Amazon – other vendors have had – and will have – interruptions in service. For most companies, the cloud will still be the most cost-effective and reliable way to host your applications; few companies can compete with the big platform cloud vendors for expertise, focus, reliability, security, economies-of-scale, and efficiency. It is only a matter of time before you are there. Today, your competitors (known and unknown) are moving there already. As a wise man once told me (citing Crossing the Chasm), the innovators and early adoptors are those companies willing to trade off risk for competitive advantage. You saw it here first: this Internet thing is going to stick around for a while. Yes, and cloud services will just make too much sense to ignore. You will be on the cloud; it is only a matter of where you’ll be on the curve.

Back to all those nines… Of course, Amazon has done nothing wrong here. I see nothing inaccurate or deceptive in their documentation. But those of us in the community need to pay closer attention to what is really being described.  So here’s a small favor I ask of this technology community I am part of: Let’s please do our homework so that when we discuss and compare the cloud platforms – on blogs, when giving talks, or chatting 1:1 - we can at least keep the discussions based on facts.

This month’s July Boston Azure User Group meeting was up for the challenge.

• In the opening act, Bill Wilder (that’s me) introduced the Windows Azure Toolkit for Social Games (see also the Cloud Cover episode 52 which also talks about the toolkit). You can find Bill on Twitter @codingoutloud.
• Mark Eisenberg of Microsoft then shared some great insights about the cloud and the Windows Azure Platform – what they really are, why they matter, and how they fit into the real world. You can find Mark on Twitter @azurebizandtech.
• Marshall Rosenstein then talked about building social games for Facebook with an Azure back-end. Marshall’s slides are on his skydrive here (the slides are here, the code download is here). You can find Marshall on Facebook (http://www.facebook.com/marshall.rosensteina) and on Twitter @mrosenstein.
• We wrapped up the meeting with a short live demonstration of the Windows Azure Portal doing its thing. Then a few of us retired to the Muddy.

Hope to see you at the Boston Azure meeting in August (Windows Phone 7 + Azure), two meetings in September (one in Waltham (first time EVER), and the “usual” one at NERD), and then kicking off a two-day Boston Azure Bootcamp!

Details on ALL upcoming Boston-area events of interest to Azure folks (that I know about) can be found in this blog post about Boston-events in August and September. Those hosted by Boston Azure are also at www.bostonazure.org and the upcoming events page.

[Note - this post originally was mis-titled to say July and August instead of the correct August and September. I have not changed its URL, but did fix the title.]

Since this summary page is – by necessity – a point-in-time SNAPSHOT of what I see is going on, it will not necessarily be updated when event details change. So please always double-check with official event information!

Know of any more cloud events of interest to the Windows Azure community? Have any more information or corrections on the events listed? Please let us know in the comments.

Events are listed in the order in which they will occur.

August Events

1. Boston Azure User Group meeting: Special Guest John Garland on Windows Phone

• when: Thu 25-Aug-2011, 6:30 – 8:30 PM
• where: Hosted at NERD Center
• wifi: Wireless Internet access will be available
• food: Pizza and drinks will be provided
• cost: FREE
• what: Windows Phone 7 expert John Garland (a Senior Consultant at Wintellect) is the featured speaker. John’s presentation will show how the Windows Azure Toolkit for Windows Phone 7 can be used to quickly create Azure-enabled applications for the Windows Phone platform.  This talk will also include a discussion of some of the new features available in the Windows Phone “Mango” release due later this Fall (update), and how they can be used to further enhance the experience of working with Azure-based applications.
• more info: See the Boston Azure cloud user group site for more info, or join the (low volume) Boston Azure mailing list to keep most up to date
• register: here
• twitter: #bostonazure

September Events

2. Vermont Code Camp 2011

While not strictly a “Boston-area” event, this may be of interest still. I attended Vermont Code Camp 2010 as both an attendee (hitting lots of great sessions) and as a speaker (spoke about Azure, of course). There was a great deal of buzz and energy at the event. There was also major swag – some really good stuff. I don’t know what this year will hold, but they did set a pretty high bar last year across the board. I will be attending again this year (and have proposed a talk: Applying Architecture Patterns for Scalability and Reliability to the Windows Azure Cloud Platform). Hope to see you there!

• when: Saturday, September 10, 2011 8am–6pm
• where: Kalkin Hall on the University of Vermont campus in Burlington, VT
• wifi: (I think so)
• food: (Pretty sure)
• cost: FREE
• what: It’s a Code Camp! (from http://vtcodecamp.org/):  “Last year’s event had four rooms with sessions on .NET, PHP, Ruby, Python, and more. Two of the rooms had .NET topics and another had sessions on free/open source software. There was a fourth room where developers were introduced to various technologies that they may not use every day. Check back for details about Vermont Code Camp 2011 or follow us on Twitter.”
• more info: http://vtcodecamp.org/
• register: http://vtcodecamp.org/register/
• twitter: @VTCodeCamp

3. Cloudy Mondays

• when: Mon 19-Sep-2011, 5:00 – ?:?? PM
• where: Venture Development Center, 100 Morrissey Blvd, Boston, MA
• wifi: (not sure)
• food: (not sure, though food and beer were offered last time)
• cost: FREE
• what: (topics not posted yet, though generally cloud and cloud startup-related)
• more info: http://www.meetup.com/Cloudy-Mondays/
• register: here
• twitter: (not sure)

4. Boston Azure User Group meeting in Waltham: Special Guest Thom Robbins!

In this special event, the Boston Azure User Group is combining forces with both the Boston .NET Architecture Study Group and the New England ASP.NET Professionals Group to host this talk from Thom Robbins. Note the location is Waltham (not NERD).

• when: Wed 21-Sept-2011, 6:00 – 8:30 PM
• where: Hosted at Microsoft Office in Waltham (201 Jones Road, Waltham, MA 02451 – come to the 6th floor) – ample free parking is available
• wifi: Wireless Internet access is NOT available to attendees
• food: Pizza and drinks will be provided
• cost: FREE
• what: Special Guest speaker is Thom Robbins

Kentico CMS: A Case Study in Building for Today’s Web

Building software is a set of smart choices to meet the needs of your customers and the possibilities of technology.  Today’s Web demands that customers have a choice in how they deploy their applications. With over 7,000 websites in 84 countries, Kentico CMS for ASP.Net is delivered as a single code base for use as a cloud, hosted, or on-premise solution. With over 34 out of the box modules and everything built on a SQL Server backend – How did we do it?  What tradeoffs did we make? In this session we will answer that question and look at how to build a rich and compelling website using Windows Azure.

About Thom Robbins

Thom Robbins is the Chief Evangelist for Kentico Software. He is responsible for evangelizing Kentico CMS for ASP.NET with Web developers, Web designers and interactive agencies. Prior to joining Kentico, Thom joined Microsoft Corporation in 2000 and served in a number of executive positions.  Most recently, he led the Developer Audience Marketing group that was responsible for increasing developer satisfaction with the Microsoft platform. Thom also led the .NET Platform Product Management group responsible for customer adoption and implementation of the .NET Framework and Visual Studio. Thom was also a Principal Developer Evangelist working with developers across New England implementing .NET based solutions. A regular speaker and writer, he currently resides in Seattle with his wife and son. He can be reached at thomasr@kentico.com or on Twitter at @trobbins.

• more info: See the Boston Azure cloud user group site for more info, or join the (low volume) Boston Azure mailing list to keep most up to date.
• register: (registration will open in early September)
• twitter: #bostonazure

5. Boston Azure User Group meeting: Special Guest Brian Prince!

• when: Thu 29-Sep-2011, 6:00 – 8:30 PM
• where: Hosted at NERD Center
• wifi: Wireless Internet access will be available
• food: Pizza and drinks will be provided
• cost: FREE
• what: Brian Prince - from Microsoft, and co-author of the most excellent Azure in Action book – is our featured speaker.
• more info: See the Boston Azure cloud user group site for more info, or join the (low volume) Boston Azure mailing list to keep most up to date.
• register: (will open in early September)
• twitter:

6. Boston Azure Bootcamp

• when: Fri/Sat Sep 30 – Oct 1 (full days, but start/end times are tbd)
• where: Hosted at NERD Center
• wifi: Wireless Internet access will be available
• food: Expected to be provided, but details being worked out
• cost: FREE
• what:This free event is a two-day, hands-on bootcamp with the goal of learning a whole lot about the Windows Azure Platform. The primary programming environment will be Visual Studio 2010 (a free version is available). Coding will be primarily done in C#. (Other programming environments and other languages are available for Windows Azure. If you plan to program in other than Visual Studio and C#, please let us know it advance in the “Any Other Comments” section of the sign-up form.)The two days will largely consist of a sequence segments where important general topics in cloud computing will be introduced, and the Windows Azure approach will be discussed in detail. Each of these segments will include both a lecture by an Azure expert followed by a hands-on lab where you code a basic solution to get these concepts to really sink in. Azure experts will be in the room to help you with any questions or issues during labs.At the end of this two days, you will have learned key cloud and Windows Azure concepts, and have hands-on experience building, debugging, and deploying real applications. You need bring your own Azure-ready laptop – or let us know on the signup form if you would like a loaner – or would like to pair with someone for the coding part.
• more info: See the Boston Azure Bootcamp page on Eventbrite for more info
• register: Registration is LIMITED BY SPACE - register here
• twitter: #bostonazurebootcamp

Omissions? Corrections? Comments? Please leave a comment or reply on the Twitters!

The deck is attached at the bottom, after a few links of interest for folks who want to dig deeper.

Command Query Responsibility Segregation (CQRS):

• I’m a big fan of Bertrand Meyer‘s work, and I just learned that CQRS is based on his earlier CQR pattern
• Martin Fowler has a entry on CQRS (recently added, I will now read this)
• CQRS on Windows Azure  (MSDN Magazine article)
• .NET Rocks podcast: Episode 639 Udi Dahan Clarifies CQRS (That same podcast episode is also included in the Azure Top 40 feed that I curate: Azure Top 40 http://bit.ly/azuretop40)
• http://abdullin.com/cqrs/

Sharding is hard:

• Foursquare down for 11 hours from imbalanced shards: http://blog.foursquare.com/2010/10/05/so-that-was-a-bummer/ (though the High Scalability blog says it was longer)

NoSQL:

• NoSQL and the Windows Azure Platform whitepaper from Microsoft (which I found to be a very good read)

CAP Theorem:

• Wikipedia
• HA MySQL (interesting links)

PowerPoint slide deck used during my talk:

• DotNetArchStudyGroup_ArchitecturePatternsForScalabilityAndReliability_BillWilder_20July2011.pptx

A. The short answer is that while there is no built-in support for this, you can get the same effect by using a simple trick to add logic to your Startup Script to provide sufficient control. Before getting into that, let’s describe the problem in a bit more detail. Update 14-Oct-2011: Stop the presses!! This capability is now built into Windows Azure! Steve Marx has a blog post on the matter. I will leave this blog post around since the details in it may be of value for other scenarios.

Suppose you want to use ASP.NET MVC 3 in your Windows Azure Web Role. At the time of this writing, MVC 2 was installed in Azure, but not MVC 3. What to do? The short answer is, you can install MVC 3 along with your application at deployment time in the cloud. This type of prerequisite installation is most conveniently handled using a Startup Task. The idea is that I include the ASP.NET MVC 3 bits with my app, and define a Startup Task that installs these bits, and I can set things up easily so that these bits are already installed before my Web Role tries to run (via a Simple Startup Task). This is a pretty clean solution. (For more on Startup Tasks and how to configure them see How to Define Startup Tasks for a Role. For specific guidance on installing ASP.NET MVC 3 as a Startup Task, see Technique #2 in the ASP.NET MVC 3 in Windows Azure post on Steve Marx’s blog.)

Example Startup Task That ALWAYS Runs

Of course, installing ASP.NET MVC 3 is only one example. Here is another example – a Startup Task that enables support for ADSI with IIS – let’s call it enable-webmetabase.cmd. First, you would add the following entry to ServiceDefinition.csdef:

<?xml version=”1.0″ encoding=”utf-8″?>
<ServiceDefinition name=”NameOfMyAzureApp” xmlns=”http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition“>
…
<Startup>
<Task commandLine=”enable-webmetabase.cmd” executionContext=”elevated” taskType=”simple” />
</Startup>
…

The contents of enable-webmetabase.cmd would be something like the following (first enabling PowerShell scripting, then executing a specific script):

powershell -command “Set-ExecutionPolicy Unrestricted”
powershell .\enable-webmetabase.ps1

Though the specifics are not important for these instructions, since this script invokes a PowerShell script – let’s call it enable-webmetabase.ps1 – here is what that might look like:

Import-Module ServerManager
Add-WindowsFeature Web-Metabase

And as a final step, you would include both enable-webmetabase.cmd and enable-webmetabase.ps1 with your Visual Studio Project, and set the Copy to Output Directory property on each of these two files to be Copy always. Now, every time you deploy this Azure solution this Startup Task will be executed – and you can feel confident that you won’t have to worry about ADSI in IIS not being available (or whatever it is your Startup Tasks do for you).

Startup Tasks Run in Development Too

But what happens when I wish to deploy this solution on my development machine so I can quickly test it out while I am in the midst of development? Since the Windows Azure Platform has an outstanding local cloud simulation environment (which can be downloaded for free), “local” is the most common deployment target! It is not ideal that the Startup Tasks will run locally – I do not want to continually install ASP.NET MVC (or re-enable web metabase support, etc.) since that will just slow me down.

The Simple Workaround

I know of no built-in support that makes it easy for a Startup Task to “know” whether it is running in the cloud or in your local development environment. But it is simple to roll your own. Here’s what I do:

• Create an Environment Variable called AZURE_CLOUD_SIMULATION_ENVIRONMENT. While the exact value of this variable does not matter, for the sake of someone else who may see it and be puzzled, I set mine to be “set manually per http://bit.ly/rs5SRN” where the bit.ly link points back to this blog post.  It also doesn’t matter if the Environment Variable is user-specific or System-wide. If it is a shared development machine, I would make it System-wide (for all users).
• It is common practice when defining Startup Tasks to create command script using a .cmd file and have that be the Startup Task. Within the Startup Task .cmd file, use the defined keyword (supported in the command shells of recent versions of Windows, such as those you will be using for Azure development and deployment) to add a little logic so that you run only those commands you wish to execute in the current environment.

To set up the AZURE_CLOUD_SIMULATION_ENVIRONMENT environment variable:

1. Run SystemPropertiesAdvanced.exe to bring up the System Properties dialog box:
   
2. Click the Environment Variables button to bring up the Environment Variables dialog box:
   
3. Click the New… button at the bottom to bring up the New System Variable dialog box:
   
4. Type AZURE_CLOUD_SIMULATION_ENVIRONMENT into the Variable name field, and set manually per http://bit.ly/rs5SRN into the Variable value field:
   
5. Hit a few OK buttons and you’ll be done.

Revised “Smart” Startup Task

Of course the trick is that the AZURE_CLOUD_SIMULATION_ENVIRONMENT variable will only be set on development machines, so it will NOT be set in the real cloud, getting you the desired results. Here is the same enable-webmetabase.cmd Startup Task script from above, except rewritten so that when you run it locally it will not do anything to your development machine.

if defined AZURE_CLOUD_SIMULATION_ENVIRONMENT goto SKIP

powershell -command “Set-ExecutionPolicy Unrestricted”
powershell .\enable-webmetabase.ps1

:SKIP

The line “if defined AZURE_CLOUD_SIMULATION_ENVIRONMENT goto SKIP” simply checks whether AZURE_CLOUD_SIMULATION_ENVIRONMENT exists in the environment, and if it does exist, the script jumps over the two powershell lines. This is pretty handy!

Again, in summary, if you follow the very simple approach in this post, the AZURE_CLOUD_SIMULATION_ENVIRONMENT will exist only on development machines – in the simulated cloud – and not out in the “real” cloud.

Not to be Confused with RoleEnvironment.IsAvailable

There is another technique – that is built into Azure – which you can use in code that needs to behave one way when running under Windows Azure, and another way when not running under Windows Azure: RoleEnvironment.IsAvailable. This is good for code that might be deployed both in, say, an Azure Web Role and in a non-Azure ASP.NET web site. For Azure applications, RoleEnvironment.IsAvailable will be true for both the local development machine and when deployed into the public cloud.

While RoleEnvironment.IsAvailable and AZURE_CLOUD_SIMULATION_ENVIRONMENT serve different purposes, they are complementary and can be used together.

For more information on RoleEnvironment.IsAvailable, there is documentation and a good description of its use.

Other Uses for the Technique

Maybe you want to do certain things ONLY in your development environment. For example, perhaps you wish to launch Fiddler. Or maybe uninstall a Windows Service (via InstallUtil /u <service exe name>). Whatever your needs – you can use the same simple technique to make this easy. The following syntax is also supported – each bullet being a single line (though some of them may appear on more than one line in this blog post):

• if defined AZURE_CLOUD_SIMULATION_ENVIRONMENT (echo AZURE_CLOUD_SIMULATION_ENVIRONMENT equals %AZURE_CLOUD_SIMULATION_ENVIRONMENT%) else (echo AZURE_CLOUD_SIMULATION_ENVIRONMENT is NOT defined)
• if defined AZURE_CLOUD_SIMULATION_ENVIRONMENT echo DOING SOMETHING
• if NOT defined AZURE_CLOUD_SIMULATION_ENVIRONMENT echo DOING SOMETHING ELSE

Is this useful? Did I leave out something interesting or get something wrong? Please let me know in the comments! Think other people might be interested? Spread the word!

As I was working through the Interesting Finds: June 23, 2011 posts on Cloud Computing I drilled into one from the Official Google Enterprise Blog titled Businesses innovate and scale faster on Google App Engine. It is a very well crafted post which includes some great customer quotes and a couple of videos. I must say, it does a great job of promoting the value in the Google App Engine (GAE) platform, essentially as mini-case studies. Well done!

What struck me as particulary interesting about this post, however, is the types of benefits the GAE customers say they value:

• The first embedded video features Dan Murray, founder and managing director of a cloud-based SEC-filings company called WebFilings. Mr. Murray mentions they needed a platform that would be secure and would support rapid growth. He goes on (at 1:50 into the video): “Google App Engine provides a platform that takes the infrastructure management off of our hands, we don’t have to worry about it, so it’s easy for us to build and deploy apps. For us right now it’s about execution and making sure that we’re scaling our business, while App Engine provides the ability to scale the technology and platform.”
• The second embedded video features Jessica Stanton from the famous Evite event invitation site. Ms. Stanton mentions (at 0:52 into the video) “the things that App Engine especially desirable for us are the autoscaling and … monitoring systems” that Google provides. Near the end (at 1:12 into the video) she emphasises: “the opportunity that App Engine has afforded to us is more time to do what we need to do. To just get things done and to get new features out and not have to worry so much about load and things going down because we take on 16-18 million unique users a month.  It’s really nice to see instances spin up and come down and we never had to touch anything.”
• Quote from Gary Koelling of Best Buy: “… we don’t have to spend any time doing system administration or setting up servers, which allows us to focus on the development and testing new ideas.”

The funny thing is, the benefits touted are really the benefits of Platform as a Service (PaaS). These services could just as easily have been built on the Windows Azure Platform!

• Mr. Murray from WebFilings mentioned the need for a a platform based on a great security infrastructure. Both Microsoft and Google have some of the industry’s best and brightest working for them in their state-of-the art, world-class data centers. Here are some good resources relating to security in the Windows Azure data centers. If you want a secure data center and secure platform, I don’t think you can go wrong with either Microsoft or Google. (Frankly, I expect you are more likely to have problems – including with cost and security – if you roll your own data center. Your company will not have the top experts in the world on your payroll.)
• Both Ms. Stanton from Evite and Mr. Koelling of Best Buy emphasize that they benefit from being able to focus on building software – and not being distracted by needing to worry about infrastructure. This is what Platform as a Service (PaaS) is all about. Both Microsoft and Google offer PaaS. GAE supports apps which run on the JVM (e.g., Java) and apps written in Python. Windows Azure supports programming in any .NET language (e.g., C#), plus a plethora of other platforms that run on Windows – PHP, Java, Python, Ruby, C++, and so many more. GAE has database support with a query language they call GQL, and Azure has SQL Azure which supports the regular SQL you know and love. Each platform has other features as well, making it a place where you can focus on your app – not your infrastructure.
• Ms. Stanton mentions that they have a team of 5 developers. I wonder how large the Evite team would need to be if they were not running on PaaS?

Mr. Murray from WebFilings mentions that they began using GAE back 2008 – and the Windows Azure Platform was not announced until late in 2008 (at Microsoft PDC in November 2008), so that was not an option yet for them. It is not mentioned when the other companies began to use GAE. If they were starting today, I wonder how many would choose Azure?

While hybrid cloud models will surely be around for many companies for a long time – we won’t all move to the cloud over night – the economics of moving to the cloud are too compelling to ignore. Many newer companies are heading directly into the cloud – never owning any infrastructure.

One of the the costs in managing a hybrid cloud model – where some data is on-prem, some data is in the cloud – is the raw data transfer when you copy bits to or from the cloud. This can cost you real money: for example, in the USA and Europe, both the Windows Azure Platform and the Amazon S3 services charge $0.10 per GB to move the data into the datacenter. If you have a huge amount of data, that cost can add up.

Announced today on the Windows Azure blog, as of July 1, 2011 the Windows Azure datacenters will no longer have a data transfer charge for inbound data. What are the implications?

Here are a few I can think of:

1. Overall cost savings can only help general overall cloud adoption
2. Backing up data from on-prem into the cloud just got more interesting (good point I stole from maura )
3. HPC applications which may have a lot of data to move into the cloud for processing – but may never need that data to come out of the cloud (other than in much smaller digested form) just became more appealing
4. Use of Windows Azure as a collection for disparate data sources from around the internet – for management, aggregation, or analysis – just became more attractive
5. While experimentation on the cloud has always been cheaper than buying boxes, it now makes it even simpler and cheaper to try out something big in the cloud because you are now an even smaller blip on the corporate cost radar – go ahead, upload that Big Data and run your experiment – you can always delete it when you are done
6. There are cloud storage vendors who sit on top of big cloud storage vendor platforms, such as on Azure and Amazon – if I was one of these vendors, I would be delighted – business just got a little easier

Points 2, 3, 4, and 5 above all deal with an asymmetric use of bandwidth where the amount of data moving into the cloud is far less than the amount of data leaving the cloud. With backups – your hope is to NEVER need to pull that data – but it is there in the event you need it. With HPC – in many cases you just want answers or insights – you may not care about all the raw data. With data aggregation – you probably just want some reports. With one-off experiments, when you are finished you just delete all the storage containers – so simple!

This is a big and interesting step towards accelerating cloud computing adoption generally, and Windows Azure specifically. This friction-reducing move brings us closer to a world where we don’t ask “should we be in the cloud?” but rather “why aren’t we in the cloud?”

[Update 23-June-2011: Joel Bennett posted his slide deck from the talk.] 
[Update 05-July-2011: Added another handy link to post from Patrick Butler called Developing and Debugging Azure Management API CmdLets.]

Some of the pointers:

• Go get the PowerShell Community Extensions (from codeplex)
• You can use the PS command line to CD into folders/directories, look at files, etc. — but you can also look at the Registry or your Certificate Store as if they were directories!
• There are no plural nouns in PS (e.g., get-provider, not get-providers)
• Learn these commands first: Get-Command, Get-Help, Get-Member, Select-Object, Where-Object, Format-Table, … others you can learn later
• Somebody needs to write a PowerShell Provider for Azure Storage
• Joel created an open-shell WPF-ish PowerShell shell called POSH

Try some commands:

• dir | get-objec
• dir | fl *  –formats as lis
• get-verb | fw -col
• get-verb | fw -col 6 -groupby Group
• Get-ExecutionPolicy
• dir | where { $_.PSIsContainer } – where $_ is “the thing over there (directory)”
• dir | select CreationTime, Name | gm
• dir | select * –will look different than command above
• $global:foo = “some value”
• cd c:\windows\system32\windowspowershell\v1.0… see Help.format.ps1xml controls the default output formatting properties for object types not already known by PowerShell – can have property names, even script blocks in it – very powerful
• # is single-line comment char; <# … #> for multi-line comments
• You can create aliases for scripts
• Powershell is an interpreted scripting language
• Can access WinForms, WPF, lots of stuff.. though not Threading

Three ways to manage Azure from PowerShell

1. Remoting
2. WASM (Windows Azure Services Management commandlets) – superceded by http://wappowershell.codeplex.com/ – developed by Development Evangelist group (e.g., Vittorio)
3. Cerebrata (3rd party, commercial)

Remoting:

• Need to get some goodness in a Startup Script, along with crecentials
• Set OS Family = 2 (so you get Windows Server 2008 R2)
• Need a certificate – can be self-signed

WAP PowerShell:

• 36 Cmdlets
• “the 80% library”
• very good example

Cerebrata Cmdlets

• Requires .NET 4.0 (which is different than baseline support for PS, which is .NET CLR 2.0
• $70
• 114 Cmdlets
• Cerebrata
• gcm –mo cerebrata | gr0up Noun | sort

Snap-ins need to be in the GAC – so put WAP PowerShell stuff where you want to keep them, since that’s where they’ll be built — or build the file in Visual Studio

• Add-Module is for SNAPINS
• IPMO is for ImportModule for Modules
• ipmo AzurePlatform
• gcm –mo AzurePlatform

PowerShell has something called “splatting”

• Starting with a hashtable… put in the parms you’ll need
• variables start with $
• retrieving (splatting) starts with @

Both cerebrata and WAP are snap-ins

WHAT FOLLOWS… are somewhat random notes I captured…

Get-Certificate $azure | Get-HostedCertificateStore

Your personal profile for PowerShell lives in c:\Users\YOURNAME\Documents\WindowsPowerShell\Modules\AzurePlatform as Startup.ps1 (?)

Two kinds of errors in PowerShell: 1. Terminating Errors (exceptions, can be “trapped” or use try/catch as of PS2) and 2. Non-Terminating Errors which are harder to deal with

$? ==> did the last command succeed

dir doesnotexist –ev er –ea “”

$er[0].categoryinfo

“Don’t Produce Snap-ins!” Here’s why: to figure out what is in there (get-command –Module

Get-Module –ListAvailable

- run the above on AZure and see “NetworkLoadBalancingCl…” – is this Azure relate

OTHER INTERESTING POWERSHELL/AZURE LINKS

• http://poshcode.org/
• http://pscx.codeplex.com
• http://wappowershell.codeplex.com/workitem/list/basic
• http://blogs.msdn.com/b/patrick_butler_monterde/archive/2011/06/30/developing-and-debugging-azure-management-api-cmdlets.aspx

To specificy your intent to use a Local Storage location, you add an entry to ServiceDefinition.csdef under the specific role from which you plan to access it. For example:

<LocalStorage name="LocalTempFolder" sizeInMB="11"
      cleanOnRoleRecycle="true" />

You can read more about the details over in Neil Mackenzie’s post, but the main thing you need to do is call a method to access the full path to the read/write folder associated with the name you provided (e.g., “LocalTempFolder” in the config snippet above). The method call looks like this:

LocalResource localResource =
      RoleEnvironment.GetLocalResource("LocalTempFolder");
var pathToReadWriteFolder = localResource.RootPath;
var pathToFileName = pathToReadWriteFolder + "foo.txt";

Now you can use ”the usual” classes to write and read these files. But calling RoleEnvironment.GetLocalResource only works from within the safe confines of a your Role code – as in a Worker Role or Web Role – you know, the process that inherits from (and completes) the RoleEntryPoint abstract class. What happens if I am inside of a Windows Service?

Your Windows Service Has Super Powers

Well… your Windows Service does not exactly have Super Powers, but it does have powers and abilities far above those of ordinary Roles. This is due to the differences in their security contexts. Your Windows Service runs as the very powerful LocalSystem account, while your Roles run as a lower priviledge user. Due to this, your Windows Service can do things your Role can’t, such as write to the file system generally, access Active Directory commands, and more.

[Your Startup Tasks might also have more powers than your Roles, if you configure them to run with elevated privileges using executionContext="elevated" as in:
<Task commandLine="startuptask.cmd" executionContext="elevated" />
See also David Aiken's post on running a startup task as a specific user.]

However, there are some things your Windows Service can’t do, but that your Role can: access RoleEnvironment!

Problem Querying Local Storage from a Windows Service

Inside of a Windows Service (which is outside of the Role environment), the RoleEnvironment object is not populated. So, for example, you cannot call

RoleEnvironment.GetLocalResource("LocalTempFolder")

and expect to get a useful result back. Rather, an exception will be raised.

But here’s a trick: it turns out that calling RoleEnvironment.GetLocalResource returns the location of the folder, but it is just the location of a folder on disk at this point – this folder can be accessed by any process that knows about it. So how about if your Web  or Worker Role could let the Windows Service know where its storage location happens to be? (As an aside, we have a good idea where they might ultimately end up on disk in practice (see last section of this post) – but of course subject to variability and change – but it is useful if you want to poke around on your local machine or through Remote Desktop to an instance in the cloud.)

The Trick: Pass the Local Storage location into your Windows Service

If you are deploying a Windows Service along with your Role, you will need to install the Windows Service and you will need to start the Windows Service. A reasonable way to install your Windows Service is to use the handy InstallUtil.exe program that is included with .NET. Here is how you might invoke it:

%windir%\microsoft.net\framework\v4.0.30319\installutil.exe
      MyWindowsService.exe

Now the Windows Service is installed, but not running; you still need to start it. Here is a reasonable way to start it:

net start MyWindowsServiceName

Typically, both the InstallUtil and net start commands would be issued (probably in a .bat or .cmd file) from a Startup Task. But there is another way to start an installed Windows Service which allows some additional control over it, such as the ability to pass it arguments. This is done with a few lines of code from within the OnStart method of your Role, such as in the following code snippet which uses the .NET ServiceController class to get the job done:

var windowsServiceController =
      new System.ServiceProcess.ServiceController
            ("MyWindowsServiceName");
System.Diagnostics.Debug.Assert(
      windowsServiceController.Status ==  
      windowsServiceControllerStatus.Stopped);
windowsServiceController.Start();

Putting together both acquiring the Local Storage location and starting the Windows Service, your code might look like the following:

string[] args = { 
      RoleEnvironment.GetLocalResource
            ("LocalTempFolder").RootPath
      }
var windowsServiceController =
      new System.ServiceProcess.ServiceController  
            ("MyWindowsServiceName");
System.Diagnostics.Debug.Assert(
      windowsServiceController.Status == 
      windowsServiceControllerStatus.Stopped);
// pass in Local Storage location
windowsServiceController.Start(args);

Within your Windows Service’s OnStart method you will need to pick up the arguments passed in, which at that point has nothing specific to Azure. Your code might look like the following:

protected override void OnStart(string[] args)
{
   var myTempFolderPath = args[0];
   // ...
}

That oughta do it! Please let me know in the comments if you find this useful.

• Development tools for Azure are available for free
• Azure provides an excellent cloud simulation environment on your desktop
• Not every application is a good fit for the cloud – for example, a small app that doesn’t need to scale and need not be highly available might fit better with a less costly hosted solution
• When comparing costs of Azure with other approaches, keep in mind that Azure is a robust, highly available, scable, flexible platform – what you get for your dollar is often of much greater value than the dollar you spend for some other types of solution
• Azure affords fantastic cost-saving opportunities through the flexible scale down model – don’t need a data or compute resource anymore? stop using it, and you’ll stop paying for it. Try that kind of “on-a-dime” manuever with a hosted solution, with hardware you purchase, or rack space you lease
• Azure services are available a la carte – though of course they are also a fantastic approach when used all together
• There are a number of ways to auto-scale, though don’t underestimate the boundary conditions and there are also some nuances

Since I did only give a taste of the Access Control Service, the plan discussed was for me to come back after the summer for a deeper dive into that fascinating topic.

Although I did not proceed linearly through it, here is the deck I used: [neasp.net-bill_wilder-intro-cloud-azure-acs-for-asp.net-devs-15-june-2011]. The Access Control Service (ACS) content did not include any slides – all talk and demo – though I gave a similar talk at Boston Azure back in February that used the following deck: [Solving the Identity Crisis-using-azure-cloud-access-control-service-(ACS)-talk-24-Feb-2011-Boston-Azure-User-Group] (Since then, the final ACS v2 has been release and changed a few things.)

[Update 12-Oct-2012] This post only applies to Windows Azure Cloud Services (which have Web Roles and Worker Roles). This post was written a year before Windows Azure Web Sites and Windows Azure Virtual Machines (including Windows and Linux flavors) were announced and does not apply to either of them.

Q. Can I write to the file system from an application running on Windows Azure?

A. The short answer is that, yes, you can. The longer answer involves better approaches to persisting data in Windows Azure, plus a couple of caveats in writing data to (virtual) hard disks attached to the (virtual) machines on which your application is deployed.

Any of your code running in either (a) ASP.NET (e.g., default.aspx or default.aspx.cs) or (b) WebRole.cs/WorkerRole.cs (e.g., methods OnStartup, OnRun, and OnStop which are derived from RoleEntryPoint class) will not have permission to write to the file system. This. is. a. good. thing.®

To be clear, if you have code that currently writes to fixed locations on the file system, you will probably need to change it. For example, your ASP.NET or Role code cannot directly create/write the file c:\foo.txt – the permissions are against you, so Windows will not allow it. (To round out the picture though… You can write to the file system directly if you are running in an elevated Startup Task, but cannot write to it from a limited Startup Task. For more on Startup Tasks and how to configure them see How to Define Startup Tasks for a Role.)

The best option is usually to use one of the cloud-native solutions: use one of the Windows Azure Storage Services or use SQL Azure. These services are all built into Windows Azure for the purpose of supporting scalable, reliable, highly available storage. In practice, this means choosing among Windows Azure Blob storage, Windows Azure Table storage, or SQL Azure.

The second-best option is usually to use a Windows Azure Cloud Drive – which is an abstraction that sits on top of Blob storage (Page blobs, specifically) – and looks and acts a lot like an old-fashioned hard disk. You can access it with a drive letter (though you won’t know the drive letter until deployment time!), it can be mounted by and read from multiple of your role instances, but only one of these at a time will be able to mount it for updating. The Windows Azure Drive feature is really there for backward compatibility – to make it easier to migrate existing applications into the cloud without having to change them. Learn more from Neil Mackenzie’s detailed post on Azure Drives.

The third-best option is usually to use the local hard disk. (And this is what the original FAQ question specifically asked about.) Read on…

Writing Data to Local Drive from Windows Azure Role

So… Can I write to the hard disk? Yes. And you have a decent amount of disk at your disposal, depending on role size. Using Azure APIs to write to disk on your role is known as writing to Local Storage. You will need to configure some space in Local Storage from your ServiceDefinition.csdef by giving that space (a) a name, (b) a size, and (c) indicating whether the data there should survive basic role recycles (via cleanOnRoleRecycle). Note – cleanOnRoleRecycle does not guarantee your data will survive – it is just a hint to the Fabric Controller that, if it is available, should it leave it around or clean it up.  That limitation is fine for data that is easily recalculated or generated when the role starts up – so there are some good use cases for this data, even for cloud-native applications – think of it as a handy place for a local cache. (Up above I refer to this as the usually being the third-best option. But maybe it is the best option! In some use cases it might be. One good example might be if you were simply exploding a ZIP file that was pulled from blob storage, but there are others too. But let’s get back to Local Storage…)

Here is the snippet from ServiceDefinition.csdef:

...
<LocalResources>
<LocalStorage name="SomeLocationForCache"
cleanOnRoleRecycle="false"
sizeInMB="10" />
</LocalResources>
...

You can also use the Windows Azure Tools for Visual Studio user interface to edit these values; double-click on the role you wish to configure from the Roles list in your Windows Azure solution. This is the easiest approach.

Once specified, the named Local Storage area can be written to and read from using code similar to the following:

// reference Microsoft.WindowsAzure.ServiceRuntime.dll from SDK
// (probably in C:\Program Files\Windows Azure SDK\v1.4\ref)
const string azureLocalResourceNameFromServiceDefinition =
"SomeLocationForCache";
var azureLocalResource =
RoleEnvironment.GetLocalResource(
azureLocalResourceNameFromServiceDefinition);
var filepath =
azureLocalResource.RootPath +
"myCacheFile.xml"; // build full path to file
// the rest of the code is plain old reading and writing of files
// using the 'filepath' variable immediately above

Learn more from Neil Mackenzie’s blog post on Local Storage.

Writing to TEMP Folder from Windows Azure Role

How about writing temporary files? Is that supported? Yes, same as in Windows. For example, in .NET one can get a temporary scratch space and write to it using code similar to the following:

var filepath = System.IO.Path.GetTempFileName();
System.IO.File.WriteAllText(filepath, "some text");


Do Not Use Environment.SpecialFolder Locations in Azure

You may also have some existing code which writes files for the currently logged in user. Check the Environment.SpecialFolder Enumeration for the full list, but one examples is Environment.SpecialFolder.ApplicationData. You would access this location with code such as the following:

string filepath = Environment.GetFolderPath(
Environment.SpecialFolder.ApplicationData,
Environment.SpecialFolderOption.DoNotVerify);

You will find that your ASP.NET code will be able to write to this location, but that is almost certainly not what you want! By default, the user account under which you will be saving this data is one that is generated when your role is deployed – something like RD00155D328831$ – not some IPrincipal from your Windows domain.

Further, for data you care about, you don’t want to store data it in the local file system in Windows Azure. Better options should be apparent from earlier points made in this article.

And, finally, you may prefer the elegance of claims-based federated authentication using the AppFabric Access Control Service.

Writing to File System from Windows Service in Windows Azure Role

If you want to do something unusual, like write to the file system from outside of Role’s code, there are ways to write to the file system from a Windows Service or a Startup Task (though be sure to run your Startup Task with elevated permissions).

Is this useful? Did I leave out something interesting or get something wrong? Please let me know in the comments! Think other people might be interested? Spread the word!

A. The anwer, of course, depends on what you are doing. Official pricing information is available on the Windows Azure Pricing site, and to help you model pricing for your application you can check out the latest Windows Azure Pricing Calculator. Also, the Microsoft Assessment and Planning (MAP) Toolkit is now in beta.

Simple cost example: Running One Instance of a Small Compute Role costs 12¢ per hour, which is around $1052 per year. A SQL Azure instance that holds up to 1 GB costs $9.99 per month. If you have Two Small Compute Instances & 1 GB of SQL Azure storage, plus throwing in some bandwidth use, a dash of Content Delivery Network (CDN) use, and your baseline cost might start at around $2,225.

Update 22-June-2011: The pricing calculators may not reflect this interesting development: data transfer into the Azure Data Centers becomes free on July 1, 2011. See: http://blog.codingoutloud.com/2011/06/22/free-data-transfer-into-azure-datacenters-is-a-big-deal/ and http://blogs.msdn.com/b/windowsazure/archive/2011/06/22/announcing-free-ingress-for-all-windows-azure-customers-starting-july-1st-2011.aspx

But it is not always that simple: this is just the simplest, pay-as-you-go model! In the short term, there are many deals, offers, and trials – some free. There are Azure benefits included with MSDN. And long term there are ways to get better rates if you have an Enterprise Agreement with Microsoft, or by selecting a more predictable baseline than pay-as-you-go. See the Windows Azure Pricing site for current options.

Further, when comparing costs with other options, consider a few factors:

• The SQL Azure storage is really a SQL Azure cluster of three instances giving you storage redundancy (3 copies of every byte), high availability (with automagic failover), high performance, and other advanced capabilities.
• Similarly, every byte written to Windows Azure Storage (blobs, tables, and queues) is stored as three copies.
• Running two Small Compute instances of a role comes with a 99.9% uptime Service Level Agreement (SLA), and a 99.95% connectivity SLA. Read more about the Compute, SQL Azure, and other Windows Azure Platform SLAs here.
• Since Windows Azure is Platform as a Service (PaaS), be careful to also consider that you may have fewer hassles and lower engineering and operational costs – these are lower staff-time costs – if you are comparing to an Infrastructure as a Service (IaaS) offering.

While you are at it, consider checking out some of these related third-party offerings:

• CloudValue – A whole company dedicated to understanding and optimizing costs in moving to the cloud. I saw them at TechEd Atlanta in May 2011. They (a) presented a generally useful talk on Cost-Oriented Development (not specific to their technology, though we saw a glimpse of their Visual Studio integrated cost analyzer); and they (b) had a booth so people could check out their CloudValue Online Service Tracking (COST) service which provides ongoing analysis of your costs in the Windows Azure cloud. I am trying out the COST product now that my beta request has been approved!
• CloudTally – A service offering from Red Gate Software – currently in beta, and currently free – will keep an eye on your SQL Azure database instance and based on how much data you have in it over time, it will report your daily storage costs via email. I’ve been using this for a few months. The data isn’t very sophisticated – of the “you spent $3.21 yesterday” variety – but I think they are considering some enhancements (I even sent them some suggestions).
• Windows Azure Migration Scanner – An open source tool created by Neudesic to help you identify changes your application might require in order to make it ready for Azure. This is not specifically a cost-analysis tool, but is useful from a cost-analysis point of view since it can help you predict operational costs of the Azure-ready version of your application – for example if you will make changes to leverage the reliable queue service in Windows Azure Storage, you will know enough to model this. Read David Pallmann’s introduction to the scanner, where he also mentions some other tools.
• Greybox – While not a core tool for calculating costs, it is a interesting open source utility to help you avoid the “I-deployed-to-Azure-for-testing-purposes-but-forgot-all-about-it” memory lapse. (If deployed, you pay – whether you are using it or not. Like an apartment – you pay for it, even while you are at work – though Azure has awesome capabilities for you to “move out of your cloud apartment” during times when you don’t need it!) You may not need it, but its existance illustrates an important lesson!

Credit: I discovered the new Windows Azure Pricing Calcular from http://twitter.com/#!/dhamdhere/status/73056679599677440.

Is this useful? Did I leave out something interesting or get something wrong? Please let me know in the comments! Think other people might be interested? Spread the word!

Here’s how my day went:

1. Spoke about the cloud and Azure’s role in the cloud. Special thanks to Corinne, Sandra, and Matthew for the excellent questions and discussion. Here is the slide deck (new-hampshire-code-camp-3-concord-bill_wilder-demystifying_cloud_computing_and_azure-04-june-2011.ppt) – though I didn’t use much of it! – we freestyled a lot. Of particular interest to attendees of this talk. check out my post called “Azure FAQ: How much will it cost me to run my application on Windows Azure?” (actually posted “tomorrow” – the day after I posted this note from code camp).
2. Was torn between Phil Denoncourt‘s talk on “25 Things I’ve Learned about C# over the past 10 years” and Andy Novick‘s talk on SQL Azure. Ended up hanging out for Andy’s talk to see if there was anything new in SQL Azure and to get his take on the awesomeness that is SQL Azure Federations.
3. Lunch break
4. Spoke about Architecture Patterns for the Cloud. Here is the slide deck: New-Hampshire-Code-Camp-3-Concord-_bill_wilder_-_cloud_scalability_patterns_in_windows_azure_platform_-_04-june-2011 – we talked focused on three specific scalability patterns and how you might implement those on the Windows Azure Platform: Sharding, NoSQL Data (and eventual consistency), and CQRS.
5. Watched Udai Ramachandran talk about Windows Azure AppFabric Caching in the final session.