Category Archives: Cloud Computing

January 2010 Boston Azure User Group Meeting Notes

This was the third meeting of the Boston Azure User Group! (You can get on the group mailing list here.)

We watched a clip from the first day of PDC where Ray Ozzie and others talk Azure in the keynote

Discussed idea of an Azure Firestarter event – possibly for May 8, 2010 – and this seems to flow nicely from our scheduled April meeting where Jason Haley is scheduled to talk about getting started programming in Azure, such as with the Azure SDK.

Ben Day spoke on Windows Azure storage. Some quick notes / points from his talk:

  • Relational databases have a schema – all rows in a table have same columns, structure is defined before pouring in any data, data is not repeated (third-normal form breaks out data to appear only once – no redundancy)
  • … database will manage transactions across tables
  • … though mixed with replication can provide performance challenges
  • This changes for Azure Table Storage!
  • … though Azure Table Storage can scale way better – horizontally (“out”) whereas traditional SQL RDBMs tend to scale best vertically (“up”) – to larger boxes – which is more limiting and tends to be more expensive.
  • Do we need to rethink what needs to be transactional? Can we use a simplified transactional model – such as just within one table – or one instance of one table…
  • … compensating transactions are another approach
  • Securely storing data
    • Encrypt (compute is cheap)
    • If you encrypt a key, it won’t work for indexing
    • Search is harder if you encrypt
  • String columns have a 64KB size limit for Table Storage – so reference larger objects in a Blob
  • Unit testability
    • Abstract away you r persistent store, such as with Repository pattern – so you can unit test
    • Encapsulate business logic, such as with Service Layer and Domain Model patterns
    • Extract logic from UI using MVP (Model View Presenter)
    • Use Mock objects
  • Ben will come back to finish the story!

Around 23 attended.

December 2010 Boston Azure User Group Meeting Notes

Second meeting of Boston Azure User Group

Guest speakers were Michael Stiefel and Mark Eisenberg

Meeting was held December 3, 2009 at the Microsoft NERD

We opened with Boston Azure Theater, kicking off a few minutes after 6:00.  For around 45 minutes we watched a video of Microsoft Director Manuvir Das’ PDC talk A Lap Around the Windows Azure Platform.

From there, Microsoft’s Mark Eisenberg walked us through a summary of key Windows Azure announcements made at (or right before) the Microsoft PDC in November. The deck Mark used is available BAUG_PDCHighlights. There was a lot of interest in the announcement details and in the pricing model.

Our keynote speaker, Michael Stiefel, followed with a detailed look into the project “Dallas” announcement, showcasing the Dallas “Data as a Service” platform, working through sample apps, a custom mashup – with code, demonstrating the straight-forward programming model (ATOM feeds), and showing use of the data directly within Excel. Michael wrapped up by reviewing the business model – and discussing the interesting possibilities (publishers can publish – and others can consume – data so much more easily than today since Microsoft will have eliminated the “contract friction” we’d have if every consumer had to strike a deal with every publisher).

October 2009 Boston Azure User Group Meeting Notes

Meeting #1 of Boston Azure User Group!

Keynote speaker: Brian Lambert of Microsoft

Meeting was held Thursday October 22, 2009 at Microsoft NERD in Cambridge, MA

Mike Werner (Microsoft evangelist for Azure in the Northeast) introduced the user group at this innaugural meeting, plugged the upcoming Microsoft PDC, then introduced Bill Wilder, the guy who started the Boston Azure User Group.

Bill Wilder provided an overview of the group (powerpoint slides):

  • All meetings will have Windows Azure focus
  • Any technology or business concept – cloud models, Microsoft dev technologies, tools, and so forth – is fair game, but must be presented with appropriate Azure slant: they need to be related back to Azure or else they are not appropriate for Boston Azure User Group meetings

Brian Lambert spoke on how to build applications for Windows Azure.

Bill’s raw notes from Brian’s talk:

  • Can call unmanaged DLLs through Full Trust
  • “Fabric Controller worries about the “shape” of your app”

Worker Roles

  • Background processing – utility computing
  • May communicate with outside services
  • Not externally visible to client
  • Queues ~ how we communicate to a Worker Role ~ since both Worker Roles and Web Roles can talk to storage (like Queues), this is a good medium

There are public and private containers in blog storage. Public is the only accessible to clients (w/o keys that is).

Partitioned for scale

Blobs = ? “cloud files”

  • Up to 50 GB / blob + 8k of metadata

Azure Dev Workflow:

  • Add some Web Role(s)
  • Add 0+ Worker roles
  • Add Service  Configuration
  • Add Service Package
  • Upload to Windows Azure Management Portal
  • Deploy to Windows Azure Fabric Controller
  • Provision / Run the roles and storage needs; monitors the health

Sticky Storage ?

….

Azure Tools + Azure SDK

….

For “dev” + “test” + “etc.” – may want to use REAL azure accounts – not just the “other account” you have in your main production account

….

Need to handle the “at least one worker role will work on it” contract – your application needs to be designed for this…

“poison message” – due to bug or oversight or bad data, a message from the queue will NEVER be successfully handled. App needs to handle this currently.

Queue timeouts can have different values – could be, say, different for different message types.

….

Worker Role => a class that inherits from WorkerRoleMain (what was this class name??)

Fabric occassionally calls GetHealthStatus() which returns a RoleStatus. If you tell the Fabric you are Unhealthy, the Fabric may slay you? Maybe your NIC card is flakey, so your performance to external web services is too slow?

….

Public container for blob – for actual image (full size) linked to from the thumb.

“Fiddler is your friend when you’re working … watch your requests.”

Dev Fabric + Cloud Storage is an excellent mode to run in for effective debugging – can watch the message traffic with Fiddler.

“There is no debugging in the cloud. There’s logging.”

RoleManager can also write to “local storage” [[more interesting access / features coming soon!]] – then can go to Portal and use “configuration” button to update your config file to “copy logs” (did they move it?)

===========

can even bring web site up with http://localhost:8020 

WorkerRole is very simple to just run it in a process

Use Test Running to fire up a WorkerRole

  • Tight dev cycle
  • Can also now run a performance profile

Tricks: log in “real fabric”, write to console in “dev fabric”

Intuit as a Service

My notes from Intuit talk at Boston Cloud Meetup on Nov 3, 2009.

Per Alex Barnett of Intuit – only 12% of 4m Small Businesses not willing to use hosted data with SaaS.

Webware 100 finalists… best software solutions.

How do cloud-based apps Integrate?

  • Finance system know the project in Basecamp is done
  • Calendar know about CRM

Simplify by mapping services to a COMMON DATA MODEL (this is a REALLY big deal, IMHO).

QuickBooks have 4 m customers who have effectively agreed on a database schema… exposed an API to this data store. Nice…

The “Small Business Data Cloud” – available today from Intuit – as the Intuit Partner Platform.

native apps

  • Flex applications hold the logic
  • Server-side Java coming in development

federated apps

  • identity
  • data
  • billing
  • most of the action is over here (rather than native apps)

They expect to ship 1.7 m units of QB 10 over next 6-12 months – this will really launch the platform.

4 m users spending 4 b hours per year of screen time

25 million users within these Small Businesses

FAST TIME TO VALUE through SaaS solutions.

Intuit keeps 20% of the rate, rest goes to developers

Can have trial-periods, entitlements (monthly-charged extra features), more options coming next year.

Developers support the apps – Intuit supports everything else.

http://code.intuit.com – some Open Source in here

  • some code uses SAML
  • support Java, Python, Ruby, PHP, and .NET versions of code, such as SAML gateway

http://developer.intuit.com

Intuit – SMB, 4m customers

Salesforce – Enterprises, 25k customers

Is it a per-user, or per-customer license? Intuit end-users don’t need a license.

Don’t need to license QuickBooks to use the overall platform – but there may be some synergies.

Intuit certifies apps for use in their marketplace. Federated apps go through a third-party security assessment – on Intuit’s dime today, though that may change in the future.

Boston Azure User Group Now on the Map!

The Boston Azure User Group is a Cloud Computing community group focusing on Windows Azure, Microsoft’s Cloud platform, and …

The Boston Azure User Group is now on the map – literally!

Check out Jim O’Neil’s user group map. Zoom in on Cambridge, MA and you will see us waving from the NERD center. Thanks Jim!

While not as awesomely interactive and visual as a Bing map, we do appreciate other people plugging the user group:

Speaking of Roger Jennings… at the first Boston Azure meeting this Thursday night (Thu 22-Oct-2009 @ 6:30), we’ll give away copy of his hot-off-the-presses book Cloud Computing with the Windows Azure Platform!

I found Roger Jennings’ book chock full of useful information – from context to detail to practical code samples. I stopped at Barnes & Noble after a user group meeting in Burlington and picked up my own personal copy – why no Kindle version!? – and quickly plowed through it. Now I want to go back and play around with the abundant code samples. And in case you are wondering – no, this isn’t the copy we’ll be giving away…we have a new one.

The Boston-area User Group Calendar

First Boston Azure User Group meeting next week

We are getting close to the kick-off meeting of the Boston Azure User Group – next week, on Thursday October 22, 2009 starting with pizza at 6:30 at the NERD in Kendall Square. Microsoft’s Brian Lambert is the featured speaker.

We have a couple of behind-the-scenes planning meetings this week then will finalize the information on the bostonazure.org web site.

Have you joined the Boston Azure User Group mailing list?

Cloud Security – A Business Tradeoff?

I took notes during the Boston Cloud Computing Group Meetup 23-Sept-2009 – the raw notes are below, but a couple of more noteworthy highlights appear first with some of my views interspersed.

Executive Summary – Key Take-Aways & Highlights

Notes from Javed Ikbal’s talk (http://10domains.blogspot.com) are in regular type. My editorial comments and thoughts are in italics or bold italics – so don’t blame these on Javed. 🙂

  • Key take-away – going to the Cloud is waaaay more about Business Tradeoffs than it is about Technology.
  • “There are 2 kinds of companies – those which have had a [data security]breach, and those which are going to have a [data security] breach” -Javed
  • Centralization of data makes insider threat a bigger risk -Javed
  • “On premise does not mean people are doing the right thing” –Javed – right on! I bet the majority of the fortune five-million (as 37 Signals refers to the medium and small business market) have insufficient IT – they just don’t know it. Any stats?
  • Someone from the audience stated there are more breaches in on-premise data centers than in cloud. Therefore cloud is safer. I don’t buy the logic. There could so many more publicized breaches in on-premise systems simply because there are so many more on premise data centers today. So this is easy to misinterpret. We can’t tell either way from the data. My personal prediction: today if there is a data breach for data stored in the cloud, people will not be able to believe you were reckless enough to store it in the cloud; 5 years from now, if there is a data breach for data stored on premise, people will not be able to believe you were reckless enough to store it locally instead of in the cloud which everyone will then believe is the safest place.
  • Someone from audience commented that business value of losing data will be balanced against business cost of it being exposed. This comment did not account for the PROBABILITY of there being a breach – how do you calculate this risk? I bet it is easier to calculate this risk on the cloud than on premise (though *I* don’t know how to do this)
  • Comment from Stefan: We can’t expect all cloud services to be up all the time (we were chatting about Google and Amazon downtime, which has been well documented). I completely agree – And many businesses don’t have the data to fairly/accurately compare their own uptimes with those of the cloud vendors – and, further, if the cloud vendors did have 100% up-time, that may destroy the economies we are seeing on the cloud today (who cares if it is 100% reliable if it is 0% affordable – that’s too expensive to be interesting)
  • Off-premise security != in cloud – different security issues for different data – Javed In other words, treat SSN and Credit Card data differently than which books I bought last year. But I can think of LOTS of data that is seemingly innocuous, but that SOME PEOPLE will balk at having it classified  as “non-sensitive” – might be my bookmarks, movie rentals, books purchased, travel plans/history, many more… not just those that support identity theft and/or direct monetary loss (bank account hacks). I think it would be a fine idea for data hosts to publicly declare their data classification scheme – shouldn’t we all have a right to know?
  • I think IT generally – and The Cloud specifically – could benefit from the kind of thinking that went into GoodGuide.com.

Raw Notes Follow

The rest of these notes are a bit rough – and may or may not make sense – but here they are anyway…

Intros

  • Pizza & drinks, some social (sat next to Stefan Schueller from TechDroid Sytems and enjoyed chatting with him)
  • Went around the room introducing ourselves
  • People who were hiring / looking for work spoke up
  • Around 30 people in attendance
  • Meeting host: Aprigo – 460 Totten Pond rd, suite 660 – Waltham, MA  02451 – USA
  • Feisty audience! Lots of participation. This added to the meeting impact.

Twisted Storage talk

From Meetup description: Charles Wegrzyn – CTO at TwistedStorage Inc. (Check actually built an Open source cloud storage system back in ’05)

TwistedStorage is open source software that converts multiple storage
repositories, legacy or green-field, into a single petabyte-scale cloud
for unstructured data, digital media storage, and archiving. The Twisted
Storage Enterprise Storage Cloud provides federated search, electronic
data discovery with lock-down, and policy-driven file management
including indexing, retention, security, encryption, format conversion,
information lifecycle management, and automatic business continuity.

History of Building Storage Management software

  • Open Source
  • Been downloaded 75k times
  • Re-wrote – now version 4 – in Python

Common anti-pattern observed in real world:

  • Users storing “stuff” in Exchange since that was a convenient place to store it
  • Results in a LOT of email storage (and add’l capacity is easy to keep adding on)
  • Can’t find your data (too much to logically manage)
  • Backups inadequate
  • Complexity, complexity, complexity

The Twisted Storage Way

  • Federated storage silos w/ adaptors/agents
  • Provide enterprise capabilities spanning sites (access control, audits, search/indexing – including support for metadata, simplified administration and recovery)
  • Petabyte-scale
  • ILM = Information Lifecycle Management
  • Open Source
  • Work-flow (Python scripts, XML coming)
  • Policy-driven (“delete this after 2 years”, “encrypt me”) (Python scripts)

Twisted Storage Design Goals

  • Always available content (via replication)
  • No back-up or recovery needed (due to replication)
  • Linear scalability (scales out)
  • Able to trade off durability with performance
  • Supports old hardware
  • Minimal admin overhead
  • Support external storage systems and linkage
  • Portable – will run on Linux, Windows, (iPhone?) – due to portable Python implementation
  • Pricing: Enterprise Edition: $500 / TB up to 2 PB (annual), minimum $10k for first 20 TB (see web site for full story)
  • versus competition like Centera which charge $15k/Silo + Enterprise Edition
  • http://www.twistedstorage.com, cwegrzyn@twistedstorage.com

Info Security & Cloud Computing Talk

From Meetup description:  Javed Ikbal (principal and co-founder of zSquad LLC)- will talk about:   “Marketing, Uncertainty and Doubt: Information Security and Cloud Computing”

  • What is the minimum security due diligence that a company needs to do before putting it’s data in the cloud?
  • Since 2007, Amazon has been telling us they are “.. working with a public accounting firm to … attain certifications such as SAS70 Type II”  but these have not happened in 2+ years.
  • On one side of the cloud security issue we have the marketing people, whohype up the existing security and gloss over the non-existing. On the other side we have security services vendors, who hawk their wares by hyping up the lack of security. The truth is, there is a class of data for every cloud out there, and there is also someone who will suffer a data breach because they did not secure it properly.
  • We will look at Amazon’s EC2, risk tolerance, and how to secure the data in the cloud.
  • Javed is a principal and co-founder of zSquad LLC, a Boston-based information security consulting practice.

Javed is a Security Consultant

Also co-founded http://www.layoffsupportnetwork.com

Formerly worked in Fidelity (in security area)

Cloud Definition

  • Elastic – provision up/down on demand (technical)
  • Avail from anywhere (technical)
  • Pay-as-you-go (business model)

Cloud Challenges

  • Data stored in China – gov’t could get at it
  • We never have direct access
  • May be locked in? (for practical reasons)
  • March 7, 2009 from WSJ – Google disclosed that it exposed a “small number” of Google docs – users not supposed to be authorized were able to view them. Google estimated < 0.05% of all stored Google docs were impacted – BUT! – this is a LOT of documents. http://blogs.wsj.com/digits/2009/03/08/1214/
  • Sept 18, 2009 from NYT – a recent bug in Google Apps allowed students at several colleges to read each other’s emails – this impacted only a “small handful” of colleges (like Brown University, for 3 days)http://www.nytimes.com/external/readwriteweb/2009/09/18/18/18readwriteweb-whoops-students-going-google-get-to-read-ea-12995.html
  • Google’s official policy for paid customers states “at your sole risk” and no guarantee it will be uninterrupted, timely, secure, or free from errors
  • Amazon states it is not responsible for “deletioreach” – Javedn, destruction, loss” etc.
  • Google will not allow customers to audit Google’s cloud storage claims
  • Amazon says PCI level 2 compliance is possible with AWS, level 1 not possible
  • SAS 70 Type II reports not meaningful unless you can see which controls were evaluated
  • “on premise does not mean people are doing the right thing” –Javed
  • Perception of more breaches in on-premise systems – but there are so many more of them, it is easy to misinterpret
  • Business value of losing data will be balanced against business cost of it being exposed – but this does not account for the PROBABILITY of there being a breach – how do you calculate this risk? I bet it is easier to calculate this risk on the cloud than on premise (though *I* don’t know how to do this)
  • We can’t expect all cloud services to be up all the time – right, and many businesses don’t have the data to fairly/accurately compare their own uptimes with those of the cloud vendors – and, further, if the cloud vendors did have 100% up-time, that may destroy the economies we are seeing on the cloud today (it may be 100% reliable, but too expensive to be interesting)
  • Off-premise security != in cloud – different security issues for different data
  • “There are 2 kinds of companies – those which have had a [data security]breach, and those which are going to have a [data security] breach” -Javed
  • Centralization of data makes insider threat a bigger risk
  • Customers should perform on-site inspections of cloud provider facilities (but rare?)
  • Ask SaaS vendor to see 3rd party audit reports – SalesForce has one, Amazon does not (Google neither? What about Microsoft – not yet?)
  • Providers need to be clear about what you will NOT support – e.g., Amazon took 2 years to provide an answer… Amazon/AWS disclaimers are excellent models
  • Providers need to understand they may be subject to legal/regulatory discovery due to something a customer did
  • Unisys has ISO 27001-certified data centers (high cost, effort)

Creating Secure Software

  • Devs care about deadlines and meeting the requirements
  • If security is not in the requirements, it will not get done
  • if devs don’t know how to code securely, it will not get done right (if at all)
  • Train your devs and archs: one day will help with 90% of issues!
  • Build security into your software dev life-cycle
  • Let security experts, not necessarily developers, write the security requirements
  • Secure Code Review can be expensive –  bake in an application security audit into your schedule, to be done before going live
  • (high customer extensibility + low provider security responsibility) IaaS – PaaS – SaaS (low customer extensibility + high provider security responsibility)

Azure Development Requirements

Executive Summary

This post describes some key aspects of your development environment that need to be in place in order to to write and test code for Windows Azure.

Windows XP does not natively support Azure Development

For all the developers running Windows XP face an obstacle to writing code for Windows Azure:  developing for Azure requires Windows 7, Windows Vista, or Windows Server 2008. The fundamental dependency is that the Azure Fabric Controller (which runs on your desktop for development purposes, simulating cloud behavior) relies on IIS 7, which (you guessed it!) ships with Vista, Windows 7, and Windows Server 2008.

One option is to upgrade your operating system. If you are not quite ready to do that, you have another option – use Virtual PC to run Windows 7 from Windows XP. (This technique also works to run a virtualized Windows 7 image from Vista – or even Win 7 itself – since maybe you don’t want to foul your machine with beta software, like a sandbox for Visual Studio 2010 while it is still in beta (beta 1 as of this writing)).

Essential Software to Develop for Azure

The four essentials are:

  • Have IIS 7.x on one of Windows Vista (Business or Ultimate, I believe) –or– Windows Server 2008 –or– Windows 7
  • Install Visual Studio 2010 – currently in beta (beta 1 as of this writing) – or Visual Studio 2008
  • Install Azure plug-in – currently in beta – to Visual Studio 2010 or Visual Studio 2008
  • Create an account on Azure hosting in order to deploy to/test on the cloud

I wrote a separate, detailed post on creating a virtual machine image for Windows 7 using Virtual PC 2007.

Boston Azure User Group

Coming soon – a new user group for the Boston/Cambridge/Waltham area:

The Boston Azure User Group will focus on Cloud Computing, specifically as it relates to Microsoft’s Windows Azure platform.

This group will likely kick-off in October 2009 – exact date to be determined – exact dates have now been determined – now working on the times 🙂  – see the Boston Azure User Group site for details and updates – and to join the mailing list.

What would YOU like to see covered in the meetings of the Boston Azure User Group? Please leave a comment with your thoughts / feedback.

And see you at the Boston Azure User Group!