Simplification is the ultimate sophistication

"Simplification is the ultimate sophistication." - Steve Jobs

After doing many software projects, I found out that I've always been trying to the same thing: Simplify unnecessarily complex designs. Not only to simplify the architecture design, but also to simplify the way we communicate the solution. Real masterpiece is like an art: Intuitively simple, while ultimately sophisticate.

This philosophy resonates well with SOA design principles. Good design encapsulates intrinsic complexities within its own boundary, while delivers promised capabilities through a very simple interface. Simplicity also fosters other key characteristics of a well-designed online system: reliability, scalability, performance, agility and manageability. 

Designing simple requires profound understanding of innate complexities. It’s a long but rewarding journey, that worth pursuing by every self-esteem designer.

This folder (http://sdrv.ms/15U0N9K) contains codes and documents of open source projects I've worked on. Every one of them embodies the same simplification spirit that I fully embrace.

Enjoy the beauty of simplicity!

Gateway Service

Gateway is a design pattern, which enables authentication, routing, monitoring, A/B and other infrastructural capabilities that you can add to your web site in a service injection manner. 

By service injection, you don’t need to change your code much, as there is no direct dependency between the Gateway and your web site. You just use the Gateway as a service.

Gateay Live testing sites:
https://dangateway.cloudapp.net  
(username/password):  user1@danyang.onmicrosoft.com/test@123 , user2/test@123
User1 will be routed to East, User2 to West

Gateway Source Code:  http://sdrv.ms/12RSZjM 

Project Cloud Service Fundamentals

Cloud Service Fundamentals (CSF) is an open source project created by Microsoft Azure CAT team. The purpose is to provide best practices on dealing with common Azure project challenges: SQL Azure Retry, SQL Sharding, Logging, Telemetry, Configuration, Caching, Server Affinity, etc.

The project can be found at http://code.msdn.microsoft.com/windowsazure/Cloud-Fundamentals-in-1a3ab1bd
Here is a decoupled and componentized version of CSF. The CSF project is decoupled into 4 components: Common Library, CSF Social Web App, Telemetry and Gateway.

Live testing sites:
•CSF Web App: http://dancsf.azurewebsites.net/
•Telemetry Report: https://5zjgywiud3.reporting.windows.net/ReportServer    dan/test@123
•Gateway: https://dangateway.cloudapp.net  user1@danyang.onmicrosoft.com/test@123
•Code:  http://sdrv.ms/12RSXID  (CSF)  , http://sdrv.ms/12RSZjM   (Gateway)

Choices of deployment on Windows Azure

Azure provides many choices of deployment, for both web/application servers and database servers. You will never fully appreciate how powerful these choices are, unless you actually played with it. 

Last week, I played with it as part of a load test project, and I was amazed by its capability even though I presented these features individually many times before.

My test setting is a very typical LOB web application. ASP.NET web tier, SQL Database, plus IE or Visual Studio load test client running on laptop. I tried many different deployment options, and compared the performance differences. It's very easy to switch from one deployment to another. Just a few clicks on the Azure management portal or simply republish the project from Visual Studio. Here are my learning.

Overall design:

Deployment Options:

Website: Azure Website free tier, Azure Website reserved instances, Web Role instances, multiple number of instances

SQL: Azure SQL Database, SQL Server in Azure VM, multiple sizes of VMs

Load test client: laptop, Azure IaaS VM

Network: Virtual Network, or not

Performance findings:

• Azure website free tier, performance quota will soon be blown, and request will be rejected. 
• Azure website reserved instance, and Web role instance. Performance is comparable between these two options. Both can scale pretty well, and performance is stable.
• SQL Server in VM. Tried different file placement. There is some perf improvement by moving log/temp files to other disks, however the improvement is not significant. Need further investigation.
• SQL Server in VM. Switching to bigger VM significantly improves performance.
• SQL Azure vs. SQL Server. This is the most interesting part. SQL Azure 1 GB Database achieved comparable performance as SQL Server in Middle sized VM. The price of SQL Azure is only 1/10 of that of SQL Server in this configuration. The result should be related to my testing workload (a mixture of read/update/insert against NopCommerce web), and should not be generalized without further investigation.

Bowlus for AWS

To learn the upcoming Azure IaaS Virtual Machine, I decided to migrate Cloud Samurai project (code named Bowlus) to IaaS infrastructure. Since Azure IaaS is not GA yet, I did this first on Amazon AWS. This  also gave me an opportunity to learn and compare the IaaS offering from both Azure and Amazon.

After the initial redesign for IaaS, I found the benefit of IaaS is rather obvious:

• Use the more mature IIS Web Deploy cmdLine tool to provision web site, instead of update IIS configure file directly.
• Updating IIS configure file is a little bit dangerous, and it will cause troubles when concurrent access is not managed well.
• Web Deploy is mature tool which is also used in Visual Studio to deploy web site to Azure. 
• Using Web Deploy generates deployment logs, which helps diagnosis and auditing.
• De-provisioning web site using Web Deploy is also cleaner and safer.
• This is only possible under IaaS, since you have to pre-install Web Deploy tool and other dependent services in the VM.
• Reduce two layers: routing + hosting, into one layer. The design is thus more simple and robust.
• This change might also work in PaaS. Need further investigation.
• IaaS gives us more flexibility to manage end points on each VM, and thus makes this design change more natural.

This is the live test link:

http://bowlus-2127885525.us-east-1.elb.amazonaws.com:8080/admin/

This is the new design for AWS:

Understand Persistent VM Disk (2)

Since all persistent PVM disks are eventually stored in blob storage, there must be a way to manage these disks and the .vhd files in the blob storage. The tools and portal to manage these are simple and straight forward, but understanding below key concepts would be very helpful for you to use those tools.

There are three stages of a .vhd file or a PVM disk:

1. raw .vhd file in blob storage (use any blob explore tool to manage this)
2. .vhd files recognized by PVM as disks (use Azure portal to create/delete disk)
3. PVM disk attached to a PVM instance (use Azure portal to attach/detach disk)

O/S disk vs. Data disk: this is decided when you recognize a raw blob file as a disk. O/S disk can be used to create a VM instance.

Image vs. Disk: Image is "sysprepared" O/S disk. You can use either Image or O/S Disk to create a VM instance. If you use O/S Disk, all the old user name, machine name etc. persist, while everything is as from scratch if you use Image.

Understand Persistant VM disks (1)

Persistent VM (PVM) is Azure's IaaS offering comparable to Amazon's EC2. Unlike EC2 which uses local disks for the majority of the time, PVM uses Azure blob storage to store disks. This design has multiple implications.

On the good side, your data in the disks are more reliable, as every Azure blob automatically has 3 local copies which are always synchronized and another 3 copies in a different data center 100+ miles away which is delay synchronized. On the bad side, the disk I/O performance might be impacted. For this reason, PVM do use both Azure blob backed disks and local only disks to optimize the disk I/O, so it's important to understand where each disk lives. Below is the explanation:

C: disk is the OS disk. Persisted in blob storage.

D: disk is local disk. Not persisted, but very fast.

F: disk is additional user data disk. Persisted in blob.