Software Architecture – Stephen Haunts { Freelance Trainer and Writer }

Before Christmas a good friend of mind, Rachel, asked me if I could help her set up a website for a business she was starting selling antique silverware (Argent Antiques) . Rachel needed the site to offer a standard set of e-commerce features where she could add and amend products and content on the site.

The software developer in me immediately starting thinking about how I could build her a solution. I was thinking that I could develop the site using ASP.NET MVC 5, Twitter Boostrap, Braintree payments integration etc. then then I thought, STOP, what are you doing? This isn’t a fun software development project, and Rachel certainly didn’t want to wait 3-6 months for me to lovingly hand craft the perfect e-commerce platform. She wanted a site up and running that fit her requirements in as short space of time as possible. After all, a site that isn’t live, is a site that isn’t generating revenue.

I took a step back and started looking at the different options available to me, these included :

Develop a custom site in ASP.NET MVC.
Use a CMS system like WordPress, Joomla.
Use a SaaS (Software as a Service) eCommerce solution like Shopify or Wix

The option that I selected is based heavily on the circumstances and restraints that I was working within. For example, I was asked to produce this site as a favour to a friend, which meant I was going to be working on it in my spare time. I have a full time career, plus I am a Dad to 2 kids, so spending a lot of time on a solution would be an overall constraint. Plus from Rachel’s perspective, she wanted to get a version 1 website out there as soon as possible. The screen shots of the site in this article are our current version 1.0, quick to market version of the site that is fully functioning.

Here are some of the Pros and Cons I used as part of the selection process.

Read More “Buying vs Building Systems”

High Availability/Software Architecture/System Design

High Availability in a Retail Environment

January 14, 2014January 14, 2014 Stephen Haunts 2 Comments

Working in a global retail environment poses some interesting availability challenges when you have physical Bricks and Mortar stores. I have been thinking about the problem of high availability in this environment for a little while now due to a project I am involved with to harmonise the retail systems used between global groups. It is quite a common problem for an organisation that has grown through acquisition that you have different systems used in different business units, but after a while it makes sense to try and go with a common platform.

This article talks about how this architecture could look and how you can support the staggered roll-out of new Point of Sale features to the store whilst still maintaining high availability.

Logical View

Before I talk about the architecture, I want to cover a scenario first of the end state. Imagine there is a global retail company based in both North America and Europe. Both territories have around 1000 physical bricks and mortar stores. These stores each have a number of tills (cash registers for my American friends). There could be between 2 and 5 tills per store depending on its size. Each till communicates with systems hosted at a centralised location. These systems consist of web services, caching servers and databases. This has been illustrated in the diagram below.

This diagram shows 2 geographic regions. Each Region contains a head office network infrastructure and a store network infrastructure. Both of the global regions are completely separate from each other. There are no shared resources between the two.

General/Software Architecture/WCF/WebAPI

Is WCF Redundant?

November 22, 2013November 22, 2013 Stephen Haunts 13 Comments

It’s funny, when alternative technologies comes out that do a similar thing to a previous technology, people are very quick to pronounce the previous technology as dead. I was talking to someone the other week at a different company who said that their architects wanted to use WebAPI because WCF wasn’t relevant any more. This made me think, WHAT!!, REALLY!!, SURELY NOT!!

In the rest of this article I want to briefly cover what WCF and WebAPI are, talk a bit about the differences, and then suggest the reasons why you may use either. This isn’t an exhaustive discussion, but the point is that you can have various technologies doing similar things, without either one of them being redundant. I have spent the last 7 years of my career being involved in service architectures built using WCF and I am certainly not thinking I need to replace any of those systems just yet.

Composing Services into Layers

November 7, 2013November 8, 2013 Stephen Haunts 6 Comments

In this article I want to discuss a web services architecture that I have worked with over the last 7 years in a couple of different companies. The architecture I am discussing composes web services into layers, but there is much more to it than that. Nothing in this article is rocket science to an experienced services developer, but it is a way of working that has proven very reliable and easy to understand for developers that gets them working in a consistent way.

The diagram above shows what you will typically be used to working with in an enterprise. It is a standard 3 tiered architecture comprising of the following layers:

Presentation Layer: This contains any views that your users will interact with like, web sites, smart phone applications, desktop clients, etc.
Business Logic Layer: This contains the under lying logic for your application which may be comprised of web services, REST Services, windows services etc. The presentation layer shouldn’t contain any business logic and therefore should be quite dumb to the inner workings of your system, this is what the Business Logic Layer is for.
Data Layer: This contains your data storage. This typically is an SQL database (SQL Server, Oracle, and MySQL) or one of the newer blend of NoSql Databases (RavenDB, MongoDB, Cassandra). Typically your data layer may expose stored procedures to the Business Logic Layer, or your business logic may connect via an Object relational mapper such as Enterprise Library, NHibernate or SubSonic.

This article is going to concern itself predominantly with the Web Services part of the Business Logic Layer.

Software Architecture/System Design

Command Query Responsibility Segregation

July 4, 2013November 9, 2013 Stephen Haunts 3 Comments

In this article I want to talk about an architectural pattern called Command Query Responsibility Segregation (CQRS). I will use an example to illustrate this pattern based on a recent problem we had at the company I work for and how we solved that problem. The system in question is an automated payment processor and runs on a schedule through the night and collects payments from our customers. The original CQRS pattern was written about by Martin Fowler over on his blog.

The concepts behind CQRS are very simple, but they give you some powerful advantages. What CQRS essentially does is get you to use a different model to update information (COMMAND) than the mode you use to read information (QUERY). If you think of a typical layered application you may have a client, a user interface or a background process that communicates with a business logic layer that may be comprised of a static set of libraries, Web Services or REST based services.

This business logic layer acts as a model into your domain that lets you perform operations against that model. These operations generally allow you to create, retrieve, update, or delete data associated with the model. For most scenarios this age old way of working is fine and that is certainly true nowadays. This way of working can cause you problems for systems that need to have high performance and be more scalable.

Command Query Responsibility Segregation

To illustrate this pattern I will discuss a payment system developed by 2 engineers on my team, Graham Johnson and Hugh Hulme. For the purposes of this article we will call it the Automated Payment System (APS). The actual system itself had a different name but APS will do for the purpose of this article. The system would run through the night and collect payments that where due from our customers. The initial system was completely synchronous and would execute in the following steps:

Software Architecture/System Design/System Monitoring

System Monitoring – Part 4 : Dashboards

March 1, 2013November 8, 2013 Stephen Haunts Leave a comment

This is the fourth article in my series on systems monitoring for software developers. In this article I want to talk about using dashboards for displaying information generated by your monitors.

The system monitor discussed previously will collect a lot of data from different systems. The amount of information that is collected can be vast and in its raw form may not be that useful except to people who really understand what the data represents. The key thing with a dashboard is to break this information down for different users. This may be a complete view of the data, or just a subset for different purposes, so you have to think about the context in which the data is going to be provided and the intended audience.

For me, a key requirement of the dashboard I built was to make the data from the monitor available to my development and operations team in an easy to view format. This dashboard contains more technical information including exceptions as that is more appropriate for the intended audience. You may need to provide another dashboard to key business stakeholders. The level of information you provide here would most likely be different and contain more business level data.

Your dashboard may be viewed in different places. This might include a large TV attached to the wall, projected against a wall, or the dashboard might be run on a person’s desktop computer. If you are displaying the dashboard in a place where people cannot interact with it, i.e. on a large TV Screen, then you want to make sure the dashboard automatically refreshes itself. I will cover this more later on in the article.

Types of Dashboard

Technology

You really are spoilt for choice these days when it comes writing a dashboard. You can keep the display simple or you can really embellish the presentation. As I said earlier, it depends on your target audience. I personally find that keeping the display simple is the best thing to do. I have seen some dashboards that go overboard with dials, speedometers and other fancy graphics, and these can confuse the information you are trying to communicate if you are not careful.

Windows Application

You can develop your dashboard as a native application. This is what I did with my own solution for the company I work for. I work at a company that is based around the Microsoft stack, so this really gave me 2 choices as a .NET developer; Winforms or WPF. As I needed to get the dashboard up and running quickly I used Winforms, mainly because I am very familiar with this particular technology. WPF would also have been a very good choice.

My dashboard was aimed squarely at a technical audience, so the level of detail could be higher. The context of the dashboard was to allow developers spot any problems in our critical systems. I decided to keep the screen design very simple. I used a tabbed control, where each tab represented one sensor from the xml stream written out by the monitoring system.

Software Architecture/System Design/System Monitoring

System Monitoring – Part 3 : Deployment

January 5, 2013November 8, 2013 Stephen Haunts 1 Comment

This is the third article in my series on system monitoring for application developers. The previous article covered the basic code architecture for the system monitor.

This article will talk about the deployment of your system monitor to start monitoring production systems. A lot of what I discuss here is from experience, especially as I work in a regulated industry (Financial Services), which threw up a few interesting challenges on how I could achieve rapid deployments and application feedback without breaking any rules.

Deploying into a Production Environment

The goal of your monitoring tool is to regularly check over your critical production systems, so that means you need to deploy into production. If you work in a smaller company that doesn’t have a long and complex deployment process (By ‘complex deployment process’, I mean a process heavily bogged down with paper work a company processes), then this should be straight forward.

Systems Monitoring : Deploy to Production Environment — Deploy to Production Environment

When developing monitors you want to be able to iterate quickly, and deploy early and often. If you work in an environment where you can do this and release into production quickly, then great. You need to do, just that. Every time you get a piece of functionality ready for prime time, release it. You just need to make sure that the server you deploy the monitor too can access any log files or databases to gather its information.

The diagram above shows what a typical deployment might look like. You have your monitoring server setup containing the monitor application and the task scheduler. This application then processes logs contained on each target system. The example above has web services, a website, and payment gateway and payment processors. This shows you your most basic type of deployment. This is great if you can do this type of deployment quickly and often.

Software Architecture/System Design/System Monitoring

System Monitoring – Part 2 : Basic Architecture

January 3, 2013November 8, 2013 Stephen Haunts 2 Comments

This is the second article in a series on application systems monitoring for software developers. In the first article I discussed the basic idea and concepts around systems monitoring. In this second article I will go over the basic architecture of how an application monitor can work. The system will start out simple to begin with and be extended over time. This iterative approach to building up the design mirrors how I implemented this type of system for my current employer. In true agile style I wanted to get something basic working as quickly as possible so we could start getting the benefit from it early on. Once the system was out there and working, I then built upon the basic idea with new features.

Use Case

First of all let’s look at the basic use case diagram above. This diagram shows 2 subsystems, the monitor sensors and the monitor dashboard. This article will focus on the first subsystem, Monitor Sensors. At this stage in the systems evolution the main actors here are Developers and Technical Support.

Roles may differ in your own organisation, but because I work in Financial Services we have to have a clear separation of concerns between the development infrastructure and the production infrastructure. This means developers can not directly deploy too or modify a production environment. This is for good reason; we are very good at breaking stuff!! This is why we and many other large organisations have a separate technical services teams who maintain the production environment.

That means both developers and technical support have a vested interest in the results of the monitor. Just because developers can’t make changes to a production environment, there is no reason why they can’t see the telemetry data collected by the monitoring systems. In fact it would be a very bad idea if they couldn’t. By seeing how the production systems are running directly from a snapshot data stream, developers can get a good impression of how their systems perform in a real production environment. This may well help to influence future design decisions when the systems are extended. This also gives developers a realistic impression of what volumes their systems cope with in real life. As we all know a development test environment, no matter how hard we try, never matches what we see in a real production scenario with real users and data.

Software Architecture/System Design/System Monitoring

System Monitoring – Part 1

January 2, 2013November 8, 2013 Stephen Haunts 3 Comments

This is the first part of a set of articles about systems monitoring for software developers. I am writing about this as it is something I have been working with a fair amount over the last 6 months in my current role here in the UK.

The company I work for is pretty typical for a large firm. We have a mix of different legacy systems that we have to keep running as we try to update them, and integrate new systems. A lot of these systems vary in quality and the majority of the original system developers no longer work for the company. When any of these systems go wrong, it can be quite difficult to diagnose what the problem is.

System Monitoring - Programmers Desk — Taken by Duncan Verrall

As an example, we have one particular system that handles file synchronisation between multiple sites that runs over night. This system does write out log files, but they are the most unfriendly log files I have ever seen. For a start they are so large, people do not even bother to look at them anymore. Even if you do open up these files, the formatting is so strange and complicated you really struggle to see what is going on. This again is why people don’t bother to look at them. This particular system is one of the more extreme cases. Other systems operate a mix of writing data to log files or adding huge amounts of data into a logging database. In some cases the amount of data logged is quite extreme, which makes searching of meaningful information in the event of a system failure extremely difficult. Even more so if you are under pressure with a system outage which is having a direct revenue impact to the business.

In the rest of this series, I will talk about the system I designed and developed. I will cover the design and implementation. I will also talk about some of the problems I faced, and how this system has helped divert a few major live incidents. For obvious reasons I can’t discuss actual systems and internal process at my company as I don’t think they would like that too much, so some details have had to be changed or omitted, but that really doesn’t matter for the purpose of this article. The intention is to discuss the architecture of the system. I hope this series will help you in thinking about your monitoring needs in your own organisation.

Read More “System Monitoring – Part 1”