The Software Architecture of a Business Application
Screens and Reports - A User's Perspective
From a user's point of view a business application consists of two main components and although it may be obvious to people who live and work in the "real" world it is inclined to be forgotten by the technical people who have to deal with the hidden complexities of computer systems or those who consider computers to be primarily an entertainment device.
Firstly there are screens or forms that appear on the computer's screen and allow a user to enter information or instructions into the "system" and observe the progress and results of various "happenings" within the "system." Secondly there are reports that the user can print out that are commonly referred to as "the hard copy."
Although this description of a business application may be accurate from a user's perspective it belies the fact that a single computer represents the most complex machine human beings have ever devised. And we haven't even begun to consider the implications of a planet filled with a diverse network of interconnected computers of various makes and models running a humanly incomprehensible multitude of software systems.
The user gives instructions to the "system" using a keyboard or pointing device and they see what happens on their screen or what comes out of the printer. And that's it. As far as the user is concerned everything else that happens inside the "system" is shrouded in mystery. This gives someone with even a little knowledge of computers a major advantage in the eyes of a user and allows them to take on the aura of a magician, a sorcerer or a witch doctor.
Because those who make the decision about whether or not to invest in or commit to a particular technology can often be described as users (albeit sophisticated users) the computer practitioner is able to turn this situation to his own advantage and profit. Because the user fails to appreciate the true nature of computers he becomes an easy prey to the black arts of the computer practitioner who can conjure up dazzling spectacles both on and off the computer screen.
Off-The-Shelf or Customization - An Investor's Choice
When searching for a software solution the business owner is often faced with two alternatives. The first and most appealing is to search for a ready-made pre-built solution. This is what has come to be known as off-the-shelf software. The idea is that the user pays for the software, installs it on his computer and after some configuration the problem is solved. This model works well for a particular type of business application with the classic examples being word processors and spreadsheets.
The difficulty with this approach is that every business is unique and has diverging requirements and expectations for a software solution. For an off-the-shelf solution to be really useful in this context it must include all the features its users and potential users are likely to need. The software in effect becomes "all things to all men" with so many features that just learning how to use it is not an exercise for the faint of heart.
The second alternative is what we call customized software. This means quite simply either build it yourself or engage someone to build it to your exact requirements. The advantage of customization is that if done correctly you will end up with a complete solution without bells, whistles and a host of other features you will probably never need or use.
There is a third alternative we have thus far failed to mention that should not be forgotten. This is what we commonly refer to as the paper-based solution. For those that don't know or can't remember this name is derived from the way things were done before we had computers. Long lists of facts and figures were created on sheets of paper using a primitive hand-held writing device known as a pen. If presentation was important then another device was used where sheets of paper were fed directly into it and the characters were impressed on the paper through an inked ribbon by mechanical hammers operated from a set of manual keys laid out in a remarkably similar arrangement to the ones on a computer keyboard. This contraption was called a typewriter and examples of it can still be seen in some museums. In many business situations the paper-based solution is still the best and most cost-effective answer to the problem.
(The remainder of this essay assumes you have abandoned the search for an elusive off-the-shelf solution and have also ruled out the prospect of returning to or remaining in the dark ages with a paper-based system. You now have a single option in mind - you have decided to either build it yourself or engage someone else to build it for you. After all it can't be that hard if everyone is doing it! Right?)
Garden Sheds or Skyscrapers - An Engineer's Perspective
The techniques used in building a garden shed are reasonably easy to apply by anyone with some experience of carpentry. Having mastered the requisite skills one can continue to successfully build sheds of varying sizes until one feels confident enough to attempt a larger project such as a house. What is not so obvious is that although the methods used to build the shed are also useful on the house there are a significant number of other skills and techniques that must be understood and applied correctly in order that the house-building project be a success.
Mastering the skills needed to build the house may give one the inclination to try an even more ambitious project such as a multistorey building. However unless one is cognizant of the fact that to succeed in such a venture will require the rigorous application of a completely different set of skills the project is doomed to failure from the outset. Although many of the shed and house building techniques are useful on the multistorey they are of only secondary importance to the successful outcome of the project.
To the building industry outsider who has never touched a hammer and nails the shed, the house and the multistorey building have much in common and he remains blissfully unaware of and unconcerned about the different techniques used in their construction.
Under Pressure - Increasing Users, Increasing Data, Increasing Complexity
The concepts involved and the problems encountered in the construction of computer systems bear a remarkable similarity and parallel to those in the building industry and anyone serious about software construction would be foolish to ignore the significant lessons learned and the experience accumulated by that industry over the last few thousand years.
The techniques used to build a business application are reasonably easy to apply by anyone with some experience of computers - create a database table, add some fields, create a data entry screen, design a report. One can continue applying these techniques developing one application after another and at the same time gaining the confidence to attempt a more ambitious project. However unless one is aware that in the same way as making the transition from shed-building to house-building requires a significant number of new skills and methods in order that the project be successful the move from a basic business application to something more advanced also requires the correct application of new knowledge and skills the project is doomed to failure.
At best a number of things will happen. As the number of users increases the application will begin to experience lock-ups where the application or computer "freezes" for no apparent reason. This will usually happen at the busiest of times when the number of active users has increased and the greatest demand is being placed on the system.
As time goes by and more data is added the application will tend to slow down and may even appear to grind to a halt in a similar way to a lock-up. This can happen when entering new data, searching for existing data or printing a report.
Finally as new features are added to the application by either the original or subsequent developers the time and effort required to do so increases inordinately making the cost of each new feature outweigh the benefit to be gained. The developer either gives up in despair or the application begins to develop bugs that are difficult to eliminate and have the potential to cause the data to become corrupt or unreliable.
Information and the Nature of Software
Although there are many similarities between software construction and building construction it should be noted that there is also a fundamental difference the world is still in the process of coming to terms with. This difference arises from the inherent nature of information expressed in its digital form.
By means of computers and computer networks digital information (including computer software, data, audio and visual media) may now be instantaneously duplicated and transmitted at the speed of light to any geographic location on earth at virtually zero cost. This fact presents a serious challenge to those who work in the software industry or are used to making a living from the sale of digital information.
If the building industry was the same as the software industry it would mean we could build a shed, house or multistorey building (with all the incumbent expense) and then at zero cost create as many identical buildings as we wanted anywhere else on the planet.
It is this ability to duplicate and distribute information and software at next to zero cost that creates the illusive promise of easy riches for the software practitioner. This dream was a contributing factor in the speculative investment frenzy of the 1990's that came to be known as the dot com bubble.
The Problem with Software Development Today
Today the biggest problem in software development arises from the fact that everyone is doing it. Computers have become ubiquitous and there is now a huge array of tools and products enabling us to do clever and amazing things. Many of these technologies are freely downloadable from the Internet and there is no barrier to entry into this vast Pandora's box we call computer programming.
This presents the business owner with a significant problem. With so many options to choose from and so many potential solutions on offer just how is he to go about making a decision. What criteria should he be using to evaluate each alternative? Does he have the time to thoroughly research all the possible options at his fingertips? And how is he to avoid becoming entangled by the alluring snares of the conjurer?
Does that computer salesperson in the nice suit with the slick presentation and impressive graphics really have the best interests of the business at heart? Or is he more interested in making a sale? And how is the business owner to know the difference?
Even if the business owner is able to find software or solutions that solve at least some of his problems he will often end up with a hotchpotch of different applications that do not work well together. This often means that the same data must be entered into the system multiple times or transferred from one application to another using time consuming and error-prone import/export procedures. An even more uncomfortable prospect is the possibility that the business will come to depend on a particular piece of software as a critical part of its operation only to discover when problems arise that the software is no longer supported by the company that developed it.
Ignorance - Leaving it to the Experts
When attempting to find a software solution most business owners initial approach is to engage an expert. They have a gut feeling that their business would benefit from and be made more efficient by using a computer solution. They also recognise that they do not have the knowledge or understanding necessary to make a wise decision that will stand their business in good stead for many years to come.
Often they will employ the services of an accountancy firm or computer consultant to advise them on the best options available. This will give rise to an expensive and time consuming search to collect and collate the many off-the-shelf solutions on offer. The consultant will examine the various features of the software to see how well they fit with the business. They will attempt to get feedback from other users of the software in order to gauge their level of satisfaction. They will also look carefully at the company that developed the software asking questions like "how long have they been in business?" and "how long are they likely to continue?"
The result of this process will often arrive in the form of a long report containing many interesting tables and diagrams (together of course with a sizable bill for services rendered which the length of the report is partly an attempt to justify!) This report is likely to be inconclusive and fail to provide the business owner with any clear direction. Having sought and paid for the advice of an expert the business owner very often finds himself back where he started (but with less money in his pocket of course!)
Openness - Getting Rid of the Magicians
There is an oath practitioners of magic are expected to commit themselves to by stating essentially that "as a magician I promise never to reveal the secrets of any illusion to a non-magician, without first swearing them to this oath. I promise never to perform any illusion for a non-magician, without first practicing it until I can perform it well enough to maintain the illusion of magic."
Although we are unaware of any similar oath among computer practitioners there is a tendency among many to want to closely guard their secrets. They see this as a way of preserving their privileged position amongst the community at large by restricting access to the knowledge and secrets they have invested many years and resources to acquire.
When investing in a computer solution that is likely to become a critical part of a business's operation the owner must be made aware of all the necessary facts pertaining to that solution if he is to make a wise decision in the best interests of the business and its long term future. Either that or he must be willing to believe that the magic will continue to work for as long as he has an interest in the business.
Professionalism - Putting the Business First
What the business owner really needs (although he may not be aware of it) is the assistance of a professional. This is a person who has neither a vested interest in recommending a particular solution nor the intention of wasting the owner's time and resources by taking him on a wild goose chase.
The true professional is the one whose ultimate intention is to act in the best interests of the business even if it means gracefully stepping aside when his assistance is no longer sought or required. His motivation is to use his knowledge, talents and experience to assist the business owner to find a real and long term solution to his problem. Although he does not have all the answers he has the humility to acknowledge this and the willingness to seek and accept advice from those more experienced.
The real question then that the business owner should be asking is this - amongst all the people queuing up to offer advice and potential solutions how can I pick out the true computer professional?
Software Ownership - The Long Term Perspective
Satisfying the immediate needs of a business by addressing its current problems with a software solution (assuming one can be bought or built at an affordable cost) may provide a certain semblance of progress. The business is able to cut costs by streamlining its administrative processes resulting in time savings, staff reductions and efficiencies. The magic works!
As time goes by a number of things begin to happen. The commercial or legislative environment changes, the computer hardware technology changes, the business diversifies or the business owner thinks of other valuable things the software could be doing with the information available.
All of these things are likely to require software modifications. However unless this eventuality has been considered beforehand the business may find itself facing an insurmountable obstacle. The cost of the modifications may significantly outweigh the value of the benefits to be gained. In the worst case where the source code is not available or the original developer has ceased trading there may be no viable alternative other than to replace the entire system.
In order for a software system to be an asset to a business in the years ahead and not a liability there are a number of precautions that must be taken by the business when the system is first implemented. Firstly and foremostly an electronic copy of the source code for the entire system (and any subsequent modifications) must be acquired by the business and kept in a safe place preferably as multiple copies in disparate geographic locations. If future modifications are to be made to the system the availability of this source code is an absolute necessity.
Secondly if the original developer is not operating under the business's direct instructions the business must ensure that it obtains from the developer a perpetual licence allowing the business to maintain the software for its own use on an on-going basis. The purpose of this is to ensure that the business is not legally dependent on the original developer in any way with regard to future modifications to their system.
Thirdly if the software is to become a real asset as time goes by the system must be architected and developed in a way that allows new features to be easily integrated into the existing system. The initial system in effect becomes a framework within which new features will be added or existing ones modified. The quality of this framework, its associated documentation and the foresight with which it was developed will have a direct effect on the ease with which subsequent developers are able to make enhancements. It will also provide a model that will either guide and inspire their work or alternatively cause them to pull their hair out in frustration and despair.
An Integrated System - The Corporate Database
In many organisations today the picture that emerges of the state of their information systems is not a pretty one. As information needs arise from within the organisation the common approach is to search for an off-the-shelf solution to meet that need. When the search proves unfruitful a customized application is created using the computer programming tools most readily available or most easily understood.
To use the building analogy the end result of this approach begins to resemble a yard filled with garden sheds rather than a house or multi-storey building. The people working in the yard become adept at moving items from one shed to another by means of a wheelbarrow along a garden path or through an underground tunnel. Similarly the people working on the computer system get used to moving information around the system by means of multiple entries or complicated import/export procedures.
The result of all this is that data that should be accurate, up-to-date and available to people within the organisation for it to function effectively is inaccurate, out-of-date or unavailable. Customers and suppliers lose confidence in the organisation and decision makers are forced to make decisions based on a "fuzzy" picture of what is happening at best.
To avoid or rectify this situation the business must make a deliberate decision at some point to develop or move towards an integrated system. An integrated system is a system where all the information used throughout the organisation is stored in a single logical place and is structured in a way that avoids unnecessary duplication of data. Unfortunately for a large organisation that already has a well-developed computer system but has not taken this integrated approach switching is in many cases not a viable option. They will probably continue to struggle with their existing systems for some time to come until they are either bought by or merged with another organisation or they cease to exist.
A Flexible System - The Incremental Approach
The traditional approach to developing a complex computer system has been to put a great deal of effort into specifying beforehand and in precise detail what is required and exactly how it is going to work. These specifications are presented in the form of a document that the business owner or users will be asked to sign before any development work begins. The purpose of this arrangement is to allow the developer to focus his effort on building a system that satisfies the specifications agreed to in the document.
Unfortunately this method presupposes that what is in the document is exactly what the business requires. The problem with this is that what is often delivered although it may meet with the documented specifications is not really what the business needs. When the users or the business owner complain about the end result the developer is able to point to the specification document and quite correctly state that they have done all that was agreed to. Any additional features or adjustments will need to be renegotiated often at a significantly increased cost.
The problem arises from the commonly held view among computer practitioners of the idea that a software system is a product rather than a service (or more correctly a means of delivering a service.) The result is that the whole effort of the software developer is misdirected toward the delivery of a finished product when in fact what the business really needs is a developer that is able to understand its complex, unique and changing needs and modify or enhance its computer systems accordingly.
Successful businesses grow and develop with the passage of time and the availability of additional investment capital may mean that a business can benefit from the enhancement of its computer systems. Furthermore the commercial, legislative and technological environments in which a business operates are subject to on-going change. It is important therefore that if a business is to reap real rewards from their investment in computer systems those systems are structured from the outset in a way that allows for this incremental approach to software development.
Instead of looking to purchase or build a product that will meet its needs the business must change its approach. It must instead begin searching for professional software developers that understand both the incremental approach to software development and the need to supply the business with an on-going service. These professional developers will also understand and fully identify with the business's requirement not to be "locked in" to a single supplier in order to satisfy their future need for software development.
Data Storage - The ER Model
The development of a business computer system is an attempt to create a representation or model within the computer of both the internal processes of the organisation and the way in which that organisation interacts with its environment so that the relevant information can easily be collated and accessed by the users within the business or its customers or suppliers (with appropriate access rights of course!) The better and more elegantly the design of this model fits with the reality of what is happening in practice the easier it will be for future software developers to modify and enhance the system as required.
The computer model used to represent an organisation can be divided into two parts. The first part is known as the entity-relationship (ER) model and is used to represent the structure of the data used by the organisation. (When the ER model is implemented on the computer it is usually referred to as the database.) The second part is known as the object-oriented (OO) model and is used to represent the processes within the organisation and the way these affect the data. (When the OO model is implemented on the computer it is commonly called a program or an application.)
The ER model attempts to identify the entities that a business needs to record information about and the way that these entities are related to each other. Common examples of such entities are customer, supplier, job, sale, invoice, order and product. Each entity is represented in the database by a table containing records (rows) and fields (columns).
The task of designing a database consists of identifying the entities to be modelled and creating the corresponding tables with the appropriate fields. At first glance this appears to be a trivial exercise and in the case of a simple application with few entities it is usually quite straightforward. However as the system becomes more sophisticated with multiple entities interacting in complex relationships the job can become decidedly non-trivial. The skills required to develop an elegant database design in this situation are often developed only after some hard-won years of experience and a careful study of mistakes made along the way.
Once a computer system begins to develop on a database suffering from a poor design it can be very difficult if not impossible to rectify. This leaves future developers with the extra work necessary to allow for and work around the design flaws in any modifications they make and also underscores the value of engaging an experienced database designer at the earliest possible opportunity.
Processing - The OO Model
The object-oriented (OO) model attempts to define the processes within a business in terms of the data that is affected by them. An entity in the database combined with the processes that are specifically related to it are collectively referred to as an object.
For example an entity in the database used to represent a customer becomes a customer object when combined with the various processes we may wish to perform in connection with that specific customer such as displaying their contact details or account details on screen, printing a statement or determining their credit rating. Because these processes are directly linked to the customer object itself it is easy to invoke them whenever they are needed within the system by a simple reference to the customer object.
The OO model will also include objects for which there is no corresponding entity in the database. These objects usually exist only in the computer's memory at the time an application is running so they are often referred to as transient objects. Examples include screens, forms, reports and the data structures used to hold data while it is being processed or moved around the system. (It is worth noting that many processes we wish to model in an organisation are not linked to a single specific entity in the database and unless we think of the organisation or the computer system itself as an object the OO model is deficient.)
As with a database designer the skills of a good OO designer are often developed over a long period of time by experience and a fair amount of trial and error. Usually errors that occur at this level in the design are less far-reaching in their consequences and easier to fix than design errors in the ER model.
Communications - Moving Data Efficiently
Because the data used in a computer system is often located on a machine other than the one the user is working on we need a means of moving data as efficiently as possible between two machines. There are a number of high-level protocols that have been developed to meet this need in various situations including XML, HTTP, SOAP, FTP, POP3 etc. The main consideration in designing these protocols is that they are intended to work between any two programs running on different types of computers with diverse hardware and operating systems.
It is worthwhile to note that these high-level protocols have been designed primarily to allow data to be accessed or manipulated by persons or organisations OTHER THAN THE ONE THAT OWNS THE SYSTEM WHERE THE DATA IS STORED. This is an important point. It is the very fact that these protocols are published protocols (i.e. intended for public use) that gives rise to many of the concerns about security.
These high-level protocols all rely on an underlying protocol known as TCP/IP that allows binary data to be exchanged directly between any two programs running on different computers on either a local area network or the Internet. (The logical connection point at each end of this conversation is called a socket.)
When moving data between computers within an organisation it is both more efficient and more secure to avoid the high-level published protocols in favour of the lower-level TCP/IP protocol. This approach allows the developer to create his own proprietary encrypted binary protocol for all data communication within the organisation and only resort to a published protocol whenever it is necessary to expose data to persons or systems outside the organisation.
Security - Encryption & Authentication
Whenever data is being transferred between two computers on a network there is a risk that it will be intercepted by a third party who will use it in an unauthorised or malicious manner. This risk is significantly lessened if the data is both encrypted and the application sending the data uses a proprietary binary protocol that can be understood only by the application for which the data is intended. A well-designed business application will use these techniques to minimize the chance of this unauthorised use of data occurring.
There is also a risk that what may appear to be an application making an authentic request to access an organisation's data may in fact be coming from what we commonly refer to as a hacker. A well-designed business application will allow an administrator within the organisation to maintain a list of users' names and passwords and will only allow those users that have entered the correct name and password to gain access to the system. It will also record unauthorised access attempts in order to determine if a hacker has been trying to break into the system.
The User Interface - Data Integrity
The data in a computer system is only as reliable as the information entered into it by the user and it is a fact of life that users can and do make mistakes. However there are a number of features that can be incorporated into the design of a business application that can lessen the risk of mistakes by the user and at the same time make the system easier and more enjoyable to use.
Firstly and foremostly the user should not under any circumstances be allowed direct access to the data in the underlying tables of the database. The only way a user should be allowed to manipulate data is through the clearly defined and purpose-built screens of the business application. Unfortunately many of the most readily available and easily understood programming tools are not very good at preventing users from gaining direct access to the underlying data. (Having said that there is some justification for allowing a sophisticated user to use an SQL-based tool to create their own views of the data for reporting purposes.)
Secondly the screens used to view and enter the data should prevent the user from accidentally changing it by forcing them to indicate when they wish to begin editing. If an error is made during this process the user should be given the option of cancelling the changes and reverting back to the original data as it exists in the database. The user should then be required to indicate whether or not they wish to save their changes to the database.
Thirdly the data entry screens should allow the user to move around the controls and enter data without requiring the use of a mouse. Users that are adept with a keyboard often find it a significant irritation when they are forced to move their hands away from the keyboard in order to use the mouse during a repetitive data entry process.
A Public Interface - The HTTP Protocol
Often a business will want to make some of its data available to the public at large through an interface known as the World Wide Web. This term refers to those computers connected to the Internet that provide information using the HTTP protocol. This information is most frequently displayed in a web client application (also known as a web browser.)
In order to ensure that the information presented to the public is as up-to-date as possible and avoid having to replicate or extract the data from the database for use by another HTTP server (also called a web server) it makes good sense for a business application to provide its own web server. This is particularly the case if the organisation wishes to provide a web site where the user can enter data directly into its database.
In a three tier architecture where there is a client application, a server application and a database server the HTTP server can be incorporated directly into the server application so that requests for data can be co-ordinated between the application clients within the organisation and the web clients in use by the public at large. This arrangement will allow data to be cached by the server application if necessary in order to improve response times for the requesting clients.