Evaluating technical architecture: 11 key criteria and how to apply them

Technical architecture provides a way to describe, evaluate, and plan the evolution of the information technology that IT manages and the enterprise relies on.

The previous installment of this series on building effective IT provided a framework for describing technical architecture, breaking it down into portfolios and sub-portfolios, including applications (systems of record, integration, and satellite apps), data (both structured and unstructured), and technology (facilities, infrastructure, and platforms).

This framework enables you to identify and categorize what you have. So far so good.

But it doesn’t tell you whether what you have is what you ought to have. That’s the challenge this article deals with, outlining how to look at your technical architecture, now that you’ve defined it, and offering key criteria for evaluating your technical architecture, now that you’ve categorized it.

The two perspectives of technical architecture

Descriptions of technical architecture fall into two complementary perspectives: holistic design and the portfolio view.

Holistic design describes what each component of the architecture does — the capabilities it provides — and how the components fit together to create a functional whole out of individual parts.

Portfolio view, on the other hand, is rooted in investment theory. It equates components in a technical architecture to equities in an investment portfolio. Just as investors periodically review their portfolios to decide which equities to buy more of, keep at current levels, or sell, technical architects, according to this model, periodically review the health of each component of their technology portfolios to decide which should continue as standards and which should be phased out in favor of alternatives that do a better job of providing the needed functionality.

But unlike an investor, a technical architect has more choices than just buy, hold, and retire. Technical architects call their choices dispositions, which I will cover in the next article in this series.

For now, just keep in mind that without holistic design, IT will be overseeing a bunch of poorly conceived stuff. Without a portfolio view, it will find itself overseeing a well-engineered house of cards: Everything fits together, but you wouldn’t want to live in it.

Business architecture and how it connects

Designing and planning a coherent technical architecture isn’t possible without mapping applications to the business functions they support. So, whoever is responsible for documenting the business architecture must provide four key pieces of information to IT’s technical architects.

Taxonomy. Here, we’re talking about a taxonomy of business functions. Three levels — capabilities (L1), responsibilities (L2), and processes (L3) — usually suffice. For example, HR (L1) includes compensation management (L2), which in turn includes payroll (L3), just as finance and accounting (L1) includes accounts receivable (L2), which includes collections (L3). If you want to be Fully Buzzword Compliant, call this taxonomy your BCM (business capability model).

Mapping. The second key piece is a mapping of which applications each function in the BCM relies on. Business architects might be tempted to map these at the capability level, but without L2 and L3 mappings, the BCM will have limited utility.

Assessment. The third key piece of information is an assessment of each BCM function’s overall effectiveness.

Importance. That leaves the fourth and most controversial: each business function’s relative importance. Two tips on this one: (1) Define importance as the impact on competitive advantage; and (2) rate them, don’t rank them.

To illustrate: You’ll never reach consensus on whether payroll is more or less important than sales, but you’ll easily reach agreement that on a five-point scale (recommended), they both deserve a top score (5) on the grounds that if you can’t sell, you’ll lose market share, and if you don’t pay your employees, you won’t have anything to sell.

These four pieces of information — taxonomy, application mappings, business function effectiveness, and business function importance — are what connect the business and technical architectures.

Worth mentioning: While the BCM often resembles the company’s organizational chart, the org chart isn’t a BCM. It’s common for organizations — especially large enterprises — to organize based on something other than function, for example, by geography, customer type, or product category. This leads to some business functions being represented in more than one part of the organization.

Evaluating your technical architecture

To evaluate technical architecture, architects need to understand: component and integration health; redundancies and opportunities for consolidation; and the quality of business function support. We’ll cover redundancies and business function support in the next installment. Here’s what you need to know about component health scoring.

Every component of each portfolio and sub-portfolio in the technical architecture ranges from being a positive asset to interfering with IT’s ability to do its work and the various supported business areas’ ability to do their work.

The complete list of criteria to use in evaluating an architecture’s components is extensive. The framework I use includes 30 potential evaluation criteria for the application layer alone. But 30 is too many even for a single layer. From a data collection and management perspective, 10 is a practical maximum.

But I like to go the extra mile, so I’ll give you 11 to get you started. Adapt the following simplified set of criteria to your portfolios and sub-portfolios and you will have a strong basis for evaluating your technical architecture:

• Functionality: This is a Captain Obvious criterion — whether the component does what the business needs it to do or not.
• Flexibility: How well the component adapts to new and changing situations.
• Stability and performance: Captain Obvious again. An application, platform, or infrastructure component that crashes a lot and is miserably slow when it is available is a problem.
• Internal engineering: How well the component is put together (easier to determine when the component is developed in house) — its conformance to your engineering standards.
• Integration and interfaces: This one applies only to applications and data repositories. It scores how each one exchanges data with other applications and data repositories so as to synchronize overlapping data and, if you’re especially sophisticated, to synchronize overlapping business logic as well.
• Adherence to architectural principles: You took time to articulate them. Your technology should conform to them.
• Security: While most intrusions these days are the result of social engineering, that doesn’t mean the technology doesn’t need to be hardened.
• Vendor and product viability: Whether the component and its vendor have critical mass in their marketplaces — i.e., will the component be around, supported, and enhanced; will you be able to recruit the best talent to work on it.
• Currency: Whether the component is no more than one version behind what its vendor is currently shipping, or, at the other extreme, if it is no longer supported by the vendor that provides it.
• Health of lower layers: As components in one layer rely on components in lower layers, they inherit the health or deficiencies of those lower-layer components. An application might, for example, rely on data stored hierarchically in a mainframe-hosted IMS database. Most IT organizations consider IMS to be an obsolete platform, contributing to a negative platform-layer score for that application. In addition, hierarchical data design would, for most IT shops, violate the structured data design standard (normalization) giving the application a reduced score based on its information repository characteristics.

• Redundancy: A component is redundant when other, functionally similar, and possibly superior alternatives are in use elsewhere in the enterprise. If so, among the redundant components one should be established as the go-forward standard and receive a high ranking; the others should be rated as problematic because they are redundant.

Scoring

Whichever attributes you decide on to gauge the health of your architecture’s components, here are three pro tips:

Establish a common scale for all attributes. In my consulting work I’ve found that a scale of + 2 to -2, integers only, works well. It’s a five-point scale, which is in line with what we’re all accustomed to. But by centering the scale at the zero point it’s a more natural system, in that negative numbers correspond to negatives, while positive numbers correspond to positives.

Forego weighting. Think long and hard before adding weighting factors to your evaluation criteria. In principle you should, because some attributes are more important than others; but in practice, you’re likely to find that the difference in impact between scoring an attribute’s importance high or medium on a three-point weighting scale (high, medium, low), for example, doesn’t affect the outcome enough to be worth the trouble. Similarly, low importance attributes are probably unimportant enough that you can just drop them altogether.

Move beyond the spreadsheet. Don’t rely on spreadsheets to manage the data you collect about your technical architecture. Establish a database, whether homegrown or a commercial architecture management system. One reason among many: Quite a lot of the data you need to manage involves many-to-many relationships. For example: Some applications support more than one business function while most business functions rely on more than one application.

A technical architecture repository built on spreadsheets quickly turns into a hard-to-manage kludge. Also, if you manage your technical architecture data in spreadsheets the Irony Police will come knocking on your door.

Now what?

You have all the data you need. You know which business functions each application supports and what hardware and software supports each application. You know how healthy every component is, and in what ways.

And for each component you know if you have alternative components that do the same work and, if so, whether one of them does a better job of it.

It’s time to figure out where the architecture of tomorrow stays the same, where it must change, and what your priorities are for making the changes.

We’ll cover the Now What question in the next installment.