Evaluating Architectures

by Philip Boxer

Architectural evaluation usually focuses on the supplier’s domain from which a product or service is to be provided in response to some customer’s direct demand.  The supplier approaches architecture from the starting point of the scenarios in which their products or services are used, defining the functional and non-functional characteristics of the customer’s demand, in relation to which the supplier can then design a satisfactory architecture (Bass, Clements, and Kazman 2008). 

The supplier combines task systems and sentient systems (Baldwin and Clark 2000), which can be described as a three-way relationship between three sub-models (shown in Figure 1) of an organization (1) controlling resources (2), constraining and constrained by the precedence relations imposed by the nature of the task system (3). Primitive relations are defined between the elements within and between these sub-models, from which complex relations can be derived that describe architectural patterns (Carley and Krackhardt 1998). 

Figure 1 assumes stakeholders with sovereignty over the supplier’s sentient system, ensuring that their interests are served. What patterns may be described depend on the primitive relations defined by the modeler (Bass, Clements, and Kazman 2008), but can be used to consider the potential consequences of misalignment between the sentient system of the supplier and the desired architecture of the task system (Cataldo et al. 2006).  Elaboration of the primitive relations within and between the sub-models can give insight into architecture not only at the level of the sentient system but also at the level of the relations between software modules and data within the task system itself (Sharman and Yassine 2004; MacCormack, Baldwin, and Rusnak 2010; Colfer and Baldwin 2009).

Figure 1: The sub-domains of the supplier

However, suppliers pursuing indirect value within an ecosystem must relate to the demands of their indirect customers, and in so doing must surrender elements of their sovereignty.  For example, the interests of clinics run by orthotics suppliers cannot be assumed to be the same as those of their patients who are using orthoses to manage their conditions.  This means adding a fourth sub-model that can be used to describe the indirect demands arising within the variety of customer situations to which the supplier must respond, shown in Figure 2

Figure 2: Adding the demand-side

Projective analysis aims to include a model of this variety of demands, creating an environment of indirect demands placed on suppliers and customers.  Adding this fourth sub-model therefore enables a model to be built of both the supply-side and the demand-side architectures, providing a basis from which to judge both whether it is in the supplier’s economic and competitive interests to satisfy indirect demand, and if so, how.

The relation to the ontologies of existing methods

A projective analysis is a model of a supplier’s models, using methods of conceptual or structural modeling to describe three levels of openness in the supplier’s relationship to the ecosystem.  (See Conceptual or Structural Modeling, for the relationship between this and other methods). 

These methods are constrained by the ontologies that they permit.  Their ontologies are ‘upper’ ontologies defining the different categories of things and relationships between things that a modeling method makes it possible to represent (Sowa 2000), determining the forms of knowledge that can be represented in particular domains (Amin and Cohendet 2004; Hopkins and Jenkins 2008).  Many such frameworks exist, for example Zachman (Zachman 1987), DoDAF (U.S. Department of Defense 2009), or Federal Enterprise Architecture (The Office of Management and Budget 2007).  The characteristic of all of these ontologies is that they model physical and digital structures and behaviors, and the accountability hierarchies under which these operate are subject to a single sovereign supplier, whether virtual or not.  They assume a closed value space, however, modeling only two levels of openness.  As a consequence, only relations reducible to dyadic form need be modeled.

In order to extend these ontologies to include a model of an open value space, projective analysis needs to admit multiple sources of sovereignty by adding representations for network relationships across multiple suppliers and customers, and for the organization of direct and indirect customers’ contexts-of-use (Boxer and Garcia 2009).  This demands an expanded ‘upper’ ontology capable of representing different ways of aligning the supply-side to the demand-side, in contrast to the solely supply-side focus of a modeling method such as IDEF or UML, working across the boundary between systems and their immediate organizational context  (Noran 2004).  This demands complex operators reducible to triadic relations.  By analyzing patterns of relationship across the sub-models in Figure 2 using this expanded ontology, the different forms of alignment can be analyzed with respect to different forms of demand. 


These modeling methods will describe the stratification of an ecosystem and provide a basis for analyzing its structural characteristics.  The resultant analyses provide support to a supplier evolving the agility of its business architecture in order to capture indirect value within the larger ecosystem.

Amin, A, and P Cohendet. 2004. Architectures of knowledge: firms, capabilities and communities (Oxford University Press).

Baldwin, C., and K. Clark. 2000. Design Rules: The Power of Modularity (MIT Press).

Bass, L., P. Clements, and R. Kazman. 2008. Software Architecture in Practice 2nd Edition (Addison-Wesley).

Boxer, Philip, and Suzanne Garcia. 2009. “Limits to the use of the Zachman Framework in Developing and Evolving Architectures for Complex Systems of Systems.” In SATURN. Pittsburgh: SEI.

Carley, K.M., and D. Krackhardt. 1998. “A PCANS Model of Structure in Organizations.” In International Symposium on Command and Control Research and Technology. Monterey, CA.

Cataldo, M., P.A. Wagstrom, J.D. Herbsleb, and K.M. Carley. 2006. “Identification of Coordination Requirements: Implications for the Design of Collaboration and Awareness Tools.” In Computer Supported Cooperative Work. Banff, Canada.

Colfer, L., and C. Baldwin. 2009. ‘The Mirroring Hypothesis: Theory, Evidence and Exceptions’, Harvard Business School Working Paper.

Hopkins, R, and K Jenkins. 2008. Eating the IT Elephant: Moving from Greenfield Development to Brownfield (IBM).

MacCormack, J., C. Baldwin, and J. Rusnak. 2010. ‘The Architecture of Complex Systems: Do Core-Periphery Structures Dominate?’, MIT Sloan School of Management, Working Paper 4770-10.

Noran, O. 2004. “UML vs IDEF: An Ontology-oriented Comparative Study in View of Business Modelling.” In 6th International Conference on Enterprise Information Systems, 674-82. Portugal.

Sharman, D.M., and A.A. Yassine. 2004. ‘Characterizing Complex Product Architectures’, Systems Engineering, 7: 35-60.

Sowa, John F. 2000. Knowledge Representation: Logical, Philosophical, and Computatinal Foundations (Brooks/Cole).

The Office of Management and Budget. 2007. “Federal Enterprise Architecture Consolidated Reference Model V2.3.” In.

U.S. Department of Defense. 2009. “DoD Architecture Framnework Version 2.0.” In.

Zachman, J.A. 1987. ‘A Framework for Information Systems Architecture’, IBM Systems Journal, 26: 276-92.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.