by Philip Boxer[1]
An ecosystem is a community of managerially and operationally independent organizations interacting with each other and with their environment. Software-intensive ecosystems—ecosystems in which the behaviors of the participating organizations are themselves dependent on software because of the intensive use they make of it—are an increasingly important social, financial, and political force in the world. We find examples of software-intensive ecosystems in industries concerned with such things as transport, healthcare, defense, government, and communications. These software-intensive ecosystems are different from traditional “closed-world” software systems that can be analyzed independently of the contexts in which they are embedded; and their emergent properties cannot be predicted by the designers of the software systems on which they depend. A number of key drivers underlie this change, challenging the former “closed-world” perspective on software engineering. Amongst these are the tempo at which the ecosystems are themselves expected to evolve, the ubiquity and criticality of the software on which they depend, and the entanglement not only between software systems and the way they are used by people, but also between interoperating software systems that are themselves managerially and operationally independent of each other [2]. To understand the behaviors of such ecosystems, the analysis of their architectures must not separate the software systems from the organizational contexts-of-use that depend on them. This is the case even where the interactions between the organizations within the ecosystem primarily concern the use of software itself, such as is to be found in the Microsoft and iPhone ecosystems [3].
The core-periphery distinction
The Metropolis Model [4] provides a starting point for understanding how software systems are constructed, maintained, and operated. At the heart of this model is the distinction between the core and the periphery, which has been often noted as a key architectural construct in complex software systems [5]. The core-periphery architectural pattern provides the maximum opportunity for developers at the periphery (and the producers and consumers—sometimes called prosumers—of content) to embed the behavior of a system into their own contexts, enabling their own activities or those of eventual customers and end-users within a larger ecosystem. Examples of cores include the Linux kernel, the Android platform, Facebook’s application platform, the Apache core, iPhone’s iOS platform, Hadoop Common, and so forth. Each of these “cores” provides an abstraction upon which the functionality of the actual customer/end-user-facing application (’App’) is built. The core itself provides little or no end-user value, but rather provides the architectural foundation upon which others build value. The core is typically relatively small, compared with the size of the periphery. For example the (iOS) core of the iPhone enables application developers and users to develop Apps independently that can in turn by taken up within a wide variety of contexts; and making GPS location data accessible to users and application developers on the iPhone platform has spawned a wide variety of location-based services.
This core-periphery distinction can be applied to the way modularity and its associated inter-dependencies are defined for a software system. For modules in the core, tight coupling between them creates mutual dependencies. In the periphery coupling is much looser, creating a measure of independence in the way modules can be used. The strength of the core-periphery pattern is therefore the relatively few constraints at the periphery on how modules may be used and combined, and how their use may be modified. Consequently a core-periphery structure is less over-determining of its use at the periphery than it is at the core, enabling the necessary dependencies defined by the core to be combined with chosen dependencies created by developers in their Apps.
Direct scenarios and direct interactions
Apps are developed to deliver their functionality within a direct scenario in which they are used. These direct scenarios, in addition to describing the direct interaction of some end-user with the App, also describe a quality attribute of the system (a latency goal, for example) within the context defined by the scenario. The achievement of these quality-attribute-related responses brings benefit to the stakeholders in the behavior of the App. For example, a correct response might be of great value to an end-user stakeholder if it arrives predictably within 100 ms., of moderate value if it arrives predictably within 1 second, and of little value if it arrives unpredictably or if it predictably arrives in 10 seconds.
This is depicted in the above figure, where there is one core and three different direct scenarios represented at the periphery. Each of these direct scenarios connects some end-user input—a stimulus—to an output. Consider, for example, the connecting lines for each scenario: α1β1, α2β2 and α3β3. These might represent different direct scenarios in which end-users are making voice-over-IP (VoIP) connections, for example conferencing, or saying hello to a colleague on the other side of the world, or giving a seminar. The end-users within each direct scenario might be employing different App software, but in the end they are using and sharing some resources at the core—name servers, protocol translators, routers, satellite links, IP stacks, fiber-optic cables, etc. Furthermore, the stakeholder perspective on each of these scenarios could be that the VoIP call would only be of value if latency and jitter were both kept within specified ranges.
Direct scenarios can be used to trace through the core-periphery interactions when trying to identify, measure, and understand the benefits radiating out to the end-users at the perimeter on the basis of their direct uses Apps. Scenarios such as these are necessary to analyze a shared core infrastructure, illuminating such things as contention for resources. Consider, for example, understanding the load on an operating system core when multiple processes are operating and competing for shared resources. On a larger scale, cloud computing platforms such as Amazon’s EC2 support large numbers of simultaneous users, each of which could request thousands of server instances. And social computing platforms such as Facebook have over 250 million active users each day, each of whom will consume core resources, potentially interacting through applications that are built on top of Facebook’s core application platform.
Indirect scenarios and indirect interactions
In the larger ecosystem within which Apps are being used, the under-determining effects of core-periphery systems enable Apps to be used by end-users independently of each other, but they also enable indirect interactions between Apps and end-users. The indirect scenarios associated with these indirect interactions are not necessarily related to their use of any one App, and may actually involve multiple core-periphery systems that are operationally and managerially independent of each other. For example, Facebook’s location features allow users and service providers to interact with each other in ways that are only made possible by their use of a smartphone. This potentially creates a much larger and more complex environment defined by interactions that are indirect (from the perspective of the core software systems). For example, the actors within scenarios 1, 2 & 3 may belong to different research institutions, but are all involved in a single research collaboration for which a number of core systems become a key enabling element. This research collaboration is shown in the figure below as an indirect scenario linking indirect interactions between particular direct uses of Apps to the ultimate end-use. The figure below represents this by adding an extra ‘App use’ layer to the onion:
A new challenge arises therefore for the analysis of architecture: what is the impact of these indirect interactions on the architecture of the operationally and managerially independent core-periphery systems (for example VoIP, Screen-sharing, etc.)? For example, consider the indirect effects that might arise where the end-users in the research collaboration are combining the use of VoIP, screen-sharing, and file sharing systems. These indirect interactions represent architectural challenges. For example, many users of the VoIP systems will be competing for resources, such as computation and bandwidth, affecting latency. Or the developers of those Apps may have chosen a common protocol affecting the development time of individual systems. Or charging schemes may affect an end-user’s choice of combined systems to employ (for example as a result of differential charging schemes for relatively time-sensitive internet-traffic such as VoIP packets, and relatively non-time-sensitive traffic, such as file sharing). Such considerations are of real concern to core providers because these indirect interactions can collectively have a huge effect on the performance and behavior of the core. As an example, Google Maps imposes resource restrictions on the services that it provides to other organizations, in an effort to moderate worst-case resource usage.
The multi-sided matrix
Consider the multi-sided matrix in the figure below, in which the sets of columns correspond to sets of direct scenarios associated with direct use of an App (for example particular uses of VoIP or screen-sharing), and in which each set of columns is operationally and managerially independent of the others. The rows correspond to the indirect interactions between multiple Apps supporting, each row corresponding to a particular indirect scenario (for example a research collaboration, or a marketing project). From the perspective of the App developers in the periphery, each row represents indirect interactions between Apps that creating indirect effects within the social collaborations that they are supporting. A one-sided approach to App development will only examine the mix of direct scenarios represented by a column. But each the indirect scenarios create multi-sided demands on the direct scenarios depending on how the App is expected to interact with other Apps:
Thus decisions made affecting the capabilities of core-periphery systems will also impact on the way they can participate in the indirect interactions chosen by stakeholders. The variety and scale of these indirect interactions arising between stakeholders within indirect scenarios will determine the emergent qualities of the larger ecosystem in which these interactions are taking place. Multiple X’s in a row reflect social collaborations that also present technical challenge: how will the Apps interoperate, and how will collaborations manage that interoperation within their larger contexts?
Platform scope
Returning to the example above, each column for VoIP stream, screen-sharing, and file-sharing session must have direct scenarios defining their direct scope across the rows. But since the rows themselves represent indirect scenarios, the dependencies and alignment processes between the direct scenarios along a row must also be considered, each of which presents an interoperability requirement and a potential contention for shared resources between operationally and managerially independent systems supporting the direct scenarios. Thus an end-user in the Research Collaboration in the first row above may wish to capture the content of a screen-sharing session and share that with other end-users. Such an indirect scenario imposes an interoperability requirement between the screen-sharing and file-sharing systems. A platform that enables this to be accomplished may be said to have a platform scope defined by the number of direct interactions it can enable to interoperate, shown below as a separate set of columns.
Analysing the technical *and* the social
The presence and impact of indirect interactions is an essential characteristic of ecosystems arising from the entanglement of the technological core-periphery systems with an environment of social systems. To understand and analyze these ecosystems, we therefore need to be able to characterize a statistically representative variety of indirect interactions between many independent actors across independent core-periphery systems. But if we just analyze the direct impact of the αβ paths of each direct scenario on the core-periphery architecture of a system in any given column—the traditional architectural analysis—we will not understand the collective impact of the indirect interactions within indirect scenarios across many core-periphery systems caused by the social collaborations taking place within the larger socio-technical ecosystem.
We will thus not be in a position to analyze the potential emergent effects and behaviors arising from the way the core systems are used within this larger context. It follows that we will also be unable to define the architectural characteristics of the supporting platforms needed.
Notes
[1] With special thanks to Rick Kazman for his contributions to an earlier draft.
[2] Coyle, L., et al., Guest Editor’s Introduction: Evolving Critical Systems. Computer, 2010. 43(5): p. 28-33.
[3] Jansen, S., A. Finkelstein, and S. Brinkkemper. A Sense of Community: A Research Agenda for Software Ecosystems. in 31st International Conference on Software Engineering (New Ideas and Emerging Results Track). 2009. Vancouver, Canada: IEEE CS Press.
[4] Kazman, R. and H.-M. Chen, The Metropolis Model: A New Logic for the Development of Crowdsourced Systems. Communications of the ACM Vol 52 No 7, 2009: p. 76-84.
[5] MacCormack, J., C. Baldwin, and J. Rusnak, The Architecture of Complex Systems: Do Core-Periphery Structures Dominate? MIT Sloan School of Management, 2010. Working Paper 4770-10.