Tag: complex system

Aha Insight from the US Army: Strategy for Complex Environments

Strategy/policy approaches are often inadequate for figuring out what we should do because they ignore reality. The world is uncertain, but we try to plan based on what we believe is certain. We should be planning based on hypotheses that the world operates like a complex adaptive system, everything is always changing, and most things about the future are uncertain.

A complex adaptive system is a good model for how the real world operates. Like the real world a complex adaptive system is constantly changing, but not changing in a predictable, linear, incremental fashion. When faced with a real-world situation—a situation that is hard to describe because of poor-quality information, many interconnections, and many uncertainties—we can start with a framework for complex adaptive systems and apply a strategy/policy decision process that will enable us to make some sense of a complex, dynamic situation, understand the limits of that sense, and generate good strategies for the situation. And because we’re dealing with a dynamic system, we need a process that will accommodate the change that will be continuous and prepare us to respond to that change as necessary.

New US Army Doctrine for Complex Environments

One very interesting approach put forth for achieving goals in a complex adaptive system is the proposed management doctrine of the US Army for designing and executing military operations in complex operational environments, like insurgency situations in Afghanistan and Iraq. The U.S. Army Capabilities Integration Center, Training and Doctrine Command, United States Army recently described that doctrine in Commander’s Appreciation and Campaign Design, Department of the Army TRADOC Pamphlet 525-5-500.

As outlined in the US Army’s pamphlet, complexity is significant to military commanders because it’s a basic characteristic of operational problems. The military defines an operational problem as a discrepancy between the state of affairs as it is and the state of affairs as it ought to be that compels military action to resolve that discrepancy. The complexity of operational problems ranges from tame, well-structured problems to those that are extremely complex and ill-structured. Unfortunately, most management doctrine today in the military—as well as in civil-government service and private corporations—is for well-structured problems hence the need for a different doctrine and the understanding for when to apply it.

Well-structured problems are controlled through technical reduction and a systematic method-based solution. They are easier to recognize and characterize. Most modern tactical doctrine of military services fits this mold, specifying the tasks, conditions, and standards for every task in warfare from tank gunnery to conducting a defense. The most structured problems often have just one correct solution, and success requires learning to perfect the established technique.

Medium-structured problems are more interactively complex, and while there is no single correct solution, personnel will agree on the structure of the problem, appropriate tasks, and the end state, but may disagree about how the general principles in doctrine are applied on a specific piece of terrain against a specific enemy. In a medium-structured problem, it is possible for a defense to succeed against one enemy commander yet fail against another under precisely the same circumstances. The difference between success and failure in this case is a function of interactive complexity, rather than a structural or technical difference between the two

In planning for a well- or medium-structured military situation, personnel will focus on the linear phenomena rather than the non-linear. They will focus on the practice of war, which is based upon professional consensus and is authoritatively prescribed in doctrine, rather than the art of war, which is based upon intuition and genius. Leader development processes are not designed to produce geniuses because geniuses are idiosyncratic. Instead, leader development processes are based on previous experience and practice and the linear phenomena that can be controlled and on whose structure personnel can agree.

Ill-structured (also called wicked) problems require a completely different orientation. Ill-structured problems are interactively complex, non-linear, and chaotic—and therefore the most challenging. Unlike well- or medium-structured problems, smart people will disagree about how to solve an ill-structured problem, what should be the end state, and whether the desired end state is even achievable.

A number of challenges need to be overcome to address an ill-structured problem.

  • The first challenge is that at the root of the lack of consensus about how to solve an ill-structured problem is the difficulty in agreeing on the structure of the problem. Unlike medium structured problems, it is not clear what action to take, because the nature of the problem itself is not clear. There’s not even a definitive way to formulate an ill-structured problem. For an ill-structured problem, the information needed to understand the problem depends upon how one defines it. And the solution depends upon how one understands the problem, or how one answers the question: “What is causing this problem?” Ill-structured problems rarely have a single cause, and different stakeholders will see the relationships between the causes and their importance differently. Thus, understanding and formulation depend to some degree upon the perspective of the problem-solver rather than some objective truth. Thus an ill-structured problem cannot be known, but must be surrounded.
  • The second challenge in addressing an ill-structured problem is one cannot understand an ill-structured problem without proposing a solution. Understanding the problem and conceiving a solution are identical and simultaneous cognitive processes. For example, if one describes bankrupt commodity producers as the result of falling demand and lower commodity prices from a weak economy, our solution will be different than if we describe bankrupt commodity producers as the result of building too much supply capacity. The formulation of the problem points in the direction of a particular solution.
  • A third challenge is every ill-structured problem is essentially unique and novel. Historical analogies can provide useful insights for individual aspects of the larger problem, but the differences among even similar situations are profound and significant. The political goals at stake, the stakeholders involved, the cultural milieu, the histories, and other dynamics will all be novel and unique to a particular situation.
  • A fourth challenge is that ill-structured problems have no fixed set of potential solutions. Since each ill-structured problem is a one-of-a-kind situation, it requires a custom solution rather than a standard solution modified to fit circumstances. For well- and medium-structured problems, best practices offer standard templates for action, standard ways of doing things that have to be adapted to specific circumstances. There is no similar kit of generic solutions for ill-structured problems. The dynamics that make an operational problem unique also demand the design of a custom solution. Additionally, there is no way to prove that all solutions to an ill-structured problem have been identified and considered.
  • The fifth challenge is that solutions to ill-structured problems are better or worse, not right or wrong. There is no objective measure of success and different stakeholders may disagree about the quality of a solution. The suitability of a solution will depend upon how the individual stakeholders have formulated the problem and what constitutes success for them.
  • The sixth challenge is that ill-structured problems are interactively complex. Operational problems are socially complex because people have tremendous freedom of interaction. Since interactively complex problems are non-linear, a relatively minor action can create disproportionately large effects. The same action performed on the same problem at a later time may produce a different result. Interactive complexity makes it difficult to explain and predict cause and effect.
  • The seventh challenge is that every solution to an ill-structured problem is a ‘one-shot operation.’ Every attempted course of action has effects that create a new situation and cannot be undone. The consequences of direct action are effectively irreversible. Whenever actions are irreversible and the duration of their effects is long, every attempted action counts.
  • The eighth challenge is there is no immediate and no ultimate test of a solution to an ill-structured problem. The perceived quality of a solution to an ill-structured problem can change over time; yesterday’s solution might appear good today, but disastrous tomorrow as the unintended effects become clearer. Measurable results to a particular action may not appear for some time. This time lag complicates assessment enormously, because in the meantime the operational command may have executed other actions, which will make assessing cause and effect even more difficult.
  • The ninth challenge is that ill-structured problems have no ‘stopping rule’. It is impossible to say conclusively that such a problem has been solved in the sense that a student knows when she or he has solved a math problem. Work on an ill-structured problem will continue until strategic leaders judge the situation is “good enough,” or until stakeholder motivations, will, or resources have been diverted or exhausted.
  • The tenth challenge is that every ill-structured problem is a symptom of another problem. The causal explanation for a problem will determine the range of possible solutions. Yet, solving one problem often reveals another higher-level problem of which the original one was a symptom. The level at which an operational problem is solved depends upon the authority, confidence, and resources of a particular commander. One should not simply cure symptoms, but should rather strive to solve the problem at the highest possible level. However, if the problem is formulated at too high a level, the broader and more general it becomes and therefore the less likely it is to solve particular aspects of the specific problem.
  • The eleventh challenge is that the problem-solver has no right to be wrong. The writ of an operational commander and his staff is to improve the state of affairs as his superiors perceive it. Like others in senior positions of an organization, he is responsible for the consequences of the actions he generates.

Given these challenges facing military leaders, the process for confronting an ill-structured problem—for trying to have a healthy future in a complex adaptive system—has to be very different from how situations were generally approached in the past. The US Army pamphlet identifies several key features for a new approach:

  • Shared development of plausible scenarios. The task of defining the problem will require much more work and insight than before, and it’s not something that can be done top down by the commander. Instead, given the uncertainties and complexities of future situations, commanders must approach the problem with a holistic systems perspective using both bottom up and top down inputs. Ultimately, developing a shared understanding of the external environment situation is a critical success factor in defining the problem and quantitative models won’t be very useful. Instead, qualitative, heuristic approaches will be needed to create a shared understanding of the circumstances and possibilities. Based on my experience, a very good holistic approach for creating a shared understanding of the problem or challenge, and one that recognizes the problem or challenge is going to evolve over time with the actions of the participants and unfolding dynamics, is the scenario development process.
  • Shared strategy decision-making. Since each strategy and action solution will be a function of the shared understanding developed for the particular problem or challenge, a top-down developed solution probably wouldn’t work. The process for developing a solution should involve all the key stakeholders and utilize the scenarios.
  • Prepare for continuous change. Since a critical feature of insurgency conflicts is how rapidly the situation or problem changes over time, all participants are in an unrelenting struggle to learn and adapt rapidly, and they do. Over time the original shared understanding of a problem or challenge will no longer be valid and will need to be changed, resulting consequently in the need for an adjusted strategy and action plan. Organizations will need:
    • A continuous process of strategy, with abilities for ongoing monitoring, assessing, and making adjustments to the action plan;
    • Companies and squads closest to the action in the field need the responsibility and authority to conduct that process and make the adjustment decisions. The best information and awareness of the changing situation is in the field and there’s often not enough time for those in the field to brief those at the top and involve them in a process to develop a suitable response;
    • Individual squad members to be utility players more than specialists, able to play multiple roles as needed. New task responsibilities of the squads are assigned to those best suited to carrying them out. The activities of individuals will shift as required by the circumstances.

Implication for Policy and Strategy Development for Global Situations

If this Army doctrine makes sense for complex operational environments, then private corporations, local government agencies, and community planning committees should all use similar principles when developing strategy. But instead they employ processes that are linear and top down and don’t have a good chance of succeeding. They focus on the certainties rather than the uncertainties; they look at issues in isolation rather than being part of an interconnected environment; they forecast or assume one future rather than anticipate a range of plausible futures; they assume a best solution can be found; and they don’t plan for the inevitable change in the future after implementation begins.

Predicting Future Commodity Developments

The world is always being surprised by the size of commodity-market changes. For something as basic and well understood as commodities, you would think it would be straightforward to predict future outcomes. But experts, strategy makers, and leaders generally fail in this regard—often big time—because social interactions, business dynamics, geopolitical forces, and the bio and physical forces of the earth unfold in complex ways, shaping commodities supply, demand, and price in unforeseen ways, and making prediction impossible. Still, policy and strategy makers have made some progress in the last fifty years. For a start, we know much more about how complex systems operate, and we have learned techniques for characterizing complex environments and projecting possible outcomes so policy and strategy makers in the face of all the uncertainty can make better decisions.

The language of complex social-physical systems provides us a tool for understanding many global trends and creating strategies for how we might respond. What is a social-physical system? It is a complex dynamic of social, political, economic, physical-environmental forces interacting with each other as one interlinked system. Every player in the system is a stakeholder, playing a role and acting in that system. Change in one type of force, social or economic or physical, inevitably affects the others, and it’s not possible to understand the dynamics of one arena in isolation from the others. Social-physical systems can be defined at many levels or scales: the highest level is the Earth system, but any real world issue, like the copper commodity market, the urbanization of Southern California, the development of renewable-energy sources in Europe, or the development of Russian oil and gas reserves above the Arctic Circle, can be defined as a social-physical system. At whatever the level or scale, the result of the cross-acting social-physical forces is a complex adaptive system that behaves in nonlinear ways and is largely unpredictable.

Resilience Thinking and Complex Systems

Much of my understanding of the basic dynamics of a complex adaptive system comes from a small but brilliant book called Resilience Thinking: Sustaining Ecosystems and People in a Changing World by Brian Walker and David Salt. This book provides a great framework for understanding a social-physical system and very useful metaphors for visualizing such a system.

According to Walker and Salt, the complexity of the many linkages, actions, and effects that make up a social-physical environment system is such we can never predict with certainty what the exact response will be to any act or input in the system. The system is relatively stable, but the unfolding behavior of the system cannot be predicted by understanding the individual mechanics of the component parts or any pair of interactions.

Another feature of a social-physical system is that it has the potential to exist in more than one kind of stable state in which the dynamics of the specific forces, structure, interactions and responses would be different. A system will transition from one state to another when changes in several interlinked forces result in a crossing of a threshold and the complete reconfiguration of the system to a different state or dynamic. Shocks and disturbances to a system, such as from a natural disaster, market disruption, etc., can push the system across a threshold into a different state or dynamic, often with unwelcome surprises.

Resilience is the capacity of the system to absorb disturbance, to undergo change, without crossing a threshold to a different system state with its different identity and dynamic. This capacity to undergo some change without a radical change in general dynamic is defined as the resilience of the system. The more resilient the system, the more anti-fragile (a term from Nassim Nicholas Taleb, the author of The Black Swan and Antifragile: Things That Gain from Disorder) it is.

In the metaphor of a ball moving in a basin, the ball is the current state of the system. The basin in which the ball is moving is the set of possible states that can be reached by the system with the general dynamic of the interlinked forces. The system is stable as long as the ball stays in the basin. The boundary or lip of the basin is the threshold. Within the basin, the ball tends to roll to the bottom. In system terms it tends toward some equilibrium state. The shape of the basin is always changing as external conditions change and so is the position of the ball. The net effect is the system is never in equilibrium (i.e., with the ball stuck at the bottom). The distance of the ball from the threshold measures the system’s resilience. The resilience of the system is how much change can occur in the basin and in the ball’s (i.e., the system’s) trajectory before the ball (system) leaves the basin. The closer one is to the threshold, the less it takes to be pushed over. If the conditions cause the basin to get smaller or the ball to be moving faster, resilience declines, and the potential of the ball (system) to cross into a different basin becomes easier.

While social-physical systems involve many interlinked forces, their trajectories in a basin are often governed by only a handful of driving forces. To prevent the ball from leaving the basin, it would be important to identify and understand the drivers that could cause the ball to cross the threshold, know where the threshold actually is, and enhance those aspects of the system that would enable it to remain resilient or adaptable. This can include moving the thresholds, moving the current state of the system away from the threshold, or making a threshold difficult to reach. If the system is stuck in an undesirable basin dynamic, it might be impossible or too expensive to manage the threshold or system’s trajectory, and it might be necessary to transform the very nature of the system.

The scale of the system that we’re focused on (usually a global market or geographic region) is connected to and affected by what’s happening at the scales above and below, both in time and space. For example, the annual maintenance activity for an urban highway system is linked to the longer scale for transportation investments in that urban area that are linked to business growth in the region and demographic changes, etc. At each scale, the system is changing, but the linkages across scales play a major role in determining how the system at another linked scale is behaving. Disturbances at lower scales can influence the dynamic of a system at a higher scale. In the end, every system is composed of a hierarchy of linked morphing systems operating at different scales (both in time and space).

So What Insights about Commodities Can We Develop with This Complex-Systems Lens?

Change is normal: Plan for Change. One reason commodities markets are constantly changing is technology innovation. Innovations can help lower production costs, develop new production sources, create new products, and enable increased demand. They can’t be individually predicted, but a wide variety in innovations will shape commodities outcomes throughout the world for the foreseeable future. The scope and scale of impacts on commodities from technology innovation:

  • Agriculture-seed hybrids and herbicides. Technology innovation by large corporations has had a tremendous impact on global food supply and the costs of food. Several large companies like Monsanto, Syngenta, Bayer, Dow Chemical, BASF, and DuPont now dominate the business of food supplies. A key competitive factor for these companies that drives corporate decisions is their ability to innovate in the future.
  • Commercialization of solar energy. The demand for solar-energy solutions is a major factor in national energy policies in both developed economies as well as developing economies. That demand is directly influenced by the production costs of the solar solutions as well as the cost of production for competitive hydrocarbon supplies. Africa is a continent with a lot of sun. It’s also a continent that lacks electricity in many parts. The rapidly falling costs of solar panels may mean that much of Africa’s growth in electricity demand could be supplied by solar.
  • Cyberattacks on financial institutions. Commodity markets are global, and buyers and sellers in different parts of the world depend on a secure global payments systems. The Society for Worldwide Interbank Financial Telecommunications (SWIFT), a network that banks use to move money around the world, recently announced its concern about cyber-heists. Experts believe many attacks have yet to be discovered because the criminals are always getting better.

Future market outcomes can’t be predicted because of the complex interactions of market, political, economic, social, and environmental forces.

  • Global Economy. Most commentators discuss the world economy as if it’s a static system where good logic should enable reasonable projections in the short term (the next year) and the long term (the next decade). Because the global economy is a complex system, it’s just not possible to predict anything. A recent headline in January 2016 in the Wall Street Journal before the World Economic Forum in Davos said, “Welcome to the Crisis Economy, Where Tumult Reigns.” The article argued the “economic and geopolitical outlook appears more unsettled this year than in the past” because of the uncertainties in geopolitics, commodities, energy, and the financial markets. But that unsettled state didn’t stop the article from including projected single-point growth rates for China, the United States, Japan, Russia, and the Eurozone for 2016. The article was correct to highlight the important uncertainties facing the world. But it should not have shown single-point forecasts for the different nations. Instead it should have argued that when tumult reigns the range of potential economic outcomes for the different nations in 2016 could be quite wide and then it should have provided some indication of those ranges. So instead of saying the growth rate for China’s GDP was going to be 6.3% in 2016 compared to 6.8% in 2015, it should have said the growth rate for China’s GDP could be as low as 3% and as high as 8%. This range of 3% to 8% has a very different implication for commodities markets than the single point of 6.3%.

Extreme change may indicate a threshold boundary is about to be or has been breached.

  • A recent article in The Economist on May 28, 2016, “Global Warming: In the red. The end of El Niño sees temperatures soar across the world,” said that the current year would most likely be the warmest on record, and by a wide margin. A major factor in the high heat could be the just-ending El Niño, the Pacific Ocean dynamic of changing atmospheric heat and moisture. The article makes links between the recent high temperatures, the ocean’s heat-storing abilities, and climate change, but says those are tenuous, and that “the complexity of climate systems means temperature variations cannot be explained by a single cause.” For those watching though, the ball might have just left the basin. Supply of agricultural commodities is affected by both the immediate and long-term heat and moisture changes in the atmosphere and clearly will be severely impacted by future higher temperatures. At the same time, warmer temperatures in the upper-latitude and Arctic areas could change the mineral and metal supply opportunities. New sources for many materials may become financially viable in the next 20 years.