Complex systems

One hundred years from now, the role of science and technology will be about becoming part of nature rather than trying to control it. - Joichi Ito, MIT Media Lab director

In complex systems, trade-offs are everywhere. Engineers designing technological artifacts carefully balance competing objectives. An economy allocates resources like land, labor and capital to one use or another. Even evolution faces trade-offs.

Optimal bundles for three different incomes--2 normal goods.
Price Theory, David D. Friedman

Economists study trade-offs using tools like utility and indifference curves. The figure shows the optimal bundle consisting of two goods (apples and oranges, of course) for three different income levels. The curves represent trade-offs of equal utility; red lines represent income. They intersect at the points X, Y and Z, the points of highest utility achievable at each level of income.

Pareto optimality

At a macro level, millions of competing buyers and sellers make thousands of such trade-offs daily. Such multiobjective optimization problems often shake out in a way described by Pareto optimality. If an allocation of resources in an economy is such that no-one could be made better off without making someone else worse off, that allocation is said to be Pareto optimal. Competitive markets can be shown to deliver a Pareto optimal allocation of resources. Rather than a single optimal point, the Pareto optimal frontier defines a surface in high-dimensional space of best possible trade-offs. The notion of Pareto optimality turns out to be portable to other fields of study.

Shoval et al., 2012

Evolutionary Trade-Offs, Pareto Optimality, and the Geometry of Phenotype Space (Shoval et al., Science 2012), a short-but-awesome paper from Uri Alon's lab, applies the concept of Pareto optimality to evolutionary biology. The morphology of finch beaks, ant heads, and bat wings are all re-examined in the light of Pareto optimality. The idea applies at the molecular level as well. Microbial gene expression moves along an axis between competing priorities of growth and stress response.

In each case, there is a trade-off among competing objectives - performance at various specialized tasks. One of the simplifying ideas in the paper is to work in terms of measurable task performance rather than the resulting contribution to fitness. Fitness, like the economic concept of utility, is hard to quantify and maybe not precisely knowable.

For example, a beak might be optimized for a particular diet: cracking hard seeds, chewing soft seeds or plucking insects from their hiding places in the bark of a tree. The Pareto frontier defines solutions in which it is not possible to improve performance of one task without sacrificing performance of another. Selection favors phenotypes near the Pareto front. Local environment dictates the distribution of species and individuals along the front.

Complex adaptive systems

Biology, economics and, to some extent, engineering are all turning into the study of complex adaptive systems. Economics is the ecology of money. Biology is just the economics of the jungle. Here are a few big ideas and questions that generalize across disciples.

• Biology - adaptability
  Living systems are adaptive. They deal with extremes of environment, degrading gracefully with damage and keep on functioning in situations that stop mechanical systems dead. Economic systems, too, are adaptable. New information is constantly being factored into prices carrying signals that balance the needs of consumers with productive capacity. So far, engineered systems fall short in terms of adaptability.
• Engineering - modularity A hallmark of engineered systems is modularity. In software, we struggle against the entropy of spaghetti-code. But, somehow modularity emerges spontaneously in living systems, their processes algorithmic in nature. Evolved systems are clearly different from engineered systems and are clearly far from fully modular, but, to a surprising degree, modularity is found in biology. What forces cause modularity to arise and what are the opposing forces?
• Economics - distributed information Information flows through complex systems. Price is an emergent property integrate over millions of individuals each making hundreds of economic decisions daily. Living things, too, aggregate information in order to compute how best to allocate their budget of energy and scarce nutrients to achieve competing objectives. How do complex systems achieve balance, find and maintain stable states (homeostasis) and tend towards optimality? How do feedback mechanisms promote or disrupt stability?

These some of the fundamental features of complex systems: adaptivity, information flow and modularity. (I'm calling the book “Darwin, Hayek, Knuth”. Nice, huh?) Adaptability and robustness emerge from information flowing across interconnected networks often hierarchical in structure.

Marshall Clemens and the New England Complex Systems Institute

After bouts of physics envy, biology and economics have arrived at roughly the same place. The toolkits used to study both are converging on the same set of statistical techniques and machine learning algorithms for computationally deriving models from big data.

Both fields are wrestling with the same problems: complexity and uncertainty. With an emerging understanding of the properties of complex systems, we might see more robust engineered systems and the ability to re-engineer systems like economies and metabolisms or ecosystems or at least know the limits to which these systems can be engineered. Learning to engineer complex systems is the frontier of the 21st century.

More

• Evolutionary Trade-Offs, Pareto Optimality, and the Geometry of Phenotype Space Shoval et al., Science 2012
• Nature, an economic history by Geerat J. Vermeij
• Robert Frank of Cornell University and author of The Darwin Economy talks about Darwin's vision of the competitive process, and appeared on PBS in 2011 Was Charles Darwin the Father of Economics as Well?
• The Architecture of Complexity (Herbert Simon, 1962)
• Barabasi also wrote a paper titled The Architecture of Complexity
• Caldwell on Hayek
• Evolutionary Algorithms for Solving Multi-Objective Problems: section 1.2.3 Multiobjective optimization problems
• Yaneer Bar-Yam's New England Complex Systems Institute predicts food riots next year.
• To Understand Is to Perceive Patterns

Phenotypic constraints drive the architecture of biological networks

Biological networks are often compared to random networks in terms of properties like degree distribution, clustering, robustness and over-representation of network motifs. In a talk this morning, Areejit Samal, a Postdoctoral Fellow in the Price lab at ISB, proposed a new null model based on Markov Chain Monte Carlo (MCMC) sampling to generate realistic benchmark ensembles for metabolic networks and gene regulatory networks. Based on this improved background model, the conclusion is that “phenotypic constraints drive the architecture of biological networks”, which is also the title of the talk.

doi:info:doi/10.1371/journal.pone.0022295.g004

Applied to the metabolic model of E. coli, the sampling technique involves two steps. In the swap step, a reaction is removed from the network and a new reaction is drawn randomly from the KEGG database to replace it. The proposed new network is then tested by flux-balance analysis. If the new network is viable, it is accepted. If not, it is rejected. So, only networks that are biochemically plausible are sampled.

The technique has implications for systems and synthetic biology as well as evolutionary theory. By approximating the space of all possible networks that might lead to a viable functioning organism, we can better understand what properties these networks have, and maybe better design new networks. By changing the acceptance criteria to enforce viability in a second environment, the algorithm nicely models the evolutionary emergence of modularity.

Samal also showed the same general algorithm applied to the gene regulatory network controlling flowering in Arabidopsis.

I especially appreciated seeing a really cool application of Markov Chain Monte Carlo sampling, a topic covered only a couple weeks back in the Probabilistic Graphical Models class.

Links

• Randomizing Genome-Scale Metabolic Networks, Areejit Samal, Olivier C. Martin (2011)
• Environmental versatility promotes modularity in genome-scale metabolic networks (2011)
• Genotype networks in metabolic reaction spaces (2010)
• Neutralism and selectionism: a network-based reconciliation, A. Wagner (2008)
• The road to modularity, G. Wagner (2007)

Network Science

Network analysis is hip. Applications range over social networks, security, biology, and economics. At this point, you'll hardly be the first one to the party, but if you want to give network science a try, here's a random grab-bag of resources to get started.

Coordination of frontline defense mechanisms under severe oxidative stress, Kaur et al. 2010

Learning network science

Jon Kleinberg, a professor of computer science at Cornell University, co-wrote Networks, Crowds, and Markets: Reasoning About a Highly Connected World along with David Easley. He also wrote Algorithm Design, an undergraduate textbook.

A 2004 review paper by Barabasi and Oltvai Network biology: understanding the cell's functional organization. covers a broad range of applications of networks in modern biology. Barabasi is also author of Linked.

A Science special issue on networks, from July 2009, revisits the foundations of network analysis, and delves into applications to ecological interactions, counter-terrorism, and finance.

Video and slides are available for Drew Conway's presentation on social network analysis in R, which mostly focuses on software tools.

Tools for analyzing networks

Software tools for working with networks include the R packages graph, igraph, network. Also, the NetworkX library for Python looks quite powerful. Visualization tools tend to come and go, but some well-known tools are: Cytoscape, Gephi, and GraphViz.

More network stuff

• Synthetic biology: predicting and optimizing gene regulatory networks
• Uri Alon's Network motifs: theory and experimental approaches