Optimizing information flow in small genetic networks. III. A self-interacting gene

Deriving a genetic regulatory network from an optimization principle

Many biological systems operate near the physical limits to their performance, suggesting that aspects of their behavior and underlying mechanisms could be derived from optimization principles. However, such principles have often been applied only in simplified models. Here, we explore a detailed mechanistic model of the gap gene network in the Drosophila embryo, optimizing its 50+ parameters to maximize the information that gene expression levels provide about nuclear positions. This optimization is conducted under realistic constraints, such as limits on the number of available molecules. Remarkably, the optimal networks we derive closely match the architecture and spatial gene expression profiles observed in the real organism. Our framework quantifies the tradeoffs involved in maximizing functional performance and allows for the exploration of alternative network configurations, addressing the question of which features are necessary and which are contingent. Our results suggest that multiple solutions to the optimization problem might exist across closely related organisms, offering insights into the evolution of gene regulatory networks.

Irreversibility in bacterial regulatory networks

Irreversibility, in which a transient perturbation leaves a system in a new state, is an emergent property in systems of interacting entities. This property has well-established implications in statistical physics but remains underexplored in biological networks, especially for bacteria and other prokaryotes whose regulation of gene expression occurs predominantly at the transcriptional level. Focusing on the reconstructed regulatory network of Escherichia coli, we examine network responses to transient single-gene perturbations. We predict irreversibility in numerous cases and find that the incidence of irreversibility increases with the proximity of the perturbed gene to positive circuits in the network. Comparison with experimental data suggests a connection between the predicted irreversibility to transient perturbations and the evolutionary response to permanent perturbations.

Information content and optimization of self-organized developmental systems

A key feature of many developmental systems is their ability to self-organize spatial patterns of functionally distinct cell fates. To ensure proper biological function, such patterns must be established reproducibly, by controlling and even harnessing intrinsic and extrinsic fluctuations. While the relevant molecular processes are increasingly well understood, we lack a principled framework to quantify the performance of such stochastic self-organizing systems. To that end, we introduce an information-theoretic measure for self-organized fate specification during embryonic development. We show that the proposed measure assesses the total information content of fate patterns and decomposes it into interpretable contributions corresponding to the positional and correlational information. By optimizing the proposed measure, our framework provides a normative theory for developmental circuits, which we demonstrate on lateral inhibition, cell type proportioning, and reaction-diffusion models of self-organization. This paves a way toward a classification of developmental systems based on a common information-theoretic language, thereby organizing the zoo of implicated chemical and mechanical signaling processes.

Proofreading does not result in more reliable ligand discrimination in receptor signaling due to its inherent stochasticity

Kinetic proofreading (KPR) has been used as a paradigmatic explanation for the high specificity of ligand discrimination by cellular receptors. KPR enhances the difference in the mean receptor occupancy between different ligands compared to a nonproofread receptor, thus potentially enabling better discrimination. On the other hand, proofreading also attenuates the signal and introduces additional stochastic receptor transitions relative to a nonproofreading receptor. This increases the relative magnitude of noise in the downstream signal, which can interfere with reliable ligand discrimination. To understand the effect of noise on ligand discrimination beyond the comparison of the mean signals, we formulate the task of ligand discrimination as a problem of statistical estimation of the receptor affinity of ligands based on the molecular signaling output. Our analysis reveals that proofreading typically worsens ligand resolution compared to a nonproofread receptor. Furthermore, the resolution decreases further with more proofreading steps under most commonly biologically considered conditions. This contrasts with the usual notion that KPR universally improves ligand discrimination with additional proofreading steps. Our results are consistent across a variety of different proofreading schemes and metrics of performance, suggesting that they are inherent to the KPR mechanism itself rather than any particular model of molecular noise. Based on our results, we suggest alternative roles for KPR schemes such as multiplexing and combinatorial encoding in multi-ligand/multi-output pathways.

Quantifying information of intracellular signaling: progress with machine learning

Cells convey information about their extracellular environment to their core functional machineries. Studying the capacity of intracellular signaling pathways to transmit information addresses fundamental questions about living systems. Here, we review how information-theoretic approaches have been used to quantify information transmission by signaling pathways that are functionally pleiotropic and subject to molecular stochasticity. We describe how recent advances in machine learning have been leveraged to address the challenges of complex temporal trajectory datasets and how these have contributed to our understanding of how cells employ temporal coding to appropriately adapt to environmental perturbations.

Prediction of Time Series Gene Expression and Structural Analysis of Gene Regulatory Networks Using Recurrent Neural Networks

Methods for time series prediction and classification of gene regulatory networks (GRNs) from gene expression data have been treated separately so far. The recent emergence of attention-based recurrent neural network (RNN) models boosted the interpretability of RNN parameters, making them appealing for the understanding of gene interactions. In this work, we generated synthetic time series gene expression data from a range of archetypal GRNs and we relied on a dual attention RNN to predict the gene temporal dynamics. We show that the prediction is extremely accurate for GRNs with different architectures. Next, we focused on the attention mechanism of the RNN and, using tools from graph theory, we found that its graph properties allow one to hierarchically distinguish different architectures of the GRN. We show that the GRN responded differently to the addition of noise in the prediction by the RNN and we related the noise response to the analysis of the attention mechanism. In conclusion, this work provides a way to understand and exploit the attention mechanism of RNNs and it paves the way to RNN-based methods for time series prediction and inference of GRNs from gene expression data.

Signaling pathways have an inherent need for noise to acquire information

BACKGROUND

All living systems acquire information about their environment. At the cellular level, they do so through signaling pathways. Such pathways rely on reversible binding interactions between molecules that detect and transmit the presence of an extracellular cue or signal to the cell's interior. These interactions are inherently stochastic and thus noisy. On the one hand, noise can cause a signaling pathway to produce the same response for different stimuli, which reduces the amount of information a pathway acquires. On the other hand, in processes such as stochastic resonance, noise can improve the detection of weak stimuli and thus the acquisition of information. It is not clear whether the kinetic parameters that determine a pathway's operation cause noise to reduce or increase the acquisition of information.

RESULTS

We analyze how the kinetic properties of the reversible binding interactions used by signaling pathways affect the relationship between noise, the response to a signal, and information acquisition. Our results show that, under a wide range of biologically sensible parameter values, a noisy dynamic of reversible binding interactions is necessary to produce distinct responses to different stimuli. As a consequence, noise is indispensable for the acquisition of information in signaling pathways.

CONCLUSIONS

Our observations go beyond previous work by showing that noise plays a positive role in signaling pathways, demonstrating that noise is essential when such pathways acquire information.

The information theory of individuality

Despite the near universal assumption of individuality in biology, there is little agreement about what individuals are and few rigorous quantitative methods for their identification. Here, we propose that individuals are aggregates that preserve a measure of temporal integrity, i.e., "propagate" information from their past into their futures. We formalize this idea using information theory and graphical models. This mathematical formulation yields three principled and distinct forms of individuality-an organismal, a colonial, and a driven form-each of which varies in the degree of environmental dependence and inherited information. This approach can be thought of as a Gestalt approach to evolution where selection makes figure-ground (agent-environment) distinctions using suitable information-theoretic lenses. A benefit of the approach is that it expands the scope of allowable individuals to include adaptive aggregations in systems that are multi-scale, highly distributed, and do not necessarily have physical boundaries such as cell walls or clonal somatic tissue. Such individuals might be visible to selection but hard to detect by observers without suitable measurement principles. The information theory of individuality allows for the identification of individuals at all levels of organization from molecular to cultural and provides a basis for testing assumptions about the natural scales of a system and argues for the importance of uncertainty reduction through coarse-graining in adaptive systems.

Dynamic positional information: Patterning mechanism versus precision in gradient-driven systems

There is much talk about information in biology. In developmental biology, this takes the form of "positional information," especially in the context of morphogen-based pattern formation. Unfortunately, the concept of "information" is rarely defined in any precise manner. Here, we provide two alternative interpretations of "positional information," and examine the complementary meanings and uses of each concept. Positional information defined as Shannon information helps us understand decoding and error propagation in patterning systems. General relativistic positional information, in contrast, provides a metric to assess the output of pattern-forming mechanisms. Both interpretations provide powerful conceptual tools that do not compete, but are best used in combination to gain a proper mechanistic understanding of robust patterning.

Dissipation in Non-Steady State Regulatory Circuits

In order to respond to environmental signals, cells often use small molecular circuits to transmit information about their surroundings. Recently, motivated by specific examples in signaling and gene regulation, a body of work has focused on the properties of circuits that function out of equilibrium and dissipate energy. We briefly review the probabilistic measures of information and dissipation and use simple models to discuss and illustrate trade-offs between information and dissipation in biological circuits. We find that circuits with non-steady state initial conditions can transmit more information at small readout delays than steady state circuits. The dissipative cost of this additional information proves marginal compared to the steady state dissipation. Feedback does not significantly increase the transmitted information for out of steady state circuits but does decrease dissipative costs. Lastly, we discuss the case of bursty gene regulatory circuits that, even in the fast switching limit, function out of equilibrium.

Maximal information transmission is compatible with ultrasensitive biological pathways

Cells are often considered input-output devices that maximize the transmission of information by converting extracellular stimuli (input) via signaling pathways (communication channel) to cell behavior (output). However, in biological systems outputs might feed back into inputs due to cell motility, and the biological channel can change by mutations during evolution. Here, we show that the conventional channel capacity obtained by optimizing the input distribution for a fixed channel may not reflect the global optimum. In a new approach we analytically identify both input distributions and input-output curves that optimally transmit information, given constraints from noise and the dynamic range of the channel. We find a universal optimal input distribution only depending on the input noise, and we generalize our formalism to multiple outputs (or inputs). Applying our formalism to Escherichia coli chemotaxis, we find that its pathway is compatible with optimal information transmission despite the ultrasensitive rotary motors.

Estimating information in time-varying signals

Across diverse biological systems-ranging from neural networks to intracellular signaling and genetic regulatory networks-the information about changes in the environment is frequently encoded in the full temporal dynamics of the network nodes. A pressing data-analysis challenge has thus been to efficiently estimate the amount of information that these dynamics convey from experimental data. Here we develop and evaluate decoding-based estimation methods to lower bound the mutual information about a finite set of inputs, encoded in single-cell high-dimensional time series data. For biological reaction networks governed by the chemical Master equation, we derive model-based information approximations and analytical upper bounds, against which we benchmark our proposed model-free decoding estimators. In contrast to the frequently-used k-nearest-neighbor estimator, decoding-based estimators robustly extract a large fraction of the available information from high-dimensional trajectories with a realistic number of data samples. We apply these estimators to previously published data on Erk and Ca2+ signaling in mammalian cells and to yeast stress-response, and find that substantial amount of information about environmental state can be encoded by non-trivial response statistics even in stationary signals. We argue that these single-cell, decoding-based information estimates, rather than the commonly-used tests for significant differences between selected population response statistics, provide a proper and unbiased measure for the performance of biological signaling networks.

Universality of biochemical feedback and its application to immune cells

We map a class of well-mixed stochastic models of biochemical feedback in steady state to the mean-field Ising model near the critical point. The mapping provides an effective temperature, magnetic field, order parameter, and heat capacity that can be extracted from biological data without fitting or knowledge of the underlying molecular details. We demonstrate this procedure on fluorescence data from mouse T cells, which reveals distinctions between how the cells respond to different drugs. We also show that the heat capacity allows inference of the absolute molecule number from fluorescence intensity. We explain this result in terms of the underlying fluctuations, and we demonstrate the generality of our work.

Optimal Decoding of Cellular Identities in a Genetic Network

In developing organisms, spatially prescribed cell identities are thought to be determined by the expression levels of multiple genes. Quantitative tests of this idea, however, require a theoretical framework capable of exposing the rules and precision of cell specification over developmental time. We use the gap gene network in the early fly embryo as an example to show how expression levels of the four gap genes can be jointly decoded into an optimal specification of position with 1% accuracy. The decoder correctly predicts, with no free parameters, the dynamics of pair-rule expression patterns at different developmental time points and in various mutant backgrounds. Precise cellular identities are thus available at the earliest stages of development, contrasting the prevailing view of positional information being slowly refined across successive layers of the patterning network. Our results suggest that developmental enhancers closely approximate a mathematically optimal decoding strategy.

Negative feedback increases information transmission, enabling bacteria to discriminate sublethal antibiotic concentrations

In the cell, noise constrains information transmission through signaling pathways and regulatory networks. There is growing evidence that the channel capacity of cellular pathways is limited to a few bits, questioning whether cells quantify external stimuli or rely on threshold detection and binary on/off decisions. Here, using fluorescence microscopy and information theory, we analyzed the ability of the transcriptional regulator TetR to sense and quantify the antibiotic tetracycline. The results showed that noise filtering by negative feedback increased information transmission up to 2 bits, generating a graded response able to discriminate different antibiotic concentrations. This response matched the antibiotic subinhibitory selection window, suggesting that information transmission through TetR is optimized to quantify sublethal antibiotic levels. Noise filtering by negative feedback may thus boost the discriminative power of cellular sensors, enabling signal quantification.

Statistics of optimal information flow in ensembles of regulatory motifs

Genetic regulatory circuits universally cope with different sources of noise that limit their ability to coordinate input and output signals. In many cases, optimal regulatory performance can be thought to correspond to configurations of variables and parameters that maximize the mutual information between inputs and outputs. Since the mid-2000s, such optima have been well characterized in several biologically relevant cases. Here we use methods of statistical field theory to calculate the statistics of the maximal mutual information (the "capacity") achievable by tuning the input variable only in an ensemble of regulatory motifs, such that a single controller regulates N targets. Assuming (i) sufficiently large N, (ii) quenched random kinetic parameters, and (iii) small noise affecting the input-output channels, we can accurately reproduce numerical simulations both for the mean capacity and for the whole distribution. Our results provide insight into the inherent variability in effectiveness occurring in regulatory systems with heterogeneous kinetic parameters.

Perspectives on theory at the interface of physics and biology

Theoretical physics is the search for simple and universal mathematical descriptions of the natural world. In contrast, much of modern biology is an exploration of the complexity and diversity of life. For many, this contrast is prima facie evidence that theory, in the sense that physicists use the word, is impossible in a biological context. For others, this contrast serves to highlight a grand challenge. I am an optimist, and believe (along with many colleagues) that the time is ripe for the emergence of a more unified theoretical physics of biological systems, building on successes in thinking about particular phenomena. In this essay I try to explain the reasons for my optimism, through a combination of historical and modern examples.

Molecular noise can minimize the collective sensitivity of a clonal heterogeneous cell population

It is now widely accepted that molecular noise, rather than be always detrimental, introduces in many circumstances the required boost to reach fundamental cellular activities or strategies otherwise unattainable. In threshold-like genetic systems, molecular noise serves to generate heterogeneous responses in a clonal population, also in a tissue, due to cell-to-cell variability. Here, we derived a mathematical framework from which we could study in detail this effect. We focused on a minimal decision-making gene circuit implemented as a transcriptional positive-feedback loop. We evidenced that when the individual responses of each cell within the population are averaged, a sort of collective behavior, the resulting dose-response curve is linearized. In other words, the population is less sensitive than the individuals, which otherwise enhances the information transfer from signal to response. We found that the distance to the ideal linear response of the cell population is minimized for a particular noise level, and also characterized the interplay between intrinsic and extrinsic noise. Overall, our results highlight how cells could, by acting as a collective, entangle their genetic systems with their environments by adjusting the intracellular noise levels.

Beyond the French Flag Model: Exploiting Spatial and Gene Regulatory Interactions for Positional Information

A crucial step in the early development of multicellular organisms involves the establishment of spatial patterns of gene expression which later direct proliferating cells to take on different cell fates. These patterns enable the cells to infer their global position within a tissue or an organism by reading out local gene expression levels. The patterning system is thus said to encode positional information, a concept that was formalized recently in the framework of information theory. Here we introduce a toy model of patterning in one spatial dimension, which can be seen as an extension of Wolpert’s paradigmatic “French Flag” model, to patterning by several interacting, spatially coupled genes subject to intrinsic and extrinsic noise. Our model, a variant of an Ising spin system, allows us to systematically explore expression patterns that optimally encode positional information. We find that optimal patterning systems use positional cues, as in the French Flag model, together with gene-gene interactions to generate combinatorial codes for position which we call “Counter” patterns. Counter patterns can also be stabilized against noise and variations in system size or morphogen dosage by longer-range spatial interactions of the type invoked in the Turing model. The simple setup proposed here qualitatively captures many of the experimentally observed properties of biological patterning systems and allows them to be studied in a single, theoretically consistent framework.

Information processing by simple molecular motifs and susceptibility to noise

Biological organisms rely on their ability to sense and respond appropriately to their environment. The molecular mechanisms that facilitate these essential processes are however subject to a range of random effects and stochastic processes, which jointly affect the reliability of information transmission between receptors and, for example, the physiological downstream response. Information is mathematically defined in terms of the entropy; and the extent of information flowing across an information channel or signalling system is typically measured by the 'mutual information', or the reduction in the uncertainty about the output once the input signal is known. Here, we quantify how extrinsic and intrinsic noise affects the transmission of simple signals along simple motifs of molecular interaction networks. Even for very simple systems, the effects of the different sources of variability alone and in combination can give rise to bewildering complexity. In particular, extrinsic variability is apt to generate 'apparent' information that can, in extreme cases, mask the actual information that for a single system would flow between the different molecular components making up cellular signalling pathways. We show how this artificial inflation in apparent information arises and how the effects of different types of noise alone and in combination can be understood.

Noise Expands the Response Range of the Bacillus subtilis Competence Circuit

Gene regulatory circuits must contend with intrinsic noise that arises due to finite numbers of proteins. While some circuits act to reduce this noise, others appear to exploit it. A striking example is the competence circuit in Bacillus subtilis, which exhibits much larger noise in the duration of its competence events than a synthetically constructed analog that performs the same function. Here, using stochastic modeling and fluorescence microscopy, we show that this larger noise allows cells to exit terminal phenotypic states, which expands the range of stress levels to which cells are responsive and leads to phenotypic heterogeneity at the population level. This is an important example of how noise confers a functional benefit in a genetic decision-making circuit.

Fluctuations, or “noise”, in the response of a system is usually thought to be harmful. However, it is becoming increasingly clear that in single-celled organisms, noise can sometimes help cells survive. This is because noise can enhance the diversity of responses within a cell population. In this study, we identify a novel benefit of noise in the competence response of a population of Bacillus subtilis bacteria, where competence is the ability of bacteria to take in DNA from their environment when under stress. We use computational modeling and experiments to show that noise increases the range of stress levels for which these bacteria exhibit a highly dynamic response, meaning that they are neither unresponsive, nor permanently in the competent state. Since a dynamic response is thought to be optimal for survival, this study suggests that noise is exploited to increase the fitness of the bacterial population.

Extending the dynamic range of transcription factor action by translational regulation

A crucial step in the regulation of gene expression is binding of transcription factor (TF) proteins to regulatory sites along the DNA. But transcription factors act at nanomolar concentrations, and noise due to random arrival of these molecules at their binding sites can severely limit the precision of regulation. Recent work on the optimization of information flow through regulatory networks indicates that the lower end of the dynamic range of concentrations is simply inaccessible, overwhelmed by the impact of this noise. Motivated by the behavior of homeodomain proteins, such as the maternal morphogen Bicoid in the fruit fly embryo, we suggest a scheme in which transcription factors also act as indirect translational regulators, binding to the mRNA of other regulatory proteins. Intuitively, each mRNA molecule acts as an independent sensor of the input concentration, and averaging over these multiple sensors reduces the noise. We analyze information flow through this scheme and identify conditions under which it outperforms direct transcriptional regulation. Our results suggest that the dual role of homeodomain proteins is not just a historical accident, but a solution to a crucial physics problem in the regulation of gene expression.

Only accessible information is useful: insights from gradient-mediated patterning

Information theory is gaining popularity as a tool to characterize performance of biological systems. However, information is commonly quantified without reference to whether or how a system could extract and use it; as a result, information-theoretic quantities are easily misinterpreted. Here, we take the example of pattern-forming developmental systems which are commonly structured as cascades of sequential gene expression steps. Such a multi-tiered structure appears to constitute sub-optimal use of the positional information provided by the input morphogen because noise is added at each tier. However, one must distinguish between the total information in a morphogen and information that can be usefully extracted and interpreted by downstream elements. We demonstrate that quantifying the information that is accessible to the system naturally explains the prevalence of multi-tiered network architectures as a consequence of the noise inherent to the control of gene expression. We support our argument with empirical observations from patterning along the major body axis of the fruit fly embryo. We use this example to highlight the limitations of the standard information-theoretic characterization of biological signalling, which are frequently de-emphasized, and illustrate how they can be resolved.

Information transfer through a signaling module with feedback: A perturbative approach

Signal transduction in biological cells is effected by signaling pathways that typically include multiple feedback loops. Here we analyze information transfer through a prototypical signaling module with biochemical feedback. The module switches stochastically between an inactive and active state; the input to the module governs the activation rate while the output (i.e., the product concentration) perturbs the inactivation rate. Using a novel perturbative approach, we compute the rate with which information about the input is gained from observation of the output. We obtain an explicit analytical result valid to first order in feedback strength and to second order in the strength of input. The total information gained during an extended time interval is found to depend on the feedback strength only through the total number of activation/inactivation events.

Optimizing information flow in small genetic networks. IV. Spatial coupling

We typically think of cells as responding to external signals independently by regulating their gene expression levels, yet they often locally exchange information and coordinate. Can such spatial coupling be of benefit for conveying signals subject to gene regulatory noise? Here we extend our information-theoretic framework for gene regulation to spatially extended systems. As an example, we consider a lattice of nuclei responding to a concentration field of a transcriptional regulator (the input) by expressing a single diffusible target gene. When input concentrations are low, diffusive coupling markedly improves information transmission; optimal gene activation functions also systematically change. A qualitatively different regulatory strategy emerges where individual cells respond to the input in a nearly steplike fashion that is subsequently averaged out by strong diffusion. While motivated by early patterning events in the Drosophila embryo, our framework is generically applicable to spatially coupled stochastic gene expression models.

Optimal census by quorum sensing

Quorum sensing is the regulation of gene expression in response to changes in cell density. To measure their cell density, bacterial populations produce and detect diffusible molecules called autoinducers. Individual bacteria internally represent the external concentration of autoinducers via the level of monitor proteins. In turn, these monitor proteins typically regulate both their own production and the production of autoinducers, thereby establishing internal and external feedbacks. Here, we ask whether feedbacks can increase the information available to cells about their local density. We quantify available information as the mutual information between the abundance of a monitor protein and the local cell density for biologically relevant models of quorum sensing. Using variational methods, we demonstrate that feedbacks can increase information transmission, allowing bacteria to resolve up to two additional ranges of cell density when compared with bistable quorum-sensing systems. Our analysis is relevant to multi-agent systems that track an external driver implicitly via an endogenously generated signal.

Bacteria regulate gene expression in response to changes in cell density in a process called quorum sensing. To synchronize their gene-expression programs, these bacteria need to glean as much information as possible about their cell density. Our study is the first to physically model the flow of information in a quorum-sensing microbial community, wherein the internal regulator of the individuals response tracks the external cell density via an endogenously generated shared signal. Combining information theory and Lagrangian formalism, we find that quorum-sensing systems can improve their information capabilities by tuning circuit feedbacks. Our analysis suggests that achieving information benefit via feedback requires dedicated systems to control gene expression noise, such as sRNA-based regulation.

Evolution of bow-tie architectures in biology

Bow-tie or hourglass structure is a common architectural feature found in many biological systems. A bow-tie in a multi-layered structure occurs when intermediate layers have much fewer components than the input and output layers. Examples include metabolism where a handful of building blocks mediate between multiple input nutrients and multiple output biomass components, and signaling networks where information from numerous receptor types passes through a small set of signaling pathways to regulate multiple output genes. Little is known, however, about how bow-tie architectures evolve. Here, we address the evolution of bow-tie architectures using simulations of multi-layered systems evolving to fulfill a given input-output goal. We find that bow-ties spontaneously evolve when the information in the evolutionary goal can be compressed. Mathematically speaking, bow-ties evolve when the rank of the input-output matrix describing the evolutionary goal is deficient. The maximal compression possible (the rank of the goal) determines the size of the narrowest part of the network—that is the bow-tie. A further requirement is that a process is active to reduce the number of links in the network, such as product-rule mutations, otherwise a non-bow-tie solution is found in the evolutionary simulations. This offers a mechanism to understand a common architectural principle of biological systems, and a way to quantitate the effective rank of the goals under which they evolved.

Many biological systems show bow-tie (also called hourglass) architecture. A bow-tie means that a large number of inputs are converted to a small number of intermediates, which then fan out to generate a large number of outputs. For example, cells use a wide variety of nutrients; process them into 12 metabolic precursors, which are then used to make all of the cells biomass. Similar principles exist in biological signaling and in the information processing in the visual system. Despite the ubiquity of bow-tie structures in biology, there is no explanation of how they evolved. Here, we find that bow-ties spontaneously evolve when the information in the evolutionary goal they evolved to satisfy can be compressed. Mathematically, this means that the matrix representing the goal has deficient rank. The maximal compression possible determines the width of the bow-tie—the narrowest part in the network (equal to the rank of the goal matrix). This offers a mechanism to understand a common architectural principle of biological systems, and a way to quantitate the rank of the goals under which they evolved.

Noise and information transmission in promoters with multiple internal States

Based on the measurements of noise in gene expression performed during the past decade, it has become customary to think of gene regulation in terms of a two-state model, where the promoter of a gene can stochastically switch between an ON and an OFF state. As experiments are becoming increasingly precise and the deviations from the two-state model start to be observable, we ask about the experimental signatures of complex multistate promoters, as well as the functional consequences of this additional complexity. In detail, we i), extend the calculations for noise in gene expression to promoters described by state transition diagrams with multiple states, ii), systematically compute the experimentally accessible noise characteristics for these complex promoters, and iii), use information theory to evaluate the channel capacities of complex promoter architectures and compare them with the baseline provided by the two-state model. We find that adding internal states to the promoter generically decreases channel capacity, except in certain cases, three of which (cooperativity, dual-role regulation, promoter cycling) we analyze in detail.

Information processing in the adaptation of Saccharomyces cerevisiae to osmotic stress: an analysis of the phosphorelay system

Cellular signaling is key for organisms to survive immediate stresses from fluctuating environments as well as relaying important information about external stimuli. Effective mechanisms have evolved to ensure appropriate responses for an optimal adaptation process. For them to be functional despite the noise that occurs in biochemical transmission, the cell needs to be able to infer reliably what was sensed in the first place. For example Saccharomyces cerevisiae are able to adjust their response to osmotic shock depending on the severity of the shock and initiate responses that lead to near perfect adaptation of the cell. We investigate the Sln1-Ypd1-Ssk1-phosphorelay as a module in the high-osmolarity glycerol pathway by incorporating a stochastic model. Within this framework, we can imitate the noisy perception of the cell and interpret the phosphorelay as an information transmitting channel in the sense of C.E. Shannon's "Information Theory". We refer to the channel capacity as a measure to quantify and investigate the transmission properties of this system, enabling us to draw conclusions on viable parameter sets for modeling the system.

Multiplexing oscillatory biochemical signals

In recent years it has been increasingly recognized that biochemical signals are not necessarily constant in time and that the temporal dynamics of a signal can be the information carrier. Moreover, it is now well established that the protein signaling network of living cells has a bow-tie structure and that components are often shared between different signaling pathways. Here we show by mathematical modeling that living cells can multiplex a constant and an oscillatory signal: they can transmit these two signals simultaneously through a common signaling pathway, and yet respond to them specifically and reliably. We find that information transmission is reduced not only by noise arising from the intrinsic stochasticity of biochemical reactions, but also by crosstalk between the different channels. Yet, under biologically relevant conditions more than 2 bits of information can be transmitted per channel, even when the two signals are transmitted simultaneously. These observations suggest that oscillatory signals are ideal for multiplexing signals.

Morphogenesis at criticality

Spatial patterns in the early fruit fly embryo emerge from a network of interactions among transcription factors, the gap genes, driven by maternal inputs. Such networks can exhibit many qualitatively different behaviors, separated by critical surfaces. At criticality, we should observe strong correlations in the fluctuations of different genes around their mean expression levels, a slowing of the dynamics along some but not all directions in the space of possible expression levels, correlations of expression fluctuations over long distances in the embryo, and departures from a Gaussian distribution of these fluctuations. Analysis of recent experiments on the gap gene network shows that all these signatures are observed, and that the different signatures are related in ways predicted by theory. Although there might be other explanations for these individual phenomena, the confluence of evidence suggests that this genetic network is tuned to criticality.

Noise in biology

Noise permeates biology on all levels, from the most basic molecular, sub-cellular processes to the dynamics of tissues, organs, organisms and populations. The functional roles of noise in biological processes can vary greatly. Along with standard, entropy-increasing effects of producing random mutations, diversifying phenotypes in isogenic populations, limiting information capacity of signaling relays, it occasionally plays more surprising constructive roles by accelerating the pace of evolution, providing selective advantage in dynamic environments, enhancing intracellular transport of biomolecules and increasing information capacity of signaling pathways. This short review covers the recent progress in understanding mechanisms and effects of fluctuations in biological systems of different scales and the basic approaches to their mathematical modeling.

Transcription factor binding kinetics constrain noise suppression via negative feedback

Negative autoregulation, where a transcription factor regulates its own expression by preventing transcription, is commonly used to suppress fluctuations in gene expression. Recent single molecule in vivo imaging has shown that it takes significant time for a transcription factor molecule to bind its chromosomal binding site. Given the slow association kinetics, transcription factor mediated feedback cannot at the same time be fast and strong. Here we show that with a limited association rate follows an optimal transcription factor binding strength where noise is maximally suppressed. At the optimal binding strength the binding site is free a fixed fraction of the time independent of the transcription factor concentration. One consequence is that high-copy number transcription factors should bind weakly to their operators, which is observed for transcription factors in Escherichia coli. The results demonstrate that a binding site's strength may be uncorrelated to its functional importance.

Positional information, in bits

Cells in a developing embryo have no direct way of "measuring" their physical position. Through a variety of processes, however, the expression levels of multiple genes come to be correlated with position, and these expression levels thus form a code for "positional information." We show how to measure this information, in bits, using the gap genes in the Drosophila embryo as an example. Individual genes carry nearly two bits of information, twice as much as would be expected if the expression patterns consisted only of on/off domains separated by sharp boundaries. Taken together, four gap genes carry enough information to define a cell's location with an error bar of ~1 along the anterior/posterior axis of the embryo. This precision is nearly enough for each cell to have a unique identity, which is the maximum information the system can use, and is nearly constant along the length of the embryo. We argue that this constancy is a signature of optimality in the transmission of information from primary morphogen inputs to the output of the gap gene network.

Cross talk and interference enhance information capacity of a signaling pathway

A recurring motif in gene regulatory networks is transcription factors (TFs) that regulate each other and then bind to overlapping sites on DNA, where they interact and synergistically control transcription of a target gene. Here, we suggest that this motif maximizes information flow in a noisy network. Gene expression is an inherently noisy process due to thermal fluctuations and the small number of molecules involved. A consequence of multiple TFs interacting at overlapping binding sites is that their binding noise becomes correlated. Using concepts from information theory, we show that in general a signaling pathway transmits more information if 1), noise of one input is correlated with that of the other; and 2), input signals are not chosen independently. In the case of TFs, the latter criterion hints at upstream cross-regulation. We demonstrate these ideas for competing TFs and feed-forward gene-regulatory modules, and discuss generalizations to other signaling pathways. Our results challenge the conventional approach of treating biological noise as uncorrelated fluctuations, and present a systematic method for understanding TF cross-regulation networks either from direct measurements of binding noise or from bioinformatic analysis of overlapping binding sites.

Time-dependent information transmission in a model regulatory circuit

Many biological regulatory systems respond with a physiological delay when processing signals. A simple model of regulation which respects these features shows how the ability of a delayed output to transmit information is limited: at short times by the time scale of the dynamic input, at long times by that of the dynamic output. We find that topologies of maximally informative networks correspond to commonly occurring biological circuits linked to stress response and that circuits functioning out of steady state may exploit absorbing states to transmit information optimally.

Self-regulatory gene: an exact solution for the gene gate model

The stochastic dynamics of gene expression is often described by highly abstract models involving only the key molecular actors DNA, RNA, and protein, neglecting all further details of the transcription and translation processes. One example of such models is the "gene gate model," which contains a minimal set of actors and kinetic parameters, which allows us to describe the regulation of a gene by both repression and activation. Based on this approach, we formulate a master equation for the case of a single gene regulated by its own product-a transcription factor-and solve it exactly. The obtained gene product distributions display features of mono- and bimodality, depending on the choice of parameters. We discuss our model in the perspective of other models in the literature.

Statistical mechanics of Monod-Wyman-Changeux (MWC) models

The 50th anniversary of the classic Monod-Wyman-Changeux (MWC) model provides an opportunity to survey the broader conceptual and quantitative implications of this quintessential biophysical model. With the use of statistical mechanics, the mathematical implementation of the MWC concept links problems that seem otherwise to have no ostensible biological connection including ligand-receptor binding, ligand-gated ion channels, chemotaxis, chromatin structure and gene regulation. Hence, a thorough mathematical analysis of the MWC model can illuminate the performance limits of a number of unrelated biological systems in one stroke. The goal of our review is twofold. First, we describe in detail the general physical principles that are used to derive the activity of MWC molecules as a function of their regulatory ligands. Second, we illustrate the power of ideas from information theory and dynamical systems for quantifying how well the output of MWC molecules tracks their sensory input, giving a sense of the "design" constraints faced by these receptors.