only accessible information is useful: insights from...

© 2015 The Authors. Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited

Review History Decision letter (RSOS-150486) 26-Oct-2015 Dear Dr Tikhonov On behalf of the Editors, I am pleased to inform you that your Manuscript RSOS-150486 entitled "Only accessible information is useful: insights from gradient-mediated patterning" has been accepted for publication in Royal Society Open Science. The editors feel that you have satisfactorily addressed the reviewers comments from Journal of the Royal Society Interface so please carefully proof read your article and ensure that all the below editorial comments are addressed as appropriate. • Ethics statement If your study uses humans or animals please include details of the ethical approval received, including the name of the committee that granted approval. For human studies please also detail whether informed consent was obtained. For field studies on animals please include details of all permissions, licences and/or approvals granted to carry out the fieldwork. • Data accessibility It is a condition of publication that all supporting data are made available either as supplementary information or preferably in a suitable permanent repository. The data

Only accessible information is useful: insights from

gradient-mediated patterning

Mikhail Tikhonov, Shawn C. Little and Thomas Gregor

Article citation details R. Soc. open sci. 2: 150486. http://dx.doi.org/10.1098/rsos.150486

Review timeline

Original submission: 14 September 2015 Revised submission: 28 October 2015 Final acceptance: 29 October 2015

Note: Reports are unedited and appear as submitted by the referee. The review history appears in chronological order.

Note: This manuscript was transferred from another Royal Society journal with/without peer review

on May 23, 2018http://rsos.royalsocietypublishing.org/Downloaded from

http://rsos.royalsocietypublishing.org/

2

accessibility section should state where the article's supporting data can be accessed. This section should also include details, where possible of where to access other relevant research materials such as statistical tools, protocols, software etc can be accessed. If the data has been deposited in an external repository this section should list the database, accession number and link to the DOI for all data from the article that has been made publicly available. Data sets that have been deposited in an external repository and have a DOI should also be appropriately cited in the manuscript and included in the reference list. If you wish to submit your supporting data or code to Dryad (http://datadryad.org/), or modify your current submission to dryad, please use the following link: http://datadryad.org/submit?journalID=RSOS&manu=RSOS-150486 • Competing interests Please declare any financial or non-financial competing interests, or state that you have no competing interests. • Authors’ contributions All submissions, other than those with a single author, must include an Authors’ Contributions section which individually lists the specific contribution of each author. The list of Authors should meet all of the following criteria; 1) substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; and 3) final approval of the version to be published. All contributors who do not meet all of these criteria should be included in the acknowledgements. We suggest the following format: AB carried out the molecular lab work, participated in data analysis, carried out sequence alignments, participated in the design of the study and drafted the manuscript; CD carried out the statistical analyses; EF collected field data; GH conceived of the study, designed the study, coordinated the study and helped draft the manuscript. All authors gave final approval for publication. • Acknowledgements Please acknowledge anyone who contributed to the study but did not meet the authorship criteria. • Funding statement Please list the source of funding for each author. Because the schedule for publication is very tight, it is a condition of publication that you submit the revised version of your manuscript within 7 days (i.e. by the 04-Nov-2015). If you do not think you will be able to meet this date please let me know immediately. To revise your manuscript, log into https://mc.manuscriptcentral.com/rsos and enter your Author Centre, where you will find your manuscript title listed under "Manuscripts with Decisions". Under "Actions," click on "Create a Revision." You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your manuscript and upload a new version through your Author Centre. When submitting your revised manuscript, you will be able to respond to the comments made by the referees and upload a file "Response to Referees" in "Section 6 - File Upload". You can use this to document any changes you make to the original manuscript. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the referees.



3

When uploading your revised files please make sure that you have:

1) A text file of the manuscript (tex, txt, rtf, docx or doc), references, tables (including captions)and figure captions. Do not upload a PDF as your "Main Document". 2) A separate electronic file of each figure (EPS or print-quality PDF preferred (either formatshould be produced directly from original creation package), or original software format) 3) Included a 100 word media summary of your paper when requested at submission. Pleaseensure you have entered correct contact details (email, institution and telephone) in your user account 4) Included the raw data to support the claims made in your paper. You can either include yourdata as electronic supplementary material or upload to a repository and include the relevant doi within your manuscript 5) Included your supplementary files in a format you are happy with (no line numbers,vancouver referencing, track changes removed etc) as these files will NOT be edited in production

Once again, thank you for submitting your manuscript to Royal Society Open Science and I look forward to receiving your revision. If you have any questions at all, please do not hesitate to get in touch.

Best wishes

Emilie Aime Senior Publishing Editor, Royal Society Open Science

Associate Editor Comments to Author (Professor Jürg Bähler): The authors have exhaustively addressed the issues raised by the 3 referees.

********************************************** Journal Name: Royal Society Open Science Journal Code: RSOS Online ISSN: 2054-5703 Journal Admin Email: [email protected] Journal Editor: Emilie Aime Journal Editor Email: [email protected] MS Reference Number: RSOS-150486 Article Status: SUBMITTED MS Dryad ID: RSOS-150486 MS Title: Only accessible information is useful: insights from gradient-mediated patterning MS Authors: Tikhonov, Mikhail; Little, Shawn; Gregor, Thomas Contact Author: Mikhail Tikhonov Contact Author Email: [email protected] Contact Author Address 1: 1 Oxford St Contact Author Address 2: Contact Author Address 3: Contact Author City: Cambridge Contact Author State: Massachusetts Contact Author Country: United States Contact Author ZIP/Postal Code: 02138 Keywords: Information theory, Genetic regulation, Drosophila, Signaling Abstract: Information theory is gaining popularity as a tool to characterise performance of biological systems. However, information is commonly quantified without reference to whether



4

or how a system could extract and use it; as a result, information-theoretic quantities are easily misinterpreted. Here we take the example of pattern-forming developmental systems which are commonly structured as cascades of sequential gene expression steps. Such a multi-tiered structure appears to constitute sub-optimal use of the positional information provided by the input morphogen because noise is added at each tier. However, one must distinguish between the total information in a morphogen and information that can be usefully extracted and interpreted by downstream elements. We demonstrate that quantifying the information that is accessible to the system naturally explains the prevalence of multi-tiered network architectures as a consequence of the noise inherent to the control of gene expression. We support our argument with empirical observations from patterning along the major body axis of the fruit fly embryo. We use this example to highlight the limitations of the standard information-theoretic characterisation of biological signaling, which are frequently de-emphasized, and illustrate how they can be resolved. EndDryadContent

Decision letter (RSOS-150486.R1) 29-Oct-2015 Dear Dr Tikhonov, I am pleased to inform you that your manuscript entitled "Only accessible information is useful: insights from gradient-mediated patterning" is now accepted for publication in Royal Society Open Science. You can expect to receive a proof of your article within approximately 10 working days. Please contact the production office ([email protected]) to let us know if you are likely to be away from e-mail contact during that period. Due to rapid publication and an extremely tight schedule, if comments are not received, your paper may experience a delay in publication. Royal Society Open Science operates under a continuous publication model (http://bit.ly/cpFAQ). Your article will be published straight into the next open issue and this will be the final version of the paper. As such, it can be cited immediately by other researchers. As the issue version of your paper will be the only version to be published I would advise you to check your proofs thoroughly as changes cannot be made once the paper is published. In order to raise the profile of your paper once it is published, we can send through a PDF of your paper to selected colleagues. If you wish to take advantage of this, please reply to this email with the name and email addresses of up to 10 people who you feel would wish to read your article. On behalf of the Editors of Royal Society Open Science, we look forward to your continued contributions to the Journal. Best wishes, Matthew Allinson [email protected] http://rsos.royalsocietypublishing.org/ *************************** Journal Name: Royal Society Open Science Journal Code: RSOS



5

Online ISSN: 2054-5703 Journal Admin Email: [email protected] Journal Editor: Emilie Aime Journal Editor Email: [email protected] MS Reference Number: RSOS-150486.R1 Article Status: ACCEPTED MS Dryad ID: RSOS-150486.R1 MS Title: Only accessible information is useful: insights from gradient-mediated patterning MS Authors: Tikhonov, Mikhail; Little, Shawn; Gregor, Thomas Contact Author: Mikhail Tikhonov Contact Author Email: [email protected] Contact Author Address 1: 1 Oxford St Contact Author Address 2: Contact Author Address 3: Contact Author City: Cambridge Contact Author State: Massachusetts Contact Author Country: United States Contact Author ZIP/Postal Code: 02138 Keywords: Information theory, Genetic regulation, Drosophila, Signaling Abstract: Information theory is gaining popularity as a tool to characterise performance of biological systems. However, information is commonly quantified without reference to whether or how a system could extract and use it; as a result, information-theoretic quantities are easily misinterpreted. Here we take the example of pattern-forming developmental systems which are commonly structured as cascades of sequential gene expression steps. Such a multi-tiered structure appears to constitute sub-optimal use of the positional information provided by the input morphogen because noise is added at each tier. However, one must distinguish between the total information in a morphogen and information that can be usefully extracted and interpreted by downstream elements. We demonstrate that quantifying the information that is accessible to the system naturally explains the prevalence of multi-tiered network architectures as a consequence of the noise inherent to the control of gene expression. We support our argument with empirical observations from patterning along the major body axis of the fruit fly embryo. We use this example to highlight the limitations of the standard information-theoretic characterisation of biological signaling, which are frequently de-emphasized, and illustrate how they can be resolved. EndDryadContent

Author's Response to Decision Letter for (RSOS-150486)

See Appendix A.



RESPONSE TO REVIEWERS

REVIEWER #1

We thank the reviewer for identifying several issues with the presentation of the model and the manuscript more generally. These issues are fully addressed in the revised version and the primary intention of the manuscript is clarified.

The following two points are related and we address them together below: 1. I found the title a bit too colloquial and not descriptive enough of the actual work in the paper.2. The authors try too hard in the introduction to imply that their findings explain a very broad categoryof biochemical networks. This is quite off-putting. The text should instead focus on gradient-sensing networks where the author's experiments and theory are directly applicable. To me, this would increase rather than decrease the impact of the work.

The reviewer is correct: our results section focuses on a very specific class of biochemical networks. The manuscript is very explicit about this, formulating the specific problem very narrowly as explaining the “pattern of broad subdivision followed by short-range refinement” observed in multiple developmental programs (lines 63, 71). However, for us, this specific result is less important than the very general point it illustrates, and the manuscript is structured accordingly. The main message of the manuscript is that assessing the usefulness of a signal must always take into account the so-called input noise of the downstream circuit interpreting this signal. This is a completely general point, which is not controversial among specialists (lines 95, 365-369, references [3, 42, 52]), and whose relevance extends far beyond graded-signal-responsive networks. Our intention was to provide a particularly instructive illustration. We neither state nor imply that our results directly “explain a broad category of biochemical networks”. To the contrary, we emphasize repeatedly that our results are derived in a highly simplified context (lines 93, 117, 253, 370) where the difference between raw and accessible information becomes especially clear; however, our goal was to communicate the generality of the caveats our example illustrates (lines 365-369).

For these reasons, we believe that the title of the work reflects its intention exactly. To clarify this intention and address the reviewer’s concerns, we toned down the abstract (it no longer contrasts our approach with “conventional theory”; and “we exhibit limitations” was replaced by “we highlight the limitations that are frequently de-emphasized”), and edited the main text to better separate the specific example we use and the general point it illustrates. In particular, we entirely re-wrote the final paragraph of the discussion summarizing our work (lines 365-369).

3. In general, I found the description of the model (in words) to be overly complicated and hard tofollow. The authors should consider simplifying it or, better yet, using analogies to other types of circuits (electronics comes to mind) and standardized terms (gain, noise, readout noise, etc.) to make the findings easier to understand. We agree with the reviewer. In the revised manuscript, we resolve the ambiguity of wordy descriptions by focusing on clarity of mathematical expressions. We introduced a consistent notation for all random variables and provide explicit expressions for all variables we construct. We also modified Fig. 2, expanding it to improve clarity. With regard to using other standard terms for noise, see next point.

Appendix A on May 23, 2018http://rsos.royalsocietypublishing.org/Downloaded from


4. Related to #3, I believe that there is a mistake in the authors description of the different types of noise being considered. The central result of the paper (Eq 6) relies on the two-tier model having both gain-dependent and gain-independent sources of noise. These are introduced in line 146 and Eq 4, respectively. The former (line 146) is listed as gain dependent and the authors explain it as reflecting "the intrinsic stochasticity of transcription and, in principle, many other noise sources." This sounds to me like output noise which should be gain *independent*. The later, Eq 4, a noisy estimate of the input to the circuit, is input noise and should be gain *dependent*, again the opposite of what is implied in the derivation of Eq 6. In the end, both noise sources are present in Eq 6 which is then correct. But, the derivation of where these terms come from in a realistic biological network is explained in too sloppy a manner. We thank the reviewer for bringing our attention to this point, which was very important to clarify in the text. Our model considers only one type of noise; it is denoted 𝜂 and always corresponds to the input noise of a readout circuit. Its biochemical origin is the intrinsic noise of transcriptional regulation, e.g. random arrival of molecules at their targets or the promoter switching noise. Its description in the original manuscript was imprecise and caused confusion. In the revised manuscript, this mistake was corrected (lines 115-116). In Eq. 6, this same noise enters twice, corresponding to the two readout operations required in the two-tier strategy. The second of these readouts occurs after amplification. Therefore, in this expression, the

input noise of the hypothetical circuit that will be interpreting signal �̂�(𝜆) formally appears in the same

way as would the output (“gain-independent”) noise of the amplification step �̂� → �̂�(𝜆). This shows that in our case, when the output of one circuit is the input to another, using terms such as “gain-dependent” or “gain-independent” noise would create more confusion than it would clarify. Since only one type of noise is considered, it is sufficient to be clear about its nature. In the revised manuscript, we:

1) Stress that 𝜂 is the only noise source considered in our model (line 117). 2) Describe its biochemical origin (lines 115-116), and use the standard term “input noise” of a

circuit (lines 96, 117, 184, 313, 320, 368). 3) Revised our notations for consistency and spelled out the expressions for all stochastic variables

we use (e.g., Eqs. 1-2). This resolves the confusion identified by the reviewer.

REVIEWER #2 We thank the reviewer for pointing out the instances where our notation was sloppy and the shortcuts

unclear. These technical concerns are fully addressed in the revised manuscript, notations were made

consistent with standard usage, and this did in fact substantially improve the clarity of the argument.

In the revised manuscript, the confusing notation c(x) was eliminated entirely (except in the

introductory cartoon where it is appropriate, as it labels the concentration profile rather than a random

variable). Following the suggestion of the reviewer, we introduced a consistent “hat” notation for all



random variables and follow it throughout the text, replacing 𝐼[𝑐(𝑥), 𝑥] by 𝐼[�̂�, 𝑥], etc. The calculation in

the SI was corrected (see next item). All usage of subscript “i” was removed.

Due to a typo in this section of SI, averaging over 𝑥 was missing from the definition of mutual

information (second equation, in the second term). We corrected this mistake. The rest of the math is

then correct.

The reviewer is correct that 𝑃(𝑥) and 𝑃(𝑐) cannot both be exactly uniform; we clarified that our

expression for information content of a linear gradient is derived under the assumption of small noise.

This is also the case for the rest of the manuscript, and is now stated explicitly (lines 146, 222).

Unlike the discrete entropy, the differential entropy, which extends the notion of entropy to continuous

probability distributions, can be negative and indeed tends to −∞ for perfectly deterministic functions.

In particular, a perfectly deterministic continuous function of a continuous input would carry an infinite

amount of information. In the limit 𝜎0 → 0, the formula calculated in this section of the SI yields this

correct result.

At the same time, the reviewer correctly points out that for a finite number of cells, the amount of

information is always finite. Using the expressions from the continuous case in our argument constitutes

an implicit assumption, namely that the number of cells is sufficiently large that their discreteness can

be neglected, and the explicit expression for the validity of this assumption invokes the magnitude of

noise. This is an important point; it is now explicitly mentioned in the main text (line 146) and detailed in

the SI. We comment on the validity of this assumption for the Drosophila system we consider (same

section of the SI).

We apologize for the confusing notation. The reviewer is exactly correct: 𝐼𝑎𝑐𝑐0 = 𝐼[�̂�(0) + �̂�; 𝑥]. We

define the accessible information in �̂� as the raw information of �̂�𝑒𝑠𝑡 ≡ �̂� + �̂�, and the accessible

information in �̂�(0) as the raw information of �̂�(0) 𝑒𝑠𝑡 ≡ �̂�(0) + �̂�, etc. We agree with the reviewer that

our use of superscript “estimate” was not clear. Writing out the full expressions for all random variables



with which we operate, as the reviewer did in their comments, is the clearest approach. The revised

manuscript follows this strategy throughout the text (new Eqs. 1, 2, 5 and the unnumbered equation

following line 212 ). The step from Eq. 4 to Eqs. 5 and 6 is now straightforward.

The reviewer is again correct. The manuscript was revised to clarify this issue (new Eqs. 2 and 6).

This is a valid point, also made by Reviewer #1. The revised manuscript stresses that our simplified

model considers only one type of noise; it is denoted 𝜂 and always corresponds to the input noise of a

readout circuit. Its biochemical origin is the intrinsic noise of transcriptional regulation, e.g. random

arrival of molecules at their targets or the promoter switching noise. Its description in the original

manuscript was imprecise and caused confusion. In the revised manuscript, this mistake was corrected

(lines 115-116). The extra figure suggested by the reviewer describing “each noise source” appears

unnecessary, since the model considers only one source of noise, as the revised manuscript clarifies (line

117).

REVIEWER #3

(…)

The reviewer is correct: the difference between raw and accessible information as we defined it is due

to input noise, whose importance has been realized in the past (lines 95-96, 365-376). The manuscript

directly states that the main message of the work “is implicit in the theoretical work investigating input

noise, but has not been emphasized” (lines 312-313). It is our view that this point, although understood

by information theorists, is not sufficiently appreciated by all practitioners; we discuss this in the

introduction and provide references (lines 13-16, 87-93). The reviewer is correct that in the example we

chose, the difference between raw and accessible information is especially clear; this is precisely why we

chose it. Our goal was to emphasize a theoretical point and demonstrate its implications in a biologically



relevant example that speaks to everyone who uses information theory, or encounters it in the

literature. The revised manuscript stresses that the caveats of over- or mis-interpreting standard

information-theoretic concepts, and their relevance reaching far beyond Drosophila is not controversial

among specialists and is supported by other studies (lines 312-313, refs. [3, 42, 52]). Our intention was

to provide a particularly instructive illustration; we believe this is timely. We edited the introduction

(lines 95-96) and rewrote the concluding paragraph of the paper (lines 365-376) to clarify this intention.

To our knowledge, our noise estimation technique is indeed new, and it is our hope that this aspect of

the work will also be useful to some readers. However, the manuscript is primarily not about Drosophila,

but about information theory, and the emphasis is placed accordingly.

The manuscript states that we focus on this region and these genes because “other inputs to eve are

negligible in this region at this time” (results, line 258); references [46,47] are provided. The reviewer is

correct that focusing on one small region would be a poor approach to understanding the details of

regulation of eve or its specific role in patterning of Drosophila. However, it is a perfectly valid setting to

illustrate the conceptual point made in the manuscript: a downstream gene may be noisier and carry

less raw information, yet its presence can serve to make this information more accessible (lines 295-

297). Elaborating on this point, line 300 in the Results states: “our framework demonstrates that even if

Eve were regulated by Hb and Kr only, and so were fully redundant in the standard information-

theoretic sense, the additional tier would still confer an advantage, because transcription is intrinsically

noisy.” Note that we do not claim that Eve is indeed regulated by Hb and Kr only, in fact it need not be

the case; once again, our focus is on the theoretical point, not on Drosophila specifically.

There are two possible answers. The shortest is that in our model, each time window of length 𝜏 gives

cells access to an independent sample of the random variable �̂� (lines 111, 124-125, 130). The

intermediate signal established by time T is based on past drawings of �̂�. Therefore, the two signals

{�̂�, �̂�(𝜆)} to which a cell has access at a given moment in time, and whose joint information we consider,

are wholly independent (within our simplified model). Accounting for correlations is possible, but not

necessary to prove the qualitative statement that serves as the title of that section: “Multiple tiers can

improve gradient interpretation even when raw information decreases”.

A more subtle observation is that the argument presented in this section of the manuscript can be

reformulated so as to become immune to the problem of correlations, even if they were present. For

this, we can introduce a discrete “segment” variable �̂� which is a binning of the original morphogen into

𝜆 bins, and use it instead of �̂� in joint information calculations. We added a new section to the SI

detailing this argument (“Joint accessible information and the zigzag profile z”). Since �̂� is a discrete



variable, in the small-noise approximations it has no correlations that could reduce the joint information

content.

We also added a mention of the issue of correlations in another place where it is relevant (lines 269-271,

280-281) and added a reference [49] that investigates it in Drosophila for the genes we consider.

The reviewer is correct that the DV dependence of gene expression has a confounding effect on noise

estimation. Our method, whereby we compare expression in a given nucleus only with its immediate

neighbors rather than with all nuclei along the imaginary DV axis, was developed precisely to mitigate

this effect. The residual systematic errors of this procedure can be quantified by observing the residual

systematic trends in the scatter plot of inferred expression fluctuations vs AP location, as we do in the SI

(Figs. S2, S3; see SI for details). The fact that the magnitude of the residual systematic bias (Fig. S3B,

solid black line) is significantly smaller than the magnitude of measured noise (root-mean-square scatter

of red datapoints) confirms that the procedure we developed successfully eliminates most of the

systematic errors due to DV dependence of expression profiles, so that the residual deviations are

dominated by a DV-independent component of the noise. We expanded this section of the SI to clarify

this point. This verification is the best test available with static data.

Cross-validating this experimental method using live imaging data would be a good test appropriate for

a Drosophila-centered study, but is beyond the scope of this work where our main point is theoretical.

This is a valid point. Our term “gradient response problem” was meant as a shorter equivalent of

“response to a graded input signal”, but we now see that this shorthand terminology is somewhat

misleading. We corrected our phrasing to avoid ambiguity.



only accessible information is useful: insights from...

Documents