{"id":665,"date":"2018-07-09T08:30:00","date_gmt":"2018-07-09T12:30:00","guid":{"rendered":"http:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/?p=665"},"modified":"2018-07-04T14:12:47","modified_gmt":"2018-07-04T18:12:47","slug":"on-the-importance-of-making-observations-and-inferences-at-the-right-hierarchical-level","status":"publish","type":"post","link":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/blog\/2018\/07\/09\/on-the-importance-of-making-observations-and-inferences-at-the-right-hierarchical-level\/","title":{"rendered":"On the importance of making observations (and inferences) at the right hierarchical level"},"content":{"rendered":"

I mentioned a couple of weeks ago that trait-environment associations observed at a global scale across many lineages don\u2019t necessarily correspond to those observed within lineages at a smaller scale (link<\/a>). I didn\u2019t mention it then, but this is just another example of the general phenomenon known as the ecological fallacy<\/a>, in which associations evident at the level of a group are attributed to individuals within the group. The ecological fallacy is related to Simpson\u2019s paradox<\/a> in which within-group associations differ from those between groups.<\/p>\n

A recent paper in Proceedings of the National Academy of Sciences<\/em> gives practical examples of why it\u2019s important to make observations at the level you\u2019re interested in and why you should be very careful about extrapolating associations observed at one level to associations at another. They report on six repeated-measure studies in which the responses of multiple participants (87-94) 1<\/a><\/sup> were assessed across time. Thus, the authors could assess both the amount of variation within individuals over time and the amount of variation among individuals at one time. They found that the amount of within individual variation was between two and four times higher than the amount of among individual variation. Why do we care? Well, if you wanted to know, for example whether administering imipramine reduced symptoms of clinical depression (sample 4 in the paper) and used the among individual variance in depression measured once to assess whether or not an observed difference was statistically meaningful, you\u2019d be using a standard error that\u2019s a factor of two or more too small. As a result, you\u2019d be more confident that a difference exists than you should be based on the amount of variation within individuals.<\/p>\n

Why does this matter to an ecologist or an evolutionary biologist? Have you ever heard of \u201cspace-time substitution\u201d? Do a Google search and near the top you\u2019ll find a link to this chapter<\/a> from Long Term Studies in Ecology<\/em> by Steward Pickett. The idea is that because longitudinal studies take a very long time, we can use variation in space as a substitute for variation in time. The assumption is rarely tested (see this paper<\/a> for an exception), but it is widely used. The problem is that in any spatially structured system with a finite number of populations or sites, the variance among sites at any one time (the spatial variation we\u2019d measure) is substantially less than the variance in any one site across time (the temporal variance). If we\u2019re interested in the spatial variance, that\u2019s fine. If we\u2019re interested in how variable the system is over time, though, it\u2019s a problem. It\u2019s also a problem if we believe that associations we see across populations at one point in time are characteristics of any one population across time.<\/p>\n

In the context of the leaf economic spectrum, most of the global associations that have been documented involve associations between species mean trait values. For the same reason that space-time substitution may not work and for the same reason that this recent paper in PNAS<\/em> illustrates that among group associations in humans don\u2019t reliably predict individual associations, if we want to understand the mechanistic basis of trait-environment or trait-trait associations, by which I mean the evolutionary mechanisms acting at the individual level that produce those associations within individuals, we need to measure the traits on individuals and measure the environments where those individuals occur.<\/p>\n

Here\u2019a the title and abstract of the paper that inspired this post. I\u2019ve also included a link.<\/p>\n

Lack of group-to-individual generalizability is a threat to human subjects research<\/strong><\/p>\n

Aaron J. Fisher, John D. Medaglia, and Bertus F. Jeronimus<\/p>\n

Only for ergodic processes will inferences based on group-level data generalize to individual experience or behavior. Because human social and psychological processes typically have an individually variable and time-varying nature, they are unlikely to be ergodic. In this paper, six studies with a repeated-measure design were used for symmetric comparisons of interindividual and intraindividual variation. Our results delineate the potential scope and impact of nonergodic data in human subjects research. Analyses across six samples (with 87\u201394 participants and an equal number of assessments per participant) showed some degree of agreement in central tendency estimates (mean) between groups and individuals across constructs and data collection paradigms. However, the variance around the expected value was two to four times larger within individuals than within groups. This suggests that literatures in social and medical sciences may overestimate the accuracy of aggregated statistical estimates. This observation could have serious consequences for how we understand the consistency between group and individual correlations, and the generalizability of conclusions between domains. Researchers should explicitly test for equivalence of processes at the individual and group level across the social and medical sciences.<\/p>\n

doi: 10.1073\/pnas.1711978115<\/a><\/p><\/blockquote>\n

    \n
  1. The studies are on human subjects. ↩<\/a><\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

    I mentioned a couple of weeks ago that trait-environment associations observed at a global scale across many lineages don\u2019t necessarily correspond to those observed within lineages at a smaller scale… Read more ><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[23,10],"tags":[],"class_list":["post-665","post","type-post","status-publish","format-standard","hentry","category-ecology","category-statistics"],"_links":{"self":[{"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/posts\/665","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/comments?post=665"}],"version-history":[{"count":3,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/posts\/665\/revisions"}],"predecessor-version":[{"id":667,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/posts\/665\/revisions\/667"}],"wp:attachment":[{"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/media?parent=665"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/categories?post=665"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/tags?post=665"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}