{"id":820,"date":"2019-09-16T08:00:00","date_gmt":"2019-09-16T12:00:00","guid":{"rendered":"http:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/?p=820"},"modified":"2019-09-14T11:48:45","modified_gmt":"2019-09-14T15:48:45","slug":"a-bayesian-approach-to-variable-selection-using-horseshoe-priors","status":"publish","type":"post","link":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/blog\/2019\/09\/16\/a-bayesian-approach-to-variable-selection-using-horseshoe-priors\/","title":{"rendered":"A Bayesian approach to variable selection using horseshoe priors"},"content":{"rendered":"\r\n

Variable selection in multiple regression<\/a><\/p>\r\n\r\n\r\n\r\n

The Lasso<\/a> has been very widely used, particularly in high-dimensional problems where the number of observations is less than the number of covariates.1<\/a><\/sup> In fact, when I checked Google Scholar on Saturday, it had been cited nearly 30,000 times.2<\/a><\/sup> Bayesians didn\u2019t want to be left out, so Trevor Park and George Casella developed the Bayesian Lasso.3<\/a><\/sup> The Bayesian Lasso overcomes what to my mind is one of the great disadvantages of the original Lasso, the difficulty of providing an assessment of how reliable the regression coefficients are. Like any other Bayesian method that uses MCMC methods, it\u2019s just as easy to get credible intervals on parameters as it is to get posterior means. The Bayesian Lasso also estimates \\(\\lambda\\) as part of the procedure rather than relying on cross-validation. The R<\/code> package monovm<\/code><\/a> provides an implementation of the Bayesian Lasso in addition to other shrinkage regression methods.<\/p>\r\n\r\n\r\n\r\n

I haven\u2019t explored monovm<\/code>, but if you\u2019re interested in the Bayesian Lasso, you might want to check it out. Instead of exploring the Bayesian Lasso, the R notebook I\u2019ve put together here explores the use of \u201chorseshoe priors\u201d in rstanarm<\/code><\/a>. The basic idea is the same. We\u2019d like to \u201cshrink\u201d some parameter estimates towards zero, and we\u2019d like to have the data tell us which estimates to shrink. The nice thing about \u201chorseshoe priors\u201d in rstanarm<\/code> is that if you know how to set up a regression in stan_glm()<\/code> or stan_glmer()<\/code> you can use a horseshoe prior very easily in your analysis simply by changing the prior<\/code> parameter in your call to one of those functions.<\/p>\r\n\r\n\r\n\r\n

    \r\n
  1. This is often referred to as an \\(n \\ll p\\) problem. I’m not going to address that problem here, but if you deal with genomic data, you’ll want to familiarize yourself with the problem and the approaches typically used for addressing it. ↩<\/a><\/li>\r\n
  2. 29,558 times to be exact. ↩<\/a><\/li>\r\n
  3. Park, T. and G. Casella. 2008. The Bayesian Lasso. Jornal of the American Statistical Association<\/em>. 103:681-686. doi: 10.1198\/016214508000000337<\/a>↩<\/a><\/li>\r\n<\/ol>\r\n","protected":false},"excerpt":{"rendered":"

    Variable selection in multiple regression The Lasso has been very widely used, particularly in high-dimensional problems where the number of observations is less than the number of covariates.1 In fact,… Read more ><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10],"tags":[],"class_list":["post-820","post","type-post","status-publish","format-standard","hentry","category-statistics"],"_links":{"self":[{"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/posts\/820","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/comments?post=820"}],"version-history":[{"count":11,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/posts\/820\/revisions"}],"predecessor-version":[{"id":831,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/posts\/820\/revisions\/831"}],"wp:attachment":[{"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/media?parent=820"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/categories?post=820"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/darwin.eeb.uconn.edu\/uncommon-ground\/wp-json\/wp\/v2\/tags?post=820"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}