Introduction
Machine learning methods, such as ensemble decision trees, are widely used to predict outcomes based on data. However, these methods often focus on providing point predictions, which limits their ability to quantify prediction uncertainty. In many applications, such as healthcare and finance, the goal is not only to predict accurately but also to assess the reliability of those predictions. Prediction intervals, which provide lower and upper bounds such that the true response lies within them with high probability, are a reliable tool for quantifying prediction accuracy. An ideal prediction interval should meet several criteria: it should offer valid coverage (defined below) without relying on strong distributional assumptions, be informative by being as narrow as possible for each observation, and be adaptive—provide wider intervals for observations that are “difficult” to predict and narrower intervals for “easy” ones. Read more…
Motivation
You have just trained a gradient boosting machine (GBM) and a random forest (RF) classifier on your data using Stata’s new h2oml command suite. Your GBM model achieves 87% accuracy on the testing data, and your RF model, 85%. It looks as if GBM is the preferred classifier, right? Not so fast.
Why accuracy alone isn’t enough
Accuracy, area under the curve, and root mean squared error are popular metrics, but they provide only point estimates. These numbers reflect how well a model performed on one specific testing sample, but they don’t account for the variability that can arise from sample to sample. In other words, they don’t answer this key question: Will the difference in performance between these methods hold at the population level, or could it have occurred by chance only in this particular testing dataset? Read more…
I am excited to let you be the second to know that Stata 19 is now available. Statalist is always the first to know!
Highlights include
And more. Visit stata.com/new-in-stata for all the details. You can also visit stata.com/help.cgi?whatsnew18to19 for the nitty gritty on every single change from Stata 18 to Stata 19.
Those of you with StataNow already received some of these features along the way in updates to StataNow. And, those of you with StataNow are eligible for an automatic upgrade to StataNow 19. Watch your inbox for an email from us with instructions on how to request your upgrade.
Categories: New Products Tags: Bayesian, biostatistics, CATE, cox, CRE, data science, econometrics, H2O, HDFE, machine learning, meta-analysis, mundlak, new release