Skip to main content

One post tagged with "ASCM"

View All Tags

· 3 min read
Arturo Esquerra

GeoLift v2.3 Release​

TL;DR​

  • The new version of the GeoLift code makes the pre-test analysis (Power Analysis and Market Selection) easier than ever before! The entire process is now handled by a single function: GeoLiftMarketSelection().
  • The new package contains dozens of new features, functionalities, and bug fixes.
  • A new and improved walkthrough guide is available to help internal and external users run their GeoLift tests from start to finish!

GeoLift v2.3: Power Calculations Made Easier​

Following the feedback we received, we re-built the Power Calculation and Market Selection functionalities from the ground up! The new code makes this process easier, more streamlined, and more powerful than ever before.

What's New?​

  • We've streamlined the entire Power Analysis and Market Selection process into a single function: GeoLiftMarketSelection().
  • Previously, the process to select the test markets was very convoluted and involved going through several functions: NumberLocations(), GeoLiftPower.search(), GeoLiftPowerFinder(), and GeoLiftPower(). The new function, GeoLiftMarketSelection(), aggregates all of their functionalities and even improves upon them.
  • We're soft-deprecating the NumberLocations(), GeoLiftPower.search(), GeoLiftPowerFinder(), and GeoLiftPower(). The development of these functions is now considered complete and they're now superseeded by GeoLiftMarketSelection().
  • We've included two new important metrics that can help us make better decisions when comparing between different test market candidates: Average_MDE and abs_lift_in_zero.
  • A new and improved ranking system takes into consideration all key model performance metrics to identify the best test markets for a given set-up.
  • We've added the option for one tailed tests. These make the GeoLift model much more powerful than it was before, by simply changing the test hypothesis we want to validate. Now you can chose from Positive, Negative and Total tests in the inference section and one sided or two sided tests in the power section.
  • The Market Selection process is more flexible and customizable than before! You can now include additional test constraints to focus only on the tests that make sense for our client. These are: the available budget (budget), acceptable holdout ranges (holdout), test markets we want to force into the test regions (include_markets), and markets that shouldn't be considered as eligible test regions (exclude_markets).
  • We've added a new plotting method for GeoLiftMarketSelection objects. Through this method you can easily plot different test market selections and compare their model fit, MDE, and power curves! Plus, they lines have the GeoLift colors! PlotATTBest
  • The new function: GetWeights() makes it easy to save the synthetic control weights into a data frame for further analysis.
  • To aid with MMM calibrations, we've included a new parameter in the Market Selection/Power Analysis function: Correlations. Setting it to TRUE allows the user to analyze the similarities between test and the control regions.
  • You can also plot historical similarities between test and control regions with plotCorrels(). PlotATTBest
  • A revamped GeoLift walkthrough vignette has been launched to accompany the new version of the package. This new material provides much more detailed explanation on our model, it's parameters, how to run a study, and how to interpret the results.
  • We've fixed multiple bug and errors across the package (thanks for the feedback!).
  • Thanks to all of these changes, we've significantly reduced the total time needed in pre-test calculations!