Inside a “Model Zoo”: How Propensity Models Ship So Fast

One of the claims that we sometimes see in regard to modern personalization systems is that they offer a “Model Zoo” of ready-to-use models, and that they can rapidly build new models in a very short amount of time (for example, 30 minutes to 3 hours).

How can a whole new model be created in such a short amount of time?

And what on earth is a model zoo?

In this context, a “model zoo” refers to a curated set of prebuilt, pre‑tuned models and pipelines that are used for common tasks like classification, regression or ranking, and that can be called upon quickly with minimal configuration and dataset mapping. Having this ready-at-hand accelerates time‑to‑value by reusing well‑tested feature pipelines, encoders and model hyperparameter defaults. And this is precisely what’s behind platforms that describe themselves as offering a large number of “standard models out-of-the-box.”

Building on this foundation, we typically also find a set of techniques and tools that automate the end-to-end process of applying machine learning to this short-listed set of best-in-breed models, enabling algorithms to automatically discover the best approaches for themselves, rather than requiring human data scientists to manually experiment with different models and settings. Such tools and methods are commonly referred to as “AutoML”.

Google CEO Sundar Pichai describes AutoML as “an approach which shows that it’s possible for neural nets to design neural nets.”

Specifically in our context, AutoML can quickly handle missing values, categorical encoding, outlier treatment, timestamp features, standard aggregations, and useful data conversions like text to numbers, dates to usable features, and so forth – even for datasets with millions of rows.

Also, AutoML can automatically create hundreds of new, relevant features from existing data by combining fields, or by applying mathematical transformations, and by testing which combinations improve model performance.

Another area where AutoML is quite useful is as an efficient alternative to grid search, which was the old brute-force way of tuning models that many of us used not that long ago, where you had to manually list a defined set of parameter values in Python (various learning rates × specified tree depths × regularization strengths) and then pick the winning combination that performed best on validation data.

Instead of this, AutoML pipelines typically to use sensible defaults, or smart starting guesses learned from what worked on similar datasets in the past, thereby quickly converging to near‑optimal model parameter settings in minutes rather than hours.

And of course, for Marketing purposes, what we care about most is propensity – the likelihood that a particular event of interest will happen for a particular customer within a given period of time – either a “good” event, such as likelihood to agree to an upgrade offer, or an event we hope to avoid, such as losing that customer to churn within 30 days.

Often this is predicted using decision trees, where the data is iteratively split into subsets based on the most relevant feature at each step, and where the goal is to create subsets that are as pure and as similar as possible with respect to the variable we’re trying to predict.

For propensity models, the degree of likelihood is usually expressed on a scale of 0-1 or on a scale of 0-100, where the record-level result is the individual customer score, and where this score, in turn, is used to rank a dataset from most-likely to least-likely in regard to what we’re seeking to predict. Then, after ranking the database, we typically set a cut-off threshold based on the data, and declare all customers above that threshold to be “likely”. This Yes/No answer can then be used as a feature in other models.

Tree-based models are popular for this because:

• The results are often fairly intuitive
• They handle nonlinear relationships well
• They automatically capture interaction effects between variables
• They work well with messy, tabular business data
• They usually don’t require much feature scaling, and
• They’re often quite robust to missing values

But today’s systems aren’t built using a single decision tree model. They’re built using ensemble methods that combine predictions from multiple models – such as multiple decision trees, for example.

And two of the top ensemble options are Random Forest and Gradient Boosting.

Random Forest creates multiple decision trees, each one trained on random portions of the data, and it then combines all those predictions together, by using majority voting (for classification problems), or by averaging (for regression problems). This delivers a more accurate final answer, because the diversity among the trees helps to prevent the model from overfitting on the training data.

Gradient Boosting takes a different approach. Instead of creating many independent models and looking for commonality among them, it creates its models sequentially, where each new model focuses on correcting the mistakes made by the previous ones, so it learns sequentially. This step-by-step improvement process often results in very accurate predictions.

Both of these methods are popular options for Marketing because:

• They integrate well with CDP-type feature tables (one row per customer with aggregated features)
• They train quickly, and
• They can handle messy real-world signals

Here are some typical examples of prebuilt templates that might be included within a Marketing-focused model zoo.

• Purchase propensity in the next ‘n’ days
• Churn risk in the next ‘n’ days
• Reactivation likelihood
• Upsell / cross-sell propensity (at the category-level)
• Channel response propensity (email vs. SMS vs. app)
• Likelihood to convert if given a discount or other incentive
• Likelihood to purchase a high-margin item
• Likelihood to visit (restaurants, travel, hospitality)
• Likelihood to accept an upgrade offer (insurance, airlines, software, hospitality tiers)
• Propensity to try a new category (in grocery, for example)

In short, the phenomena of ready-to-deploy out-of-the-box models, as well as the ability to spin up new models in minutes or hours, are inter-related developments, and are both enabled by the creation of a model zoo, in conjunction with AutoML.

In fact, under the hood, all of these out-of-the-box models might be handled by the same or similar AutoML mechanics – just with different labels and feature sets, so that helps to explain the kind of scale we’re seeing today: Numerous models, out-of-the-box, ready-to-go on Day One.