Billions and Billions: 
Machines, Algorithms, and Growing 
Business in Programmatic Markets 
Ari Buchalter COO, MediaMath 
PhD, Astrophysics
What do these things have in common? 
The Digital Advertising Universe The Actual Universe 
• Both are complex systems 
• Math can be applied to understand both
The evolution of media decision making 
Past 60 years – “Audience-based” Today – “Goal-based” 
Describe your audience 
Figure out media they consume 
Buy placements, wait, and hope 
Get report, manually adjust 
Define your marketing goal 
Capture all the data (media, user) 
Model it to identify what works 
Automate the buying 
Humans making coarse 
decisions based on proxies, 
averages, and indexes 
Machines making exact 
decisions based on granular 
user data
We’ve seen this movie before…
We’ve seen this movie before…
…and we understand the benefits 
Buy in batch (wheat + chaff) Buy what you want (wheat only) 
Fixed price, regardless of value Variable bidding, aligned with value 
Little/no insight into true drivers Full insights into “what” & “why” 
Manual, labor-intensive (~5/FTE) Fully automated, scalable (~50/FTE) 
And the results are typically 10x better, BUT there’s a cost…. 
Analyze 10-20 buys weekly Analyze 1MM opps. per sec.
Let’s talk about Big Data in Programmatic 
~100 BILLION impressions per day 
~100 variables per impression 
~100 values per variable 
EQUALS 
~1,000,000,000,000,000 
Possible combinations of data per day 
(1015 = ONE QUADRILLION)
Making sense of the chaos 
Algorithms 
Optimization 
Programmatic 
Automation 
Predictive Modeling 
Machine learning 
Decision engines
Getting inside the RTB transaction 
SSP or 
Exchange 
Publisher 
Consumer 
DSP 
Advertiser 
Agency or 
Trading Desk
The two (buyer) questions that matter 
What is the right bid for each impression? 
Which impressions should I buy?
Why does question #1 matter? 
What is the right bid for each impression? 
Too high 
Overpay & 
underperform 
Too low 
Lose out & 
underspend 
“Goldilocks” bid 
Maximize scale & 
performance
Why does question #2 matter? 
Which impressions should I buy? 
• ~$100MM/day of RTB supply 
• Typical campaign spends ~$1K/day 
(0.001% of total supply) 
• Not buying the RIGHT 0.001% is 
throwing money away
Answering the questions ain’t easy 
Data is large, and growing  need technology at scale 
It’s called different things  need to “normalize” data 
Data interactions are complex  need sophisticated models 
Mix of goals (upper/lower funnel)  need flexible methodology 
Supply & demand constantly changing  need to remodel often 
Clients need to understand  need intuitive, transparent output 
It’s all in real-time (100ms)  need speed without latency 
Only a machine-learning algorithmic approach can handle 
the size, variability, complexity, and speed required
Question #1 – A simple exercise 
What is the right bid for each impression? 
$1 prize 
Flip a coin to 
win 1 dollar 
50% chance $0.50 
Bid Price 
Goal Value x Action Rate = Bid Price
Question #1 – The real thing 
What is the right bid for each impression? 
Goal Value x Action Rate = Bid Price 
1% chance consumer 
takes desired action 
(purchase) 
$50 value to 
advertiser 
(CPA) 
Bid for an 
RTB ad 
$0.50 bid 
price 
(breakeven) 
YOUR AD HERE
Question #1 – The Goal Value 
(Input) (Prediction) (Output) 
Goal Value 
x Action Rate = Bid Price 
The goal can be anything at all: 
• Branding: positive survey response (awareness, intent, etc.) 
• Engagement: site visit, site action (locate store, post comment) 
• Conversion: signup, application, purchase, etc. 
• Retention: repeat purchase, renewal, upsell 
If it can be measured, it can be made better by math
Question #1 – The Action Rate 
(Input) (Prediction) (Output) 
Goal Value x Action Rate 
= Bid Price 
Predictive modeling: the process by which a mathematical model is created to 
predict the probability of an outcome, usually based on historical input data 
The model should base the prediction on all available data: 
• User: site activity (1p), interests & behaviors (3p), geo, TOD, DOW, etc. 
• Media: channel, publisher, page, ad size, above/below fold, etc. 
• Creative: image, offer, call to action, etc.
Answering question #1 
Video 
Publisher: YouTube 
Unit: 15 sec pre roll 
Time: 16.46 – 17.00 
Age: 25-34 
Gender: Male 
Price: $15.76 CPM 
Social 
Publisher: FBX 
Unit: Newsfeed 
Day: Tuesday 
Time: 5.00pm – 5.15pm 
Price: $2.30 CPM 
Display 
Publisher: Rubicon 
Data: Rakuten Male 
Location: Tokyo 
Creative size: 160 x 600 
Price: $0.63 CPM 
A different model for every creative in every 
campaign of every advertiser – all in real time!
Question #2 – Which ones to bid on? 
Optimization: the process of making 
the best choice among a set of 
options to achieve a desired goal, 
usually under some constraints 
Example – Shopping for food 
Constraints: fixed budget, 
nothing artificial 
Goal: Most mass of food? 
Most volume of food? 
Healthiest mix?
Question #2 – Two important concepts 
1) Bid Price: How much the impression is worth to the buyer 
• Depends on who the publisher is and who the advertiser is 
• Is a measure of quality (i.e., what it’s worth to the buyer) 
2) Market Price: The price the impression will clear for 
• Depends on the entire marketplace 
• Also obtained through predictive modeling
Question #2 – A meaty example 
Bid Price: 
$30 
Bid Price: 
$30 
$30 
High 
(good quality) 
Bid Price 
Low 
(poor quality) 
Bid Price: 
$2 
Bid Price: 
$2 
$2 
YES! 
Market Price 
High 
Eh, OK 
(not a deal) 
Low 
(a deal!) 
NO! 
Eh, OK 
$30 
Selling for: 
$30 
Selling for: 
$30 
$2 
Selling for: 
$2 
Selling for: 
$2
Answering question #2 – Which to bid on? 
III IV 
Quality-driven 
performance 
<10% of impressions 
I II 
performance 
40-70% of impressions 
Value-driven 
performance 
<5% of impressions 
Cost-driven 
performance 
20-50% of impressions 
Relative Value 
Low High 
Non 
Low High 
Bid Price
Putting it all together 
1) Use a predictive model 
to determine what each 
impression is worth 
2) Use optimization to 
determine which 
impressions to bid for 
What is the right bid for each impression? 
Which impressions should I buy?
So where do I get me some of those? 
Find a partner who: 
 Leverages robust technology – ask to see the scale & speed 
 Has proven results – across verticals, geographies, over time 
 Will expose the “black box” – transparency & insights are key! 
 Has cross-channel capabilities – display, video, social, mobile, premium, BYO 
 Has broad integrations – 3p data, surveys, viewability, attribution, etc. 
 Can incorporate 1st party offline data – increasingly important 
 Develops custom solutions – to suit your unique business needs 
 Makes it easy – execution, workflow, reporting, testing, etc. 
 Provides thought partnership & great service – it’s not just machines! 
(machines just enable people to do the REAL value-added work)
Forrester DSP Wave: MediaMath is #1 
“MediaMath boasts excellent algorithmic 
optimization capabilities (including a 
multifaceted view of the decisioning engine’s 
output), and its multichannel media and data 
access is both broad and deep.” 
“MediaMath is a great all-around choice 
for buyers in market for a DSP.” 
“Its large employee base and diverse, well-tenured 
management team also provide the 
necessary foundation for it to execute 
effectively on its strategic vision: to 
empower marketing professionals with a 
flexible, easy-to-use, multichannel platform.”

Billions and Billions: Machines, Algorithms, and Growing Business in Programamtic Markets

  • 1.
    Billions and Billions: Machines, Algorithms, and Growing Business in Programmatic Markets Ari Buchalter COO, MediaMath PhD, Astrophysics
  • 2.
    What do thesethings have in common? The Digital Advertising Universe The Actual Universe • Both are complex systems • Math can be applied to understand both
  • 5.
    The evolution ofmedia decision making Past 60 years – “Audience-based” Today – “Goal-based” Describe your audience Figure out media they consume Buy placements, wait, and hope Get report, manually adjust Define your marketing goal Capture all the data (media, user) Model it to identify what works Automate the buying Humans making coarse decisions based on proxies, averages, and indexes Machines making exact decisions based on granular user data
  • 6.
    We’ve seen thismovie before…
  • 7.
    We’ve seen thismovie before…
  • 8.
    …and we understandthe benefits Buy in batch (wheat + chaff) Buy what you want (wheat only) Fixed price, regardless of value Variable bidding, aligned with value Little/no insight into true drivers Full insights into “what” & “why” Manual, labor-intensive (~5/FTE) Fully automated, scalable (~50/FTE) And the results are typically 10x better, BUT there’s a cost…. Analyze 10-20 buys weekly Analyze 1MM opps. per sec.
  • 9.
    Let’s talk aboutBig Data in Programmatic ~100 BILLION impressions per day ~100 variables per impression ~100 values per variable EQUALS ~1,000,000,000,000,000 Possible combinations of data per day (1015 = ONE QUADRILLION)
  • 10.
    Making sense ofthe chaos Algorithms Optimization Programmatic Automation Predictive Modeling Machine learning Decision engines
  • 11.
    Getting inside theRTB transaction SSP or Exchange Publisher Consumer DSP Advertiser Agency or Trading Desk
  • 12.
    The two (buyer)questions that matter What is the right bid for each impression? Which impressions should I buy?
  • 13.
    Why does question#1 matter? What is the right bid for each impression? Too high Overpay & underperform Too low Lose out & underspend “Goldilocks” bid Maximize scale & performance
  • 14.
    Why does question#2 matter? Which impressions should I buy? • ~$100MM/day of RTB supply • Typical campaign spends ~$1K/day (0.001% of total supply) • Not buying the RIGHT 0.001% is throwing money away
  • 15.
    Answering the questionsain’t easy Data is large, and growing  need technology at scale It’s called different things  need to “normalize” data Data interactions are complex  need sophisticated models Mix of goals (upper/lower funnel)  need flexible methodology Supply & demand constantly changing  need to remodel often Clients need to understand  need intuitive, transparent output It’s all in real-time (100ms)  need speed without latency Only a machine-learning algorithmic approach can handle the size, variability, complexity, and speed required
  • 16.
    Question #1 –A simple exercise What is the right bid for each impression? $1 prize Flip a coin to win 1 dollar 50% chance $0.50 Bid Price Goal Value x Action Rate = Bid Price
  • 17.
    Question #1 –The real thing What is the right bid for each impression? Goal Value x Action Rate = Bid Price 1% chance consumer takes desired action (purchase) $50 value to advertiser (CPA) Bid for an RTB ad $0.50 bid price (breakeven) YOUR AD HERE
  • 18.
    Question #1 –The Goal Value (Input) (Prediction) (Output) Goal Value x Action Rate = Bid Price The goal can be anything at all: • Branding: positive survey response (awareness, intent, etc.) • Engagement: site visit, site action (locate store, post comment) • Conversion: signup, application, purchase, etc. • Retention: repeat purchase, renewal, upsell If it can be measured, it can be made better by math
  • 19.
    Question #1 –The Action Rate (Input) (Prediction) (Output) Goal Value x Action Rate = Bid Price Predictive modeling: the process by which a mathematical model is created to predict the probability of an outcome, usually based on historical input data The model should base the prediction on all available data: • User: site activity (1p), interests & behaviors (3p), geo, TOD, DOW, etc. • Media: channel, publisher, page, ad size, above/below fold, etc. • Creative: image, offer, call to action, etc.
  • 20.
    Answering question #1 Video Publisher: YouTube Unit: 15 sec pre roll Time: 16.46 – 17.00 Age: 25-34 Gender: Male Price: $15.76 CPM Social Publisher: FBX Unit: Newsfeed Day: Tuesday Time: 5.00pm – 5.15pm Price: $2.30 CPM Display Publisher: Rubicon Data: Rakuten Male Location: Tokyo Creative size: 160 x 600 Price: $0.63 CPM A different model for every creative in every campaign of every advertiser – all in real time!
  • 21.
    Question #2 –Which ones to bid on? Optimization: the process of making the best choice among a set of options to achieve a desired goal, usually under some constraints Example – Shopping for food Constraints: fixed budget, nothing artificial Goal: Most mass of food? Most volume of food? Healthiest mix?
  • 22.
    Question #2 –Two important concepts 1) Bid Price: How much the impression is worth to the buyer • Depends on who the publisher is and who the advertiser is • Is a measure of quality (i.e., what it’s worth to the buyer) 2) Market Price: The price the impression will clear for • Depends on the entire marketplace • Also obtained through predictive modeling
  • 23.
    Question #2 –A meaty example Bid Price: $30 Bid Price: $30 $30 High (good quality) Bid Price Low (poor quality) Bid Price: $2 Bid Price: $2 $2 YES! Market Price High Eh, OK (not a deal) Low (a deal!) NO! Eh, OK $30 Selling for: $30 Selling for: $30 $2 Selling for: $2 Selling for: $2
  • 24.
    Answering question #2– Which to bid on? III IV Quality-driven performance <10% of impressions I II performance 40-70% of impressions Value-driven performance <5% of impressions Cost-driven performance 20-50% of impressions Relative Value Low High Non Low High Bid Price
  • 25.
    Putting it alltogether 1) Use a predictive model to determine what each impression is worth 2) Use optimization to determine which impressions to bid for What is the right bid for each impression? Which impressions should I buy?
  • 26.
    So where doI get me some of those? Find a partner who:  Leverages robust technology – ask to see the scale & speed  Has proven results – across verticals, geographies, over time  Will expose the “black box” – transparency & insights are key!  Has cross-channel capabilities – display, video, social, mobile, premium, BYO  Has broad integrations – 3p data, surveys, viewability, attribution, etc.  Can incorporate 1st party offline data – increasingly important  Develops custom solutions – to suit your unique business needs  Makes it easy – execution, workflow, reporting, testing, etc.  Provides thought partnership & great service – it’s not just machines! (machines just enable people to do the REAL value-added work)
  • 27.
    Forrester DSP Wave:MediaMath is #1 “MediaMath boasts excellent algorithmic optimization capabilities (including a multifaceted view of the decisioning engine’s output), and its multichannel media and data access is both broad and deep.” “MediaMath is a great all-around choice for buyers in market for a DSP.” “Its large employee base and diverse, well-tenured management team also provide the necessary foundation for it to execute effectively on its strategic vision: to empower marketing professionals with a flexible, easy-to-use, multichannel platform.”