Artificial Intelligence & Machine Learning
Elena Ehrlich, PhD
eeehrlic@amazon.com
What is AI?
https://www.geospatialworld.net/blogs/difference-between-ai%EF%BB%BF-machine-learning-and-deep-learning/
Agenda
• Image & Video Recognition Rekognition
• Deep-Learning Enabled Video Cameras DeepLens
• Natural Language Understanding Comprehend
• Voice & Convseration Bots Polly, Lex, & Alexa
• Fully-Managed Machine Learning Sagemaker
Image Analysis
AWS Rekognition
Bay
Beach
Coast
Outdoors
Sea
Water
Palm_tree
Plant
Tree
Summer
Landscape
Nature
Hotel
99.18%
99.18%
99.18%
99.18%
99.18%
99.18%
99.21%
99.21%
99.21%
58.3%
51.84%
51.84%
51.24%
Category Confidence
Rekognition: Object & Scene Detection
Rekognition: Facial Analysis
"FaceDetails": [{
"BoundingBox": {
"Height": 0.22111110389232635 ,
"Left": 0.29600000381469727,
"Top": 0.08888889104127884,
"Width": 0.4000000059604645
},
"Confidence": 99.9970474243164,
"Emotions": [{
"Confidence": 98.48326110839844,
"Type": "HAPPY"
}, {
"Confidence": 15.214723587036133,
"Type": "CALM"
}, {
"Confidence": 1.2157082557678223,
"Type": "CONFUSED"
}],
"AgeRange": {
"High": 47,
"Low": 30
},
"Beard": {
"Confidence": 95.77610778808594,
"Value": false
},
"Eyeglasses": {
"Confidence": 99.68527221679688,
"Value": true
},
"EyesOpen": {
"Confidence": 99.99991607666016,
"Value": true
},
"Gender": {
"Confidence": 99.92896270751953,
"Value": ”Female"
},
"MouthOpen": {
"Confidence": 99.90928649902344,
"Value": true
},
"Mustache": {
smart cropping
& ad overlays
sentiment
capture
demographic
analysis
face editing
& pixelation
DetectFaces
{ "contentString":
{
"Attributes": [
"ALL"
],
"Image": {
"Bytes": "..."
}
}
}
Rekognition: Compare Faces
Face Comparision
Hierarchical taxonomy
Confidence score
"ModerationLabels": [
{
"Confidence": 82.7555923461914,
"Name": "Suggestive",
"ParentName": ""
},
{
"Confidence": 82.7555923461914,
"Name": "Female Swimwear or Underwear",
"ParentName": "Suggestive"
},
{
"Confidence": 50.11056137084961,
"Name": "Covered Nudity",
"ParentName": "Nudity and Sexuality"
},
{
"Confidence": 50.11056137084961,
"Name": "Nudity and Sexuality",
"ParentName": ""
},
]
Rekognition: Image Moderation
Suggestive 82.7%
Female Swimwear or Underwear 82.7%
Nudity and Sexuality 50.1%
Covered Nudity 50.1%
https://console.aws.amazon.com/rekognition/home?region=us-east-1#/label-detection
https://d3qtbfbtl5c95j.cloudfront.net/Main.html
http://iad-front.deepvideoanalysis.cloud/results.html#!?identifier=introducingamazongo.mp4
https://aws.amazon.com/rekognition/customers/
Interesting Demos…Time permitting
Deep-Learning Enabled
Video Cameras
AWS DeepLens
DeepLens: Deep-Learning Enabled Video Camera
A DL video camera uses deep convolutional neural networks (CNNs) to analyze visual imagery.
The device itself is a development environment to build computer vision applications.
AWS DeepLens communicates with the following ML endpoints:
• Amazon SageMaker, for model training and validation
• AWS Lambda, event-driven triggers run inference against CNN models
• AWS Greengrass, for deploying updates and functions to your device and other IoT devices
April 2018
Natural
Language Understanding
AWS Comprehend
https://www.ip-watch.org/2018/01/24/itu-4-5-people-ldcs-can-access-mobile-networks-not-using-internet/
Comprehend: Keyword, Sentiment, & Topic Modeling
Comprehend: Keyword, Sentiment, & Topic Modeling
Comprehend: Keyword, Sentiment, & Topic Modeling
Comprehend: Keyword, Sentiment, & Topic Modeling
Life-like Speech
AWS Polly
Polly: Life-like Speech Service
Plain Text SSML Lexicons
Plain Text SSML Lexicons
Speech Synthesis Markup Language
<speak> - Start Tag
<break> - Pause in Speech
<lang> - Specifies the language
<mark> - Tag Name for specific word
<p> - Indicates Paragraph
<phoneme>- phonetic pronunciation
<prosody> - Controls the volume
<s> - Indicates a sentence
<say-as>- Interpretation
<sub> - Alias words
<w> - Customize pronunciation
<amazon:effect name="whispered">
<lexeme>
<grapheme>espresso</grapheme
>
<alias>ess-press-oh</alias>
</lexeme>
Conversational Engines
AWS Lex
Lex: The Advent Of Conversational Interactions
1st Gen: Machine-oriented
interactions
2nd Gen: Control-oriented
& translated
3rd Gen:
Intent-oriented
Speech Recognition Language UnderstandingBusiness Logic
Disparate Systems Authentication
Messaging platforms
Scale
Testing
Security
Availability
Mobile
Lex – Converstaional Engines
Informational Bots
Chatbots for everyday consumer requests
Application Bots
Build powerful interfaces to mobile applications
• News updates
• Weather
information
• Game scores ….
• Book tickets
• Order food
• Manage bank accounts ….
Enterprise Productivity Bots
Streamline enterprise work activities and improve efficiencies
• Check sales numbers
• Marketing performance
• Inventory status ….
Internet of Things (IoT) Bots
Enable conversational interfaces for device interactions
• Wearables
• Appliances
• Auto ….
Operational Bots
Chatbots for IT automation
• Reset my
Password
• TCO analysis
• Productivity….
Most Common Algorithms Provided
•Linear Learner
•Factorization Machines
•XGBoost Algorithm
•Image Classification Algorithm
•Amazon SageMaker Sequence2Sequence
•K-Means Algorithm
•Principal Component Analysis (PCA)
•Latent Dirichlet Allocation (LDA)
•Neural Topic Model (NTM)
•DeepAR Forecasting
•BlazingText
import sagemaker
Sagemaker: Fully-Managed Machine Learning
10x
Performance
Single-Click Training
Train Models at Petabyte Scale
Deploy in Production
Auto-Scaling Cluster of AWS EC2 Instances
OpenSource tools
TensorFlow
Apache MXNet
A/B Testing
Built-in
AI/ML Adoption Benefits
CONVERTING
THE POWER OF
MACHINE
LEARNING INTO
BUSINESS VALUE
MAKING THE
BEST USE OF A
DATA SCIENTISTS
TIME
EMBEDDING
MACHINE
LEARNING INTO
THE FABRIC OF
YOUR BUSINESS
While the power of ML is unrivaled, “data scientists spend around
80% of their time on preparing and managing data for analysis” …
hence only 20% of their time is used to derive insights
The value of data science relies upon operationalizing models
within business applications and processes, yet “50% of the
predictive models [built] don’t get implemented”
While “60% of companies agree that big data will help improve their
decision making and competitiveness … only 28% indicate that they
are currently generating strategic value from their data”
1
2
3
Thank you
Elena Ehrlich, PhD
eeehrlic@amazon.com
Appendix
Services
Platforms
Frameworks
AWS AI/ML: The Stack
Apache
MXNet
KerasGluonPyTorch
Cognitive
Toolkit
Caffe2
& Caffe
Tensor-
Flow
AWS Deep Learning AMI
SageMaker
Mechanical
Turk
AWS
DeepLens
Amazon ML
Spark &
EMR
Speech:
Polly &
Transcribe
Vision:
Rekognition Image &
Rekognition Video
Language:
Lex, Translate &
Comprehend
AWS AI/ML: Notable Successes
Services
Platforms
Frameworks
AWS AI/ML: Solutions for Every Skill Level
• Designed for Developers & Data Scientists
• Solution-oriented Prebuilt Models Available via APIs
• Image Analysis, NLU, NLP, Translation, Text-to-Speech & Speech-to-Text
• Designed for Data Scientists to Address Common Needs
• Fully Managed Platform for Model Building
• Reduces the Heavy Lifting in Model Building & Deployment
• Designed for Data Scientists to Address Advanced / Emerging Needs
• Provides Maximum Flexibility to develop on the leading AI Frameworks
• Enables Expert AI Systems to be Developed & Deployed
Services
Platforms
Frameworks
Real-time &
batch image
analysis
Object & Scene
Detection
Facial Detection Face SearchFacial Analysis
Rekognition: Search & Understand Visual Content
Image Moderation Celebrity Recognition
Rekognition: Image Moderation
Manual
Review
Approved
Rejected
Picture posted
to end users
No inappropriate
content detected
Inappropriate
content detected
Object CreationUpload picture
Users
S3 Bucket Rekognition
Lambda
User
Notification
Rekognition: Video - Case Study Architecture
Polly: Life-like Speech Service
Converts text
to life-like speech
47 voices 27 languages Low latency,
real time
Fully managed
Lex: Build Natural, Conversational Interactions In Voice & Text
Voice & Text
“Chatbots”
Powers
Alexa
Voice interactions
on mobile, web
& devices
Text interaction
with Slack & Messenger
Enterprise
Connectors
(with more coming) Salesforce
Microsoft Dynamics
Marketo
Zendesk
Quickbooks
Hubspot
BOT Intent Slot & Slot type
BOT Intent Slot & Slot type
An intent represents an action that
the user wants to perform
Intent name– A descriptive name for
the intent.
Sample utterances – How a user
might convey the intent.
How to fulfill the intent – How you
want to fulfill the intent after the user
provides the necessary information
Slot - An intent can require zero or
more slots or parameters
Slot type – Each slot has a type.
You can create your custom slot
types or use built-in slot types
Lex: Build Natural, Conversational Interactions In Voice & Text
An Amazon Lex bot is powered
by Automatic Speech
Recognition (ASR) and Natural
Language Understanding
(NLU) capabilities
Response Cards
• Simplify interactions for your users
• Increase bot's accuracy
• Can be used with Facebook Messenger, Slack, and Twilio as well as your own client
applications.
DeepLens Architecture
IoT Anomaly Detection
AWS Kinesis Analytics
Kinesis Analytics: real-time insights from streaming data
Kinesis Analytics: real-time insights from streaming data
AI Inquisitors AI Adopters AI Experts
Interested in AI but
have limited expertise
and/or resources
Limited expertise
and/or use of AI for
one-off projects
Advanced expertise
and/or use of
embedded AI in apps
AI/ML Assessment
Business Value Ability to Execute Data Availability
Assessing POC Targets: Criteria
Prep Question Sample Answer
What Business or Operational benefits are you trying to drive? • Improve content personalization
How will you consume the outputs and put them into action?
• Content will be distributed at a
targeted level
What types of data is available today? Where does the data
reside?
• Content and subscription data
What types of analytics and/or machine learning are being
employed today?
• Business Intelligence
• Predictive Analytics
What staff and/or consultants currently support these activities?
• Data Engineers
• Data Scientists
What software currently supports these activities? • R / Python
What is your ideal scenario in tackling these business objectives? • One-to-one content for individuals
What challenges have you experienced when deploying AI?
• Prioritization of Targets
• Operationalization
AI/ML Assessment

AI and machine learning

  • 1.
    Artificial Intelligence &Machine Learning Elena Ehrlich, PhD eeehrlic@amazon.com
  • 2.
  • 3.
    Agenda • Image &Video Recognition Rekognition • Deep-Learning Enabled Video Cameras DeepLens • Natural Language Understanding Comprehend • Voice & Convseration Bots Polly, Lex, & Alexa • Fully-Managed Machine Learning Sagemaker
  • 4.
  • 5.
  • 6.
    Rekognition: Facial Analysis "FaceDetails":[{ "BoundingBox": { "Height": 0.22111110389232635 , "Left": 0.29600000381469727, "Top": 0.08888889104127884, "Width": 0.4000000059604645 }, "Confidence": 99.9970474243164, "Emotions": [{ "Confidence": 98.48326110839844, "Type": "HAPPY" }, { "Confidence": 15.214723587036133, "Type": "CALM" }, { "Confidence": 1.2157082557678223, "Type": "CONFUSED" }], "AgeRange": { "High": 47, "Low": 30 }, "Beard": { "Confidence": 95.77610778808594, "Value": false }, "Eyeglasses": { "Confidence": 99.68527221679688, "Value": true }, "EyesOpen": { "Confidence": 99.99991607666016, "Value": true }, "Gender": { "Confidence": 99.92896270751953, "Value": ”Female" }, "MouthOpen": { "Confidence": 99.90928649902344, "Value": true }, "Mustache": { smart cropping & ad overlays sentiment capture demographic analysis face editing & pixelation DetectFaces { "contentString": { "Attributes": [ "ALL" ], "Image": { "Bytes": "..." } } }
  • 7.
  • 8.
    Hierarchical taxonomy Confidence score "ModerationLabels":[ { "Confidence": 82.7555923461914, "Name": "Suggestive", "ParentName": "" }, { "Confidence": 82.7555923461914, "Name": "Female Swimwear or Underwear", "ParentName": "Suggestive" }, { "Confidence": 50.11056137084961, "Name": "Covered Nudity", "ParentName": "Nudity and Sexuality" }, { "Confidence": 50.11056137084961, "Name": "Nudity and Sexuality", "ParentName": "" }, ] Rekognition: Image Moderation Suggestive 82.7% Female Swimwear or Underwear 82.7% Nudity and Sexuality 50.1% Covered Nudity 50.1%
  • 9.
  • 10.
  • 11.
    DeepLens: Deep-Learning EnabledVideo Camera A DL video camera uses deep convolutional neural networks (CNNs) to analyze visual imagery. The device itself is a development environment to build computer vision applications. AWS DeepLens communicates with the following ML endpoints: • Amazon SageMaker, for model training and validation • AWS Lambda, event-driven triggers run inference against CNN models • AWS Greengrass, for deploying updates and functions to your device and other IoT devices April 2018
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
    Polly: Life-like SpeechService Plain Text SSML Lexicons Plain Text SSML Lexicons Speech Synthesis Markup Language <speak> - Start Tag <break> - Pause in Speech <lang> - Specifies the language <mark> - Tag Name for specific word <p> - Indicates Paragraph <phoneme>- phonetic pronunciation <prosody> - Controls the volume <s> - Indicates a sentence <say-as>- Interpretation <sub> - Alias words <w> - Customize pronunciation <amazon:effect name="whispered"> <lexeme> <grapheme>espresso</grapheme > <alias>ess-press-oh</alias> </lexeme>
  • 19.
  • 20.
    Lex: The AdventOf Conversational Interactions 1st Gen: Machine-oriented interactions 2nd Gen: Control-oriented & translated 3rd Gen: Intent-oriented Speech Recognition Language UnderstandingBusiness Logic Disparate Systems Authentication Messaging platforms Scale Testing Security Availability Mobile
  • 21.
    Lex – ConverstaionalEngines Informational Bots Chatbots for everyday consumer requests Application Bots Build powerful interfaces to mobile applications • News updates • Weather information • Game scores …. • Book tickets • Order food • Manage bank accounts …. Enterprise Productivity Bots Streamline enterprise work activities and improve efficiencies • Check sales numbers • Marketing performance • Inventory status …. Internet of Things (IoT) Bots Enable conversational interfaces for device interactions • Wearables • Appliances • Auto …. Operational Bots Chatbots for IT automation • Reset my Password • TCO analysis • Productivity….
  • 23.
    Most Common AlgorithmsProvided •Linear Learner •Factorization Machines •XGBoost Algorithm •Image Classification Algorithm •Amazon SageMaker Sequence2Sequence •K-Means Algorithm •Principal Component Analysis (PCA) •Latent Dirichlet Allocation (LDA) •Neural Topic Model (NTM) •DeepAR Forecasting •BlazingText import sagemaker Sagemaker: Fully-Managed Machine Learning 10x Performance Single-Click Training Train Models at Petabyte Scale Deploy in Production Auto-Scaling Cluster of AWS EC2 Instances OpenSource tools TensorFlow Apache MXNet A/B Testing Built-in
  • 24.
    AI/ML Adoption Benefits CONVERTING THEPOWER OF MACHINE LEARNING INTO BUSINESS VALUE MAKING THE BEST USE OF A DATA SCIENTISTS TIME EMBEDDING MACHINE LEARNING INTO THE FABRIC OF YOUR BUSINESS While the power of ML is unrivaled, “data scientists spend around 80% of their time on preparing and managing data for analysis” … hence only 20% of their time is used to derive insights The value of data science relies upon operationalizing models within business applications and processes, yet “50% of the predictive models [built] don’t get implemented” While “60% of companies agree that big data will help improve their decision making and competitiveness … only 28% indicate that they are currently generating strategic value from their data” 1 2 3
  • 25.
    Thank you Elena Ehrlich,PhD eeehrlic@amazon.com
  • 26.
  • 27.
    Services Platforms Frameworks AWS AI/ML: TheStack Apache MXNet KerasGluonPyTorch Cognitive Toolkit Caffe2 & Caffe Tensor- Flow AWS Deep Learning AMI SageMaker Mechanical Turk AWS DeepLens Amazon ML Spark & EMR Speech: Polly & Transcribe Vision: Rekognition Image & Rekognition Video Language: Lex, Translate & Comprehend
  • 28.
    AWS AI/ML: NotableSuccesses Services Platforms Frameworks
  • 29.
    AWS AI/ML: Solutionsfor Every Skill Level • Designed for Developers & Data Scientists • Solution-oriented Prebuilt Models Available via APIs • Image Analysis, NLU, NLP, Translation, Text-to-Speech & Speech-to-Text • Designed for Data Scientists to Address Common Needs • Fully Managed Platform for Model Building • Reduces the Heavy Lifting in Model Building & Deployment • Designed for Data Scientists to Address Advanced / Emerging Needs • Provides Maximum Flexibility to develop on the leading AI Frameworks • Enables Expert AI Systems to be Developed & Deployed Services Platforms Frameworks
  • 30.
    Real-time & batch image analysis Object& Scene Detection Facial Detection Face SearchFacial Analysis Rekognition: Search & Understand Visual Content Image Moderation Celebrity Recognition
  • 31.
    Rekognition: Image Moderation Manual Review Approved Rejected Pictureposted to end users No inappropriate content detected Inappropriate content detected Object CreationUpload picture Users S3 Bucket Rekognition Lambda User Notification
  • 32.
    Rekognition: Video -Case Study Architecture
  • 33.
    Polly: Life-like SpeechService Converts text to life-like speech 47 voices 27 languages Low latency, real time Fully managed
  • 34.
    Lex: Build Natural,Conversational Interactions In Voice & Text Voice & Text “Chatbots” Powers Alexa Voice interactions on mobile, web & devices Text interaction with Slack & Messenger Enterprise Connectors (with more coming) Salesforce Microsoft Dynamics Marketo Zendesk Quickbooks Hubspot
  • 35.
    BOT Intent Slot& Slot type BOT Intent Slot & Slot type An intent represents an action that the user wants to perform Intent name– A descriptive name for the intent. Sample utterances – How a user might convey the intent. How to fulfill the intent – How you want to fulfill the intent after the user provides the necessary information Slot - An intent can require zero or more slots or parameters Slot type – Each slot has a type. You can create your custom slot types or use built-in slot types Lex: Build Natural, Conversational Interactions In Voice & Text An Amazon Lex bot is powered by Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) capabilities
  • 36.
    Response Cards • Simplifyinteractions for your users • Increase bot's accuracy • Can be used with Facebook Messenger, Slack, and Twilio as well as your own client applications.
  • 37.
  • 38.
    IoT Anomaly Detection AWSKinesis Analytics
  • 39.
    Kinesis Analytics: real-timeinsights from streaming data
  • 40.
    Kinesis Analytics: real-timeinsights from streaming data
  • 41.
    AI Inquisitors AIAdopters AI Experts Interested in AI but have limited expertise and/or resources Limited expertise and/or use of AI for one-off projects Advanced expertise and/or use of embedded AI in apps AI/ML Assessment
  • 42.
    Business Value Abilityto Execute Data Availability Assessing POC Targets: Criteria
  • 43.
    Prep Question SampleAnswer What Business or Operational benefits are you trying to drive? • Improve content personalization How will you consume the outputs and put them into action? • Content will be distributed at a targeted level What types of data is available today? Where does the data reside? • Content and subscription data What types of analytics and/or machine learning are being employed today? • Business Intelligence • Predictive Analytics What staff and/or consultants currently support these activities? • Data Engineers • Data Scientists What software currently supports these activities? • R / Python What is your ideal scenario in tackling these business objectives? • One-to-one content for individuals What challenges have you experienced when deploying AI? • Prioritization of Targets • Operationalization AI/ML Assessment