THE WHAT, WHY & HOW
OF (WEB) ANALYTICS
TESTING
Anand Bagmar
Software Quality Evangelist
ABOUT ME
@BagmarAnand
about.me/anand.bagmar
Web Analytics
WHAT IS WEB ANALYTICS?
WHY DO WE NEED WEB ANALYTICS?
WEB ANALYTICS SOLUTIONS
Google Analytics
SiteCatalyst
AWStats
WebTrends
….
Testing Web Analytics
TESTING AT THE REPORT LEVEL
Pros
¨ Ensure report is setup correctly
Cons
¨Licensing
¨Reports not yet setup
¨Validate all requests are sent / captured
Biggest Problem
It is TOO LATE!
The Solution
Web Analytics Testing
Challenges
MANUAL
REPEATING OVER-AND-OVER AGAIN
The Automation
Solution
WAAT - Web Analytics
Automation Testing
WHAT DOES WAAT DO FOR ME?
q Plugs into existing Test Automation Framework
q With minimal changes
q Web Analytic tool independent
q UI Driver framework independent
q Browser independent
FLAVORS
WAAT-JAVA
q Original flavor
q Supports 2 plugins
q Omniture Debugger
q Proxy
q HttpSniffer
q JsSniffer
q Available on github
Will be eventually available as a Maven Dependency!
WAAT-RUBY
q Ruby gem implemented over WAAT-Java
q Uses RJB – Ruby-Java-Bridge
q Supports 2 plugins
q HttpSniffer
q JsSniffer
q Available on github and rubygems.org
Will be changing soon!
WAAT-JAVA
q Original flavor
q Supports 3 plugins
q Omniture Debugger
q HttpSniffer
q JsSniffer
q Available on github
WAAT-RUBY
q Ruby gem implemented over WAAT-Java
q Uses RJB – Ruby-Java-Bridge
q Supports 2 plugins
q HttpSniffer
q JsSniffer
q Available on github and rubygems.org
OMNITURE DEBUGGER
OMNITURE DEBUGGER
Pros
q OS independent
q Run using the regular test-user
Cons
q Browser dependent – need to implement ScriptRunner
for the UI-driver in use
q Web-Analytic solution dependent – Adobe Marketing
Cloud / Omniture SiteCatalyst
HTTPSNIFFER
Pros
q Web-analytic solution independent
q Browser independent
q UI-driver independent
Cons
q 3
rd
party libraries are OS Dependent
q HTTPs not supported out-of-the-box
q Run tests as ‘root’
JSSNIFFER
Pros
q Web-analytic solution independent
q Browser independent
q HTTPs supported out-of-the-box
q No dependency on any 3
rd
party library
Cons
q Need to write JavaScript to get the URL from the browser
q UI-driver dependent
@BagmarAnand #waat how can I contribute?
PROXY DEBUGGER
Pros
q Web-analytic solution independent
q Browser independent
q UI-driver dependent
q HTTPs supported out-of-the-box
q No dependency on any 3
rd
party library
Cons
q Making it work for Mobile needs manual setup
ARCHITECTURE
Is that all to
Web Analytics?
The new “kids” in town
IoT & Big Data
Some popular use cases
https://www.kaaproject.org/iot-use-cases/
IOT – INTERNET OF THINGS
Opportunity to
¨ Create new value propositions
¨ Be Innovative & Creative
IOT – HOW TO DELIVER VALUE?
¨ Automate the manual processes
¨ Integrate data capabilities
¨ Collect – integrate from various sources
¨ Repeat collection – automate the collection
¨ Analyze – manual & machine learning
¨ Optimize / Pivot
¨ Repeat
IOT – CHALLENGES
¨ Federated devices,
¨ Different types of networks
¨ Different communication channels
¨ Physical (hardware) & Virtual (software)
IOT – BIGGER CHALLENGES
¨ Too many devices
¨ Lots of data
BIG DATA …. IS GETTING BIGGER
¨ Volumes of data generated
¨ A jet engine generates 1TB of data per flight.
¨ A large refinery generates 1TB of raw data per day.
¨ As cars get smarter, the number of sensors is projected to reach as many as 200
per car.
¨ Sensors of all types will generate immense amounts of data. In fact, analysts
estimate that by 2020, 40 percent of all data will come from sensors.
¨ IoT leads to massive volumes of data
http://www.cisco.com/web/solutions/trends/iot/docs/iot-data-analytics-white-paper
What does this mean
for Testing?
TESTING OPPORTUNITIES
¨ “Testing” of these new integrations & its value
propositions
¨ Performance
¨ Reliability
¨ Security
https://www.wired.com/2017/03/medical-devices-next-
security-nightmare/?mbid=social_twitter_onsiteshare
TESTING OPPORTUNITIES ...
¨ Build capabilities (and tools) to validate –
¨ Data collection is working well
¨ From all sources
¨ Analyzing data, capturing patterns and trends
¨ Optimize business value
¨ Create new opportunities and value propositions
@BagmarAnand
about.me/anand.bagmar
THANK YOU
RESOURCES
¨ https://github.com/anandbagmar/waat
¨ https://github.com/anandbagmar/waat-ruby
¨ http://essenceoftesting.blogspot.in/search/label/waat
¨ http://www.cisco.com/web/solutions/trends/iot/docs/io
t-data-analytics-white-paper
¨ http://www.dezyre.com/article/how-big-data-analysis-
helped-increase-walmart-s-sales-turnover/109

The What, Why and How of (Web) Analytics Testing (Web, IoT, Big Data)

  • 1.
    THE WHAT, WHY& HOW OF (WEB) ANALYTICS TESTING Anand Bagmar Software Quality Evangelist
  • 2.
  • 4.
  • 5.
    WHAT IS WEBANALYTICS?
  • 6.
    WHY DO WENEED WEB ANALYTICS?
  • 8.
    WEB ANALYTICS SOLUTIONS GoogleAnalytics SiteCatalyst AWStats WebTrends ….
  • 9.
  • 13.
    TESTING AT THEREPORT LEVEL Pros ¨ Ensure report is setup correctly Cons ¨Licensing ¨Reports not yet setup ¨Validate all requests are sent / captured
  • 14.
  • 15.
  • 19.
  • 20.
  • 21.
  • 23.
  • 24.
    WAAT - WebAnalytics Automation Testing
  • 25.
    WHAT DOES WAATDO FOR ME? q Plugs into existing Test Automation Framework q With minimal changes q Web Analytic tool independent q UI Driver framework independent q Browser independent
  • 26.
  • 27.
    WAAT-JAVA q Original flavor qSupports 2 plugins q Omniture Debugger q Proxy q HttpSniffer q JsSniffer q Available on github Will be eventually available as a Maven Dependency!
  • 28.
    WAAT-RUBY q Ruby gemimplemented over WAAT-Java q Uses RJB – Ruby-Java-Bridge q Supports 2 plugins q HttpSniffer q JsSniffer q Available on github and rubygems.org Will be changing soon!
  • 29.
    WAAT-JAVA q Original flavor qSupports 3 plugins q Omniture Debugger q HttpSniffer q JsSniffer q Available on github
  • 30.
    WAAT-RUBY q Ruby gemimplemented over WAAT-Java q Uses RJB – Ruby-Java-Bridge q Supports 2 plugins q HttpSniffer q JsSniffer q Available on github and rubygems.org
  • 31.
  • 32.
    OMNITURE DEBUGGER Pros q OSindependent q Run using the regular test-user Cons q Browser dependent – need to implement ScriptRunner for the UI-driver in use q Web-Analytic solution dependent – Adobe Marketing Cloud / Omniture SiteCatalyst
  • 33.
    HTTPSNIFFER Pros q Web-analytic solutionindependent q Browser independent q UI-driver independent Cons q 3 rd party libraries are OS Dependent q HTTPs not supported out-of-the-box q Run tests as ‘root’
  • 34.
    JSSNIFFER Pros q Web-analytic solutionindependent q Browser independent q HTTPs supported out-of-the-box q No dependency on any 3 rd party library Cons q Need to write JavaScript to get the URL from the browser q UI-driver dependent
  • 35.
    @BagmarAnand #waat howcan I contribute?
  • 36.
    PROXY DEBUGGER Pros q Web-analyticsolution independent q Browser independent q UI-driver dependent q HTTPs supported out-of-the-box q No dependency on any 3 rd party library Cons q Making it work for Mobile needs manual setup
  • 37.
  • 38.
    Is that allto Web Analytics?
  • 39.
    The new “kids”in town IoT & Big Data
  • 40.
    Some popular usecases https://www.kaaproject.org/iot-use-cases/
  • 41.
    IOT – INTERNETOF THINGS Opportunity to ¨ Create new value propositions ¨ Be Innovative & Creative
  • 42.
    IOT – HOWTO DELIVER VALUE? ¨ Automate the manual processes ¨ Integrate data capabilities ¨ Collect – integrate from various sources ¨ Repeat collection – automate the collection ¨ Analyze – manual & machine learning ¨ Optimize / Pivot ¨ Repeat
  • 43.
    IOT – CHALLENGES ¨Federated devices, ¨ Different types of networks ¨ Different communication channels ¨ Physical (hardware) & Virtual (software)
  • 44.
    IOT – BIGGERCHALLENGES ¨ Too many devices ¨ Lots of data
  • 45.
    BIG DATA ….IS GETTING BIGGER ¨ Volumes of data generated ¨ A jet engine generates 1TB of data per flight. ¨ A large refinery generates 1TB of raw data per day. ¨ As cars get smarter, the number of sensors is projected to reach as many as 200 per car. ¨ Sensors of all types will generate immense amounts of data. In fact, analysts estimate that by 2020, 40 percent of all data will come from sensors. ¨ IoT leads to massive volumes of data http://www.cisco.com/web/solutions/trends/iot/docs/iot-data-analytics-white-paper
  • 46.
    What does thismean for Testing?
  • 47.
    TESTING OPPORTUNITIES ¨ “Testing”of these new integrations & its value propositions ¨ Performance ¨ Reliability ¨ Security https://www.wired.com/2017/03/medical-devices-next- security-nightmare/?mbid=social_twitter_onsiteshare
  • 48.
    TESTING OPPORTUNITIES ... ¨Build capabilities (and tools) to validate – ¨ Data collection is working well ¨ From all sources ¨ Analyzing data, capturing patterns and trends ¨ Optimize business value ¨ Create new opportunities and value propositions
  • 49.
  • 50.
    RESOURCES ¨ https://github.com/anandbagmar/waat ¨ https://github.com/anandbagmar/waat-ruby ¨http://essenceoftesting.blogspot.in/search/label/waat ¨ http://www.cisco.com/web/solutions/trends/iot/docs/io t-data-analytics-white-paper ¨ http://www.dezyre.com/article/how-big-data-analysis- helped-increase-walmart-s-sales-turnover/109