Downloaded 42 times
Uploaded byRobert Grossman
Big Data, The Community and The Commons (May 12, 2014)
The document discusses the challenges and opportunities presented by big data in biomedical research, highlighting the need for new computational models and infrastructures to manage and analyze massive datasets. It emphasizes the importance of community-driven data commons and the integration of cloud computing for accessing and utilizing biomedical data. Moreover, the document outlines the potential impacts of analyzing large genomic datasets on healthcare and research methodologies.
More Related Content
Similar to Big Data, The Community and The Commons (May 12, 2014)
![[DSC Europe 25] Stefan_Milosevic - AI and digital twins in precision oncology...](https://cdn.slidesharecdn.com/ss_thumbnails/nnmuciuxr2ugh4d9pzkg-1-251126104228-148c7fe8-thumbnail.jpg?width=640&height=640&fit=bounds)
![[DSC Europe 25] Milan Misic - RAG, recommenders and face recognition applica...](https://cdn.slidesharecdn.com/ss_thumbnails/mxe0wzfeqkortbfecopo-8-251128093135-51a402bb-thumbnail.jpg?width=640&height=640&fit=bounds)
Big Data, The Community and The Commons (May 12, 2014)
- 1. Big Data, The Community and The Commons Robert Grossman University of Chicago Open Cloud Consor?um May 12, 2014 TCGA Third Annual Scien?fic Symposium
- 2. Outline 1. Big data and the problems it creates 2. Compu?ng over big data 3. Science clouds 4. Data analysis at scale 5. Genomic clouds 6. How might we organize?
- 3. Four ques?ons and one challenge
- 4. 1. What is the same and what is different about big biomedical data vs big science data and vs big commercial data? 2. What instrument should we use to make discoveries over big biomedical data? 3. Do we need new types of mathema?cal and sta?s?cal models for big biomedical data? 4. How do we organize large biomedical datasets to maximize the discoveries we make and their impact on health care?
- 5. One Million Genome Challenge • Sequencing a million genomes would likely change the way we understand genomic varia?on. • The genomic data for a pa?ent is about 1 TB (including samples from both tumor and normal ?ssue). • One million genomes is about 1000 PB or 1 EB • With compression, it may be about 100 PB • At $1000/genome, the sequencing would cost about $1B • Think of this as one hundred studies with 10,000 pa?ents each over three years.
- 6. Part 1: Biomedical compu?ng is being disrupted by big data Source: Michael S. Lawrence, Petar Stojanov, Paz Polak, et. al., Muta?onal heterogeneity in cancer and the search for new cancer-‐associated genes, Nature 449, pages 214-‐218, 2013.
- 7. Standard Model of Biomedical Compu?ng Public data repositories Private local storage & compute Network download Local data ($1K) Community soeware Soeware & sweat and tears ($100K)
- 8. We have a problem … Image: A large-‐scale sequencing center at the Broad Ins?tute of MIT and Harvard. Growth of data New types of data It takes over three weeks to download the TCGA data at 10 Gbps Analyzing the data is more expensive than producing it
- 9. The Smart Phone is Becoming a Home for Medical & Environmental Sensors Source: LifeWatch V from LifeWatch AG, www.lifewatchv.com.
- 10. Source: Interior of one of Google’s Data Center, www.google.com/about/datacenters/
- 11. Computa?onal Adver?sing and Marke?ng • Computa?onal adver?sing finds the “best match” between a given user in a given context and a suitable adver?sement*. • In 2011, over $100 billion was spent in online adver?sing (eMarketer es?mates) • Contexts include: – Web search context (SERP adver?sing) – Web page content context (content match adver?sing and banners) – Social media context – Mobile context *Andrei Broder and Vanja Josifovski, Introduc?on to Computa?onal Adver?sing.
- 12. Why Do We Care? • A modern adver?sing analy?c planorm: – Will build behavioral profiles on 100 million plus individuals – Use full sta?s?cal models (not rules) for targe?ng – Re-‐analyze all of the data each night – Serve 10,000’s of ads per second using sta?s?cal models – Respond < 100 ms (with analy?cs < 10 ms) – Use real ?me geoloca?on data – Do analy?cs at machine speed – Be driven by an analyst with only modest training • More than pay for the data centers we just saw.
- 13. New Model of Biomedical Compu?ng Public data repositories Private storage & compute at medical research centers Community soeware Compute & storage Community resources
- 14. The Tragedy of the Commons Source: Garrep Hardin, The Tragedy of the Commons, Science, Volume 162, Number 3859, pages 1243-‐1248, 13 December 1968. Individuals when they act independently following their self interests can deplete a deplete a common resource, contrary to a whole group's long-‐term best interests.
- 15. 1. Create community commons of biomedical data. 2. Use a cloud compu?ng and automa?on to manage the commons and to compute over it. 3. Interoperate the data commons.
- 16. 2005 -‐ 2015 Bioinforma)cs tools & their integra)on. Examples: Galaxy, GenomeSpace, workflow systems, portals, etc. 2010 -‐ 2020 ??? 2015 -‐ 2025 ???
- 17. Part 2 Data Center Scale Compu?ng (aka “Cloud Compu?ng”) Source: Interior of one of Google’s Data Center, www.google.com/about/datacenters/
- 18. What instrument do we use to make big data discoveries in cancer genomics and big data biology? How do we build a “datascope?”
- 19. Source: Luiz André Barroso, Jimmy Clidaras and Urs Hölzle, The Datacenter as a Computer, Morgan & Claypool Publishers, Second Edi?on, 2013, www.morganclaypool.com/doi/pdf/10.2200/S00516ED2V01Y201306CAC024
- 20. Self Service Scale Soeware stack that scales to a data center Forgot cloud compu?ng. Focus on data centers & the soeware they run. Use of automa?on
- 21. Sevng up and opera?ng large scale efficient, secure and compliant racks of compu?ng infrastructure is out of reach for most labs, but essen?al for the community. This is not a cloud…
- 22. Commercial Cloud Service Provider (CSP) 15 MW Data Center 100,000 servers 1 PB DRAM 100’s of PB of disk Automa?c provisioning and infrastructure management Monitoring, network security and forensics Accoun?ng and billing Customer Facing Portal Data center network ~1 Tbps egress bandwidth 25 operators for 15 MW Commercial Cloud
- 23.
- 24. Open Science Data Cloud (Home of Bionimbus) 6 PB (OpenStack, GlusterFS, Hadoop) Infrastructure automa?on & management (Yates) Compliance, & security (OCM) Accoun?ng & billing Customer Facing Portal (Tukey) Data center network ~10-‐100 Gbps bandwidth 6 engineers to operate 0.5 MW Science Cloud Science Cloud SW & Services
- 25. Why not just use (only) Amazon Web Services (AWS)? Community science and biomedical clouds • Scale / capacity • Simplicity of a credit card • Wide variety of offerings. vs. It is s?ll essen?al to interoperate with CSP whenever possible by compliance and security policies. Commercial Cloud Service Providers (CSP) • Lower cost (at medium scale) • We can build specialized infrastructure for science. • We can build specialized infrastructure for security & compliance. • The data is too important to trust exclusively with a commercial provider.
- 26. Cost of a medium private/community cloud vs large public cloud. Source: Allison P. Heath, Maphew Greenway, Raymond Powell, Jonathan Spring, Rafael Suarez, David Hanley, Chai Bandlamudi, Megan McNerney, Kevin White and Robert L Grossman, Bionimbus: A Cloud for Managing, Analyzing and Sharing Large Genomics Datasets, Journal of the American Medical Informa?cs Associa?on, 2014. PB Cost (thousands of $)
- 27. Reliability over Commodity Compu?ng Components Is Difficult as is Data Locality • Hadoop enables reliable computa?on over thousands of low costs, unreliable compu?ng nodes. • Hadoop efficiently computes over the data instead of moving the data. • The programming model of Hadoop (MapReduce) is in prac?ce more efficient in terms of soeware development than the programming tradi?onally used by high performance compu?ng (message passing) (but usually does not fully u?lize the underlying hardware)
- 28. The Tail at Scale, Jeffrey Dean, Luiz André Barroso Communica?ons of the ACM, Volume 56 Number 2, Pages 74-‐80 Latency is Difficult
- 29. Call an algorithm and compu?ng infrastructure is “big-‐data scalable” if adding a rack of data (and corresponding processors) does not increase the ?me required to complete the computa?on but enables the computa?on to run on a rack more of data. Most of our community’s algorithms today fail this test.
- 30. 2005 -‐ 2015 Bioinforma)cs tools & their integra)on. Examples: Galaxy, GenomeSpace, workflow systems, portals, etc. 2010 -‐ 2020 Data center scale science. Examples: Bionimbus/OSDC, CG Hub, Cancer Collaboratory, GenomeBridge, etc. 2015 -‐ 2025 ???
- 31. Part 3 Science Clouds Source: Lincoln Stein
- 32. Discipline Dura)on Size # Devices HEP -‐ LHC 10 years 15 PB/year* One Astronomy -‐ LSST 10 years 12 PB/year** One Genomics -‐ NGS 2-‐4 years 0.4 TB/genome 1000’s Genomics as a Big Data Science *At full capacity, the Large Hadron Collider (LHC), the world's largest par?cle accelerator, is expected to produce more than 15 million Gigabytes of data each year. … This ambi?ous project connects and combines the IT power of more than 140 computer centres in 33 countries. Source: hpp://press.web.cern.ch/public/en/Spotlight/SpotlightGrid_081008-‐en.html **As it carries out its 10-‐year survey, LSST will produce over 15 terabytes of raw astronomical data each night (30 terabytes processed), resul?ng in a database catalog of 22 petabytes and an image archive of 100 petabytes. Source: hpp://www.lsst.org/ News/enews/teragrid-‐1004.html
- 33. vs Source: A picture of Cern’s Large Hadron Collider (LHC). The LHC took about a decade to construct, and cost about $4.75 billion. Source of picture: Conrad Melvin, Crea?ve Commons BY-‐SA 2.0, www.flickr.com/photos/58220828@N07/5350788732 • Ten years and $10B • No business value • Big data culture • Liple compliance • Five years and $1M • Business value • Culture of small data • Compliance Common compu?ng, storage & transport infrastructure
- 34. UDT Tuning: Buffers and CPUs Configuration Change Observed Transfer Rate Time to Transfer 1 TB (minutes) UDT and Linux Defaults 1.6 Gbps 85 Setting buffers sizes to 64 MB 3.3 Gbps 41 Improved CPU on sending side with processor affinity 3.7 Gbps 36 Improved CPU on receiving side with processor affinity 4.6 Gbps 29 Improved CPU on both sides with processor affinity on both sides 6.3 Gbps 21 Turn off CPU frequency scaling and set to max clock speed 6.7 Gbps 20
- 35. Science vs Commercial Clouds Science CSP Commercial CSP Perspec?ve Democra?ze access to data. Integrate data to make discoveries. Long term archive. As long as you pay the bill; as long as we keep the our business model. Data Data intensive compu?ng Internet style scale out Flows Large data flows in and out Lots of small web flows Lock in Data and compute portability essen?al Lock in is good
- 36. A Key Ques?on • Is biomedical compu?ng at the scale of a data center important enough for the research community to do or do we only outsource to commercial cloud service providers (certainly we will interoperate with commercial cloud service providers)?
- 37. Open Science Data Cloud
- 38. Part 4: Analyzing Data at the Scale of a Data Center Source: Jon Kleinberg, Cornell University, www.cs.cornell.edu/home/kleinber/networks-‐book/
- 39. experimental science simula?on science data science (big data biology, medicine) 1609 30x 1670 250x 1976 10x-‐100x 2004 10x-‐100x
- 40. Is More Different? Do New Phenomena Emerge at Scale in Biomedical Data?
- 41. Complex models over small data that are highly manual. Simpler models over large data that are highly automated. Small data Medium data GB TB PB W KW MW
- 42. Several ac?ve voxels were discovered in a cluster located within the salmon’s brain cavity (Figure 1, see above). The size of this cluster was 81 mm3 with a cluster-‐level significance of p = 0.001. Due to the coarse resolu?on of the echo-‐planar image acquisi?on and the rela?vely small size of the salmon brain further discrimina?on between brain regions could not be completed. Out of a search volume of 8064 voxels a total of 16 voxels were significant.
- 44. Environmental Factors and Cancer
- 45. Building Models over Big Data • We know about the “unreasonable effec?veness of ensemble models.” Building ensembles of models over computer clusters works well … • … but, how do machine learning algorithms scale to data center scale science? • Ensembles of random trees built from templates appear to work beper than tradi?onal ensembles of classifiers • The challenge is oeen decomposing large heterogeneous datasets into homogeneous components that can be modeled.
- 46. New Ques?ons • How would research be impacted if we could analyze all of the data each evening? • How would health care be impacted if we could analyze of the data each evening?
- 47. 2005 -‐ 2015 Bioinforma)cs tools & their integra)on. Examples: Galaxy, GenomeSpace, workflow systems, portals, etc. 2010 -‐ 2020 Data center scale science. Interoperability and preserva?on/peering/ portability of large biomedical datasets. Examples: Bionimbus/OSDC, CG Hub, Cancer Collaboratory, GenomeBridge, etc. 2015 -‐ 2025 New modeling techniques. The discovery of new & emergent behavior at scale. Examples: What are the founda?ons? Is more different?
- 48. Part 5: Genomic Clouds Cancer Genome Collaboratory (Toronto) Cancer Genomics Hub (Santa Cruz) Bionimbus Protected Data Cloud (Chicago) (Cambridge) (Mountain View)
- 49.
- 50. Analyzing Data From The Cancer Genome Atlas (TCGA) 1. Apply to dbGaP for access to data. 2. Hire staff, set up and operate secure compliant compu?ng environment to mange 10 – 100+ TB of data. 3. Get environment approved by your research center. 4. Setup analysis pipelines. 5. Download data from CG-‐ Hub (takes days to weeks). 6. Begin analysis. Current Prac)ce With Bionimbus Protected Data Cloud (PDC) 1. Apply to dbGaP for access to data. 2. Use your exis?ng NIH grant creden?als to login to the PDC, select the data that you want to analyze, and the pipelines that you want to use. 3. Begin analysis.
- 51. 1. What scale is required for biomedical clouds? 2. What is the design for biomedical clouds? 3. What tools and applica?ons do users need to make discoveries in large amounts of biomedical data? 4. How do different biomedical clouds interoperate?
- 52. 6. How Do We Organize?
- 53. Cyber Condo Model • Research ins?tu?ons today have access to high performance networks – 10G & 100G. • They couldn’t afford access to these networks from commercial providers. • Over a decade ago, they got together to buy and light fiber. • This changed how we do scien?fic research. • Cyber condos interoperate with commercial ISPs
- 54. Science Cloud Condos • Build Science Cloud condo. • To provide a sustainable home for large commons of research data (and an infrastructure to compute over it). • “Tier 1” Science Clouds need to establish peering rela?onships. • And to interoperate with CSPs • The Open Cloud Consor?um is planning to develop a science cloud condo.
- 55. Some Biomedical Data Commons Guidelines for the Next Five Years • There is a societal benefit when biomedical data is also available in data commons operated by the research community (vs sold exclusively as data products by commercial en??es or only offered for download by the USG). • Large data commons providers should peer. • Data commons providers should develop standards for interopera?ng. • Standards should not be developed ahead of open source reference implementa?ons. • We need a period of experimenta?on as we develop the best technology and prac?ces. • The details are hard (consent, scalable APIs, open vs controlled access, sustainability, security, etc.)
- 56. Source: David R. Blair, Christopher S. Lyple, Jonathan M. Mortensen, Charles F. Bearden, Anders Boeck Jensen, Hossein Khiabanian, Rachel Melamed, Raul Rabadan, Elmer V. Bernstam, Søren Brunak, Lars Juhl Jensen, Dan Nicolae, Nigam H. Shah, Robert L. Grossman, Nancy J. Cox, Kevin P. White, Andrey Rzhetsky, A Non-‐Degenerate Code of Deleterious Variants in Mendelian Loci Contributes to Complex Disease Risk, Cell, September, 2013 7. Recap
- 57. 100,000 pa?ents 100 PB 1,000,000 pa?ents 1,000 PB 10,000 pa?ents 10 PB 1000 pa?ents • What are the key common services & APIs? • How do the biomedical commons clouds interoperate? • What is the governance structure? • What is the sustainability model?
- 58. 2005 -‐ 2015 Bioinforma)cs tools & their integra)on. Examples: Galaxy, GenomeSpace, workflow systems, portals, etc. 2010 -‐ 2020 Data center scale science. Interoperability and preserva?on/peering/ portability of large biomedical datasets. Examples: Bionimbus/OSDC, CG Hub, Cancer Collaboratory, GenomeBridge, etc. 2015 -‐ 2025 New modeling techniques. The discovery of new & emergent behavior at scale. Examples: What are the founda?ons? Is more different?
- 59. Thanks To My Colleagues & Collaborators • Kevin White • Nancy Cox • Andrey Rzhetsky • Lincoln Stein • Barbara Stranger
- 60. Thanks to My Lab Allison Heath Ray Powell Jonathan Spring Renuka Ayra David Hanley Maria Paperson Map Greenway Rafael Suarez Stu? Agrawal Sean Sullivan Zhenyu Zhang
- 61. Thanks to the White Lab Jason Grundstad Chaitanya Bandlamidi Jason Pip Megan McNerney
- 62.
- 63. For more informa?on • www.opensciencedatacloud.org • For more informa?on and background, see Robert L. Grossman and Kevin P. White, A Vision for a Biomedical Cloud, Journal of Internal Medicine, Volume 271, Number 2, pages 122-‐130, 2012. • You can find some more informa?on on my blog: rgrossman.com. • My email address is robert.grossman at uchicago dot edu. Center for Research Informatics
- 64. Major funding and support for the Open Science Data Cloud (OSDC) is provided by the Gordon and Bepy Moore Founda?on. This funding is used to support the OSDC-‐Adler, Sullivan and Root facili?es. Addi?onal funding for the OSDC has been provided by the following sponsors: • The Bionimbus Protected Data Cloud is supported in by part by NIH/NCI through NIH/SAIC Contract 13XS021 / HHSN261200800001E. • The OCC-‐Y Hadoop Cluster (approximately 1000 cores and 1 PB of storage) was donated by Yahoo! in 2011. • Cisco provides the OSDC access to the Cisco C-‐Wave, which connects OSDC data centers with 10 Gbps wide area networks. • The OSDC is supported by a 5-‐year (2010-‐2016) PIRE award (OISE – 1129076) to train scien?sts to use the OSDC and to further develop the underlying technology. • OSDC technology for high performance data transport is support in part by NSF Award 1127316. • The StarLight Facility in Chicago enables the OSDC to connect to over 30 high performance research networks around the world at 10 Gbps or higher. • Any opinions, findings, and conclusions or recommenda?ons expressed in this material are those of the author(s) and do not necessarily reflect the views of the Na?onal Science Founda?on, NIH or other funders of this research. The OSDC is managed by the Open Cloud Consor?um, a 501(c)(3) not-‐for-‐profit corpora?on. If you are interested in providing funding or dona?ng equipment or services, please contact us at info@opensciencedatacloud.org.