This experience is optimized for Internet Explorer version 9 and above.

Please upgrade your browser

Video title

Priority Code

Cancel
Big Data: How Data Analytics Is Transforming the World

Big Data: How Data Analytics Is Transforming the World

Professor Tim Chartier Ph.D.
Davidson College
Course No.  1382
Course No.  1382
Share:
Video or Audio?
While this set works well in both audio and video format, one or more of the courses in this set feature graphics to enhance your learning experience, including illustrations, images of people and event, and on-screen text.
Which Format Should I Choose? Video Download Audio Download DVD CD
Watch or listen immediately with FREE streaming
Available on most courses
Stream using apps on your iPad, iPhone, Android, or Kindle Fire
Available on most courses
Stream to your internet connected PC or laptop
Available on most courses
Download files for offline viewing or listening
Receive DVDs or CDs for your library
Play as many times as you want
All formats include Free Streaming
All formats include Free Streaming

Course Overview

About This Course

24 lectures  |  31 minutes per lecture

Data is everywhere, shedding light on all aspects of life. Retailers know what’s selling and who’s buying. Pollsters test opinions on everything from candidates to consumer goods. Doctors follow their patients’ vital signs. Social networks register the interactions of millions. Sensors measure the changing weather. And as athletes play, fans collect exhaustive statistics on their performance.

If something can be measured, then in all likelihood a vast archive of data is already being compiled—and it is growing daily. Often, the data is unprocessed, waiting for someone to analyze it and discover new, valuable knowledge about the world.

View More

Data is everywhere, shedding light on all aspects of life. Retailers know what’s selling and who’s buying. Pollsters test opinions on everything from candidates to consumer goods. Doctors follow their patients’ vital signs. Social networks register the interactions of millions. Sensors measure the changing weather. And as athletes play, fans collect exhaustive statistics on their performance.

If something can be measured, then in all likelihood a vast archive of data is already being compiled—and it is growing daily. Often, the data is unprocessed, waiting for someone to analyze it and discover new, valuable knowledge about the world.

This is the role of data analytics, a powerful set of tools for making sense of datasets of all sizes—from a personal exercise log to the massive collections of “big data” that define our information age. From science to sales, from sociology to sports, data analytics is unraveling the fascinating secrets hidden in numbers, patterns, relationships, and information of every kind.

Consider these examples:

  • Cell phone science: If you are an avid user of your cell phone, try downloading several months of your calling data. You may see daily and long-term patterns in your usage that surprise you. Plus, any changes in your routine, such as a vacation, will show up prominently.
  • Hardball analytics: The book and film Moneyball tell how the Oakland A’s overcame one of the smallest budgets in major league baseball to assemble a division-winning team. The secret? Managers used overlooked data analytics to hire undervalued, high-performing players.
  • Presidential prediction: In the 2012 presidential election, statistician Nate Silver and a few others correctly predicted the winner of all 50 states and the District of Columbia. Here, weighting criteria make it possible to analyze data collected by hundreds of pollsters from thousands of distinct polls.

In our age of accelerating progress in so many fields, it’s easy to lose sight of the underlying innovation that makes this revolution possible. In case after case, the big breakthrough comes from data analytics, the mathematical magic that turns undigested information into life-transforming insights and advances.

Big Data: How Data Analytics Is Transforming the World introduces you to the key concepts, methods, and accomplishments of this versatile approach to problem solving. Taught by Professor Tim Chartier, an award-winning Associate Professor of Mathematics and Computer Science at Davidson College, these 24 half-hour lectures give you the big picture on big data, highlighting the crucial role of data analytics in today’s world and the even greater impact it will have in the future.

A Course for Data Users at All Levels

You need no expertise in mathematics to follow this exciting story. Professor Chartier explains the basic computational techniques used in data analytics, but his focus is on how these ideas are applied and the amazing results they achieve. His wealth of case histories and his many helpful graphics make Big Data both accessible and entertaining. Those who will benefit from his presentation include

  • those in business, government, science, and other endeavors, who want a view into what data analytics can do for them;
  • the intellectually curious, eager to investigate the role of computing and “data scraping” in the modern-day miracles of the information age;
  • math enthusiasts who relish seeing a wide range of mathematical techniques address practical challenges;
  • those considering, or already pursuing, work with data and aspiring to explore the full scope of their remarkable field; and
  • anyone who relies on the Internet, smart phones, social media, or other tools that make them a participant in the data analytics revolution.

Big Data at Work

The volume, velocity, and variety of available data have increased at an astonishing rate during the last twenty years. That is to say, there are vast amounts of stockpiled data, and more is being generated constantly; the speed at which data is used, updated, and overturned in favor of newer data continues to accelerate; and data comes from many different sources and can be put to diverse uses. The miracle of data analytics is that ingenious algorithms are able to process this data deluge, which has been compared to trying to drink from a fire hose of information.

For instance, in just fifteen minutes the number of photos uploaded to Facebook exceeds the total number of photographs stored in the New York Public Library’s photo archives. Yet you can see a picture on your Facebook news feed within seconds after it’s posted. A high-speed computer algorithm allows the flood of imagery to be managed in a way that’s both timely and orderly. Professor Chartier explains how programmers achieve such feats by focusing only on the data that’s crucial to a specific task, while ignoring everything that’s irrelevant.

Big Data takes you behind the scenes to witness many examples of data analysis in action, including the following:

  • Google Flu Trends: Google search queries on flu symptoms have sometimes proved more accurate and up-to-date at plotting the spread of flu than reports issued by doctors and hospitals. Explore the pitfalls and enormous potential of Internet traffic for charting many different trends.
  • Online recommendations: Predictive analytics deals with forecasting the future, a task taken very seriously by companies like Netflix and Amazon that aim to predict what customers want. Learn how Netflix came up with an impressively accurate movie recommendation algorithm.
  • March Madness: A classic exercise in data analytics is predicting the playoff winners of the NCAA basketball tournament, held every March. Follow the system for filling the game brackets, designed by Professor Chartier, and see how it applies to many other problems.

But big data and data analytics can also be a mixed blessing. While the field has revolutionized fraud detection, making many kinds of transactions much more secure, it has the potential to threaten personal privacy in ways that can be hard to spot. In this course, you learn that one of the best defenses for privacy is to know how data is compiled and processed, and which activities are the most compromising.

A Tool for Everyone

Honored as the Mathematical Association of America’s first ever Math Ambassador, Professor Chartier is a champion of the fun, challenge, and breathtaking power of mathematics—qualities that are beautifully illustrated in data analytics.

He especially relishes the links between sports and math. Not only does data analytics give you deep insight into the relative qualities of players, but it can establish a theoretical limit on performance—as when you learn how to estimate the fastest possible time for the 100-meter dash.

Professor Chartier also describes how simple analysis improved his own performance as a swimmer—which illustrates a key point: data analytics can be put to use by anybody for any problem that involves a dataset, no matter what size.

With Big Data, you discover tools that are transforming the world and that you can use to transform your own life. It’s like watching a thrilling spectator sport that invites you to suit up and join the action!

View Less
24 Lectures
  • 1
    Data Analytics—What’s the “Big” Idea?
    Sample the tremendous scope and power of data analytics, which is transforming science, business, medicine, public policy, and many other spheres of modern life. Investigate why this revolution is happening now, and look at some common misconceptions about data analysis. x
  • 2
    Got Data? What Are You Wondering About?
    Data analysis is not just for large organizations and large datasets; it’s also for the average person. Learn how to put data to work in your own life—from charting your cell phone usage to personalizing your medical care or improving your exercise routine. x
  • 3
    A Mindset for Mastering the Data Deluge
    Today’s data users often feel like they’re drinking from a fire hose of information. Investigate strategies that help manage the data deluge, and learn efficient ways to think about data that separate what’s genuinely useful from what can be strategically ignored. x
  • 4
    Looking for Patterns—and Causes
    Humans are experts at pattern recognition, which is a key skill in data analysis. But when are patterns real and when are they imagined? Study some surprising correlations between apparently unrelated phenomena, asking whether there is a cause-and-effect relation or mere coincidence is involved. x
  • 5
    Algorithms—Managing Complexity
    Algorithms—rules to follow for solving problems—are the secret of managing huge datasets. Start by looking at simple algorithms, including an amazingly effective sorting procedure that you can perform by hand. Then see how these concepts apply to more complex problems, such as web search engines. x
  • 6
    The Cycle of Data Management
    Study what happens after you gather data. It must first be stored, then organized, integrated with data from other sources, and analyzed. Now you are ready to act on the information that the data provides. Determine how this cycle works in practice, and uncover some hidden pitfalls. x
  • 7
    Getting Graphic and Seeing the Data
    Graphics have long been a compelling way to present and understand data. Survey some unusually effective graphics from the pre-computer era. Then explore the wealth of graphical tools available today. Graphics can reveal new information, but they can also obscure it when used poorly. x
  • 8
    Preparing Data Is Training for Success
    “Garbage in, garbage out” is a famous expression in computer science, underscoring the importance of starting with reliable data. Learn how data is prepared to remove errors and ambiguities. As an example, see how the US Postal Service perfected machines that can read hastily scribbled addresses. x
  • 9
    How New Statistics Transform Sports
    Follow the saga of the 2002 Oakland A’s, famously depicted in the book and film Moneyball. Thanks to data analytics, the A’s made it to the major league playoffs with a roster of undervalued players. Survey the increasing role of data at all levels of sports competition. x
  • 10
    Political Polls—How Weighted Averaging Wins
    Study the role of big data in predicting election results. Contrast the disastrous 1936 presidential poll by the Literary Digest with today’s impressively accurate aggregators of polls, such as statistician Nate Silver. Analyze what makes aggregation more effective than any single poll. x
  • 11
    When Life Is (Almost) Linear—Regression
    Explore the power of regression analysis for modeling the past and future, focusing on a technique called the linear least squares method. As an example, use data from Olympic gold medal times for the 100-meter dash. Calculate a theoretical fastest possible time for the event. x
  • 12
    Training Computers to Think like Humans
    Delve into the field of artificial intelligence, discovering how computers are programmed to think and make decisions like humans. An automated version of the 20 questions game illustrates how neural networks are the key to machine learning—a technology that is now in widespread use. x
  • 13
    Anomalies and Breaking Trends
    Sometimes it is the odd bit of data—the outlier in a sea of statistics—that is crucial to solving a mystery. See how sophisticated anomaly detection has led to a significant drop in credit card fraud. The same approach helps understand cultural trends that go viral. x
  • 14
    Simulation—Beyond Data, Beyond Equations
    Enter the world of simulation, which allows researchers to model behavior that would otherwise be too dangerous or expensive to study. Investigate the history of the subject and its multiplying applications—from science and engineering to entertainment. x
  • 15
    Overfitting—Too Good to Be Truly Useful
    Learn how to avoid the perils of overfitting, which is when an overly complex model or noisy data leads to flawed conclusions. Explore object lessons in this common pitfall, including an earthquake forecast that was disastrously wrong. x
  • 16
    Bracketology—The Math of March Madness
    Every year, millions of people engage in a hugely popular data exercise called March Madness. See how a mathematical approach called bracketology helps you excel at picking winners in the playoff games of the NCAA basketball tournament. x
  • 17
    Quantifying Quality on the World Wide Web
    Internet searches used to be frustratingly hit-or-miss. See how Google changed that by creating a realistic model of the way web surfers use the Internet. Then look at attempts to hijack search results to improve page rankings and how programmers thwart these tactics. x
  • 18
    Watching Words—Sentiment and Text Analysis
    We are nearing the point where every book ever written is accessible and searchable in digital form—as already exists for the even more voluminous texts from Twitter, Facebook, and other media. Learn how data analysts mine this limitless storehouse of words for new cultural and business insights. x
  • 19
    Data Compression and Recommendation Systems
    Data compression is crucial for storing and transmitting digital images at a fraction of their original size. See how compression also improves online recommendations, as shown by the Netflix million dollar competition, which led to a new algorithm for personalized recommendations. x
  • 20
    Decision Trees—Jump-Start an Analysis
    Probe the power of decision trees by breaking down the demographics of survivors of the Titanic disaster, an analysis that tells the tragic story of events aboard the sinking ship. Then test decision trees in other applications, marveling at their ability to carve quickly through data. x
  • 21
    Clustering—The Many Ways to Create Groups
    Clustering is a powerful way to discover new relationships in data by sorting it into groups, called clusters. Explore this family of techniques by searching for clusters in the Million Song Dataset. Then try other examples that show the exceptional flexibility of clustering. x
  • 22
    Degrees of Separation and Social Networks
    Test the popular theory that six steps, at most, connect you to any person on the planet. Social networks like Facebook provide a wealth of data for quantifying our relative connectedness. See how graph theory helps you to visualize any linked phenomena. x
  • 23
    Challenges of Privacy and Security
    Big data can be a big threat to privacy. Learn how surveillance cameras, smart phones, and Internet use provide a wealth of opportunities for tracking specific individuals. Examine privacy issues raised by corporate and government activity, and review what you can do to lead a more secure life. x
  • 24
    Getting Analytical about the Future
    Focus on a branch of data analytics called predictive analytics, concerned with predicting the future. Imagine attending such a conference years from now. What can you expect? Answer the question with the tools you have learned in the course, and come up with some surprising forecasts! x

Lecture Titles

Clone Content from Your Professor tab

Your professor

Tim  Chartier
Ph.D. Tim Chartier
Davidson College

Dr. Tim Chartier is an Associate Professor of Mathematics and Computer Science at Davidson College. He holds a B.S. in Applied Mathematics and an M.S. in Computational Mathematics, both from Western Michigan University. He received his Ph.D. in Applied Mathematics from the University of Colorado Boulder.

Professor Chartier is a recipient of a national teaching award from the Mathematical Association of America (MAA). He is the author of Math Bytes: Google Bombs, Chocolate-Covered Pi, and Other Cool Bits in Computing and coauthor (with Anne Greenbaum) of Numerical Methods: Design, Analysis, and Computer Implementation of Algorithms. As a researcher, he has worked with both Lawrence Livermore National Laboratory and Los Alamos National Laboratory, and his research was recognized with an Alfred P. Sloan Research Fellowship.

Dr. Chartier is a member and past chairperson of the Advisory Council for the National Museum of Mathematics, and was named the first Math Ambassador of the Mathematical Association of America. He fields mathematical questions for ESPN’s Sport Science program and has served as a resource for the CBS Evening News, National Public Radio, The New York Times, and other major news outlets.

View More information About This Professor

Reviews

Rated 4 out of 5 by 6 reviewers.
Rated 3 out of 5 by Big Data, Slim Content Video Review: The Great Courses markets this as: "A Course for Data Users at All Levels". Unfortunately, that isn't the case. The course is listed under topic "Science and Mathematics" and sub-topic "Mathematics". Yet there is very little mathematics in this course. In fact, it is as if the instructor is purposely trying to avoid presenting too much mathematics on the assumption that the audience can't handle it. I suspect that he is being coached by The Great Courses in this regard as he seems to struggle at times to hold back an urge to present more depth in his presentation. I find this disappointing as it was clear from feedback in a customer survey that customers wanted more depth in TGC's science and mathematics courses. Anyone with any significant background in the sciences, mathematics, or computer science is already likely a user of data analytics at or near the level of this course. The first dozen or so lectures of this 24 lecture course cover some very basic concepts devoting a long time to their explanation. For example, and entire lecture is spent to explain that "correlation does not equal causation" and 20mins of another is used to explain "Garbage In Equals Garbage Out". Not that these concepts aren't important, but clearly the time to explain these rather simple and obvious concepts could be compressed. The content of the course as it stands now, could easily be compressed into 18 or fewer lectures. The real meat of this course finally comes through in the last nine lectures. Here is where Dr. Chartier explains (conceptually) how various data algorithms work such as: Google's search engine link analysis, text searching and analysis, data compression and recommendation analyses, decision trees, clustering/ranking, and degrees of separation/social networks. The conceptual explanations are great, but Dr. Chartier misses an opportunity to put more meat on the bone by not explaining the mathematics and statistical analyses behind these further. My recommendation to make this a truly great course would be to shorten the first half of the course considerably and add more depth to the algorithms and math in the later half. Dr. Chartier uses sports examples such as Oakland A's "money-ball", NCAA Final Four basketball "bracketology", and Davidson basketball team performance analyses. Apparently this is Dr. Chartier's area of research. I'm a sports fan, and these are interesting, but they don't present a fair representation of the real world problems that data analytics is being used to solve daily. For awhile, I thought that this was course was an adaptation of a "Math for Jocks" course that enabled the NCAA Division I athletes at Davidson to satisfy their mathematics requirement without having to really take math. But since no such course exists on the Davidson website, I suspect this emphasis is more a reflection of Dr. Chartier's own research area. As someone who spent an entire career in the electronics and computer industry, I think there are much better examples of the utility of data analytics for "Transforming the World" than sports statistics. To his credit, Dr. Chartier does mention several of these: Political Polling, the Human Genome Project, and a new Materials Genome Initiative, but with far less emphasis. Of course, he also shows several examples using social media (Twitter and Facebook) which have become business and cultural tools. A "March Mathness" appendix is included in the Course Guidebook to show the math behind "Bracketology" of NCAA basketball's March Madness. The math depth here is a welcome addition but similar mathematics should be included in the lectures and/or course guidebook for other algorithms introduced in the course.. Despite some of the above comments, I found Dr. Chartier to be a capable presenter who obviously has a passion for the subject. He does tend to read from the teleprompter but does give appropriate emphasis and inflection to his points. This course does seem to be geared toward "Liberal Arts" majors. I did major in physics, but I have a liberal arts degree and I am a trustee of a Liberal Arts college (from which TGC has drawn instructors).. Any definition of Liberal Arts instruction includes teaching mathematics and science. So why the trepidation on TGC's part to have their instructors deliver mathematical substance in a math/science course like this one? Lifelong learners who chose to take a course on "Big Data" can handle more math. I have taken several TGC math/science courses and this one seems to be "dumbed down" more than most. This is a pity given that this course was produced in 2014 following lots of customer survey feedback for more rigor in science and math courses. A good example of a TGC course which includes science and math appropriate for "Users at All Levels" is Mark Whittle's Cosmology course. Dr. Whittle gives a heads up before lectures that are more mathematically challenging and includes a detailed guide to the mathematics in the course guidebook. I would have liked to see this Big Data course more along that model. Out of more than 40 TGC courses in history, philosophy, science, social science, etc, I have taken, this is the first one that I cannot say I would recommend to a friend as most of my friends would understand much of this course's content already. Any of the people I know who have an educational background (or are getting one) or work in physical sciences, biological sciences, engineering, economics, accounting or finance would not find much new in the content of this course until perhaps the last 9 or so lectures. However, folks whose background is in marketing, political science, journalism, or humanities may find much more of this course new to them and would find it worthwhile. I guess I could also recommend this course to folks who are sports gamblers involved in things like Fantasy Football or March Madness office pools (but don't expect to get an edge). September 15, 2014
Rated 2 out of 5 by didn't learn much Review of Prof. Chartier's 'Big Data (2014)'. I have a lot of background in math and computer science and 20+ years of work experience in the field. I seldom order science courses as I figure I got my fill of those in school. (Normally I order history courses and I've really enjoyed most of the 20+ history series I've watched so far.) I decided to try this Big Data course however because it had a lot of new-sounding topics of interest that I did not take in school. I thought a professor might provide deeper insight than the various online articles or blogs I'd read on these subjects. Unfortunately I didn't find this to be the case. The series got off to an irritating start, telling me about the terabytes, petabytes and exabytes of data out there. There were weak examples such as a study of someone's cell phone data and finding the lack of calls from 3am-10am consistent with when he slept. In Lecture 3, I started fast-forwarding, something I'd never done before in a TTC course. I ended up skipping over a lot of Lectures 3 through 8, which seemed to cover topics I already knew, such as quadratic running time and database keys. I found the discussion of baseball analytics to be off the mark, overly emphasizing the "Pythagorean expectation formula" as a key to the success of the 2002 Oakland A's. The prof often repeated claims from industry uncritically, such as that neural networks have been used "to time when to buy and sell stocks" (quote from page 77 of the guidebook). The explanation of neural networks was not sufficient to understand why they might work. To say something positive, I thought the lecture on overfitting was the best one, with a good example of how it may have led to underestimating the likelihood of the 9.0 earthquake that caused the Fukushima nuclear disaster. Overall, if you already have some familiarity with a lot of the topics, I don't think you will get much out of this series. August 30, 2014
Rated 4 out of 5 by Left wanting a bit more This course is not intended to present detailed 'how to' algorithms to technically savvy students. The majority of the course can be appreciated by many high school students. There are some discussions using matrix algebra in a few lessons, but it is not enough to discourage those with limited math backgrounds. Here is my summary of the 24 lessons: Pros - The professor is a very smooth and engaging presenter. His skills in data analysis/manipulation are obvious. I suspect his more rigorous #i.e., hands-on applications# college course would be quite enjoyable for a retired technical person such as myself. These 24 lessons get progressively more interesting, particularly the last 10 or so. For example, I found the lessons on political polling, data compression, degrees of separation and privacy to certainly be worthwhile. Cons - Perhaps half of the course material was quite generic, especially the beginning lessons. It could have been condensed without compromising much of the groundwork he intended. If the course content was moved a bit more toward a college course, it would probably be somewhat more satisfying. #I am presuming that many of the technical Great Course purchases are made by people with BS degrees, at the minimum??# I will be interested to see what future courses are offered by this excellent instructor. August 30, 2014
Rated 5 out of 5 by Quite informative I have a professional interest in learning more about the analytics and statistics behind Big Data, which is a very hot topic in high tech today. I found Prof. Chartier's course quite informative and useful, especially his use of numerous accessible real-world examples in explaining various analyses. If one is experienced in computer science, you're likely to find several of the first 8 lectures to be very elementary but the rest quite valuable. August 25, 2014
2 next>>

Questions & Answers

Customers Who Bought This Course Also Bought

Some courses include Free digital streaming.

Enjoy instantly on your computer, laptop, tablet or smartphone.