Video Transcript: Review of Data - Part 2
Welcome. In this lecture, we're going to continue on with the review of all that we've covered by now looking at a complete example with our bike data. Let's imagine you've been hired as a data analyst by CLI Bike Sharing Incorporated. We're going to refer to them as the customer from now on, and your task is now to provide key insights and recommendations about their business. Well, the customer is a bike rental business operating in the Washington, DC, and Arlington, Virginia area. And here's their mission statement: Bike-sharing systems are a new generation of traditional bike rentals, where the whole process, from membership to rental and return, has become completely automatic through these systems. A user can easily rent a bike from a particular position and return it back to another position. Currently, there are over 500 bike sharing programs around the world, which are composed of over 500,000 bicycles. Today, there exists great interest in these systems due to their important role in traffic, environment, and health issues. Now, compared to other transfer services, such as a bus or a subway, the duration of travel from the departure to the arrival position is explicitly recorded in these systems. Now, this feature turns bike sharing systems into a virtual sensor network that can be used for sensing mobility inside of the city. Now the company has provided detailed rental and environmental data for a two year period, specifically 2011 and 2012 The data are based on their Washington DC operations and cover measures such as daily rental counts, precipitation, day of week, season, and other variables, which might have a potential impact on rental behavior. So, here are some key goals of analysis. Specifically, part one of the analysis. Our goal is to describe the key statistical measures of registered and casual users, then we're asked to identify any extreme observations in the counts of registered and casual users. Do they tend to appear on certain days, maybe certain seasons? Then we're going to describe any changes in registered users from 2011 to 2012 really, whether or not we see any improvements, and if these improvements are evident across all months, and are there any months that stand out in terms of over or under performance? These are the questions we're going to answer in this lecture. In the next lecture, we're going to do part two of the analysis, where we identify key probabilities around these customers, namely the probability a customer is a registered user given the season is fall, as well as a probability a customer is a registered user given the season is summer. Then we're going to build interval estimates for those probabilities. Now the customer's marketing division also has preconceived notions on the average number of total users for the 2012 seasons, as follows: to help develop their marketing budgets, they think that the average number of daily users, daily total users in the summer is no more than 6500 while the average number of daily total users in the fall is no less than 6000 we're going to validate those above claims using statistical hypothesis testing. All right, so that lays out the premise of what we've been charged with doing with the data set that we've been given. Let's jump into part
one of the analysis. Let's describe the key statistical measures of registered and casual users. A great way of being able to do this is looking at a five number summary. Remember, the five number summary is basically the summary of a boxplot. I've also added in the mean here as well, just to give us an idea of that, but we can look at registered users here on the left hand side and casual users on the right hand side, so on the left hand side we can see that when it comes to daily total users we have everything in terms of 20 all the way up to 6946 for registered users the median value of registered users across our whole data set on a given day is 3662 with the inner quartile range being 2493 all the way up to 4790 Those two numbers define the middle 50% of our data. Now we can also see that the mean and the median are approximately equal to each other. The mean registered users on any given day, the average is 3,656.2 Notice how this. Is much bigger than the casual users that you see over here on the right hand side. On the right hand side, we can see our worst day in terms of number of casual users was 2, where our best day was 3410 However, that appears to be a notion of an outlier, the interquartile range for the number of casual users is anywhere from 315 to 1097 on a given day, with the median number of casual users on any given day being 713. Because of those outliers, with the average is a little bit taller than the median, we have 848.2 users per day on average in terms of casual users, so again, if we wanted to look at things in terms of center or typical, we can again summarize those numbers, right? We have the mean and the median that we talked about previously on the last slide for both registered users, as well as casual users. However, if we want to look at variability, we can look at range standard deviation, as well as the interquartile range. So, I mentioned the interquartile range of both registered and casual users based on the boxplots, but let's take a look at total range. The total range for registered users is 6926 Our maximum value of 6946 minus our minimum value of 20. That's a wide range, at least almost double what the casual users range is. The casual users range is 3410 minus two, which is 3408 Now, although the range for registered users is bigger than the range for casual users, it looks like the standard deviation is actually smaller. So, the standard deviation of 686.6 users per day is actually almost half that of the standard deviation of 1,560.3 users per day. This is most likely because of the fact that although there is a wide range for registered users, it is a tighter distribution overall in terms of where a majority of the data is, however, those casual users. Remember, we have some outliers, and those outliers can drastically change what we're looking at around the mean, so that's why the standard deviation is probably much higher. Awesome. So we just got done describing key statistical measures of both registered and casual users. Now let's identify any extreme observations in the counts of registered and casual users. Do they tend to appear on certain days, certain seasons? That's what we're looking for. So let's look at some of these extreme counts. When looking at some of these extreme
counts again, we can see, especially on the low side, registered users as well as casual users, we had a very single bad day. 20 registered users total, and 2 casual users total. Those are rather extreme when it comes to the upper side of things. Again, considering the average of registered users is in the middle 3000s having almost 7000 registered users in a single day seem to be rather high. Same for casual users, considering our average was around 800 Having a single day of casual users above 3000 seems rather extreme. So, what we're going to do is, we're going to look at the five highest registered user days, and see if we notice any kind of patterns. So, these are the five highest registered user days. These days all had users above 6844 in terms of registered users. Do you notice any kind of patterns? Take a look over on the left hand side, four out of these five days were Wednesdays, the middle of the week, most likely when these registered users are probably going to work. Any other patterns that you notice, four out of these five days are in the fall, so we have fall on Wednesdays seems to be a very popular day for registered users, but let's look a little bit deeper. Let's take a look at the weather overall. Wow, four out of five days were clear or partly cloudy, with temperatures ranging from 60 degrees or 61.1 degrees all the way up to 73.2 degrees. They just seem like nice mild weather days. So we have a work day Wednesday on a seasonal. Nicely weathered day, and that seems to lead to a high number of registered users. Let's see if we see the same kind of patterns for casual users. For casual users, here are the five highest casual user days, with all five of these days having more than 3155 casual users. It looks like we have some changes, though. Take a look at the weekday here. Unlike registered users, which mostly had their big days on Wednesdays, when it comes to casual users, four out of the five highest days were Saturdays, the other being a Sunday. All five of our highest casual user days were weekend days. What else do we see? Three out of the five of them were in the spring, one of them was in the summer, and wow, one of them was actually in the winter, but let's dive in a little bit more again. If we were to look at temperature and weather, we see the same general pattern: clear, partly cloudy, and we see temperatures ranging from 54.6 degrees all the way up to 76 degrees, so weekends with nice mild weather seems to be our biggest set of days to be able to get a high number of casual users. These would make sense now. Let's look at the flip side. Let's look at the five lowest registered user days, we had registered users all the way down to 20, as well as in the four hundreds. Let's see if we see any patterns here. The weekday seems to be a rather mix of everything: Monday, Thursday, Wednesday, Sunday, Wednesday, but look at the season, winter. Hmm, look at the weather type: rain, snow, clear rain, snow, clear rain, snow, as well as look at some of those temperatures. Whoo 34.1 degrees all the way up to 54.8 degrees. You know, I think again, weather plays a big role in the number of registered users. So, yes, we do have weekdays here, but inside of the winter, especially when it's raining or snowing, it doesn't look
like we're going to see a lot of registered users using our bike rental service. Let's flip over to casual. Here are the five lowest casual user days: 2, 9, 9, 15, 25, In terms of number of casual users, and we see again a very similar pattern to what we saw for registered user days. The day of the week doesn't matter as much, it's a hodgepodge of all of them, but the season winter weather type rainy or snowy or misty, and the temperature rather cold. So, when it comes to high usage for both casual and registered, there were some slightly different patterns. Nice weather, but for high usage, registered users during the weekdays, casual users on the weekends. However, when it comes to low usage for either registered or casual, it's usually revolving around the idea of weather. Cold days, rainy days, snowy days, typically in the winter months, we're not going to see a lot of registered or casual users. Awesome, so we've just gotten done identifying extreme observations. Let's now describe any changes in the registered users that we've seen from 2011 through 2012 Again, the best way to do this is to visualize, so we can visualize how it looks in terms of daily number of registered users over time. So, when looking at registered users over time, we start off rather low early on in 2011 but as more people find out about our company and our bike rental service, we can see that registered users seems to be trending upwards over time. As time goes on, we're getting more and more and more registered users. Now, if we want, we can visualize this both for 2011 and 2012 So, I've highlighted both of those years here for you. When looking at this, it looks like 2012 has been a better year than 2011 which again shouldn't be too surprising if we're going up over time. 2012 may represent again that we're doing better as time's going on. However, both of the years have a very similar pattern, kind of like what we just saw in the winter months, we see low usage as compared to the summer, spring, and fall months. However, when looking here at the end, we see that we're sort of trending downwards here at the very, very end, so maybe it's something we should look out for to make sure we're still trending upwards in 2013, 2014 and so on, but at least, like, in that, at least it looks like in 2011, 2012 we're trending in the right direction. Now, let's look at the average registered users year over year. So, essentially, what we're trying to do is compare the month of the year, January, February, March, and so on, that's the x axis to the registered users per day. Now, when looking at the average registered users per day, year over year, we're breaking this down by 2011 and 2012 so we can see that again in all of the months 2011 was lower on average than 2012 That's wonderful, so again we can see that we've made strides in 2012 and increased our business, kind of like we saw with the line graph before, however, I want you to look at a couple of different things. Take a look at January, February, March, and April, as well as August, September, and October. Notice how 2012 seems to be much higher than 2011 in these months, where it looks like we're making big strides. However, in December, maybe not as many big strides as we had seen, so although yes, 2012 is better, 2011 is not
as far below 2012 in December. Again, we kind of saw this previously with the line graph, right? It looked like at the very end of 2012 we did taper off. Now that could be completely due to weather, and we had a weather event at the end of 2012 and that could lead to why we do not have as big a gains in 2012 for December that we did in some of the other months. However, overall it looks like things are doing well. We're trending upwards, as well as noticing that our months are making significant gains year over year. Wonderful. Well, now we've covered part one of the analysis. In our next lecture, we'll cover part two, but that is the end of this lecture, and I look forward to seeing you in the next one.