3

Excuse the potentially dumb question, I've only just started learning about data science.

How do we find out how much data we should collect before using it to start making decisions?

Is there a way of knowing if we should wait to collect more data? Or if the quantity of data can lead to meaningful results?

Background: we have a mobile app and some stats about it, shown below. It's early days and the data set is small. We want to start using this data to make better decisions about improving the product.

enter image description here

tobinharris
  • 131
  • 1
  • Could you tell us more about the data and give an example of what kind of decisions you want to make using it? If I understand this data right, you have 28 users who opened the app at least once, 21 of 28 who imported a contact, 18 of 28 accessed their profile and so on... is that correct? And what decisions would you like to make from that? – stmax Feb 23 '17 at 11:26
  • Thanks @stmax. We wanted to know if we can make decisions from this. For example, the data suggests that there's a large drop-off before Rate Contact, which means they're abandoning the app before getting much value from it. It could suggest usability problems, or lack of understanding etc. I guess with such a small sample set of data this could be random chance, we just need more samples? – tobinharris Feb 26 '17 at 13:14

1 Answers1

1

Short answer is: as much as possible.

The more data you have, the more likely you can make correct inference. In the worse case, you could simply discard anything that you don't like. It's like money, the more the better - you always have an option to spend or use less.

Statistics could be doggy without sufficient sample size, your statistical power would be affected and any model you have could be biased.

Having said that, you don't want everything from your user. Use your common sense, and ask yourself what exactly you want to do.

Based only on the data in your question, I'd say: (I could be wrong because I don't understand where you got the data)

  • How many total users you have that didn't drop out? If your total user base is like 10K, 28 users dropping out is not a problem. But if you only have 30 users...
  • Your sample size is small. Is that one-day installation? In any case, 28 users for a mobile app is a tiny data set. You may want to focus more on marketing.

EDIT:

How many respondent you need is hard to tell. Apple probably needs better data quality than you. If you'd like to do it statistically, you may calculate the minimum sample size and it's related to statistical power.

SmallChess
  • 3,540
  • 2
  • 18
  • 30
  • Many thanks for this.

    Yes, we've only marketed to a small number of people. But wondered how many people we'd need to analyse before being able to get reliable insights.

    I've looked at sample size calculations but it's not quite making sense. I'm not sure if the population is our entire market size, or our 28 people! Or even if it's the right tool for working this out.

    If we did some marketing, would we need 100 users or 1000 to get meaningful results? (appreciate the more the better)

     – tobinharris Feb 23 '17 at 08:34