2

I am interested in computing the margin-of-error for a metric computed on a random sample. The underlying distribution (finite) from which the random sampling is done is not normal (its extremely skewed; something like a zipf/power-law curve).

Planning on estimating the margin-of-error at 95% confidence by Monte Carlo Simulation:

  • Repeat large number of times:
    • Sample $n$ data points (without replacement) out of $N$
    • Compute metric on $n$ points and store it somewhere
  • Use the large number of metrics to estimate margin of error

Question1 :: I am not sure about the sampling without replacement part (should it be with replacement)?

Question2 :: How to compute the margin-of-error? (I can get the histogram, but then what?)

Edit: The metric I am concerned with is "defect rate" in %tage. The reason for sampling is to lower the cost of evaluation of item.

BiGYaN
  • 121
  • Are you computing a metric on a tuple of $n$ points, or on $n$ individual points and then averaging? In the former case, it's you who decides whether the tuple should allow replacement or not, since you define the metric; in the latter case, your approach of bunching the sample into groups of $n$ is a distraction and you should just take a single very big sample. Also you should say something about whether your samples are correlated (which they usually are in Monte Carlo simulations). – joriki Jan 17 '13 at 09:41

0 Answers0