2

Now I am writing my thesis The topic is about error analysis on turbocharger test bench. In order to estimate uncertainties at test bench first of all I used Monte Carlo Method. Now I have to validate my results. I do not want to use Gauss Method. Do you have any idea how can I validate my results? Is adaptive Monte carlo method such a method to validate ? Thanks a lot.

Scientifica
  • 8,781
cemik88
  • 21
  • 1
    There is not a single Monte-Carlo method, Monte-Carlo refers to a very broad class of methods that deal with random numbers to compute stuff. I would add more details if I were you. – Winther Oct 03 '15 at 21:22
  • And shouldn't your advisor be able to help with stuff like this? – pjs36 Oct 03 '15 at 21:55
  • ı used standard monte carlo method with 10^6 numbers.My advisor does not help me because he has not any idea about topic....For example if use adaptive monte carlo method does it mean that ı could validate my results ? – cemik88 Oct 03 '15 at 22:21
  • First of all, I would plot my results and see what they look like - Gaussian, Poisson, Weibull, ... – marty cohen Oct 03 '15 at 23:09
  • we assume that all our initial inputs like temperature pressure etc are gaussian.I calculated uncertainties with monte carlo.Question is how can i validate monte carlo method? – cemik88 Oct 03 '15 at 23:29
  • Are you Turkish? ('Cause "ı" is a Turkish letter.) – Akiva Weinberger Oct 04 '15 at 03:10

1 Answers1

2

The comments are correct that there are various paths and that it would be helpful if you could give more information. However, here are some generic investigations that I have found useful. They might not be the end of your 'validation', but they may raise issues that point towards the end.

(1) If you have $m = 10^6$ independent iterations in a Monte Carlo simulation of a mean or proportion, then it is reasonable to assume that mean or proportion is nearly normal. So find the relevant SD and use $2SD/\sqrt{m}$ to give a serviceable estimate of the margin simulation error.

(2) If you have questions about independence (as in Markov Chain Monte Carlo, for example) then make an ACF plot for lags out to about 40, and see how long it takes for dependence to decay. Then 'thin' the results; for instance if ACF for lag 10 and beyond is essentially 0, then use every 10th observation. Then adjust m in step (1) accordingly.

(3) If you have questions about the stability of the simulation, then look at 'traces' of running averages (cumulative sums up to n divided by n) against n. They may be very rough-looking as the simulation begins, but they should be quite smooth near the end.

(4) In the case of dependence there is always the risk that the simulation is 'getting stuck' for periods of time (or forever). A 'history plot' of the observations against time will help you to see any serious problems along these lines.

(5) If you are generating several quantities that ought to be stochastically or functionally independent, make scatter plots of pairs of such quantities (along with Pearson and Spearman correlations) to see if there are unexpected relationships. If there are many such quantities, you might try making 'matrix plots' of 'pairs' of variables.

(6) Make histograms of individual simulated quantities. Don't just look at means (or medians) and standard deviations (or interquartile ranges). A histogram of a random quantity should have an 'envelope' or 'density estimator' that is nearly the same as the distribution. If you see unexpected evidence of strong multiple modes, you should try to figure out why.

(7) Sometimes an easily computed by-product of a simulation is a quantity that is has a known value or distribution. Compute them even if they are not part of the reason for doing the simulation. If they do not have the anticipated values or approximate the anticipated distributions, that is a sign of trouble. It is usually better to compare empirical cumulative distribution functions (ECDFs) with known CDFs of distributions than to compare histograms with known PDFs (no loss of acuity due to binning with ECDFs).

I doubt that all of these suggestions will be applicable to any one simulation, but give serious consideration to these and other numerical and graphical descriptions of the output of your simulation. You may find unexpected glitches in the simulation program that are not hard to fix. You may also see that everything is working wonderfully well, and then you will feel much better about the usefulness of your simulation.

Most of the words I have put in 'single quotes' have some chance of fetching useful information in an Internet search if you need further information. Also, when you know a little more about some of these methods and about how your simulation works, you might try posting more specific and technical follow-up questions on our sister 'statistics' site.

BruceET
  • 51,500