Expected accuracy from Monte Carlo simulations?

[Three]

This is good. Who would have thought I will be interested in a disadvantage forum discussion

iCountNTrack is another of the very experienced BJ software poster's on this forum. I hope you show both iCount and Eric more respect than you have shown other BJ software experts. They all have earned the respect of the forum members here so attacking them doesn't help your credibility here. I am not an expert in BJ software but am in most of the rest of the stuff being discussed. My assessment will rely mostly on how well you work and agree with the experts that I do respect. Maybe some day you will earn the same respect. You haven't done much to help yourself on that front.

Hopefully iCount will weigh in along with Eric. Perhaps you will keep the exchange of ideas on a more civil front.

[tomker]

... You stated your SE calculation in post # 68 of the original and now closed thread. Here is the quote:
...
That is .0009% so I stand corrected. Thanks for pointing that out. Your only SE stated by you is not 0.1% but 0.0009%. I don't think that helped you any bit you asked for it. Nobody believed that SE was right anyway. ...

Looks like I overlooked the square root. If you go back and read the post, you'll see that my experimentally-measured sum of squares was 90095. Plugging this into the equation you gave me:

square root of (((the sum of the squares of the deviations from the mean)/(sample size - 1))/sample size) equals standard error

= sqrt(((90095) / (100000 - 1)) / 100000)
= sqrt(0.9 / 100000)
= sqrt(0.000009)
= 0.003

I hope this sorts things out?

[Norm]

The simple fact is that NYNE's post and Tthree's post were accurate. I think you may have also confused the concepts of difference in percentage with percentage of percentage. Your numbers should not be used.

[tomker]

The simple fact is that NYNE's post and Tthree's post were accurate. I think you may have also confused the concepts of difference in percentage with percentage of percentage. Your numbers should not be used.

I never said that Nyne's post was inaccurate. In fact I think everything he posted is extremely accurate and everything he said matches my experimental results exactly.

Tthree's post was inaccurate because he said I claimed a standard error of 0.1%, which I didn't. He went digging through posts and the number he found didn't match 0.1% at all.

[Norm]

To be fair, I must mention that tomker's posts are now moderated and may take time to appear. See http://www.blackjacktheforum.com/sho...l=1#post140482.

[tomker]

iCountNTrack is another of the very experienced BJ software poster's on this forum. I hope you show both iCount and Eric more respect than you have shown other BJ software experts. They all have earned the respect of the forum members here so attacking them doesn't help your credibility here. ...

I have had nothing but extremely pleasant interactions here with Don Schlesinger and Eric Farmer.

[tomker]

I am clearly unwelcome on this forum considering that my posts are now being moderated/censored and my threads have been moved to the area for "voodoo."

Thus I am "banning" myself from the forum and will not be posting anymore.

If anyone is interested in discussing blackjack programming or has any questions about it that I might be able to answer, I welcome emails at [email protected].

[Three]

In a way I can't blame you but you engineered this situation. You haven't tried to have what I will refer to as adult conversation. You have engaged in a he said she said arguments which move nothing forward. You have taken to personal attacks and paranoia about the whole board being against you. A position usually held by people that can't defend their ideas with cogent thought either because they are out of their element in the discussion or are wrong. It is a shame because I was one of the few that hoped your app could live up to its potential. The experts here seemed to think it was not likely to be able to be accurate enough given your estimation technique. Rather than working with them to improve your app where it could be improved you decided to act like you knew more than the smartest and most experienced in the field.

I gave you a heads up as to exactly who everyone was whose opinions should be treated more respectfully. But you chose rather to act like you could teach them all about the field of study they are experts in and engaged in childish arguments, each of which hurt your reputation and made you look immature. Many of us tried to steer you in a direction that would allow you to build your own reputation here but you balked at every opportunity. And you admitted you didn't understand statistics of BJ AP play. And even worse you didn't seem to want to take a few minutes on the web learning the basics about statistics to understand whether your app had a shot at being accurate. Everyone here has knowledge of both these things as they are necessary to running a sim that you can be sure would get accurate results.

You have insulted almost all on this site. Not by your childish personal attacks but by acting like your results were statistically significant when you really had no idea because you don't understand statistics which is the cornerstone building a statistically accurate estimates of a population. Nyne's post showed that the SE for a 100K Monte Carlo sampling that accurately reflects the total population is 0.00361 or 0.361%:

Specifically for finding the house edge of a game, and using the example of 6 deck, S17, DAS, RSA, LS, 75% pen, flat betting and playing all, the standard deviation is 1.1417. After 100,000 hands, the standard error is 0.00361, or 0.361%. This game has a house edge of 0.29%, so after 100k hands, there is greater than a 20% chance that your sample mean will show that this game has a player edge. If you show a significantly smaller standard error than 0.361% after 100k hands, it is a certainty that you aren't sampling the population effectively. You simply cannot have a sample standard deviation that differs substantially from the population standard deviation unless you failed to select a suitable sample.

The question you should be asking when your apps accuracy is concerned is, Is 0.361% accuracy good enough to give statistically significant results for the use that you are applying the statistics to. That tells you whether the Monte Carlo technique you are employing gives significant results or useless results if the highest degree of accuracy possible with a100K Monte Carlo simulation is achieved by your app. That was the relevant question, not all this he said she said. I will leave it to Norm (or another respected software expert on the forum) to answer that question for everyone as you didn't even seem to understand that was the important statistical question concerning your app. If it isn't statistically significant, your app needs to be modified greatly to get accurate results. If it is, you need to make sure you get to .361% SE for your Monte Carlo simulation. The primitive run of SE that is your only SE estimate in all your threads is (as modified for the radical error) .3% which is about 17% off the maximum accuracy for 100K Monte Carlo simulations used to estimate the whole population for the 6 deck rules that Nyne stated.

We shouldn't have to spoon feed you the relevant statistical questions about your app that must be threshed out to get at whether or not your results are statistically significant. For you to boast of the accuracy of your app you should have already done these statistical computations because they are the only way for you to know one way or another.

[craigh01]

I came across this thread and I have a lot of questions, I hope this is not a bad place to post.

Several times "good RNGs" and "bad RNGs" are mentioned. Where would I find or what is a good random number generator?

I was using a random number generator based on the algorithm in a FORTRAN version published by George Marsaglia and Arif Zaman, Florida State University, the comments say "his random number generator originally appeared in "Toward a Universal Random Number Generator" by George Marsaglia and Arif Zaman. Florida State University Report: FSU-SCRI-87-50 (1987) It was later modified by F. James and published in "A Review of Pseudo-random Number Generators" THIS IS THE BEST KNOWN RANDOM NUMBER GENERATOR AVAILABLE.".

(The "this is the best" comment seems unlikely).

Or what about using "true" random numbers like "random.org" which uses atmospheric noise...

My second question is, I'm seeing comments like "It is a certainty that you aren't sampling the population effectively". I am trying to do a simple Monte Carlo blackjack simulation, and I just randomly generate a shoe of X decks, I don't understand what it's meant my "sampling the population".

Sorry for my newbie questions. I'm really interested in this topic but I guess I need to brush up on my statistics knowledge.

[Norm]

Marsaglia and Zaman are the experts. Where did you see the comment: "It is a certainty that you aren't sampling the population effectively"? If the RNG is good, and you run enough hands, your sampling is fine.

[craigh01]

Marsaglia and Zaman are the experts. Where did you see the comment: "It is a certainty that you aren't sampling the population effectively"? If the RNG is good, and you run enough hands, your sampling is fine.

Thank you! I'm glad I'm on the right track.

In several comments in this thread, they talk about "sampling the population", like for example in the post on 9/1/14 at 5:00 AM by TThree, he says, "f you show a significantly smaller standard error than 0.361% after 100k hands, it is a certainty that you aren't sampling the population effectively.".

So I guess I just wanted to understand what they mean by "sampling the population".

But if all I need to do is have a good RNG, then I'm happy...

[Nyne]

That quote was from my response on the first page of this thread. This is an old thread, and I don't recall the full details of what the OP was doing in his software. I don't recall if he was just using a bad RNG or was choosing a subset of all hands with the relevant total rather than using the set of all possible hands that produce that matchup. I do remember that there were claims about the standard error of his 100k hand sample that were not possible from a truly random sample. In a game like blackjack, it is not usually practical to simulate every single possible hand from every possible ordering of the shoe. The population of datapoints includes all possible hands. The sample is whatever subset of hands you are actually looking at. There are tons of different ways you can select what hands you use for your simulation. You could choose a sample that only includes starting hands that contain an ace, or only even totals, or only look at 2 card totals of 16 while ignoring any multicard 16s. Of course, none of those samples would accurately represent the population, which is what we are trying to do. If you are generating your sample by simulating a random shuffle and dealing the cards normally, then as long as your software uses a good RNG and your shuffle algorithm is correct, you shouldn't have to worry about your sampling. That really only comes into play if your RNG is bad or if you are doing a bad job of forcing certain matchups for whatever reason. Just make sure you simulate enough hands to get the standard error down to an appropriate level for whatever you are trying to calculate.

[Three]

In several comments in this thread, they talk about "sampling the population", like for example in the post on 9/1/14 at 5:00 AM by TThree, he says, "f you show a significantly smaller standard error than 0.361% after 100k hands, it is a certainty that you aren't sampling the population effectively.".

This was a comment on standard error and what it means. It was a long time ago I don't talk about SE much but I will try to explain. Those that deal with it more often can correct me because I am sure I will screw it up.

SE can tell you your sample in insignificant by having too large a standard error. That is not what I am talking about. Having too low a standard error when you know what the standard error should be for a accurate sample of the population shows your sample is most likely not random. There is some sort of sample bias. This is what my comment was referring to I believe. It was a while back and my memory of it is not good. It may be the last time I looked much at SE. No let all those that really use SE a lot tell it better to you.