Expected accuracy from Monte Carlo simulations?

[tomker]

I have written some software that calculates blackjack EVs (via Monte Carlo simulation) to within 0.01 after 100k iterations about 97% of the time.

Some people on this forum doubt that this level of accuracy is possible.

So, if there's anybody on the forum who knows about statistics and probability, can you please help answer a question--what percentage of the time would you expect such a simulation to be accurate to within 0.01, mathematically speaking?

Thanks for any insights!

[tomker]

To provide a little more information, I found this:

http://www.pas.rochester.edu/~stte/p...8/notes-8.html

The first question seems to be a similar problem. This person works out that tossing a fair coin will be within 0.01 (0.49 to 0.51) with 95% probability after 10k iterations.

So if blackjack is similar to tossing a coin, then the level of accuracy I'm seeing with my simulations seems to be very believable.

Of course, blackjack is not the same as tossing a coin, but the only thing I see with these equations that's specific to tossing a coin is the probability distribution function. Does anybody know the probability distribution of successive hands of blackjack?

Or am I completely off track with all of this? I admit ignorance.

[Nyne]

Specifically for finding the house edge of a game, and using the example of 6 deck, S17, DAS, RSA, LS, 75% pen, flat betting and playing all, the standard deviation is 1.1417. After 100,000 hands, the standard error is 0.00361, or 0.361%. This game has a house edge of 0.29%, so after 100k hands, there is greater than a 20% chance that your sample mean will show that this game has a player edge. If you show a significantly smaller standard error than 0.361% after 100k hands, it is a certainty that you aren't sampling the population effectively. You simply cannot have a sample standard deviation that differs substantially from the population standard deviation unless you failed to select a suitable sample.

[tomker]

Specifically for finding the house edge of a game, and using the example of 6 deck, S17, DAS, RSA, LS, 75% pen, flat betting and playing all, the standard deviation is 1.1417. After 100,000 hands, the standard error is 0.00361, or 0.361%. This game has a house edge of 0.29%, so after 100k hands, there is greater than a 20% chance that your sample mean will show that this game has a player edge. If you show a significantly smaller standard error than 0.361% after 100k hands, it is a certainty that you aren't sampling the population effectively. You simply cannot have a sample standard deviation that differs substantially from the population standard deviation unless you failed to select a suitable population.

Thanks Nyne, really appreciate these numbers. In other words, there's a 20% chance that the EV will be >= 0.0029 high (which is what would be necessary to show a player edge), and presumably an equal chance that it will be >= 0.0029 low. So 60% of the time, the EV will be accurate to +- 0.0029.

In a different thread, I ran an experiment and made a graph of my simulation's accuracy after 100k nodes. The graph agrees with this prediction EXACTLY.

Hopefully this will put to rest any claims (made by Norm and others) that my simulation results are invalid because they converge faster than they should, since they apparently converge at exactly the rate they should.

[Norm]

Unbelievable. NYNE's excellent post shows that his sims can show a positive EV in a negative situation, a deadly result, and he claims it proves he is correct. This thread will remain in the Disadvantage Forum.

[tomker]

Unbelievable. NYNE's excellent post shows that his sims can show a positive EV in a negative situation, a deadly result, and he claims it proves he is correct. This thread will remain in the Disadvantage Forum.

It's a simple question about blackjack statistics, Norm.

Surely you can make a contribution other than moving the thread from point A to point B?

[tomker]

Unbelievable. NYNE's excellent post shows that his sims can show a positive EV in a negative situation, a deadly result, and he claims it proves he is correct. This thread will remain in the Disadvantage Forum.

Oh BTW -- just noticed your edit.

"Deadly," huh?

I see you are trying to pivot your argument from "your rate of convergence is impossible, your random number generator is flawed, your results are invalid" to "your level of accuracy can result in 'deadly' mistakes."

Let's just stick to one or the other, no?

[Norm]

1. Your rate of convergence is impossible
2. Your random number generator is flawed
3. Your results are invalid
4. Your level of accuracy can result in 'deadly' mistakes.

ALL of these statements are correct as has been explained in detail by multiple people.

I don't have time for this nonsense.

[Bodarc]

My ignore button works quite well.

[tomker]

1. Your rate of convergence is impossible

...

The question at hand is a simple math question. If your answer doesn't have numbers in it, you know something has gone wrong.

I've posted numbers and Nyne has posted numbers. (And happily our numbers agree.) If you're not going to post numbers I don't see why you're "contributing" to this thread.

... I don't have time for this nonsense. ...

Let's be honest though, does it take a lot of your time to write posts that contain nothing useful?

[Norm]

Yeah, I'm useless around here.

[Three]

You simply cannot have a sample standard deviation that differs substantially from the population standard deviation unless you failed to select a suitable sample.

Tomker needs to reread this part of the Nyne's post because he isn't understanding that his standard error is making his results dangerous to rely on.

If you show a significantly smaller standard error than 0.361% after 100k hands, it is a certainty that you aren't sampling the population effectively.

Tomker said his standard error was .1% in a previous post rather than Nyne's .361% for the sample size of his Monte Carlo simulations. Nyne was showing that your SE should be .361% for your sample technique and that this SE is too high to make results reliable for betting applications. Your SE of .1% showed your sample populations are not indicative of of the sample. And that if they had a SE of .361% the results would be absurdly misleading about half the time.

Not understanding the statistics doesn't instill confidence in the way you interpret them or your procedures.

it is a certainty that you aren't sampling the population effectively.

Above is the results of Nyne's checking of your use of the Monte Carlo simulation. Google bell curve and read about the statistics. To increase certainty you want to tighten the bell curve.

We are used to getting SE down to .01-.03% % for our simulations to be considered reliable. Much above that and graphs of hand EV per true count that should look linear start to waiver and wander. At that point the sample size is too small to rely on the results. Norm and Don have told you this over and over. Nyne did the same with more detail hoping to educate you. Pretty much everyone here has simulation software and knows how to get accurate results and have questioned why results weren't accurate and found out their sample size was too small. You aren't instilling confidence when you make statements about your statistics that everyone here knows are grossly inaccurate.

You have been given a chance to learn and make your program something that has useful and reliable results by some of the best in the field. It seems pretty much everyone here knows more about the statistics of BJ than you do. That is not surprising as playing a winning game is dependent on understanding the statistics and applying it to the game. Some simply use software as a crutch and don't have a great grasp of statistics. They must learn enough about SE and other metrics to get useful results from their software. I suggest you start to learn how to apply the statistics of BJ in a way that will produce meaningful results. Most of the players that don't learn this end up going bust because counting without the knowledge of statistics of the game can give yourself just enough knowledge to hang yourself (lose your entire BR).

[ericfarmer]

Thanks Nyne, really appreciate these numbers. In other words, there's a 20% chance that the EV will be >= 0.0029 high (which is what would be necessary to show a player edge), and presumably an equal chance that it will be >= 0.0029 low. So 60% of the time, the EV will be accurate to +- 0.0029.

In a different thread, I ran an experiment and made a graph of my simulation's accuracy after 100k nodes. The graph agrees with this prediction EXACTLY.

Hopefully this will put to rest any claims (made by Norm and others) that my simulation results are invalid because they converge faster than they should, since they apparently converge at exactly the rate they should.

I'm jumping into this conversation pretty late, but it's interesting, so hopefully I can catch up. I started with your previous post about your iOS app, which I downloaded and played around with, and it looks pretty cool. Unfortunately, the most interesting aspects of the app under discussion here are behind your paywall, so hopefully we can resolve any issues with some example data here instead.

First, though, I am rather disappointed in the reaction of this community to what I perceive to be a developer who is interested in blackjack, who has potentially made available some useful computational tools (although obviously not for use at the table), and who seems to welcome *quantitative* investigation, criticism, etc., of the data provided by those tools. The response has not been impressive. We accuse this guy of not knowing his stuff, while at times demonstrating the same lack of mathematical or technological expertise that we accuse this guy of. The discussions about "the cloud," multi-core speedups, and RNGs were particularly embarrassing.

But back on subject: Nyne's example calculation is a good one, and your response suggests that perhaps we are just confusing units, i.e. *fractions* of initial wager, versus *percent* of initial wager? In other words, when you say "accurate to within 0.01," do you mean within a dime of a $10 wager, or within a tenth of a penny?

Also, I can't seem to find the graph in a different thread that you mention (unless it's the coin-tossing example). Can you provide a link?

Thanks,
Eric