See the top rated post in this thread. Click here

Page 2 of 2 FirstFirst 12
Results 14 to 25 of 25

Thread: "Probability of Being Ahead by a Certain Amount" Question

  1. #14


    1 out of 1 members found this post helpful. Did you find this post helpful? Yes | No

    5000-hand sim results are meaningless

    Quote Originally Posted by DSchles View Post
    The validity of the results from a SIM of 5,000 hands is meaningless. But that has nothing to do with what Dog Hand calculated. His work had nothing to do with a sim, and the results are accurate, no matter what the number of hands.

    Don
    Just to elaborate on Don's comment concerning 5000-hand sims:

    I have an Excel file containing 300,000 hands of BJ played by a heads-up B.S. player flat-betting $5/round for a 6D, H17, DAS game with 75% penetration. For this game, billion-round simulation results give the player's IBA as -0.5580% and his SD as 1.16.

    I used Excel to look at every 5000-hand set in these data: thus hands 1-5000, 2-5001, 3-5002, etc. The best set had the player winning $1,285 on total initial bets of $24,425 (4,885 rounds, EV = -$136.29), for an edge of a whopping +5.2610% and a z-score of 3.5061. The worst set had the poor guy losing $1,557.50 on total initial bets of $24,375 (4,875 rounds, EV = -$136.01), for an edge of -6.3897% and a z-score of -3.5102.

    The wide range of results for 5000-hand sims indicates clearly why such small sample sizes are not useful when the edge is so close to zero.

    In fact, for these 300,000 hands the player was "lucky": instead of losing the expected $8,133.35, he lost "only" $7,070.00, for a z-score of 0.3396. Thus, even 300,000-hand sim are not sufficient to generate useful results.

    Hope this helps!

    Dog Hand

  2. #15
    Banned or Suspended
    Join Date
    Jul 2018
    Posts
    326


    Did you find this post helpful? Yes | No
    Quote Originally Posted by Dog Hand View Post

    The wide range of results for 5000-hand sims indicates clearly why such small sample sizes are not useful when the edge is so close to zero.

    What would a sufficient number of trials be (not blackjack related, but in general) to establish reliable Means and the SD for those Means? Is it possible that the type of statstic one is attempting to identify has an impact on the number of trials needed to be considered statistically reliable? 5000 trials seems to me to be sufficient to establish Mean and SD, and your examples clearly show that 5000 trials is not a large enough sample to establish EV, but I am not a statistician and am wondering if different types of statistics require different sample sizes in order to be considered reliable.
    Last edited by Wave; 01-27-2020 at 11:55 AM.

  3. #16


    Did you find this post helpful? Yes | No
    Great! And it also clearly shows why, so often, when people wonder what they’re doing wrong because they have lost when they should have been winning, or think that this is such an easy game because they can win flat betting with basic strategy, when they should be losing, it’s all utterly meaningless statistical noise.

    Don

  4. #17


    Did you find this post helpful? Yes | No
    Quote Originally Posted by Wave View Post
    What would a sufficient number of trials be (not blackjack related, but in general) to establish reliable Means and the SD for those Means? Is it possible that the type of statistic one is attempting to identify has an impact on the number of trials needed to be considered statistically reliable? 5000 trials seems to me to be sufficient to establish Mean and SD, and your examples clearly show that 5000 trials is not a large enough sample to establish EV, but I am not a statistician and am wondering if different types of statistics require different sample sizes in order to be considered reliable.
    The term(s) that you're looking for is how many trials it takes for the values to "converge." And yes, they are quite different, depending on the statistic you're trying to determine. Norm will tell you that he wants to run hundreds of millions of hands before "declaring" any number to be accurate or reliable, but I will allow him to weigh in further on this.

    Suffice it to say that 5,000 hands is, obviously, good for absolutely nothing whatsoever.

    Don

  5. #18
    Banned or Suspended
    Join Date
    Jul 2018
    Posts
    326


    Did you find this post helpful? Yes | No
    Quote Originally Posted by DSchles View Post
    The term(s) that you're looking for is how many trials it takes for the values to "converge." And yes, they are quite different, depending on the statistic you're trying to determine. Norm will tell you that he wants to run hundreds of millions of hands before "declaring" any number to be accurate or reliable, but I will allow him to weigh in further on this.

    Suffice it to say that 5,000 hands is, obviously, good for absolutely nothing whatsoever.

    Don
    Thanks Don.

    I am in no way trying to make this political, but we are constantly bombarded with public opinion polls in the media where they act so sure of the results of polls based on "1000 registered voters", or "1000 likely voters", +/- 3% margin of error, blah, blah, blah...so if I'm interpreting your response correctly, these types of polls are basically worthless?

  6. #19


    Did you find this post helpful? Yes | No
    Quote Originally Posted by Wave View Post
    Thanks Don.

    I am in no way trying to make this political, but we are constantly bombarded with public opinion polls in the media where they act so sure of the results of polls based on "1000 registered voters", or "1000 likely voters", +/- 3% margin of error, blah, blah, blah...so if I'm interpreting your response correctly, these types of polls are basically worthless?
    No, didn't say that! With 1,000 people polled, the typical margin of error is, indeed, a bit higher than 3%. So, it might say that candidate A leads B by 55 to 45, give or take 3 percentage points. That means the vote could actually be as close as 52-48 or as far apart as 58-42 ... or more (see below). And that's a big difference (as Hillary might tell you!)!! So, the values simply aren't very reliable.

    For 5,000 observations, standard error drops to 1.4%, and so, if I tell you that your SCORE in a game is, say 50, you now can feel reasonably confident that it probably is somewhere between 50.71 and 49.29. For most, that might be acceptable, but for researchers, and those of us who obsess over every decimal point, it isn't nearly accurate enough. And, the true result will lie outside of these upper and lower limits almost one-third of the time, so, again, the results can leave much to be desired.

    Don

  7. #20
    Banned or Suspended
    Join Date
    Jul 2018
    Posts
    326


    Did you find this post helpful? Yes | No
    Quote Originally Posted by DSchles View Post
    No, didn't say that! With 1,000 people polled, the typical margin of error is, indeed, a bit higher than 3%. So, it might say that candidate A leads B by 55 to 45, give or take 3 percentage points. That means the vote could actually be as close as 52-48 or as far apart as 58-42 ... or more (see below). And that's a big difference (as Hillary might tell you!)!! So, the values simply aren't very reliable.

    For 5,000 observations, standard error drops to 1.4%, and so, if I tell you that your SCORE in a game is, say 50, you now can feel reasonably confident that it probably is somewhere between 50.71 and 49.29. For most, that might be acceptable, but for researchers, and those of us who obsess over every decimal point, it isn't nearly accurate enough. And, the true result will lie outside of these upper and lower limits almost one-third of the time, so, again, the results can leave much to be desired.

    Don
    Ahhh...now I think I get it, so then it is simply a matter of acceptable degrees of accuracy.
    Last edited by Wave; 01-27-2020 at 03:42 PM.

  8. #21


    Did you find this post helpful? Yes | No
    4 out of 5 dentists recommend...

  9. #22


    Did you find this post helpful? Yes | No
    APOLOGIES to all, I misread my own data. I actually was inaccurate on 2 or so fronts, the biggest inaccuracy being that the number of rounds was 2,550, not 5,000.

    Rather than a z-score of 3.38, the data reflects a standard deviation of 2.397. So, it looks like the probability is 0.82%.

    Notwithstanding my errors, I wanted to address Dog Hand's latest post. I understand what you mean by a very large spread of outcomes. But, in doing 295,000 sets of 5,000 round sims (out of 300,000 total rounds), my previous (now known to be mistaken) finding was way at the end of positive player expectation. Indeed, Dog Hand's best set out of 295,000 sets reflected a z-score of +3.5061 while my sim of my strategy reflected a z-score of +3.38.

    So, before I discovered my errors an hour ago, wouldn't you agree that the sim I did of my strategy reflected an outcome EXTREMELY unlikely to occur by chance (or 'luck') alone?

    As it stands now, as I said above, I am working with a probability of 0.82% with an standard deviation of 2.397.

    Finally, for those who still radically discount the validity of findings from sims with a low number of rounds, please see Norm's qfit outcome calculator. You can calculate the probability of being ahead "X" number of dollars after inputting player advantage, average bet size, and NUMBER OF ROUNDS.

    That is, one can enter a very, very low number of rounds (let's say, 1,000 rounds to reflect a blackjack trip to Vegas) and the calculator, rather than saying "this is meaningless," will still yield a probability. Norm's calculator is of course not yielding misleading, inaccurate, or meaningless information.

    I apologize once again for my errors. Nonetheless, this thread has been heretofore stimulating, and I welcome further comments based on this post.

  10. #23
    Random number herder Norm's Avatar
    Join Date
    Dec 2011
    Location
    The mote in God's eye
    Posts
    12,461
    Blog Entries
    59


    Did you find this post helpful? Yes | No
    You enter the Std. Dev. and EV which are calculated by simming hundreds of millions or billions of hands. The calculations are based on those numbers. The calculation would be meaningless if those inputs were based on 1,000 hands.
    "I don't think outside the box; I think of what I can do with the box." - Henri Matisse

  11. #24


    Did you find this post helpful? Yes | No
    Thanks, Norm, for the reply.

    But I am confused. Your free qfit outcome calculator I am referring to yields a standard deviation - one does not enter a standard deviation as you indicated above. Here is the link: https://www.qfit.com/calcrisk.htm

    However, one does enter the number of rounds into this calculator. If one enters a low number of rounds, are the results shown by your calculator meaningless?

  12. #25


    Did you find this post helpful? Yes | No
    All the risk and goal calculators are above. You're using the wrong one. Go to BJ Resources and see all the calculators there.

    Don

Page 2 of 2 FirstFirst 12

Similar Threads

  1. Basic Strategy question regarding "soft" and "hard" hands
    By Letangs in forum General Blackjack Forum
    Replies: 73
    Last Post: 08-22-2018, 07:02 AM
  2. A curiosity ~ "Probability and Strategy in a weird BJ variant"
    By ZenMaster_Flash in forum General Blackjack Forum
    Replies: 0
    Last Post: 03-11-2016, 04:20 PM
  3. Replies: 3
    Last Post: 10-15-2015, 11:37 AM
  4. Replies: 0
    Last Post: 03-29-2015, 08:44 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

About Blackjack: The Forum

BJTF is an advantage player site based on the principles of comity. That is, civil and considerate behavior for the mutual benefit of all involved. The goal of advantage play is the legal extraction of funds from gaming establishments by gaining a mathematic advantage and developing the skills required to use that advantage. To maximize our success, it is important to understand that we are all on the same side. Personal conflicts simply get in the way of our goals.