Or it just suggests that something unlikely happened with those 10 runs. You're the apparent expert on statistics, and you're trying to prove a point that knowing statistics is valuable, so maybe you can do a calculation for us and tell us what the odds are of such clumping?Originally Posted by Norm
Meanwhile, made the same graph of 1000 runs of my program and got a nice, smooth curve. What does that "suggest" to you?
cointoss.jpg
Very high with an LCG, or combination LCG. From WP, "This phenomenon is sometimes called the Marsaglia effect, and means that n-tuples with coordinates obtained from consecutive use of the generator will lie on a small number of equally spaced hyperplanes in n-dimensional space."
A combined LCG with more runs may require that you plot a 3D dispersion chart and rotate it in three-dimensions. What may originally appear as a random dispersion may line up in planes upon rotation. Marsaglia's comment, with apologies to Alan Lerner, "Random numbers fall mainly in the planes."
But, I never said I was an expert on statistics. I'm not.
Last edited by Norm; 08-24-2014 at 10:49 AM.
"I don't think outside the box; I think of what I can do with the box." - Henri Matisse
You might also note the sentence in the Wikipedia article on LCGs "LCGs should not be used for applications where high-quality randomness is critical. For example, it is not suitable for a Monte Carlo simulation because of the serialcorrelation (among other things)." There is a great animated gif there, copied below. Watch how the dispersion appears random, but then lines up into hyperplanes that appear at 2,000 calls and solidifies at 60,000.
Lcg_3d.gif
"I don't think outside the box; I think of what I can do with the box." - Henri Matisse
In that case, I will defer to the experts. What data do you need from me to calculate the quality of my EVs to your satisfaction?
I have done a quick experiment by calculating the difference between my Monte Carlo EVs (100k iterations) and my exact EVs for 185 different situations/actions. I sorted the differences and plotted them (attached). That should give you enough information to be able to calculate whatever statistics you want but please let me know if you need anything else.
I will point out again that my app runs MC simulations with WAY more iterations than 100k, so if you are using the app you can expect much more accuracy than this graph indicates. But even with 100k iterations (which the app simulates in a fraction of a second), the EVs are within 0.01 of each other 97.3% of the time.
But again, what data and analysis do you want from me for me to prove that my EVs are good?
evdiff.png
Last edited by tomker; 08-24-2014 at 11:11 AM. Reason: fixed typo
Yes, having been a professional programmer for maybe 2 decades now, I'm familiar with these well-known weaknesses of LCGs, which is why I'm not using a LCG. I think you have made an unjustified logical leap that glibc's rand function suffers from the same flaws as LCGs because its code makes it look similar to a LCG.
Now, if all I was doing was calling an LCG 3 times and combining the middle-order bits, I wouldn't be surprised if it exhibited this hyperplane behavior too, although you might need more dimensions to see it.
But the fact that the result of the combination is being used as the seed for the next random number makes this PRNG so much more complicated than an LCG that my brain hurts to even contemplate trying to analyze how weaknesses of an LCG might carry over to it.
So people are complaining that I'm calculating EVs without knowing about statistics, yet you have written the de facto standard software for calculating EVs and you are saying that you're not an expert on statistics. So how much knowledge of statistics do people on this forum think is ideal for calculating EVs? Not too much, not too little? Or can we all just agree that no knowledge of statistics is necessary to average numbers and see if one number if close to another number?
I don't see any value in my continued participation in this discussion. You are welcome to believe that there is no need for an understanding of standard error to sim blackjack and you can use whatever RNG you wish. You can also continue the discussion here.
"I don't think outside the box; I think of what I can do with the box." - Henri Matisse
I still don't really know what standard error is, but as you can see from the graph I posted above, my simulations are accurate to <= 0.01 after a fraction of a second (100k iterations) 97% of the time.
Which is a degree of accuracy that I'm happy with AND one that you claimed was impossible earlier in this thread.
So the burden of proof is really on you here, it's up to you to explain how knowing statistics will increase or decrease the accuracy of a simulation that has nothing to do with statistics. You telling me that this is important makes basically the same amount of sense as me telling you that the color of your car affects the accuracy of your blackjack simulations. If I made such a claim, you would probably expect me to explain myself too.
If the standard deviation (or standard error) of the sample does not match the known standard deviation (or standard error) of the population, that should tell you that you haven't chosen a representative sample of the population. As norm says, there is 1 correct standard error after n trials, based on the properties of the population, so if your simulation converges faster than it "should", it indicates that there is a problem with the randomness of your trials. That is something that would concern me. I suggest you research the difference between accuracy and precision.
My simulation converges on the correct numbers, so that means 1 of 2 possible things:
1. My simulation is correct and converges at the expected rate (which Norm has miscalculated since he thinks that rate is impossible)
... or ...
2. I have, by some miracle, inadvertently created a simulation that's better than one that's random, which means I should probably be writing a scientific paper about it and possibly be inducted in the blackjack hall of fame
Personally, I think possibility (2) is very unlikely.
I don't just care about the answers; I care about the method you use to get there. If your method isn't sound, I can't trust your answers, even if they're apparently right.
I'm really not a potential customer for your application, anyway. Don't feel bad about that; I'm not a potential customer for lots of other people's blackjack software, too - you're in good company.
May the cards fall in your favor.
Okay, sure. Of course there is a possibility that I have bugs in my code that will give incorrect EVs. I would be naive if I didn't admit that.
But I think there are a few reasons to have high confidence in the EVs that my app generates:
1. This is something I've said before in this thread: I've written two completely different bodies of code that calculate EVs in completely different ways using completely different methods, and I've debugged both to the point where they now agree with each other. It seems extremely unlikely that they would both have bugs that are completely different (remember, different code and methods) that cause them to yield exactly the same EVs.
2. I've used my code to calculate all my basic strategy tables and they agree with the ones found on the Wizard of Odds web site, except for some rare, minor differences that are easily explained by the assumptions made about the cards in a player's hand. I believe the author of that web site used Norm's software to generate his tables. Some of the decisions in the basic strategy tables for certain rule variations are pretty close, so the EVs have to be pretty accurate to end up with the same results.
3. I've also compared the EVs produced by my "exact" algorithm to a bunch of the ones published on this page and they match down to the last decimal place:
http://wizardofodds.com/games/blackjack/appendix/1/
4. I've also used my code to confirm the Illustrious 18 to make sure its EVs are correct when simulating particular high-low true counts.
So I'm pretty confident in the numbers produced by my algorithms.
I don't feel bad that you're not interested in buying my software. Like I said in a previous post, I'm not expecting any significant amount of sales by posting to this forum.
But I did think it would be interesting to blackjack enthusiasts that there's now a mobile app available for only $6 that can almost instantly calculate the EV for any action for any given hand for any set of rules for any given true count.
I did not expect the pushback I've received from people who obviously haven't even tried the software but who nonetheless want to cast doubt (for an unknown reason) on my code/methods/results.
A suggestion: Being APs, I think a lot of people here actually don't want to spend the $6 to give your app a spin. But, I think if you made it available to them through some sort of private code or way to allow them to try it for free, you would get more valuable feedback and more important input than you would ever dream of.
Don
Posting Permissions
-*-
Bookmarks