New iOS app shows real-time EVs as you play

[tomker]

Again, there is no such processor. I have never heard of the company you are using -- sounds like they are exaggerating.

And again, what makes you think the computer only had one processor? Maybe it had two 8-core processors. Or four 4-core processors.

Ultimately it doesn't matter. You seem to think that I bought into some hyped-up marketing claims, ran my code on a no-name service, and called it good. In fact, my code constantly printed out performance statistics about how quickly it was running and I compared those statistics to what I get with my desktop computer, which has Ivy Bridge cores. The DigitalOcean computer was able to do the workload exactly as fast as 16 2.3GHz Ivy Bridge cores. I saw the numbers with my own eyes, so nothing you say on this thread will somehow convince me that my own code ran slower than it did in reality.

But whatever, I was simply making a friendly suggestion about something you might consider doing in the future, since I had a great experience with the service myself. But if all you're going to do is imply that I'm lying or that I don't know what I'm doing, then I rescind my suggestion and we don't have to talk about it again.

[Norm]

I'm not sure what you mean by standard error but I disagree that a few hundred thousand hands is "useless."

Please take this the right way. After you posted the above, you probably lost most of the statisticians on the site. Here is a good article: Standard Error

[tomker]

Please take this the right way. After you posted the above, you probably lost most of the statisticians on the site. Here is a good article: Standard Error

I don't see how I could take it a wrong way. I am happy to learn new things. I admit that my education is in electrical engineering and we only had to take one semester of statistics at my university, which I only barely remember. So this article, or at least how it applies to determining the convergence rate of Monte Carlo simulations, has unfortunately gone over my head.

But I would be happy to write some code to run my simulation in a loop with 100k hands and post the results. From that I could easily calculate the standard deviation, error, etc. between each run of the simulation and the correct EV and see if those match your statistical expectations.

[bigplayer]

I don't see how I could take it a wrong way. I am happy to learn new things. I admit that my education is in electrical engineering and we only had to take one semester of statistics at my university, which I only barely remember. So this article, or at least how it applies to determining the convergence rate of Monte Carlo simulations, has unfortunately gone over my head.

Yet you've written software that requires a decent understanding of statistics in order to properly model the game of blackjack and are, in my view, excessively defensive, when some of the games top theoreticians are critical of your methods. Do you get where the inconsistency is in this?

[tomker]

Yet you've written software that requires a decent understanding of statistics in order to properly model the game of blackjack and are, in my view, excessively defensive, when some of the games top theoreticians are critical of your methods. Do you get where the inconsistency is in this?

No, I don't "get" the inconsistency because I completely disagree with your premise. You don't need to know anything about statistics to simulate blackjack hands and average the results. And you don't need to know anything about statistics to see if your results are any good.

As a proof by induction, let's consider tossing a coin. The probability of tossing a head is very obviously 0.5. Now, as long as you know how to divide, you can toss a coin X number of times, figure out the average, and eyeball your average to see how close it is to 0.5.

I have written a quick program to simulate tossing a coin 100k times, in a loop. Here are some lines of output:

average coin toss value: 0.499570
average coin toss value: 0.502760
average coin toss value: 0.502410
average coin toss value: 0.497520
average coin toss value: 0.500420
average coin toss value: 0.502530
average coin toss value: 0.498850
average coin toss value: 0.498970
average coin toss value: 0.498960
average coin toss value: 0.500180

So, as you can see very quickly, and without any knowledge of statistics, the result is (let's say almost) always within 2 decimal places of what you know to be correct.

Now, what statistics will be able to tell you is, what is the probability of your result being within a certain range of accuracy? This is valuable and interesting if you're writing a scientific paper about something, or you have to legally guarantee the accuracy of a calculation. But in the case of an iOS app that calculates blackjack odds, I would argue that it has minimal practical value over a human being simply eyeballing the numbers and saying "hey, these numbers always seem to be correct to at least 2 digits."

[bigplayer]

So, as you can see very quickly, and without any knowledge of statistics, the result is (let's say almost) always within 2 decimal places of what you know to be correct.

Except that with coin-tosses you KNOW the results are supposed to be 0.5 so when you look at a bunch of 0.49xxxx and 0.50xxxxx results you fairly confident. With blackjack you DON'T KNOW what the results are suppose to be so you cannot just eyeball the results and feel confident about them.

[tomker]

Except that with coin-tosses you KNOW the results are supposed to be 0.5 so when you look at a bunch of 0.49xxxx and 0.50xxxxx results you fairly confident. With blackjack you DON'T KNOW what the results are suppose to be so you cannot just eyeball the results and feel confident about them.

I think in your haste to prove a point you've said something you probably didn't intend. In fact, blackjack EVs are all over: online, in books, apparently Norm's software can generate them, etc.

[Dieter]

you don't need to know anything about statistics to see if your results are any good.

As a proof by induction, let's consider tossing a coin. The probability of tossing a head is very obviously 0.5.

You absolutely do need to know some things about statistics to determine if your results are any good. The mechanics of gathering and tabulating the results isn't nearly so important as the analysis of the results.

Absolutely none of the people I toss coins with expect the results to be 50/50.* I don't even expect 50/50 in a supposedly "fair" coin toss.

Now, as long as you know how to divide, you can toss a coin X number of times, figure out the average, and eyeball your average to see how close it is to 0.5.

That's why you need to understand statistics, to determine how valid your results are.

*The people I toss coins with don't determine the results based on one toss, but based on 2 or more simultaneous (unfair) tosses. This somewhat normalizes the outcome when multiple AP's are competing.

[tomker]

You absolutely do need to know some things about statistics to determine if your results are any good. The mechanics of gathering and tabulating the results isn't nearly so important as the analysis of the results. ...

In that case, I will defer to the experts. What data do you need from me to calculate the quality of my EVs to your satisfaction?

I have done a quick experiment by calculating the difference between my Monte Carlo EVs (100k iterations) and my exact EVs for 185 different situations/actions. I sorted the differences and plotted them (attached). That should give you enough information to be able to calculate whatever statistics you want but please let me know if you need anything else.

I will point out again that my app runs MC simulations with WAY more iterations than 100k, so if you are using the app you can expect much more accuracy than this graph indicates. But even with 100k iterations (which the app simulates in a fraction of a second), the EVs are within 0.01 of each other 97.3% of the time.

But again, what data and analysis do you want from me for me to prove that my EVs are good?

evdiff.png

[Dieter]

what data and analysis do you want from me for me to prove that my EVs are good?

I don't just care about the answers; I care about the method you use to get there. If your method isn't sound, I can't trust your answers, even if they're apparently right.

I'm really not a potential customer for your application, anyway. Don't feel bad about that; I'm not a potential customer for lots of other people's blackjack software, too - you're in good company.

[tomker]

I don't just care about the answers; I care about the method you use to get there. If your method isn't sound, I can't trust your answers, even if they're apparently right.
...

Okay, sure. Of course there is a possibility that I have bugs in my code that will give incorrect EVs. I would be naive if I didn't admit that.

But I think there are a few reasons to have high confidence in the EVs that my app generates:

1. This is something I've said before in this thread: I've written two completely different bodies of code that calculate EVs in completely different ways using completely different methods, and I've debugged both to the point where they now agree with each other. It seems extremely unlikely that they would both have bugs that are completely different (remember, different code and methods) that cause them to yield exactly the same EVs.

2. I've used my code to calculate all my basic strategy tables and they agree with the ones found on the Wizard of Odds web site, except for some rare, minor differences that are easily explained by the assumptions made about the cards in a player's hand. I believe the author of that web site used Norm's software to generate his tables. Some of the decisions in the basic strategy tables for certain rule variations are pretty close, so the EVs have to be pretty accurate to end up with the same results.

3. I've also compared the EVs produced by my "exact" algorithm to a bunch of the ones published on this page and they match down to the last decimal place:
http://wizardofodds.com/games/blackjack/appendix/1/

4. I've also used my code to confirm the Illustrious 18 to make sure its EVs are correct when simulating particular high-low true counts.

So I'm pretty confident in the numbers produced by my algorithms.

I don't feel bad that you're not interested in buying my software. Like I said in a previous post, I'm not expecting any significant amount of sales by posting to this forum.

But I did think it would be interesting to blackjack enthusiasts that there's now a mobile app available for only $6 that can almost instantly calculate the EV for any action for any given hand for any set of rules for any given true count.

I did not expect the pushback I've received from people who obviously haven't even tried the software but who nonetheless want to cast doubt (for an unknown reason) on my code/methods/results.

[DSchles]

A suggestion: Being APs, I think a lot of people here actually don't want to spend the $6 to give your app a spin. But, I think if you made it available to them through some sort of private code or way to allow them to try it for free, you would get more valuable feedback and more important input than you would ever dream of.

Don

[tomker]

A suggestion: Being APs, I think a lot of people here actually don't want to spend the $6 to give your app a spin. But, I think if you made it available to them through some sort of private code or way to allow them to try it for free, you would get more valuable feedback and more important input than you would ever dream of.

Don

Hi Don,

Thank you for the suggestion. This got buried in a post pages ago, but I did make an offer that the first 10 people from this forum to buy the $6 upgrade in my app could email me their iTunes receipt and I would happily PayPal them the full amount that they paid. So far nobody has taken me up on the offer but it still stands.

Unfortunately Apple doesn't have a way to gift in-app purchases to other people so this system of emailing receipts and PayPaling money is the best I can do.