Optimal Strategy & Blackjack

Why didn't I just calculate all the strategies' expected value against the "optimal hands" and use that as my fitness algorithm instead of playing x games against each other? Well, I think I'll just calculate the expected value for my next program.

I haven't decided weather to use C++ or Java, but I want to output a table of the optimal blackjack strategy (the basic strategy should be for when the count is at 0), based on the count.

First of all, I want to see, based on the rules at Spirit Mountain, is this the optimal strategy when the count is 0. Second, I want to know how the strategy changes based on the count. Third, I want to know, do I have an advantage, if I simply bet the minimum, but play the optimal strategy. If not, then how wide of a spread do I need to play in order to cut my losses and maximize my wins in order to gain an advantage. I'm guessing playing the minimum when I expect to lose, and 2x when I expect to win might be enough if I use the optimal strategy, but I want to know for sure.

This all started because I have some friends who play blackjack. I played blackjack in a casino for the third time in my life, lost $20, bringing my lifetime wins/losses to a grand total of -$10. I won $15 using a coupon at a $10 min. bet table in Vegas, and I lost $5 at Spirit Mountain on my 21st birthday. Anyways, I realized I didn't even know basic strategy, and I really wanted to learn. Plus, I knew about counting cards, and yet none of my friends seemed like they were serious about doing it, even though they all knew how too.

Honestly, I think if you're going to play blackjack, you may as well count cards to gain a little edge. And as long as you're only playing $5-$10 a hand, I don't think anyone will even really care if you try to count cards. Chances are you will mess up or go broke trying anyways.

I've been practicing, and last night I counted down a deck in 42 seconds. I heard in an episode of Breaking Vegas that one team of pros required you to be able to count down the deck in less than 30 seconds, so I still have a ways to go. But it took a lot of work for me to even get under 1 minute, so I'm proud of that (a lot of work = 2 days practice ~1 hour or less each day, lol).

So anyways, weather I do it in C++ or Java I just plan on outputting to an html file. Html makes such pretty tables you know. ;)

Thanks for reading.

AI Aiming Problem

My brother and I were working on an aiming problem that was quite interesting and similar to some things we worked on in our AI class. The problem is for two tanks with constant velocity, initial position, and projectile speed for tank 1's weapon (which can be fired from the turret in any direction), where should we aim in order to hit tank 2?

First we can simplify this by calling tank 1's center the origin. Then we just need to find the relative position to aim at. With this assumption tank 2's velocity and position will be relative to tank 1 (which can easily be calculated by tank2.pos - tank1.pos, and tank2.vel - tank1.vel where pos is the position vector and vel is the velocity vector for a tank). We will call tank 2's position x = (x1, x2) and velocty v = (v1, v2). We will call the bullet's speed s.

The solution can be expressed as target y, with

y = x + vt

where t is the time it will take for the bullet to travel that distance. I will not refer to this again, as the interesting part of the problem is finding t. But the algorithm should use this to calculate the position of the target once we find t. (In the game the AI specifies a target and there already exists a function to fire at that target).

It took a lot of work to find the appropriate way to express these relationships (including simplifications) in order to find a solvable equation, but here is the solution. It came from thinking of this in terms of polar form, where each position (bullet and tank2) is expressed as a pair of distance and angle (d, theta). But actually we only need the distance to be equal. Since we can fire in any direction, we can think of our projectile going out in all directions at once.

Assume we fire at time t=0. Then when the distance of the projectile from the origin (tank 1) is equal to the distance of tank2 from the origin, a solution exists. So for the distance of tank 2 relative to tank 1 T2(t)= |x + vt|, and the distance of the bullet of tank 1 relative to itself B(t)=ts, we get

B(t)=T2(t)

ts = |x + vt|

ts = sqrt((x1 + v1t)^2 + (x2 + v2t)^2)

(ts)^2 = (x1 + v1t)^2 + (x2 + v2t)^2

0 = (v1^2 + v2^2 - s^2)t^2 + 2(x1v1 + x2v2)t + x1^2 + x2^2

a = v1^2 + v2^2 - s^2

b = 2(x1v1 + x2v2)

c = x1^2 + x2^2

t1 = -(b - sqrt(b^2 - 4ac))/(2a)

t2 = -(b + sqrt(b^2 - 4ac))/(2a)

And we want the smallest of t1 and t2 that is bigger than 0 and less than the lifetime of the bullet. Also if a = 0 then we have a linear case with solution

t = -(x1^2 + x2^2)/(2*(x1v1 + x2v2)

Anyways, I was proud of myself for being able to find the solution after some work. My brother already tested it and it works better than the previous version, which was to approximate 2 times. First one based on the time it would take for the projectile to reach the tank at its current position. Second the time it would take to reach the position it would be at after that first time. This worked pretty good, but I knew we could find the time exactly (for this simple model).

The problem is still open as this only works for one type of "floating" kind of projectile. Some projectiles take into account gravity, and the arc angle is calculated automatically based on your target. So the time it takes depends on the future position, and the arc needed to reach that. And that time dictates what the future position is. So we have a similar problem, where we could use the solution above to estimate this time, and iterate, but that would only be an approximate solution.

I ran a 2 hour test where my compute played a bunch of poker with itself. I played 100 strategies for 1st position (for those of you who know poker) and 100 strategies for 2nd position against eachother. Every 1st position strategy played every 2nd position strategy 10,000 hands. Then the best one for each strategy (the one who had the most winnings after playing 1,000,000 hands) is bread into 100 more AI. The breeding used a random binomial to determine the percent it should deviate from a strategy point in the positive or negative direction. (-1 would result in a 0 for that point and 1 would result in a 1.) Anyways, here are the pretty results (p.s. bet size = 1, pot size = 1, 1 round of betting, no draws etc, 1 bet cap. A player bets hands outside of the range, and checks hands inside the range. A player player will call with hands better than or equal to the middle value (better means less), and fold worse hands. Hands are random values between 0 and 1.)

Poker AI

Okay, I'm working on a genetic algorithm to produce a good poker AI. It's not using Bayesian statistics or anything to analyze and attack your style of play, but it should be a near optimal value (i.e. some value near that of part of an optimal pair of strategies for a two player game).

Here is a trial run. This is a display of the best strategy at the end of each generation. The values listed are used to create 20 new AI with points near those, with some variance based on a kind of a skewed binomial squared (with sign preserved) mutation function. The fitness function is just 1,000 hands of poker played between every AI for P1's position and P2's position.

For poker nerds this is a one bet limit (ante 1, bet size 1) with a 1 bet cap (no raising), limit poker setting.

Every time you run the program it asks for new number of AI per generation, new bet and ante sizes, new number of hands per game (every AI plays one "game" (fixed number of hands) with every AI for its opponent's position (every P1 plays every P2)), as well as how many generations should run.

Here is a screne shot:

How he was born

I was thinking the core of Strong AI would really be the same. It should be able to use periferials to see and hear human feedback. It should be able to assess its own performance in terms of success and failure. It needs to be able to alter its behavior in creative ways.

If you can get some AI to do this, it will not be useful for just some problems, but it perhaps can be part of all AI. All AI seems like it could benefit from this behavior. Hopefully, it would be overkill for some tasks, and it would come to this conclusion on its own and remove this "creative/Strong AI" part from its code, becoming more traditional of an AI.

HRM

The hard part is trying to come up with a way for AI to decide what needs to be done, and how to do it. Once you have an AI performing a task, a Strong AI can assess its success based on feedback, and with negative feedback, regress to older behaviors, and with positive feedback, continue to progress coming up with new innovations.

Strong AI

Alexander Marsh is a computer programmer who works for Strong Intelligence, and artificial intellegence research company. His job is simply to translate the reviews of the different interesting AI written by the researchers into a database that can be read and viewed by the AI programs. This is his story.