Opponent Modeling

(continued from previous post)

Now let's start to look at some opponent modeling.

There have been some attempts recently to build a large database of all online players. I believe that modeling opponents from a long-term database may be useful for identifying strong/weak players, but is not as useful for predicting playing style as short-term modeling (i.e. modeling each opponent as you play them in the current sitting).

So here I'll discuss some ideas for modeling and using your opponents' preflop behavior as you play them. That "and using" clause is key; there's not much point to any piece of analysis unless you can rationally scale and combine it with the other available data and statistics.

There is an unbounded amount of modeling you can do from observing what cards the opponent has shown in the past, in what position, etc. In reality, however, it will take a very long time to model that level of detail, because very few of an opponent's hands actually get shown. Online, only called or winning hands on the river are regularly shown (although any hand played through the river is available by request). So this type of model may take too long to build before you can apply it. More seriously though, an opponent may choose to show a hand when folding, which allows him to selectively misshape this type of model. Any opponent that voluntarily shows you his cards is either showing off or intentionally misguiding your mental modeling of him.

So I'll talk about a very simple type of preflop model: one that simply looks at percentage of flops seen by each opponent. Even this simple statistic gets complex in implementation.

To build this statistic, simply keep a #flopsSeen/#totalHandsratio for each opponent. You may halve both numerator and denominator periodically to reduce impact of older data.


Opponent Model for the preflop itself

There are 4 pieces of information available to you on the preflop:

• a. Your hole cards
• b. Your betting position
• c. Opponent bets behind
• d. Opponent betting history (i.e. some kind ofopponent model)

(this is ignoring opponent appearance and behavior, as we're talking about what is available to a bot)

There is this corresponding set of questions whose answers need to be combined somehow:

• a. Your hole cards: In what situations are they profitably playable?
• b. Your betting position: How much can I predict about the situation at the end of the preflop?
• c. Opponent bets behind: What is the significanceof their bets?
• d. Opponents ahead: How should I expect them tobet?

If it were just (a) and (b), you could use a precomputed preflop model like Sklansky's. To be fair, Sklansky does pepper his advice with clauses like 'if the game is loose preflop', etc, but we need something deterministic for every situation.

We're still not discussing bluffing here, or influencing the opponents ahead; just trying to play optimally for an honestly profitable hand at the flop.

Your preflop model (e.g. Sklansky), tells you that certain hole cards are playable against a certain number of opponents, as if they were all the same. Now you want to apply a preflop model that attributes unique characteristics to each opponent. The only thing to do is to take the available statistics when it's your turn to bet (the bets behind and the opponents ahead) and scale them somehow. But how?

a). For the "made" hands (pairs and high cards), you want few opponents. Lots of opponents behind is bad. To scale the number of bets behind based on opponent's looseness, start with a preconceived figure of the "ideal" flops-seen rate for the game size, e.g. 20%. If the opponent is seeing twice the "ideal" number of flops, you may count his bet behind as half a bet. A tight bettor behind might count as more than one scaled bet, etc. For the opponents ahead, however, their looseness doesn't count; you simply make a statistical estimate of how many stronger hands there are ahead.

b). For the "draw" hands (e.g. suited connectors), the equation is entirely different. For draw hands, the desire is to have many callers who will pay you off in the minority of cases where you make your hand. In this case, the bets behind are not scaled; they represent money that is there to pay you off; each bet behind is counted as one caller. It is the opponents ahead who are scaled on their looseness. Loose opponents ahead means a greater likelihood of enough callers to make your hand (e.g. 78s) playable. Scale each opponent ahead by his flops-seen percentage, and add that to the bets behind, to obtain an estimate for the number of players seeing the upcoming flop. Here, a tight player is scaled down instead of up. Obviously, the later your position, the more accurate your estimate is going to be.

Now you have a decision to make. You have a fixed number of callers behind, a scaled estimate of number of callers for the flop, and a range of number of callers against which your hole cards are playable according to your preflop strategy. At the conservative extreme, you can choose to play only if your hand is playable against the entire possible range of callers for the flop; or, you can choose to play if your cards are playable against the number of opponents given by your scaled estimate; or you may require a 'playable range' of #callers around the estimate.

Applying preflop modeling to RHO later in the hand

Earlier I discussed Reflective Hand Odds, which is a way of filtering the hand spectrum based on opponent betting behavior to obtain more accurate hand odds. The RHO filter assumes rational behavior by the opponents, who are usually betting according to the strength of their hands. Part of that filtering is for the preflop, i.e. if an opponent bet voluntarily on the preflop, the hand spectrum is filtered for "playable" hole cards.

The opponent preflop model is applicable to this filter. If an opponent should "ideally" be playing only the top 25% Sklansky hands in his position (and given the betting situation at the time that he bet), but he has a modeled preflop looseness factor of 2x, then the filter may adjust to consider the top 50% of Sklansky hands as having been "playable".

Depending on the board, this could make your RHO higher or lower - if only a maniac would play 46 offsuit but that is now the nuts, the loose player may have you beat; but if the board is playable with top Sklansky hole cards, then the loose player is less likely to be holding them. The RHO computation will express that numerically.

Note that it's easier to have the "Reflective Nuts" against a tight player. In many situations you can actually bet stronger against a tight player due to having made the "Reflective Nuts" (or close).