|
|
 |
Poker Botting |
 |
1
 |
jchysk United States. Jan 31 2015 21:09. Posts 435 | | |
Some background:
I played poker pretty much full time from 2007-2009. I played about 3 million hands, mostly cash games, over 10,000 tournaments, and had done a great deal of analysis on my play. At the end of 2009, even though I was doing better than ever in poker, I felt like the game had become a job.
After doing research and speaking with a bunch of botters I knew it was possible to be fairly profitable. Already, many of the highest performing low-stakes SNG players on Sharkscope were bots. My long term goal was to build a self-improving botting system that could take on any poker game of any type.
Having spent such an immense amount of time analyzing the game and my play, I felt like utilizing all the information to write an AI would be the best use of that knowledge. I spent a good chunk of 2010 both writing an AI and setting up a botting system that would use that AI to play real money games online. I basically had my bots play until Poker Black Friday. In that time hundreds of thousands of SNGs and over 10 million hands were played.
So how does a bot work?
You must completely automate everything a human would do. Open the client, log in, pick the games using the filters and clicking through the results. Approve or close popups. Join games. Play games. Leave games. Close client.
Botting undetected:
Unfortunately most poker sites terms of service disallowed the use of bots with confiscation of funds and account closure as penalties for violation. The biggest and best sites to play on had these terms, so the first order of business is how to play and not get caught.
There were a number of mechanisms sites employed to enforce these terms including:
Resizing tables
Captchas
Different images for the same cards (although inperceptible to humans)
Pop-up boxes with a human asking a simple question
Scanning running software
Tracking mouse clicks
Checks for non-human play schedules
Also, when pokertableratings started getting popular and allowing the purchasing of hand histories, players themselves were finding and reporting bots by auditing play stats.
After looking at what tools were currently available I ended up choosing a piece of open source software called OpenHoldem and forking it for my own purposes. OpenHoldem had a tool called a TableMap that allowed you to capture images and text from a poker table and translate them into variables that could be used for the logic. Since this software couldn't be made visible to the poker clients being used, I ran poker clients in a headless virtual machine and output the display to a separate computer. Within the virtual machine I used software to place tables at specific resolutions and locations on the screen. A friend helped me write a program that turned the graphic windows from the virtual machine's output into their own separate windows that OpenHoldem could lock to and use the appropriate table map for. Since only the graphic display is being output from the virtual machine, there isn't even a remote desktop client such as VNC or Teamviewer running within.
So far this solves poker clients detecting running software and the resizing of tables. For images being slightly different, I just had to make very robust table map sets where they could account for a certain degree of error and utilize fuzzy fonts. For pop-ups and captchas I had alerts pop-up when there was a certain degree of deviation and I could manually take over for a minute to correct the issue. For the first few months I was always within range of a computer that would give an alert so I could respond to an anomaly. After an account had successfully solved a captcha the risk was lowered and I could feel safe going out to dinner or taking a nap without having to actively monitor play.
The same program that generated the graphic display windows for tables from the virtual machine would accept incoming clicks from OpenHoldem for when an action needed to be done and then would convey those clicks into the virtual machine. Since there were regions where buttons existed, using certain distribution rules, the location on the button could be randomly clicked so as to emulate human clicking behavior.
As far as bots not being caught for their measurably similar behavior, I gave them different personalities. Logic was slightly tweaked for more/less aggressive play in certain areas or overall. For example I could just slightly increase the aggression factor for a player and across every possible scenario they would take the slightly more aggressive approach. I could pick just a few stats such as call preflop raise and 3bet post-flop and tweak them by a percentage. In many cases, this made the bot play less optimally, but different enough to be unique. If a bot unexpectedly performed better in certain areas, I could figure out why and use that for future builds.
Running multiple accounts:
The most obvious advantage of a bot playing is that it can play endlessly at its most optimal settings. You couldn't have a single account play forever because that would be impossible for a human to do and the most obvious red flag of botting. So creating multiple accounts across multiple sites was the best way to get the most of the bots and cover multiple stakes. I set up different VPNs in the same general area as where the account owner's location claimed to be.
At first I never had any of the bots on the same site and stake play at the same time since I figured it would be sub-optimal to ever have them potentially play against each other. Later I realized that could be a flag and I started semi-randomizing when they would activate and play.
More Automation:
I set up a scheduling system where each bot had its own schedule (e.g. Mon-Thu 11:00AM-7PM with 15% variance). When a bot was set to play, its appropriate VPN was turned on, the virtual machine would boot, the clients would be opened and logged into using a separate script that was able to graphically interact with the virtual machine. Once everything was open a program like TableNinja could be used to register for games that were to pre-defined settings. For sites without TableNinja compatibility I had to write a set of scripts that would check the filters and scroll through to register for the appropriate games.
A forked version of the Table Map system with OpenHoldem was designed to assist with picking games. Going down the games in the lobby or opening up a game lobby the players could be scraped and checked against the database of players and hands to figure out how juicy the game was. I never had this actually stop a bot from joining a game, but I was collecting the data with the plan for better game selection in the future.
Stop losses was a pretty important feature that I built early on. I would have regular iterations of the logic system and sometimes there would be flaws that would make the bot unprofitable. It wasn't always obvious in their play, but over hundreds of thousands of games the losses would prove otherwise. Essentially an account balance check was done at the beginning of play and then every time a new game was registered. If the balance was below the stop loss limit, it would stop registering for new games.
Actual AI:
So the high level logic was something along the lines of:
Evaluate current actions, current stack, potential effects of all available actions, pick the best one.
All knowledge on all the players would be taken into consideration from previous games by utilizing a database of hand histories. In a particular decision the main variables that influenced the decision were current bets by which players, stack sizes of all remaining players to act, the odds of all those players acting in which ways with what kinds of hands, ICM values, position.
For a simple example, if it was folded to on the button it would first evaluate the EV or raising. It first checked the call preflop raise rates of the blinds and overcall rate of the BB. That stat would be determined by any knowledge of those players from those positions at the current blinds. That stat gives a pretty good idea of what kinds of hands those would be, and then it can be calculated with the current hand and ICM equity values what the EV was. Next the EV of limping and folding would be evaluated in a similar fashion and the highest EV score would determine the action.
I build formulas to quickly evaluate all the major stats that would be used for measurement: PFR, CPR, 3Bet, 4Bet, CBet, etc. I would do some statistical analysis to build a dataset that I thought accurately reflected the stat using the variables I wanted. Then I would use symbolic regression with those variables to generate a reasonable formula. This was time consuming and required a lot of computing power. An example formula for cpfr looked like this: f(nplayersdealt,stackbbs,temppos) = (temppos)/(log((temppos)*((temppos)*(((stackbbs) + ((11.601799964904785)*((nplayersdealt)/((stackbbs) + (sin((((-5.0985198020935059)*(temppos)) - (nplayersdealt))/(nplayersdealt))))))) + (((mod((nplayersdealt) + (((nplayersdealt) + ((5.1521501541137695)*(temppos)))/(nplayersdealt)), -1.5626200437545776)) - (mod(nplayersdealt, log(nplayersdealt)))) - (nplayersdealt)))))). It took number of players in the hand, current stack size relative to big blind, and the current position relative to the dealer.
The formulas I was describing above were used to figure out the baseline adjustments of a particular stat based on the most important general knowledge factors. During actual gameplay there was another set of functions that ran in order to weight the stats based on the players' known data. This was grabbed from my shared PostgreSQL database across the bots per poker room. There was also a level of confidence associated with each stat based on how much data I had for it for that particular player.
A major issue with this approach was dealing with heteroscedasticity with the initial analysis. I had way more data on how people play AA than anything else simply because it shows down more often since players are more likely to play it. It becomes increasingly difficult to find out how players play a hand like 22 in rare scenarios, even with millions of hands to analyze.
After awhile of accumulating data I felt that playing against an unknown was possibly the place where I felt most optimized. Not in the sense that I could play better against them than a known player, but relatively to someone else approaching an unknown. Using data I could figure out the distribution of common play tactics of an unknown. One tricky part was you couldn't simply average the distributions by their likelihood of occurring. Playing sub-optimally against one type of strategy could have a far more negative impact than playing sub-optimally against another type. So in some instances it's better to assume the player is going to be aggressive even if it's actually more common for the player to be passive just because it's less harmful against the passive players than it is when they're aggressive.
Improvements were surprisingly slow and undoubtedly where my system needed the most work. After a bot was consistently beating a certain game, moving up in stakes usually didn't go well. It would take quite some time of break even or even negative play before adjustment pushed it into the positive zone. I'd say it took well over half a million hands at each stake to get it where I wanted it to be. Utilizing a great deal of the incoming revenue as a research budget for higher stakes made it so there wasn't very much profit, but kept progress up.
Conclusions:
It was a fun project. My original long term goal was to build a self-improving NL cash game AI and I did spend quite some time building out logic systems for cash games. I figured building SNG bots would be much easier to make profitable, and then once they were running I'd switch over to cash games. I had many of my friends who were profitable at all different stakes and games send me collectively millions of hands to help me out. In exchange I could analyze their respective games and show them very specific instances where they could improve. Having the hole cards of many different profitable players helped immensely with finding core patterns that link to profitability.
To model some of the decision making I put all the players' hand histories into separate databases so that only the knowledge that the player had on the opponents was available. A model I would put together on something as simple as preflop actions from a particular position I could compare with another player's hand histories to see if they would take the same approach using the knowledge they uniquely had on their opponents.
This project helped me learn a lot about programming, statistics, and higher level mathematics. Poker AI specifically isn't something I would ever get into again. For the amount of work that is required there are similar kinds of projects that can yield much higher gains. For example an automated financial trading system. I also can't recommend that anyone else to get into poker botting beyond a hobbyist activity. The number of specialized areas that I needed to branch into in order to piece everything together was much greater than anticipated and often required me to reach out to those more proficient in those areas. Honestly, I think it's something that a strong team should be assembled to pursue and then financially it's likely not worth it compared to other ventures.
|
| |
|
| 1
|
Smuft Canada. Jan 31 2015 22:02. Posts 633 | | |
Interesting stuff, you're probably going to get some angry posts for this but thanks for sharing.
Some questions:
1. Does your system get passed PS security?
2. What stakes were you profitable to at what win rate?
3. What stakes do the best botters beat and for what win rate?
4. What are you going to do next?
|
|
| 1
|
jchysk United States. Jan 31 2015 22:39. Posts 435 | | |
| | On January 31 2015 21:02 Smuft wrote:
Interesting stuff, you're probably going to get some angry posts for this but thanks for sharing.
Some questions:
1. Does your system get passed PS security?
2. What stakes were you profitable to at what win rate?
3. What stakes do the best botters beat and for what win rate?
4. What are you going to do next?
|
1) I haven't run any bots since mid-2011. At the time there weren't any security systems employed that could detect them. PS had the best anti-botting system of any room though.
2) Depended on the site, the game, and how long was played there. For example, double or nothings or superturbos had a much easier time ranking up with fewer hands than regular sngs. Some rooms the winrate was much higher for the same stakes, but significantly fewer games available.
3) I have no idea what the current landscape looks like, but I know up to $20 regular NL sngs and micro-stakes short and full stacked NL cash games were being beaten when I was working. The winrates were again pretty widespread. Some people were fine breaking even as long as rakeback or points could bring in some profit. There were many limit game botters as well, but I had no interest in that.
4) I am entrepreneur that builds tech companies. I plan on continuing that until I'm a billionaire. |
| |
|
| 1 | |
Great post, thanks for sharing. |
|
| Our deepest fear is not that we are inadequate. Our deepest fear is that we are powerful beyond measure. It is our light not our darkness that most frightens us and as we let our own lights shine we unconsciously give other people permision to do the same | |
|
| 1
 |
bigredhoss Cook Islands. Jan 31 2015 22:50. Posts 8648 | | |
| | On January 31 2015 20:09 jchysk wrote:
(Damn it feels good to be a gangster) |
1. what is your education and programming background?
2. i know it's not related to this blog, but i remember from previous blogs you were involved in cryptocurrency, what do you think of BTC's crazy recent downswing?
3. since you mention building an automated financial trading system as a comparatively more profitable use of time, is that something you're pursuing or plan to pursue?
4. is the reason you no longer run the bots primarily because of site's evolving security, your algorithms becoming unprofitable as players improved, or something else? |
|
| Truck-Crash Life | Last edit: 31/01/2015 22:51 |
|
| 1 | |
I never wanted to accepted the fact of bots when i played on PS. i always thought their security was really good with those capcha's you had to type in. I am just in total awe.
seems like you invested so much already to give up. if you could at least get a break even one its free money. |
| |
|
| 1
|
jchysk United States. Jan 31 2015 23:39. Posts 435 | | |
| | On January 31 2015 21:50 bigredhoss wrote:
Show nested quote +
On January 31 2015 20:09 jchysk wrote:
(Damn it feels good to be a gangster) |
1. what is your education and programming background?
2. i know it's not related to this blog, but i remember from previous blogs you were involved in cryptocurrency, what do you think of BTC's crazy recent downswing?
3. since you mention building an automated financial trading system as a comparatively more profitable use of time, is that something you're pursuing or plan to pursue?
4. is the reason you no longer run the bots primarily because of site's evolving security, your algorithms becoming unprofitable as players improved, or something else?
|
1) I've been programming at least a little bit since I was 12 years old. I'd like to say I'm mostly self-taught, but I've found quite a few very intelligent people over the course of my life that I've managed to wrangle a good bit of education out of.
2) I can't say I've been pleased. The vast majority of my liquid worth is in cryptocurrencies. I have way larger swings now than when I played poker. I am a large stakeholder in a Bitcoin exchange though and any crazy movement means greater trading volume.
3) I've casually built some systems for analysis and do a tiny amount of trading on the side. I also have built arbitrage systems for sports betting and bitcoin trading. The difficulty is in shipping large amounts of money to various overseas accounts. Especially since many bitcoin sites and sportsbooks have a high shutdown rate as they are prone to being hacked or being fiscally/morally irresponsible.
4) After Black Friday, I felt a little bit defeated and after taking a short break I decided it was a good opportunity to reevaluate what I was doing in life and I took up full time entrepreneurship. Overall it was a bit draining to be operating in a constant gray area and in the back of my mind there was a recurring feeling of not contributing anything of value to the world. |
| |
|
| 1
 |
whamm! Albania. Jan 31 2015 23:44. Posts 11625 | | | |
|
| 1
 |
NewbSaibot United States. Feb 01 2015 00:26. Posts 4943 | | |
I call BS, namely because I thought of this exact implementation years ago entirely on my own and your roadmap for a bot is nearly identical to what I personally pioneered in my own head, and I'm an idiot.
I wont believe it until you post some source code / screenshots of it in action. I expect excuses for why you have none of the above or refuse to do so, especially the source code, to properly conclude this troll post.
|
| |
|
| 1
 |
Trav94 Canada. Feb 01 2015 00:36. Posts 1785 | | |
| | On January 31 2015 23:26 NewbSaibot wrote:
I wont believe it until you post some source code / screenshots of it in action. |
I as well would like to see some source code |
|
| 1
|
jchysk United States. Feb 01 2015 00:56. Posts 435 | | |
| | On January 31 2015 23:26 NewbSaibot wrote:
I call BS, namely because I thought of this exact implementation years ago entirely on my own and your roadmap for a bot is nearly identical to what I personally pioneered in my own head, and I'm an idiot.
I wont believe it until you post some source code / screenshots of it in action. I expect excuses for why you have none of the above or refuse to do so, especially the source code, to properly conclude this troll post.
|
I doubt that what I built is "identical" to what you thought might be implemented, although there is a difference between thinking of doing something and actually executing. For one, that person that actually executes also gets to learn about the obstacles that may have been difficult to foresee.
There's no need to bait or attempt reverse psychology in order to trick me into releasing valuable source code as I don't particularly care if you believe me or not. You merely could have asked.
I absolutely do have all of the source code somewhere and if mods are not opposed with me doing so, I am willing to dig some of it up and post it. |
| |
|
| 1
|
bigredhoss Cook Islands. Feb 01 2015 01:44. Posts 8648 | | |
| | On January 31 2015 22:39 jchysk wrote:
3) I've casually built some systems for analysis and do a tiny amount of trading on the side. I also have built arbitrage systems for sports betting and bitcoin trading. The difficulty is in shipping large amounts of money to various overseas accounts. Especially since many bitcoin sites and sportsbooks have a high shutdown rate as they are prone to being hacked or being fiscally/morally irresponsible.
4) After Black Friday, I felt a little bit defeated and after taking a short break I decided it was a good opportunity to reevaluate what I was doing in life and I took up full time entrepreneurship. Overall it was a bit draining to be operating in a constant gray area and in the back of my mind there was a recurring feeling of not contributing anything of value to the world. |
re: sports betting
what is the reason for depositing at any of the "less reputable" sportsbooks? between pinnacle/5d/bovada/bookmaker/probably a couple others that are fairly reputable (at least as far as these things go...i have heard some complaints about these sites but nothing like being hacked or shutting down and taking customers money like you see in smaller books) what do you need other books for?
can you give some kind of generalization of what your system does? like does it just track lines at different sites and alert you when there's a rogue/slow-moving line that allows for arbitrage?
re: bots
i understand shifting focus to entrepreneurship, but couldn't you still run the bots if they're profitable without taking any of your time? maybe i'm naive about what's required to run a pokerbot even after it's created.
thanks for answering these questions btw |
| |
|
| 1
|
jchysk United States. Feb 01 2015 04:52. Posts 435 | | |
| | On February 01 2015 00:44 bigredhoss wrote:
Show nested quote +
On January 31 2015 22:39 jchysk wrote:
3) I've casually built some systems for analysis and do a tiny amount of trading on the side. I also have built arbitrage systems for sports betting and bitcoin trading. The difficulty is in shipping large amounts of money to various overseas accounts. Especially since many bitcoin sites and sportsbooks have a high shutdown rate as they are prone to being hacked or being fiscally/morally irresponsible.
4) After Black Friday, I felt a little bit defeated and after taking a short break I decided it was a good opportunity to reevaluate what I was doing in life and I took up full time entrepreneurship. Overall it was a bit draining to be operating in a constant gray area and in the back of my mind there was a recurring feeling of not contributing anything of value to the world. |
re: sports betting
what is the reason for depositing at any of the "less reputable" sportsbooks? between pinnacle/5d/bovada/bookmaker/probably a couple others that are fairly reputable (at least as far as these things go...i have heard some complaints about these sites but nothing like being hacked or shutting down and taking customers money like you see in smaller books) what do you need other books for?
can you give some kind of generalization of what your system does? like does it just track lines at different sites and alert you when there's a rogue/slow-moving line that allows for arbitrage?
re: bots
i understand shifting focus to entrepreneurship, but couldn't you still run the bots if they're profitable without taking any of your time? maybe i'm naive about what's required to run a pokerbot even after it's created.
thanks for answering these questions btw
|
Generally when you're arbing, you have a very small edge after rake and fees to move your money around. So in order to make it worth it, you need to be making substantial sized bets. Even if you're guaranteeing profit regardless of the outcome of the game, you don't necessarily know where your money is going to end up and where you will need to move it from.
Many sites will only let you bet so much. Sometimes it's so they can adjust the line and then allow you to bet again or you're just capped out. So having money on many different sites when an arb presents itself gives you a better opportunity to fully take advantage.
There are also places that allow individuals to put up their own lines to any takers. This may be because they can't get enough action on a book or it's a relatively unpopular game. What you can do is take their action for a really good line and then split up the other side across as many books as you can.
Overall though, because many of the smaller books are shady, and money can be difficult or slow to move around I deemed it not worth the hassle.
As far as running a bot, you need accounts. I live in the US so all the sites that my software was made compatible with I can't legally play on. Trying to run bots on other peoples' accounts from other countries sounds like a troublesome task for numerous reasons. Also, the software would need to be updated every time additional security measures were put into place and the graphical interfaces of the lobby or a table changed. I remember dreading updates because I would have to figure out if anything broke. |
| |
|
| 1
|
ReSpOnSe United States. Feb 01 2015 12:32. Posts 405 | | |
This is really interesting/impressive if it's legit. |
|
| 1
 |
Romm3l Germany. Feb 01 2015 13:06. Posts 285 | | |
ur obv v smart, dedicated and different to normal people.. good to hear there's something in the back of your mind about using those talents to contribute to the world instead of purely self-enriching zerosum activity  |
|
| 1
|
NewbSaibot United States. Feb 01 2015 13:49. Posts 4943 | | |
Will you post some arbitrary source code please? |
| |
|
| 1
|
Rinny United States. Feb 01 2015 17:42. Posts 600 | | |
holy fucking shit... in awe. this is so beautiful. Very humbling + inspiring to read. |
|
| 1 | |
sounds like you made a better bot than 99% of solo botters |
|
| 1
|
Smuft Canada. Feb 03 2015 04:02. Posts 633 | | |
It seems you've successfully been your own boss for a long time; something that's very hard that most poker players have failed horribly at (including me). Some comments and advice on how you've accomplished that would be very interesting. More specifically:
1. What's your daily schedule like?
2. Where do you get the motivation to put in 12+ hour long days in year after year?
3. How do you decide what work needs to be done and how do you prioritize it?
4. How do you manage those priorities (tasks)? like do you setup a timeline months in advance and know what you'll be working on in June of this year?
5. Any material you've read on this subject that helped you a lot?
|
|
| | Last edit: 03/02/2015 04:46 |
|
| 1 | |
| | On February 01 2015 12:06 Romm3l wrote:
ur obv v smart, dedicated and different to normal people.. good to hear there's something in the back of your mind about using those talents to contribute to the world instead of purely self-enriching zerosum activity |
By pursuing self interest and maximizing our own good, through trade and entrepreneurship, society as a whole is better off then it would be if otherwise.
The saving the wales, holding hands and dancing around fires singing songs idea is over rated |
|
| Our deepest fear is not that we are inadequate. Our deepest fear is that we are powerful beyond measure. It is our light not our darkness that most frightens us and as we let our own lights shine we unconsciously give other people permision to do the same | |
|
| |
|
|
 Poker Streams | |
|
Canada. Jan 31 2015 22:50. Posts 20070
