Bill is good at Skat now!
Bill and Mike have been practicing Skat for a while now, and they have finally reached a level where they can play competitively!
In reality, what happened is that I needed a bit of a break from my usual projects. I had never been very satisfied with the level of play the Skat computer opponents provided, so I decided to go out and do proper research into how computer Skat players actually work - the most up-to-date and widely accepted strategies, ideas, and algorithms out there. Some of it went over my head, but I finally got a good enough grip on the matter to give it a try.
The result? Bill and Mike started utterly outplaying me every hand. Then again I'm not very good at Skat myself, so I needed a better yardstick: an A/B test. For the past week, half of you have been playing with the old bots, and half of you have been playing with the new bots. The results are in, and they are very promising.
Some statistics!
After the A/B test had run for a week it became extremely clear that the new bots beat the old ones on every metric. They win more hands and score more points, they defend better, and they make a lot more of their declared contracts, so they win more lists. The bottom line is simple: The new bots are as good as, and even a bit better than, the average human who plays our Skat game.
Hand scores
Previously the bots, on average, were in the red. On the average hand you could expect the bots to either score nothing, or declare and lose enough contracts for the penalties to outweigh whatever meagre points they managed to earn themselves. Their average score per hand was negative, as opposed to the pretty decent score the humans generally got - likely a combination of the humans bidding better contracts, and the bots being somewhat bad at defending against weak human contracts.
The new bots, however, are much more competent. Being very careful not to bid on contracts they don't think they can win, the bots now earn points more often than they lose them and thus they get some excellent average scores per hand. The human players, on the other hand, are now up against bots that defend much better, so risky contracts are a lot harder to pull off, leading to a lower score per hand. Still in the black, though!
Contracts bid, contracts won
As is probably obvious from the hand score chart the robots are now competent enough to know how high to bid, and how to actually complete a contract once they have won it.
As a rule of thumb you want to win at least two thirds of the contracts you declare simply because the penalty for losing is twice what you get for winning. Safe to say, the old bots did not achieve this lofty goal.
The old computer players generally bid too much, and as a result failed many of their contracts. Not most of them, mind you, but enough to fall short of the magic 66% they needed to stay out of the negative.
The new bots are a lot more deliberate. They know what bids they are likely to be able to achieve, and won't bid higher than that. Bill is set up to take more risks than Mike, but auctions tend to end long before they get anywhere near the red line.
As a result when the new bots declare they almost always complete their contracts. With good teamwork and a little luck they are by no means impossible to take down - it just isn't as comically easy as before.
Games won
And finally, the whole reason for this journey of mine: trying to fix the incredibly lopsided win rates. The old bots, frankly, weren't very good. The humans playing against them won a supermajority of games: 77%. That's simply absurd, too lopsided to justify even by our general philosophy of being a 'casual site for some relaxed games'.
After the rework I can proudly state that the computer opponents are winning at about the same rate as the human players - and even a bit better - with the humans only managing to pull in 30% of all list victories. That makes Bill and Mike easily the most competent computer players on our entire site! The previous best was Cribbage, whose computer opponent wins 51% of games on a good day.
Now, the new bots might feel a bit overtuned and too challenging. We didn't exactly set out to make a professional-level Skat computer player. While I am sure an actually good Skat enthusiast could run circles around the new bots they are undoubtedly quite challenging. However, they're a wonderful starting point for tuning things down a bit until we get something closer to an even 33% split all around. That's for a future A/B testing session to determine, and a future project for me!
The new bots are live!
As of this post, the new bots are live for everyone to play against.
The new bots are on by default through the "Difficult Computer Opponents" option, but for casual players like me who need a bit of practice before taking them on, that option can be turned off in the options menu. Doing so switches to easier opponents that use the same system as the new bots but have been tuned so that they, between the two of them, should only win 50% of the lists against the human. I will be keeping an eye on their performance, though, and if they still turn out to be too challenging I'll tone them down further until the humans are winning half of all games. I didn't specifically run an A/B test to arrive at that 50% number, I just had the easier bots play the harder bots (being stand-ins for a human player) and twisted knobs until the hard bots beat the easy bots about the right amount.
Because of the new strategy, Bill and Mike's Skat algorithm might be the most complicated and involved one of all the bot players on our site, but if this goes well I might be convinced to try adapting it for more of our games. We'll see!
I hope you enjoy the new bots and that they will provide a more challenging and fun experience for everyone!
If you want to give the new bots a go simply head over to Skat here on Cardgames.io, and if you have any comments or feedback on the new bots, don't hesitate to reach out to me via support@cardgames.io or any of our social media channels.
Until next time!
