Every now and then I manage to write a piece of code that really makes me smile for a long time. It happens about twice a year, sometimes more often. The shorter and more useful the code is, the better. Yesterday I felt enlightened again, and I cannot resist showing you the result of my internal brainstorm.
To start with, I read many bad opinions about multiprocessing (MP) in Python — useless, slow, badly designed. But to be fair, first steps with multiprocessing may be difficult in any programming language. Queues, locks, races, synchronization, shared memory — they all make…
When I start to solve a new machine learning task with Python and TensorFlow, at the beginning I try to keep things simple. I focus on the core of the problem and try to isolate it. Usually it is very good to start with the minimum amount of the data to set small, simple and effective models as the baseline. At this stage you should forget about corner cases for a while. I think that it is a workable scenario for most of the problems. It is a good strategy to not fall into big trouble early.
To solve poker…
Let’s build a neural network (NN) for the task introduced in my previous post. I expect the network to learn an algorithm of a poker hand ranking from the given examples. It is only a part of solving poker game with reinforcement learning (RL) and neural networks, but as we will see later, it is very interesting and important one.
Let’s look at the following example (single case for two players hands). It shows possible steps of the ranking algorithm.
Implementing reinforcement learning in such complex environment is not straightforward, but I can split the problem into smaller factors. For sure, it will take time. In the meantime, I will report the progress of the project and my observations here in the subsequent posts.
In general, I’ll have to think about the input data, build poker environment representation for neural network and think about neural architectures. Moreover, I’ll need to write the code — I have chosen Python, mainly with Numpy and TensorFlow. There’s also room for multiprocessing, genetic algorithms and many other interesting experiments. …
It is the right time to give the pypoks project a new “medium”. In the following posts, you will find some info, concepts & updates about the project.
pypoks is an attempt to implement reinforcement learning into deep neural networks for poker game environment. I will use Python with Tensorflow mainly.
You may have a look at the code here.