1. Python Bytecode Evolution using Stack-based Genetic Programming
for Regression Problems
Hyunsoo Park Kyung-Joong Kim
Department of Computer Engineering, Sejong University, Seoul, South Korea
Introduction
Traditional genetic programming uses tree-like data structure to represent a program. But, recently there is study on genetic programming directly using Java bytecode, a practical intermediate language. It evolves a series of commands that manipulate stack and register with the virtual machine and represents them with a simple list data structure instead of complex tree structure. Evolving the intermediate language is promising because 1) it is easy to combine an existing program with the program automatically generated , 2) there are several available development tools and environments for the language including virtual machine, decompiler, optimizer and so on, and 3) incorporating the list data structure into the evolutionary algorithm is simple and straightforward. In this research, we propose to evolve bytecode of Python programming language’s bytecode by stack-based genetic programming. In detail, PyPy is used as a Python interpreter to verify the correctness of bytecode and byteplay module can manipulate the bytecode in runtime. We show this method can solve simple regression problems.
Fig 3. Example of crossover
Python Script
Python Bytecode
def f(x):
x = x + x
x = 2 * x
return x
LOAD_FAST x
BINARY_ADD
STORE_FAST x
LOAD_CONST
BINARY_MULTIPLY
RETURN_VALUE
Python Bytecode
Python scripts (left) is converted into Python bytecode (right). Python bytecode is a command list, manipulating virtual machine’s stack and registers. This code means 𝑦=𝑥+2𝑥. Python bytecode can be edited by byteplay module.
Evaluation
We describe an evaluation function defined.
𝑓𝑖𝑡𝑛𝑒𝑠𝑠= 1 𝑖=1 20 𝑓(𝑥 𝑖 ) +1
Function 𝑓(𝑥): each individual (generated bytecode)
Input 𝑥 𝑖 : randomly selected 20 numbers in range [0, 1)
Selection
Population size is N, and N new offspring is generated from the crossover stage. So, the size is now 2N. We choose half of them based on their fitness for the next generation.
Python implementations
Typical Python implementations are Cpython (C), Jython (Java), IronPython (C#), PyPy (Python, Generally, Cpython is Python implementation using C programming language. It is de facto standard in Python community. It is fast and many modules are provided. PyPy is implementation using Python. It is getting attention recently. Because, PyPy is faster than Cpython in many cases.
We use PyPy 1.6, implementation using Python. It can catch necessary exceptions and verify the correctness of code. But, CPython and PyPy generated different executions for the same code.
Experiments
We solve two regression problems. It runs 100 times. Yield means the success ratio among the 100 runs.
Table 1. Problem I
Problem I
𝑦= 𝑥 4 + 𝑥 3 + 𝑥 2 +𝑥
Population Size
1000
Max. Generation 50
Yield at 30 gen
62 %
Yield at 50 gen
94 %
LOAD_GLOBAL
LOAD_FAST
BINARY_MULTIPLY
BINARY_ADD
RETURN_VALUE
'cos‘
'x‘
None
'x'
Algorithm
We follow typical genetic programming steps, but mutation stage is skipped. Therefore we describe Initialization, crossover, evaluation and selection stages.
Initialization
Seed function
def func(x):
x = cos(sin(log((((x+x)-x)*x)/x)))
return x
Initial population is not randomly generated. Initial population is needed to be runnable. So we use a seeding technique. We use a seeding function for the initial population. As a result, all individuals in the initial population is the same.
Table 2. Problem II
Problem II
𝑦= 𝑥 9 + 𝑥 8 + 𝑥 7 + 𝑥 6 + 𝑥 5 + 𝑥 4 + 𝑥 3 + 𝑥 2 +𝑥
Population Size
1000
Max. Generation
300
Yield at 150 gen
23 %
Yield at 300 gen
31 %
Fig 4. A successful code for the problem I
Fig 1. Example of initialization
Crossover
Because each individual code is a command list, we can use the same crossover technique used in simple Genetic Algorithm. We use two-point crossover.
But, the offspring is needed to be runnable and we verify the correctness of the code. If the code is not correct, then it generates new offspring.
Conclusion
We use a stack-based Genetic Programming to evolve a Python bytecode.
It shows the proposed method can generate successful regression Python codes for the simple regression tasks.
Fig 2. Crossover