Structural Test Data Generation using Genetic Algorithm and Java
Budget: £250 – £750 GBP
Structural Test Data Generation Exercise
This piece of coursework is based around the problem of using search-based strategies to automatically generate test data to achieve a level of coverage (in this exercise we will focus on statement coverage). The exercise is based around two methods included at the end of this page: one which takes integer inputs and the other yet to be defined (but you need to tackle the first problem before moving onto the second).
The main challenge with this problem is that unlike the other situations we have explored where all the necessary data was available in a file, here the GA needs to interact with the program under test and execute it to establish the fitness values of the population. So the program under test needs to be instrumented (have code inserted) to gain feedback on which parts of the program it has executed and thereby provide the data which will will feed into the fitness function. For the purposes of this exercise this instrumentation can take place by hand. A further complication is that to work effectively the GA needs to evolve data for just one "target" - a particular point in the program it is trying to reach - and to test the how program means identifying and evolving data for several targets. So as you can see it is quite an awkward (but very real) problem.
Marks will be based on:
• The representation and fitness function used (it is fine to use the approach level and branch distance covered in
the lecture but don't feel constrained by this). The targets identified, the fitness information collected, and the mechanism to do this (inserting pieces of code by hand is fine for this exercise)
• The design and implementation of the GA (it is fine to use either a framework or your own code), the associated crossover and mutation operations, and how this interacts with the program under test.
• Presentation of results and comparison with random: A summary of how successful this was (targets achieved), if it didn't work for any cases then where and why, how it compared with random search, and any other interesting observations and insights.
• Addressing difficult targets: Any additional interventions made to try and make it work for difficult cases and how successful these were.
Submission
Source code for your implementation along with a report which for each problem identifies:
• The targets identified in the code
• The representation
• The fitness function
• The modifications made to calculate the fitness value and the data collected
• Brief details of the GA implementation (own implementation/framework, any deviations from the standard, e.g. for the crossover and mutation operators) and how it interacts with the method under test.
• How successful the strategy was (which targets were covered and how long this took)
• The targets covered by a random strategy
• Comments on the relative performance of the GA and random strategies
• For any uncovered targets, any additional modifcations made (e.g. to the information collected) and how successful this was.
• Any other observations or comments not included above.
Please see the attached papers with the code of methods to test and some papers, describing using Genetic Algorithm for Structural Test Data Generation
This piece of coursework is based around the problem of using search-based strategies to automatically generate test data to achieve a level of coverage (in this exercise we will focus on statement coverage). The exercise is based around two methods included at the end of this page: one which takes integer inputs and the other yet to be defined (but you need to tackle the first problem before moving onto the second).
The main challenge with this problem is that unlike the other situations we have explored where all the necessary data was available in a file, here the GA needs to interact with the program under test and execute it to establish the fitness values of the population. So the program under test needs to be instrumented (have code inserted) to gain feedback on which parts of the program it has executed and thereby provide the data which will will feed into the fitness function. For the purposes of this exercise this instrumentation can take place by hand. A further complication is that to work effectively the GA needs to evolve data for just one "target" - a particular point in the program it is trying to reach - and to test the how program means identifying and evolving data for several targets. So as you can see it is quite an awkward (but very real) problem.
Marks will be based on:
• The representation and fitness function used (it is fine to use the approach level and branch distance covered in
the lecture but don't feel constrained by this). The targets identified, the fitness information collected, and the mechanism to do this (inserting pieces of code by hand is fine for this exercise)
• The design and implementation of the GA (it is fine to use either a framework or your own code), the associated crossover and mutation operations, and how this interacts with the program under test.
• Presentation of results and comparison with random: A summary of how successful this was (targets achieved), if it didn't work for any cases then where and why, how it compared with random search, and any other interesting observations and insights.
• Addressing difficult targets: Any additional interventions made to try and make it work for difficult cases and how successful these were.
Submission
Source code for your implementation along with a report which for each problem identifies:
• The targets identified in the code
• The representation
• The fitness function
• The modifications made to calculate the fitness value and the data collected
• Brief details of the GA implementation (own implementation/framework, any deviations from the standard, e.g. for the crossover and mutation operators) and how it interacts with the method under test.
• How successful the strategy was (which targets were covered and how long this took)
• The targets covered by a random strategy
• Comments on the relative performance of the GA and random strategies
• For any uncovered targets, any additional modifcations made (e.g. to the information collected) and how successful this was.
• Any other observations or comments not included above.
Please see the attached papers with the code of methods to test and some papers, describing using Genetic Algorithm for Structural Test Data Generation