efficientMutation flag for neat + XOR test case utilizing it

[D-Nice]

Implements an efficientMutation flag that can now be passed in as a boolean option. Before it attempts to do any mutations, it checks which mutations are possible on the genome from the selected mutation types, and passes them to be utilized from the selectMutationMethod. There shouldn't be a case where no mutations are returned (as long as one of ADD_NODE, MOD_WEIGHT, MOD_BIAS, is amongst the selected mutations, and maxNodes isn't enabled).

So far it appears to provide more consistent results and solutions in less generations than without, from limited testing. Some benchmarks may be useful as well, this will likely see more benefit on larger datasets.

The efficient can be a misnomer, as the flag will use additional computational resources than without, but it's a tradeoff to avoid wasting mutations and creating more diverse generations.

Tested both with evolve and custom evaluations using fitness function.

[D-Nice]

Also, any opinion on onlyPossibleMutations being a more fitting flag (tried to keep it short, and was not sure of all the side-effects), because as mentioned, it's not necessarily more efficient depending on other variables, especially if using a greater than one mutationAmount or if using all possible mutations, it will have to check all of them on each mutation selection.

[wagenaartje]

Seems like a good idea. Another way we could do this is just by passing the list of possible mutations to a new Network.prototype.efficientMutate function (while leaving the option open for 'inefficient' mutation).

This then selects a random mutation method from that list and attempts to mutate it through that method (by passing it to Network.prototype.mutate, which returns false or true), if it does not succeed, another mutation method from the list will be chosen (the previous one will be removed from the list). This continues until one of the methods is successful.

The advantages of this is that we don't check every mutation method. Instead, it is kind of like 'brute forcing'.

What do you think?

[D-Nice]

Yes, I had considered, and actually started with this method. I thought it more elegant to avoid the brute-force approach. Do you think the brute-force method will always be faster? The other way will tend to be more consistent in speed, maybe I'll see the ms performance hit of it checking all mutations and not checking them.

[wagenaartje]

I think it will be faster most of the times. The only requirements that network.mutate returns false as fast as possible when mutation is not possible. However we could do some speed comparisons. You're right about the consistency in speed that your method gives.

Should I make an implementation and compare them?

[D-Nice]

Yes, I'd be quite interested, and think that definitely the one with the higher average speed should be gone with. The brute-force method will probably be the fastest most times as you mentioned, only in the case where it runs through all the possible mutations like this current implementation, does it really have a chance to be slower, which should be rare.

Don't know if it's worth it to take deviation into account as well?