Validation patience is the number of epochs that the algorithm tries to improve the performance before giving up (if the error is not decreasing).
For example:
Epoch: ... 191 192 193 194 195 196 197 198 199 200 201 202 203 204
Performance ... 10 9 2 1.2 1.2 1.2 1.2 0.8 0.7 0.7 0.7 0.7 0.7 0.7
In this case, if validation patience were 2, the algorithm would stop at epoch 196. If it were 5, it would stop at epoch 204.
Normally, it is assumed that if the performance does not decrease for a few epochs in a row, then a minimum has been found. However, in rare cases where the minimum can be surpassed with some 'patience', a high number for validation patience can help the network attain lower performances.
There is a trade off between waiting a long time for nothing and missing out on better performances.
I personally leave ValidationPatience at 6.
