maxSize argument

[Nick Smallbone]

Hi Moritz,

On 24 July 2012 13:57, Moritz Fürneisen <fuerneis at cs.uni-bonn.de> wrote:
>         Test.QuickCheck> quickCheckWith stdArgs{maxSize = 4, maxSuccess = 50000} (( \x -> property $  ((x >= -4) &&  (x <= 4))) :: Int -> Property )
>         +++ OK, passed 50000 tests.
>
>         Test.QuickCheck> quickCheckWith stdArgs{maxSize = 4, maxSuccess = 50000} (( \x -> x >= 0 ==>  (x <= 4)) :: Int -> Property )
>         *** Failed! Falsifiable (after 52 tests and 2 shrinks):
>         5
>
> Is this a bug or a feature(like simulated annealing)?
> If it is a feature, is there a possibility to disable it?
> I wasn't able to determine from the sources, why the number of discarded tests relates to testsize.

Ah, you are right! What happens is that if you use ==>, and a lot of
tests are discarded (the precondition is false), QuickCheck will
eventually increase the test size. For every ten discarded tests, the
size goes up by one.

The reason for this is in case you write a property like
   prop_something xs = xs /= [] ==> ...
If the size is 0, xs will always be [] and the test case will be
discarded. So you need QuickCheck to increase the size to 1
eventually, even if all test cases were discarded.

The problem is that once QuickCheck increased the size after
discarding tests, it would never decrease it again! So if it had
discarded 500 tests, the size would be permanently increased by 50. It
would also increase the test size beyond maxSize, like you pointed
out.

So I've pushed a patch that fixes these two problems. The test size
will never go beyond maxSize, and after a sequence of discarded tests
has increased the size, the next successful test will reset the size
again.

Nick