t/response.t randomly fails

[kentfredric]

I had this failure occur, and then I responded "retry" to my cpanm prompt and it went away:

#   Failed test at t/response.t line 143.
#          got: '180.2'
#     expected: '180'
# Looks like you failed 1 test of 68.
t/response.t ................ 
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/68 subtests 

Test Summary Report
-------------------
t/response.t              (Wstat: 256 Tests: 68 Failed: 1)
  Failed test:  47
  Non-zero exit status: 1
Files=17, Tests=772,  2 wallclock secs ( 0.19 usr  0.04 sys +  2.81 cusr  0.47 csys =  3.51 CPU)]

There's one reproduction on CPAN Testers on Dragonfly BSD with Perl 5.18.4

But I'm failing here on Linux + 5.27.9

Summary of my perl5 (revision 5 version 27 subversion 9) configuration:
   
  Platform:
    osname=linux
    osvers=4.10.13-gentoo
    archname=x86_64-linux
    uname='linux katipo2 4.10.13-gentoo #48 smp preempt tue may 2 15:18:23 nzst 2017 x86_64 intel(r) core(tm) i5-2410m cpu @ 2.30ghz genuineintel gnulinux '
    config_args='-de -Dprefix=/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot -Ddefault_inc_excludes_dot -Doptimize= -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native -Dman1dir=none -Dman3dir=none -Dusedevel -Accflags= -fstack-protector-strong -fno-stack-protector -DPERL_DISABLE_PMC -Aldflags= -fstack-protector-strong -fno-stack-protector -Aeval:scriptdir=/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/bin'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-fstack-protector-strong -fno-stack-protector -DPERL_DISABLE_PMC -fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    optimize=' -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native'
    cppflags='-fstack-protector-strong -fno-stack-protector -DPERL_DISABLE_PMC -fwrapv -fno-strict-aliasing -pipe'
    ccversion=''
    gccversion='7.3.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='  -fstack-protector-strong -fno-stack-protector -L/usr/local/lib'
    libpth=/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/include-fixed /usr/lib /usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lpthread -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.26.so
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.26'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared  -fstack-protector-strong -fno-stack-protector -O3 -march=native -mtune=native -L/usr/local/lib'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DISABLE_PMC
    PERL_DONT_CREATE_GVSV
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under linux
  Compiled at Mar 19 2018 14:35:38
  %ENV:
    PERLBREW_BASHRC_VERSION="0.80"
    PERLBREW_HOME="/home/kent/.perlbrew"
    PERLBREW_MANPATH="/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/man"
    PERLBREW_PATH="/home/kent/perl5/perlbrew/bin:/home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/bin"
    PERLBREW_PERL="5.27.9-nossp-oaath-nopmc-nodot"
    PERLBREW_ROOT="/home/kent/perl5/perlbrew"
    PERLBREW_VERSION="0.80"
  @INC:
    /home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/lib/site_perl/5.27.9/x86_64-linux
    /home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/lib/site_perl/5.27.9
    /home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/lib/5.27.9/x86_64-linux
    /home/kent/perl5/perlbrew/perls/5.27.9-nossp-oaath-nopmc-nodot/lib/5.27.9

[kentfredric]

My test failure: http://www.cpantesters.org/cpan/report/60e196d0-2b1e-11e8-ba7a-ff0ab828c311

[eserte]

Just saw a variation of this failure (perl 5.20.2, debian/jessie):

#   Failed test at t/response.t line 142.
#          got: '90.1'
#     expected: '90'

#   Failed test at t/response.t line 143.
#          got: '180.2'
#     expected: '180'
# Looks like you failed 2 tests of 68.
t/response.t ................ 
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/68 subtests 

[eserte]

Looks like a timing issue. May be reproduced by putting a sleep() within the test, e.g.

diff --git a/t/response.t b/t/response.t
index 2412e52..4f5a2fd 100644
--- a/t/response.t
+++ b/t/response.t
@@ -139,6 +139,7 @@ is($r2->freshness_lifetime(h_default => 900), 900);
 is($r2->freshness_lifetime(h_min => 7200), 7200);
 is($r2->freshness_lifetime(time => time), 3600);
 $r2->last_modified(time - 900);
+sleep 1;
 is($r2->freshness_lifetime, 90);
 is($r2->freshness_lifetime(h_lastmod_fraction => 0.2), 180);
 is($r2->freshness_lifetime(h_min => 300), 300);

[eserte]

A possible solution is to hack around system's time() to make sure that the clock does not move: eserte@e045b2c

However, I feel somewhat uncomfortable using CORE::GLOBAL stuff and overriding builtins. Maybe a simpler and more pragmatic solution would be to just accept both return values, 90 and 90.1 resp. 180 and 180.2.

[karenetheridge]

I like this module - https://metacpan.org/pod/Test::MockTime

[eserte]

However not perfect: http://matrix.cpantesters.org/?dist=Test-MockTime%200.17;reports=1
(This reminds me that I wanted to create an issue about the possible test failures)

[eserte]

Here's another possible solution: eserte@d57c758

The test code does not look pretty anymore, but it is quite robust against all kind of delays (try sleep $something with any value after the last_modified call, it should always work).

[oalders]

Here's another possible solution: eserte/HTTP-Message@d57c758

LGTM.

[neilb]

How about using Test::Number::Delta, so

is($r2->freshness_lifetime, 90);

becomes something like:

use Test::Number::Delta qw/ delta_within /;
delta_within($r2->freshness_lifetime, 90, 0.2);

[oalders]

Sounds great!