segmentation fault in pure perl

[p5pRT]

Migrated from rt.perl.org#77082 (status was 'open')

Searchable as RT77082$

[p5pRT]

From @mauke

Created by @mauke

So pippijn on freenode/#perl had this program that made his perl
segfault. We reduced it to the following​:

#!perl

use strict; # not needed

sub list_rule {
  my ($term, $sep) = @​_;

  {
  rhs => [
  {
  name => 'term',
  },
  ],
  },
  {
  rhs => [
  {
  expr => $term . '_list',
  },
  ],
  }
}

my %lists;

sub quantise {
  my ($term) = @​_;

  if (my ($sep) = $term =~ s/(?​:\[(.*)\])?\+$/__list/) {
  @​{ $lists{$term} } = list_rule
  }

}

sub assemble_rules {
  my ($nodes, $rule) = @​_;

  map { quantise $_->{expr} } @​{ $rule->{rhs} };

  0 for ();
}

sub gen_yacc_rules {
  my ($rules) = @​_;

  while (my ($nterm, $rule) = each %$rules) {
  map { assemble_rules undef, $_ } @​$rule
  }
}

sub gen_rules {
  my $rules = {
  foo => [{
  rhs => [{ }]
  }]
  };

  gen_yacc_rules $rules;
  gen_yacc_rules \%lists;
}

gen_rules;

__END__

This segfaults his 5.10 as well as my 5.12.1 and my system perl 5.8.8.

The latter doesn't exactly segfault on this version of the code; instead
I get this​:

*** glibc detected *** /usr/bin/perl​: munmap_chunk()​: invalid pointer​: 0x097cf574 ***
======= Backtrace​: =========
/lib/libc.so.6[0x4a450861]
/lib/libc.so.6[0x4a451aee]
/usr/bin/perl(Perl_safesysfree+0x21)[0x80b0d31]
/usr/bin/perl(Perl_mg_free+0x32)[0x80b23a2]
/usr/bin/perl(Perl_sv_clear+0x2dc)[0x80c7ffc]
/usr/bin/perl(Perl_sv_free+0xbe)[0x80c865e]
/usr/bin/perl(Perl_leave_scope+0x1b2)[0x80e4862]
/usr/bin/perl(Perl_pop_scope+0x3a)[0x80e541a]
/usr/bin/perl(Perl_pp_leaveloop+0x13b)[0x80e91ab]
/usr/bin/perl(Perl_runops_standard+0x25)[0x80bcd75]
/usr/bin/perl(perl_run+0x2f7)[0x8068e17]
/usr/bin/perl(main+0xf6)[0x8062f36]
/lib/libc.so.6(__libc_start_main+0xe6)[0x4a3fbbb6]
/usr/bin/perl[0x8062d81]
======= Memory map​: ========
08047000-08143000 r-xp 00000000 08​:06 11020884 /usr/bin/perl5.8.8
08143000-08144000 r-xp 000fc000 08​:06 11020884 /usr/bin/perl5.8.8
08144000-08147000 rwxp 000fd000 08​:06 11020884 /usr/bin/perl5.8.8
0814700-0814a000 rwxp 0814700 00​:00 0
097b3000-097d4000 rwxp 097b3000 00​:00 0 [heap]
4a3c5000-4a3e1000 r-xp 00000000 08​:06 4614127 /lib/ld-2.11.2.so
4a3e1000-4a3e2000 r-xp 0001b000 08​:06 4614127 /lib/ld-2.11.2.so
4a3e2000-4a3e3000 rwxp 0001c000 08​:06 4614127 /lib/ld-2.11.2.so
4a3e5000-4a525000 r-xp 00000000 08​:06 4614130 /lib/libc-2.11.2.so
4a525000-4a527000 r-xp 0013f000 08​:06 4614130 /lib/libc-2.11.2.so
4a527000-4a528000 rwxp 00141000 08​:06 4614130 /lib/libc-2.11.2.so
4a528000-4a52b000 rwxp 4a528000 00​:00 0
4a52d000-4a551000 r-xp 00000000 08​:06 2287197 /lib/libm-2.11.2.so
4a551000-4a552000 r-xp 00023000 08​:06 2287197 /lib/libm-2.11.2.so
4a552000-4a553000 rwxp 00024000 08​:06 2287197 /lib/libm-2.11.2.so
4a555000-4a557000 r-xp 00000000 08​:06 2285536 /lib/libdl-2.11.2.so
4a557000-4a558000 r-xp 00001000 08​:06 2285536 /lib/libdl-2.11.2.so
4a558000-4a559000 rwxp 00002000 08​:06 2285536 /lib/libdl-2.11.2.so
4a55b000-4a570000 r-xp 00000000 08​:06 2287028 /lib/libpthread-2.11.2.so
4a570000-4a571000 r-xp 00014000 08​:06 2287028 /lib/libpthread-2.11.2.so
4a571000-4a572000 rwxp 00015000 08​:06 2287028 /lib/libpthread-2.11.2.so
4a572000-4a574000 rwxp 4a572000 00​:00 0
4b28d000-4b2a0000 r-xp 00000000 08​:06 2287277 /lib/libnsl-2.11.2.so
4b2a0000-4b2a1000 r-xp 00012000 08​:06 2287277 /lib/libnsl-2.11.2.so
4b2a1000-4b2a2000 rwxp 00013000 08​:06 2287277 /lib/libnsl-2.11.2.so
4b2a2000-4b2a4000 rwxp 4b2a2000 00​:00 0
4b5b2000-4b5b4000 r-xp 00000000 08​:06 2287276 /lib/libutil-2.11.2.so
4b5b4000-4b5b5000 r-xp 00001000 08​:06 2287276 /lib/libutil-2.11.2.so
4b5b5000-4b5b6000 rwxp 00002000 08​:06 2287276 /lib/libutil-2.11.2.so
4b5cb000-4b5d4000 r-xp 00000000 08​:06 2287275 /lib/libcrypt-2.11.2.so
4b5d4000-4b5d5000 r-xp 00008000 08​:06 2287275 /lib/libcrypt-2.11.2.so
4b5d5000-4b5d6000 rwxp 00009000 08​:06 2287275 /lib/libcrypt-2.11.2.so
4b5d6000-4b5fd000 rwxp 4b5d6000 00​:00 0
b7d1c000-b7f1c000 r-xp 00000000 08​:06 11054077 /usr/lib/locale/locale-archive
b7f1c000-b7f1f000 rwxp b7f1c000 00​:00 0
b7f29000-b7f41000 r-xp 00000000 08​:06 8652675 /home/mauke/usr/local/lib/libgcc_s.so.1
b7f41000-b7f42000 r-xp 00017000 08​:06 8652675 /home/mauke/usr/local/lib/libgcc_s.so.1
b7f42000-b7f43000 rwxp 00018000 08​:06 8652675 /home/mauke/usr/local/lib/libgcc_s.so.1
b7f43000-b7f44000 rwxp b7f43000 00​:00 0
b7f44000-b7f45000 r-xp b7f44000 00​:00 0 [vdso]
bfc2f000-bfc45000 rw-p bffea000 00​:00 0 [stack]
zsh​: abort /usr/bin/perl irc.pl

Anyway, I think perl shouldn't crash here.

Perl Info

Flags:
    category=core
    severity=medium

Site configuration information for perl 5.12.1:

Configured by mauke at Thu Jun  3 20:09:15 CEST 2010.

Summary of my perl5 (revision 5 version 12 subversion 1) configuration:
   
  Platform:
    osname=linux, osvers=2.6.27-gentoo-r8, archname=i686-linux
    uname='linux nora 2.6.27-gentoo-r8 #4 preempt mon apr 13 21:59:34 cest 2009 i686 amd athlon(tm) 64 processor 3200+ authenticamd gnulinux '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='gcc', ccflags ='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -march=native',
    cppflags='-fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.4.3', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='gcc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
    libc=/lib/libc-2.10.1.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.10.1'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -march=native -L/usr/local/lib -fstack-protector'

Locally applied patches:
    


@INC for perl 5.12.1:
    /home/mauke/usr/local/lib/perl5/site_perl/5.12.1/i686-linux
    /home/mauke/usr/local/lib/perl5/site_perl/5.12.1
    /home/mauke/usr/local/lib/perl5/5.12.1/i686-linux
    /home/mauke/usr/local/lib/perl5/5.12.1
    /home/mauke/usr/local/lib/perl5/site_perl/5.10.1
    /home/mauke/usr/local/lib/perl5/site_perl
    .


Environment for perl 5.12.1:
    HOME=/home/mauke
    LANG=en_US.UTF-8
    LANGUAGE (unset)
    LC_COLLATE=POSIX
    LD_LIBRARY_PATH=/home/mauke/usr/local/lib
    LOGDIR (unset)
    PATH=/home/mauke/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/4.4.3:/opt/sun-jdk-1.4.2.13/bin:/opt/sun-jdk-1.4.2.13/jre/bin:/opt/sun-jdk-1.4.2.13/jre/javaws:/opt/dmd/bin:/usr/games/bin:/opt/vmware/server/bin
    PERL_BADLANG (unset)
    PERL_UNICODE=SAL
    SHELL=/bin/bash

[p5pRT]

From @avar

On Sun, Aug 8, 2010 at 11​:57, l.mai@​web.de <perlbug-followup@​perl.org> wrote​:

-----------------------------------------------------------------
[Please describe your issue here]

So pippijn on freenode/#perl had this program that made his perl
segfault. We reduced it to the following​:

#!perl

use strict; # not needed

sub list_rule {
  my ($term, $sep) = @​_;

  {
     rhs => [
        {
           name => 'term',
        },
     ],
  },
  {
     rhs => [
        {
           expr => $term . '_list',
        },
     ],
  }
}

my %lists;

sub quantise {
  my ($term) = @​_;

  if (my ($sep) = $term =~ s/(?​:\[(.*)\])?\+$/__list/) {
     @​{ $lists{$term} } = list_rule
  }

}

sub assemble_rules {
  my ($nodes, $rule) = @​_;

  map { quantise $_->{expr} } @​{ $rule->{rhs} };

  0 for ();
}

sub gen_yacc_rules {
  my ($rules) = @​_;

  while (my ($nterm, $rule) = each %$rules) {
     map { assemble_rules undef, $_ } @​$rule
  }
}

sub gen_rules {
  my $rules = {
     foo => [{
        rhs => [{ }]
     }]
  };

  gen_yacc_rules $rules;
  gen_yacc_rules \%lists;
}

gen_rules;

__END__

This segfaults his 5.10 as well as my 5.12.1 and my system perl 5.8.8.

This will fail an assert on blead compiled with debugging flags, or
segfault otherwise. It fails this assert in Perl_sv_clear​:

  @​@​ -5788,7 +5788,10 @​@​ Perl_sv_clear(pTHX_ register SV *const sv)

  PERL_ARGS_ASSERT_SV_CLEAR;
  assert(SvREFCNT(sv) == 0);
  assert(SvTYPE(sv) != SVTYPEMASK);

Without the debugging assert it'll continue to​:

  } else if (SvMAGIC(sv))
  mg_free(sv);

Where it'll segfault on​:

  Perl_mg_free(pTHX_ SV *sv)
  {
  MAGIC* mg;
  MAGIC* moremagic;

  PERL_ARGS_ASSERT_MG_FREE;

  for (mg = SvMAGIC(sv); mg; mg = moremagic) {
  => const MGVTBL* const vtbl = mg->mg_virtual;

That's not magic​:

  (gdb) p mg->mg_virtual
  Cannot access memory at address 0xff00000008

Looks like perl is trying to free an already freed scalar.

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From @iabyn

On Sun, Aug 08, 2010 at 12​:23​:14PM +0000, Ævar Arnfjörð Bjarmason wrote​:

On Sun, Aug 8, 2010 at 11​:57, l.mai@​web.de <perlbug-followup@​perl.org> wrote​:
This will fail an assert on blead compiled with debugging flags, or
segfault otherwise. It fails this assert in Perl_sv_clear​:

@​@​ \-5788\,7 \+5788\,10 @​@​ Perl\_sv\_clear\(pTHX\_ register SV \*const sv\)

     PERL\_ARGS\_ASSERT\_SV\_CLEAR;
     assert\(SvREFCNT\(sv\) == 0\);
     assert\(SvTYPE\(sv\) \!= SVTYPEMASK\);

Without the debugging assert it'll continue to​:

  \} else if \(SvMAGIC\(sv\)\)
      mg\_free\(sv\);

Where it'll segfault on​:

Perl\_mg\_free\(pTHX\_ SV \*sv\)
\{
    MAGIC\* mg;
    MAGIC\* moremagic;

    PERL\_ARGS\_ASSERT\_MG\_FREE;

    for \(mg = SvMAGIC\(sv\); mg; mg = moremagic\) \{
=>      const MGVTBL\* const vtbl = mg\->mg\_virtual;

That's not magic​:

\(gdb\) p mg\->mg\_virtual
Cannot access memory at address 0xff00000008

Looks like perl is trying to free an already freed scalar.

The assert-failing test can be reduced to

  my @​a = ( {}, {} );

  sub f {
  my ($x) = @​_;
  @​a = ( {}, {} );
  0 for ();
  }

  map { f $_ } @​a;

which is just an instance of the "don't modify an array you're iterating
over" bug​: essentially unfixable without making the stack reference
counted.

--
In the 70's we wore flares because we didn't know any better.
What possible excuse does the current generation have?

[p5pRT]

From @avar

On Wed, Aug 18, 2010 at 16​:43, Dave Mitchell <davem@​iabyn.com> wrote​:

On Sun, Aug 08, 2010 at 12​:23​:14PM +0000, Ęvar Arnfjörš Bjarmason wrote​:

On Sun, Aug 8, 2010 at 11​:57, l.mai@​web.de <perlbug-followup@​perl.org> wrote​:
This will fail an assert on blead compiled with debugging flags, or
segfault otherwise. It fails this assert in Perl_sv_clear​:

    @​@​ -5788,7 +5788,10 @​@​ Perl_sv_clear(pTHX_ register SV *const sv)

         PERL_ARGS_ASSERT_SV_CLEAR;
         assert(SvREFCNT(sv) == 0);
         assert(SvTYPE(sv) != SVTYPEMASK);

Without the debugging assert it'll continue to​:

      } else if (SvMAGIC(sv))
          mg_free(sv);

Where it'll segfault on​:

    Perl_mg_free(pTHX_ SV *sv)
    {
        MAGIC* mg;
        MAGIC* moremagic;

        PERL_ARGS_ASSERT_MG_FREE;

        for (mg = SvMAGIC(sv); mg; mg = moremagic) {
    =>      const MGVTBL* const vtbl = mg->mg_virtual;

That's not magic​:

    (gdb) p mg->mg_virtual
    Cannot access memory at address 0xff00000008

Looks like perl is trying to free an already freed scalar.

The assert-failing test can be reduced to

   my @​a = ( {}, {} );

   sub f {
       my ($x) = @​_;
       @​a =  ( {}, {} );
       0 for ();
   }

   map { f $_ } @​a;

which is just an instance of the "don't modify an array you're iterating
over" bug​: essentially unfixable without making the stack reference
counted.

Even if that's unfixable we could still do better than segfault here,
maybe that assert needs to be turned into a croak() ?

Is the assert only reached due to this sort of bug? If so perhaps a
more pertinent error message is in order.

[p5pRT]

From @iabyn

On Thu, Aug 19, 2010 at 10​:59​:36PM +0000, Ævar Arnfjörð Bjarmason wrote​:

On Wed, Aug 18, 2010 at 16​:43, Dave Mitchell <davem@​iabyn.com> wrote​:
Even if that's unfixable we could still do better than segfault here,
maybe that assert needs to be turned into a croak() ?

Maybe; it increases the cost of every SV freeing, though.

Is the assert only reached due to this sort of bug? If so perhaps a
more pertinent error message is in order.

That assert fails any time an already-freed SV is freed. Any internal
inconsistency or bug could potentially trigger it.

--
A walk of a thousand miles begins with a single step...
then continues for another 1,999,999 or so.