5.8, 5.6 and earlier had a good core perl compiler, but we try to be better.

Basic testsuite added, coverage TODO.
Bytecode compiler disabled for 5.6. 
The core compiler fails 50% but in mine is a blocker.

Same ok and less failures as with 5.6, 5.8 and 5.9.4.
In 5.8.8 and 5.8.9 we have much less errors in the testsuite for the
new CPAN compiler and the CORE compiler, just __DATA__ is still 
unsupported. See below.

Several B::C and B::CC bugs fixed. 5.10 compatibility added.

Open Problems:
5.6:
    reading from __DATA__ handles (15)
    test 25
    XSUB load order via B::Stash (?). Usually fixed via perlcc.
5.8:
    none else
5.10:
    +
    test 16
    destruction of static pvs for -O1
5.11:
    +
    special our handling: (tests 14 + 23)
See below at detailed 5.6, 5.8, 5.10, 5.11 status

TEST STATUS
-----------
Tested on cygwin1.7, centos5, debian4, freebsd7, solaris10.
List of failed tests.

CORE (old)
----
p=perl5.6.2
t/testc.sh -q -c
for t in $(seq -f"%02.0f" 26); do rm ccode 2>/dev/null; $p -MO=C,-occode.c ccode$t.pl 2>/dev/null && $p script/cc_harness ccode.c -o ccode >/dev/null; echo -n "$t: "; $p ccode$t.pl; echo -n " => "; ./ccode; echo; done
p=perl5.8.8
for t in $(seq -f"%02.0f" 26); do rm ccode 2>/dev/null; $p -MO=C,-occode.c ccode$t.pl 2>/dev/null && $p script/cc_harness -Bstatic ccode.c -o ccode >/dev/null; echo -n "$t: "; $p ccode$t.pl; echo -n " => "; ./ccode; echo; done
p=perl5.8.8-nt
for t in $(seq -f"%02.0f" 26); do rm ccode 2>/dev/null; $p -MO=C,-occode.c ccode$t.pl 2>/dev/null && $p script/cc_harness -Bdynamic ccode.c -o ccode >/dev/null; echo -n "$t: "; $p ccode$t.pl; echo -n " => "; ./ccode; echo; done

5.6.2:
t/bytecode      3,6,8..10,12,15,16,18,25,26
t/c             8,15,16,22
t/c_o1          8,15,16,22
t/c_o2          8,15,16,22
t/cc            15,18,21,25
t/cc_o1         15,18,21,25
t/cc_o2         10,15,16,18,21,25,26

5.8:                                    | non-threaded
t/bytecode      ok
t/c             11,14,15,20,23          | 5,7-12,14-20,22-23,25
t/c_o1          1,3-8,10-12,14,15,17-24 | 5,7-12,14-20,22-23,25
t/c_o2          1,3-12,14,15,17-25      | 5,7-12,14-20,22-23,25
t/cc            7,11,14,15,18-21,23-25
t/cc_o1         7,11,14,15,18-21,23-26
t/cc_o2         7,10,11,14-16,18-21,23-26

B::C (new)
----
t/testc.sh -q -c; t/testcc.sh -q -c or make test TEST_VERBOSE=1

5.6.2:					| non-threaded
t/bytecode      broken
t/c             15,25             	| 15
t/c_o1          15,22			| 15
t/c_o2          15,24			| 15
t/cc            15,18,21
t/cc_o1         15,18,21
t/cc_o2         10,15,16,18,21,26

5.8.8: (5.8.8 + 5.8.8-nt)		| non-threaded
t/bytecode      ok
t/c             15		     	| 15
t/c_o1          11,15  			| 15
t/c_o2          11,15	 		| 15
t/cc            15,18,21,25		| 15,18,21,25
t/cc_o1         15,18,21,25,26		| 15,18,21,25,26
t/cc_o2         10,15..16,18,21,25,26	| 10,15,16,18,21,25,26

5.8.9:
t/bytecode      ok
t/c             15
t/c_o1          15
t/c_o2          15
t/cc            15,18,21,25	 
t/cc_o1         15,18,21,25,26	 
t/cc_o2         10,15..16,18,21,25,26

5.10.1: (same results as non-threaded)
t/bytecode 	9..12,20
t/c 		15,16
t/c_o1          10,12,13,15..20,22
t/c_o2          10,12,13,15..20,22
t/cc 		12,15,16,18,21,25
t/cc_o1         12,15,16,18,21,25,26
t/cc_o2         10,12,15,16,18,21,25,26

5.11.2: (same results as non-threaded)
t/bytecode 	9..12,16,20,21
t/c 		14..16,23
t/c_o1          10,12,13,15..20,22
t/c_o2          10,12,13,15..20,22
t/cc 		12,15,16,18,21,25
t/cc_o1         15,16,18,21,25,26
t/cc_o2         10,15,16,18,21,25,26

5.10, 5.11 status
-----------------
TODO for B::C:
  __DATA__ IO (test 15)
  destruction: undef comppad_name, -O1 static PVs and HEKs
  Compiler errors: GvCV RV of stash (test 16)
  namesv: test 10 -O1, 14, 23

DONE:
index (fbm_compile) for GVs fixed with 1.04_31

test 14+23 destruction of GvPVX of the empty main:: stash fixed by adding a dummy.

cccode19 sort failure via custom sortcv fixed with B::CC 1.02_03. endless loop

autoload subs from main::, from packages it works ok (test 8)
	fixed with 1.04_25

panic: illegal pad in pad_new: 0x18c4368[0x18cf6e8] with DEBUGGING only
        CvPADLIST: curpad<=>comppad
        fixed with 1.04_22

pvx: seems to be fixed now in bc, and c
  With the move of the pvx field from xpv to the sv, we have to solve 
  that differently for the Bytecode and C backend.
  Bytecode can simply mimic the old XPV behaviour of a 3 field struct 
  (pvx, cur, len) to simplify pv handling.

hv: crash at invalid entry in hv_store in B::HV::save fixed

hek: new implementation, needs static optimization via heksect

regexp: match works, split still broken

bc 10: padv+sassign => Modification of a read-only value attempted at 
bytecode10.pl line 1. Only on cygwin, not on linux!
The bytecode is exactly the same, it must be pp_entersub() with &$cv()
Is FAKE flag of the padsv is missing or should we check for readonly pads?
  g  <1> entersub[t4] vKS/TARG,1
  => Perl_sv_force_normal_flags()
	if (SvREADONLY(sv) && (!SvFAKE(sv)) && (IN_PERL_RUNTIME)) => die
	SV = NULL(0x0) at 0x12207c0
	  REFCNT = 2147483129
	  FLAGS = (READONLY)
-Dt crash fixed by core patch pl-dump-const.patch

cop_io status?
  http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2006-05/msg00770.html

5.8 status
----------
I've restored 5.8 backwards compatibility, mainly for test comparison.
But since the new B::C compiler is better than the CORE compiler,
it is installed.

Details:

bytecode11 - 5.11d
assertion "isGV_with_GP(shplep)" failed: file "B.c", line 4961
#0  XS_B__GV_GP (my_perl=0x14222d0, cv=0x152a750) at B.c:15887
#1  0x521ca568 in Perl_pp_entersub () from /usr/local/bin/cygperl5_11d.dll
#2  0x52165d4c in Perl_runops_debug () from /usr/local/bin/cygperl5_11d.dll
#3  0x521a9730 in Perl_call_sv () from /usr/local/bin/cygperl5_11d.dll
#4  0x521b1640 in Perl_call_list () from /usr/local/bin/cygperl5_11d.dll
#5  0x521a596f in perl_parse () from /usr/local/bin/cygperl5_11d.dll
#6  0x00401136 in main (argc=4, argv=0x14221a0, env=0x1420090)
(gdb) p *gv
$2 = {sv_any = 0x14d1d48, sv_refcnt = 3, sv_flags = 32777 (0x8009), sv_u = {
    svu_iv = -5931823458977729848, svu_uv = 12514920614731821768,
    svu_rv = 0x14db2c8, svu_pv = 0x14db2c8 "E\001", svu_array = 0x14db2c8,
    svu_hash = 0x14db2c8, svu_gp = 0x14db2c8}}

cccode3.c - 5.11 dstr assert
	XPUSHs(GvSV(PL_curpad[1])); /* oops, this GV is empty */
	/* stack =  */
	/* BINOP (0x15ec140) sassign [OPf_STACKED] */
	dst = POPs; src = TOPs; /* empty var dst at stack ! */
	MAYBE_TAINT_SASSIGN_SRC(src);
	SvSetSV(dst, src);

ccode3 - 5.10 (fixed via workaround, dynamic init)
-------------
SEGV     $_ = "xyxyx"; %j=(1,2); s/x/$j{print("z")}/g; print $_ 
empty data. cause: no pv flag and refcnt in the sv
#0  0x004a38c3 in Perl_fbm_instr (my_perl=0x18926d0, big=0x18d4bc8 "xyxyx",
    bigend=0x18d4bcd "", littlestr=0x0, flags=0) at util.c:577
#1  0x006481d4 in Perl_re_intuit_start (my_perl=0x18926d0, prog=0x18d2088,
    sv=0x18d3fb8, strpos=0x18d4bc8 "xyxyx", strend=0x18d4bcd "", flags=0,
    data=0x0) at regexec.c:561
#2  0x005a13f3 in Perl_pp_subst (my_perl=0x18926d0) at pp_hot.c:2105
(gdb) p *rx
$7 = {engine = 0x6a06a0, mother_re = 0x0, extflags = 1126170624, minlen = 0,
  minlenret = 0, gofs = 0, substrs = 0x18c0908, nparens = 0, intflags = 0,
  pprivate = 0x18d1588, lastparen = 0, lastcloseparen = 0, swap = 0x0,
  offs = 0x18cf748, subbeg = 0x0, sublen = 0, prelen = 0,
  precomp = 0x18ab870 ")", wrapped = 0x18ab868 "(?-xism:)", wraplen = 9,
  seen_evals = 0, paren_names = 0x0, refcnt = 1}

ccode2 - 5.11 (fixed via workaround, dynamic init)
-------------
match with /\d/
(gdb) p *pm
$1 = {op_next = 0x66513c, op_sibling = 0x665198, op_ppaddr = 0x59a6e9 <Perl_pp_match>, op_targ = 0, op_type = 31,
  op_opt = 0, op_latefree = 1, op_latefreed = 0, op_attached = 0, op_spare = 0, op_flags = 2 '\002',
  op_private = 64 '@', op_first = 0x0, op_last = 0x0, op_pmoffset = 46, op_pmflags = 0, op_pmreplrootu = {
    op_pmreplroot = 0x0, op_pmtargetoff = 0}, op_pmstashstartu = {op_pmreplstart = 0x0, op_pmstashpv = 0x0}}
(gdb) n
1216        register REGEXP *rx = PM_GETRE(pm);
(gdb) p *rx
$2 = {engine = 0x69f690, mother_re = 0x0, extflags = 52428800, minlen = 1, minlenret = 1, gofs = 0,
  substrs = 0x1920188, nparens = 0, intflags = 0, pprivate = 0x1915448, lastparen = 0, lastcloseparen = 0,
  swap = 0x0, offs = 0x191f6c8, subbeg = 0x0, sublen = 0, prelen = 1, precomp = 0x18fb870 "0)",
  wrapped = 0x18fb868 "(?-xism:0)", wraplen = 10, seen_evals = 0, paren_names = 0x0, refcnt = 1}

TEST COVERAGE
-------------
Bytecode:
coverage for ret(0) ldsv(1) ldop(2) stsv(3) stop(4) stpv(5) ldspecsv(6) 
ldspecsvx(7) newsv(8) newsvx(9) nop(10) newop(11) newopx(12) newopn(13) 
newpv(14) pv_cur(15) pv_free(16) sv_upgrade(17) sv_refcnt(18) sv_refcnt_add(19) 
sv_flags(20) xrv(21) xpv(22) xpv_cur(23) xpv_len(24) xiv(25) xnv(26) 
xlv_targoff(27) xlv_targlen(28) xlv_targ(29) xlv_type(30) xbm_useful(31) 
xbm_previous(32) xbm_rare(33) xfm_lines(34) comment(35) xio_lines(36) xio_page(37) 
xio_page_len(38) xio_lines_left(39) xio_top_name(40) xio_top_gv(41) xio_fmt_name(42) 
xio_fmt_gv(43) xio_bottom_name(44) xio_bottom_gv(45) xio_type(46) xio_flags(47) 
xcv_xsubany(48) xcv_stash(49) xcv_start(50) xcv_root(51) xcv_gv(52) xcv_file(53) 
xcv_depth(54) xcv_padlist(55) xcv_outside(56) xcv_outside_seq(57) xcv_flags(58) 
av_extend(59) av_pushx(60) av_push(61) xav_fill(62) xav_max(63) xav_flags(64) 
xhv_name(65) hv_store(66) sv_magic(67) mg_obj(68) mg_private(69) mg_flags(70) 
mg_name(71) mg_namex(72) xmg_stash(73) gv_fetchpv(74) gv_fetchpvx(75) gv_stashpv(76) 
gv_stashpvx(77) gp_sv(78) gp_refcnt(79) gp_refcnt_add(80) gp_av(81) gp_hv(82) 
gp_cv(83) gp_file(84) gp_io(85) gp_form(86) gp_cvgen(87) gp_line(88) gp_share(89) 
xgv_flags(90) op_next(91) op_sibling(92) op_ppaddr(93) op_targ(94) op_type(95) 
op_opt(96) op_latefree(97) op_latefreed(98) op_attached(99) op_first(102) op_last(103) 
op_pmreplroot(105) op_pmreplstart(106) op_pmreplrootpo(108) op_pmstash(109) 
op_pmreplrootgv(110) pregcomp(111) op_pmflags(112) unused(113) op_reflags(114) 
op_sv(115) op_pv(117) op_pv_tr(118) op_redoop(119) op_nextop(120) op_lastop(121) 
cop_label(122) cop_stash(125) cop_filegv(126) push_begin(134) push_init(135) 
push_end(136) curstash(137) defstash(138) data(139) incav(140) load_glob(141) 
regex_padav(142) comppad_name(144) xgv_stash(145) signal(146) formfeed(147)

JIT PLANS
---------
B-C-1.04_10:
Playing with the idea to target against parrot pir instead of lightning
and use the parrot jit, with pbc_to_exe or exec_save.
Then we need no jit support in the ByteLoader and get rid of all the Jit 
stuff for now. 
=> B::PIR, Doing mentoring for Google Soc 2008

B-C-1.04_03:
added Jit and Asm layout, with the idea of using either 
lightning as jit backend (Jit), or creating our own assembler (Asm) 
to be able to use a high-level language.

Abbrevations
------------
bc B::ByteCode 
c  B::C
cc B::CC, 
-D -DDEBUGGING perl, also used as d version suffix, eg 5.8.8d
-nt not threaded perl.

2009-12-14 22:03:25 rurban
