Blame HACKERS

Packit 7d6a7d
=head1 NAME
Packit 7d6a7d
Packit 7d6a7d
HACKERS - Devel::PPPort internals for hackers
Packit 7d6a7d
Packit 7d6a7d
=head1 SYNOPSIS
Packit 7d6a7d
Packit 7d6a7d
So you probably want to hack C<Devel::PPPort>?
Packit 7d6a7d
Packit 7d6a7d
Well, here's some information to get you started with what's
Packit 7d6a7d
lying around in this distribution.
Packit 7d6a7d
Packit 7d6a7d
=head1 DESCRIPTION
Packit 7d6a7d
Packit 7d6a7d
=head2 How to build 366 versions of Perl
Packit 7d6a7d
Packit 7d6a7d
C<Devel::PPPort> supports Perl versions between 5.003 and bleadperl.
Packit 7d6a7d
To guarantee this support, I need some of these versions on my
Packit 7d6a7d
machine. I currently have 366 different Perl version/configuration
Packit 7d6a7d
combinations installed on my laptop.
Packit 7d6a7d
Packit 7d6a7d
As many of the old Perl distributions need patching to compile
Packit 7d6a7d
cleanly on newer systems (and because building 366 Perls by hand
Packit 7d6a7d
just isn't fun), I wrote a tool to build all the different
Packit 7d6a7d
versions and configurations. You can find it in F<devel/buildperl.pl>.
Packit 7d6a7d
It can currently build the following Perl releases:
Packit 7d6a7d
Packit 7d6a7d
    5.003
Packit 7d6a7d
    5.004 - 5.004_05
Packit 7d6a7d
    5.005 - 5.005_04
Packit 7d6a7d
    5.6.x
Packit 7d6a7d
    5.7.x
Packit 7d6a7d
    5.8.x
Packit 7d6a7d
    5.9.x
Packit 7d6a7d
    5.1x.x
Packit 7d6a7d
Packit 7d6a7d
=head2 Fully automatic API checks
Packit 7d6a7d
Packit 7d6a7d
Knowing which parts of the API are not backwards compatible and
Packit 7d6a7d
probably need C<Devel::PPPort> support is another problem that's
Packit 7d6a7d
not easy to deal with manually. If you run
Packit 7d6a7d
Packit 7d6a7d
    perl Makefile.PL --with-apicheck
Packit 7d6a7d
Packit 7d6a7d
a C file is generated by F<parts/apicheck.pl> that is compiled
Packit 7d6a7d
and linked with C<Devel::PPPort>. This C file has the purpose of
Packit 7d6a7d
using each of the public API functions/macros once.
Packit 7d6a7d
Packit 7d6a7d
The required information is derived from C<parts/embed.fnc> (just
Packit 7d6a7d
a copy of bleadperl's C<embed.fnc>), C<parts/apidoc.fnc> (which
Packit 7d6a7d
is generated by F<devel/mkapidoc.sh> and simply collects the rest
Packit 7d6a7d
of the apidoc entries spread over the Perl source code) and
Packit 7d6a7d
C<parts/ppport.fnc> (which lists all API provided purely by
Packit 7d6a7d
Devel::PPPort).
Packit 7d6a7d
The generated C file C<apicheck.c> is currently about 500k in size
Packit 7d6a7d
and takes quite a while to compile.
Packit 7d6a7d
Packit 7d6a7d
Usually, C<apicheck.c> won't compile with older perls. And even if
Packit 7d6a7d
it compiles, there's still a good chance of the dynamic linker
Packit 7d6a7d
failing at C<make test> time. But that's on purpose!
Packit 7d6a7d
Packit 7d6a7d
We can use these failures to find changes in the API automatically.
Packit 7d6a7d
The two Perl scripts F<devel/mktodo> and F<devel/mktodo.pl>
Packit 7d6a7d
repeatedly run C<Devel::PPPort> with the apicheck code through
Packit 7d6a7d
all different versions of perl. Scanning the output of the compiler
Packit 7d6a7d
and the dynamic linker for errors, the files in F<parts/todo/> are
Packit 7d6a7d
generated. These files list all parts of the public API that don't
Packit 7d6a7d
work with less than a certain version of Perl.
Packit 7d6a7d
Packit 7d6a7d
This information is in turn used by F<parts/apicheck.pl> to mask
Packit 7d6a7d
API calls in the generated C file for these versions, so the
Packit 7d6a7d
process can be stopped by the time F<apicheck.c> compiles cleanly
Packit 7d6a7d
and the dynamic linker is happy. (Actually, this process may generate
Packit 7d6a7d
false positives, so by default each API call is checked once more
Packit 7d6a7d
afterwards.)
Packit 7d6a7d
Packit 7d6a7d
Running C<devel/mktodo> takes about an hour, depending of course
Packit 7d6a7d
on the machine you're running it on. If you run it with
Packit 7d6a7d
the C<--nocheck> option, it won't recheck the API calls that failed
Packit 7d6a7d
in the compilation stage and it'll take significantly less time.
Packit 7d6a7d
Running with C<--nocheck> should usually be safe.
Packit 7d6a7d
Packit 7d6a7d
When running C<devel/mktodo> with the C<--base> option, it will
Packit 7d6a7d
generate the I<baseline> todo files by disabling all functionality
Packit 7d6a7d
provided by C<Devel::PPPort>. These are required for implementing
Packit 7d6a7d
the C<--compat-version> option of the C<ppport.h> script. The
Packit 7d6a7d
baseline todo files hold the information about which version of
Packit 7d6a7d
Perl lacks a certain part of the API.
Packit 7d6a7d
Packit 7d6a7d
However, only the documented public API can be checked this way.
Packit 7d6a7d
And since C<Devel::PPPort> provides more macros, these would not be
Packit 7d6a7d
affected by C<--compat-version>. It's the job of F<devel/scanprov>
Packit 7d6a7d
to figure out the baseline information for all remaining provided
Packit 7d6a7d
macros by scanning the include files in the F<CORE> directory of
Packit 7d6a7d
various Perl versions.
Packit 7d6a7d
Packit 7d6a7d
The whole process isn't platform independent. It has currently been
Packit 7d6a7d
tested only under Linux, and it definitely requires at least C<gcc> and
Packit 7d6a7d
the C<nm> utility.
Packit 7d6a7d
Packit 7d6a7d
It's not very often that one has to regenerate the baseline and todo
Packit 7d6a7d
files. If you have to, you can either run F<devel/regenerate> or just
Packit 7d6a7d
execute the following steps by hand:
Packit 7d6a7d
Packit 7d6a7d
=over 4
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
You need a whole bunch of different Perls. The more, the better.
Packit 7d6a7d
You can use F<devel/buildperl.pl> to build them. I keep my perls
Packit 7d6a7d
in F</tmp/perl>, so most of the tools take this as a default.
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
You also need a freshly built bleadperl that is in the path under
Packit 7d6a7d
exactly this name. (The name of the executable is currently hardcoded
Packit 7d6a7d
in F<devel/mktodo> and F<devel/scanprov>.)
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
Remove all existing todo files in the F<parts/base> and
Packit 7d6a7d
F<parts/todo> directories.
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
Update the API information. Copy the latest F<embed.fnc> file from
Packit 7d6a7d
bleadperl to the F<parts> directory and run F<devel/mkapidoc.sh> to
Packit 7d6a7d
collect the remaining information in F<parts/apidoc.fnc>.
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
Build the new baseline by running
Packit 7d6a7d
Packit 7d6a7d
    perl devel/mktodo --base
Packit 7d6a7d
Packit 7d6a7d
in the root directory of the distribution. When it's finished,
Packit 7d6a7d
move all files from the F<parts/todo> directory to F<parts/base>.
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
Build the new todo files by running
Packit 7d6a7d
Packit 7d6a7d
    perl devel/mktodo
Packit 7d6a7d
Packit 7d6a7d
in the root directory of the distribution.
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
Finally, add the remaining baseline information by running
Packit 7d6a7d
Packit 7d6a7d
    perl Makefile.PL && make
Packit 7d6a7d
    perl devel/scanprov --mode=write
Packit 7d6a7d
Packit 7d6a7d
=back
Packit 7d6a7d
Packit 7d6a7d
=head2 Implementation
Packit 7d6a7d
Packit 7d6a7d
Residing in F<parts/inc/> is the "heart" of C<Devel::PPPort>. Each
Packit 7d6a7d
of the files implements a part of the supported API, along with
Packit 7d6a7d
hints, dependency information, XS code and tests.
Packit 7d6a7d
The files are in a POD-like format that is parsed using the
Packit 7d6a7d
functions in F<parts/ppptools.pl>.
Packit 7d6a7d
Packit 7d6a7d
The scripts F<PPPort_pm.PL>, F<PPPort_xs.PL> and F<mktests.PL> all
Packit 7d6a7d
use the information in F<parts/inc/> to generate the main module
Packit 7d6a7d
F<PPPort.pm>, the XS code in F<RealPPPort.xs> and various test files
Packit 7d6a7d
in F<t/>.
Packit 7d6a7d
Packit 7d6a7d
All of these files could be generated on the fly while building
Packit 7d6a7d
C<Devel::PPPort>, but not having the tests in C<t/> will confuse
Packit 7d6a7d
TEST/harness in the core. Not having F<PPPort.pm> will be bad for
Packit 7d6a7d
viewing the docs on C<search.cpan.org>. So unfortunately, it's
Packit 7d6a7d
unavoidable to put some redundancy into the package.
Packit 7d6a7d
Packit 7d6a7d
=head2 Adding stuff to Devel::PPPort
Packit 7d6a7d
Packit 7d6a7d
First, check if the code you plan to add fits into one of the
Packit 7d6a7d
existing files in F<parts/inc/>. If not, just start a new one and
Packit 7d6a7d
remember to include it from within F<PPPort_pm.PL>.
Packit 7d6a7d
Packit 7d6a7d
Each file holds all relevant data for implementing a certain part
Packit 7d6a7d
of the API:
Packit 7d6a7d
Packit 7d6a7d
=over 2
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
A list of the provided API in the C<=provides> section.
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
The implementation to add to F<ppport.h> in the C<=implementation>
Packit 7d6a7d
section.
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
The code required to add to PPPort.xs for testing the implementation.
Packit 7d6a7d
This code goes into the C<=xshead>, C<=xsinit>, C<=xsmisc>, C<=xsboot>
Packit 7d6a7d
and C<=xsubs> section. Have a look at the template at the bottom
Packit 7d6a7d
of F<PPPort_xs.PL> to see where the code ends up.
Packit 7d6a7d
Packit 7d6a7d
=item *
Packit 7d6a7d
Packit 7d6a7d
The tests in the C<=tests> section. Remember not to use any fancy
Packit 7d6a7d
modules or syntax elements, as the test code should be able to run
Packit 7d6a7d
with Perl 5.003, which, for example, doesn't support C<my> in
Packit 7d6a7d
C<for>-loops:
Packit 7d6a7d
Packit 7d6a7d
    for my $x (1, 2, 3) { }    # won't work with 5.003
Packit 7d6a7d
Packit 7d6a7d
You can use C<ok()> to report success or failure:
Packit 7d6a7d
Packit 7d6a7d
    ok($got == 42);
Packit 7d6a7d
    ok($got, $expected);
Packit 7d6a7d
Packit 7d6a7d
Regular expressions are not supported as the second argument to C<ok>,
Packit 7d6a7d
because older perls do not support the C<qr> operator.
Packit 7d6a7d
Packit 7d6a7d
=back
Packit 7d6a7d
Packit 7d6a7d
It's usually the best approach to just copy an existing file and
Packit 7d6a7d
use it as a template.
Packit 7d6a7d
Packit 7d6a7d
=head2 Implementation Hints
Packit 7d6a7d
Packit 7d6a7d
In the C<=implementation> section, you can use
Packit 7d6a7d
Packit 7d6a7d
  __UNDEFINED__ macro    some definition
Packit 7d6a7d
Packit 7d6a7d
instead of
Packit 7d6a7d
Packit 7d6a7d
  #ifndef macro
Packit 7d6a7d
  #  define macro    some definition
Packit 7d6a7d
  #endif
Packit 7d6a7d
Packit 7d6a7d
The macro can have optional arguments and the definition can even
Packit 7d6a7d
span multiple lines, like in
Packit 7d6a7d
Packit 7d6a7d
  __UNDEFINED__ SvMAGIC_set(sv, val) \
Packit 7d6a7d
                STMT_START { assert(SvTYPE(sv) >= SVt_PVMG); \
Packit 7d6a7d
                (((XPVMG*) SvANY(sv))->xmg_magic = (val)); } STMT_END
Packit 7d6a7d
Packit 7d6a7d
This usually makes the code more compact and readable. And you
Packit 7d6a7d
only have to add C<__UNDEFINED__> to the C<=provided> section.
Packit 7d6a7d
Packit 7d6a7d
Version checking can be tricky if you want to do it correct.
Packit 7d6a7d
You can use
Packit 7d6a7d
Packit 7d6a7d
  #if { VERSION < 5.9.3 }
Packit 7d6a7d
Packit 7d6a7d
instead of
Packit 7d6a7d
Packit 7d6a7d
  #if ((PERL_VERSION < 9) || (PERL_VERSION == 9 && PERL_SUBVERSION < 3))
Packit 7d6a7d
Packit 7d6a7d
The version number can be either of the new form C<5.x.x> or of the older
Packit 7d6a7d
form C<5.00x_yy>. Both are translated into the correct preprocessor
Packit 7d6a7d
statements. It is also possible to combine this with other statements:
Packit 7d6a7d
Packit 7d6a7d
  #if { VERSION >= 5.004 } && !defined(sv_vcatpvf)
Packit 7d6a7d
    /* a */
Packit 7d6a7d
  #elif { VERSION < 5.004_63 } && { VERSION != 5.004_05 }
Packit 7d6a7d
    /* b */
Packit 7d6a7d
  #endif
Packit 7d6a7d
Packit 7d6a7d
This not only works in the C<=implementation> section, but also in
Packit 7d6a7d
the C<=xsubs>, C<=xsinit>, C<=xsmisc>, C<=xshead> and C<=xsboot> sections.
Packit 7d6a7d
Packit 7d6a7d
=head2 Testing
Packit 7d6a7d
Packit 7d6a7d
To automatically test C<Devel::PPPort> with lots of different Perl
Packit 7d6a7d
versions, you can use the F<soak> script. Just pass it a list of
Packit 7d6a7d
all Perl binaries you want to test.
Packit 7d6a7d
Packit 7d6a7d
=head2 Special Makefile targets
Packit 7d6a7d
Packit 7d6a7d
You can use
Packit 7d6a7d
Packit 7d6a7d
    make regen
Packit 7d6a7d
Packit 7d6a7d
to regenerate all of the autogenerated files. To get rid of all
Packit 7d6a7d
generated files (except for F<parts/todo/*> and F<parts/base/*>),
Packit 7d6a7d
use
Packit 7d6a7d
Packit 7d6a7d
    make purge_all
Packit 7d6a7d
Packit 7d6a7d
That's it.
Packit 7d6a7d
Packit 7d6a7d
=head2 Submitting Patches
Packit 7d6a7d
Packit 7d6a7d
If you've added some functionality to C<Devel::PPPort>, please
Packit 7d6a7d
consider submitting a patch with your work to GitHub here:
Packit 7d6a7d
L<https://github.com/mhx/Devel-PPPort/issues/>, or by sending a
Packit 7d6a7d
Pull Request.
Packit 7d6a7d
Packit 7d6a7d
When submitting patches, please only add the relevant changes
Packit 7d6a7d
and don't include the differences of the generated files. You
Packit 7d6a7d
can use the C<purge_all> target to delete all autogenerated
Packit 7d6a7d
files.
Packit 7d6a7d
Packit 7d6a7d
=head2 Integrating into the Perl core
Packit 7d6a7d
Packit 7d6a7d
When integrating this module into the Perl core, be sure to
Packit 7d6a7d
remove the following files from the distribution. They are
Packit 7d6a7d
either not needed or generated on the fly when building this
Packit 7d6a7d
module in the core:
Packit 7d6a7d
Packit 7d6a7d
  MANIFEST
Packit 7d6a7d
  META.yml
Packit 7d6a7d
  PPPort.pm
Packit 7d6a7d
Packit 7d6a7d
=head1 COPYRIGHT
Packit 7d6a7d
Packit 7d6a7d
Version 3.x, Copyright (C) 2004-2013, Marcus Holland-Moritz.
Packit 7d6a7d
Packit 7d6a7d
Version 2.x, Copyright (C) 2001, Paul Marquess.
Packit 7d6a7d
Packit 7d6a7d
Version 1.x, Copyright (C) 1999, Kenneth Albanowski.
Packit 7d6a7d
Packit 7d6a7d
This program is free software; you can redistribute it and/or
Packit 7d6a7d
modify it under the same terms as Perl itself.
Packit 7d6a7d
Packit 7d6a7d
=head1 SEE ALSO
Packit 7d6a7d
Packit 7d6a7d
See L<ppport.h> and L<devel/regenerate>.
Packit 7d6a7d
Packit 7d6a7d
=cut