[See this talk online for graphics and stuff at:
http://www.astray.com/yapc/2000/graphing_perl/]

                 [1]yapc::Europe 19100 Graphing Perl talk 
                                      
   yapc::Europe 19100 Graphing Perl talk / By Leon Brocard / Last updated
                          24th September June 2000
                                      
   Contents
   
     + [2]1. Meta Introduction
     + [3]2. Introduction
     + [4]3. Quote
     + [5]4. What this talk is not about
     + [6]5. What this talk is about
     + [7]6. Why?
     + [8]7. Graphs?
     + [9]8. How?
     + [10]9. Example GraphViz input
     + [11]10. GraphViz.pm
     + [12]11. GraphViz.pm example
     + [13]12. Data structures
     + [14]13. XML
     + [15]14. Subroutine cross-reference
     + [16]15. Code profiling (line level)
     + [17]16. Code profiling (subroutine level)
     + [18]17. Overview and future
     + [19]18. Questions
     
   1. Meta Introduction
   
   These are most of the slides of a talk named "Graphing Perl" I gave at
   [20]yapc::Europe. 19100. Comments added after the talk are in square
   brackets.
   
   2. Introduction
   
   Graphing Perl
   
   Leon Brocard
   
   acme@astray.com
   
   3. Quote
   
   "Did you ever see such a thing as a drawing of a muchness?" --
   Dormouse, Alice in Wonderland
   
   The point of this talk is that, not only that you can draw a muchness,
   you can do it in Perl and of Perl and make it pretty too!
   
   4. What this talk is not about
   
   Bar charts!
   
                               [A bar chart]
                                      
   This talk is NOT about producing barcharts or plots. See
   Chart::PNGgraph and other modules on the CPAN instead.
   
   5. What this talk is about
   
   Pretty pictures.
   
   No, really. Oh, alright, it's really about the visualisation of
   various parts of Perl using directed graphs. We're going to explore
   data structures, XML, call graphs, code profiling.
   
                              [A simple graph]
                                      
   6. Why?
   
   Observation aids comprehension. That's a fancy way of saying that
   popular faux-Chinese proverb (you can join in with me if you want): "a
   picture is worth ..." [a few people chanted out "a thousand words",
   and one chanted out "what you paid for it"].
   
   Text isn't always the best way to represent anything and everything to
   do with a computer programs. Pictures and images are easier to
   assimilate than text. The ability to show foo graphically can aid a
   great deal in comprehending what foo represents.
   
   Diagrams are computationally efficient, because search for information
   can be indexed by location; they group related information in the same
   area. They also allow relations to be expressed between elements
   without labelling the elements.
   
   [The following was spontaneous, due to seeing my friend Mark sitting
   in front of me]. For example, my friend Mark Fowler used this to his
   advantage when trying to remember important dates in computer history.
   He didn't sit down and try to remember everything, he printed over a
   hundred posters (each with a date and event) and plastered these
   throughout his house. His spatial memory was so good, he would answer
   "oh, I know that, it's opposite John's door on the second floor so it
   must be about August 1982".
   
   Spreadsheets are a wonderfully simple graphical representation of
   computational models. They've taken off. You may know this already.
   
   7. Graphs?
   
   What is a graph? A collection of nodes linked together with edges.
   
   What is a directed graph? A graph - but the edges have a direction.
   
   8. How?
   
   Luckily, a wonderful automatic graph layout and drawing package
   already exists and is out there: GraphViz, developed by some guys at
   AT&T.
   
   It takes input in a fairly simple text file format (well, Kevin Lenzo
   says it's simple and who am I to argue ;-) and can output in a variety
   of graphic formats (PostScript, PNG, etc.).
   
   It's actually a pain to install and use. The latest version depends on
   all sorts of libraries that are used nowhere except in AT&T... but we
   still like it as it does the hard part, which is laying out the graph
   in an optimal way.
   
   9. Example GraphViz input
   
   Example input to create the graph shown above:
   
                               digraph test {
                            node [ label = "\N" ];
                            node1 [label=London];
                          node2 [label="New York"];
                       node3 [label="City of\nlurve"];
                               node1 -> node3;
                         node1 -> node2 [label=Far];
                               node3 -> node1;
                                     }
                                      
                              10. GraphViz.pm
                                      
   Now, I didn't want to output the text file in my example programs, so
   I constructed a Perl module, called rather obviously "GraphViz" (it's
    on the CPAN of course). It provides a very simple object-orientated
    interface and takes a great deal of the load off the programmer when
               dealing with large graphs and fancy features.
                                      
    I'm not going to go into much detail about how I used the module in
   this talk, mostly because the Perl code wouldn't fit on one slide ;-).
   Just bear in mind that whenever I present a graph it's been generated
                       automatically with my module.
                                      
                          11. GraphViz.pm example
                                      
     An example of using GraphViz.pm to create the same graph as shown
                                   above:
                                      
                               use GraphViz;
                                      
                          my $g = GraphViz->new();
                                      
                     $g->add_node({ name => 'London'});
        $g->add_node({ name => 'Paris', label => 'City of\nlurve'});
                    $g->add_node({ name => 'New York'});
                                      
             $g->add_edge({ from => 'London', to => 'Paris',});
                                      
             $g->add_edge({ from => 'London', to => 'New York',
                                     label => 'Far'});
                                      
            $g->add_edge({ from => 'Paris',  to => 'London',});
                                      
                             print $g->as_png;
                                      
     This may seem like a lot of work for a simple graph, but trust me,
    once you get complicated graphs with anonymous nodes and start using
   ports (subparts of nodes) and clusters (grouping nodes together) this
       is a great deal simpler than printing out the evil dot format
                          throughout your code...
                                      
                            12. Data structures
                                      
    Programs consist of data structures and code. If the data structures
     used aren't clear to you, you will never gain understanding of the
       code. This principle is upheld by the Data Display Debugger, a
      debugger which allows visual inspection of data structures. Perl
     programmers know this, and everyone uses the Data::Dumper module.
                                      
                               $VAR1 = 'red';
                                 $VAR2 = {
                                       'a' => [
                                              3,
                                              1,
                                              4,
                                               1
                                             ],
                                       'b' => {
                                          'w' => 'b',
                                          'q' => 'a'
                                              }
                                         };
                              $VAR3 = 'blue';
                               $VAR4 = undef;
                                      
      The above may be clear if you're a good Perl programmer, but how
                                   about:
                [21][A representation of a data structure] 
                                      
                                  13. XML
                                      
    This talk is fully buzzword-compliant! I can overhype, overbuzz, and
                   overstate like the best of them, baby.
                                      
        XML is a very hyped up format, which is basically all about
      representing data in a tree. While it is human-readable, gaining
      information about the structure is quite hard. If only we could
    represent the following XHTML (HTML forced into XML) graphically...
                                      
                <html><head><title>news</title></head><body>
           <p>Check out the <a href="/news/">latest news</a>.</p>
                 <p>Under construction!!!</p></body></html>
                                      
       Well, with a bit of help from XML::Twig and GraphViz, you can!
                  [22][A representation of an XHTML file] 
                                      
                       14. Subroutine cross-reference
                                      
     How can subroutines in a module call each other? Well, the B::Xref
   module generates cross reference reports for Perl programs, and with a
   bit a glue we can generate pretty call graphs like the following of my
    Functional module. Call graphs are graphs which show the ordering of
                          calls within a program.
                                      
    Note that the B::Xref module has hooks into the Perl compiler: these
      links are calls that could be made, not that they actually are.
                 [23][A subroutine cross-reference graph] 
                                      
    The picture above shows my Functional module, which allows you to do
   kind of functional programming in Perl. It shows various possible call
   paths, for example zip3 can call maximum, which can call foldl1, which
   can call foldl. Also prime can call Length and factors, show can call
        show_aux which can call show, and gcd_aux and Until can call
                                themselves.
                                      
   This is fairly useful for determining the structure of a program. And
                                pretty ;-).
                                      
                      15. Code profiling (line level)
                                      
      Can we do better than potential code graphs? Yes, we can do code
   profiling to find how many calls were actually made and where the code
   spent all its time. Representing this graphically brings new meanings
                         to the phrase "hot spot".
                                      
       Using a modified version of Devel::SmallProf, a per-line Perl
    profiler, we can clearly see the hotspot in this program (written by
    Greg McCarroll to err, sound good ;-). Nodes are coloured from black
    to red depended on how much time they took of the total time taken.
     Edges are coloured depending on the number of times that path was
                                   taken.
                  [24][A line-level code profiling graph] 
                                      
                   16. Code profiling (subroutine level)
                                      
    Line-level profiling isn't useful when programs get very large, but
        luckily subroutine-level profiling is made possible with the
     Devel::DProf module. The graph clusters subroutines from the same
    module together, and labels edges with the amount of times that the
                              path was taken.
               [25][A subroutine-level code profiling graph] 
                                      
                          17. Overview and future
                                      
     + Graphing is Good
     + Graphing Perl is possible as easy (as we just provide glue
     between existing modules and GraphViz)
     + A new version of my module will be out on the CPAN next week
     which supports clusters and ports
     + Future: graph internal Perl datastructures (B::Graph, perlguts
     illustrated)
     + Graphing Perl6: bytecode? parse trees?
     + GraphViz does good layout, but it doesn't draw particularly well.
     For example, one of the ideas is to draw Flash graphs with Simon
     Wistow and Mark Fowler's PerlFlash stuff - this is currently Hard,
     having to hack lots of undocumented C code. I don't like C. I like
     Perl. Perhaps my module could make this kind of thing easier?
     + It would be nice to do interactive graphs, perhaps which are
     zoomable with a fisheye-lens kind of thing. Kevin Lenzo pointed me
     at Tk for this kind of thing.
     + ... in my Copious Spare Time ;-)
     
                               18. Questions
                                      
                   I had time for a couple of questions.
                                      
     + Mark Fowler asked how fast the graph layout was. I replied it was
     very fast, two seconds or so for fairly large graphs
     + Simon Cozens asked what kind of internal datastructures I wanted
     graphed. I replied that I wasn't really a Perl internals guy, but
     he should look at B::Graph and we should get in contact later.
     + Someone asked a question about whether cumulative graph layout
     was possible, that is given an existing graph layout adding an
     extra edge or node can be done without too much work. I answered
     that the GraphViz package supported this, but had an amazing lack
     of documentation and used data structures I couldn't understand
     (and muttered out loud that perhaps I could graph C data structures
     too...).
     + ... and there were a couple of questions that I've forgotten.
     Sorry.
     
                               [26][<xml/>] 
                              [27][<email/>] 
                                      
   [www.astray.com]

References

   1. http://www.astray.com/yapc/2000/
   2. http://www.astray.com/yapc/2000/graphing_perl/#1__Meta_Introduction
   3. http://www.astray.com/yapc/2000/graphing_perl/#2__Introduction
   4. http://www.astray.com/yapc/2000/graphing_perl/#3__Quote
   5. http://www.astray.com/yapc/2000/graphing_perl/#4__What_this_talk_is_not_about
   6. http://www.astray.com/yapc/2000/graphing_perl/#5__What_this_talk_is_about
   7. http://www.astray.com/yapc/2000/graphing_perl/#6__Why_
   8. http://www.astray.com/yapc/2000/graphing_perl/#7__Graphs_
   9. http://www.astray.com/yapc/2000/graphing_perl/#8__How_
  10. http://www.astray.com/yapc/2000/graphing_perl/#9__Example_GraphViz_input
  11. http://www.astray.com/yapc/2000/graphing_perl/#10__GraphViz_pm
  12. http://www.astray.com/yapc/2000/graphing_perl/#11__GraphViz_pm_example
  13. http://www.astray.com/yapc/2000/graphing_perl/#12__Data_structures
  14. http://www.astray.com/yapc/2000/graphing_perl/#13__XML
  15. http://www.astray.com/yapc/2000/graphing_perl/#14__Subroutine_cross_reference
  16. http://www.astray.com/yapc/2000/graphing_perl/#15__Code_profiling__line_level_
  17. http://www.astray.com/yapc/2000/graphing_perl/#16__Code_profiling__subroutine_level_
  18. http://www.astray.com/yapc/2000/graphing_perl/#17__Overview_and_future
  19. http://www.astray.com/yapc/2000/graphing_perl/#18__Questions
  20. http://yapc.org/Europe/
  21. http://www.astray.com/yapc/2000/graphing_perl/graphics/dumper.png
  22. http://www.astray.com/yapc/2000/graphing_perl/graphics/xml.png
  23. http://www.astray.com/yapc/2000/graphing_perl/graphics/Functional.png
  24. http://www.astray.com/yapc/2000/graphing_perl/graphics/primes.png
  25. http://www.astray.com/yapc/2000/graphing_perl/graphics/tmon.png
  26. http://www.astray.com/yapc/2000/graphing_perl/index.xml?xmlmode=raw
  27. mailto:acme@astray.com
