After a long break, an apple, a cup of coffee, and a beer, I’m back in the Perl track.

The full title of this session is, Scratching the 40-Year Itch of Inheritance with Smalltalk-style Traits.

This is not a tutorial. How to use traits is easy, but why to use them is a more complex discussion.

Inheritance is a very complex problem and an easy one to get wrong. Then people start doing things with multiple inheritance and, even if they’re not doing something deliberately stupid, they end up with diamond inheritance. Not only is this a problem, but it’s been a problem for a very long time—40 years, in fact.

Complex systems can lead to deep class hierarchies. When hierarchies are deep, in particular with a dynamic language like Perl, it becomes difficult to determine where a method came from. Even when its known where a method comes from, undesired behavior may be inherited. This becomes worse when multiple inheritance is used.

As systems grow, the problem becomes two-fold:

1. Class responsibility – larger classes are desired
2. Class reuse – smaller classes are desired

Inheritance, by itself, cannot solve this problem. So the solution is to
decouple the sub-problems.

Several solutions have been tried:

• Interfaces
• Delegation
• Mixins – incredibly popular

As expected by the name of this session, traits (or roles in the nomenclature of Moose) solve the problem far better than any of the above solutions. Much of the session involved showing real-world application of roles to clean up code at the BBC.

After reviewing today’s session schedule, I quickly came to the conclusion that I will spend my entire day sitting in the room “Portland 256.” This is, apparently, where the Perl track is located.

Paul Fenwick introduced Piers in song, to the tune of Gilligan’s Island.

Piers switched from Perl to Ruby a while back and swore that he wouldn’t return to Perl until 6. Facetiously, the reason he switched to Ruby was the handsome community associated with it and he reason he switched back to Perl was the amazingly supportive community associted with it. He began with a point about programming style. We think of code as describing what we are doing, but in reality the majority of our code actually describes how we are doing it. This infrastructure code is noise.

More seriously, he absolutely hated unrolling the @_ variable in every function. In such a high level language like Perl, why must we pop arguments off the stack in the same manner we would in an assembly language? This leads to long subroutines, every single one containing anti-patterns designed to implement the language infrastructure, instead of the language doing the work for us.

Moose does a lot to improve writing classes, using a more declarative syntax. However, even within Moose methods we need to write the infrastructure code. The MooseX::Declare module solves this problem, giving method syntax a more declarative style. By moving the infrastructure code out of sight, we can better focus on what we are trying to do, rather than how we are doing it.

Piers proceeded to list the modules that “rock” and brought him back to Perl:

• Plack
• Devel::NYTProf
• Moose

Perl’s object-orientation absolutely “sucks.” However, this turns out to be a good thing. It allows very clever people to create modules that extend the semantics of the language. In a language like Ruby, which has a good object-orientation built-in, it’s essentially stuck. If, in the future better ideas of object-orientation are developed, they can be implemented in Perl far more easily than in Ruby. An interesting point: sometimes when the tool sucks, things are better. People develop layers of tools that enhance and extend the original.

It also helps that the Perl release schedule has accelerated.

Piers continued with a high-level, hand-waving explanation of how MooseX::Declare works. While not informative, it was entertaining. Including a video of Matt Trout attempting to hypnotize the room.

Piers ended by thanking the Perl community and expressing how good it feels to be back into it and developing in Perl again.

The Devel::DProf module is old and a waste of time and is broken. Stop using it. Take it out and shoot it.

The first obvious distinction between profilers is CPU time versus real time. CPU time tends to be highly granular, but doesn’t include I/O, context switching, or other kinds of blocking. That’s where real time comes in. It’s far more useful in the real world.

Tim, as with many of us, is interested in line-based profiling. It provides a high level of granularity The total subroutine time is not always useful, particularly in larger subroutines.

The NYTProf module is exremely fast, discounting the time taken by profiling overhead, making it quite a bit more useful for real world analysis. It also allows profile times per block, and can be aggregated up to the subroutine level. It’s a module with dual profilers: line-based and subroutine-based.

It gets better, every location that calls the subroutine keeps separate track of the subroutine time. This allows us to determine where the majority of the subroutine calls are coming from. For control flow statements, the decision expression is not taken into account when profiling the block that is executed. This is useful if the loop control itself takes time that should be discounted.

And that’s it for the description. Now we have half an hour to play with it.

The HTML-based reporting is inspired by Devel::Cover‘s reporting. Reported for each file are the number of statements executed, the time spent in the source file and the line, block, and subroutine reports. The subroutine reports include the amount of time spent within the subroutine and the amount of time spent in other called subroutines. The coloring of each line of the report—red, orange, yellow, and green—give a relative measure of deviation from the norm. Very impressive.

Even more impressive, Devel::NYTProf is capable of reporting exactly what a subroutine reference is called, even when it’s an anonymous subroutine compiled within an eval. With a handy link also provided, the called code can be easily inspected.

In summary, Devel::NYTProf is awesome. Use it. I know I will.

Tim Bunce is even more impressive than most people think he is. He is the only presenter I’ve seen so far who has managed to use IRC while giving his talk. Well, he didn’t actually type on IRC, but he had Colloquy running in the background. This particular IRC client uses Apple’s Growl feature to display notifications when you are mentioned in a channel. After he’s opened up the session to questions, one of those notifications pops up on the projected display:

<sirhc> Adam Kennedy (to Tim Bunce): Why are you so awesome?

It got a laugh, and Tim seemed to take it all in stride, even joking that he was not looking very professional on his screen cast. Important safety tip for session presenters, don’t leave your IRC client open.

[tags]oscon, oscon08, oscon2008, Perl, programming, profiling[/tags]

A computer once referred to a human worker who would perform calculations. This was a fairly easy thing to cluster and “run” several computers in parallel. As time progressed, more and faster work was desired. Enter the electronic computer, and specifically for this talk, the Cray. As with anything, the inner workings of the Crays of old can be recreated in Perl. Just use the Cray module, no problem (if only it existed).

After the history lesson, we move into high level overviews of parallelism and pipelineing, and a note about Amdahl’s Law. This was followed up with an example for detecting prime numbers by partitioning the work.

The slide presentation was over in under 20 minutes. Instead, we’re jumping straight into code examples. Awesome.

Or so I thought. Unfortunately, he’s been interrupted by multiple people in the audience, who keep wanting to move off into tangential conversations. Eric is having difficulty bringing the talk under his own control—it’s no longer his talk, but that of the somewhat rude fellow in the front row. Neither is Eric as eloquent when he switches from a prepared talk to demonstrating and explaining real code. It’s become far more difficult to pay attention to this session, and I find myself looking at the clock to see how much time we have until the next session.

For real fun, be sure to check out Brad’s post on Schwern’s session about skimmable code.

[tags]oscon, oscon08, oscon2008, Perl, programming[/tags]

my $code = qq{
    print 1+1 . "\n";
    $code =~ m/(\d+)\+(\d+)/;
    $new = $1 + $2;
    $code =~ s/\d+\+(\d+)/$2+$new/;
};

for ( 1 .. 10 ) {
    eval($code);
}

What’s the first bug? Yes, it should use q{}, or the variables will interpolate on the initial assignment to $code. To their credit, they initially used single quotes, but people said it was too hard to read.

I wasn’t content with just figuring out what the code did and fixing a small bug. I think it can be written better.

eval($code = q{
    print 1+1 . "\n";
    $code =~ s/(\d+)(\+)(\d+)/"$3$2" . ($1 + $3)/e;
    eval $code;
});

Much better. Not only is it more concise, I was able to remove that pesky loop, so I wouldn’t be bothered by any silly upper bounds.

So what does it do? Should be obvious. Head over to the Amazon booth and let them know.

Nearly everyone agrees that code reviews are a good idea, so why aren’t they done more often? In fact, this is the very same problem we’ve had at work. We’ve been talking about code reviews for two years, but we’ve never had one.

There are some barriers to entry to doing code reviews. If revision control is not in use or automated tests aren’t being run, tackle those problems first. Also, the need for a team process is necessary, from ticket tracking to release plans.

Pair programming, that tenet of XP, is a poor substitute for code reviews. Two people working together will not magically turn one or the other into what is essentially a disinterested third party, who may catch bugs simply because they weren’t there when it was written.

Test-driven development is also a great way of coding, but not a substitute for reviews. Often for the same reasons. Tests are often just more code and the code tested is only when someone thinks to test it.

Even during a code review, a reverence for authority can get in the way of getting things done. A poor, intimidated programmer may not have the courage to criticize a more senior programmer. Instead, this can be turned around with something I use a lot myself. I like to call it, “playing dumb.” Instead of saying, “this won’t work,” ask what will happen for a suspicious case.

Socially, the only way for code reviews to work is universal buy-in. Everyone is subjected to code reviews by everyone else. No exceptions. Make them a habit, a regularly-scheduled meeting. At work, I’ve even suggested bi-weekly, or perhaps monthly, catered, lunch time code reviews. Just to get us into the habit of doing it.

Code review time should not be wasted on things such as code formatting, best practices, or test coverage. This is stupid. These are objective tasks that can be automated.

Instead, look for subjective things, which can’t be automatically found. Such as code readability, algorithmic clarity, and consistent identifier naming. Other targets for code reviews are the usual things we here over and over again as development best practices: consistent documentation that follows the internal standard, that kind of thing.

The remainder of the talk is essentially an enumeration of all the things to look for in code reviews. All of them are, at least to me, common sense. So I’m not going to spend any time writing them down. If you don’t already know them, well go find some common sense.

One thing that he recommends that I like is code reviews by e-mail. It’s an old, well-understood, and (usually) reliable tool. So why not combine e-mail with a version control system—particularly one of the newer distributed version control systems—to perform out-of-band code reviews. It actually sounds like a good idea to me, and I’ve done it at work a couple of times with code written by an intern.

What I’m starting to notice is that many of the later the recommendations for reviewing code are personal opinions of the presenter. I think the way in which code reviews are performed are highly dependent on what works best for the group reviewing code. It’s like so many things, from cameras to backup solutions: the best one is not the shiniest or the one with the most bells and whistles, it’s the one that’s actually used.

[tags]oscon, oscon08, oscon2008, programming[/tags]

Erlang is a pure functional language (and thus no side-effects) with strong dynamic typing and syntax similar to Prolog and ML. Most notably, it contains concurrency primitives, which is what we’re here to hear about today.

Erlang concurrency primitives include spawn, to create a process, !, to send a message to a process, and receive, to listen for a message. These are not system level processes, but other Erlang processes. It’s a lot like using fork in imperative languages, but less messy.

Erlang, like many functional languages, can implement quick sort in three lines of code. I was having a discussion with a friend of mine about this topic yesterday. It’s very nice, and demonstrates the power of functional languages to trivially solve an already solved set of problems, but is it any use in the real world? Maybe. While I’ve not seen any non-trivial examples, I’m reserving judgment.

The first example is a demonstration on how simple it is to parallelize the quick sort algorithm. It’s not a worthwhile example, in fact, it’s a particularly bad idea, but it serves as a reasonable example of the ease of use of the concurrent features in Erlang. So far, it seems like changing a map call—something I love from Perl—to pmap.

The pmap function is not a built in function (BIF), but a library function built on top of the built in concurrency primitives. The code implementing the function is actually quite simple, and should be available in the slides available at the end of the conference. Conceptually, it spawns as many processes as necessary and uses them to call the function being mapped. It then gathers the results, waiting for each process to complete. It’s quite similar to code I’ve written to do scientific processing using MPI, but I’ve always thought functionally when coding.

After explaining concurrency, we make the jump to distributed systems. What’s everyone’s favorite distributed system? Twitter! Twitter, while not designed as such, is essentially a messaging system. Erlang does message passing very well, and almost all programs are designed using this paradigm. So Kevin took a stab at implementing a Twitter-like system in Erlang, the key ideas of which he will present to us.

The lightweight and convenient process architecture of Erlang lends itself to the problem. Every user can be represented as a process. Each process can then send and receive messages. In effect, the problem—the messaging part anyway—is now solved. But, what about scaling to multiple machines?

It turns out to easy (but you knew it would, right?). All we need to do is pull in the global module and we can bind our users not only to a process identifier, but combine that with a given machine as well.

However, we still don’t have a reliable system. If a process dies, that user is no longer in the system. So it really is a lot like Twitter.

OTP, the Open Telecom Platform (a legacy name from Erlang’s history at Ericcson), provides a set of common behaviors and patterns for writing reliable and distributed system. The programmer simply declares what interface they would like to use, then implement a set of callbacks defined for that behavior. Reminds me a bit of roles (because I have an unhealthy need to relate everything back to Perl).

As with everything in Erlang, it is almost impossibly easy to set up this reliability. I still can’t get over how well the syntax maps to how I actually think about code.

A question was raised about how to go about setting up the necessary cluster of hosts used in Erlang’s mesh network. Kevin went into it briefly, but it’s unfortunately out of scope for this session.

And, with that, it’s time for lunch. Thanks, Kevin!

[tags]oscon, oscon08, oscon2008, Erlang, concurrency, programming[/tags]

Over 90 people registered for this tutorial, and the room is almost full. Save for the handful of available chairs, I’d feel guilty about auditing it instead of attending the Real Time 3D on the Web with Open Source I had originally registered for. This will be a two and a half day course compressed into three hours. Should be fun, and useful for Kevin’s session tomorrow, Beautiful Concurrency with Erlang. After seriously considering the relative merits and general usefulness of the tutorials, I decided Erlang would be much more interesting. I had made my original choice with the equivalent of a dart board, so I don’t feel too bad about changing my mind.

The tutorial started with a quick tour of Erlang’s syntax. It looks odd, but I’ve used Lisp and ML in the past, and I’m a rather good Perl hacker, so it isn’t proving too difficult to pick up. The concept of pattern matching intrigues me. It appears to use equivalency, in the mathematical sense to handle both boolean and assignment operations with the same syntax. For example,

[A,B,C] = [1,2,3]    % A is 1, B is 2, C is 3
[A,B,C] = [1,2]      % error, size mismatch
[A,B,A] = [1,2,3]    % error, A already bound to 1
[A,B,A] = [1,2,1]    % okay, A bound to 1, then equivalent to 1

Shortly into the discussion of syntax, Francesco asked that anyone who hasn’t yet installed Erlang do so. I executed yum install erlang, which pulled in unixODBC, tcl, and tk as dependencies. Well, 45 megabytes and 45 minutes later—an impressive speed of 1 MBpm—I now have Erlang installed and ready to run. Just in time for a 10 minute break.

During this first break, we were asked to do a simple exercise in Erlang: write a module, boolean.erl, that implements b_not(), b_and(), b_or(), and b_nand(), without using the built in logical operators. I’ve been able to define the structure of the module, but I don’t know how boolean values are represented in Erlang, so I may have to wait until he gives us the answer. Vim’s syntax highlighting tells me that true and false are reserved words, so I can use those.

The solution for this involves writing a simple truth table. In Erlang, functions are subject to pattern matching in the same way that many programming languages allow for function overloading. For the logical or, we start with the basic truth table:

b_or(true,true)   -> true;
b_or(true,false)  -> true;
b_or(false,true)  -> true;
b_or(false,false) -> false.

That’s downright simple and extremely easy to grasp on a conceptual level, particularly for anyone with any background in mathematics. However, and this appeals to me as Perl hacker, Erlang allows the programmer to be lazy, but in a good way. The null variable—as I’m calling it due to the analogy with /dev/null on Unix-like systems (or undef in Perl)—_, allows a kind of lazy matching:

b_or(false,false) -> false;    % the only false case with OR
b_or(_,_)         -> true.     % any other case is true

The other functions can be written in a similar way.

Back from the break, and the population of the room has thinned very slightly. Francesco immediately jumped into conditional evaluation, starting with the case clause. I suspect this may be one of the answers to the exercise. He followed that with the if clause. I find it interesting that he’s done it in that order. In most languages, the if statement is a much simpler case (no pun intended) and is covered first, before moving into more complex territory. I think I understand why, the two clauses are implemented in a very similar fashion. I’m not sure how equivalent they are, I’d have to play with them a bit.

As with any functional language, Erlang has strong support for recursion as well as a handful of built in functions (BIFs) implemented in C to accomplish things that are difficult or impossible to do directly in Erlang. After all, at a certain point, things like date and time require system calls. Also available are convenience functions to do things like convert tuples to lists or back.

At the second, official, break—taken after an official entered the room to scold Francesco for being 15 minutes late—we were presented with two more exercises. First, to write a function, sum/1, which, given a positive integer N, will return the sum of all the integers between 1 and N. As an extension, write a function, sum/2, which, given two integers N and M, return the sum of the interval between them, first ensuring N <= M. Second, write a function, create/1, which will return the list 1 through N given N as its argument. As an extension, write a function, reverse_create/1, which does the same in reverse.

As I suspected, both exercises are perfect candidates for recursion, which is quite simple to do in Erlang:

sum(N) when N > 0 ->
    N + sum(N-1);
sum(0) ->
    0.

The simpler list creation function is actually the second, and is solved similarly, but by accumulating a list instead of adding to a sum (which is, actually, also a method of accumulation):

reverse_create(0) ->
    [];
reverse_create(N) ->
    [N|reverse_create(N-1)].

The first thing I notice is, again, how mathematical Erlang is. The solution is written in exactly the same way I do it when I’m jotting down notes while thinking about how to solve the problem. To me, the syntax is quite elegant.

After going over the solutions to the exercises, we moved into concurrency. As with most languages worth using, Erlang has a spawn() BIF, used to create processes. What’s interesting about spawning processes in Erlang is that the function to do it does not take a system command. Rather, it takes another Erlang function to run. It’s quite a bit more elegant (there’s that word again) than the equivalent fork() dance done in most imperative languages.

Communication between Erlang processes is done via message passing; data is never shared. As with everything else, the method for doing so is quite elegant: Pid2 ! {self(), foo}. Okay, maybe someone has to be me to find that elegant.

The whole process concept in Erlang is quite nice and, again, elegant. It’s plain that it is the primary method by which systems in Erlang are designed. So far, though, we’ve only seen trivial examples. That’s okay, because this is only a three hour tutorial. However, as Larry Wall once said about Perl: It makes the easy things easy and the hard things possible. It’s a good litmus test for any language. It’s far too early for me to pass any judgment on Erlang. I’d like to use it in anger sometime, to see how it performs for me. Perhaps I can get my local Perl Mongers interested in chatting about it.

The idea behind Mastering Perl is not to talk about Perl to a group of Perl masters. Instead, it’s about mastering Perl in the guild sense (and not of the SCA variety). Back in the day, and still existing in some professions today, there was an apprentice system. A neophyte—in today’s nomenclature, a noob—would begin acquiring skills under a master of the art. As he progressed, he would be entrusted with more and more responsibility, until finally he became a master himself and took people under his own wings.

This apprenticeship system, somewhat unfortunately, does not exist in the computing world. That’s where brian d foy feels that Mastering Perl fits. Lacking true masters, the book acts as a substitute. Someday, we may even create a guild system. But then we’d probably have to pay dues and follow rules, and that’s not very attractive. That said, it’s the model I’m hoping to use at my own place of work. I’d like to hire one or two developers who I can take under my own wing and mentor them in the ways of Perl and the grid.

The first two topics covered are tools for optimization, profiling and benchmarking. Often mis-attributed to Donald Knuth, Tony Hoare once said, “Premature optimization is the root of all evil.” What this means is that one should never assume what requires optimization. Let the testing be the guide.

While profiling is objective, benchmarks, like statistics, are not always objective. Everyone has an agenda and benchmarks are subjective. Often, benchmarks are short-sighted. For example, benchmarking code run time and attempting to optimize for it may not be worth the expense of the developer time required to make the requisite changes. It’s worth analysing what is important before blindly following benchmarks.

I’ve been on the receiving end of misplaced premature optimization. I worked with a development group that put far too much emphasis on achieving perfect results on their Devel::Cover reports. This led to strange bugs in their code, and a strong belief that “new() doesn’t work that way.” As it turns out, their test suite was calling new() in two ways. I forget what the second method was, but it was not used anywhere else in their code. However, in order to get this test code to run, and get 100% coverage, they added code to the constructor for every class. Code that prevented inheritance of the method. The team then convinced themselves that constructors could not be inherited in Perl, rather than realizing that their own habits were the problem.

After the mid-morning break, we wrapped up the discussion on profiling and benchmarking, and moved into configuration. This is a vital topic for anyone who desires the ability to pass a program off to users without being bothered to modify it later in response to users’ desire to customize the program for a slightly different use.

External configuration, particularly via the command line, is something I depend on heavily, even in very simple Perl or Bourne shell scripts. I almost always create command line options for performing a dry run or output debugging information. Not only are these useful for development, they can live on in the final program, providing help to the final user, who more often than not is me. Sometimes I will even add configuration to values that never change, just for when they eventually do.

Jumping past configuration, we move on to logging. It’s really easy to add to a program, and it’s really useful to leave in a program when it’s released. The ability to enable logging on the fly sure beats adding a bunch of print() calls in the code when it inevitably breaks at three in the morning. The Log::Log4perl module is a particularly powerful method of adding logging to programs. It’s well worth investigating for anyone who wants to easily add logging functionality to their code.

The final topic of the day is lightweight persistence. It’s always nice to have data stick around between program invocations. The easy way (and everything in the second half of the tutorial is easy) to add persistence to code is to not use DBI. While DBI is powerful, it also tends to require a database server (ignoring SQLite for the moment). Modules such as Data::Dumper, YAML or Storable are ideal for easily storing and retrieving data in code.

After the tutorial, brian will be available at the Powell’s Books mini store, located near the registration desk, to sign copies of Mastering Perl. I already have a copy, thanks to my local Perl Mongers group, but it’s all marked up with the group name, and I wouldn’t mind having a signed copy.

Now it’s time for lunch, which is good, because I’m quite hungry. I hope the conference-provided lunch is decent during the tutorials, as it was last year.

[tags]oscon, oscon08, perl[/tags]