I’m trying to extract key/value pairs from a file with the following contents:

- name = gcc_xo_src_clk, type = rcg
+ name = cxo_clk, type = xo, fgroup = xo, wt = 10, bloo = blah
? type = hm_mnd_rcg, name = bo : type = rcg_mn
+ name = pxo_clk

I was hoping to do something like this:

@list = $_ =~ m{ ^[-+?] \s* (\S+) \s* = \s* (\S+) \s* (?:, \s* (\S+) \s* = \s* (\S+) \s*)* }xms;

Thinking @list would be assigned the alternating key/value pairs. But the above doesn’t extract anything sane. Adding the /gc modifiers doesn’t make any difference.

If I do the following, it extracts the first two key/value pairs correctly (if the line has more than one pair).

@list = $_ =~ m{
    ^[-+?] \s* (\S+) \s* = \s* (\S+) \s*
    , \s* (\S+) \s* = \s* (\S+) \s*
}xms;

If I keep repeating the pattern in the second line, it keeps matching more key/value pairs.

I would expect using (?: )* should mean zero or more instances of match inside the parentheses, but obviously it’s not working. What am I doing wrong?

When I’m presented with a problem like this, that is some kind of structured data, I immediately think of writing a parser. I’ll get back to that in a bit, but I wanted to address the confusion about capturing in the pattern. And, in fact, that’s how the discussion on the mail list proceeded.

Repeated Capturing

First, let’s simplify the example to demonstrate why our seeker of wisdom isn’t getting back the list of items he expected.

my @matches = 'a b c d e' =~ /^(a) \s* (?: ([bcde]) \s* )*/xms;

say "(@matches)";   # prints "(a e)"

Capturing parentheses in Perl are treated somewhat like registers. Most Perl programmers are familiar with the $n variables, which hold the values of a successful pattern match. For example $1 holds the value matched by the first set of parentheses, $2 holds the value of the second set, and so on.

When a pattern is matched in list context, as above, it’s effectively the same as writing,

'a b c d e' =~ /^(a) \s* (?: ([bcde]) \s* )*/xms;

my @matches = ( $1, $2 );

These pattern match variables are scalars and, as such, will only hold a single value. That value is whatever the capturing parentheses matched last. So, in our simplified example, $1 matches a, which is obvious enough. As the pattern repeats, $2 would be set to b, then c, and so on until the final match of e.

That explains why the pattern match wasn’t returning the expected list. What can be done about it?

Capturing Along the Way

If we break down the sample data, we see that it generalizes to,

prefix key = value[, ...] [: key = value[, ...]]

The first approach that came to mind is to split the data into multiple lines. Each line can then have its initial prefix removed and saved, then parsed for its key/value pairs. That’s starting to look a lot like parsing, which I promised to get to later. For the purposes of this discussion, I wanted to be able to accomplish the task with a single regular expression.

To capture all of the values we want, we need to remove the repeating set of non-capturing parentheses. However, we still need to repeat the match, ideally returning all of the captured values in one statement. We can do that with the /g and /c regular expression modifiers.

my @list = $string =~ m{ ([-+?,:]) \s* (\w+) \s* = \s* (\w+) \s* }xmsgc;

I’ve done two things here. First, I replaced the \S character classes, used to match the key and value, with \w. The + pattern in a Perl regular expression is greedy, so the former character class was also matching the comma used to separate key/value pairs in the data. This left the literal comma with nothing to match, so was one source of confusion.

Second, I noted that the initial prefix, while syntactically important, could be viewed in the same way as the comma and colon separators. I combined all of these separators and added a capture around them so we can later make sense of the parsed data.

When matched against the data, the pattern results in a list like,

("-", "name", "gcc_xo_src_clk", ",", "type", "rcg", "+", "name", "cxo_clk", ...)

Now we can process the data using a simple state machine.

my $state = undef;

while ( my $token = shift @list ) {
    if ( $token eq '-' ) { $state = 'dash'; next; }
    # ...
    if ( $token eq ',' ) { next; }

    my $key   = shift @list;
    my $value = shift @list;

    if ( $state eq 'dash' ) {
        # ...
    }
}

Even though we did all of the data extraction using a single pattern match, it looks remarkably like … a parser! The pattern is simply the tokenizer used to feed tokens into our state machine, the parser.

Parsing

I stated at the outset that I looked at this as a parsing problem, so the solution I would use is most likely a parser. For simple, one-off scripts, I’d use a technique similar to the one I described in the previous section. However, for more complex data or a more complex script, I’d turn to a real parser.

In fact, one of my contributions to the thread that led me to compose this post included an example of using the $^R and $^N variables in embedded code blocks to demonstrate a rudimentary parser that allowed a simulated form of capturing within a repeated non-capturing group. I won’t go into any detail beyond showing what I wrote. As this was from an early point in the thread, the prefix is ignored in this example.

my @list = ();

my $kv = qr{
    (\w+) (?{ $^N; })           # capture the key
    \s* = \s* (\w+)
    (?{ $^R = [ $^R, $^N ]; })  # capture the value, saving the key
    (?{ push @list, @{ $^R } }) # push the key/value onto @list
}xms;

$data =~ m{ (?: ^[-+?] \s* $kv \s* (?:[,:] \s* $kv \s* )* )* }xms;

Fortunately for us, there are parsing modules on the CPAN.

Prior to Perl 5.10, Damian Conway had written Parse::RecDescent, but with the introduction of grammar-like facilities like named captures and named backreferences, Damian improved upon his original work and presented the Perl community with Regexp::Grammars.

What does a parser for this data built with Regexp::Grammars look like?

my $parser = qr{
    <[Line]>+

    <token: Prefix>   <MATCH= ([-+?]) >
    <token: Key>      <MATCH= (\w+) >
    <token: Value>    <MATCH= (\w+) >

    <rule: Line>      <Prefix> <Pairs> <Options>?
    <rule: Pairs>     <[Pair]>* % ,
    <rule: Pair>      <Key> = <Value>
    <rule: Options>   : <[Option]>* % ,
    <rule: Option>    <Key> = <Value>
}x;

if ( $data =~ $parser ) {
    # Do something with %/
}

This is a trivial example and all the work is left to be done by inspecting the parse tree in %/. However, the module supports embedded code that will be called when a token or rule matches, which can be used to process the data as its parsed.

References

• perlre
• perlretut
• perlvar
• Parse::RecDescent
• Regexp::Grammars

After my near complete lack of interest in the keynotes I saw on Wednesday and Thursday mornings, I paid little attention to those on Friday. I thought the message Karen Sandler had about open health was good, but that’s about all I can say about them.

By far I was the most pleased by the sessions I attended on Friday. First, Kevin Falcone’s Shipwright: Application Distribution Simplified. Kevin works for Best Practical, a company with the best shirts. I plan on doing some evangelizing of Shipwright at work, as it would help a lot of people, including me, to better develop and deploy their applications.

I wasn’t planning on attending OSCON this year. I was perfectly happy skipping it and staying home during the last week of July. Then I happened to be looking over the list of Perl sessions and saw, at the very end of the list, Easy Distributed Computing with Perl and Grid::Request. It seems that Victor Felix has released a module that does exactly the same thing as some of the modules I’ve maintained at work, only the design is much better. However, it doesn’t support the batch system we use. I emailed Victor to discuss some collaboration and registered for OSCON so I could meet him. So yeah, I attended OSCON for one session. But it was worth it. The module looks great and Victor seems happy that I have an interest to contribute. It will be much better use of my time to contribute to a module on the CPAN than to continue pouring effort into what we have today.

Since, after chatting for a bit with Victor, I was already standing outside the room well into the next time slot, I popped into Git for Ages 4 & Up. Michael Schwern and Ricardo Signes demonstrated the Git commands everyone should know to get started with the version control system. As an added bonus, they used tinkertoys to help the audience visualize what Git’s internal representation of the repository looked like after each command. It was definitely a different and entertaining talk.

Prior to the closing keynote, Piers Cawley was invited to sing his library song, which I mentioned in Thursday’s post, again for the benefit of all OSCON attendees.

Paul Fenwick delivered the closing keynote. If you haven’t seen one of his talks, shame on you. Here, to help you fix that, I’ll refer you to his keynote, All Your Brains Suck—Known Bugs and Exploits in Wetware.

After three days in Portland, I finally ate at Burgerville. Eating at this regional chain is something I look forward to every time I’m in the area. Though, I suppose my change in diet may have suppressed my eagerness and led me to put it off until Friday. In any case, I ordered a cheeseburger with grilled onions (ditching the bun) and a large raspberry shake. While I prefer their blackberry shakes when available, the meal was delicious.

The high point of the conference happened, oddly enough, after it had ended. For whatever reason, I happened to wander into a different area of the convention center, in which a sock knitting conference was taking place. Outside of their expo hall was the Sockgate, a cardboard replica of a Stargate. As we were waiting to take pictures with it, Paul Fenwick happened by and offered to take some photos. He’s a really nice guy and I enjoyed finally getting the chance to meet him. After the photo op, he headed into the knitting expo hall. In retrospect, I should have done the same. It would have been interesting to see what it was like.

Photo Credit: Paul Fenwick

Finally, I learned that when I attend OSCON, I really do need to go for the entire week. Apparently, it takes me about two days to acclimate myself to the environment and really start interacting with people. Of course, by arriving Tuesday night, I was ready to interact on Friday, just as everyone was heading home. It didn’t help that I was staying in a hotel way out by the airport, with MAX service ending before 11:00 PM. With a new baby at home, I certainly don’t regret my choice to be away for a shorter period of time, but if I go next year, I’ll probably go for the entire week.

In any case, the Thursday morning keynotes were far better than those foisted upon us on Wednesday morning. Gabe Zichermann’s talk, in particular, caught my attention. In Game theory applied to user engagement in Open Source he talked about using so-called gamification techniques to draw people into using Open Source software. Many of his examples had to do with using game theory to alter real life behavior, such as a lottery to reward good drivers in Sweden or the use of consumption graphs in hybrid vehicles. On a separate note, I tend to grow annoyed at the latter, having been stuck behind too many hypermiling drivers.

Getting into the sessions, I favored those more in line with the work I do as a Perl programming system administrator. Also, it didn’t hurt that The Conway Channel 2011 happened to take place during the first time slot of the day. I’m a bit sorry I passed up DIY Clinical Trials (Or: How to Guinea Pig Your Way to Scientific Truth and Better Health), if only for the reason that it would have been completely different from anything I normally do. But, I attended those types of sessions on Wednesday, so it was back to business, so to speak. Damian Conway was in his usual top form, as entertaining as he is educational. I won’t go into too much detail, only to note that he demonstrated four of his modules, using a theme I’m sure most will recognize. First, something old, updates to the Regexp::Grammars module. He then introduced something new, the IO::Prompter module, which supersedes his older IO::Prompt. There was something borrowed, the Data::Show module, which serves as a convenience wrapper around the Data::Dump module. And finally, something blue, the Acme::Crap module, which seems oddly cathartic.

I like to think I’m a halfway decent Perl programmer, but that doesn’t mean I think I can ignore things like Jacinta Richardson’s Perl Programming Best Practices 2011. The talk was a round-up of the tools and modules that are generally considered to be the best practices by the Perl community today. Yes, generally. People will have their differences of opinion, and I don’t always agree with the advertised best practices. However, if followed, the practices will lead to better code, and if violating a practice, I like to be able to back that up with a well thought out reason (it doesn’t necessarily have to be a good reason). The first of two, possibly pithy, examples of this is the local::lib and it’s default use of ~/perl5 as its include path. I prefer to use ~/local/lib/perl5 and, sure, the module allows me to do that easily enough, but it’s an extra, non-standard step. Second, the cpanm has been touted as the best way to install modules from CPAN. As a control freak with a highly customized CPAN configuration, I’ve never liked the way cpanm seems to do things its way. Admittedly, it may be customizable, but I’ve never had the need to look into it.

There’s been some noise around the office about testing Amazon’s EC2 offering. To that end, I thought James Loope’s Utility and Automation: Low Overhead Operations with Amazon & Puppet would be educational, possibly giving me some ideas about how to managing our own potential EC2 environment. Unfortunately, it didn’t work out that way for me. The talk was heavily focused on the way the web application was designed and pieces of Amazon’s infrastructure were used. We’re not creating or running web applications, so none of it was beneficial to me. There was nothing about Puppet aside from explaining that using it (or another configuration management tool) is vital for keeping everything running.

At this point, I was turned off from any cloud talks at OSCON. There seems to be, with probably good reason, an inextricable tangling of cloud and web applications. Because of this, I decided to pass on Achieving Hybrid Cloud Mobility with OpenStack and XCP and instead attended Piers Cawley’s Polymorphic Dispatch—It’s Not Just a Good Idea, It’s the Law. I’m glad I did, because there were definitely some very useful ideas presented. The idea, taken from Smalltalk, of passing messages to objects has a lot of merit. Combining this with polymorphism, sending a message and allowing different objects to act on it differently, vastly simplifies code. Simple code, of course, is easier to test and easier to debug when things go horribly wrong (and actually is less likely to go horribly wrong in the first place). Of particular interest to me were the Null Object pattern and what Piers referred to as the key tenant of object-oriented programming: tell, don’t ask. That is to say, if I understood correctly, instead of querying an object for information and using it to determine which action to perform, give the information to the object and have it perform the action. Finally, Smalltalk Best Practice Patterns was recommended as the best book on good coding practices out there. According to Piers, it “will change the way you think about programming.”

I was in way over my head in Tom Christiansen’s Unicode in Perl Regexes. The only thing I managed to learn is that I don’t know nearly enough about Unicode to actually understand using it. I’ll leave it at that. It was a very information-dense session and it’s possible that Tom knows more about Unicode than those who designed it. Other choices during this time slot, which may have been better for me, were Connecting iOS to the Real World with Arduino, presented by my friend Alasdair Allan, or, venturing again into the realm of health geekery, Open Source Preventive Medicine: Citizen Science Genomics

The last session I attended on Thursday had so much potential, but, for me, it fell flat. I expected A. Sinan Unur‘s Visualizing Economic Data Using Perl and HTML5′s Canvas to focus far more on visualization than it did. Instead, the majority of the presentation was about the difficulty of parsing public data published by the United States government. For this, Sinan uses Spreadsheet::ParseExcel and explained a few of the techniques he uses to extract data from tables designed primarily for visual consumption. Unfortunately, very little time was spent showing how Canvas was used. We were given one static example and an explanation that there is no method available for determining the height of text in a Canvas element. I had hoped to return to work with some ideas for using Canvas to visualize data from our batch scheduling system, but ultimately left disappointed.

After the last session, I met up with a coworker, an old friend, and a new friend to have dinner at Chipotle. Normally, I like to avoid chain restaurants—national chains in particular—when traveling, preferring to sample the local cuisine. But, we wanted a quick dinner and it was nearby. My opinion was requested, on the relative healthfulness of pinto versus black beans. I simply stated that I would be ordering my carnitas bowl without any beans.

After dinner, we returned to the convention center for the Perl Lightning Talks and the State of the Onion. As always, the talks were quite entertaining. Of note was a juggling demonstration, illustrating various programming languages and databases. Near the end, Ricardo Signes recounted a conversation he had with a couple of women from the knitting conference sharing the convention center with us. Its presence provided a wonderful juxtaposition. While OSCON is male-dominated and many don’t know how to act when women brave their way into our midst, the knitting convention is completely opposite. Ricardo’s message to us was, take the time to look up from our laptops and chat with those around us. We might just have a better time and make new friends.

Finally, Piers Cawley favored us, as he does every year, with a song. This year, however, he did not bear a tale of levity, but a message of deadly seriousness. The United Kingdom is closing libraries in an attempt to reduce public spending. As a protest, Piers wrote a song, “Child of the Library”. There doesn’t appear to be any video (yet) of Piers performing at OSCON, but I’ve gone ahead and embedded one that I found. It’s catchy, I had it stuck in my head for a couple of days after the conference.

We could easily see the same thing happen in the United States—and in fact I have already seen it proposed in San Diego. I’ll first admit that I have not set foot inside a library since college, over a decade ago (high school, if only counting public libraries). Do libraries still matter, or is the concern over their closing merely the knee-jerk nostalgia of those of us who came of age in a world that didn’t yet know the Internet? I can’t, and won’t, take a side on this issue until I’ve taken the time to visit my local library. If I can recognize it as something I saw in my childhood, perhaps it should be closed. If it has adapted to the so-called Information Age, maybe it’s worth funding.

As a final, humorous note, I almost didn’t make it back to my hotel. At least, not without finding an alternate method of transportation. At 10:22 PM, excusing myself and apologizing for staying so far away from the conference, I left Media Temple party at the Jupiter Hotel, arriving at the convention center MAX station at 10:32 PM. The schedule at the station listed 10:42 PM as the last red line train to the airport, with Google Maps concurring that a train was 10 minutes away. About two minutes later an unmarked blue line train arrived at the station, traveling east. At this point, Google Maps had decided it would rather show me its trip planner instead of the previous screen which showed the impending arrival of the red line. Forced to make a split-second decision, I hopped on the train. I knew that I could take it at least as far as the Gateway station, where I could transfer to the red line if it was still behind me. Around 11:00 PM I arrived at Gateway, after spending the ride thinking about how much a cab would cost. This station had a real-time display with train arrival times. The last red line of the day was only three minutes out. Whew.

I released a new version of a command line program. It’s an elegant piece of work, combining a marvelously complex-but-intuitive configuration for system administrators with a absolutely simple interface for users. To use the command, the user runs it with a couple of arguments and it prints out a single line of useful text derived from the marvelously complex configuration.

But, it doesn’t print a newline.

Anyone who has encountered a command like this knows well my irritation. You end up with something like this:

my awesome prompt> some_lame_command
my awesome prompt>e answer

Argh!

The workaround most of us use is to see the above, face-palm, then run something like this:

my awesome prompt> echo `some_lame_command`
42 is obviously the answer
my awesome prompt>

Being the arrogant bastard programmer that I am, I decided to fix this. Since all commands print newlines, everyone should already be assuming that this one does too and should already be handling it in the proper manner. When writing a shell script, the distinction between newline-printing and non-newline-printing commands is irrelevant. In either Bourne shell,

FROBBED=`frobnosticate`

or C shell,

setenv FROBBED `frobnosticate`

the shell is benevolent enough to remove the newline, if it exists. After all, this is the most commonly desired behavior when assigning command output to a variable. However, things are a bit different when switching to a programming language, like Perl:

$ENV{FROBBED} = `frobnosticate`; # Caution, newline ahead!

Sure, it looks more or less the same, but veteran Perl programmers will immediately grimace when reading the above. Unlike the shell, Perl, like other programming languages, will preserve the output of the command. In this case, preserving data and letting the programmer decide how to use it is the most commonly desired behavior. Since everything coming from an external command ends with a newline, the environment variable being set in this case will have a newline. This will almost always cause a problem. One that, as I’ve learned, is not always easy to find. Since stripping input of newlines is just as common as the desire to preserve data, Perl makes this easy and most Perl programmers will habitually write this:

chomp( $ENV{FROBBED} = `frobnosticate` );

Now it doesn’t matter if the command prints a newline or not, the chomp function has your back. It’s just like being in the warm embrace of the shell, only with a little extra syntax.

So it turns out that one of the engineering groups I support was using a Perl script that set an environment variable as in the first example. The value of this environment variable was then being passed off to the batch system and used by an engineering program as a network address to connect to. Of course, the program made the fatal mistake of trusting user input and, in a spectacular fashion, failed to connect to the server whose name just happened to contain a newline.

After chasing down a couple of red herrings which left me flummoxed, one of the affected users shared with me an error log and the script that generated it. There, in all its syntax highlighted, monospaced glory was the environment variable being set without attempting to trim off the newline. I quickly released an update that reverted the newline behavior and the problem went away. My engineers—at least, the subset using this particular script—could once again get their work done.

By far, this isn’t the worst thing I’ve done to our batch system. One time I caused all jobs that launched on Solaris hosts to immediately fail. Whoops.

Anyway, what’s the lesson to be learned from today’s experience?

Never—and I’ll repeat that, never—assume everyone will be doing the right thing. Inevitably, someone won’t be.

There’s a corollary to today’s lesson. When coming across something that could be improved with a small change, don’t. Seriously, just don’t. Inevitably, someone will be depending on the current behavior, no matter how right or wrong it may seem.

Tuesday was the second and final day of the tutorial sessions. In the morning I attended a tutorial on PostgreSQL’s new hot stand-by and streaming replication features; and, in the afternoon I attended part of a tutorial on Cassandra. Why only part? I’ll get to that.

I didn’t feel like going across the river to the food trucks for lunch, so I joined Debbie for lunch at Burgerville. Aside from the delicious food made from local ingredients, there are two things that struck me about Burgerville. The first I noticed when I walked in the door: for the first time, disposing of my trash would require me to read instructions. Burgerville uses three bins for trash: one for recyclable materials, one for compost, and finally one for trash that can neither be recycled nor composted. I thought this was neat, though I did get a kick out of the soft drink cup. It’s from the Coca-Cola company and advertises itself as something that can be composted; with the footnote that this was only possible in a large facility capable of composting such cups. Not something one can throw into their garden compost pile, I guess. The second thing I noticed caused me immediate regret: the receipt lists the calorie count of the foods ordered, along with carbohydrate and fiber content. Looking over the details of the burger, onion rings, and raspberry milkshake I ordered, I decided that it would not be a very paleo day for me. Oh well, the milkshake was very good.

While enjoying our carb-loaded, calorie-filled lunch, Debbie noticed someone wearing a pair of Vibram FiveFingers that we hadn’t seen before. From a distance, they looked almost like normal shoes and appeared to be made with a dark brown suede. With both of us deciding that a post-lunch, calorie-burning walk was called for, and sharing a desire to buy a new pair of FiveFingers, we set out for Portland’s REI store. A trip on the MAX, a walk, a few blocks on the trolley, and another walk brought us to the store.

The shoes turned out to be the KSO Trek. They’re very nice and I’m considering purchasing a pair for hiking. Unfortunately, I struck out on the trip. REI has been having a hard time keeping FiveFingers in stock, so I wasn’t able to find or buy a pair of the Classic version. Fortunately, I’m still satisfied with my KSOs, which I was wearing at the time.

Our impromptu quest for footwear took us well beyond the alloted time for lunch. Fortunately, this time was not wasted. While walking, we had received a call from our coworker back in the expo hall, who needed help setting up the QuIC booth. For some reason, it was fun being allowed into the expo hall while booths were still being constructed. Not sure why, other than that I enjoy seeing things taken apart and (sometimes) being put back together. After getting the booth set up, I made it to the second half of the Cassandra tutorial. I’m told by those who attended the first half that I didn’t miss much.

We had some time to kill between the end of the day’s sessions and the evening’s Ignite talks. So we walked a few blocks to a place called rontoms. Had I not been looking for the specific address, I would have walked right past, not noticing that this was either a restaurant or a bar. The cavernous interior was devoid of anyone save the bartender and a waitress, who would disappear as quickly as she appeared. The photographs on the wall, ost of which featured a man in an animal costume, ranged from strange to disturbing. After a moment’s hesitation, we ventured out back to find a patio crowded with patrons enjoying food, beer, and spirits. With what appeared to be only a single waitress working and not having particularly strong appetites, we went back inside, obtained pints directly from the bartender, and found a comfortable area to sit and chat. Twice we encountered people entering the restaurant, looking for people they didn’t know by sight. Both times my colleagues convinced them that we were those people; one girl even sat down with us for a few minutes before we let her in on the joke. After a while, I received a page from Jonathan that there was beer, salami, and cheese being served outside the ballroom at the convention center. This sounded like an excellent and delicious dinner to me, so I made my way back.

I hadn’t been to an Ignite session before, so I was looking forward to this one. Right off the bat we were warned that we would likely enjoy some talks and dislike others. Fortunately, each talk would only last five minutes, so we were free to use the time to retrieve another beer. By the time we returned, the talk would be over. I don’t believe I took advantage of this, instead waiting for the break, during which some awards were being presented.

Two talks stand out in my memory. The first, perhaps appropriately, was the first in the lineup: Paul Fenwick talking about Maximum XP: Optimising life for adventure (which he gave again, at a much better pace, at the Perl Lightning Talks). Presented in song, Paul’s message seemed to be to enjoy travel and to take advantage of opportunities to meet people and have fun. Based on what I’ve read on his Twitter stream, I’d say he’s been successful.

The other talk, Your Infinite Do-Loop Exercises Bores Me, struck a chord with me. John Scott and Jim Stogdill paired up for this talk, one would perform exercises while the other would speak, switching places at the halfway mark. Not only was it refreshing to see a talk about fitness at a convention populated by a class of people not known for their physical exertion, but it was about a method of fitness I’ve recently become interested in. While I don’t practice CrossFit myself, I frequently look at the exercises on the site and prefer it to the typical, repetitive gym workout. They also mentioned the paleo diet, which, along with the primal lifestyle, I’ve become a big fan of.

My coworkers all turned in early, so I hopped back on the MAX and headed downtown to have drinks with Kevin at Bailey’s Tap Room. I had a wonderful sour beer, which I no longer remember the name or origin of, and had the pleasure of meeting Steve, Jeff, and Michael Schwern. Jeff and Schwern were discussing the use of the Log4perl module in the latter’s gitpan project.

After last call at Bailey’s, I caught the last yellow line across the river and turned in myself.

Larry claims that so rarely does he talk about Perl in the States of the Onion addresses that he has brought his conscience with him today to prod him in the right direction (the aforementioned angel and devil).

The current state of the onion is segmented into left, central, and right sections. It can be labeled, say, 5 and 6. They can also be labeled 0 and 1, for false and true. Larry then asked a series of boolean questions, asking the audience to weigh in on the veracity.

Do you think Perl 5 and Perl 6 are really the same language?

Do you think Perl 5 and Perl 6 are really different languages?

As the angel and the devil argued, Larry pointed out that an important skill for a language designer is to be able to stay on the fence long enough until he can determine which side the grass is greener on. Sometimes you discover that you’re sitting on the wrong fence and the voices in your head start to argue about which side has the greener grass.

When the voices in your head start arguing if the purple cow eats greener grass than the brown fence, it’s time to see a doctor. Or find a better drug dealer.

— Larry Wall

This is, of course, a metaphor for being a language designer. Sometimes you sit on the fence for language features, without ever knowing which direction is the better one.

Next up is a live demo of Perl 6; or, more specifically, of Rakudo Star, which is scheduled to be released next week. Some of the demos, without comment:

.say if 6 %% $_ for 1..^6

[+] gather { take $_ if 6 %% $_ for 1..^6 }

[+] grep { 6 %% $_ }, 1..^6

~[+] grep 6 %% *, 1..^6

-> $n { $n == [+] grep $n %% *,  ..^ $n }

-> $n { $n == [+] grep $n %% *,  ..^ $n }(6)

At this point, the examples scrolled off the screen due to a “whatever” example being run. That’s good news, though. It means Rakudo Star supports lazy lists and, as such, we finally have those infinite lists we’ve been promised:

0, 1, ... *

The whatever star can, in addition to being used as in an infinite series, can be used to curry a function:

(1, 1, *+* ... *)[^20]    # Fibbonacci

(0, !* ... *)[^20]        # 0 1 0 1 0 1 ...

In a recent video interview, Larry was asked, if he were hit by a bus, has he designated anyone to be his successor as the leader of the Perl 6 project? His response was that he trusts the Perl community to choose the right person.

Onions can make you cry, so can disruptive technologies or innovations. Almost everyone has labeled their technology as disruptive. As such, the phrase has lost most of its meaning.

A disruptive technology simultaneously does something worse and does something better than its competitors. In a time of the Unix philosophy of “do one thing and do it well,” Perl came along and attempted to do everything, but didn’t necessarily do any of it well. The Unix philosophy was broken by its own utilities. No one knew what a “thing” was, and no utility of the time did it well. By the time Perl 4 turned into Perl 5, it demonstrated that a tool that was itself an entire tool shed could run circles around shell scripts.

In California, we once had many, many colonies of ants. Now, most of California is populated by a single colony of Argentine ants. This is because the colonies have forgotten how to fight with each other. Perl 6 has benefited from multiple teams creating multiple implementations, in the end working together to create a better product, even if that product takes longer to complete.

If you don’t like Camelia, you can just fork off.

— Larry Wall

The takeaway, I think: It is up to all of us to determine what Perl 6 will be. What kind of disruptive technology will it be?

Ever since I saw An Illustrated History of Failure two years ago, I’ve made it a point to see @pjf‘s talks. That’s how I find myself in his mid-afternoon session, Awesome Things You’ve Missed in Perl. Judging by the size of the crowd, I’m not the only one. However, I won’t attempt to pass along his humour in this post. I’d never do it justice.

In his introduction, Piers Cawley asked that we go wild when Paul took the stage, so the folks in the Google Wave session next door would be taken aback, and realize that Perl is not, in fact, dead.

People are still out there writing Perl as if still in the dark ages of 2008. Paul doesn’t want us to write old Perl, but only new and shiny Perl. This talk only covers practices that have come about since Perl Best Practices was released.

Object-oriented Perl is not awesome. Not even close. If you look at the old ways of doing it, all of them are either wrong, stupid, or both. The rest are too hard. There’s a simple way to fix this: use Moose. This module does so much of the infrastructure work of composing classes, it makes object-oriented programming enjoyable again.

Paul spent a lot of time giving a humorous, high-level overview of the features available in Moose.

The Moose module contains a huge number of extension modules in the MooseX namespace.

When I have a problem, I go down to the pub with other Perl mongers and bitch.

One of the limitations of Perl, that is exposed to Moose, is that not everything is an object. This means methods like push() or isa() can’t be called on everything. And checking types defeats the purpose of polymorphism. Enter the autobox module, which turns everything into an object. As a bonus, it operates in lexical scope. Moose exposes autobox through the Moose::Autobox module.

A module that Paul wrote, autodie, which is now included in core. This lexically scoped module removes all of the boilerplate code that goes along with trapping errors from subroutines.

Not only is Perl 5.10 awesome, but Perl 5.10 regular expressions are awesome. In particular, the introduction of named captures (via %+) made regular expressions extremely awesome.

Perl 5.10 also provides grammars in the regular expression engine. This is the basis for Damian Conway’s Regexp::Grammars module.

Referring to an article on SweeperBot in The Perl Review. However, there’s the problem of distributing a program that uses half of CPAN to users of inferior operating systems, such as Microsoft Windows. That’s where the PAR module comes in. It will pack up all of the modules used by the program, including the Perl interpreter itself if necessary, so a single, self-reliant file can be distributed to users who need it.

Remember to never optimize code. Programmer time is far more valuable than CPU time. However, when you must optimize code, profile first. The Devel::NYTProf makes profiling awesome.

Code reviews are important, but Perl programmers are lazy. Fortunately, the Perl::Critic module has read Perl Best Practices for you and will complain about where your code violates the practices in the book. At my day job, it does about half the work of code reviews for me, loudly announcing violations of the coding standards that I enforce with an iron fist.

If you find an awesome module, buy the author a beer if you have the opportunity. There’s also CPAN Ratings to leave feedback or perlthanks in recent versions of Perl.

The full title of this session is, 21st Century Systems Perl – the New Perl Enlightment for sysadmins

Introduction

While Perl isn’t dying, “PERL” most certainly is dying. This is a good thing, because it includes all the really crappy stuff, such as Matt’s Script Archive. Thank goodness for that. To be fair, this code would have been horrible written in any language. Remember, blame the artist, not the tool.

We have a very mature community, which means we also have very mature practices. We are also converging on a standard platform, even if there are more than one ways to do something.

Part 1: Minimising Developer Fatalities

As a developer, we should do what we can to make our sysadmins’ lives easier.

Right off the bat, we should use the local::lib module, which allows an application to use custom library areas without polluting the system installation areas. It can even work with /etc/skel. Matt is a big fan of using a local library path, included with the application, so it can be maintained separately from both the operating system vendor’s modules and even other applications.

Improve module installation using Module::Install.

Package modules for your distribution of choice using cpan2dist.

Improve the CPAN experience using App::cpanminus, which is amazing easy to bootstrap:

> wget cpanmin.us
> ./cpanm

Start using all of the modules associated with best practices by installing Task::Kensho.

Vendors are getting better at distributing Perl and keeping up with module releases. The Debian Perl team is the strongest, with Fedora lagging quite a bit far behind. Fedora is finally getting better, now that members of the Perl community have a say in the packaging of Perl and the modules.

After many debug sessions, Matt has come to the conclusion that mod_$lang is evil. Jamming languages into the web server is a bad, bad idea. However, actually hooking into the different handlers can be useful. Matt’s preference now is now FastCGI.

Part 2: Maximising Automation Banality

“In the systems world, shiny and exciting is not good.”

Use the autodie (in core as of 5.10) and the IPC::System::Simple modules to reduce the repetitiveness and the common errors of systems programming.

Use IO::All to fix the syntax and semantics of I/O operations.

Systems script shouldn’t need to be deployed. It should be possible to just drop the script onto a host and it will Just Work. That’s where PAR::Packer.

The full title of this talk is, Dist::Zilla – Maximum Overkill for CPAN Distributions.

Every CPAN distribution contains a significant amount of crap. It’s infrastructure used for the distribution tools.

ExtUtils::MakeMaker has been the traditional way to work on the infrastructure code. By necessity, it contains a lot of legacy, which can be cumbersome to maintain. Enter Module::Install, which can look in the expected places for the necessary information, such as the author name. But, the author still must write all the boilerplate. Module::Starter was written to address this, composing all the boilerplate on behalf of the author. There is so much boilerplate that, by default, Module::Starter also provides a boilerplate test to detect it.

Why are we doing all of this? How much repetitive work are we doing?

What can Dist::Zilla do for us? For starters, we can remove some files:

• LICENSE
• MANIFEST.SKIP
• Makefile.PL
• README
• t/pod.t
• t/pod-coverage.t

Leaving us with only our Changes file, our code, and our tests. The non-infrastructure parts. On top of that, Dist::Zilla does all of the boring distribution bits for us. It only handles the make dist command. It does not handle the make install command, which means the users who install the module don’t need all of the dependencies.

Dist::Zilla puts all of its functionality into plugins, which will be the meat of the rest of this session. It also uses a very simple INI-style configuration file.

The main command provided by the module is dzil build. This bundles the distribution, which will contain all of the infrastructure necessary for users to install the module. When building, it follows a simple work flow:

1. Gather files
2. Munge files
3. Collect metadata
4. Write out

There is no default configuration, but there is a Basic plugin bundle that will include all of the most common plugins.

What followed were examples of what the plugins can do. Of course, all of them are designed to reduce cruft—the non-code, non-documentation bits that we’re forced to maintain. The philosophy is the same one I advocate to anyone who will listen: computers are good at doing boring, repetitive tasks with derived data; why don’t we let them do more of that stuff?

I’ve followed @rjbs on Twitter for a while, and I’ve seen him talk about Dist::Zilla. I’ve wanted to try it out for a while, to simplify my distributions—both for CPAN and for my day job—but I didn’t realize until this session just how awesome the tool is. It’s a complete framework for managing Perl module distributions. Dist::Zilla will give my Laziness score a huge bump.

After a long break, an apple, a cup of coffee, and a beer, I’m back in the Perl track.

The full title of this session is, Scratching the 40-Year Itch of Inheritance with Smalltalk-style Traits.

This is not a tutorial. How to use traits is easy, but why to use them is a more complex discussion.

Inheritance is a very complex problem and an easy one to get wrong. Then people start doing things with multiple inheritance and, even if they’re not doing something deliberately stupid, they end up with diamond inheritance. Not only is this a problem, but it’s been a problem for a very long time—40 years, in fact.

Complex systems can lead to deep class hierarchies. When hierarchies are deep, in particular with a dynamic language like Perl, it becomes difficult to determine where a method came from. Even when its known where a method comes from, undesired behavior may be inherited. This becomes worse when multiple inheritance is used.

As systems grow, the problem becomes two-fold:

1. Class responsibility – larger classes are desired
2. Class reuse – smaller classes are desired

Inheritance, by itself, cannot solve this problem. So the solution is to
decouple the sub-problems.

Several solutions have been tried:

• Interfaces
• Delegation
• Mixins – incredibly popular

As expected by the name of this session, traits (or roles in the nomenclature of Moose) solve the problem far better than any of the above solutions. Much of the session involved showing real-world application of roles to clean up code at the BBC.

I’m just back from lunch at Burgerville with Juan and Jonathan. On the way back into the convention center, I ran into Alasdair, who has been attending the hardware hacking sessions. That made me think that I may want to try to find non-Perl sessions to attend. After all, I tend to keep up with Perl news, so the sessions are of marginal usefulness. Unfortunately, nothing on the schedule looked very interesting to me. I was curious about the session on Open Source Tool Chains for Cloud Computing until I read the description. While it looked cool, it wouldn’t be useful for me in my work. The session would go through provisioning, setup, and maintenance of hosts, all of which we already have well-entrenched solutions for in my day job. So, I ended up back in the Perl track. My friends in the San Diego Perl Mongers group will appreciate that, I think.

Anyway, on to the session.

The name Perl 6 is a language specification, rather than any particular implementation. All of the references and links off-handedly mentioned in this post are available from the Perl 6 website.

Patrick is the lead developer of Rakudo Perl, which is the most feature complete and up-to-date.

Perl 6 has a language specification and a test suite. There are still many places in Perl 6 that are not being tested yet.

Rakudo * (Star) is scheduled to be released a week from tomorrow, targeted at being a useful, usable, early adopter distribution.

At this point, Patrick began to enumerate the new language features and how they work in Perl 6, such as variables, loops, interpolation, and so on. I won’t go into these here, since there are numerous places on the Web where this has been documented.

About half way through this session, I realized that “r0ml” was presenting in another room. If I’d noticed that before, I would have attended that session.

This talk could be titled something along the lines of “Lessons Learned from Project Management.”

Jesse Vincent is the current Perl 5 pumpking, which for the moment can be thought of as the project janitor.

People who say “Perl is dead” or that Perl hackers are “desperate” are behind the times.

There are a lot of exiting things happening that are not in the Perl core. Audrey Tang has said that “CPAN is the language, Perl is the syntax.” Like Piers in the previous session, Jesse enumerated a handful of things that make Perl awesome:

• Moose
• Plack
• cpanm – makes installing CPAN modules Just Work
• Devel::Declare
• Devel::NYTProf

While some of the coolest new things happening in the CPAN world, it merely scratches the surface of what is available.

About three months ago, Jesse uploaded Perl 5.12. Amazingly, no one has reported any critical regressions.

Jesse has been assured that Rakudo * will be out next week, on 29 July. However, Perl 6 will not replace Perl 5, which has paid Jesse’s mortgage for many, many years. Also, thanks to Perl 5.12, Perl 5.10 is no longer “too new to use.”

Perl 5.12 marks the latest release in the process of cleaning up the inernals and adding much desired features. Some of the highlights:

• Deprecations warn by default
• suidperl is dead
• package Foo::Bar 1.0; – better version import syntax
• Y2038 compliant – thanks to Schwern
• Unicode improvements; upgrade to 5.2
• Pluggable keywords
• Overridable function lookup
• Dtrace support
• Deprecated modules – Class::ISA, Pod::Plainer, Shell, Switch (but still on CPAN)
• Yadda, yadda, yadda operator

Jesse believes the best new thing in Perl 5.12 is the release process, including him as the pumpking. Twenty years ago, Perl didn’t use version control. He recommends learning from this mistake.

It took five years to release Perl 5.10, after burning through two pumpkings.

Before 5.12, maintenance releases contained all sorts of bug fixes and updates, but could not break binary compatibility. Doing so was a huge task, was very difficult, and, contrary to its name, is unmaintainable. Even without all this work, the pumpking’s job is a lot of work. Jesse really doesn’t want to burn out after a release of Perl.

Traditionally, the process of turning someone with the necessary skills to be the pumpking involves preventing them from using those skills and replacing them with management skills. It’s a shame.

The system is broken and Perl 5 isn’t going anywhere, so how can it be fixed? We can reinvent it, but that’s already being done by Perl 6. Alternatively, we can refactor it. There is no reason many of the skills and duties required of the pumpking can’t be delegated out to people with those skills. In effect, the most important skill and duty for the pumpking is project management.

The 5.9 releases, leading up to 5.10, were haphazard. The 5.11 releases, leading up to 5.12, have settled into a new release every month on the twentieth, with a couple of exceptions. The 5.13 series has followed suit. One of the reasons this was possible was documenting the entire release process.

Releases in the 5.12 series are on a fixed schedule, every three months. A release schedule has been created for 5.14, too.

One of the things I’ve learned working in an enterprise and my observations of the Fedora Project is that good project management is vital. Jesse Vincent is exactly what Perl needed and he continues to demonstrate that, with regular, high quality releases of Perl. What’s more, he is a good spokesman for the project, being able to come to OSCON and give a session on all of this detail in a cojent and interesting format.

After reviewing today’s session schedule, I quickly came to the conclusion that I will spend my entire day sitting in the room “Portland 256.” This is, apparently, where the Perl track is located.

Paul Fenwick introduced Piers in song, to the tune of Gilligan’s Island.

Piers switched from Perl to Ruby a while back and swore that he wouldn’t return to Perl until 6. Facetiously, the reason he switched to Ruby was the handsome community associated with it and he reason he switched back to Perl was the amazingly supportive community associted with it. He began with a point about programming style. We think of code as describing what we are doing, but in reality the majority of our code actually describes how we are doing it. This infrastructure code is noise.

More seriously, he absolutely hated unrolling the @_ variable in every function. In such a high level language like Perl, why must we pop arguments off the stack in the same manner we would in an assembly language? This leads to long subroutines, every single one containing anti-patterns designed to implement the language infrastructure, instead of the language doing the work for us.

Moose does a lot to improve writing classes, using a more declarative syntax. However, even within Moose methods we need to write the infrastructure code. The MooseX::Declare module solves this problem, giving method syntax a more declarative style. By moving the infrastructure code out of sight, we can better focus on what we are trying to do, rather than how we are doing it.

Piers proceeded to list the modules that “rock” and brought him back to Perl:

• Plack
• Devel::NYTProf
• Moose

Perl’s object-orientation absolutely “sucks.” However, this turns out to be a good thing. It allows very clever people to create modules that extend the semantics of the language. In a language like Ruby, which has a good object-orientation built-in, it’s essentially stuck. If, in the future better ideas of object-orientation are developed, they can be implemented in Perl far more easily than in Ruby. An interesting point: sometimes when the tool sucks, things are better. People develop layers of tools that enhance and extend the original.

It also helps that the Perl release schedule has accelerated.

Piers continued with a high-level, hand-waving explanation of how MooseX::Declare works. While not informative, it was entertaining. Including a video of Matt Trout attempting to hypnotize the room.

Piers ended by thanking the Perl community and expressing how good it feels to be back into it and developing in Perl again.

Based on past experience, this is likely to be a fast-paced, highly-entertaining talk. One which will be impossible to summarize, or no doubt even to explain, here. Needless to say, if you’re not here, you’re missing out. I intend to sit back, relax, and enjoy.

He’s talking about the defining characteristic of a hacker. Particularly when they’re told that something is impossible and can’t be done. The reaction is typically, “you wanna bet?”

He just presented a slide that read, “Let’s leave behind the shackles of sanity…”

Now I’m scared.

This is a great talk. It’s a series of examples of things “you can’t do in Perl.” At least, not until Damian shows us how.

I think Brad may have taken notes. Which is good, because now I wish I had.

[tags]oscon, oscon08, ocon2008, Perl, Damian Conway[/tags]

There are so many holy wars debates about whether one language is better than another. Instead, the right question to ask is whether or not the developer’s skill set is right for the job. I agree. When I look for a developer, I’m more concerned with how they think than in what language they think.

Unfortunately, Tim is preaching to the converted in this talk. Nearly the entire attendance already uses Perl and don’t believe the myths. With that, let’s conquer them anyway.

Perl is Dead

No it isn’t. It’s two decades old and still growing strong. The books aren’t flying off the presses with great speed because the Perl community already has excellent books.

The trend when searching for “web development” jobs shows Perl growing very slowly in relation to other languages, particularly PHP. However, searching for “developer” jobs shows Perl growing very strongly and holding its own extremely well.

As a lurking member of the Perl community and an active member of my local Perl Mongers group, it’s been my experience that Perl programmers tend to be quite happy with their jobs. Which, unfortunately, has made it very difficult for me to find talent.

In fact, Perl is growing faster than ever. A simple look at how much work is going into CPAN will show that. The community is strong and Perl is everywhere.

Perl Is Hard to Read / Test / Maintain

Only if you’re doing it wrongly. We have Perl Best Practices, to use as the default documentation for coding standards, leaving developers with the need to only document when they deviate from the norm. There’s Perl::Tidy, to force any Perl code into one’s own personal style. Perl::Critic for ensuring that code is being well-written and follows best practices. And there’s no end to the Test::* modules and the work being done to make testing easy. There’s even a coverage analysis tool.

Perl 6 is Killing Perl 5

In fact, Perl 6 saved Perl 5, but one has to be close to the center of the community to see that. One should notice that Perl 5.8 and 5.10 have both been released in the time that Perl 6 has been in development.

There is a culture of testing around Perl. So many tests have been written for Perl 6, and the language is being defined by its test suite. This culture has leaked out to the community. In fact, I find there now exists a lot of peer pressure in the community to do proper testing.

Perl 6 Is Not Perl

Yes, and no. Unfortunately, I was so busy trying to catch up with the last section that I missed most of the points Tim made. In the end, I feel that this is fine. If Perl 6 was supposed to be Perl 5, why not just use the perfectly decent, already existing Perl 5? Which is still being actively developed.

Perl 6 Will Never Be Ready

It’s not on a schedule and, if it were on a schedule, it would be crap. It will be ready when it’s ready. Better to do it right than screw it up. The development model encourages a lot of experimentation, and it’s difficult to schedule experimentation.

There’s No Perl 6 Code

Sure there is. Thousands of lines of Perl 6 code exist in the test suite that came about from Pugs. These very same tests are being used in Perl 6 development today in the form of Rakudo, Perl 6 on Parrot.

The important thing to note is that Perl 6 refers to a specification. It does not refer to a particular implementation. Any implementation that passes the test suite may call itself Perl 6.

From an authority in the audience (who I don’t recognize, unfortunately), we have been told that there will be a useable Perl 6 by this Christmas. A round of applause ensued.

[tags]oscon, oscon08, oscon2008, Perl, Tim Bunce, myths[/tags]

If he had Perl to do all over again, what would he do different? Only two things, nothing, and everything. Perl 6 is the everything part of the answer.

In Perl 5, one of the problems that creeps up is that regular expressions (regexes) are strings. The best example of this is variable interpolation in regexes. In Perl 6, this has been fixed. They are now their own language.

Like cargo-cult programming, parsing has turned into its own cargo-cult. Perl 6 breaks the mold when it comes to copying languages (the old lex/yacc loop), and instead uses polymorphism in its sub-language design.

Both regexes, double quoted strings, and single quoted strings are examples of sub-languages in Perl 6. Each of these sub-languages has its own parsing rules and therefore parsing implementations. This allows is code reuse. Parsers can derive behavior from other parsers, but treat the tokens differently as necessary.

Fundamentally, Perl 6 is very simple. It has no CORE. It has no built-ins and no operators. What Perl 6 has given us (will give us?), in effect, is a just in time lexer. Tokens and their behavior can be defined on the fly, on a per-sub-language basis.

There are quite a few changes to the regularity of regular expressions. Mostly what this means is that Perl 6 regexes are incompatible with those used in Perl 5, and that Perl-compatible regular expressions (PCRE) aren’t (or won’t be).

All languages tend to fall into the One True Syntax trap. Perl 6 has aimed to break out of that trap. By giving the user enough power over the syntax (rope) to design the language that suits them (hang themselves).

I didn’t enjoy the State of the Onion as much as I have in the past. I suppose that’s to be expected. Larry did warn us at the top of the talk that it would be serious and contain only a single joke. For as great a writer as Larry is, his ability as a public speaker is lacking. That’s okay, though. I’d rather he not shift focus away from the design and development of Perl.

[tags]oscon, oscon08, oscon2008, Perl, State of the Onion, Larry Wall[/tags]

Sorry guys, these are five minute talks. If I start summarizing, I’ll fall way behind. You’re lucky I even take the time to write this.

If you really want to know what’s going on, there’s a schedule.

For those of you still reading, here’s a bit of stream-of-consciousness for you. Note, if trying to match these up to the schedule, they are in order, but I didn’t comment on all of them.

Testing databases with TAP is cool. You really can test anything with it.

Nice to see The Perl Foundation get some slots in Google’s Summer of Code this year.

It’s interesting to see how much Perl is used to compile USA Today every day. Without Perl, it would be a very empty paper. Though I’m not convinced the content would be much different.

Schwern tells us that, in thirty years, time will wrap.

$time = 2**31 - 1;
print scalar gmtime $time;

Tue Jan 19 03:14:07 2038

$time = 2**31;
print scalar gmtime $time;

Fri Dec 13 20:45:52 1901

Wait, that’s not good. But he’s fixed it.

Sweet, Perl on Google App Engine!

Use autodie instead of Fatal. It’s better.

Also, Paul Fenwick is one of the best speakers I’ve seen in ages. I hope he becomes an OSCON staple.

F*ck, the F*cking thing is F*cked had the best slides.

IPv6Experiment.com (warning: there may be porn).

[tags]oscon, oscon08, oscon2008, Perl, lightning talks[/tags]

The Devel::DProf module is old and a waste of time and is broken. Stop using it. Take it out and shoot it.

The first obvious distinction between profilers is CPU time versus real time. CPU time tends to be highly granular, but doesn’t include I/O, context switching, or other kinds of blocking. That’s where real time comes in. It’s far more useful in the real world.

Tim, as with many of us, is interested in line-based profiling. It provides a high level of granularity The total subroutine time is not always useful, particularly in larger subroutines.

The NYTProf module is exremely fast, discounting the time taken by profiling overhead, making it quite a bit more useful for real world analysis. It also allows profile times per block, and can be aggregated up to the subroutine level. It’s a module with dual profilers: line-based and subroutine-based.

It gets better, every location that calls the subroutine keeps separate track of the subroutine time. This allows us to determine where the majority of the subroutine calls are coming from. For control flow statements, the decision expression is not taken into account when profiling the block that is executed. This is useful if the loop control itself takes time that should be discounted.

And that’s it for the description. Now we have half an hour to play with it.

The HTML-based reporting is inspired by Devel::Cover‘s reporting. Reported for each file are the number of statements executed, the time spent in the source file and the line, block, and subroutine reports. The subroutine reports include the amount of time spent within the subroutine and the amount of time spent in other called subroutines. The coloring of each line of the report—red, orange, yellow, and green—give a relative measure of deviation from the norm. Very impressive.

Even more impressive, Devel::NYTProf is capable of reporting exactly what a subroutine reference is called, even when it’s an anonymous subroutine compiled within an eval. With a handy link also provided, the called code can be easily inspected.

In summary, Devel::NYTProf is awesome. Use it. I know I will.

Tim Bunce is even more impressive than most people think he is. He is the only presenter I’ve seen so far who has managed to use IRC while giving his talk. Well, he didn’t actually type on IRC, but he had Colloquy running in the background. This particular IRC client uses Apple’s Growl feature to display notifications when you are mentioned in a channel. After he’s opened up the session to questions, one of those notifications pops up on the projected display:

<sirhc> Adam Kennedy (to Tim Bunce): Why are you so awesome?

It got a laugh, and Tim seemed to take it all in stride, even joking that he was not looking very professional on his screen cast. Important safety tip for session presenters, don’t leave your IRC client open.

[tags]oscon, oscon08, oscon2008, Perl, programming, profiling[/tags]

Pudge is, quite famously, a Republican, so he wants poor people to die, he asserts his right to shoot people who jaywalk, and he hates puppies. Now that we have that out of the way, this will not be a political talk. Instead, it will be a talk that just happens to use politics as the problem domain for which Perl was the solution (but isn’t it always?). Pudge happens to volunteer for the Republican party in Snohomish county, Washington. I actually know the area fairly well, as my grandmother happens to live there.

Winning elections is all about knowledge. And blackmail. But, mostly knowledge.

This session is essentially about data mining. There are a number of disparate data sources available with information about voters. From registration and voting history to contact information and preferences—can or can they not be contacted. This data is not always easy to access. For example, there is something called the Voter Vault, which is a super secret database of voter information controlled by the Republican party (there’s an NDA involved, so we won’t see any of it).

Essentially, Voter Vault is a really crummy Web application that only works for IE (hence the crummy part). That’s where WWW::Mechanize comes in. Using this brilliant module, data on any Web site can be retrieved, even if it requires a certain amount of user interaction to access. This, along with other sites, like the Washington State Public Disclosure Commission, provide all the raw data Pudge needs.

However, raw data is, by itself, not useful to anyone. This is the reason behind Pudge’s efforts. He uses Perl (and some JavaScript) to collect and aggregate all of this data. Then, once it’s all compiled, he can use a bit of Perl glue to use the data in Apple’s Address Book and Mail applications. But, more importantly, he can visualize it.

For the visualization, Pudge uses everyone’s favorite new tool, Google Maps. Using the Ajax API provided by Google, he can embed a map in his own Web application and, next to it, provide controls to enable and disable different views of the data on the map. For example, candidate donations by city and how much each candidate received.

It gets better. With the Google Earth APIs available to Google Maps, KML files can be generated (again, with Perl) to provide even better data visualizations. For example, precinct boundaries can be imported and colored based on voting history.

Initially, I wasn’t sure how I’d feel about this talk, but I ended up enjoying it. It was an excellent presentation on how to take data and display it to users in a useful manner.

[tags]oscon, oscon08, oscon2008, Perl, politics, visualization[/tags]

A computer once referred to a human worker who would perform calculations. This was a fairly easy thing to cluster and “run” several computers in parallel. As time progressed, more and faster work was desired. Enter the electronic computer, and specifically for this talk, the Cray. As with anything, the inner workings of the Crays of old can be recreated in Perl. Just use the Cray module, no problem (if only it existed).

After the history lesson, we move into high level overviews of parallelism and pipelineing, and a note about Amdahl’s Law. This was followed up with an example for detecting prime numbers by partitioning the work.

The slide presentation was over in under 20 minutes. Instead, we’re jumping straight into code examples. Awesome.

Or so I thought. Unfortunately, he’s been interrupted by multiple people in the audience, who keep wanting to move off into tangential conversations. Eric is having difficulty bringing the talk under his own control—it’s no longer his talk, but that of the somewhat rude fellow in the front row. Neither is Eric as eloquent when he switches from a prepared talk to demonstrating and explaining real code. It’s become far more difficult to pay attention to this session, and I find myself looking at the clock to see how much time we have until the next session.

For real fun, be sure to check out Brad’s post on Schwern’s session about skimmable code.

[tags]oscon, oscon08, oscon2008, Perl, programming[/tags]

my $code = qq{
    print 1+1 . "\n";
    $code =~ m/(\d+)\+(\d+)/;
    $new = $1 + $2;
    $code =~ s/\d+\+(\d+)/$2+$new/;
};

for ( 1 .. 10 ) {
    eval($code);
}

What’s the first bug? Yes, it should use q{}, or the variables will interpolate on the initial assignment to $code. To their credit, they initially used single quotes, but people said it was too hard to read.

I wasn’t content with just figuring out what the code did and fixing a small bug. I think it can be written better.

eval($code = q{
    print 1+1 . "\n";
    $code =~ s/(\d+)(\+)(\d+)/"$3$2" . ($1 + $3)/e;
    eval $code;
});

Much better. Not only is it more concise, I was able to remove that pesky loop, so I wouldn’t be bothered by any silly upper bounds.

So what does it do? Should be obvious. Head over to the Amazon booth and let them know.

Recently, Adam has been funded by The Perl Foundation, Perl in Israel, and Stonehenge to use Perl from nothing but his flash drive. This provides an excellent motivation to get Strawberry Perl working in a highly portable way.

Originally, Perl was awesome and worked everywhere—except Windows. That was okay, because Windows didn’t matter. No one did any real work on Windows. Then, around 1995, Windows started to matter. A brief history of Perl on Windows followed, resulting in what is today ActiveState.

Much of what Adam wrote for PPI does not work in ActivePerl, which makes it a non-starter for him, as he tends to work on Windows. Anything depending on Scalar::Util or List::MoreUtils modules will not work with the ActivePerl build system. This led to an embarrassing problem for Adam when he gave a talk three years ago at OSCON. He couldn’t give his demo, because PPI would not build in ActivePerl. In fact, ActiveState’s package manager has gotten so much worse that almost any module that is at all useful does not exist—and thus nothing useful can be done on Windows (big surprise).

Moving away from ActiveState, this talk is essentially about Adam trying to get his own laptop to work. That’s really all he wants. It’s a modest desire. More importantly, the CPAN module has to work. Without that, what’s the use of Perl?

So Adam offered a prize: a yard-high stack of cases of any beer desired by the first person who could provide a fully-installable and working (by the above definition of working) version of Perl for Windows. After six months and no sign of a winner, he changed the prize to “craploads” of beer. In 24 hours, he received two entries. The winner cheated a lot, but the loser was Vanilla Perl, which has become a testing ground for experimentation.

Strawberry Perl is the Perl for Windows designed for people who don’t use Windows. That is, the people who do all of their work on Unix or Unix-like systems—Linux, Solaris, and Mac OS X. The main goal of the project is to make it easy—it is Perl, after all.

In the future will come Chocolate Perl—completing the holy trinity of neopolitan flavors—for people who know Windows, but don’t know Perl, and thus the Unix-like characteristics of Perl.

The target of Adam’s financial support is Portable Perl: Perl for flash drives. Carry it around, install CPAN modules onto, or from, the flash drive. It’s network-aware, does the right thing, and juliennes fries. An excellent standard being developed for portable apps is, in fact, PortableApps.com, where applications such as Firefox or Putty can be downloaded and installed to those ever-growing flash drives.

Available Thursday at the Perl Foundation‘s booth in the expo hall will be branded flash drives with Portable Perl on them. At least, I think I heard that correctly.

I really like the work Adam is doing. He’s accomplished so much to get Perl everywhere. That’s a cause I can get behind.

“The main problem today is Vista.”
— Adam Kennedy

Okay, I took that out of context, but I couldn’t resist capturing the quote. What he really means is that changes made to Windows in Vista have made things not work, in particular the access control. It’s not an unusual problem when upgrading to new systems, but it is more difficult with proprietary platforms, which Open Source authors have very little access to.

Most of us are, of course, aware of the concept of best practice when coding. Writing code that’s maintainable, predictable, and follows the rules. Oh, and uses Java.

Worst practice is, by contrast, code that is obfuscated, unmaintainable, and breaks all of the rules. Today, we will be studying code that Damian has submitted to the Obfuscated Perl contest. This promises to be very, very scary.

Damian’s entry to this contest was SelfGOL, a program capable of self-replication, rewriting other Perl programs to themselves self-replicate, detecting un-rewritable programs, playing Conway’s “Game of Life,” and, as if that wasn’t enough, animating any text as a cycling marquee banner. The main constraint of the contest is that the entry must be under 1,000 bytes of code, so it shouldn’t be too difficult to understand. Obviously it doesn’t use any modules, because that would be too easy. Not only that, but it doesn’t use a single control structure. This is going to be great.

Following an amusing demonstration of SelfGOL, we moved into treating it as a case study for a set of principles. Principles that will focus on the very practices SelfGOL embodies, and why they should never, ever be used. As I intend to enjoy the discussion, I won’t spend much time writing about the discussion and examples accompanying these principles, but rather simply note the principles for my own benefit (documentation for the win). After all, sharing all my new tips and tricks would suck all the fun out of it.

Principle 1: Sane and consistent layout makes code more maintainable (but it isn’t a magic bullet if the code itself is beyond help).

Principle 2: Using built-in features isn’t necessarily smarter or cleaner (even though fellow developers’ futile struggles to recall those features can be highly amusing).

Principle 3: Obscure obsolete features are obscure and obsolete for a reason (and restasking them for even more obscure purposes is not helping).

Principle 4: Each statement should do one thing only (since that’s the upper limit most brains can comprehend).

Principle 5: Relying on default behavior makes code very slightly easier to write and vastly harder to read (because most readers can see better than they can think).

Principle 6: Randomly placed subroutine definitionss are static (in the radio interference sense).

Principle 7: Choose data structures that simplify your task (even if the task is to make those data structures incomprehensible).

Principle 8: Just because you use some operation frequently doesn’t mean it should be in a utility function (especially if it’s in a function merely to abbreviate its name).

Principle 9: Encapsulating the familiar can decrease maintainability (refactoring isn’t a substitute for sanity).

Principle 10: Treat any clever one-line solution as an alarm bell (or as an antipersonnel mine with a six-month delay fuse).

Principle 11: Familiarity breeds comprehension (it breeds contempt (but hey, what’ doesn’t?)).

Principle 12: Table-driven solutions are clean, efficient, and extensible (as long as you don’t mind losing a little comprehensibility).

Principle 13: Building a messy data structure and then cleaning it up is often easier than building it cleanly in the first place (and to hell with the purists).

Principle 14: Some code is better compiled at run-time (but the urge to use eval is Nature’s way of letting you know there’s not yet enough pain or misey in your life).

Principle 15: Parentheses are our friends (cos, if you can remember all 24 levels of Perl’s precedence, you gotta get a life, dude!).

Principle 16: Edge cases suck (and edge cases of familiar constructs suck worst of all).

Principle 17: Code should do what it seems to be doing (especially when it seems to be doing something subtle).

Principle 18: Conceptual elegance is no guarantee of actual maintainability (nor a good substitute for it).

Principle 19: If you’re going to have default values, define them near the place they may actually be used (or, at least, somewhere they have a slim chance of being discovered).

Principle 20: No matter how good you think your error messages are, they’re still too brief, too obscure, and too hard to decipher (even if you’ve already taken Principle 20 into account).

Principle 21: Avoid using obsolete and arcane magic punctuation variables with unfamiliar default values and unexpected global effects (even if you happen to enjoy a little self-inflicted pain in other recreational activities).

Principle 22: The fundamental complexity of any problem is irreducible (optimizations merely redistribute the pain differently).

Principle 23: Code that breaks when it’s reformatted is already broken (though on a much more profound and interesting level).

Principle 24: If it’s impossible to understand, it’ll be impossible to maintain (on the bright side, of course, such code is highly stable).

This last one should, but often doesn’t, go without saying.

Principle 25: Phenomimetic retrodeterministic nominativism generally does not improve code comprehension (then again, did it sound like it would?).

Principle 26: Don’t allow dynamic behavior to violate static expectations (and the easiest way to do that is reusing over-scoped variables for unrelated purposes).

Principle 27: Explicit behaviors are better than implicit behaviors (especially when the specification of the implicit behavior is syntactically baroque and hard-to-spot, and the behavior itself is unknown to the majority of developers).

At this late point of the tutorial, Brad pointed out to me that all of these principles are in the included materials. Now that I’ve already transcribed so much from the slides, I don’t have the heart to delete it all. Of course, since I haven’t been commenting on all of the black magic to this point, there would then be very little in the end to post. Brad also has a much better post about this tutorial, since he actually took real notes.

Principle 28: Code that pre-caches or precomputes its data is much easier to maintain than code that caches or computes on-the-fly (when you’re running at multiple gigahertz, acquiring your data a few thousand operations early is still plenty JIT enough).

Principle 29: Coding is an art, but code shouldn’t be art (evolution made programmers boring, pedestrian, and aesthetically challenged for good reasons).

It’s mesmerizing to listen to the thought process behind Damian’s obfuscated code. I can’t help but wonder if this well-organized, well-thought-out explanation is anything close to how Damian designed this program. Or, rather, if there are extremely convoluted, scary, and most importantly, evil gears grinding away inside his head. In fact, I suspect this entire tutorial may have been designed purely as a way of documenting SelfGOL so Damian himself can remember how it works. Clever.

This kind of programming is silly and fun, but it serves a real purpose. Pushing the limits of a language teaches about its dark places. The understanding that comes from it vastly improves the skills of the programmer, even if—especially if—the bad things are never, ever used. Perl, even more than other languages, encourages this kind of play, thanks to its rich diversity and culture.

Important safety tip: keep these tricks and contrivances for recreational purposes only.

I don’t know what’s more disturbing, how much of the tutorial I understood, or how much I already knew coming in.

[tags]oscon, oscon08, Perl, Damian Conway[/tags]

“A computer is secure if you can depend on it and its software to behave as you expect.”
—Simson Garfinkel and Gene Spafford in Practical UNIX & Internet Security

In a nutshell, that’s what security is. If a computer behaves as expected, it is secure. That is, unless it’s expected to be insecure, I suppose. I’m not sure I’d enjoy that situation, so I’ll assume the assumption of expected behavior is both expected and secure.

Most security boils down to common sense. Unfortunately, this mythical state of being is far less common than its name would imply. Sad, but true. People are often lazy or distracted, and these usually lead to really stupid mistakes.

There is a key acronym when thinking about security: CIA. No, not that CIA. Yes, I thought so, too, at first. What it really means is, Confidentiality, Integrity, and Accessibility. Confidentiality, because information will not remain secure if it does not remain confidential. Integrity, because information must remain known and trusted to remain secure. Accessibility, because denial of access to information may result in insecurity. I may not have done justice to this acronym, because the tutorial moved on quickly after this point. I’m sure there are web sites dedicated to security that can better define the term.

Perhaps the most important piece of advice for the unwitting Perl programmer is to always perform data validation. Never, ever trust input, regardless of where it came from. Fortunately, Perl provides Taint Mode, which forces the program to mistrust input.

Paul shared with us a variety of examples to demonstrate why input should not be trusted, as well as a number of examples of how to properly untaint data. As with anything, it’s easy to become lazy when untainting data, which can sometimes be as bad as not using Taint Mode at all.

The tutorial continued into what is essentially a list of best practices to follow when programming securely with files.

• Do: Use the three argument version of open(), to prevent attacks using file names with magic characters in them.
• Do: Use sysopen() instead of open(), which provides ways to avoid overwriting a file, thus helping to prevent symlink attacks often as a result of race conditions.

The common attack vector in so many of the examples given so far has been via file names. Wouldn’t it be great if we could write programs without file names at all? Well, when working in a Unix-like environment, we can. Perl has the ability to use anonymous files by passing undef as the third argument to open(). He was even kind enough to provide us with a way of passing these anonymous file handles to child processes, by disabling the close-on-exec flag prior to calling system(). Sorry, the slide went by too quickly for me to transcribe the method. It, along with all the other examples, are available online.

Calling system() and using backticks make Paul really, really angry. Why? Because doing it right is hard. In fact, just correctly checking the result in $? requires 10 lines of code, according to the documentation for system() in the perlfunc manual page. So, 10 lines just to verify that a single line of code executed successfully.

I briefly became distracted by news of a fire back home. However, what I was able to get is that Paul has written a module, IPC::System::Simple, which, as the name implies, makes the process of calling system commands quite simple.

After the mid-afternoon break, we ventured into setuid and setgid programs. Perl provides ways to determine who is really running the program ($<, $() and who is effectively running the program ($>, $)). Perl is, however, ignorant of the saved UID, which is the third UID in Unix, along with real and effective. Unfortunately, the standard for setuid scripts is confusing and implemented differently on various systems, so don’t use it. Really.

Even worse, the $< and $> variables are cached by Perl, so they may lie to the program, especially when using the setresuid() system call to properly drop privileges, as recommended. Fortunately, another useful module from Paul, Proc::UID provides a solution to this caching problem.

Now we move into DBI security. SQL injection attacks are very similar to the file name or shell attacks covered previously. Any database programmer worth his salt should be aware of the hazards of composing SQL, so I won’t go into the examples here. Programmers should, of course, use placeholders if they’re available. The DBI module itself provides its own Taint Mode, both for input and output, adding all the benefits of Perl Taint Mode to database interface code. Even better, it can be controlled on a per-statement basis.

All of this careful taint checking we’ve done and Perl may end up sabotaging us anyway. When presented with files on the command line, Perl is happy to just open them using the simplistic, dangerous, single argument open() call. Typically, this is done when using the <> operator in a while loop. Also, everyone forgets to use Taint Mode in cron jobs. Don’t do that. Really.

Because Perl is written in C, the null byte becomes very interesting. While it is a perfectly valid character in Perl strings, it marks the end of a C string. In most circumstances, this is not a problem. However, it can mean bad things when making systems calls, which are written in C. Normally, at a terminal, null bytes don’t occur in user input, unless that input comes from the Web. Null bytes can be trivially represented by the %00 escape sequence.

I need to go through the list of Paul’s modules, since they appear to be ideal for the type of programming I tend to do, as an IT developer. In fact, he’d like to see some Solaris patches for Proc::UID, so I can probably help him with that.

I noticed during the tutorial that Paul must read the Fail Blog and I Can Has Cheezburger, or at least knows someone who does. Quite a few of the images that have appeared on his slides have graced the pages of those web sites.

As an added bonus, the tutorial ended 40 minutes early, and Paul had bonus material. What a guy.

The tutorial, and with it the day, is now over. It’s time for dinner, then maybe a BOF session or maybe just a trip to a pub.

[tags]oscon, oscon08, perl, security[/tags]

The idea behind Mastering Perl is not to talk about Perl to a group of Perl masters. Instead, it’s about mastering Perl in the guild sense (and not of the SCA variety). Back in the day, and still existing in some professions today, there was an apprentice system. A neophyte—in today’s nomenclature, a noob—would begin acquiring skills under a master of the art. As he progressed, he would be entrusted with more and more responsibility, until finally he became a master himself and took people under his own wings.

This apprenticeship system, somewhat unfortunately, does not exist in the computing world. That’s where brian d foy feels that Mastering Perl fits. Lacking true masters, the book acts as a substitute. Someday, we may even create a guild system. But then we’d probably have to pay dues and follow rules, and that’s not very attractive. That said, it’s the model I’m hoping to use at my own place of work. I’d like to hire one or two developers who I can take under my own wing and mentor them in the ways of Perl and the grid.

The first two topics covered are tools for optimization, profiling and benchmarking. Often mis-attributed to Donald Knuth, Tony Hoare once said, “Premature optimization is the root of all evil.” What this means is that one should never assume what requires optimization. Let the testing be the guide.

While profiling is objective, benchmarks, like statistics, are not always objective. Everyone has an agenda and benchmarks are subjective. Often, benchmarks are short-sighted. For example, benchmarking code run time and attempting to optimize for it may not be worth the expense of the developer time required to make the requisite changes. It’s worth analysing what is important before blindly following benchmarks.

I’ve been on the receiving end of misplaced premature optimization. I worked with a development group that put far too much emphasis on achieving perfect results on their Devel::Cover reports. This led to strange bugs in their code, and a strong belief that “new() doesn’t work that way.” As it turns out, their test suite was calling new() in two ways. I forget what the second method was, but it was not used anywhere else in their code. However, in order to get this test code to run, and get 100% coverage, they added code to the constructor for every class. Code that prevented inheritance of the method. The team then convinced themselves that constructors could not be inherited in Perl, rather than realizing that their own habits were the problem.

After the mid-morning break, we wrapped up the discussion on profiling and benchmarking, and moved into configuration. This is a vital topic for anyone who desires the ability to pass a program off to users without being bothered to modify it later in response to users’ desire to customize the program for a slightly different use.

External configuration, particularly via the command line, is something I depend on heavily, even in very simple Perl or Bourne shell scripts. I almost always create command line options for performing a dry run or output debugging information. Not only are these useful for development, they can live on in the final program, providing help to the final user, who more often than not is me. Sometimes I will even add configuration to values that never change, just for when they eventually do.

Jumping past configuration, we move on to logging. It’s really easy to add to a program, and it’s really useful to leave in a program when it’s released. The ability to enable logging on the fly sure beats adding a bunch of print() calls in the code when it inevitably breaks at three in the morning. The Log::Log4perl module is a particularly powerful method of adding logging to programs. It’s well worth investigating for anyone who wants to easily add logging functionality to their code.

The final topic of the day is lightweight persistence. It’s always nice to have data stick around between program invocations. The easy way (and everything in the second half of the tutorial is easy) to add persistence to code is to not use DBI. While DBI is powerful, it also tends to require a database server (ignoring SQLite for the moment). Modules such as Data::Dumper, YAML or Storable are ideal for easily storing and retrieving data in code.

After the tutorial, brian will be available at the Powell’s Books mini store, located near the registration desk, to sign copies of Mastering Perl. I already have a copy, thanks to my local Perl Mongers group, but it’s all marked up with the group name, and I wouldn’t mind having a signed copy.

Now it’s time for lunch, which is good, because I’m quite hungry. I hope the conference-provided lunch is decent during the tutorials, as it was last year.

[tags]oscon, oscon08, perl[/tags]

A couple of things resulted from this change. First and foremost, we have a lot of slack to take up, so the rest of the year will be very busy for us. Second, I am now the de facto lead developer in the group. A group for which we need to hire two more developers (we had an open position before the loss of our comrade).

Two fresh, new, dreamy eyed developers. For me to lead, to teach, to mold. I like to think of these potential developers as my minions, willing to do my bidding.

For a while, we filled our open developer position with a temporary employee. We tasked this person with the creation of a process work flow for our development efforts. Something we could use to identify tasks, categorize them, prioritize them, assign them, and sometimes even work on them. The final result of this effort looks something like this:

No, no, no. This will never do. I can’t use this. Look at how many boxes there are. Not only that, look the sheer complexity introduced by all those decision branches! I could never trust my minions with so much independent thought. Also, I have no desire to confuse my minions any more than they already are. So I designed a new process flow, which I believe is far simpler and easier to remember.

Yes, this is more like it. I suspect even the simplest of minions can effectively follow this process. And if they can’t, well, we have ways of dealing with them.

So I need minions. There are a few requirements, however.

• Familiarity with Perl (other programming languages are acceptable—except Python)
• Experience administering Linux (or another Unix-like system, I guess)
• Fascination for grid computing
• Misplaced enthusiasm for supporting users
• Blind devotion to me

Not necessarily in that order.

My task this week was to compensate for one of these bugs. Basically, the request was to somehow lie to License Scheduler’s data collection process, convincing it that the license counts are different than the reality. The collection process uses Macrovision‘s lmstat(1) command to gather license counts. Okay, no problem. Twenty lines of Perl later, and I have my own lmstat command, which behaves identically to the real version (which I simply execute) except the license counts have been altered.

In my group, we’re supposed to be working primarily on projects. All of these projects are assigned awkward, forgettable acronyms. So I decided that this project needed an acronym, too. Not just any old acronym, either, but something memorable. After a bit of searching through /usr/share/dict/words, I finally settled on Project FALSE: Falsifying Answers in the License Scheduler Environment.

So with my quick hack, I’ve both defeated an expensive piece of software and won the prize for the best project name so far.