Tag Archives: oscon08

OSCON 2008: Ultimate Perl Code Profiling

Posted on by
Reply

Lunch is over and I’m here to listen to Tim Bunce talk about Ultimate Perl Code Profiling with Devel::NYTProf.

The Devel::DProf module is old and a waste of time and is broken. Stop using it. Take it out and shoot it.

The first obvious distinction between profilers is CPU time versus real time. CPU time tends to be highly granular, but doesn’t include I/O, context switching, or other kinds of blocking. That’s where real time comes in. It’s far more useful in the real world.

Tim, as with many of us, is interested in line-based profiling. It provides a high level of granularity The total subroutine time is not always useful, particularly in larger subroutines.

The NYTProf module is exremely fast, discounting the time taken by profiling overhead, making it quite a bit more useful for real world analysis. It also allows profile times per block, and can be aggregated up to the subroutine level. It’s a module with dual profilers: line-based and subroutine-based.

It gets better, every location that calls the subroutine keeps separate track of the subroutine time. This allows us to determine where the majority of the subroutine calls are coming from. For control flow statements, the decision expression is not taken into account when profiling the block that is executed. This is useful if the loop control itself takes time that should be discounted.

And that’s it for the description. Now we have half an hour to play with it.

The HTML-based reporting is inspired by Devel::Cover‘s reporting. Reported for each file are the number of statements executed, the time spent in the source file and the line, block, and subroutine reports. The subroutine reports include the amount of time spent within the subroutine and the amount of time spent in other called subroutines. The coloring of each line of the report—red, orange, yellow, and green—give a relative measure of deviation from the norm. Very impressive.

Even more impressive, Devel::NYTProf is capable of reporting exactly what a subroutine reference is called, even when it’s an anonymous subroutine compiled within an eval. With a handy link also provided, the called code can be easily inspected.

In summary, Devel::NYTProf is awesome. Use it. I know I will.

Tim Bunce is even more impressive than most people think he is. He is the only presenter I’ve seen so far who has managed to use IRC while giving his talk. Well, he didn’t actually type on IRC, but he had Colloquy running in the background. This particular IRC client uses Apple’s Growl feature to display notifications when you are mentioned in a channel. After he’s opened up the session to questions, one of those notifications pops up on the projected display:

<sirhc> Adam Kennedy (to Tim Bunce): Why are you so awesome?

It got a laugh, and Tim seemed to take it all in stride, even joking that he was not looking very professional on his screen cast. Important safety tip for session presenters, don’t leave your IRC client open.

[tags]oscon, oscon08, oscon2008, Perl, programming, profiling[/tags]

OSCON 2008: Perl for Political Campaigns

Posted on by
1

There was nothing interesting for me scheduled for the second session today, so I ended up in Perl for Political Campaigns, presented by Chris “Pudge” Nandor. I’m not entirely sure why I’m here, but it likely has something to do with Perl in the title and Pudge as the presenter. I must be in the right place, though. Both Damian Conway and Adam Kennedy are present.

Pudge is, quite famously, a Republican, so he wants poor people to die, he asserts his right to shoot people who jaywalk, and he hates puppies. Now that we have that out of the way, this will not be a political talk. Instead, it will be a talk that just happens to use politics as the problem domain for which Perl was the solution (but isn’t it always?). Pudge happens to volunteer for the Republican party in Snohomish county, Washington. I actually know the area fairly well, as my grandmother happens to live there.

Winning elections is all about knowledge. And blackmail. But, mostly knowledge.

This session is essentially about data mining. There are a number of disparate data sources available with information about voters. From registration and voting history to contact information and preferences—can or can they not be contacted. This data is not always easy to access. For example, there is something called the Voter Vault, which is a super secret database of voter information controlled by the Republican party (there’s an NDA involved, so we won’t see any of it).

Essentially, Voter Vault is a really crummy Web application that only works for IE (hence the crummy part). That’s where WWW::Mechanize comes in. Using this brilliant module, data on any Web site can be retrieved, even if it requires a certain amount of user interaction to access. This, along with other sites, like the Washington State Public Disclosure Commission, provide all the raw data Pudge needs.

However, raw data is, by itself, not useful to anyone. This is the reason behind Pudge’s efforts. He uses Perl (and some JavaScript) to collect and aggregate all of this data. Then, once it’s all compiled, he can use a bit of Perl glue to use the data in Apple’s Address Book and Mail applications. But, more importantly, he can visualize it.

For the visualization, Pudge uses everyone’s favorite new tool, Google Maps. Using the Ajax API provided by Google, he can embed a map in his own Web application and, next to it, provide controls to enable and disable different views of the data on the map. For example, candidate donations by city and how much each candidate received.

It gets better. With the Google Earth APIs available to Google Maps, KML files can be generated (again, with Perl) to provide even better data visualizations. For example, precinct boundaries can be imported and colored based on voting history.

Initially, I wasn’t sure how I’d feel about this talk, but I ended up enjoying it. It was an excellent presentation on how to take data and display it to users in a useful manner.

[tags]oscon, oscon08, oscon2008, Perl, politics, visualization[/tags]

OSCON 2008: Stick a fork() in It

Posted on by
Reply

First session of the day and I’m in room F150 (brought to you by Ford). The F wing, bereft of wifi. I’m here for Stick a fork() in It: Parallel and Distributed Perl with Eric Wilhelm of Scratch Computing. It’s great to see how popular Perl still is. It’s standing room only in here.

A computer once referred to a human worker who would perform calculations. This was a fairly easy thing to cluster and “run” several computers in parallel. As time progressed, more and faster work was desired. Enter the electronic computer, and specifically for this talk, the Cray. As with anything, the inner workings of the Crays of old can be recreated in Perl. Just use the Cray module, no problem (if only it existed).

After the history lesson, we move into high level overviews of parallelism and pipelineing, and a note about Amdahl’s Law. This was followed up with an example for detecting prime numbers by partitioning the work.

The slide presentation was over in under 20 minutes. Instead, we’re jumping straight into code examples. Awesome.

Or so I thought. Unfortunately, he’s been interrupted by multiple people in the audience, who keep wanting to move off into tangential conversations. Eric is having difficulty bringing the talk under his own control—it’s no longer his talk, but that of the somewhat rude fellow in the front row. Neither is Eric as eloquent when he switches from a prepared talk to demonstrating and explaining real code. It’s become far more difficult to pay attention to this session, and I find myself looking at the clock to see how much time we have until the next session.

For real fun, be sure to check out Brad’s post on Schwern’s session about skimmable code.

[tags]oscon, oscon08, oscon2008, Perl, programming[/tags]

OSCON 2008: Thursday Morning Keynotes

Posted on by
3

Thursday morning, the conference is more than half way over. It’s once again time for some keynotes. They opened with an open content video from REM. I don’t know why. It wasn’t very good.

Our first speaker this morning is Keith Bergelt of the Open Invention Network, speaking about Open Invention Network and Its Role in Open Source and Linux. He’s speaking about patents and intellectual property in Open Source, the realities of it today and where he sees it going tomorrow. He’s big on the buzzwords, and this is not the right audience for it. In fact, a game of Buzzword Bingo has already broken out in the IRC channel.

In summary, “Blah blah patent blah blah buzzword blah blah we care blah blah.”

Oh wait, he droned his way to a point. One of the things the Open Invention Network does, and I should have known because I’ve seen this before, is to buy up patents and keep Open Source safe from them. At least, until their funding dries up and they turn to their patent portfolios to squeeze money out of everyone.

I seem cynical this morning. Maybe I didn’t get enough sleep. Or maybe the first keynote today is boring. The back-channel conversation on IRC is actually quite entertaining, though. I need to whip up a quick IRC log file analyzer to correlate IRC traffic to keynote speaker. Then I can use it as a tool to rate speakers.

The pain is finally over, and the program chair has caught buzzworditis from the last speaker. Next up is Peter H. Salus to speak to us about Anniversaries. I’m told by Nat Torkington that Peter is an Unix historian. He’s started off by showing us a picture of the first transistor, which is about 20cm and a bit more than that around. It’s amazing to see how far we’ve come in 60 years—how many iPhones can fit in the same volume?

Anniversaries, in this case, are major milestones in computer history. The first electronic computer; the first time-sharing system; the first Unix paper by Ritchie and Thompson; the GNU project. One of the interesting things to learn is that history repeats itself. Back in the days of ARPANET, there was an issue involving the exhaustion of address space on the network. Short-sighted problems like that would never happen today, right?

I enjoyed this keynote speech, but probably because I really enjoy history.

Next up, Supporting the Open Web with David Recordon of Six Apart. It’s not just the open nature of the software or the platform that matters, but the openness of the data. Without open data, the Open Web can’t work. Interoperability and open specifications are vital to moving forward with the technology. The Web must be accessible, not just available on one device or another.

The majority of the talk is dedicated to talking about the various organizations doing work to keep everything free and open, including the Open Source Initiative, Creative Commons, and the Apache Foundation. There are also quite a few people donating a lot of their time to help.

He’s announcing the formation of the Open Web Foundation. They don’t necessarily want to form their own foundation, but they have had little luck finding an existing one to do what they’ve asked.

The Open Web Foundation will focus on four areas: incubation, licensing, copyright, and community. Many companies, such as Google and Yahoo have already shown support for this new foundation.

Following David is Danese Cooper of the Open Source Initiative and Intel Corporation to speak about Why Whinging Doesn’t Work. A catchy title, and she introduced her talk with a funny video of a choir of Finnish women singing about all of the complaints they have (search YouTube for “complaints choir“).

She’s making a very good point. There are so few women in Open Source. Geek are often intimidated by women and women are so often objectified. It’s true, there is a huge gender imbalance in the geek community. Of all the geeks I know, I can name very few women. I’m having a daughter soon, and you know what, she’s going to learn to code.

However, the feminist angle is merely a way of personally relating to the main point of her talk. People complain. I do it, you do it, the guy sitting next to you does it. But whinging doesn’t help. Mostly, all whinging does is beget more whinging. That energy used to complain needs to be channeled into something constructive.

For seven years, Danese was the only female member of the Open Source Initiative’s board. Now 30% of the board members are female. Progress.

Finally, Nathan Torkington, former OSCON program chair and recently of He Hononga Software, Limited and his keynote, fork() && exec(): Spawning the Next Generation of Hackers. Thank goodness, this talk is not about geeks having sex.

I’ve been looking forward to this keynote for a couple of reasons. First, I’ve missed hearing Nat speak this year. Second, I’m expecting my first child in a couple of months. Not only that, two other members of my local Linux User Group are either recent or expecting fathers. Suddenly, topics involving children are much more interesting to me.

Nat recently moved his family back to New Zealand. One of the things he does now is to help teach children about computing. In his school district, the computing infrastructure was awful—and used Windows. So he got a handful of Macs and became the Bastard Operator from Hell for his kids’ school. Then he started teaching the schoolchildren. Quickly, he discovered that the teachers needed teaching as well.

One more thing he wanted to do was to teach programming. He feels it’s a very important skill. But it has to be done right. Avoid the frustration that so many of us experience with computing and programming, but something consistent, easy-to-learn, but still powerful. Nat’s introduced Scratch. The kids loved it.

Lessons learned:

  • Lectures suck (you have two minutes to say what you want)
  • The gender gap is not what you think (girls are smarter and more focused than boys)
  • Keyboards are a challenge
  • Not a lot of experience with math
  • Robots are lame

So please, volunteer in schools. Perhaps remove Windows and bring the joy of Linux to their lives. Find, or create, good courseware, such as Scratch. Post it on your blog, so everyone can find it. Finally, don’t profit. Do this for the good of the children, our future generation of geeks.

With that, we’re off to the expo hall for the break.

[tags]oscon, oscon08, oscon2008[/tags]

OSCON 2008, Day 4

Posted on by
1

Thursday morning and day four of OSCON is sunnier than the last two have been. Though it’s still chilly outside, it’s comfortable inside the convention center, so far. I’m once again having breakfast in the expo hall after getting too little sleep.

Sadly, yesterday during the morning keynotes, Al was called back home abruptly. Hopefully, he made it back to the UK quickly and safely.

After all the sessions were said and done for the day, we found our way to the expo hall, where beer and appetizers were being served. Alas, we did not stay long. We caught wind that Google would be hosting pizza across the river at Old Town Pizza, an event we never made it to. It turned out to be a pizza dinner for Summer of Code participants. We finally ended up at Rogue for dinner, and I finally got myself a growler for my collection—currently being held (safely?) in Brad’s hotel room refrigerator.

After dinner, we swung by the supposed Amazon party. Only, there wasn’t one. It was only held between 8:00pm and 9:00pm. Seriously? This is how Amazon throws a party?

Fortunately, the Sun party was a better this year. First of all, they had no stupid lolspeak flyers. Second, bottled beer instead of kegs, which is difficult for incompetent bartenders to over-prime and serve nothing but head. Third, sumo wrestling! Brad and I also participated; those photos are coming soon, I promise.

However, as I actually enjoy attending the keynote sessions—scheduled far too early in the morning—I was back in my hotel just after 11:00pm. I ran into Dan and his fellow TierraNet colleagues in the hotel bar. Unfortunately, I had missed last call, but I sat down for a bit anyway. We had some laughs with Margaret, the bartender. I tried to get her to slap Tyler, but sadly it never happened.

Today’s session tracks begin with a dilemma. Unfortunately, I’d like to be in three places, simultaneously.

Fortunately, Brad wants to go to Michael Schwern’s talk, so I’ve agreed to attend Eric Wilhelm’s talk. We’ll write summaries and both be happy. The microblogging session was just a curiosity for me anyway.

The rest of the day won’t require quite as much rolling of dice.

The only potential conflict is during the second half of the Perl lightning talks, A Tasting Tour of Haskell (Bryan O’Sullivan).

Just about time for the morning keynotes, and I’m looking forward to seeing Nat Torkington speak. If I can reconnect to the wifi network, I can even post this entry.

OSCON 2008: An Illustrated History of Failure

Posted on by
1

For my final session of the day, I’m in D139/140 for An Illustrated History of Failure with Paul Fenwick. I attended Paul’s Perl security talk yesterday, which was deciding factor in my attendance here. I figure it will have to be good, I’m sitting a few seats away from Damian Conway.

Paul has started out by describing the world’s oldest computer in terms of modern computing.

From there, he’s providing examples of major computing and engineering failures throughout modern history. It’s amazingly entertaining. I can’t summarize it. If you’re not here, you fail. I’m just going to sit back and enjoy it.

[tags]oscon, oscon08, oscon2008, history, failure[/tags]

OSCON 2008: Moblin.org

Posted on by
Reply

Continuing my afternoon tradition of attending sessions with absurdly long names, I’m in D136 at Moblin.org: The Community for Linux on Mobile Internet Devices (MID), netbooks, nettops and More…. It’s being presented by Dirk Hohndel, who I just overheard agreed at the last minute to substitute for the original author of the presentation. He’s nervous, so I hope it goes well. He is, however, the same person who gave the keynote this morning.

I work for a small telecommunications design company, so this venture into Linux on mobile platforms holds quite a bit of interest for me. Granted, I work in a support capacity for the folks who do real work, but knowledge is always a good thing, right?

Intel has chosen a Fedora- and GNOME-based platform for Moblin. I’ve contributed a couple of packages to Fedora, which means users of these Intel mobile systems can play Zork.

Dirk wasn’t able to have any sample devices with him, so he was left to describe what a “net book” is. Fortunately, in a room full of geeks in a mobile computing presentation, several people had ASUS EEE PCs, which he could show off to the audience. There were also a Nokia N800, N810, and of course several iPhones in the crowd. Obviously I mobile-savvy audience.

Linux is often touted as the obvious first choice for these mobile devices because of its price. One of the more important reasons is the ability to strip down Linux so much to fit on these devices, but still be incredibly usable.

This session ended up being exactly what I thought. It’s essentially a marketing spiel masquerading as a technical talk. The slides are far too slick, and the only reason any technical details are being given at all is because of the last-minute speaker substitution. Our new speaker is a technical guy who has been promoted to a managerial role. The presentation was apparently designed by a marketing guy with enough technical knowledge to be dangerous. I hope Brad is having more fun in the Moose talk.

I’m really regretting where I’ve chosen to sit. Someone in front of me is wearing way too much pungent cologne. I may be sick.

[tags]oscon, oscon08, oscon2008, Intel, Moblin, mobile, Linux[/tags]

OSCON 2008: Linux on the Corporate Desktop: We Did It, and You Can Too

Posted on by
Reply

The second of my mid-afternoon sessions is Linux on the Corporate Desktop: We Did It, and You Can Too with John Goerzen. This session popped out at me because we have a similar initiative at work. The company John works for has about 400 employees, so obviously no where near the scale we’d be deploying on. Hopefully, I’ll learn a few lessons from someone who’s done it before.

There are a multitude of troubles with using a proprietary operating system, as anyone attending OSCON is familiar. From cost to forced upgrades to vendor lock-in. Suddenly, companies are at the mercy of the vendor, and have lost so much of their own self-direction.

Not only has John’s company benefited from the Open Source community, they’ve contributed back to the community. That’s key, I feel. I’d like to see my own company contribute much more than they do.

I’m not sure who this talk was targeted for. It wasn’t really a good sales pitch to business-type people, and it wasn’t very high level for IT-type people. I don’t know what I expected from it, but I don’t think I got what I wanted out of it. Most of the challenges they faced, we’ve already solved. We’ve already created a standard image and can already deploy it on standard hardware. We already have Windows virtual machines for anyone who still needs to run Windows applications. We already have enough management buy-in for the project, too.

I do, however, like the sound of this “seamless RDP” he talked about. I will need to investigate it further. Also, it’s refreshing to hear from someone who has successfully (mostly) removed Windows from their enterprise.

[tags]oscon, oscon08, oscon2008, Linux[/tags]

“Ninja” Code

Posted on by
1

The Amazon booth at OSCON 2008 is advertising heavily that they are hiring. They are also holding a raffle. To enter, simple look over some Perl code they have written out on some poster board and tell them what it does. It looks a little something like this (transcribing from memory):

my $code = qq{
    print 1+1 . "\n";
    $code =~ m/(\d+)\+(\d+)/;
    $new = $1 + $2;
    $code =~ s/\d+\+(\d+)/$2+$new/;
};

for ( 1 .. 10 ) {
    eval($code);
}

What’s the first bug? Yes, it should use q{}, or the variables will interpolate on the initial assignment to $code. To their credit, they initially used single quotes, but people said it was too hard to read.

I wasn’t content with just figuring out what the code did and fixing a small bug. I think it can be written better.

eval($code = q{
    print 1+1 . "\n";
    $code =~ s/(\d+)(\+)(\d+)/"$3$2" . ($1 + $3)/e;
    eval $code;
});

Much better. Not only is it more concise, I was able to remove that pesky loop, so I wouldn’t be bothered by any silly upper bounds.

So what does it do? Should be obvious. Head over to the Amazon booth and let them know.

OSCON 2008: Code Reviews for Fun and Profit

Posted on by
Reply

Lunch is over and I’m sitting in Code Reviews for Fun and Profit with Alex Martelli. I really wanted to go to the Perl 6 talk, but I always end up going home disappointed, because I don’t yet have Perl 6. It’s maddening, so here I am, sitting in something that may be useful. And we’re off.

Nearly everyone agrees that code reviews are a good idea, so why aren’t they done more often? In fact, this is the very same problem we’ve had at work. We’ve been talking about code reviews for two years, but we’ve never had one.

There are some barriers to entry to doing code reviews. If revision control is not in use or automated tests aren’t being run, tackle those problems first. Also, the need for a team process is necessary, from ticket tracking to release plans.

Pair programming, that tenet of XP, is a poor substitute for code reviews. Two people working together will not magically turn one or the other into what is essentially a disinterested third party, who may catch bugs simply because they weren’t there when it was written.

Test-driven development is also a great way of coding, but not a substitute for reviews. Often for the same reasons. Tests are often just more code and the code tested is only when someone thinks to test it.

Even during a code review, a reverence for authority can get in the way of getting things done. A poor, intimidated programmer may not have the courage to criticize a more senior programmer. Instead, this can be turned around with something I use a lot myself. I like to call it, “playing dumb.” Instead of saying, “this won’t work,” ask what will happen for a suspicious case.

Socially, the only way for code reviews to work is universal buy-in. Everyone is subjected to code reviews by everyone else. No exceptions. Make them a habit, a regularly-scheduled meeting. At work, I’ve even suggested bi-weekly, or perhaps monthly, catered, lunch time code reviews. Just to get us into the habit of doing it.

Code review time should not be wasted on things such as code formatting, best practices, or test coverage. This is stupid. These are objective tasks that can be automated.

Instead, look for subjective things, which can’t be automatically found. Such as code readability, algorithmic clarity, and consistent identifier naming. Other targets for code reviews are the usual things we here over and over again as development best practices: consistent documentation that follows the internal standard, that kind of thing.

The remainder of the talk is essentially an enumeration of all the things to look for in code reviews. All of them are, at least to me, common sense. So I’m not going to spend any time writing them down. If you don’t already know them, well go find some common sense.

One thing that he recommends that I like is code reviews by e-mail. It’s an old, well-understood, and (usually) reliable tool. So why not combine e-mail with a version control system—particularly one of the newer distributed version control systems—to perform out-of-band code reviews. It actually sounds like a good idea to me, and I’ve done it at work a couple of times with code written by an intern.

What I’m starting to notice is that many of the later the recommendations for reviewing code are personal opinions of the presenter. I think the way in which code reviews are performed are highly dependent on what works best for the group reviewing code. It’s like so many things, from cameras to backup solutions: the best one is not the shiniest or the one with the most bells and whistles, it’s the one that’s actually used.

[tags]oscon, oscon08, oscon2008, programming[/tags]