OSCON 2011: Thursday

Thursday was the second day of sessions at the O’Reilly Open Source Convention (OSCON) and my third day in Portland, Oregon. Overall, the sessions I attended were arguably more relevant to my work than those I attended on Wednesday. Still, the day left me feeling unsatisfied. At past OSCONs, I ended each day with my mind brimming with new ideas, scarcely able to wait until I could put some of them into practice. So far, this year’s conference hasn’t had the same effect on me.

In any case, the Thursday morning keynotes were far better than those foisted upon us on Wednesday morning. Gabe Zichermann’s talk, in particular, caught my attention. In Game theory applied to user engagement in Open Source he talked about using so-called gamification techniques to draw people into using Open Source software. Many of his examples had to do with using game theory to alter real life behavior, such as a lottery to reward good drivers in Sweden or the use of consumption graphs in hybrid vehicles. On a separate note, I tend to grow annoyed at the latter, having been stuck behind too many hypermiling drivers.

Getting into the sessions, I favored those more in line with the work I do as a Perl programming system administrator. Also, it didn’t hurt that The Conway Channel 2011 happened to take place during the first time slot of the day. I’m a bit sorry I passed up DIY Clinical Trials (Or: How to Guinea Pig Your Way to Scientific Truth and Better Health), if only for the reason that it would have been completely different from anything I normally do. But, I attended those types of sessions on Wednesday, so it was back to business, so to speak. Damian Conway was in his usual top form, as entertaining as he is educational. I won’t go into too much detail, only to note that he demonstrated four of his modules, using a theme I’m sure most will recognize. First, something old, updates to the Regexp::Grammars module. He then introduced something new, the IO::Prompter module, which supersedes his older IO::Prompt. There was something borrowed, the Data::Show module, which serves as a convenience wrapper around the Data::Dump module. And finally, something blue, the Acme::Crap module, which seems oddly cathartic.

I like to think I’m a halfway decent Perl programmer, but that doesn’t mean I think I can ignore things like Jacinta Richardson’s Perl Programming Best Practices 2011. The talk was a round-up of the tools and modules that are generally considered to be the best practices by the Perl community today. Yes, generally. People will have their differences of opinion, and I don’t always agree with the advertised best practices. However, if followed, the practices will lead to better code, and if violating a practice, I like to be able to back that up with a well thought out reason (it doesn’t necessarily have to be a good reason). The first of two, possibly pithy, examples of this is the local::lib and it’s default use of ~/perl5 as its include path. I prefer to use ~/local/lib/perl5 and, sure, the module allows me to do that easily enough, but it’s an extra, non-standard step. Second, the cpanm has been touted as the best way to install modules from CPAN. As a control freak with a highly customized CPAN configuration, I’ve never liked the way cpanm seems to do things its way. Admittedly, it may be customizable, but I’ve never had the need to look into it.

There’s been some noise around the office about testing Amazon’s EC2 offering. To that end, I thought James Loope’s Utility and Automation: Low Overhead Operations with Amazon & Puppet would be educational, possibly giving me some ideas about how to managing our own potential EC2 environment. Unfortunately, it didn’t work out that way for me. The talk was heavily focused on the way the web application was designed and pieces of Amazon’s infrastructure were used. We’re not creating or running web applications, so none of it was beneficial to me. There was nothing about Puppet aside from explaining that using it (or another configuration management tool) is vital for keeping everything running.

At this point, I was turned off from any cloud talks at OSCON. There seems to be, with probably good reason, an inextricable tangling of cloud and web applications. Because of this, I decided to pass on Achieving Hybrid Cloud Mobility with OpenStack and XCP and instead attended Piers Cawley’s Polymorphic Dispatch—It’s Not Just a Good Idea, It’s the Law. I’m glad I did, because there were definitely some very useful ideas presented. The idea, taken from Smalltalk, of passing messages to objects has a lot of merit. Combining this with polymorphism, sending a message and allowing different objects to act on it differently, vastly simplifies code. Simple code, of course, is easier to test and easier to debug when things go horribly wrong (and actually is less likely to go horribly wrong in the first place). Of particular interest to me were the Null Object pattern and what Piers referred to as the key tenant of object-oriented programming: tell, don’t ask. That is to say, if I understood correctly, instead of querying an object for information and using it to determine which action to perform, give the information to the object and have it perform the action. Finally, Smalltalk Best Practice Patterns was recommended as the best book on good coding practices out there. According to Piers, it “will change the way you think about programming.”

I was in way over my head in Tom Christiansen’s Unicode in Perl Regexes. The only thing I managed to learn is that I don’t know nearly enough about Unicode to actually understand using it. I’ll leave it at that. It was a very information-dense session and it’s possible that Tom knows more about Unicode than those who designed it. Other choices during this time slot, which may have been better for me, were Connecting iOS to the Real World with Arduino, presented by my friend Alasdair Allan, or, venturing again into the realm of health geekery, Open Source Preventive Medicine: Citizen Science Genomics

The last session I attended on Thursday had so much potential, but, for me, it fell flat. I expected A. Sinan Unur‘s Visualizing Economic Data Using Perl and HTML5′s Canvas to focus far more on visualization than it did. Instead, the majority of the presentation was about the difficulty of parsing public data published by the United States government. For this, Sinan uses Spreadsheet::ParseExcel and explained a few of the techniques he uses to extract data from tables designed primarily for visual consumption. Unfortunately, very little time was spent showing how Canvas was used. We were given one static example and an explanation that there is no method available for determining the height of text in a Canvas element. I had hoped to return to work with some ideas for using Canvas to visualize data from our batch scheduling system, but ultimately left disappointed.

After the last session, I met up with a coworker, an old friend, and a new friend to have dinner at Chipotle. Normally, I like to avoid chain restaurants—national chains in particular—when traveling, preferring to sample the local cuisine. But, we wanted a quick dinner and it was nearby. My opinion was requested, on the relative healthfulness of pinto versus black beans. I simply stated that I would be ordering my carnitas bowl without any beans.

After dinner, we returned to the convention center for the Perl Lightning Talks and the State of the Onion. As always, the talks were quite entertaining. Of note was a juggling demonstration, illustrating various programming languages and databases. Near the end, Ricardo Signes recounted a conversation he had with a couple of women from the knitting conference sharing the convention center with us. Its presence provided a wonderful juxtaposition. While OSCON is male-dominated and many don’t know how to act when women brave their way into our midst, the knitting convention is completely opposite. Ricardo’s message to us was, take the time to look up from our laptops and chat with those around us. We might just have a better time and make new friends.

Finally, Piers Cawley favored us, as he does every year, with a song. This year, however, he did not bear a tale of levity, but a message of deadly seriousness. The United Kingdom is closing libraries in an attempt to reduce public spending. As a protest, Piers wrote a song, “Child of the Library”. There doesn’t appear to be any video (yet) of Piers performing at OSCON, but I’ve gone ahead and embedded one that I found. It’s catchy, I had it stuck in my head for a couple of days after the conference.

We could easily see the same thing happen in the United States—and in fact I have already seen it proposed in San Diego. I’ll first admit that I have not set foot inside a library since college, over a decade ago (high school, if only counting public libraries). Do libraries still matter, or is the concern over their closing merely the knee-jerk nostalgia of those of us who came of age in a world that didn’t yet know the Internet? I can’t, and won’t, take a side on this issue until I’ve taken the time to visit my local library. If I can recognize it as something I saw in my childhood, perhaps it should be closed. If it has adapted to the so-called Information Age, maybe it’s worth funding.

As a final, humorous note, I almost didn’t make it back to my hotel. At least, not without finding an alternate method of transportation. At 10:22 PM, excusing myself and apologizing for staying so far away from the conference, I left Media Temple party at the Jupiter Hotel, arriving at the convention center MAX station at 10:32 PM. The schedule at the station listed 10:42 PM as the last red line train to the airport, with Google Maps concurring that a train was 10 minutes away. About two minutes later an unmarked blue line train arrived at the station, traveling east. At this point, Google Maps had decided it would rather show me its trip planner instead of the previous screen which showed the impending arrival of the red line. Forced to make a split-second decision, I hopped on the train. I knew that I could take it at least as far as the Gateway station, where I could transfer to the red line if it was still behind me. Around 11:00 PM I arrived at Gateway, after spending the ride thinking about how much a cab would cost. This station had a real-time display with train arrival times. The last red line of the day was only three minutes out. Whew.

OSCON 2010: Friday Morning Plenary Sessions

I’m tired this morning after a long week at OSCON, so my ability to understand and summarize the Friday plenary sessions is diminished. As such, what follows won’t be terribly useful to anyone.

Your Work in Open Source, 3 years of Incremental Change

Chris DiBona (Google, Inc.)

Google crawled 40 million files in Google Code to generate statistics on what’s in there. Lines of code and numbers of commits are not the most useful of metrics but that’s what they have to use.

The Gnu General Public License is the most used license, at over 50%. Of those, more than half have moved to GPL version 3. Perl has declined a bit, but C has the most use, at about 40%.

Many companies are committing code, too.

Mayor Sam Adams

Sam Adams (City of Portland, Oregon)

Last September, Portland adopted one of the first Open Source policies in the nation. They’ve committed themselves to open software, open data, and Open Source in the procurement process for software.

It’s pretty cool when a politician gets it.

Situation Normal, Everything Must Change

Simon Wardley (Leading Edge Forum (CSC))

Simon started with a recap of the talk he gave last year, which showed correlations between the ubiquity and certainty. All technologies follow the same curve, from having both low ubiquity and certainty up to having both high ubiquity and certainty. The stages tend to be the innovation of a technology, the productization, and finally the comoditization.

The basic idea was that the cloud, as it is known, is still in its infancy. As it matures, we will see innovations built on it at an accelerated rate. If we don’t pay attention to it, we’ll be left behind.

Well defined processes stifle innovation.

Projects or teams can be organized by lifecycle: innovation, leverage, and commoditize. This circles back on itself. When one thing is commoditized, a new innovation can be built on top of it.

OSCON 2010: Thursday Plenary Session

This morning’s plenary session, based on the scheduled speakers, is focused on the nebulous cloud. The cloud is what everyone in technology talks about, but everyone defines differently. It’s the section of the flow chart where magic happens. Somehow, we will send our data into the cloud and all our wishes will be fulfilled.

To be fair, this vagueness and my pessimism are precisely why these speakers have been invited to the O’Reilly Open Source Convention. Tim O’Reilly has a grand vision for the cloud, for ubiquitous computing, and the use of technology to help solve the world’s problems. I commend him for that. I hope this morning’s speakers do justice to his vision and that, if there are valuable lessons to be learned, that we learn them.

Today’s LAMP Stack

David Recordon (Facebook)

Over the last decade, the LAMP stack hasn’t been fundamentally updated. A cache, such as memcached has been added. Different languages (Perl, Python, Ruby) have been used in place of the original PHP. Even different web servers have been used in place of Apache.

Facebook created HipHop for PHP, which compiles PHP into C++. Creating native executables in this way reduces CPU use by a large factor (a number I didn’t catch).

There are alternatives for the database component in the stack, too. MySQL is ubiquitous at this point. Facebook doesn’t really use the relational bits of MySQL very much. So they have been using databases from the NoSQL family—Hadoop, according to the presentation.

David made a point I think a lot of people miss. When evaluating databases, or any other software, first look at what problem needs solving, then find a product that solves it in the correct way. Too often I see people advocating their preferred solution without even looking at the problem.

Data is the lifeblood of Facebook (and we all have our own opinions about that). They are able to use a plethora of Open Source tools to store the data, scale the data, and analyze the data.

This talk wasn’t very focused on the cloud, aside from Facebook being a nebulous site where people store their data without really knowing where it goes or how it’s used. I expect this was more for public relations, given all the bad press they’ve had. Not that anyone stops using Facebook.

Open SETIQuest – It Will Be What You Make It!

Jill Tarter (SETI Institute)

Jill started her talk by explaining what SETI is and why it exists, which I won’t go into here, since it’s just a Google search away. I used to run SETI@home a bit over a decade ago when I was in college and felt like using my computer as a space heater.

Jill is here, representing SETI, because she wants to involve the world in their search. SETI has classroom materials covered, but they are lacking in the social networking world. Jill wants people to first identify themselves as Earthlings, recognizing our place in the Universe.

SETI, with the development of the setiQuest community, hopes to use the vast resources available in the Open Source world to improve the project. These include physical resources, such as cloud storage and compute cycles, to human resources, such as programmers and analysts.

Cloudant has created a SETI stack on the Amazon AWS infrastructure.

Open Cloud, Open Data

Jean Paoli (Microsoft)

I’m always a little concerned when I see a speaker from Microsoft at OSCON. While I can imagine that there are employees at the company who genuinely embrace Open Source—and, presumably from this talk, open data—I can’t lay aside my suspicion. Microsoft does not have a benevolent history with competition, so when a representative shows up to talk about an open cloud with open data, I instinctively look for the company’s angle. What is their nefarious plan?

Jean talked about open standards and open data. Data portability, standards, easy migration and deployment, and developer choice. For some reason, when he talks about the “open cloud,” I have thoughts about Microsoft’s OpenDocument move a few years ago. Sure, parts of it were open, but the format as a whole was useless for non-Microsoft tools.

He claimed that Microsoft Windows Azure is an open and interoperable platform. I have a hard time swallowing that. The #oscon IRC channel was not kind in its commentary. A selection from the channel logs:

<b3gl> "Microsoft totally agrees..." as long as you pay your Windows, Azure and MSSQL license fees

<alapapa> standards are great…as long as they're ours

<dbrewer> wow, thanks Microsoft.  You think I should be able to use any language I want, I appreciate your permissions.

<b3gl> dbrewer: Notice he didn't say "We believe if you want to use Linux ...."

Public Static Void

Rob Pike (Google, Inc.)

Programming languages used to be relatively simple, but still fairly powerful. They’ve gotten considerably more complex and confusing. The C++ language was used as an easy target during the talk. Rob went on to bash various (in most cases deservedly) programming languages as a way to lead into what he called the renaissance of language design.

Many of the emerging languages are dynamic and interpreted, and there’s a false dichotomy that they are considered good while the static and compiled languages are considered bad. Part of the problem is that the latter class of languages are old, designed in a different age of computing.

Obviously, the end goal of this talk was to talk about the Go language, which tries to bridge the gap between the dynamic interpreted languages and the static compiled languages.

Toward an Open Cloud

Lew Moorman (Rackspace.com)

Lew’s talk was to introduce OpenStack. Rackspace took the internal software that powers their cloud and donated it to OpenStack. I wonder if this is something we can use at my day job to build an internal cloud. The stack is licensed under the Apache 2 license and they don’t use a dual licensing model, which sounds nice.


I was wrong, the talks weren’t really about demonstrating the wonder of ubiquitous computing and how we can move in that direction so much as a showcase of organizations in the cloud or using the cloud. It was really just one long commercial.

OSCON 2010: Building Applications with the Simple Cloud API

Doug Tidwell (IBM)

http://www.oscon.com/oscon2010/public/schedule/detail/13976

I finally left the Perl track. I attended Tim Bunce’s presentation on Devel::NYTProf at OSCON two years ago and, while there have been many enhancements made to module since that time, I expect this year’s talk won’t differ much from the previous one.

This session on Simple Cloud is being presented by IBM’s Cloud Computing Evangelist. The drivers behind this product (is it a product?) are the development and promotion of a standard cloud API. There is some relevancy with my day job, not only because of the possibility of using cloud services, but as a way of getting ideas for the API I develop for our engineers to interact with the batch compute system.

There are several levels of where we can work. The levels start at the wire, where we have to generate and parse data ourselves. From there, we have vendor-specific APIs, service-specific APIs, and finally service-neutral APIs. This last level is where we want to be.

The Simple Cloud API covers three areas: file storage, document storage, and simple queues. Once thought of in these simplified concepts, there really isn’t any reason the interface used by a program can’t be standardized. A program should no more need to concern itself with the implementation details of an individual cloud provider than it does the details of the file system of the computer on which it runs.

The API uses the Factory and Adapter design patterns, with a configuration file used by the Factory object to determine which Adapter should be created. These patterns are exactly what I’ve been looking at for the API I work on at my day job.

A demo of the Simple Cloud API followed. There wasn’t much to these demos. The first showed listing data stored at two different providers. The second showed queue manipulation.

After the demo, the Apache libcloud, which is getting a good deal of vendor support.