links for 2008-03-20
-
I just ran into this ridiculous behavior, hopefully you won't too!
links for 2008-03-06
-
Git as a backup system makes a lot of sense, and this looks like a great way to try it out.
-
I think this is my favorite of the ruby 'ignore nil' memes I've seen floating around.
links for 2008-03-05
-
Great post about how language/framework success isn't a zero sum game.
Hype is a good thing
While listening to the Ruby on Rails podcast, one of the guests mentioned that one of the things behind their decision to use Rails for their project was the popularity of the language. Now, social news sites are weird. Browsing reddit and digg, you get the impression that popularity and hype, in the context of programming languages, concepts, and libraries, is a bad thing. I never really understood why popularity is seen as negative, especially in an area where community is important.
One of the enormous benefits of popularity, especially in the programming world, is the benefit of a large, active community of early adopters. Given an unpopular language and a popular language, the popular language will have more libraries, more support, and more experimentation than the unpopular language. As long as those early adopters are hammering on the language, it will quickly improve and give programmers in that language the resources they need to get work done.
Take two imaginary languages, Blub and Frob. They are fundamentally pretty equivalent, except Blub uses tabs instead of spaces and Frob has an awesome emacs mode but sucks with vi. Blub is way more popular (annoyingly so) than Frob in the blog circuit, but Frob has a better implementation and runtime than Blub. These differences, although minor, make for heated discussions among a team that is interested in the productivity benefits either language will give them.
I'll pick Blub every time, because when I run into an esoteric error message, I'll find 10 blogs from google explaining the problem and workarounds, instead of half a blog post saying, "I ran into this problem, does anyone have a solution?" (with no comments, of course). When I need a library to interface with our new caching service, I'll have tons to choose from (and modify), instead of one that I may or may not like using. With the number of eyes looking at Blub and how it compares to Frob, there will be people working to mitigate and solve issues that make it harder to work in Blub than Frob.
Even though there will be dedicated, hardworking, genius programmers working in Frob, they probably won't be able to produce as much help and resources as the early adopters that are jumping on the Blub bandwagon, documenting their experiences, writing code to experiment with features, and making Blub better and easier to use.
Of course, popularity does not necessarily make a language better -- look at Java for one great example of a popular language that I never want to touch again. Popularity as a language "feature" makes the most sense with up and coming languages. Hyped languages also attract people who will just as easily flock to the "next big thing," reducing the pool of contributing users that give it this advantage in the first place, and attracting a large community will often leave the average quality of work lacking. But as annoying as an overhyped language can be, it's often a good reason to pick a language over other, mostly equivalent languages when it comes to building on the work of others.
links for 2008-02-27 1
-
Rails Core is investigating switching to git.
-
I've never really been into hosted source control, but I'm going to have to check GitHub out.
Bootstrapping content with Hpricot 1
On my latest project, I discovered I had to pre-populate the project's database with existing content. Jon Udell just posted about how much of a waste of time this can be in some circumstances, but in this case, Hpricot and database migrations made it easy. This wouldn't be a solution I'd use if I needed the data as anything beyond a one-off bootstrap, but in this case it worked really well.
Hpricot, for those who don't know, is an HTML parser for Ruby that's fun to use. When I was first learning Ruby, most of the simplest yet useful projects I could come up with used Hpricot to grab content off of websites and format or combine it in different ways. Its syntax looks like this:
require 'hpricot'
require 'open-uri'
uri = URI.parse(link)
doc = Hpricot(open(uri))
name = (doc/"li.active a").inner_html
page_title = (doc/"title").inner_html
body = (doc/"#content_body").html
In this example, Hpricot is using CSS selectors to grab different pieces of content out of the page in link. The nice thing about using CSS selectors here is the code tends to be less fragile than screenscrapers that depend on the architecture of the page.
Page scraping can be a frustrating art, especially if the page layout changes or if pages are inconsistent, or have unique properties. Luckily, in this case, I only had to get it right once, and even then, I didn't have to get it completely right. I used this four-stage process:
- Use Hpricot to get as much data off the page and into our data structures as possible.
- Persist this data to the database, and make appropriate changes that Hpricot missed, or couldn't catch.
- Dump the database to a file, and use it to bootstrap our production database.
- Repeat until finished.
Rails database migrations made this relatively easy. I ended up with three migrations. The first migration created the structure of the database. The second loaded the current page data dump from the dump file. The third grabbed a few pages I still needed to parse, and I was left with data that I could tweak and dump, overwriting it with a dump containing all the page data (including the stuff I just tweaked). I could then blow away the database and repeat until I didn't have any more pages to parse.
This worked perfectly, since I didn't have to spend time getting my Hpricot parsing perfect (since I could modify the resulting data using our CMS and re-dump), and I was left with a dump of all the data that I needed in order to dynamically generate these formerly mostly static pages.
links for 2008-02-22
-
I did something similar in a previous job. This pretty well describes why writing Web Services in Java almost made me quit programming.
-
Just yesterday I thought 'I wonder if anyone has written a git bundle for TextMate.' I have nothing if not great timing.
-
Certificates are awesome, and I wish they were used in more places (and could be shared between more applications)
Counterintuitive thinking
Like all good programmers, I read Paul Graham's essays. One of his more recent ones got a lot of attention, especially these two quotes:
Here it is: I like to find (a) simple solutions (b) to overlooked problems (c) that actually need to be solved, and (d) deliver them as informally as possible, (e) starting with a very crude version 1, then (f) iterating rapidly.
and
So when you look at something like Reddit and think "I wish I could think of an idea like that," remember: ideas like that are all around you. But you ignore them because they look wrong.
The first is interesting because it's the same sort of principles most of the agile programmers have, but it's the second that mentions what I found insightful about the essay. There's another paragraph in the essay that explains it better:
I'd noticed, of course, that people never seemed to grasp new ideas at first. I thought it was just because most people were stupid. Now I see there's more to it than that. Like a contrarian investment fund, someone following this strategy will almost always be doing things that seem wrong to the average person.
When I read this section, I thought "yeah, but that's the point." Which is why I liked what came next:
As with contrarian investment strategies, that's exactly the point.
Something that I've been interested in lately is the counterintuitive. In a lot of cases, I've found that by doing counterintuitive things, I've had way more success than I had by following everyone else.
There are a few counterintuitive rules of thumb that are particularly interesting:
- "Adding manpower to a late software project makes it later."
- The success of Ruby on Rails, despite having very strong opinions on how software should be built, (and even discouraging breaking those guidelines)
- Intentionally excluding people from a group often results in more people being interested and eventually joining that group (the "I only want it because I can't have it" principle)
The problem with the counterintuitive is that if it is successful, it often no longer becomes counterintuitive and loses the power of originality it once had. The Red Queen is a great book about this, but I've noticed it in businesses that try to predict market trends (once a trend is identified as a trend, it's exploited until it no longer becomes a trend) and in technology (as Graham outlines). Still, looking for ways in which counterintutive thinking can lead to success will usually, if nothing else, bring you one step ahead of everyone else.