Does your product pass the subway test?
A few days ago a friend of mine posted his thoughts about 37signals and Basecamp, their flagship product. He mentions that users he’d signed up on his Basecamp account didn’t end up using it, and speculates that it’s because the application isn’t user-friendly. As evident from the comments, a number of other readers share the same frustration with Basecamp. A few people also chime in to voice dissatisfaction with other 37signals products.
For those of you that don’t know, 37signals is one of the darlings of the Web 2.0 world. They’ve built a suite of successful web applications, have a large number of readers on their blog, and published a popular book on how to create great web applications. To suggest that their products aren’t well designed and user-friendly is outright blasphemy in a lot of circles.
As I was thinking about Doug’s post, I came across an interesting article in the Washington Post by way of the Freakonomics blog. The Post ran an experiment where they took Joshua Bell, an expert violinist, to a Washington D.C. subway stop and had him play beautiful pieces on his $3.5 million violin.
I hate to spoil the ending, but Bell played for about an hour and went largely unnoticed. He collected a mere $32.17 for his time, even though he can sometimes earn $1,000 per minute for his work.
I would like to think that I’d walk by Bell in the subway and be struck by his talent and the beauty of the music. But in reality I’d probably rush by, annoyed that some guy was allowed to play that loudly in public. And with all honesty, even if the venue had been more appropriate and Bell had my complete attention, I probably wouldn’t sense that I was in the presence of a master. But if I knew who he was beforehand I would probably feel overwhelmed with appreciation.
Here is the key question posed by the Post article:
IF A GREAT MUSICIAN PLAYS GREAT MUSIC BUT NO ONE HEARS . . . WAS HE REALLY ANY GOOD?
We ask this kind of question about classical music and abstract art, but why don’t we ask it about technology more often?
It’s hard to say that Basecamp isn’t user-friendly when everyone points to 37signals as the experts on creating user-friendly web applications. It’s hard to say that the iPod is mediocre if everyone feels like it was an object of perfection handed down to Steve Jobs from above. It’s hard to say that Google Search returns irrelevant search results when everyone calls the engine the greatest tool of the web age.
The Post asked Leonard Slatkin, music director of the National Symphony Orchestra, what he thought would happen in the Bell experiment. He was pessimistic, but his estimate still far exceeded the actual results. I think those of us that subscribe to dozens of “what’s new in technology” blogs and digest the content every day are in a lot of ways like Slatkin. We live and breath technology, so that puts us closer to Arrington, Godin, Graham, Kawasaki, Scoble and Spolsky than the 1,000s of people walking through the subway.
It seems right to say that Basecamp is awesome. But if nobody on your team wants to use it that doesn’t compute. Are they ignorant, or are they just seeing it sans hype?
Would Basecamp pass the subway test? How about the many products featured on Engadget and Techcrunch every day?
Douglas Karr on April 10th, 2007
Oh man, you really want to share in the wrath of the cult of the 37s? And you threw Apple and Google on there, too? Are you crazy???
I’m kidding of course - but I’m sure the response would be the same had Joshua Bell’s biggest fans stood and argued with someone going by when they didn’t stop and listen or throw their dollar in the ($3.5MM) violin case. They would have been treated as I was. I’m still actually shocked at how mean people were (and the comments only got worse on reddit.com).
When you challenge someone’s faith, I suppose it’s different than simply challenging them as a person. You’re actually challenging what they enjoy, have dedicated themselves to, and what they believe in. To some extent, cult really is a good word for it. Apple can come out with a cell phone after 10 years of cell phones, have it cost twice as much and not do any more - and it will sell like hotcakes.
I suppose if I was a company, that’s the kind of following I want! Anyways, it’s a fantastic post and a proper follow-up. Had I been as eloquent in my prose, I may not have stirred such a hornet’s nest. It’s not how I wanted my blog to break into the big time.
Ed S. on April 11th, 2007
Excellent post. I think one thing to note is that these people developed that following because of a reason. It may have been because it was a great product for a certain niche of users and then it got media attention or simply because it was rebellious.
It’s essential to remember why products and people got to where they are today. I truly believe that the iPod was a great product not because of Steve Jobs but because when coupled with iTunes (which only ran on a mac for a long time) it “Just Worked(TM)”. Steve knows the cell phone market is incredibly difficult to penetrate but he believes at his core that his company can take something that mostly works and make it work better. I’ll say the same about Google and probably Basecamp. The problem is that they won’t suite everyone and if that’s the case then how do you gracefully deal with criticism and continue to evolve your products.
Doug hit a nerve because he rightly pointed out some valid criticisms that deserved to be addressed. Instead of gracefully dealing with these criticisms, Jason Fried took an alternative path and basically pissed on them which is unfortunate.
Blogs may be the new way to communicate with your partners, customers and the general public but the rules are the same as always. Those who are humble and respectful of others usually get a lot further than those who don’t. The Cluetrain Manifesto would be a good read for Jason and crew.
Ian on April 11th, 2007
You hyperlinked a lot of text in the Joshua Bell section… a closet classical music fan?
Hugh Bien on April 11th, 2007
Great post, to be honest a lot of web applications I use are just for personal use. Whenever I work on a team, everyone else usually favors emails over logging into and using Basecamp or Trac. I think it could be a ‘why learn something new when emails already work’ feeling.
It’s hard to tell if a web app is user friendly or not, because we’re all tech oriented people. We MAKE applications, we read about them, and we get excited about them.
Berkay on April 11th, 2007
You should be at awe for 37Signals marketing skills. As for the “user friendliness” of their software? I can’t figure out where this myth is originating from. They seem to be more interested in “beauty” of their software code then the functionality or user friendliness of their software.
I had the same experience as Doug, I’ve promoted and subscribed to Basecamp, but our team did not want to use it. Primary complaint? user interface, too many steps to do things, etc. Basically complete opposite of what one may think about 37signals software. It was an informative experience for me.
I have a lot of respects for the business skills of 37Signals. They are a model for how a small company may succeed and use the web effectively. I even think that they have some very good advice for software developers, but their software is not very good, and their arrogant attitude is something to steer clear for sure.
ade on April 11th, 2007
@Doug: Ha! No, I don’t want to share in the wrath that you’ve received this week. But yeah, I can’t refrain from taunting the Apple fanboys whenever I get a chance, especially since I know most of my loyal readers are Mac users
Obviously, most of the questions in my post where rhetorical … I don’t think there’s a clear yes/no to any of it, just food for thought. I’m not a hater of 37signals, Apple, or Google, but I’m not a fanboy either. A lot of things I love, and some things I can’t stand about each.
@Ed: Great points. All those companies have had to deal with going from cult favorites to the successful mainstream. The game changes when you make that leap … you can’t rest on your earlier successes to carry new initiatives, and you have to tweak what works with your newfound size.
@Ian: I don’t listen to classical music often, but do like it. I’m usually listening to electronica or jazz, and there are a lot of similarities to classical music. That Chaconne movement of Bach’s Violin Partita no. 2 in D minor (which Bell played in the subway) is absolutely beautiful. I just got inspired to listen to it again
I’d love to hear a DJ Tiesto remix of that.
Frank Chickens on April 11th, 2007
Douglas Carr:
“When you challenge someone’s faith, I suppose it’s different than simply challenging them as a person.”
It wasn’t the fact that you challenged “faith” that drew ire.
It was the lack of substance of your “criticism”… i.e. critcizing 37S’s blog for using the “f-bomb”.
If you’re going to throw shit around, don’t complain when some of it comes back to you. Have substance to your complaint or take your lumps.
This blog post at least has a point, albeit weak and obvious.
Frank Chickens on April 11th, 2007
Ed S.: “The Cluetrain Manifesto would be a good read for Jason and crew.”
That’s pretty ironic given what Chris Locke, co-author of the manifesto, has recently been involved in.
Douglas Karr on April 11th, 2007
Frank,
Fact: Use of the word was a factor in me leaving Basecamp.
Fact: I was a paying Basecamp customer.
Fact: I am no longer a paying customer.
Conclusion: 37Signals use of the f-bomb contributed to them losing money and a customer.
No complaints here,
Doug
Pete Skomoroch on April 11th, 2007
Reminds me of Babette’s Feast, web 2.0-style
http://en.wikipedia.org/wiki/Babette’s_Feast
Steven Harman on April 11th, 2007
I use another *highly scientific and empirically backed* method of testing the true usability/coolness of a web application - I call it the Girl Friend Test.
It’s pretty simple: I show the application/site to my girl friend and let her use it for a short while. And if she “gets it” - it passes the test!
Sometimes we are too close to our products and/or the technology that fresh pair of eyes can really bring a new perspective and clarity. Think long and hard about who your key user-group is and the make sure they can actually use it. This is where Low-Fi Usability Testing really comes in to play (http://www.codinghorror.com/blog/archives/000779.html).
Frank Chickens on April 11th, 2007
“Conclusion: 37Signals use of the f-bomb contributed to them losing money and a customer.”
With a decision making process like that, I have to wonder what kind of value you deliver to your customers.
Douglas Karr on April 12th, 2007
Frank, it’s pretty simple. I advise my clients not to swear on their blog because it can alienate customers that may otherwise appreciate your products and services. If swearing is a core competency and attracts business, go for it. Otherwise, it’s simply not necessary.
You may not think so, but I respect your opinion. I know plenty of people who are not offended. I was in the USN for 6 years and can cuss with the best of them. I don’t do it on my blog, though. I believe that it’s more respectful to others to take their feelings into consideration than pushing your own.
I’m confident you read many sites and blogs that do not cuss, correct? So a blogger that doesn’t cuss can attract people who don’t mind cussing as well as people who do. But a blogger that does cuss may alienate people who do mind. That’s not a good strategy for attracting readers and customers.
Doug
Douglas Karr on April 12th, 2007
Steven, that’s fantastic!
ade on April 12th, 2007
@Pete: thanks for the Babette’s Feast reference. I already read the spoiler, but I just added it to my queue nonetheless.
Frank Chickens on April 12th, 2007
“…a blogger that does cuss may alienate people who do mind. That’s not a good strategy for attracting readers and customers.”
There are many ways to alienate people.
I’m not one of the “faithful”. I’ve rarely read 37S’s blog. As someone who’s followed the “web 2.0″ revival of web application development, however, I have a massive respect for what they’ve given to it: through setting an example of how to exist free of venture capital, through giving away books and tools, and through their contributions to design philosophy. They’ve done well financially by helping many, many people.
By dismissing people who have given so mcuh for such trivial reasons, you, like it or not, alienate people as much any anyone who uses the “f-bomb”.
Dare on April 12th, 2007
great post! The Tipping Point kept coming to mind when reading your post and the importance of maeve’s in making an idea/product spread. They are the ones that essentially know and decide whether a product in a particular category (music/technology) is superior and then inform the rest of us of the product’s superiority (with help from connectors and others, according to Malcolm Gladwell).
It seems at that point, the product begins to spread to the masses and there’s another step in making it user-friendly not just for those that are familiar with technology or music, but also for the masses.
I know in regards to music, an artist can reach hardcore music fans and be critically acclaimed but if they want mainstream success they need to “dumb down” their music another level to make it easily relatable and appeal to the masses.
My point is, although many of the maeve’s within technology may love 37signals’ products, they may not have made them user-friendly enough for the rest of us (or even for all the maeve’s within their category).
brock on May 11th, 2007
oh holy hell. the only people that say 37 Signals is user-friendly is 37 Signals themselves or the extremely deluded.
In fact they are explicitly anti-user, opining instead that their great wisdom is better than what users think. They also don’t believe in documentation, preferring to
There is truth in what they are saying. If you build what users tell you they want, you’ve got a mess. But persuing just what you think is important is equally as misguided. The way to get innovative, user-centered products is to fully understand what users are trying to achieve. The hard part is developing insights about those tasks and goals and in finding elegant ways to satisfy them. A few gel buttons, clever names, and horrible URLs aren’t they way to get there.
37 Signals is equally misguided on number of other fronts. They have a lot of good ideas, and some tools with a lot of potential, but they’d do us all a lot of good get their head out of their ass.
The subway test itself is full of flaws. Why is the value of the music dependant on some schmucks stoping and giving a buck? Mr. Bell is a talented musician (I remember when he was just a lad, btw), but how much does his $3.5M violin contribute to that? From my experience in other arenas, I’d guess not that much. Sure a $50k car is better and has more features than a $25k car - maybe even 2X as much; but a $250k Bentley isn’t 10X better.
I’d buzz by as well, but the reason would be that my frame of mind would be about commuting (and i generally have my own tunes.) I don’t think I need to be told that he is a master to appreciate, but I do have to be told to switch my frame from getting to work to listening to music.
Roger on May 25th, 2007
Very interesting post. I’ve been watching the rise of 37s, and I have slowly been coming to a conclusion similar to Berklay’s and brock’s. That is, I appreciate that they are a “star” in the community drawing attention to design issues, but I’m not really sold on their output. I’ve been using Backpack for a while now (partly lured by the glorious praise heaped upon it at the signup page), but I have consistently found myself muttering, “I really should keep track of this on Backpack, but it’s such a pain.”