Great coverage of the event is happening on the IDEA Conference Blog. I will likely make comments there in order to participate in the ongoing conversation on that site, rather than posting anything here.
Overall, it has been a great experience; very heady presentations mixed with a few case studies. Very little in the way of “here’s how you do X and Y” talks, which is fine by me. The Seattle Library rocks. I feel like perhaps I pay too much attention in my daily life; several of the speakers regurgitated the same material they write about elsewhere, and I didn’t learn anything new from them. That said, it was cool to meet the folks behind the ideas, and be able to ask questions in real time.
Onward, ho.
Today I am attending the WebGuild Silicon Valley Annual Conference in Santa Clara, CA. One of the keynote speakers was Marissa Mayer. Marissa is the VP of User Experience at Google. The format of the talk was a directed Q&A session, and then they opened it up to the audience at the end of her talk.
Marissa had some really interesting things to say, and I think that it is worth noting a few of my key takeaways. They are nothing but common user experience sense; Product managers everywhere should take notice.
On Google’s homepage/search simplicity
Google’s simple UI went like this: Sergei was an engineer, and didn’t want to build a “better” page. He just wanted to test his search engine. Initially Sergei didn’t even have a Search button. He figured the search field was enough, since the Return keyboard key worked well enough.
On Designing, Developing, and Launching products
Don’t overinvest upfront. Don’t overthink a product. Think lean from the getgo - not about everything it could do. Get it out there early, change it often. Getting it out there will determine if it’s going to take off; getting out there early will also beat the competition.
Google often walks away from revenue for the sake of the user experience. They believe that if they nail the user experience first, the revenue will come. Never compromise the user experience for the sake of immediate revenue. User experience is the best investment they can make.
Speed is at least important in user experience as any other factor. Don’t forget about speed.
On Social/Knowledge Search
Google is focused on social/knowledge search, and they’re looking at companies in Asian countries who have done a great job with the concept.
On advertising
No ads on the homepage. Why? There’s no content, so how could ads be well-targeted? It simply makes no sense. And if there is content, it still isn’t well-targeted, so ads might still not make sense. Wait until the user gives you explicit input about their interests; you can always target ads better that way.
– END NOTES –
You might hear people say that Google is a cold, machine-driven company that is solely focused on technology; that they don’t have any user experience representation or focus. Don’t believe the hype. They’re a big company that still operates its product teams as lean, mean, aggressive-as-hell user experience and business and engineering monsters.
As they spread themselves thinner, and become involved across more and more disparate verticals it will be interesting to see if Google can maintain their momentum.
I was unfortunately unable to attend BayCHI this week when Peter, Tim and Larry spoke on designing systems with emergent behavior. Lucky for me, Peter posted some of his thoughts and takeaways from the evening.
A couple of things Peter wrote struck a nerve:
Peter first says:
“There are no good user research methods of emergent behavior
The standard tools that a user researcher has tend to focus on an individual’s interaction with the system. Or, if we do engage with groups, we’re limited to the scope of groups we can interact with. So, how can user research inform the design of systems with emergent behavior? We need new methods and approaches that allow us to work with a crowd.”
Peter also goes on to say:
“Embrace the chaos
…I’m surprised that most companies don’t take the Amazon.com approach of testing designs with small percentages of the users and gauging behavior before unleashing it on the world at large.”
Companies do embrace the chaos.
I don’t know if most companies do it, but I know that all the big companies with massive user bases do it. At Yahoo and elsewhere, it’s called bucket testing. Based on any number of factors, including but not limited to revenue risk, research goals and time/budget constraints, a new rev of the product will be launched to a tiny percentage of the population. So when you have 400,000,000 users and you let 5% of them see the new product, you get a rich and reliable data set, as well as a wealth of information on how the product will be used, and how it will perform when you do open the flood gates 100%.
Bucket testing is constantly happening at Yahoo, Google, MSN, Amazon and others. Have you ever seen a new feature for a week or two, and then it vanishes into thin air? You were in a bucket test.
A Strong Statement
I know I’m going to get heat for stating that bucket testing is a user research tool, but I’m going to say it anyway: Bucket testing is a great tool for designing emergent systems. It is not the typical lab-based, or field-based “user research” that we all think of, especially because it happens after the system is designed. However, because of the limitations of human beings’ abilities to predict (unless there are clairvoyant UE people out there) what other people are going to do, bucket testing is the best tool to date that I’ve seen for informing my design decisions in SWEBs (Systems with Emergent Behavior).
As designing emergent systems become increasingly complex, bucket testing is possibly the best user research tool available; something that no amount of up front research can provide, and something that no design firm or user experience strategy firm can account for unless they are embedded in your company for a long-term relationship. I hated to say that last sentence given my consulting history, but it seems to be more and more often the case. It’s time for consultancies to innovate on how to provide this service in a long-term valuable way.
A confession
When I was first exposed to bucket testing, I hated the idea; I was also from a user experience consulting background. I loathed putting my hard work and Fantastic Design Decisions in front of a small piece of the population, just to observe and then tweak and change my minor failures in near-real time. But over time, I have actually learned to embrace The Bucket Test Methodology as a key tool in researching how users respond to the emergent systems I design, and making those systems or products better.
The other tool I should mention, which only few products and companies are lucky to have, is the loud-mouthed users themselves. If you design a product/system that people are passionate about, they will constantly give you, ahem, advice on how the system should emerge to meet their needs. For every five you hear from there are 50,000 that feel the same way. In this case it is your job to be the bullshit filter and improve your product accordingly.
Not the be all end all choice
Bucket testing isn’t The Best Way, or the only way to gain insight into a SWEB. In fact, in many cases, such as games, I don’t see how it has a ton of applicability. However, for large content-driven and transaction-driven and user-driven and recommendation-driven and algorithm-driven online ecosystems, bucket testing might well be one of the best tools at our disposal.
The key is being sure to make bucket testing not just a Product Management-driven exercise; involve User Experience folks from the get go. Otherwise, your bucket tests will be too heavily focused on numbers, and not on evolving based on behavior.
Oh, one last comment on Peter’s post. Many contemporary systems do seem to live in perpetual beta, and I think it’s fantastic. It is a clear philosophical shift from the engineering/shrink-wrapped wait-to-pay-for-the-next-release software model of Yore, and evidence that companies are trying harder to evolve their products at a rate which keeps up with our behavior. And I welcome that. It forces designers and engineers to work more closely; that historical separation are why products have sucked for so long.
Blogger tells me that my email address does not exist. The truth of the matter is that I simply entered my password incorrectly.
It continues to amaze me that after 12 years (or more) of entering user names and passwords online, designers still can’t get this simple interaction, and its simple error cases, down pat.
This morning I was greeted by this ad on the Wall Street Journal Online:
So now that we see the mainstream intellectually embracing the idea of simplicity, will they also internalize the concept and act/design/hire accordingly? I hope John is getting his cut…