In my experience it was a fool’s errand that was perhaps useful in optimizing some impulsive, synaptic user twitch but was utterly useless in providing any meaningful understanding of user motivation or desire.

But maybe I just had a bad run of it.

Still, my original question remains: can you name any innovative products or services that were developed with this method?

In the quotes you give from Peter, it appears he contradicts himself, but in his post, where you see his train of thought organized on the page, his contradiction isn’t so apparent.

There are two types of research at play here. The initial up front research to categorize or place the problem and define the solution (i.e. the problem definition), then there’s, what we at IDEO might call, the prototyping stage, where we learn even more from the solution in existance and in the hands of the ‘user’. This leads us to being able to determine the problem solution.

I think you said as much in: “The Bucket Test Methodology as a key tool in researching how users respond to the emergent systems”. And so my thought was that wouldn’t prototyping + iteration be the best method of research activity at this stage? Which I imagine is simply Bucket Testing as you describe.

…something that no amount of up front research can provide, and something that no design firm or user experience strategy firm can account for unless they are embedded in your company for a long-term relationship.

Though, I do (as you predicted) have a problem with your suggestion that up-front research is ever mistaken as the same as prototyping the solution in the hands of the users. I don’t see up-front research trying to compete with Bucket Testing, in fact is really something quite different.

In your comment you explain it better that as a design firm (I might as well say it:) ‘our’ mode of thinking will be in a different context from that of our clients. We think in terms of solving problems, or making something new and improved and our clients tend to think in terms of marketing: with contingency and necessity. So as a design thinker inside a client organization, formerly within a consultancy organization, I wonder how you’ve changed or evolved.

I think consulting can help clients by providing the analysis and insights to tackling complex problems with emergent behaviours, and then planning for and helping to manage the prototyping or testing of the developed solutions. I believe we do this on a near-regular basis.

Assuming that not all consultancies are equal, I think some successful ones realize the value in teaching their clients about the design process as well as developing one specific to their client’s business and environment. New types of research and prototyping methods included. But to put it back on to the client, in this context, much of the time the client also has to be looking for this type of service. Not many are.

My second thought, while reading this, was of Tom Moran’s plenary from DIS2002 on Everyday Adaptive Design (which Dan Hill documented very well here). This doesn’t speak to the research method, but more to the need to consider what does it mean to actually design for adaptive design because of emergent behaviour. And as you mention, stuff stays in Beta for a long time these days, shrink wrapped software is becoming a thing of the past and hackability is not just limited to software, but to cars and all sorts of products.

I’m disappointed to being absent from IDEA - but Gino will definitely document it well for us.

I’m really excited to see what happens at the upcoming IDEA conference. It is truly the first of its kind; I can only imagine that as physical space, people and technology collide more and more, these types of problems will only get more interesting, and more difficult. All of these disciplines can stand to learn from each other.

You stated, “I believe very strongly that today’s internet/technology based designers do not think enough about design as it applies to a person’s environment.”

I agree. And the reason is that until now, we haven’t had to. Technology/bandwidth/mobility didn’t allow for it.

What a cool place to be right now!

In general, the panel at various points in time tended to refer to product design and traditional social design fields such as planning and architecture. As a trained Landscape Architect, I believe very strongly that today’s internet/technology based designers do not think enough about design as it applies to a person’s environment – physical and social. The value of thinking broader is increased awareness of ALL that shapes behavior at an individual, group and system level.

Let’s be honest, emergent behavior is not new nor are many of the concepts that pop up in our industry (e.g. social networking, micro-formats). I want to believe that these new buzz words will prompt a new desire for designers to look aggressively beyond their own narrow domain and think about design as a big picture.

As we increase our design awareness we will need to evolve our approach. Specifically, ethnography will take on a whole new meaning – static home visits won’t be enough. I think the next few years in internet design will be challenging for designers – it will require us to be humbled and to recognize that our current myopic view of design is not the entire reality.