Recently I read some papers about how cultural probes evolved since their introduction by Dunne, Gaver and Pacenti. There is one that I found particularly interesting: „How HCI interprets the probes“ by Boehner, Vertesi, Sengers and Dourish. The paper does not critique on the probes as a method directly, but takes the adaptation of cultural probes in HCI as symptomatic for deeper epistemological concerns in the field, as they claim probes to „frame an alternative account to knowledge production“.
The authors’ basic critique is that there has been a desire in HCI to „domesticate“ the probes into an objective, reliable and formal method. They also claim the same attitude to misconstrue many other reflective and interpretative approaches in HCI.
Having examined a large number of paper mentioning the method, they show that there has been adaptation and variation with regard to the field of investigation (identity probes, urban probes, domestic probes, value probes) and the implementation of the probe (mobile probes, digital cultural probes, cognitive probes, technological probes).
While their material form as a package is frequentyl picked up, they are often used not to gather inspiration, but information. As such, they may complement or even substitute social science approaches or be interpreted with the help of social science methods. However, the probes’ sensitivity and provocative stance is rather neglected in the adaptation. In many cases, the initial subvertive attitude of the original probes has disappeared. Also the uncertainty that probes inherently carry is often regarded as a problem to be solved and not as an opportunity; at the same time, the idiosyncrasie of the probes is further neglected, pushing them towards a standartized method.
The authors finally argue that the probe adaptation reveals two fields of tension within HCI: The handling of interpretation (that tends towards positivist underpinnings) and to apply the method without the corresponding research attitude.
It is somehow interesting to see how a good part of the HCI community seems to struggle with the uncertainty of cultural probes. But even as a designer, you can stumble about your own assumptions and attitude. We tried to design our own probe package and some of the tasks we ended up with clearly aimed at information gathering. (Still it was way nicer to do than a questionnaire!) The more open-ended you want to be, the harder it gets to estimate the quality of your questions.
Also if you judge your probe package in terms of its usefulness, it is hard to predict how well it might do. Obviously the success of the probes depends heavily on the designer and his professional abilities. In the paper, the authors also mention how the part between the probe returns and the final design is often left out in the documentation. They acknowledge the difficulty to relate both, as the designer adds another interpretation layer. At this point, I am not sure to what extend the connection between inspirational material and design can be made explicit.