650.823.3042
I certainly enjoyed reading Dick McCullough’s editorial and I am in total agreement with almost all of his points. Been there, done that, over and over and over again.
I do think there are a few exceptions to his taxonomy, most notably the annual MRA conference, where there is a very healthy blend of activities of both the selling and networking type and of the substantive learning type. But in general, he is right on the money.
Initially, I planned to respond to Dick’s article with a dissertation on the growth, progression, and death of research conferences in North America and their impact on the marketing research community. But that quickly got too long and far too boring. So I will just cut to the chase – what went wrong with the ART Forum?
In a nut shell, as Dick so bluntly stated – it has lost its relevance – at least its relevance to practicing methodologists. Before going on, I need to disclose that I have not attended an ART Forum for many years. But, I do study the conference brochure in detail each year and weigh the potential gain with the potential pain. The pain has been winning out for many years now. However, this last conference showed a lot more potential gain than previous ones, and I would have attended if not for a prior engagement.
The original ART Forum had a dual mission:
The original ART Forum was based on a concept extension of Rich Johnson’s Sawtooth Software Conference. And, I need to make sure the record is straight – Rich Johnson is also the “father” of the original ART Forum. It was his concept and his execution that we were attempting to duplicate to extend the coverage of subject matter beyond that which was traditionally being covered at the Sawtooth Conference at that time. It worked, for a while. All of us who developed the initial design of the ART Forum were dedicated attendees and participants in the Sawtooth Software Conference, and we were deeply appreciative of all the work and care that went into its planning and organization. Our motive was to supplement Sawtooth, not replace it.
Those original ART Forums were characterized by impassioned debates on methodological issues, irreverent intolerance for any “black box” presentations, and a demand for methodological transparency. You needed a tough skin and a quick mind to be a presenter, lest you be eaten alive by the audience.
By design, there were long breaks between sessions that allowed for continuing open-ended discussions after each presentation and informal discussions outside the presentation hall. And we all had a great time and learned immensely in the process. We also had the opportunity to get to better know our fellow methodologists – both practitioners and academics.
The academics that attended were almost exclusively well-published, tenured, dedicated teachers who kept their feet firmly planted in both the academic and the practitioner worlds – folks like Paul Green, Bill Dillon, Jordon Louviere, Joel Huber, Vijay Mahajan, Greg Allenby, and many others too numerous to mention. Their questions and observations were invaluable to the learning experience.
So what did go wrong? More importantly, why?
Has it gotten too academic as Dick suggested? Yes, I believe so. Having five presenters listed for one 30 minute presentation and a 10 minute Q&A is a sure sign that a bunch of people are “resume building.” But being “too academic” is an insufficient descriptor. I’ve been to a lot of academic conferences where I’ve learned a lot. I think the problem is more one of relevance. The conference has lost its focus on practical issues and applications – those that have relevance to the real world of marketing. The models and modeling techniques have become more important than the outcomes and their applicability to practical issues. (Again, the 2012 conference appeared to be an exception, based on the presentation titles and presenters.)
As Dick also implied, there is the generalizability issue – can the methods and procedures presented be generalized to a class of problems that we know already exist or are likely to exist in the environment? It seems to me that the generalizability issue has taken a back seat to the perceived elegance of the model or technique. Seldom do you see the presenter who has walked their pet technique through multiple scenarios and multiple samples.
In the early ART Forums we put a premium on finding presentations where a generally accepted model or technique didn’t work. We assumed that failures were of high educational value and provided an opportunity to reinforce the skepticism that is a necessary trait for our field. I fondly recall Steve Cohen standing in front of the audience stating that he “tortured the data until it confessed.”
But all this has to do with the presentations themselves, which is a relatively easy fix.
In my opinion, the real underlying problem is the management of the conference. This is where ART Forum has diverged significantly from the Sawtooth Conference. The Sawtooth conference was conceived and actively managed by Rich Johnson for many years. Subsequently, the Sawtooth staff remained true to Rich’s vision and management style.
In contrast, the ART Forum is managed by an ever-changing committee selected by an ever-changing Marketing Research Division Council and the AMA staff. The original vision has vanished and the continuity of focus seems to have vanished with it. Maybe David Bakken and David Lyon will respond to this observation since they have been on recent ART Forum committees and are long time attendees.
There seems to also be a scarcity of interaction time. There is no cocktail reception, the breaks between sessions have vanished, and there seems to be little time available for the often workplace-isolated methodologists to renew friendships and make new ones. I believe this was one of the most important features of the ART Forum and contributed immensely to the learning experience. But it seems to have disappeared. Even the unsanctioned floating poker game of those early years contributed greatly to building friendships and lasting experiences.
ART Forum seems to have lost its grove. Or maybe it’s just that Dick and I are getting older and more curmudgeon-like. Dick?
