Monday, October 08, 2007

There's jargon, and there's what I understand

Prompted by Espen, and because his name had been mentioned in different contexts, I sat down to read some of John Hopson's writings at Gamasutra. The most recent work was his "We are not listening", a featured article adressing game researchers from the position of the game industry.

After the keynote at DiGRA 2007, where Mark Prensky talked about: Keynote: "Escape from Planet Jargon: Making Research More Useful to Practitioners", this was kind of a in-the-moment-topic. The Prensky talk had been fairly unimpressive, and I knowing Hopson to be both intelligent and thoughtful, I hoped he would have a better take on the issue than Prensky. I am afraid to say, my hopes were sadly dashed. Hopson did however make it clear that he didn't mean to address ALL game resarchers, only the one targetting the game industry, which I guess is a mitigating circumstance. Still - well, here's my thoughts while reading "We're not listening".

Page 1: caveat
I’m not saying that academics have to care about what the industry thinks of them, but for those who do this is the best advice I can give on how to make sure the industry takes your work seriously. Secondly, I’m going to assume that you’ve already done your research and have the findings ready to go. This article is about the final mile, going from finished research to real implementation in a shipped game.

Hopson here assumes that it's the job of the researcher to present the material, to "sell" it. Others have professionals to do that, it's called marketing departments. So what he says here is basically: if the business is to care about your work, good doesn't matter, marketing does.

Page 1, rule 1
The researcher must lay out the entire impact of the idea, from the cost of implementing the proposal to the resulting changes in player experience and the metrics for measuring that impact. Getting players to identify with the main character is great, but researchers have to finish the rest of the sentence: “This will help players identify more strongly with the main character which will result in an improvement in measures of overall player satisfaction and an increase in total playing time.”

Hopson makes the assumption that the researcher is doing research in order to identify exactly the kind of questions which are important for game developers, namely, how to make a player play more. That is probably true for his research, at the Microsoft lab, but it's certainly not true for other types of research. Knowing that a player has a very strong identification with the main character can be a way to discuss time management in education and development, dramatic structures in critical studies, the impact of games on society in sociology. The researcher does not have the same agenda as the industry, unless he or she is paid by the industry. It also assumes that an academic who knows a lot about how gamers react to the game, is also an insider on the questions game developers ask, and are able to identify the important findings in their own material, which would be important to the game companies. This demands an inside knowledge which researchers who already work in the game industry, for the industry, has. The rest, as he himself points out, although in a roundabour fashion, don't have access.

Page 2, rule 2: here we go with the jargon thingy.
Academic writing is abnormal. I know that by the time you escape grad school the rolling cadences and ritualized forms of the journal article are graven on your very soul. But really, you might as well present your research in the form of an interpretive dance as hand a producer an article written for academic publication. Reading an academic article is an obscure and highly specialized job skill, one which most of your potential audience doesn’t have the time or desire to learn. It’s up to the researchers to make their work accessible to the audiences they want.

This is as uninformed coming from Hopson as it was coming from Prensky. Every specialised field is packed with specialised concepts. Have anybody ever tried to read a user manual? Yes, there are bad academic articles out there, where content is questionable and form is taken to the extreme. But Hopson also knows very well that a lot of important terms and structures are there for precision and clarification, not to ritualize the communication. Academic writing directed at other academics is not directed at specialists in some other field, or laymen, and to exclude the precise language developed to deal with complex terms would be irresponsible. So the game developers can't read it? It's why they hire academics in their research departments, people who know the language, have the connections, understand the field. In a game developing company it's Hopson's job to keep updated on the language, the structures and the references, and bring that knowledge back into his organisation.

Use examples from bestsellers. A good example from a popular game is more effective than a great example from something they’ve never heard of. Industry people often suffer from an “if-they’re-so-smart-, why-ain’t-they-rich” attitude towards smaller titles. Even if the small title is a perfect example of how the theory works, they’re going to be less likely to listen if they haven’t heard of the game ahead of time.

This basically says: It's all about making more money, not about making better games, so if a game is better but isn't a huge bestseller, the developers aren't interested. "Good = huge selling game, screw all other analysis." And with that I think game researchers can just pack up and go back to our departments, as nothing innovative will ever reach the ears of developers. I'll not even start bashing the other advice on that page. It's all about turning your thesis into a 5 line ad. Sure, if that's what you want to do, please listen to Hopson. If you had hoped to communicating a complex understanding or a new idea, well, your odds look very short.

Page 3, rule 3
Working in the games industry can be brutal, involving fast-paced schedules and eighty-hour work-weeks at times. The people listening to your talk already have a full workload. They’ve already been cutting features to make their production milestones, often features that represent some of their best ideas and strongest held beliefs about games. And then there’s this academic who’s never shipped a game standing up there telling them to rip out weeks of work in order to implement some pet theory. Give us a break!

Yes, I'll give you a break. After rule 2, I think we already agree that innovation, complexity and precision is a waste of time when talking to developers. They are all lab rats anyway, going by programmed responses, so trying to communicate new knowledge is a waste of time. It is a pretty bleak presentation of game developers though, but I guess we already knew it was a cut-throat business where the thoughtful die fast.

Page 3 rule 4:
However, to make that research useful to developers, it’s important to take the next step and give concrete examples of how classifying one’s players helps to make a better game. Ok, so we now know that 10% of our players are “Type 3a explorer-monkeys.” Now what? Does that mean 10% of our content should be exploration-focused?

This is about normative research vs other types of research. What Hopson says here is: Game developers want cookbooks, and they want to be told how to do things right. The problem is again that no researcher can know that something they have observed will make things better if implemented in a different setting. This is something only game researchers already involved in game development can have a chance to test out. John Hopson can say: "Before we finish this, let's try this approach instead, because I have read this analysis which says that..." and then Microsoft can run a test on that, and see if his theory actually works. Game researchers outside of companies don't have that kind of access or that kind of tools.

Second, he says: Do only one kind of research. Don't do explorative or descriptive research, do normative research, create rules for how things should be done. Basically: Do the kind of research we like, and we'll listen.

Page 4, rule 5 and 6:
The question of access also makes rule 5 and 6 useless to a game developer. There's no way a scholar can prove that a theory will work unless somebody builds the game where it's been implemented. To do that we'd have to do all the things that Hopson specifically says not to do, such as citing other research, referencing, quoting lesser-known games and looking at what has been done before. It's the only way an academic not working inside a company can do these things, and accordign to Hopson, it's not valid.

The same goes with asking a developer what they need. Let me point to rule 3: The developers don't have enough resources to tell a researcher what time of the day it is. Why should they answer more complex questions, which might force them to listen to a complex theory?

I know John Hopson means well with his advice, but it's advice from a scholar who since his Ph D has worked inside of a game development firm. It basically says two things:

1) To be able to talk to game developers, you need the kind of access you have when you are already working with game developers.

2) Game developers are narrowminded, only motivated by profit, and don't want to learn new things.

The first is useless advice to any academic who wants to get into the field. The second is very close to being insulting to his colleagues, the game developers. But most of all I wonder at the job of game researchers in the development firms. Aren't they the ones who are supposed to bridge this gap? Isn't that why they are hired, with all their education and their connections to the research field? Sure, I have enough skills and knowledge about marketing to dumb down - eh, I mean, clarify my writing for a target group when I need to. I don't, however, have the dual skills that somebody like John Hopson has a chance to develop. He and those of his ilk are the ones who can bring new and interesting research to the eye of the game developers community. They are the ones who may really make a difference.

4 comments:

Jan van der Crabben said...

Being a game designer and academic myself, I see where he is coming from. Hopson's points are indeed true, sadly. The games industry is not a very academic industry, people aren't interested in the finer points of how games affect people, or how they relate to society (etc). The games industry is purely interested in:

- what makes the game better
- what makes the game cheaper
- what makes more money

There are only a few companies that truly innovate in terms of game design (Lionhead comes to mind), but most companies aren't they create a hit game, and do the same game over and over again, improving graphics and adding some new features as they go along.

Here are my answers to your criticism to his article:
- It is indeed the academic's duty to market his own article. But, let's be honest, that's quite similar when we present our ideas to other academics.
- Hopson is pointing out that if you want to make research that has any impact on game development, there must be an incentive to developers to listen. That incentive is generally money. If you write an article and don't hope it'll change anything about game design, then you don't need to worry about what makes games (sell) better.
- Game designers don't want to read academic articles. Most game developers laugh at academia. Remember, one of the main rules of game design is: Keep it as simple as possible, but make it as interesting as possible. You can't say that of academic writing. If you want developers to read it, make it simple. Oh, and most developers don't have an academic at hand who "translates" for them...
- Last, but not least, if you're researching something, don't hesitate to contact the developer. Try to go past the PR department if possible (they'll ignore you). Game designers, AI programmers, and graphic artists are very willing to talk about their work.

Anyway, this comes from my perspective of being in both worlds.

Torill said...

Yes, I read what you are saying. I think the real problem is your point about how game designers laugh at academia. Match that with how academia skuffs at popular culture, and you have two groups who will just not talk.

I just happen to be one of those academics who don't believe the rest of the world is made out of idiots, and I don't believe game designers are all so stupid that they can't see how knowledge can make their work easier and better.

I guess that's my main weakness - I think the world is made up from people who should be able and willing to communicate, without too much dumbing down in either direction. Because the game designers really need to consider their own jargon too. Don't tell me their world isn't packed with it.

*sigh*

Jan van der Crabben said...

You're right there, the main problem is academics laughing at academia.

One of the problems with that, though, is that academia often doesn't deliver what the industry needs. I know that all applicants to our company who graduated from game design courses were utterly useless and had no clue about game design.

Now, in my opinion the right thing to do would be to call them up, and send designers, artists, and programmers up there for some guest lectures. That doesn't happen though. Games companies are usually short on staff, short on money, and above all short on time (which is why so many game releases get pushed back in date). Oh yes, and then there's the publisher, who demands that everything is justified in terms of profit.

Anyway, I stray from the point. I just read your article on gender in Neverwinter Nights (because I played it myself, as the opposite gender, and found that quite interesting). The article is really valuable, as it really examines what's going on.

If you want someone from the games industry (or even someone from Bioware) to pay attention to it, though, write another article, a shorter one, that simply details your research findings. THEN people will definitely read it.

Having said that, there are some people in games development who're interested in academia. Apart from myself I know of 2 others in this company... out of a hundred. Just to give you an idea...

*sigh*

Torill said...

thamis, I agree, Academics laughing at Academia is a problem ;)

As for using the Neverwinter Nights article as an example: Thank you for noting that it would be valuable, but what should I do about a short version of an article like that? What are the findings which might be important to show off? Where do I send it, to publish it so that a game developer might read it?

What I also tried to point out in the criticism of the Hopson article is that with game development being such a closed field, only those who have access to both worlds, such as you and John, have a chance to know exactly what is useful. The Neverwinter Nights article is a decent and thorough work of game analysis, done in a classic academic style. It has no big numbers (in all industries numbers are treated like magic), it has no interviews (no actual users speaking except from the despised academics) and it has no findings beyond what can be seen by a trained analytical mind with a wide range of specialised knowledge.

If the person I try to send the "findings" to don't happen to understand the value of rigid academic training, scholarly effort, good quotes and citations and systematic analysis, then that article has absolutely nothing to offer them. So, if the person I try to sell my knowledge to is already laughing at me the moment they see the "Ph D" attached to my title online, what I have to offer is useless to them, no matter how few short words I might be able to press the contents of that article down to.

As for the people being educated being useless - well, that's something both sides need to take responsibility for. I educate people for the media, and know how to design an education which creates useful people who can fill jobs. It's done in cooperation with a profession that's willing to think long-term about their investments.