The Latest Crappy Video Game Study

By Daniel Bigler :: Monday March 22, 2010

There’s a new video gaming research study making the news rounds that examines video gaming’s impact on boys’ test scores – and it’s unfortunately just plainly questionable as research.

The gist of the study, which is titled “Effects of Video-Game Ownership on Young Boys’ Academic and Behavioral Functioning” and was in the April 2010 issue of Psychological Science:

New research shows that young boys who own a video game system don’t do as well academically as their non-playing peers, suggesting that time spent playing video games is supplanting time spent on homework.

Study author Robert Weis, an associate professor of psychology at Denison University in Ohio, said that “we can never say with 100 percent certainty that it’s playing video games that causes kids to have delays or deficits in reading and writing performance, but … we can be pretty confident that it’s the game ownership and the amount of time they spend playing that causes these academic delays.”

The problem with this claim is that it’s just a flat-out falsified oversimplification of a more complex issue: it makes a broad (and fairly true) point – that kids who do things other than homework will get worse test scores when tested on that work – but it then lands the blame for that specifically on video gaming, when really any time-consuming activity could be a potential culprit based on their measures.

Amy Kraft brought the research to my attention first, calling it out as “Another flawed study with totally speculative results.” I agree, and even an initial examination of the research can draw out its shortcomings on many levels, beyond the grandly falsified oversimplification of the issue as a whole.

First, there’s no mention of the inherent (and in this case significant) limitations of research methodology: I’d encourage you to revisit this primer by Karen Sternheimer on how to evaluate studies about video games to get into that critical mindset, because once you are the research just starts unravelling. For instance, one particular concern is that we don’t get any explanation of the tests used to evaluate the children, or why we should care about them. How strong is the correlation, and can it really be used to make a reliable prediction about future behavior?

Perhaps equally appalling was the limited, negatively defined scope of the research question: Asking “Do video games negatively impact boys’ math and language test scores?” is a very different question from “What variables have an influence on boys’ math and language test scores, and to what extent do those variables matter?” In only examining this one negative impact of gaming, too, we don’t get any idea about whether in fact it could be a worthwhile trade-off: that is, gaming could offer children something that’s perhaps more valuable in their lives than test scores. A sense of context is vitally important, in this case – as video games and gaming must be positioned within the larger picture of children’s lives and their benefits as well as drawbacks must be considered as a whole. It seems like the study’s authors start to go this way, toward a more holistic examination of children’s lives and how they spend their time, but they clearly biff it halfway through.

Finally, it’s a far leap to do a study and then publicize your additional speculation about why something is the case if you didn’t research that.

“Video games could affect a child’s brain, particularly executive functioning, and it could compromise his or her ability to do well in certain academic activities,” Weis said. In children, executive functioning refers to such things as their ability to manage time and keep track of more than one thing at once.

If Weis didn’t study executive functioning and video games, he should shut up about it. Weis conducted a behavioral study that solely examined the impact of the gaming variable on test scores – he didn’t hook the kids up to neuro-scanning machines to examine their executive functioning and brain activity. In any case, Wil Wright would certainly have something to say about executive functioning and video games – and while the research just isn’t there to say anything either way, I’d be more inclined to support Wright’s belief, that video games serve as a calibrating simulation that can offer a very positive outcome for executive functioning.

Ironically, an undergraduate student who worked with Dr. Weis on the study seems to be far more conscientious of its shortcomings than he is:

Cerankosky emphasized that “there isn’t necessarily something inherent in video games that negatively affects kids; it’s an activity that detracts from time that could be spent on schoolwork.” “The kids in our study could have been reading Stephen Hawking, and we might have found similar results,” she noted.

To me, this is the real kicker. In effect, headlines are getting splashed all over about the damaging impact video games have on children’s test scores – yet, wait, kids could equally read a book and that could have a similarly damaging impact on these scores? Uhhh….

So basically, we could have thrown the entire study out and just asked the logical question: What is important for our children to spend time on in their lives? How useful is studying for tests, as compared to playing video games, or doing something else? I think that’s really what’s infuriating about this study: It’s cultural value-setting in the guise of “social science research.” (In this case, lifting up the value of testing and achievement in boys’ lives, but not considering the value other enriching activities.) Because the research is “Science!” we’re inclined to believe its importance – but what we should really be doing is considering whether our values really match up with the study’s values.

In the end, I suppose we shouldn’t be surprised by this kind of thing, though. We’re dying to look for a nice and simple scapegoat to pin our fears about our children onto, and video games have historically been just that. Come up with a botched research design, throw in a little speculation – and voila, now we have “research” to back up the scapegoating too.

Tagged under: social science research, video games :: #