Cell Phone Anxiety: Extended Edition

Yesterday I posted some research about cell phone anxiety. You may have noticed that while I referenced a science journal article I didn't talk much about the results. Indeed, the only results I listed were descriptive (basic counting that describes a behavior) and qualitative (as opposed to quantitative). This is because of some fundamental flaws with the data as described in the article. So for those of you interested in such things here is the extended review enumerating three big problems I had with the data:

1. Sampling Error. The author state that they began with 47 participants but 24 had dropped out or otherwise been disqualified before the start of the experiment.  The problem is we can't be sure if there was some other extraneous variable contributing to the high attrition rate. For example, if those 24 dropped out because the idea of being without their cell phone for 3 or 5 days was too threatening then the study missed the opportunity to measure their anxiety. Or maybe not. The thing is that we don't, and can't, know the  possible confound of the sample selection.
2. Median Split. The authors used a median split (M=92) to determine high- vs low- texting participants. The problem here is that someone with 91 texts per day is in one group while someone with 93 is in another. Sure the line has to be drawn somewhere, but when you have a small sample size, it is hard to distinguish the difference between groups with a median split, especially if data is clustered near the median (although we don't know if that was the case here). A better solution might be to use thirds so you have a high-, moderate, and low-texting groups, which might lead to more meaningful comparisons between high- and low-.
3. Low Sample Size. Of course this is the big one. With only 23 participants stretched across four groups there simply is not enough data for meaningful statistical analysis, not with humans anyway. When doing animal research your samples are so homogeneous (there is little individual difference between rats) that you can get away with small sample sizes. In fact to do regression the rule of thumb I'd always heard was that you'd need at least 60 per group. That might be hard to get for some researchers and there are statistical tricks that get used (like bootstrapping) to work around that number. But for this study, the sample is just way to small and, to be fair, the authors acknowledge this in the discussion.

That said the  lit review and theory in the paper were good. Some of the descriptive and qualitative results were interesting. So while statistically almost meaningless, I feel as though the paper is still a valuable contribution to the relatively barren landscape of research into cell phone restriction anxiety.

• Dorothy Skierkowski & Rebecca M. Wood (2012). To text or not to text? The importance of text messaging among college-aged youth Computers in Human Behavior, 28 (2), 744-756 DOI: 10.1016/j.chb.2011.11.023