Jacking in from the "Point-Five Percent Solution" Port:
Washington, DC -- Time magazine's credibility is hemorrhaging.
The magazine's recent "Cyberporn" cover story has ignited a fire storm of criticism owing to its overblown coverage of a statistically inconsequential study, written by a university undergraduate.
Time's story is being assailed as "reckless," "shoddy work" and an outright "fraud" by academics and civil liberties groups.
Martin Rimm, who as an electrical engineering major at Carnegie Mellon University took 18 months to complete the study, says 90% of the criticism "is junk."
The writer of the Time story, Philip Elmer-DeWitt, characterized the attacks as "a lot of rhetoric from a professional lobbyist and a professor who called it reckless and criminal before she had read" the study.
Besides the pejoritives used to question how academically rigorous the Rimm study is, Time's critics also are chaffing at the veil of secrecy that has surrounded the study.
Time, the Georgetown Law Review (where the study was formally published, despite the fact that it only deals with points of law inside footnotes) and ABC's Nightline, in a kind of media collusion, refused to let anyone outside those organizations do an independent review of the study before publication. Each cited secrecy and a prior arrangement with Rimm as the reason.
At least a week before publication, Time magazine was alerted to several potential problems in the study's methodology. "I rasied what I thought were several red flags," said Donna Hoffman, an associate professor of management at the Owen School at Vanderbilt, and one of the most respected researchers on Net access issues. "Those concerns were apparently ignored," she said.
Further, at least two legal experts, Mike Goodwin of the Electronic Frontier Foundation and Danny Weitzner of the Center for Democracy and Technology, were refused access to the study, despite being asked by Rimm to review the report's legal footnotes. Both declined to provide any legal analysis, issuing warnings that such analysis was impossible without seeing the footnotes in context.
Time magazine, aware of all this, ran its story without noting any of the criticism.
The .5 Percent Solution
One of the most egregious spin elements that Time used on the story was hyping Rimm's claim that 83.5% of all images on Use net are "pornographic."
That 83.5% figure has already been sized on in by some members of Congress looking to bludgeon the First Amendment by placing unconstitutional constraints on Internet content. This figure is likely to become a rallying cry of the First Amendment impaired; it has been trumpeted in at least one Senate floor speech.
Small problem: That figure -- and the study which ejaculated its results to a select media group under the cloak of secrecy -- is severely flawed, accordig to several academics and civil liberties groups that hav since obtained and analyzed a copy.
By Rimm's own admission, the 83.5% figure is derived from a seven day time slice of the postings to only 17 of some 32 Usenet groups which typically carry image files. Usenet is comprised of thousands of newsgroups, the vast majority of which are text based.
Further, Rimm's own figures show that his so-called "pornographic" images comprise merely ONE-HALF OF ONE PERCENT (.5) of all Internet traffic.
Time reporter, Philip Elmer-DeWitt did report this fact. Sort of. But readers of the Tie story have to wade nearly 1,000 words into the story before stumbling across this passage: "As the Carnegie Mellon study is careful to point out, pornographic image files... represent only about 3 percent of all the messages on the Usenet newsgroups, while the Usenet itself represents only 11.5 percent of the traffic on the Internet."
DeWitt would later claim during an online discussion on the WELL that he didn't finish the math, citing the .5% figure, because readers tend to get lost when more than two figures are cranked into a paragraph. (See, Time takes care of you!)
Cooking the Books
To juice the coverage, Time also cited that the study had "surveyed 917,410 sexually explicit pictures, descriptions, short stories and film clips." These files, however, were dredged up from adult BBS systems, not Internet newsgroups, a point that is not entirely clear when reading the article.
The 917k figure is further msleading because even Rimm admits in his paer that he winnowed out so many files that his analysis is based on merely 294,114 files. And that STILL doesn't tell the whole story.
To analyze such a huge number of files, by visually verifying that something called "Naked Bitch with Mardi Gras Beads" is actually a woman and not a hoaxed picture of a female dog (which actualy happened), would have taken years. Instead, Rimm's analysis is based overwhelmingly on file *descriptions* only, not actual viewing, using an artificial intelligence program.
Yet a reader of Time's cover story gets none of this analysis.
Walking Back the Cat
How did a major magazine like Time get roped into reporting as "exhaustive" such an apparently flawed document? It was likely a combination of several factors, including errors in judgment, fatigue and the need to scoop the competition on a hot button issue of the day.
The intelligence community often debriefs its operations through an exercise called "walking back the cat." During this exercise, the major players are gathered and the mission is examined in detail.
While not all the information surrounding the events that led up to the Time cover story are known, let's walk back the cat on what we do know:
Early 1994:
Rimm assembles his "research team" to begin trolling some 68 adult BBSs. His team is instructed to try and obtain as much as possible data on the BBS customers through a kind of "social engineering."
Dispatch interviewed 15 major adult BBS operators to ask about their participation with Rimm. None of them remember ever having spoken to Rimm or a member of his research team about the study.
Dispatch asked Rimm: "Did your team go uncover, as it were, when getting permission from these [BBS operators] to use their information?" He replied only: "Discrete, ain't we?"
When asked how he was able to obtain detailed customer profiles from usually skeptical operators of adult BBSs he says: "If you were a pornographer, and you don't have fancy computers or Ph.D. statisticians to assist you, wouldn't you be just a wee bit curious to see how you could adjust your inventories to better serve your clientele? Wouldn't you want to know that maybe you should decrease the number of oral sex images and increase the number of bondage images? Wouldn't you want someone to analyze your logfiles to better serve the tastes of each of your customers?
October 1994:
Eight months before the "exclusive first look" that Tie touts about its story on Rimm's findings, "people involved in the study were pitching it to the media," reports Michael C. Berch, editor of INFOBAHN magazine, in a posting to the alt.internet.media-coverage newsgroup.
Berch said he took a flyer on the story because he had "other coverage of Internet erotica" in the works.
Rimm says he has no knowledge of the exclusive offered to Infobahn or any other publication before shopping it to Time.
During this time, Rimm also shops a draft of his study to the CMU administration, according to a Time magazine report last year. Shocked at the findings, the school scurries to implement a full scale censorship of alt.sex groups from the school's Usenet feed.
November 1994:
All hell breaks loose. Word gets out that Carnegie Mellon University has decided to make public its policy to censor all Alt.Sex newsgroups from flowing into its computers.
The ensuing turmoil surrounding the CMU decision draws media attention and Time is there.
Time reporter DeWitt hooks up with Rimm and using sparse stats drawn from the Rimm paper, he writes in the November 21, 1994 a story headlined "Censoring Cyberspace."
In the story he refers to Rimm as only a "research associate." DeWitt's story says the CMU administration acted on a draft of Rimm's study "about to be released." In actually, the study doesn't see the light of day until some seven months later and only then under a secrecy agreement between Time and Georgetown Law Review.
DeWitt writes in that November article that Rimm has "put together a picture collection that rivaled Bob Guccione's (917,410 in all)."
In reality, Rimm had few, if any, actual images. The 917k figure then, as now, refers only to descriptions of images. And when the data was finally washed, only some 214k of those image *descriptions* were valid.
Rimm finally finds a place to publish: The Georgetown Law Review. But he cuts a deal first: No one -- absolutely no one -- outside of the law review's immediate staff is allowed to read the full study.
David G. Post, a visiting associate professor of law at the Georgetown University Law Center is approached "to help several of the student editors with questions that they had arising out of the study," he writes in a "Preliminary Discussion of Methodological Peculiarities in the Rimm Study of Pornography on the 'Information Superhighway,'" distributed after the Time article runs.
But when Post, who says he has "research interests in this area," asks to be shown a copy of the study before advising the students, he too is rebuffed. "[T]hey were unable to do so because of a secrecy arrangement they had made with Mr. Rimm," he writes in his preliminary discussion.
Post also writes: "One would have, perhaps, more confidence in the results of the Rimm study had it been subjected to more vigorous peer review."
Law review journals, however, unlike rigorous scientific journals, are not routinely peer reviewed.
But this study and it purported results were anything but "routine." The potential magnitude of the study, which was not lost on Rimm -- he'd already seen the white bread Administration at CMU rush to trample the First Amendment after reading an early draft -- should have been enough for the Georgetown Law Review, not to mention the editors at Time, to *demand* outside review and Rimm be damned.
Hoffman readily acknowledges that law reviews aren't subject to peer reviews. (Note: Maybe this is why the majority of lawyers can't write their way past a moderately bright 14-year-old.) However, she says quite bluntly and correctly: "A study lik this belongs in a peer reviewed journal if it's going to be used to impact public policies and stimulate public debate on an important societal issue."
June 1995:
Mike Godwin, online council for the Electronic Frontier Foundation and Daniel Weitzner, deputy director of the Center for Democracy and Technology, have, at separate times, been asked by Rimm to review the legal footnotes for accuracy.
Godwin and Weitzner say the task is impossible without seeing the full report. They are denied that request.
Weitzner fires off several critical concerns he has about the footnotes anyway, noting that any kind of real analysis is impossible.
Rimm later "thanks" Weitzner for his "participation," even though Weitzner clearly had denied the review request.
June 8-18, 1995:
A copy of the study arrives at Time magazine where it sits idle. DeWit is up to his journalistic elbows trying to edit a major Time cover story on Estrogen. The story is complex and riding herd on it stresses DeWitt.
The good news: word filters down to him that his promotion, which has "been in the works for some time," he says, will be official in a couple of weeks, about the time of his vacation and right after he puts another major cover story bed: the flash point "Cyberporn" story."
Four Time correspondents are assigned to the story to help with the research.
Time passes quickly. Rimm's story, like a forest fire, begins to create its own atmosphere, that rarefied air of "The "Exclusive." In the unrelenting, brutalizing competition of the newsweeklies, the scoop is the ace in the hole.
The Time editors were convinced the Rimm study was their Ace. Somebody should have told them it was dealt from the bottom of the deck.
So now DeWitt begins pushing for his story, citing its exclusive nature. But DeWitt is negotiating the story's placement based on character flaw: He was already sold on the story, having used it back in November during the CMU censorship dust up. The story held up then, it should hold up on the cover. Besides, if it were good enough for the Georgetown Law Review, it was good enough for Time.
And DeWitt plays the law review card readily, aditting: "If [Georgetown] hadn't accepted [Rimm's study] for publication, we wouldn't have done our story."
At this point, DeWitt has too much invested in the story. Somehow he ignores the lingering doubts and presses forward with the writing. Later, on the WELL he will admit to personally be "pulling for" the validity of Rimm's study.
Meanwhile, one of his reporters, Hannah Bloch, is picking up some bad vibes from Professor Hoffman.
Hoffman and her husband/research partner, Tom Novak, have tagged-teamed some of the Net's trickiest usage based problems, developing some of the first quantitative models for accurate WEB "traffic accounting." And even from reading the abstract of Rimm's study, Hoffman smells sloppy research. "This is a nice example of bad research," she says.
After the Bloch-Hoffman telephone tag review finally ends, Hoffman says she still feels like Bloch "didn't get it." Hoffman E-mails DeWitt directly with her concerns.
When Hoffman asks DeWitt to see a copy of the study, he balks, citing the secrecy arrangement with Rimm. Hoffman lays out her concerns about Rimm's methodology and E-mails them to DeWitt. Among those concerns, Hoffman notes that a study of such reported significance should have been subject to some kind of peer review.
But DeWitt blows off Hoffman's concerns, not because of flawed logic or some perceived hidden agenda. Nope, DeWitt decides to dismiss Hoffman out of hand when he discovers -- quite suddenly -- that law review journals are rarely peer reviewed. This somehow significantly lowers the credibility factor of Hoffman's concerns in DeWitt's mind and for whatever reason, he ignores them.
The concerns are never raised. Not in editorial meetings, not in the text of the story. Nowhere. A Time reader is lead to believe that the study was rigorous and without fault.
In truth, the story had been criticized on several levels and by several different people. The connection? None, save for their concern about sloppy research.
So DeWitt presses on. Don't let facts stand in the way... he has a story to write, a vacation to get ready for. This is his baby and he's under the gun to deliver.
June 19-23
With barely a chance to breathe after the work on Time's Estrogen cover story, as well as several other stories, DeWitt wades into the reports from his other correspondents.
He fields editorial questions from higher up. There are still gapping, mawing holes in the story. By end of the day Monday, the 19th, he knows he has to start writing come Tuesday morning. This is crunch time. There is no more slack in the schedule. Artwork has been commissioned. The cover slot secured. His vacation is looking better all the time....
Meanwhile, Time's public relations arm is cranking into high gear. They know they have a hot cover coming up. They want to get the most mileage out it they can. Where do they turn? Television.
They consult with Rimm. He's pitched the idea of giving the story to 20/20's Barbara Walters. Rejected. Too light weight. Larry King Live is suggested. Good talk hype, high visibility, but not a serious enough venue. Rejected. Conan and the Late Show were never considered.
Finally, the Time spin doctors decide on Ted Koppel and Nightline. "We thought Koppel would do a more balanced job," DeWitt said.
Time calls ABC. "It's an exclusive and it's yours if you want it." Nobody mentions the fact that ABC was the third choice...
Another secrecy deal is cut. Nightline can't give the study to anyone else either The article hits the stands on the 26th, but by that time DeWitt will be vacationing. The ABC producers decide to tape him Friday, the 23rd.
Thursday hits and DeWitt mets the 6 p.m. deadline. Researchers comb the story. Top editors read it, too. "Needs some work," they say and DeWitt cranks up the computer to satisfy his bosses. The issue is put to bed.
Friday, June 23rd -- It's Darkest Before the Dawn
At 22 hundred hours, 43 minutes, Jim Thomas uploads to the WELL, under a new topic residing inside the "media" conference, an urgent message being sent through Cyberspace by Voters Telecom Watch.
The VTW alert puts the Net on notice: Time is ready to publish on Monday a study of porn on the Net. The VTW alert acts like an early warning flare: "The catch is that no one even knows if the study's methods are valid, because no one is being allowed to read it due to an exclusive deal between Time and the institution that funded the study."
Saturday, June 24th -- Bad Moon Rising
Early in the morning Hoffman logs on to the WELL and jolts the media conference, calling the Rimm study "reckless research" and noting how difficult it is to discuss porn on the Net without throwing fuel on the fire.
DeWitt follows some five hours later with his own assessment of Hoffman's opening salvo. He says that Hoffman is right about fueling the fire. But he drops a bomb of his own: He wonders aloud how Hoffman can call the study reckless when she's never even read it.
However, he conveniently forgets to tell other WELL members that he denied several requests -- Hoffman's among them -- from people to see the study before they commented on the record. He also fails to mention that it was a secret agreement with Rimm that made any independent review of the study impossible.
This early exchange, in a topic called merely "Newsweeklies," set the stage for what would become a romp into "way new" journalism of the first degree.
Over the course of the next eight days, this topic on the WELL would ignite a grassroots investigative team held together with no particular agenda other than seeing all the facts about the Time story vetted.
Steven Levy, a writer for Newsweek, weighs in. He's also written something about Porn and the Net for his publication that will run on Monday. The Rimm study gets a single, dubious paragraph.
Levy would have missed the Rimm reference altogether, but Georgetown law professor David Post tips him to the fact that Time is running the story.
Levy scrambles himself to get a copy of the study. He gets shutout. The law review won't give him a copy, citing the secrecy arrangement with Rimm.
Levy tries to find out what Rimm or the Law Review are getting in return for all their secrecy. Each tells Levy to talk to the other. He gets no answer.
In the WELL conference he voices his concern about such secrecy arrangements, wondering if it was trade off for assurances that the story would get a cover.
What Levy doesn't know is that in the coming days, the mere mention of Rimm's study in his story causes the blood pressure to rise within the Time top editorial staff. Gone was their "exclusive," or so they thought, despite the fact that Levy had virtually no detailed knowledge of the Rimm paper. DeWitt will be made to answer for "the leak" when Time does a postmortem on the story.
DeWitt barks back at Levy, defending the secret agreement with Rimm. He says he's "much more comfortable" with that arrangement than with some that Newsweek has made made with top business executives. He drops Levy a compliment, calling him "one of the best," and then backhands him: "It's not my fault he works for the magazine that secured exclusive right to Hitler's 'diaries.'"
He later takes back the remark about the Hitler Diaries, admitting it was "a low blow," explaining he found it a bit ironic for Newsweek to be claiming the high moral ground.
A critical mass begins to form; WELLites begin to limber up, taking free shots at Time and DeWitt... and all before anyone has seen the story.
EFF's Godwin weighs in, the voice of reason: "Let's hold off criticizing Time until we see what the story looks like." And yet, in the cming days, it will be Godwin that rises up as judge, jury and executioner of DeWitt and Time.
The fun has just begun and DeWitt is about to step into a virtual home only the Menendez brothers could love.
June 25, 7:36 p.m. -- The Feeding Begins
"The Time article is available on America Online right now," is the single line message posted to Newsweeklies on the WELL.
A feeding frenzy is about to take place and over the course of the next several days the topic will resemble a great roiling, shark infested pool. Time and DeWitt are the chum.
The events that shake out over the next few days, while localized on the WELL, are significant. First, the article's principal author has his virtual "home base" here. Second, the WELL becomes the focal point of the most intensive and extensive critiques of the Rimm study, a factor that proves invaluable, considering that Rimm was successful in bypassing this traditional academic gauntlet.
The early reviews of the Time story are horrendous. Someone suggests that the phrase "Rimm Job" will be used to identify overhyped undergraduate studies that masquerade as major newsmagazine cover stories.
Monday June 26, O-Dark-Thirty
DeWitt logs and posts a comment at 2:38 a.m. That prompts John Seabrook of the New Yorker magazine to query nearly 3 hours later: "You're up early. Trouble sleeping?"
At 2:39 p.m. Godwin's life for the next eight days is defined by this posting: "Philip's story is an utter disaster, and it will damage the debate about thisissue because we will have to spend lots of time correcting misunderstandings that are directly attributable to the story."
Godwin proceeds to take huge, vicious chunks from the underbelly of ime article by attacking it's least defensible position: The infamous 83.5% figure.
Godwin will continue to feast at table of Time for days to come, at times posting several devastating comments in a row. He is a machine. He admits to "obsessing" on the issue, but "I'm obsessing over what is the truth," he tells Dispatch about midnight.
He is on the edge of a day too far gone to care about, at the brink of the next too dark to foretell.
He has been unrelenting in his strategic dismantling of DeWitt and the Rimm paper. Even his voice sounds tired. But all this takes its toll: DeWitt had been a friend. "I feel like something has died," he will say later. And to a large extent, something has.
The packaging of the story gets hammered as well. The shock artwork, which includes a damn near pornographic image in its own right -- what can only be described as a man fucking a computer terminal -- is outrageously sensationalistic. DeWitt even admits at one point that he agrees with views that the art is "over the top."
9:30 Monday Evening...
By now DeWitt and Time are bloody if not bowed. A crack in Time's story begins to surface.
DeWitt admits it himself, acknowledging that he "should have had a graph" in the story that referenced the advance criticism of the study that he knew about. "That was probably a screw up," writes on the WELL. He says he "couldn't risk" giving anyone, such as Hoffman, and advance copy of the study for fear it would "leak."
Tuesday June 27th -- The Plot Moistens
Virtually bleeding from a thousand cuts, DeWitt acknowledges that the pressure got to him while writing the story. In fact, he says that if he and his team had had more time and "more presence of mind" they would have called in an "outside expert" to review the study.
But "presence of mind" was apparently lacking. DeWitt admits that he had to go from editing one cover story to writing the next with only the weekend to rejuvenate. "Such is the life at a newsmagazine these days," he writes.
Jim Thomas surfs into a WEB site that i supposed to carry the Rimm study. What Thomas finds instead is a brief description of the study, a pointer to the law review article and a phone number were you can buy it -- not download it.
And then he points out a curious note contained on the page: "Current plans for pages include the Introductory text from this article and the conspiracies which have reached the ears of the researchers." But there's no other explanation.
Nightline runs its exclusive by arrangement segment. DeWitt has already been taped the previous Friday. Godwin goes head to head with Ralph Reed of the Christian Coalition.
Godwin becomes an instant hero: He jumps into first into the discussion and is able to play the "family values" card before Reed. But Reed is tossing out facts and figures as if he has somehow been given an advance copy of the so-secret study.
When Rimm is asked if Reed had some kind of advance peek at the study, Rimm says: "Ralphy never saw the fucking study."
Wednesday, June 28th
Hoffman appears back on the WELL after a two day absence. She is shocked: In the media topic alone there have been 250 new posts.
Hoffman announces that she and her husband/partner, having finally obtained a copy of the study, are beginning a systematic critique of the Rimm report.
Six days later the Hoffman/Novak report is complete, all 9,000 words of it. It turns out to be devastating.
Professor David Post, from the Georgetown University Law Center, crusises onto the Net with his own detailed critique of the Rimm study. Post deconstructs Rimm's report in the same manner as the Hoffman/Novak paper.
Thursday, June 29th
Hoffman discovers that the cryptic WEB page message alluding to "conspiracies" is aimed at her. On the WEB site, it seems Hoffman is being singled out for being a bit too vocal.
Hoffman fires off a nasty note to Rimm's faculty advisors at CMU. They answer quickly, apologizing for "conspriacy" language that "has no place in academic discourse," according to Marvin Sirbu, one of Rimm's advisors.
Rimm answers Hoffman, too. He apologizes for the WEB page, saying that the person who put it up had done so "accidentally."
The WEB page goes back to "normal."
Friday, June 30-Monday, July 3rd
There is not a minute's rest for DeWitt. He is continuously hounded whenever he goes online. All this is very tiring for DeWitt. Finally, after a long protracted battle on the WELL, DeWitt seems to be inching near defeat, at least on certain points.
David Kline, a freelance writer and contributor to Wired magazine, logs in and writes that DeWitt didn't conduct what he calls "journalistic due dilligence" by investigating the study throughally and by not mentioning that other experts raised several doubts.
Kline's message has rung the brass bell.
The next time DeWitt logs in, he cites Kline's message saying: "I think he's put his finger on precisely where I screwed up."
And yet, the story won't die. Going into Monday night (July 3), Rimm himself was preparing a detailed assault the Hoffman/Novak critique.
I asked for an advance copy... Rimm said it was secret until he was ready to announce it.
Why am I not surprised
Meeks (whew... finally) out...
On the first line of the message (no subject needed) type:
subscribe cwd-l
Back issues are archived on gopher and World-Wide Web servers at Liberty Hill Cyberwerks. These are accessible by typing:
gopher cyberwerks.com
and choosing "CyberWire Dispatch, by Brock N. Meeks" or by opening the URL
http://cyberwerks.com:70/1/cyberwire
from within your favorite Web browser.