Why humans, not machines, make the hard calls on comments
Times Insider explains who we are and what we do, and provides a behind-the-scenes look at how our journalism comes together.
An abbreviation considered vulgar by New York Times standards also refers to perfectly innocent people aspiring to become bachelors of science. Here’s a small but telling reason why artificial intelligence can’t replace human judgment in Times journalism.
The software helps the Times track a deluge of 12,000 reader comments that pour in every day. Comments appear on 15% of all articles published online. Since 2017, The Times has relied on a program designed by Alphabet, Google’s parent company, which helps automate the review process by drawing on patterns from more than 16 million comment acceptances and rejections by Times moderators.
Despite how powerful the software is, it can’t tell the difference between gross and acceptable uses of the same word or phrase – like “BS”. Deeper questions also arise. Where to draw the line between criticizing and insulting, naming and insulting, extrapolating and digressing?
It is Erin Wright’s job to render verdicts on these questions of ethics and etiquette. She’s an editor in the Times’ Community Office, a small team dedicated to moderating comments.
Last month, Ms Wright allowed a reporter to observe her work through a queue of submitted comments responding to a profile of Jen Psaki, the Biden administration’s press secretary.
“We’ve had some criticism of her, and as long as it’s not name-calling or any kind of information that we know is untrue, criticism is fair game,” Ms Wright said.
By ensuring that the comments section reflects the Times’ civil tone, Ms. Wright and her colleagues help engage readers and give them a sense of involvement and ownership around the work of the newspaper.
The Times software automatically approves some comments, and moderators can simply delete anything posted in error. But the software is not allowed to summarily dismiss a comment.
Each rejected comment is “hand-rejected” by a human moderator, said Marcia Loughran, editor-in-chief of the community office. “We want to make sure we’re fair to readers.”
As Ms Wright read through the comments on Ms Psaki’s profile, she explained her moderation decisions, speaking slowly and enunciating every word. She wears long rectangular glasses, as if to ensure that she detects out of the corner of her eye the kind of minor subtleties that easily go unnoticed.
Ms Wright gives most comments a swipe of about 10 seconds and agrees with what she sees.
“We’re not going to dismiss someone just for criticizing our reporting,” she said of a valid comment.
In another submission, a reader used the Psaki profile to make a broader point about news media habits. “We like it when commentators extrapolate from our report,” Ms. Wright said.
One submission called Donald J. Trump the “liar in chief.”
To reject. “We don’t swear,” Ms Wright said. “We’ll have a wonderful commentary – it’s perfect in every way – but then we have name-calling that creeps in and we have to throw it out.”
Another comment started off uncontroversial, saying that Ms Psaki exemplifies “Gen X’s greatest strengths,” then continued, “Am I allowed to point out how easy it is for my eyes to watch her briefings?
The software indicated that the comment would be accepted. Mrs. Wright wasn’t sure.
“We don’t want to have too much of it in the comments, where it becomes a referendum on her appearance,” she said. “Yeah, it’s in the Styles section, but it’s about her doing her job. That doesn’t mean I’m going to reject it. »
Ms. Wright and Ms. Loughran have spoken. Did the article raise the subject of Ms. Psaki’s appearance? Ms. Wright observed that there was talk of being left-handed, but not much more. To reject.
Ms Wright moved on to other stories, including an advice column which discussed the theft of sandwiches from a work fridge. “We can let the commentators joke about some things,” Ms. Wright said, “but we don’t want to have such a suggestion of putting a laxative in a decoy sandwich.” To reject.
With characteristic tact, Ms. Wright indicated that she could not continue to tell a reporter every decision she made. It was time to focus on work.
“Our commenters love that we moderate,” she said. “They don’t have to worry about trolls or spam. But they don’t like to wait either.