Wouldn’t it be great if machine learning could be applied toward improving comments and other conversations online? Big and small publishers alike, from NYT to the site you’re now reading, are spending significant resources to stop trolls from bombarding readers with toxic comments.

A new Google tech based on machine learning strives to automate the process of sorting through millions of comments, helping identify and flag abusive comments that undermine a civil exchange of ideas.

In partnership with Jigsaw, Google launched Perspective, an early-stage technology based on machine learning that can help identify toxic comments on the web. The official Perspective API allows publishers to use this technology for their websites.

Google explains how it works:

Perspective reviews comments and scores them based on how similar they are to comments people said were “toxic” or likely to make someone leave a conversation. To learn how to spot potentially toxic language, Perspective examined hundreds of thousands of comments that had been labeled by human reviewers.

Each time Perspective finds new examples of potentially toxic comments, or is provided with corrections from users, it can get better at scoring future comments.

After the system has identified toxic comments, publishers can flag them for their own moderators to review and include them in a conversation. Readers could sort comments by toxicity, too, in order to surface conversations that matter. The system could even let commenters see the potential toxicity of their comment as they write it.

You think trolling isn’t such a big problem?

Thing again—The New York Times has an entire team charged with reviewing an average of 11,000 comments every day. Due to the sheer manpower required to review the comments, the paper has comments on only about ten percent of its articles.

Google and the Times have worked together to train machine learning models so that the moderators can sort through comments more quickly. When Perspective launches publicly and many more publishers embrace it, the system will be exposed to more comments and develop a better understanding of what makes certain comments toxic.

“Our first model is designed to spot toxic language, but over the next year we’re keen to partner and deliver new models that work in languages other than English as well as models that can identify other perspectives, such as when comments are unsubstantial or off-topic,” said Google.

According to DataSociety, 72 percent of American Internet users have witnessed harassment online and nearly half have personally experienced it. Almost a third of respondents said they self-censor what they post online for fear of retribution. It’s estimated that online harassment has affected the lives of roughly 140 million people in the U.S., and many more elsewhere.

Source: Google

  • ixl

    No it wouldn’t be great you imbecile. Censorship of speech by machines is a step towards complete government mind control you cretin.

    “wouldn’t it be great” if bots could censor free speech where your feelings start? NO.

    In case you don’t censor this, I hope with the help of God or what you probably call a magical force, you open your eyes before you become a complete automaton.

    • Bill

      On a related note, they may not need to censor individual comments if they simply disable them entirely- case in point is the iDB story on “transgender bathrooms”…they aren’t allowing comments at all on that one.

      I identify as a mandatory commenter, and iDB is discriminating against me by not allowing me to comment on that story. Sad!

      • I’m seriously hoping you are being sarcastic here.

      • Bill

        Of course the last sentence was sarcasm, Sebastien, but I hope the overall point I intended to make was understood.

    • :D

      It’s only supposed to replicate manual censorship

  • burge

    And opening sentences like that just proves that it could be needed. Not liking what a website has put has resulted in name calling of the author of the post. You sir are the reason that websites are looking at options like this.

    • :D

      You should put that in as a direct reply to his comment you stupid idiot

      • burge

        Why was I talking directly at the person ?

      • :D

        Wasn’t your comment directed to ixl? You posted your comment as a new, separate one rather than a reply.

      • burge

        I know what I did.

    • ixl

      Name calling is considered free speech, even if in my case it was over the top or “tin foil”… Disagreeing with people is part of having a conversation. Debating with others is part of life and just because it hurts your feelings, deploying bots or programs looking for certain keywords to censor those who have opposing views is one step closer towards tyranny.

      Maybe you should try a tinfoil hat. Might open your eyes a bit.

      • burge

        Free speech is one thing but as I was Implying when it’s your website your can use what ever features you like to moderated it but when it’s not your site it’s not your decision on what’s acceptable.

      • BINGO!

        You are right on the money. People pull the free speech card without really understanding what it entails. Free speech means you can spray-paint all sorts of profanities on the wall of your house. However, you can’t legally do this on the walls of my house in the name of free speech. Same applies to a website.

      • GDD2010

        Hence vandalism laws…. but hurt feelings bc of words… there is no constitutional law that says you have a right to not be offended.

      • GDD2010

        I agree to that. 1A applies only to govt restrictions. Not private citizens to one another.
        On that note: if this applies here, social media, et al.,.. it applies everywhere… florists, bakeries, coffee shops… EVERYWHERE. It’s called Freedom of Association.

        Also, may I add; nowhere, in the constitution, do you have the right to NOT be offended. I don’t recall there being a “Warm and Fuzzy” clause.

  • Stoffsprenger ✓ᵛᵉʳᶦᶠᶦᵉᵈ

    What do you idiots have against trolling??

    We don’t want censorship, we don’t want tyranny, we want freedom of speech and freedom of making fun of stupid SJW’s!

    How do you call people who hate/don’t understand trolls?