By Pol Allingham via SWNS

Artificial intelligence makes harsher judgments than humans on whether rules have been broken, a new study claims.

And scientists fear it might lead it to overstep the mark on punishments depending on what information it's been given.

When it is programmed based on the rules alone, with no human nuance, it is much harsher than when it's based on our responses.

The study by the Massachusetts Institute of Technology (MIT) looked at how AI would judge perceived breaches of a code.

They found the best data to program AI with is normative - where humans have stated whether or not a post breaks a certain rule.

However, many models are erroneously based on descriptive data where people label the factual features of a post and AI applies the code and decides whether there’s a breach.

For the study, dog images that could violate an apartment’s rule against aggressive breeds were collected and groups were asked to give normative and descriptive responses.

The descriptive team was not told about the overarching dog policy and was asked to indicate whether three factual features were presented in the image or text, such as whether the dog appears aggressive.

Their responses were used to craft judgments - if a user said the photo showed an aggressive dog, the policy was violated.

Meanwhile, the normative group was told about the rules on aggressive dogs and asked whether it had been violated by each image and why.

Participants were 20 percent more likely to declare a code breach if they used the descriptive method versus the normative one.

Had the descriptive dog data been used to program an AI model, the machine would skew towards heavy reprimands.

Scaling these inaccuracies up to the real world could have serious implications.

If a descriptive model is used to decide whether a person may re-offend, it may cast more severe judgments than a person and hand out higher bail amounts or longer criminal sentences.

As a result, the experts have called for greater data transparency, stating if researchers know how data is gathered then they can figure out how it can be used.

Professor Marzyeh Ghassemi, head of the Healthy ML Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL), said: “I think most artificial intelligence or machine-learning researchers assume that the human judgments in data and labels are biased, but this result is saying something worse.

“These models are not even reproducing already-biased human judgments because the data they’re being trained on has a flaw.

"Humans would label the features of images and text differently if they knew those features would be used for a judgment.

“This has huge ramifications for machine learning systems in human processes.

“The way to fix this is to transparently acknowledge that if we want to reproduce human judgment, we must only use data that were collected in that setting.

“Otherwise, we are going to end up with systems that are going to have extremely harsh moderations, much harsher than what humans would do.

“Humans would see nuance or make another distinction, whereas these models don’t.”

In the study published in Science Advances, three more datasets were tested.

The results ranged from being eight percent more likely to find a rule breach when using the descriptive responses to decide a dress code breach, to 20 percent for the aggressive dog images.

Lead author Aparna Balagopalan said: “While we didn’t explicitly test why this happens, one hypothesis is that maybe how people think about rule violations is different from how they think about descriptive data.

"Generally, normative decisions are more lenient.

“This shows that the data do really matter. It is important to match the training context to the deployment context if you are training models to detect if a rule has been violated."

Next, the team wants to investigate the effects of asking experts, such as lawyers and doctors, to complete the data entry.

Originally published on talker.news, part of the BLOX Digital Content Exchange.