All you non-social justice warrior “liberals” need to get hip to this and join hands with the small government folks on this issue if we are to keep this in check. This sort of massive hive-mind machine is a clear and present danger to liberty. Social credit scores could be used to oppress people on a grand scale.
What actions raise one’s score? What actions lower one’s score? At first it’ll be the behavioral economists determining these things, but with time the algorithms will do more and more. The economists are dangerous enough. But AI bots managing society? Think of a fusion between HAL from 2001 A Space Odyssey, and Obama’s economic advisor Cass Sunstein. That is a scary beast.
Social credit scoring (and the incredible implications of such scoring) is a bonkers thing to think about, but even still we need to think about it. And we need to get in gear now. At stake is our very humanity.
(From The Atlantic)
Perhaps spurred on by their distaste for everything implied by such liberality, the Chinese government has become convinced that a far greater degree of social control is both necessary and possible. It now has access to a set of tools for managing the complexity of contemporary life that it believes will deliver better, surer, and more reliable results than anything produced by the model of order from below.
Known by the anodyne name “social credit,” this system is designed to reach into every corner of existence both online and off. It monitors each individual’s consumer behavior, conduct on social networks, and real-world infractions like speeding tickets or quarrels with neighbors. Then it integrates them into a single, algorithmically determined “sincerity” score. Every Chinese citizen receives a literal, numeric index of their trustworthiness and virtue, and this index unlocks, well, everything. In principle, anyway, this one number will determine the opportunities citizens are offered, the freedoms they enjoy, and the privileges they are granted…
…But the system provides abundantly for sticks as well as carrots. Attend a “subversive” political meeting or religious service, for example, or frequent known haunts of vice, or do under-the-table business with an unregistered, informal enterprise, and the idea is that the network will know about it and respond by curtailing one’s privileges. The state wants its citizens to believe that there’s little point in trying to evade detection of such acts, especially when they are strongly correlated with suspicious sites, either by mobile-phone location data or by China’s extensive national network of facial-recognition-equipped surveillance cameras. Once detected, the system promises to pass judgment on the things a citizen is and is not permitted to do, buy, or access. And with no recourse in real time, no ability to appeal, and nowhere to turn for help.
The consequences are nontrivial. The scheme is fused to a repertory of existing techniques the Chinese state retains to guarantee control, including the “hukou” system of internal passports that restricts the free movement of individuals, right up to detention, incarceration, and execution.