Digging this. One thought I had is if we could we use this to kicksart 'we' again. By that I mean It feels like over the last few decades, we've moved to a "me" society from a 'we' one. More focus on getting what's ours and less awareness of how our behavior affects the people around us.
So maybe the interesting question is that while this is extrinsic in nature, ie not coming from our own value system, maybe these extrinsic incentives could actually help reinvigorate intrinsic ones?
Maybe a better hotel room or a discount. Some small recognition for being the kind of person other people actually want to deal with.
We begin by behaving a little differently because there's something in it for us, but as we say in our house, 'the way you do anything is the way you do anything.' and that behavior becomes habit. eventually perhaps 'this is how I get rewarded' starts becoming 'this is just how people should treat each other.'
100% agreed with each and every thought you have here!
In a world that is more "me" than ever and politeness is on the decline, this type of system — a system that could reward good conduct — would, ideally, elevate the common system (the "we") in how we all treat each other out there.
Agree with Annalise Killian. This WILL be manipulated and corrupted downstream regardless of intentions at conception, a surveillance and punishment system that will bend toward Right Thinking, Right Saying, and Right Doing. Think Chinese Social Scoring. There is no bright, big, beautiful tomorrow at the end of this. We’re just going to have to depend on our own assessments, individual by individual, interaction by interaction.
The "bends toward Right Thinking, Right Saying, Right Doing" worry is one I also thought deeply about. Society is littered with past failures where value systems aim to frame behaviors that, really, are control systems. Sci-fi, like that Black Mirror episode I mentioned above, is littered with dystopian versions of the negative impact of positive intentions.
With the design of this system, I deliberately focused on removing mechanisms and influence of things like cheerfulness, deference, agreeableness, friendliness, or cultural conformity. It's written so you can be angry, blunt, or in open disagreement and still score well.
But... whether that holds up under real-world pressure is a HUGE question.
Your last line is what I keep chewing on. Individual assessment, interaction by interaction, is how it should work. But it doesn't scale, and something is going to fill that gap.
Probably cameras and AI reading us without asking. (I hit on this a bit in my reply to Annalie below as well).
I'd rather that thing be human-scored and visible.
So... any of this change your thinking? New concerns? Would love the challenging to continue.
As a person who thinks I generally conduct myself well in dealings with others, the idea has merit … but it feels like surveillance creep. Most ideas start out with good intentions and shortly after it becomes distorted by greed, ulterior motives and manipulation. I don’t like rating people or being rated - it reminds me of the awful performance appraisal systems in companies that are deeply flawed and reward the squeakiest wheels, the narcissists, or the most politically astute and prejudice the quiet achievers, the collaborators, the team players.
I’m not a fan. This too will be gamed to advance inequality and class warfare.
Annalie, thank you for this! The performance appraisal comparison is the sharpest version of this objection I've heard, and it was highly top of mind for me. (My poor HR team at work had many conversations with me recently, where I was picking their brains on the pros/cons of human performance rating scales.)
For this idea, the difference is rooted in the context of the engagement: the person (company side employee) rating you (customer side) has no power over you (a gate agent can't affect your career), nothing about it touches your livelihood, employees can abstain, and, based on how the sharing mechanics (should!) work, a low score can never cost you the service you'd get anyway. It can only ever add something positive on top.
Does that solve the gaming problem?
No. You're right that something like this will be gamed, and, transparently, I don't have a clean answer yet.
So why did I keep pushing this idea forward?
We already live inside a reality of business surveillance today, it's just all transaction data. And businesses are quickly adding new layers to their surveillance of all of us: cameras at hotel check-in reading and grading customer/employee interactions, AI grading how we spoke to a support rep on the phone. AI systems make it easy for businesses to analyze and score each of us based on their emotion detection capabilities today.
The thought with my proposed "Conduct" idea is this:
I'd rather we get ahead of businesses individually, opaquely (even invisibly), measuring each of us by creating something that is consumer-driven/expected, human-scored, and transparent.
Or, stated another way, if we can make measuring human conduct something that requires humans-in-the-loop and limits the impact of future (what I see as inevitable) AI-powered surveillance states, then we keep the "power" of the system with the collective (warts and all).
Does any of that shift your read?
And please keep pushing and challenging this idea. It is exactly what I was hoping to see.
I can see you have given this a lot of thought but you will have to do a lot more thinking ....some comments...even a once off- if it has a certain colour- like the gate agent calling my friend a "security risk" ...these kinds of weaponised slurs are never "lost in the wash" - algorithms will flag them and elevate them. This is exactly how ICE operates, how police systems operate, how authoritarian states operate. So I think you are a bit naïve ....you are designing from a mindset of good...the real kicker here is who controls the input and the system, and the algorithms that attach weight to the language. The person being rated has neither agency, nor acess, transparency or recourse. So ...information and power assymetry puts the person being rated at an invisible and potentially permanent disadvantage.
In a reply below, I explain a bit more about the rating system and how there is no space in it for free-text, so things like "security risk" would never make it into a person's score -- and the algorithms would never try to decode a comment into something more than it is. Plus, the system has other weights-and-balances (like using hard business data as part of the score, plus time decay for scores).
But (!) these your thoughts have had me adding in a new line of specification that would be needed: abuse management + score challenging by consumers.
Dear Brent, Your example of the gate agent hit a particular nerve because a single biased gate agent at an airline can enter a comment in a system that would mean that for the rest of your life - every time you fly, you could be flagged as a security risk, held without explanation and searched and interrogated for no reason at all. Worse is- you have no recourse, there is no transparency and there is no way to establish an objective truth - not all employees are equal. ( this exact scenario happened to a friend of mine at a British Airways gate check where the gate agent claimed she was cheeky because she asked a valid but slightly challenging question)
We are already seeing how ICE agents manufacture « behavioral claims » about people. No, nothing in your arguments persuade me that, in this era or expanded surveillance, disproportionate power of tech titans controlling platforms, and information assymetry, that this is a good idea. In fact, the more I think about it in the context of what is already happening in our society, the more this idea gives me the creeps. I think you should invite extreme black hat thinking on all the ways this can be abused against innocent people by bad actors, before pursuing it . Even one life or career or future destroyed is too great a price.
Hi Annalie -- thank you for these reflections as well!
This is helping me see some of the messaging and framing and (!) system challenges this idea will face!
On the framing side, in my writing on this, I am now seeing that I haven't made one distinction in this proposal clear enough.
As someone who loves a storyline where an injustice is righted and equality hates realities where a misunderstanding leads to unintended consequences, I worked hard to ensure this concept avoids the concerns above.
Let's use your (very valid, very frustrating!) example:
What happened to your friend was a comment. Free text, a narrative about her character, was entered once and is permanent. That's exactly the failure I'm trying to design against.
In the spec for Conduct, there are two parts of a score — one is human (NCS), and one is based on system data.
The human aspect -- what a gate agent would fill out -- is answering the NCS (net conduct score) question.
NCS has no comment field. No description, no story about who someone is.
An angry gate agent's only move is a single tap on a four-point scale, and it lands in a pool with every other interaction.
One bad tap can't move a score much. Older ones age out over time. And it sits alongside hard records (the other half of the CQ score), like whether I paid, showed up, and made the flight. Because NCS, is only part of the formula and it has logic around it, a single human impression never stands on its own and ages out as time goes by.
To avoid that "free-text" challenge, my aim here is that a bounded question refusing to measure cheekiness becomes the alternative to it, rather than more of it.
Here is a page explaining NCS a bit more, including showing the question and potential responses the gate agent could select:
On the system side, though, you've named something I very much need t work through further — an "abuse-case" response track. This would cover black hat scenarios, bad actors, and more.
Because I share many of your same concerns, I've pushed deep on this with the spec, but (!) please keep pushing where/when/if the above still does not resonate.
These challenges are exactly what I was hoping would surface here!
This was a big part of my review of this idea. When I ran some adversarial AI agents against it, they quickly brought that up as well. In the piece above, I linked to this story — https://datasociety.net/wp-content/uploads/2020/10/Rosenblat_et_al-2017-Policy_amp_Internet.pdf — and, while not perfect (far, far from it) used it as a base for many of the specifications and ideas that came together into the details of Conduct as well. To make this work, it would need to avoid these issues with this idea!
Digging this. One thought I had is if we could we use this to kicksart 'we' again. By that I mean It feels like over the last few decades, we've moved to a "me" society from a 'we' one. More focus on getting what's ours and less awareness of how our behavior affects the people around us.
So maybe the interesting question is that while this is extrinsic in nature, ie not coming from our own value system, maybe these extrinsic incentives could actually help reinvigorate intrinsic ones?
Maybe a better hotel room or a discount. Some small recognition for being the kind of person other people actually want to deal with.
We begin by behaving a little differently because there's something in it for us, but as we say in our house, 'the way you do anything is the way you do anything.' and that behavior becomes habit. eventually perhaps 'this is how I get rewarded' starts becoming 'this is just how people should treat each other.'
100% agreed with each and every thought you have here!
In a world that is more "me" than ever and politeness is on the decline, this type of system — a system that could reward good conduct — would, ideally, elevate the common system (the "we") in how we all treat each other out there.
Thank you for digging into this, Joe!!
Agree with Annalise Killian. This WILL be manipulated and corrupted downstream regardless of intentions at conception, a surveillance and punishment system that will bend toward Right Thinking, Right Saying, and Right Doing. Think Chinese Social Scoring. There is no bright, big, beautiful tomorrow at the end of this. We’re just going to have to depend on our own assessments, individual by individual, interaction by interaction.
Russell! Thank you for digging into this with me!
The "bends toward Right Thinking, Right Saying, Right Doing" worry is one I also thought deeply about. Society is littered with past failures where value systems aim to frame behaviors that, really, are control systems. Sci-fi, like that Black Mirror episode I mentioned above, is littered with dystopian versions of the negative impact of positive intentions.
With the design of this system, I deliberately focused on removing mechanisms and influence of things like cheerfulness, deference, agreeableness, friendliness, or cultural conformity. It's written so you can be angry, blunt, or in open disagreement and still score well.
But... whether that holds up under real-world pressure is a HUGE question.
Your last line is what I keep chewing on. Individual assessment, interaction by interaction, is how it should work. But it doesn't scale, and something is going to fill that gap.
Probably cameras and AI reading us without asking. (I hit on this a bit in my reply to Annalie below as well).
I'd rather that thing be human-scored and visible.
So... any of this change your thinking? New concerns? Would love the challenging to continue.
Thank you!
As a person who thinks I generally conduct myself well in dealings with others, the idea has merit … but it feels like surveillance creep. Most ideas start out with good intentions and shortly after it becomes distorted by greed, ulterior motives and manipulation. I don’t like rating people or being rated - it reminds me of the awful performance appraisal systems in companies that are deeply flawed and reward the squeakiest wheels, the narcissists, or the most politically astute and prejudice the quiet achievers, the collaborators, the team players.
I’m not a fan. This too will be gamed to advance inequality and class warfare.
Annalie, thank you for this! The performance appraisal comparison is the sharpest version of this objection I've heard, and it was highly top of mind for me. (My poor HR team at work had many conversations with me recently, where I was picking their brains on the pros/cons of human performance rating scales.)
For this idea, the difference is rooted in the context of the engagement: the person (company side employee) rating you (customer side) has no power over you (a gate agent can't affect your career), nothing about it touches your livelihood, employees can abstain, and, based on how the sharing mechanics (should!) work, a low score can never cost you the service you'd get anyway. It can only ever add something positive on top.
Does that solve the gaming problem?
No. You're right that something like this will be gamed, and, transparently, I don't have a clean answer yet.
So why did I keep pushing this idea forward?
We already live inside a reality of business surveillance today, it's just all transaction data. And businesses are quickly adding new layers to their surveillance of all of us: cameras at hotel check-in reading and grading customer/employee interactions, AI grading how we spoke to a support rep on the phone. AI systems make it easy for businesses to analyze and score each of us based on their emotion detection capabilities today.
The thought with my proposed "Conduct" idea is this:
I'd rather we get ahead of businesses individually, opaquely (even invisibly), measuring each of us by creating something that is consumer-driven/expected, human-scored, and transparent.
Or, stated another way, if we can make measuring human conduct something that requires humans-in-the-loop and limits the impact of future (what I see as inevitable) AI-powered surveillance states, then we keep the "power" of the system with the collective (warts and all).
Does any of that shift your read?
And please keep pushing and challenging this idea. It is exactly what I was hoping to see.
Thank you again!
I can see you have given this a lot of thought but you will have to do a lot more thinking ....some comments...even a once off- if it has a certain colour- like the gate agent calling my friend a "security risk" ...these kinds of weaponised slurs are never "lost in the wash" - algorithms will flag them and elevate them. This is exactly how ICE operates, how police systems operate, how authoritarian states operate. So I think you are a bit naïve ....you are designing from a mindset of good...the real kicker here is who controls the input and the system, and the algorithms that attach weight to the language. The person being rated has neither agency, nor acess, transparency or recourse. So ...information and power assymetry puts the person being rated at an invisible and potentially permanent disadvantage.
Hi Annalie -- our comments crossed!
In a reply below, I explain a bit more about the rating system and how there is no space in it for free-text, so things like "security risk" would never make it into a person's score -- and the algorithms would never try to decode a comment into something more than it is. Plus, the system has other weights-and-balances (like using hard business data as part of the score, plus time decay for scores).
But (!) these your thoughts have had me adding in a new line of specification that would be needed: abuse management + score challenging by consumers.
Loving the challenging push you are bringing!
Dear Brent, Your example of the gate agent hit a particular nerve because a single biased gate agent at an airline can enter a comment in a system that would mean that for the rest of your life - every time you fly, you could be flagged as a security risk, held without explanation and searched and interrogated for no reason at all. Worse is- you have no recourse, there is no transparency and there is no way to establish an objective truth - not all employees are equal. ( this exact scenario happened to a friend of mine at a British Airways gate check where the gate agent claimed she was cheeky because she asked a valid but slightly challenging question)
We are already seeing how ICE agents manufacture « behavioral claims » about people. No, nothing in your arguments persuade me that, in this era or expanded surveillance, disproportionate power of tech titans controlling platforms, and information assymetry, that this is a good idea. In fact, the more I think about it in the context of what is already happening in our society, the more this idea gives me the creeps. I think you should invite extreme black hat thinking on all the ways this can be abused against innocent people by bad actors, before pursuing it . Even one life or career or future destroyed is too great a price.
Hi Annalie -- thank you for these reflections as well!
This is helping me see some of the messaging and framing and (!) system challenges this idea will face!
On the framing side, in my writing on this, I am now seeing that I haven't made one distinction in this proposal clear enough.
As someone who loves a storyline where an injustice is righted and equality hates realities where a misunderstanding leads to unintended consequences, I worked hard to ensure this concept avoids the concerns above.
Let's use your (very valid, very frustrating!) example:
What happened to your friend was a comment. Free text, a narrative about her character, was entered once and is permanent. That's exactly the failure I'm trying to design against.
In the spec for Conduct, there are two parts of a score — one is human (NCS), and one is based on system data.
The human aspect -- what a gate agent would fill out -- is answering the NCS (net conduct score) question.
NCS has no comment field. No description, no story about who someone is.
An angry gate agent's only move is a single tap on a four-point scale, and it lands in a pool with every other interaction.
One bad tap can't move a score much. Older ones age out over time. And it sits alongside hard records (the other half of the CQ score), like whether I paid, showed up, and made the flight. Because NCS, is only part of the formula and it has logic around it, a single human impression never stands on its own and ages out as time goes by.
To avoid that "free-text" challenge, my aim here is that a bounded question refusing to measure cheekiness becomes the alternative to it, rather than more of it.
Here is a page explaining NCS a bit more, including showing the question and potential responses the gate agent could select:
https://openconduct.org/netconductscore/
On the system side, though, you've named something I very much need t work through further — an "abuse-case" response track. This would cover black hat scenarios, bad actors, and more.
Because I share many of your same concerns, I've pushed deep on this with the spec, but (!) please keep pushing where/when/if the above still does not resonate.
These challenges are exactly what I was hoping would surface here!
Thank you again!
Hi Brent This case study on how ratings on Uber and Uber’s way of dealing with it created a human nightmare for drivers and a massive legal penalty for Uber might be informative for your design thinking https://www.perplexity.ai/page/dutch-regulator-fines-uber-eur-JZWPNgHCQHuQGu.qW24PnA
This was a big part of my review of this idea. When I ran some adversarial AI agents against it, they quickly brought that up as well. In the piece above, I linked to this story — https://datasociety.net/wp-content/uploads/2020/10/Rosenblat_et_al-2017-Policy_amp_Internet.pdf — and, while not perfect (far, far from it) used it as a base for many of the specifications and ideas that came together into the details of Conduct as well. To make this work, it would need to avoid these issues with this idea!