The increasing use of AI in critical decision-making systems, demands fair and unbiased algorithms to ensure just outcomes. An important example is the COMPAS scoring system that aimed to predict the rates of recidivism in defendants. Propublica dug into the COMPAS system and found that black defendants were more unfairly treated than white defendants.

With the rise of LLMs, there is a concern that using LLMs for decision making systems could give rise to unintended consequences. I am not aware of any LLM-based decisioning systems, and thankfully the debate over biases in LLMs has so far been mostly confined to benign political controversies: e.g. the Google Gemini fiasco.

There are 3 common perspectives that debates around fairness in AI/ML take:

1. Biased training data will lead to biased model predictions: This argument is often used when discussing statistical classification models such as COMPAS, where that claim is that algorithms are not to blame and that they merely reflect the underlying training data. If we change the training data, the model behavior will also change. The counter-argument is that data will never be perfect and that we need to design systems that can compensate for problems in the data.
2. Humans are no better than machines: This argument is pointed out to highlight that the real goal of machine based decisioning is to avoid biases in human decision making. Humans are good at masking their true intentions and biases, and machines are better at providing consistent results for everyone. There are studies that show Judges hand out stricter sentences when they are hungrier during the day (e.g. closer to lunch time). Machines will never have that prooblem.
3. Bias is inevitable because it is in the eye of the beholder: I came across this argument more recently in the context of discussions around LLM chatbot outputs. Most open LLMs, such as ChatGPT, Google Gemini etc. have some base controls over what kind of content they produce and what they refuse to provide. The debate around these models is whether there should be any such censorship or if they should be left free. The argument here claims that no matter what kind of additional fine tuning is done to the models, there will always be someone that claims offense.

At this point, I’m digging into this space and I don’t have any strong opinions on these arguments one way or another. I found this to be a good way to summarize the common positions. There is a great deal of work done to provide mathematical frameworks around how to measure fairness and bias.I’ll write more about that in the coming days.

I want to end this with a framing that I found very useful: “The real challenge is how do we make algorithm systems support human values?“