The tryst with trust when it comes to AI is an ongoing and much-talked-about topic of discussion for corporations across the world. While they spend big bucks and time to regulate AI, a trend for Explainable AI has emerged.
According to Reuters–Amazon.com Inc’s AMZN.O machine-learning specialists uncovered a big problem: their new recruiting engine did not like women. The team has been building computer programs since 2014 to review job applicants’ resumes with the aim of mechanizing the search for top talent. But by 2015, the company realized its new system was not rating candidates for software developer jobs and other technical posts in a gender-neutral way. That is because Amazon’s computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period.
In order to trust the decisions of the AI systems, humans must be able to fully understand how decisions are being made. The lack of explainability and trust will be a hindrance to fully trust AI systems. This is known as Explainable AI (XAI).
Explainable AI: Decoding the Black Box
An emerging field, Explainable AI (XAI) addresses concerns about how black box decisions of AI systems are made. It inspects and tries to understand the steps and models involved in making decisions.
XAI answers some hot questions such as:
- Why did the AI system make a specific prediction or decision?
- Why didn’t the AI system do something else?
- When did the AI system succeed and when did it fail?
How can we gain explainability in AI systems?
One way to gain explainability would be to use machine learning algorithms that are inherently explainable. Simpler forms of ML such as decision trees, Bayesian classifiers, and other algorithms have certain amounts of traceability and transparency in their decision-making. These Ml forms do not sacrifice much on performance or accuracy and provide the visibility needed for critical AI systems.
However, there are complicated and powerful algorithms such as neural networks, ensemble methods including random forests, and other similar algorithms that sacrifice transparency and explainability for power, performance, and accuracy.
In 2019 US Defense Advanced Research Project Agency (DARPA) noticed the need to provide explainability for deep learning and other more complex algorithmic approaches and pursued it through various funded research initiatives.
DARPA describes AI explainability in three parts
- Prediction accuracy: models will explain how conclusions are reached to improve future decision-making.
- Decision understanding and trust from human users and operators.
- Inspection and traceability of actions undertaken by the AI systems.
What can we gain from interpretable and traceable ML models?
Fairplay: When we ensure our predictions are unbiased, we prevent discrimination against under-represented groups.
Trust: if people understand how our model reaches its decisions, it’s easier for them to trust it.
Traceability will help us to get into AI decision loops and empower us with the ability to stop or control its tasks whenever the need arises.
The way forward for XAI
In conclusion, the lack of explainability in AI systems remains a major hurdle for trust and adoption. Fortunately, the field of Explainable AI (XAI) is emerging to address this challenge. Explainable AI aims to demystify the “black box” nature of complex algorithms, allowing users to understand how AI arrives at its conclusions.
Emly Labs understands that interpretability shouldn’t come at the cost of technical complexity. Our approach empowers users to question AI models, understand the reasoning behind their outputs, and evaluate their performance. This user-centric approach increases transparency and fosters trust in AI results, allowing organizations to harness the full potential of AI with confidence.
“ Organizations also need to have governance over the operation of their AI systems. Oversight can be achieved through the creation of committees or bodies to regulate the use of AI. These bodies will oversee AI explanation models to prevent the rollout of incorrect systems. As AI becomes more profound in our lives, explainable AI becomes even more important.” Ron Schmelzer, Managing Partner & Principal Analyst at AI-Focused Research and Advisory firm.
Start Building Trustworthy AI: Sign up for your Free Trial at Emly Labs. www.emlylabs.com
For any queries, you can reach out to us at: [email protected]