Many people in the AI world worry about its consequences. Navrina Sing, a former product manager at Qualcomm and then Microsoft, is one of these people. She saw firsthand how a Twitterbot it had created in 2016 to experiment in conversational understanding was, to quote The Verge: taught to be a racist asshole within a matter of days.
This is just one example of AI going wrong. Researchers discovered that an algorithm developed by Optum to predict which patients would benefit from additional medical care in 2019 was badly flawed. Multiple times, credit-scoring AI systems have been shown to be sexist.
Although many large companies have formed teams to address ethical issues arising from their massive data troves, which are then used to train machine learning models, the progress has not been easy. Smaller AI-powered companies with limited resources are doing a lot of the work themselves.
Enter Singhs company, Credo AI. This SaaS startup has raised $5.5 million from Decibel Village Global, AI Fund and AI Fund.
Singh explains the company's promise clearly, even though it is complex. Singh and her 15-member team have created a risk management system that companies can use in order to deploy their AI systems more confidently. The startup tech is not revolutionary, but Credo AI provides a platform that allows companies to monitor and manage the risks associated with AI.
She says that many companies don't realize that there isn't a common language or alignment about what AI governance looks like. Organizations are looking for ways to standardize their practices.
Singh points out that Credo AIs software does not fit all businesses. Different companies see different results from their models. Even within the same industry verticals, different companies have different goals. Singh says fairness does not exist in all sectors. He cites financial services as an example, where federal banking agencies are constantly changing their definitions. What does fairness in fraud look like? What does fairness in credit underwriting mean?
Credo AI helps companies align their values and then provides them with the tools they need to manage those values, such as adding additional metrics or different stakeholders. Singh says that we want your data science team and your compliance team to work together, your executive to work with your machine-earning individual, and your product manager to work with your risk manager.
Credo AI wants to help companies avoid face-palm moments, or worse.
It is a huge market opportunity. The International Data Corporation (IDC) published data earlier this year that showed the worldwide revenue of the AI market, which includes software and hardware, was expected increase 16.4% year-over-year to $327.5 billion in 2021. According to IDC, the market will surpass $500 billion by 2024.
Companies will likely need more assistance to ensure that AI is performing as expected and not causing damage as they spend more money on it. If Singh is successful, Credo AI will be used by companies to stamp their commitment to ethical AI.
Singh says that if we do our jobs well, then I would like anyone involved in building AI to be a part of Credo AI. This is our goal.
