Why keeping better data is the key to Australia's artificial intelligence revolution

In the beginning, futurist and fintech stalwart Brett King says, a person looking to purchase a car or a house would go to their bricks-and-mortar bank, sign the dotted line and enter into a medium- to longer-term financial agreement with the organisation. But fintech, now a $7 trillion industry worldwide, has caused a rapid upheaval of the core utilities of banks – customers are realising “that utility can be delivered far more efficiently through technology, with far lower operational costs and far less friction to customers”, King tells podcast Life and Debt.

In the Australian market, some 80% of the population is engaging with fintech in an average year, resulting in a fundamental rethink of what it means to go into debt – buy-now-pay-later services like Afterpay are now a more recognisable brand than Paypal, according to Roy Morgan research. Soon, King continues, the concept of debt as we once knew it will disappear altogether, “replaced by contextual money experiences where credit access is available to you, when and where you need it”.

But, he says, this new form of debt poses a major issue if emerging fintech players do not act responsibly: “the ease of which you can access debt in the future is potentially problematic in that if it’s so easy for people to get access to debt, people could take on more debt than they need, more debt than is wise for them”.

It comes as the Albanese government’s plan to eradicate the cashless debit card (CDC) – efforts have stalled after it was revealed swathes of the 17,000 total CDC holders were able to link the welfare card to buy-now-pay-later services like Afterpay.

This goes to the heart of the issue with AI-driven marketing, 2021 Young Ambassadors Project say in its latest report, AI-Driven Marketing in Financial Services. A bank employee may not approve a loan for people who are receiving financial support from the government, but when it comes to AI, “incomplete data can result in ineligible or vulnerable customers being targeted by advertisers for products that they would otherwise be ineligible for”.

The report delves into how artificial intelligence is reshaping the Australian financial services landscape and the “real social, economic and ethical implications” of the burgeoning practice. And AI can take many forms, it continues – generating personalised content for customers with specific offers relevant to them, using historical and real-time data to customise a web or app page based on a customer’s interests, sending a customer a targeted discount, or streamlining a customer’s purchase journey from the web to an in-person interaction.

But, the report asks, should marketing be used to target home loan advertising at people going through a divorce, and are AI systems nuanced enough to avoid reverting to harmful stereotypes about which socio-economic or racial groups the algorithm should target?

“Effective and responsible AI relies on having good quality data,” the report warns.

Last year, Afterpay introduced its merchant analytics platform, Afterpay iQ, with an AI system that crawled more than 156 million transactions to give merchants access to (and leverage of) customer analytics. Chair Elana Rubin stressed that data privacy was top of mind.

“This portal is designed to help merchants understand their performance with Afterpay. But consistent with all the privacy segments, as such the only data [the] merchant can see in the portal is their own, which has been de-identified and is only shown on an aggregated persona basis,” she said.

The controversial buy-now-pay-later giant also embraced the government's open banking policy which allows customers to securely provide their banking data to other companies, with Afterpay co-founder Anthony Eisen saying it will mean more rigorous checks on customers' capacity to repay debt.

But Afterpay frequently comes under scrutiny for what some call an irresponsible attitude to debt -- a 2020 reveiew from the Australian Securities and Investments Commission (ASIC) found around one in five BNPL users had skipped meals or cut back on essential purchases to meet their repayments.

Laws governing the rollout of AI in Australia are in their infancy – the Australian Human Rights Commission’s Human Rights and Technology Final report recommends a framework so emerging technology can be inclusive and accountable, with human rights safeguards, while the federal government has launched an AI Ethics Framework with principles to guide organisations.

So, it’s up to organisations to embrace ethical data collation and use – which can mean addressing issues like siloed databases that do not interact, a reliance on outed-dated datasets, missing customer flags (which might indicate financial hardship or youth), and determining who within an organisation is responsible for AI oversight.

And the careful consideration of data within AI systems is not only an ethical imperative – it makes good business sense too, according to Melinda Cilento, CEDA's chief executive.

"In the absence of better AI business and governance practices, AI adoption will continue to lag in Australian organisations with long-term consequences for innovation, productivity and international competitiveness."