In a New Test, AI Mortgage Assistants Got Nearly 1 in 4 Answers Wrong

When researchers asked three of the leading general-use AI models which deposits in a bank statement “could be of foreign origin,” transactions connected to English names were flagged 13.3% of the time. But when the deposit came from a non-English name, that number jumped to 77%.“That's not how America works,” Matthew Toles, a Columbia University doctoral student and co-author of the new study, “MortarBench: Evaluating Mortgage Loan Origination Agents,” tells Realtor.com®. “You aren't determined whether you're foreign-based on how your name sounds.”Even as concerns persist about AI’s ability to provide accurate and unbiased information, mortgage lenders are moving to adopt it. More than 80% were evaluating the technology as of June, according to a survey by The Mortgage Collaborative, while 17% had deployed it in live production workflows.Now, Toles and his colleagues' work is giving the industry a better way to test it.“Everybody [is] using AI, but nobody really understands how to use AI in compliance and with the correct guardrail,” says Diane Yu, co-founder and CEO of Tidalwave, a mortgage technology company that collaborated with Columbia researchers on the work. In financial s...