ChatGPT, the AI chatbot that has been making waves in the academic world for its impressive academic achievements, has finally met its match in an accounting class. Despite acing multiple tests, including the Wharton MBA, the bar exam, and various AP courses, the AI chatbot only managed a score of 47.4% in the accounting exam, failing to even get a D grade.
According to a recent study from Brigham Young University, ChatGPT was given a total of 25,181 classroom accounting exam questions, with an additional 2,268 textbook test bank questions fed to it by undergrad BYU students. The questions covered various accounting topics, including AIS, auditing, financial accounting, managerial accounting, and tax. These questions varied in difficulty and type, including true/false, multiple choice, short answer, and more.
Although ChatGPT did outperform students on 11.3% of questions, mainly on AIS and auditing, it did worse than human students on tax, financial, and managerial assessments. This could possibly be because ChatGPT is primarily made for language, whereas math is not its strongest suit. During testing, ChatGPT did not always recognize that it was performing mathematical operations and made nonsensical errors, such as adding two numbers in a subtraction problem or dividing numbers incorrectly.
The study involved 327 co-authors from 186 educational institutions in 14 countries, making it a truly global effort. Despite ChatGPT’s less-than-stellar performance in accounting, its achievements in other fields continue to be impressive. The study serves as a reminder that AI is not infallible and has its limitations, particularly when it comes to subjects that require mathematical expertise.
