Covered California worked with KPMG to deploy CiCi in 2018, a chatbot that went live in mid-October of that year, to guide residents through its Oct. 15 open enrollment. A KPMG team helped the department — reportedly, the nation’s largest health exchange — architect and implement the virtual assistant, and develop insight from data collected. In a four-month period during Covered California’s 2019 open enrollment, the bot answered more than 111,000 questions and used its AI to evolve responses, KPMG said in a client story. The agile development process also utilized Microsoft Cortana and Microsoft cloud, with the initial goal of providing password assistance, but soon implemented further change. The agency also ventured into robotic process automation (RPA) last year in a partnership with Kofax, in hopes of processing 40 percent of the 1 million documents it receives every year. Among the takeaways:
• Giving your bot a clearly defined focus can help reach that goal, Covered California Chief Information Officer Karen Ruiz told audience members during a recent panel discussion at the California Digital Government Summit in Sacramento. She cited the California Department of Motor Vehicles’ launch of its chatbot and a focus on assisting residents with the federally mandated Real ID. By contrast, CiCi, Covered California’s chatbot, was initially left freeform, which meant it was “only able to help answer the question in a very small percentage of the time,” Ruiz said.
• Increasing a chatbot’s profile can boost its usage. Covered California used Google Optimize to perform A/B testing on public engagement with CiCi — and discovered that direct engagement with website users dramatically increased their utilization. The agency showed half its visitors what Ruiz described as a “call-out” pop-up window saying “Hi, this is CiCi, how can I help you today?” and tracked a 240 percent increase in chatbot usage “just by adding that on.”
During the technology’s second year of operation, Ruiz said officials would like to integrate it with text recognition. They’re also planning to use CiCi to get word out about the state subsidy program that’s one of Gov. Gavin Newsom’s initiatives in open enrollment this year. The department also hopes to integrate CiCi so that users can move directly from the chatbot into live chat.
• Ruiz also recommended that agencies developing their own chatbots consider analytics while a project is still in its early stages. Her staff has found potential in Dashbot’s chatbot analytics platform; but regardless of a department’s sophistication level, the CIO recommended collaborating with data staff to ensure information like chatbot inputs and outputs are fully scrutinized. Staff training is also a must when standing up a chatbot, Ruiz said. Her agency uses Pluralsight, she said; but the underlying point is to simply find training for your developers.
• Covered California’s RPA work is also in its second year, and has found its highest volume of use in facilitating the electronic verification process for enrollees who seek financial assistance. The initiative began with an “unassisted RPA process,” in which it was hoped the technology would move around 400,000 documents a year through the system without human intervention. However, image quality and the fact that some residents still use a fax machine to send documents have resulted in a roughly 13 percent pass-through, the CIO said, and necessitated a move to an assisted RPA process.
“This is another thing you can look at, when you do RPA, it doesn’t have to be all unassisted,” Ruiz said. “You can have an assisted process and again pick out something small, something high-volume, something very repetitive that can help you be successful.”
