At the California Department of Developmental Services (DDS), our mission is to ensure that Californians with developmental disabilities have the opportunity to lead independent, productive lives in their community of choice. CA DDS oversees the coordination and delivery of services to over 350,000 individuals who have cerebral palsy, intellectual disabilities, autism, epilepsy, and related conditions through a network of 21 regional centers and state-operated facilities.
A mission this important – and at this scale – is incredibly complex. So many stakeholders have to be in sync to optimize service delivery, and as we assessed our operations back in 2018, we realized we weren’t maximizing the value of our data. In order to be more proactive and anticipate the needs of the citizens we serve, our talented personnel needed real-time insight into underserved populations, equity and disparity of quality and cost across demographics, trend analyses to facilitate forecasting and budgeting, and easier access to data insights across organizations.
The problem was our data was everywhere. It was siloed across our 21 regional centers. Multiple databases existed, from system-wide applications to in-house custom apps, and some of these databases were duplicative. Sometimes key information wasn’t documented anywhere and would leave the organization when talented and experienced personnel retired. We knew we needed a different approach in order to achieve a more proactive service delivery posture.
The vision was to bring all of this critical data together for analysis and empower CA DDS personnel with self-service tools to access the information and insight they need, when they need it. This is obviously easier said than done. We were faced with a decreasing budget and a severe lack of transparency for our data assets. And all of this was happening against the backdrop of mandates to move offices and shed at least 70 percent of our computer infrastructure and migrate to the cloud by June 2021.
Given our big goals and some equally big challenges, the cloud was the only technology vehicle agile enough to facilitate such a decisive move, and we needed to do so while simultaneously running day-to-day operations. Within this cloud environment we needed to establish a modern data warehouse infrastructure to optimize information dissemination and real-time data streaming to our stakeholders.
To execute this strategy, we needed a solution capable of supporting a diverse subset of data platforms. Each of our 21 regional centers has an IBM AS400 running a variety of applications, and we also have a mainframe and Microsoft SQL Server in various flavors to support. We began working with Qlik on a data capture and data warehousing proof of concept in partnership with Snowflake and its Cloud Data Warehouse Platform. The proof of concept met all of our data format requirements, outlined success metrics and identified gaps. With these tools in place, backed by the power of the AWS cloud, we had the right team to implement the self-service vision.
After we acquired the technology new priorities came to light. Not only did we need to get the data to Snowflake, but we also needed to make it easy to search and distribute to our diverse set of data consumers using different tools. As part of the Qlik Data Integration Suite, we were able to establish a full data governance catalog with data lineage, active business metadata, and an Amazon Store-like shopping experience that makes data easy to find and use. We cataloged all the Snowflake data coming from all our diverse databases in less than 3 months, and now we have a living catalog that will continue to grow as we grow our abilities to serve our data customers.
The results have been transformative. We have reduced our computer infrastructure footprint by 78 percent. Personnel are running their own reports in real-time rather than asking the IT shop to run custom reports – an increase of approximately 20 percent with a goal of 50 percent by the end of the year. We’ve established new ongoing reports to leadership and will continue to see an increase in users and content as our data analytics maturity level grows. We’ve also automated approximately 30 percent of our data ingestion and transformation processes and our service delivery metrics, and ticket turnaround time, data availability and cost per unit of work have all improved dramatically.
