CalCloud recently celebrated its one-year anniversary. The calendar turned quietly this summer, though — at least at first.
In fact, it was tough in that first year to get state leaders, workers and vendors to say much about California’s first foray into its enterprise private cloud. Perhaps it was because the project was still so new or that everyone involved was waiting for CalCloud to really take off. But that silence sparked speculation about the success of the platform and its immediate future.
Some answers finally came in August. While the Department of Technology still hadn’t released hard numbers as of press time as to how many agencies had joined the 20 or so believed to be using the service earlier this year, it’s clear it hadn’t attracted as many departments as the state had initially counted on to sign up for CalCloud. So to expand the vendor pool for cloud services and keep the state from having to potentially pay hundreds of thousands of dollars each month out of pocket to make up the shortfall, leaders have rolled out a new strategy aimed at turning the tide.
CalCloud’s rates have been reduced roughly 50 percent or more, depending on the configuration, in hopes of making CalCloud more competitive with other public cloud services already on the market. The state is boosting security so departments can feel more comfortable migrating sensitive information to the platform. And in a change of course, the state also is opening the field to more vendors so they can compete to host services, converting CalCloud from a private cloud into more of a hybrid model.
“What we’ve done is learn a lot from standing up the business in the first year and how to strengthen and reinforce CalCloud,” said Chris Cruz, chief deputy director of operations for the Department of Technology. “So we should see a huge spike in adoption based on the new strategy we have in place.”
Some industry insiders remain quietly critical of CalCloud’s original framework, arguing that plan should have more clearly defined ways for other vendors to get involved in providing cloud services and that its original rate structure was doomed for failure. Such concerns have contributed to a recent surge of support for newly upgraded mainframes.
CalCloud supporters argue that the contract included room for negotiation because the first year had to be about laying the groundwork, getting users accustomed to the idea and working out the kinks. And with a new vision in place as CalCloud moves into its second year, supporters hope to see California’s government embrace the platform, use it to save money, break down silos and broaden the state’s IT capabilities in a way that’s just not possible today with physical servers.
“I think there’s a lot of momentum, and I think it’s starting to pick up,” Department of State Hospitals CIO Jamie Mangrum said, estimating at least 50 to 70 percent of California’s service-based technology will be on the cloud in the next five to 10 years. “I really see cloud as an opportunity to do things we couldn’t do before.”
CalCloud Is Born
As early as a decade ago, states like Delaware, Utah and Michigan became early adopters of cloud technology as they shifted from in-house data centers to IT services delivered from a remote location through a network.
The federal government announced a “cloud first” policy in 2010, requiring its agencies to use the platform “whenever a secure, reliable and cost-effective cloud option exists.” That same year, California launched a cloud email solution in partnership with Microsoft.
While state CIO Carlos Ramos and other public-sector leaders continued expressing concerns about security and reliability of a larger cloud effort — especially the prospect of migrating systems in the public cloud – leaders such as Lt. Gov. Gavin Newsom started to criticize California for dragging its feet when it came to embracing the inevitable change. Many state workers were soon clamoring for cloud services, seeing how virtual systems were allowing their colleagues to do more with less — particularly in the wake of California’s budget crisis.
“I don’t have the staff or resources to manage on-premises technology,” Mangrum said. “We can only do it with the cloud.”
After lengthy planning and an eight-month procurement process, the Department of Technology officially unveiled CalCloud on July 24, 2014.
CalCloud is a private cloud offering a pool of IT infrastructure resources on a subscription basis. State agencies, local governments and education institutions can purchase bundled packages of Windows, Linux or AIX operating systems, storage, backup, security and disaster relief. And those customers can modify the system via an online, self-service portal.
While California was far from the first state to turn to the cloud, it might be the first to take its public-private approach. Unlike states such as Texas that have tried in the past to purely outsource their data centers, the CalCloud platform’s hardware is housed inside the state’s secure data centers in Rancho Cordova and Vacaville. However, CalCloud is operated by an outside vendor. That strategy was needed because the state didn’t have the budget to cover the infrastructure itself, according to Ron Hughes, who helped create Cal- Cloud while serving as Cruz’s predecessor. Hughes retired from state service one year ago and now runs his own consulting firm.
IBM won the $37 million CalCloud contract after a competitive bid process. The company now operates the platform in partnership with AT&T under a five-year contract, with a commitment to constantly update CalCloud and train state workers along the way so they can eventually manage the platform in-house.
IBM (which deferred comment to the state) is providing the CalCloud infrastructure at no upfront cost to California, making its money as tenant users are charged fees for the service. However, the contract, as originally written, does guarantee IBM payment for at least 500 virtual machines with within the first 12 months, equal to $645,436 each month or $7.7 million each year. And if the company doesn’t get that many subscriptions, the state would be on the hook to make up the difference.
More than 20 government customers had signed up as tenants when the system launched, the state touted. No details were provided as to how many virtual servers those users requested, though the state later identified the Department of Motor Vehicles, State Controller’s Office and city of Rohnert Park among early adopters.
Under original rates, virtual machines ranged from $680 to $6,302 per month. It’s a volume-based price structure, meaning the more subscribers CalCloud gets, the cheaper rates are for users. The contract also included a clause that said if they weren’t getting the subscribers they expected, the state and IBM would renegotiate.
The original rates weren’t intended to compete with public clouds, Hughes said, which are a different animal. “Obviously no one is ever going to be as cheap as Amazon,” he said. Instead, in evaluating the IBM contract, Hughes said the state compared what it would cost if the Department of Technology did the cloud entirely in-house. The IBM deal represented a significant savings there, Hughes said. But he added that the state always intended to try to get rates down from that original structure.
Navigating the First Year
The summer 2014 rollout of Cal- Cloud attracted national attention, with other states watching to see how California’s unique approach to cloud services would play out.
Things were quiet at first. By the start of this year, some struggles became apparent as leaders like Davood Ghods, former chief of the state’s Office of Technology Services, said in interviews that they needed to streamline CalCloud services and bring more subscribers onboard.
In March, during a joint legislative committee on CalCloud, more issues bubbled to the surface. Participants voiced concerns about the security and scalability of the state’s cloud, asking for an update on how the project was going. “We would love to see those statistics and those things revealed in a more transparent process,” said Andrea Deveau, executive director covering California for TechNet, a bipartisan advocacy group.
Then vendors pressed the Department of Technology to create an exemption process granting waivers to agencies that wanted to use alternatives to the state’s private cloud. The idea, Deveau said, was to aim for a contract that didn’t pick “winners and losers,” instead allowing agencies to choose the best fit and least costly cloud solution.
Ramos said that alternatives would be considered on an individual basis, but he would not create an official exemption process because CalCloud was meant to be all encompassing.
Assemblyman Rudy Salas, D-Bakersfield, chaired that March 11 hearing. He drafted a letter in April, asking Ramos to provide a list of agencies that have signed on to participate in CalCloud and commercial cloud services. He asked for an overview of how the state evaluates and addresses security risks, and he asked for an explanation of the exemption process for state agencies to consider cloud alternatives. “That leads to more competition, which leads to more innovation,” Salas told Techwire.
The Department of Technology has until Sept. 15 to respond to Salas’ request for information.
It makes sense to want to host sensitive data on government property, Mangrum acknowledged. But without the freedom to leverage software as a service and other off-the-shelf cloud technologies — as the Department of Hospitals already does for things like electronic record sharing — Mangrum said the procurement process could take longer than implementing some new systems.
“I really do see hybrid cloud as what we’re going to be doing,” Mangrum said. “That’s the only way I can really deliver new services to my clients.”
Cost, competition and security concerns aside, Mangrum said converting to cloud services hasn’t been a technology challenge. It’s been a “people and process” challenge.
“It’s kind of a scary proposition for many IT people,” said Mangrum. “They wonder, is it going to displace me and my role?”
It’s the same question they asked when departments moved from mainframes with terminals to servers with desktops. Or when the Internet took off, or email was outsourced.
The Department of State Hospitals is evaluating all of its legacy systems now to see which ones are candidates to migrate to CalCloud, Mangrum said. His is the only department he knows of that engaged a private contractor to help with such an evaluation, which should be complete in October.
Mangrum doesn’t foresee any workforce reduction as a result of a move to cloud-based computing. Instead, it’s a matter of transitioning from managing on-premises equipment to administering virtual technology. Once workers see that, Mangrum said, they should realize that tapping cloud technology will allow them to focus on higher-end work rather than running around fixing machines all the time.
“It’s come to a point where we have to ask why we even have physical servers,” Mangrum said. “That’s just one less thing I have to have a headache over.”
Many experienced IT workers still struggle with handing over control to a third-party vendor, Mangrum said. But as the “silver tsunami” of baby boomer retirements continues, he said, state technology positions will be increasingly staffed by younger workers who are more accustomed to looking for existing solutions before trying to build their own.
“I think initially there was a lot of hesitation,” said Mangrum. “I don’t see it as much now.”
What’s Next?
The Department of Technology heard the concerns from customers and vendors during that first year, Cruz said. That’s where CalCloud 2.0 comes in, which the department is rolling out to vendors and stakeholders through a series of forums.
Rates have been slashed, allowing CalCloud to hopefully take advantage of the economy of scale it set out to tap into through as many as 10,000 potential clients.
While CalCloud was originally slated to offer platform as a service, application as a service, IT services management as a service and virtual desktop infrastructure by summer 2015, the state is now looking to the vendor community to help ramp up those offerings through a program called CalCloud Services.
Under the revised strategy, Cruz said the state plans to act as a broker of platform and software cloud services available from vendors. His department is working with the Department of General Services to widen the vendor pool by using existing procurement vehicles, such as the Software License Program.
The state is also working with AT&T to develop a subscription-based Security Event Management service, Cruz said. That should help CalCloud meet FedRAMP and National Institute of Standards and Technology requirements, Cruz said, which should help more agencies feel confident shifting their servers to the virtual platform.
While he’s still waiting on the data he requested in April, Salas said he’s confident CalCloud is now on track. He’s also anxious to see how expanding the cloud might lead to new applications that can improve transparency and streamline services for citizens.
“It will make us more nimble and responsive,” Salas said. “I’m excited to see where it’s going to take us.”
Ironically the more mobile devices and apps are launched, the greater the need for a reliable and powerful engine to keep them running. That’s why banks, airlines and other big firms that process lots of transactions still rely on mainframe technology, said Hughes.
In a recent editorial, Claire Bailey, director of government solutions for Compuware and former CIO of Arkansas, pitched a “mainframe renaissance.” She recommended that CIOs more aggressively leverage the systems they’ve already invested in so heavily, arguing that mainframes remain more scalable, reliable and secure than cloud platforms.
As evidence of the ongoing need for mainframes, IBM in January debuted the z13. It’s being billed as “the most powerful and secure system ever built” and the first mainframe specifically designed to accommodate the booming mobile app economy.
The Franchise Tax Board also recently received approval for nearly $9 million to expand the capacity of its mainframe infrastructure. The upgrade is needed, spokeswoman Denise Azimi said, to support existing legacy systems and to expand to accommodate new taxpayers, increased security demands and changing legislation.
“Our goal moving forward is to modernize our technologies and infrastructure utilizing whatever statewide services are available to reduce cost, save resources and minimize redundancy to save taxpayer money,” said Azimi. That includes, she added, considering options like CalCloud.
Mangrum expects there will be some systems that will stay on mainframes. “There are going to be those few things that it’s always going to be cheaper to leave them where they are until the end of life,” Mangrum said. But those systems, he predicts, will be the exception and not the norm.
This story appeared in the September 2015 issue of Techwire magazine.
