Commentary: Defining Terms, Streamlining PAL, and a Tip of the Hat

This week I’d like to comment on four miscellaneous topics: a quick definition of “modern IT,” a comment on budgeting for modernization, an opinion on the California Department of Technology's Project Approval Lifecycle (PAL) process, and a shoutout. …

If you were to spin through the past blogs in this thread, you would find that a definition of modern IT emerges. Let me lay it out explicitly:
1. A modern system is cloud-native. Cloud-native means that the resulting application can scale up and down using only the cloud resources required in the instant.
2. A modern system is built according to the Twelve-Factor Application manifesto. 
3. A modern system is made using continuous integration tools and a modern development environment with automated testing that ensures a high-quality result.

I do not believe this is too controversial. Further, I think that CDT should establish a simple definition like this and enforce it as part of the PAL process. Every new system should be designed to these modern standards.

Most readers will be familiar with the meme around upgrading the plane while you are flying. IT modernization is like that:  You have to build the modern system incrementally while in flight. What's more, you have to start with a 747 and end up with a 777 … and this is very difficult. I think that it is nearly impossible. Better to build a new plane on the ground, test it, and then briefly land and swap planes. If you upgrade on the 747 airframe, you end up with an updated 747, not with a modern airframe. And there is one more consideration (to drive this plane metaphor further toward the absurd): If you take the cost of running the original 747 — the cost of fuel, maintenance, and staff — and expect to build a 777 without investment, you are bound to fail. You might cut back on long-term care and feeding or redirect a few staff, but there is just no way to get a 777 out of this approach.

Next, there have been articles critical of the CDT PAL process, items that suggest that the PAL process should be able to stop software project failures and cost overruns. I do not see it that way. I think that the PAL process asks a set of questions that need to be considered before starting any program. But the idea that if you get a proper start, you cannot fail is a misguided notion. Most projects fail during execution, and these failures cannot be caught up front.

Further, the long duration required by PAL is not all the CDT’s fault. I have had PAL processing stopped when departmental staff refused to consider the documentation because it was not grammatically correct, even though the entire paper had been through a grammar-checker and there were actually no grammar errors. I have had PAL proposals stopped by departmental enterprise architects because the plan did not follow departmental enterprise architecture guidelines. When I pointed out that there were no department guidelines, they said, “You are right … we will start developing them now.”

Don’t get me wrong; a number of the gates in the PAL process could be combined to reduce the amount of time the process slows for review. The deliverables could be reviewed to ensure that the proposal is generally sound and not rejected for some lack of compliance to form. But these are easy changes.

What is required is a reduction in the redundant oversight layered on the process. Currently, divisional, departmental, agency and CDT staff all review the same PAL documentation and impose their prejudices and political two cents. CDT should delegate oversight and be sure that every area is reviewed just once. CDT should see that their delegates are competent and responsible and then trust them. Delegates should be accountable for their oversight. If there is a failure in their domain, it is on them. Overseers should not be able only to criticize; they should be responsible for offering solutions where they think that the program is wrong-headed. They should be responsible for getting projects done, not just for saying “no.”

What is required is to monitor execution and fail fast. Consider this: If, in a waterfall project, we are to capture requirements up front, and design and develop to those requirements, then invoking change management to make a substantive change should be a signal that maybe you need to start over. Changing the scope should always signal a stop and restart for any project.

The last note: I was sorry to see California Health and Human Services Secretary Mike Wilkening depart. While I do not believe that we ever met, I was an admirer of the leadership he exhibited to start the Child Welfare Digital Services development program using thoroughly modern methods. CWDS execution slowed as they made a few of the mistakes outlined in the previous posts on agile methods, but in a risk-averse government IT culture, the strength to take CWDS on and to try to do it right is worthy of note.