IE11 Not Supported

For optimal browsing, we recommend Chrome, Firefox or Safari browsers.

San Diego Among First to Build Open Data Portal

Over the last year, the city became one of the first to build and host its own data portal using a number of open source tools.

Over the last year, the city of San Diego has implemented an open data policy, and it didn’t just upload data sets to a data portal and leave residents to interpret the information. It became one of the first cities to build and host its own data portal, according to San Diego’s Performance and Analytics Department director Almis Udrys.

At the beginning of their process, the city purchased off-the-shelf software. After nine months, staff members began designing a personalized version for the city.

“It wasn’t anything they did that caused us to try doing it ourselves, more our own curiosity and innovative spirit and belief that it would be cool to be among the first governments to use our internal expertise to build something others are selling,” Udrys wrote in an email to Techwire.

Built by city employees, the new portal is proving to be cost-effective. It saves $60,000 a year and only about $1,200 a year to manage.

The new site also includes a blog that helps residents interpret the information included on the portal. The city is working on automating the data and speeding up the process of making information available.

“Being able to design and host our own portal gives us better control over the features and making changes to it as our users require and request,” said Udrys in a phone interview.

Staff team members used multiple tools to create the data portal, including Github for collaborative coding, and they managed to avoid using a content management system by working with Jekyll, an open source site generator.

The site is hosted on Amazon S3, where code is deployed by CircleCI and CloudFront copies static site code to servers all over the world and routes users through the nearest server, which speeds up access to the data.

Data previews are created by Polymer, which makes a template for data set pages, and Lambda streams the first 10 rows of information back from S3. This way users can determine if they need specific data without having to download it.

More about how the architecture was built can be seen here.

Data on the new system includes parking meter tracking information, police stop data and crime statistics.

“It kind of runs the gamut of various data sets that we over the course of our implementation of our open data policy have identified,” Udrys said.

The source code for the entire project is open source and available for anyone who wants to try to build something similar.

Kayla Nick-Kearney was a staff writer for Techwire from March 2017 through January 2019.