Open Missouri Blog

New state data portal in the works

Stephanie Kinney — Wed, 04 Apr 2012 18:47:38 +0000

The Missouri Office of Administration is developing a new online portal at data.mo.gov to provide access to data from state agencies.

Data.mo.gov currently holds 16 data sets, including 2010 State Expenditures, County Sheriff’s Offices and Credit Unions.

“It is still being developed and populated with information,” Wanda Seeney, public information officer for the Office of Administration said in an email. “When it is launched, it will replace our current site, mo.gov/data.”

On the new site, users can generate data in machine-readable formats that researchers and developers will find easier to use, Seeney said.

For example, users can export data to Microsoft Excel spreadsheets, delimited text files, XML and other formats. Web developers can access data through an application programming interface (API). In addition, users can visualize data as a map or chart.

The Office of Administration is using the Socrata open data platform to host the data. Socrata has a contract with the state that runs through the current fiscal year, which ends on June 30, and has been paid around $7,460 so far, state records say.

A number of other governments have contracted with Socrata to provide their open data platforms: Data.Gov, which is the federal government’s data portal, as well as the United Nations Development Programme, New York City, and the City of Chicago.

Some of the governments using Socrata, such as Oregon, allow users to suggest data sets that should be posted. Unfortunately, Missouri’s site does not invite suggestions from the public. The only opportunity for users to interact is via a help form link.

Resources for Missouri data and open records

David Herzog — Fri, 16 Mar 2012 13:57:34 +0000

Last night I spoke at the Sunshine Week event hosted by the Missouri Sunshine Coalition at the Reynolds Journalism Institute. The subject: access to government information. OpenMissouri.org is one tool that helps you find and obtain information from government agencies. In my presentation, you can see a bunch more.

Resources for Missouri open records

View more PowerPoint from David Herzog

Looking ahead to 2012

David Herzog — Tue, 03 Jan 2012 15:51:52 +0000

Here at OpenMissouri, we’re not counting on a Mayan cataclysm anytime soon, so we have some plans to make the site more better, more useful, for you in 2012.

In the weeks to come, we’re going to post information about some more data sets from state agencies and from the City of Columbia.

We’re going to share some sample data sets that you can then download with one click.

Here on the blog, we hope to post articles that give you a deeper look at some of the data sets and tell you why you might find them interesting.

Behind the scenes, we’re going to make a push to get some cities across the state more involved with OpenMissouri. Also, we want to make OpenMissouri more attractive to web developers and open data enthusiasts.

–David Herzog, OpenMissouri founder

How big is OpenMissouri? Here’s our growth chart

David Herzog — Fri, 23 Dec 2011 14:35:21 +0000

One of my favorite holiday traditions comes when I visit my sister’s family. My sister, who has two grown kids of her own, lines my teenage son against the wall in her kitchen. Then she grabs a pencil and marks a spot to show how my son has grown. For anyone who has watched kids sprout, it’s a pretty amazing experience.

In that spirit, I thought it would be great to give OpenMissouri a chart of its own to show you how much we’ve blossomed since our launch on March 17, 2011 during Sunshine Week.

You can see that we nearly doubled the number data sets that we’ve cataloged on the site. Definitely great news. However, as I mentioned before, we still have a lot of work to do, because 10 out of 19 state agencies have failed to provide full information.

Other accomplishments since launch include:

Adding a Sunshine request letter generator for registered site users
Allowing registered site users to share sample data sets that they’ve obtained
Letting anyone download sample data with one click
Releasing the OpenMissouri data catalog software as an open-source project on GitHub. Developers can contribute to the software code. Groups in other locales can use the free code to host their own sites.
Reporting more in depth about U.S. newspaper journalists use of data on government web sites.

It’s been a busy and productive year. I’ll look ahead into plans for 2012 another time soon.

–David Herzog, OpenMissouri founder

2011 year-end agency scorecard update

David Herzog — Thu, 22 Dec 2011 20:16:25 +0000

The OpenMissouri team made progress during the past few months by posting information about data sets kept offline by some additional state government agencies. Master’s students from the Missouri School of Journalism working with me gathered the information from the departments of Higher Education and Public Safety. We’re grateful to have them on board, working together to make government more transparent in the Show-Me State.

Here’s a quick rundown of where stage agencies stand, in terms of providing information to OpenMissouri:

Provided requested information at no cost

Department of Agriculture

Department of Conservation

Department of Elementary and Secondary Education

Ethics Commission

Department of Higher Education

Historical Society

Department of Natural Resources

Department of Public Safety

UM System

Provided partial information at no cost

Department of Labor and Industrial Relations

Department of Revenue

Have not provided the information we requested

Office of Administration

Department of Corrections

Department of Economic Development

Department of Health and Senior Services

Department of Insurance, Financial Institutions and Professional Regulation

Department of Mental Health

Department of Social Services

Said it would provide information for a fee

Department of Transportation ($270)

You can see a good number of agencies are working with us. A good number still are not, but we hope to change that so we can become an even more valuable resource for you.

–David Herzog, OpenMissouri founder

Download sample data with one click

David Herzog — Mon, 28 Nov 2011 23:01:02 +0000

We’ve been adding sample data files to the site, so now you can see what other OpenMissouri.org community members have gotten from state and local government agencies.

We’ve made it really easy for you to download copies of those files. Just click the “Download sample data” button and the file is yours.

Remember that if you want to share any data that you’ve gotten, just click the “Suggest a Data Set” button.

-David Herzog, OpenMissouri founder

Deploy, contribute to our software

David Herzog — Tue, 18 Oct 2011 15:59:58 +0000

One of of the goals I had for OpenMissouri, when I started the project as a 2010-2011 fellow at the Reynolds Journalism Institute, was to provide inspiration and tools to groups in other locales that want to deploy data catalogs of their own.

To that end, we’ve made available, at no cost, the software that drives OpenMissouri.org. You can find it on GitHub, a social coding site.

The University of Missouri System Board of Curators holds the copyright on what we developed, but we’re releasing the code under an open-source permissive license. That means groups with the technical chops can grab the code and use it as they wish. It also means developers can join in and enhance the software with new features.

We built the site using Rails 3 web framework and Ruby 1.9.2 programming language. For more details about the components, see our readme file on GitHub.

In the weeks ahead, I’ll talk about some of the choices we made during the development and stumbling blocks we encountered.

–David Herzog, OpenMissouri.org founder

Making it easier to find, request, share data

David Herzog — Thu, 06 Oct 2011 17:27:32 +0000

The OpenMissouri.org team has been hard at work these past few months, since our March 17 launch at the Reynolds Journalism Institute during Open Missouri Day. We’re offering new features that make it easier for you to find, request and share data sets held by state and local government agencies.

Sunshine This

We added a button that lets registered site users generate, on the fly, a customized Sunshine request letter for a data set. Just register for an account and provide your complete contact information. Click on Sunshine This and OpenMissouri.org creates the letter. You’ll need to copy and paste the letter into your email or word processing program and put on the finishing touches before sending it along.

Suggest a Data Set

If you are a registered user, you can now tell us about data sets that you’ve discovered by clicking on the Suggest a Data Set button. You can even upload a copy of the data to share it with others. We’ll verify the information before we post it for the public.

Now with more data

We added information about more data sets and now are up to around 250. We’ll add more from the Missouri of Department of Public Safety in the coming weeks. Master’s degree students from my Computer-Assisted Reporting class at the Missouri School of Journalism will continue to gather information about data sets from state and local government agencies this fall.

Other improvements

Moved off Heroku, an Amazon cloud-based development service, to a faster production server.

Added a contact form so it’s easier for you to contact us.

Redesigned the site. It’s a little more eye pleasing, we think.

Thanks and remember that you can get the latest OpenMissouri.org news by following @OpenMissouri on Twitter, subscribing to our email newsletter (look for the box on the bottom right of the home page) and subscribing to our RSS feeds for data sets and blog posts.

-David Herzog. OpenMissouri.org founder

Agency scorecard: Victories and obstacles

David Herzog — Mon, 13 Jun 2011 15:53:37 +0000

Last week was a big one for Open Missouri. We added details about databases from the state Department of Elementary and Secondary Education, which collects a whole lot of information, thanks to federal and state education laws. We now list information about nearly 220 databases and 19 state agencies.

The Open Missouri team has made great strides these past several months gathering inventories of data sets held offline by state government agencies. We’ve been asking agencies to tell us for each database:

Name

Description

Data format

Time span covered by the data

Update frequency

Cost (if any)

We’re grateful for the help from the seven agencies that compiled information about their databases for Open Missouri. But we still have gaps to fill. Another seven agencies have not provided the information that we’ve requested. Two others have given us some information and we need to circle back and get the missing details. Here’s a list of agencies and where they stand:

Provided requested information

Department of Agriculture

Department of Conservation

Department of Elementary and Secondary Education

Ethics Commission

Historical Society

Department of Natural Resources

UM System

Provided partial information

Department of Labor and Industrial Relations

Department of Revenue

Processing our request

Department of Public Safety

Have not provided the information we requested

Office of Administration

Department of Corrections

Department of Economic Development

Department of Health and Senior Services

Department of Insurance, Financial Institutions and Professional Regulation

Department of Mental Health

Department of Social Services

We have not yet asked

Department of Higher Education

Department of Transportation

You can see that we have plenty to do. If you know about a data set that a state agency keeps, please join the Open Missouri community and then click on the “Suggest a data set button” to tell us as much as you know about it.

-David Herzog

Open Missouri Day replay

David Herzog — Thu, 07 Apr 2011 20:34:43 +0000

If you want to catch up on the March 17 Open Missouri Day sessions, you can watch the recorded video. You can see:

RJI fellow David Herzog’s opening remarks

Sunlight Foundation Editorial Director Bill Allison’s keynote about how journalists are using data from federal, state and local government agencies

The idea session with IRE training director Jaimi Dowdell, Kansas City Star reporter Mike Mansur and Springfield News-Leader digital editor Matt Peterson showing how Missouri news organizations are using data for their reporting and websites

A roundtable discussion about best practices for obtaining data under the Missouri Sunshine Law, with IRE executive director Mark Horvit (moderator), Missouri Sunshine Coalition President Jim Robertson and Jean Maneke, a Kansas City attorney specializing in media law.

Presentation by Patricia Churchill, head of the governmental affairs division in the Missouri Attorney General’s office.