TycheManual:PublicData
In a perfect world, all public data is available for free and readily available on open data sites. We always point to Louisville’s open data site as an aspirational example for how a city handles open data. If you are able to draw from such a site that’s great - just track when you downloaded the data, its vintage, the URL and any other information you need to recollect your process.
If open data is not available. you need to use a Freedom of Information Act or Law request to get data (or a similar non-USA practice). We typically first search a government’s web site (“FOIA request”, “FOIL process”, “public information officer”) to see what the existing process is. Typically, these web-based processes will have a request or ticketing system to process and track the request.
If no such system exists, you may have to write a letter - various resources include:
https://brechner.org/guides/foia-requests-letter/
https://www.nfoic.org/sample-foia-request-letters/
A process that you might use for acquiring data could look like this:
- Identify if data are available from your local government on a public, open data website
- If an open data website exists but the data are not available, send an email to the open data website owners inquiring about the availability of the data
- If no open data website exists then send an email to the government department that is most likely to own the data, asking about its availability
- If the above is not possible or productive, identify if your local government has an online public information request system and make a FOIL/FOIA request through the system
- If no online public information request system exists, you will need to write a letter or email
As you make requests, include in your requests:
- machine readable (csv or excel formats, not scanned pdfs)
- an expectation for what data fields or columns of information should be included
- as much of a time period as possible ("from 2015 to present" or similar)
- any documentation or data descriptions that accompany the data
You do not need to explain why you want the data, what you are going to use the data for, or make any commitments on what you will or won't do with the data.
Next: Data wrangling