Technical Specification
A very high level start to an OMG technical specification to get the ball rolling. Can also be output in JSON and CSV and other formats. What's important here is defining the standard fields, not the file format!Why Not Use An Existing Standard?
So why not just use an existing data standard? Well, as far as we know, there is not a standard that is made for this purpose, defining the optimal fields for civic data sharing. If you know of one, let us know.Filed Formatting
The fields chosen for the OMGStandard can be chosen from Dublin Core or Schema.org, as part of its "ontology" or "taxonomy" of fields. As a subset of these broad standards, the OMGStandard is immediately simpler and more accessible to government workers and regular citizens.Data Formats
OMG data fields can be output and stored in any number of data formats, including CSV, RSS, XML, KML, JSON, MySQL tables, etc. We recommend initial output and storage be in CSV, JSON, or XML since they have the most granular fields. KML and RSS is more suited to the final machine-output of the data, with styles included, not really for data transfer and export from city databases. Many fields like Name, Description, and Data must get merged into a single un-parsable field in these formats. Another limitation of KML or GeoRSS is lack of Latitude and Longitude from government sources, which is a required part of those formats, but optional under our standard.Field Introduction
High-level examples of the data fields used for a single location.Required Location Details
- Unique ID - Identifies the location so you can update it later if needed. Example: "26", "342ax", "DJWU-339832-KS"
- Name - Descriptive short title of the location/marker. Example: "Assault with a Weapon (2 counts)", "City Cafe - Rating B+", "John Doe Smith - Unregistered"
- Address - Street address of the location. Example: "624 West Main St", "Baxter Ave and Cherokee Parkway", "Everett Ave"
- City - City name of the location. Example: "Louisville", "Queens", "Fargo"
- State - State of the location. Example: "KY", "NY", "CA"
- Category ID - ID of the category this belongs to. Categories can be anything, like crime types, permit codes, or grades. This number ID just identifies them for your data. These are defined with the dataset metadata. Example: "1", "7", "23"
- Latitude - Latitude coordinate of the location. Example: "37.392984392"
- Longitude - Longitude coordinate of the location. Example: "-84.239837184"
Optional Location Details
- Description - Detailed HTML description. Example: "Sex: Male‹br/›Race: White‹br/›Details: Suspect attempted to rob victim at gunpoint."
- Address 2 - Extra information about the location. Example: "Apt 2", "USA Town Mall", "On the sidewalk"
- Zip - Zip code of the location. Example: "40206"
- Date - ISO formatted date of the event/review/etc. Example: "2005-09-28"
- Time - ISO formatted time of the event/review/etc. Example: "14:59:01"
- Full URL - Link to a page on your site with more information. Example: "http://www.mywebsite.com/detailpage/infoonmyplace.htm"
- Image Path - Full path to an image on your site. Example: "http://www.mywebsite.com/images/locationimage2918.jpg"
- Detail Link - True or false to show if this location links to a dynamic page on your site that uses your Unique ID. Example: "1", "0" for "http://www.mywebsite.com/detailpage/info.asp?id=[Unique ID]"
- Details - Extended HTML description of your location for the detail page. This can be very long if needed. Example: "See case 29491-OE for more details..."
- Field1, Field2, etc - A custom field for a specific dataset. There can be any number of these, allowing filterable data to be placed here instead of Description. These are defined with the dataset metadata. Example: "Open", "Closed", "Category2"
Formatted Examples
CSV Example
This format is the best for significantly smaller file sizes. It communicates the most data with the least field name redundancy. Many cities use simple CSV because of this, and because it is human readable in spreadsheet programs. The first row is always the OMG field names.
Restaurant inspection example:
View CSV
ID,Name,Description,Category,Address,Address2,City,State,Zip,Date,Latitude,Longitude,URL,ImagePath,DetailLink
43367,EL CAMINO,<b>Score: 99</b><BR>RESTAURANT<i></i>,1,1314 Bardstown Rd,,LOUISVILLE,KY,40204,11/2/12,38.224098,-85.674797,http://elfreakingcamino.com/,https://fbcdn-profile-a.akamaihd.net/hprofile-ak-prn1/s160x160/1378696_253912441427943_1454540082_a.jpg,http://portal.louisvilleky.gov/dataset/restaurant-inspection-data?id=43367
33659,HOMEMADE ICE CREAM,<b>Score: 95</b><BR>FOOD SERVICE<i></i>,2,2525 BARDSTOWN RD,,LOUISVILLE,KY,40204,12/12/12,38.222698,-85.682999,http://www.piekitchen.com,http://www.piekitchen.com/pk%20images/Hdr-PK-2012.jpg,http://portal.louisvilleky.gov/dataset/restaurant-inspection-data?id=33659
JSON Example
Larger file sizes than CSV, but good for processing on-the-fly in web pages.
Nursing home example:
View JSON
{ "items" :
[
{
"id" : "185237",
"name" : "FOUR COURTS AT CHEROKEE PARK",
"description" : "<b>Ratings (out of 5):</b><br>Health Inspection: 1<br>Nurse Staff: 3<br>Quality Measures: 3<br>RN Only: 4<br>(502) 451-0990",
"categoryid" : "1",
"address" : "2100 MILLVALE RD.",
"city" : "LOUISVILLE",
"state" : "KY",
"zip" : "40205",
"latitude" : "38.230888",
"longitude" : "-85.686569",
"field1" : "1",
"field2" : "3",
"detaillink" : "http://www.yourmapper.com/details/178/185237/four-courts-at-cherokee-park.htm"
}
]
}
XML Example
XML provides the most structured data format, but is larger than JSON files and can be difficult to read at a glance.
Crime example:
View XML
<?xml version="1.0" encoding="utf-8"?>
<dataset name="Crime Reports">
<metadata>
<sourcename>Louisville Metro Police Department</sourcename>
<sourceurl>http://www.louisvilleky.gov/MetroPolice/default.htm</sourceurl>
<sourcecity>Louisville</sourcecity>
<sourcestateid>18</sourcestateid>
<sourcecountryid>216</sourcecountryid>
<description>All reported crimes for the city of Louisville in Jefferson County, Kentucky, since 2002.</description>
<updatefrequency>Weekly</updatefrequency>
<lastupdated>2008-05-19 03:23:24.22</lastupdated>
<startdatetime>2002-01-01 02:11:15.97</startdatetime>
<enddatetime>2008-05-18 23:15:54.22</enddatetime>
</metadata>
<locationdata>
<centercity>Louisville</centercity>
<centerstateid>18</centerstateid>
<centercountryid>216</centercountryid>
<coveragecity>Louisville</coveragecity>
<coveragestateid>18</coveragestateid>
<coveragecountryid>216</coveragecountryid>
<minlatitude>33.8232387</minlatitude>
<maxlatitude>34.2312221</maxlatitude>
<minlongitude>-85.3871842</minlongitude>
<maxlongitude>-84.1028518</maxlongitude>
</locationdata>
<item>
<id>C34928341</id>
<name>Assault</name>
<description><![CDATA[Assault with a Deadly Weapon, 2nd Degree<br>Time: 1:40 AM<br>Race: White<br>Sex: M<br>Born: Nov 24, 1979]]>
</description>
<categories>
<categoryid>21</categoryid>
</categories>
<address>Baxter Ave And Tyler Park Dr</address>
<city>Louisville</city>
<state>KY</state>
<date>2008-04-15</date>
<detaillink>1</detaillink>
</item>
</dataset>
XOMGL Examples
XOMGL is the name of the XML formatted version of the OMGStandard data.
Wikipedia
XOMGL Wikipedia Article
XOMGL Spreadsheet Example
Download this spreadsheet to get started formatting your data by seeing a few examples.
Empty XOMGL
All possible fields with comments
View XML
<?xml version="1.0" encoding="utf-8"?>
<dataset name=""> <!-- * name:string - name of the dataset -->
<metadata> <!-- * metadata about the dataset -->
<sourcename><!-- * string: name of the source of the data --></sourcename>
<sourceurl><!-- string: full path to the page that has the source data --></sourceurl>
<sourcecity><!-- string: name of the originating city --></sourcecity>
<sourcestateid><!-- int: id of the originating state --></sourcestateid>
<sourcecountryid><!-- int: id of the originating country --></sourcecountryid>
<description><!-- string: text description of source of the data --></description>
<updatefrequency><!-- string: frequency of data updates, daily, monthly, weekly, etc --></updatefrequency>
<lastupdated><!-- datetime: stamp of last update to this dataset --></lastupdated>
<startdatetime><!-- datetime: earliest date of data coverage --></startdatetime>
<enddatetime><!-- datetime: latest date of data coverage --></enddatetime>
</metadata>
<locationdata> <!-- * location based information -->
<centercity><!-- string: name of the city of the data center --></centercity>
<centerstateid><!-- int: id of the state of the data center --></centerstateid>
<centercountryid><!-- int: id of the country of the data center --></centercountryid>
<centerlatitude><!-- float: latitude of center --></centerlatitude>
<centerlongitude><!-- float: longitude of center --></centerlongitude>
<coveragecity><!-- string: city of the dataset coverage area. Leave out if coverage area is greater --></coveragecity>
<coveragestateid><!-- int: id of the state of the dataset coverage area. Leave out if coverage area is greater --></coveragestateid>
<coveragecountryid><!-- * int: id of the country of the dataset coverage area. --></coveragecountryid>
<minlatitude><!-- float: based on data coverage area --></minlatitude>
<maxlatitude><!-- float: based on data coverage area --></maxlatitude>
<minlongitude><!-- float: based on data coverage area --></minlongitude>
<maxlongitude><!-- float: based on data coverage area --></maxlongitude>
</locationdata>
<itemdata> <!-- data concerning all items -->
<externalurl><!-- full path to more info with [ExternalID] in place of item->id in a url --></externalurl>
</itemdata>
<categories> <!-- category to id relationship for a dataset -->
<category>
<categoryid><!-- int: id of categorization of item. --></categoryid>
<categoryname><!-- string: short name of category. --></categoryname>
</category>
<category>
<categoryid><!-- int: id of categorization of item. --></categoryid>
<categoryname><!-- string: short name of category. --></categoryname>
</category>
</categories>
<item> <!-- * one item of data. there can be many of these -->
<id><!-- * string: unique id of this item based on internal system id. used for updating data later --></id>
<name><!-- * string: name of the item --></name>
<description><!-- string: html formatted or plain text full description of item --></description>
<categories> <!-- one or more categories the item can be classified as. id links to other xml formatted data that shows category name-->
<categoryid><!-- int: id of categorization of item. --></categoryid>
</categories>
<!-- * Note: one of the following must be provided: (location) or (address, city, state) or (latitude, longitude) -->
<location><!-- string: full text string of location. this can be used instead of address, city, state, zip--></location>
<address><!-- string: street address with house number --></address>
<address2><!-- string: more detailed street address info or location description --></address2>
<city><!-- string: city of item location --></city>
<state><!-- string: state of item location --></state>
<zip><!-- string: zip code of item location --></zip>
<date><!-- date: date associated with item --></date>
<latitude><!-- float: latitude of item location --></latitude>
<longitude><!-- float: longitude of item location --></longitude>
<fullurl><!-- string: full path to more information about this item --></fullurl>
<imagepath><!-- string: full path to location of image on a server --></imagepath>
<detaillink><!-- boolean: if true, will us item->id to link to another XML dataset with full details --></detaillink>
</item>
</dataset>
Minimum XOMGL
Minimum required fields with comments
View XML
<?xml version="1.0" encoding="utf-8"?>
<dataset name=""> <!-- * name:string - name of the dataset -->
<metadata> <!-- * metadata about the dataset -->
<sourcename><!-- * string: name of the source of the data --></sourcename>
</metadata>
<locationdata> <!-- * location based information -->
<coveragecountryid><!-- * int: id of the country of the dataset coverage area. --></coveragecountryid>
</locationdata>
<item> <!-- * one item of data. there can be many of these -->
<id><!-- * string: unique id of this item based on internal system id. used for updating data later --></id>
<name><!-- * string: name of the item --></name>
<!-- * Note: one of the following must be provided: (location) or (address, city, state) or (latitude, longitude) -->
<location><!-- string: full text string of location. this can be used instead of address, city, state, zip--></location>
</item>
</dataset>
Example: Pollution Sources
An example of a pollution source dataset
View XML
<?xml version="1.0" encoding="utf-8"?>
<dataset name="EPA Toxic Pollution Sources">
<metadata>
<sourcename>Environmental Protection Agency</sourcename>
<sourceurl>http://www.epa.gov/tri/tridata/index.htm</sourceurl>
<sourcecountryid>216</sourcecountryid>
<description>All sources of toxic pollution from the EPA's monitoring and reporting system from 2006-2007</description>
<updatefrequency>Yearly</updatefrequency>
<lastupdated>2008-01-30 03:23:24.22</lastupdated>
<startdatetime>2006-01-01 02:11:15.97</startdatetime>
<enddatetime>2007-12-31 23:15:54.22</enddatetime>
</metadata>
<locationdata>
<centerstateid>17</centerstateid>
<centercountryid>216</centercountryid>
<coveragecountryid>216</coveragecountryid>
<minlatitude>15.82323</minlatitude>
<maxlatitude>63.23122</maxlatitude>
<minlongitude>-178.38718</minlongitude>
<maxlongitude>-63.10285</maxlongitude>
</locationdata>
<itemdata>
<externalurl>http://www.epa.gov/tri/tridata/datapoint/source.aspx?id=[ExternalID]</externalurl>
</itemdata>
<item>
<id>439-B.3148253</id>
<name>HEXION SPECIALTY CHEMICALS INC.</name>
<description><![CDATA[Emissions (pounds): 12,500<br>Major Toxic Chemical: Hexaflorine]]>
</description>
<categories>
<categoryid>3</categoryid>
<categoryid>4</categoryid>
</categories>
<address>7635 National Turnpike Unit 130</address>
<address2>Campmore Building</address2>
<city>Kansas City</city>
<state>KA</state>
<latitude>45.39281332</latitude>
<longitude>-152.34718334</longitude>
</item>
</dataset>
CSV Example
This format is the best for significantly smaller file sizes. It communicates the most data with the least field name redundancy. Many cities use simple CSV because of this, and because it is human readable in spreadsheet programs. The first row is always the OMG field names.Restaurant inspection example: View CSV
ID,Name,Description,Category,Address,Address2,City,State,Zip,Date,Latitude,Longitude,URL,ImagePath,DetailLink 43367,EL CAMINO,<b>Score: 99</b><BR>RESTAURANT<i></i>,1,1314 Bardstown Rd,,LOUISVILLE,KY,40204,11/2/12,38.224098,-85.674797,http://elfreakingcamino.com/,https://fbcdn-profile-a.akamaihd.net/hprofile-ak-prn1/s160x160/1378696_253912441427943_1454540082_a.jpg,http://portal.louisvilleky.gov/dataset/restaurant-inspection-data?id=43367 33659,HOMEMADE ICE CREAM,<b>Score: 95</b><BR>FOOD SERVICE<i></i>,2,2525 BARDSTOWN RD,,LOUISVILLE,KY,40204,12/12/12,38.222698,-85.682999,http://www.piekitchen.com,http://www.piekitchen.com/pk%20images/Hdr-PK-2012.jpg,http://portal.louisvilleky.gov/dataset/restaurant-inspection-data?id=33659
JSON Example
Larger file sizes than CSV, but good for processing on-the-fly in web pages.Nursing home example: View JSON
{ "items" :
[
{
"id" : "185237",
"name" : "FOUR COURTS AT CHEROKEE PARK",
"description" : "<b>Ratings (out of 5):</b><br>Health Inspection: 1<br>Nurse Staff: 3<br>Quality Measures: 3<br>RN Only: 4<br>(502) 451-0990",
"categoryid" : "1",
"address" : "2100 MILLVALE RD.",
"city" : "LOUISVILLE",
"state" : "KY",
"zip" : "40205",
"latitude" : "38.230888",
"longitude" : "-85.686569",
"field1" : "1",
"field2" : "3",
"detaillink" : "http://www.yourmapper.com/details/178/185237/four-courts-at-cherokee-park.htm"
}
]
}
XML Example
XML provides the most structured data format, but is larger than JSON files and can be difficult to read at a glance.Crime example: View XML
<?xml version="1.0" encoding="utf-8"?> <dataset name="Crime Reports"> <metadata> <sourcename>Louisville Metro Police Department</sourcename> <sourceurl>http://www.louisvilleky.gov/MetroPolice/default.htm</sourceurl> <sourcecity>Louisville</sourcecity> <sourcestateid>18</sourcestateid> <sourcecountryid>216</sourcecountryid> <description>All reported crimes for the city of Louisville in Jefferson County, Kentucky, since 2002.</description> <updatefrequency>Weekly</updatefrequency> <lastupdated>2008-05-19 03:23:24.22</lastupdated> <startdatetime>2002-01-01 02:11:15.97</startdatetime> <enddatetime>2008-05-18 23:15:54.22</enddatetime> </metadata> <locationdata> <centercity>Louisville</centercity> <centerstateid>18</centerstateid> <centercountryid>216</centercountryid> <coveragecity>Louisville</coveragecity> <coveragestateid>18</coveragestateid> <coveragecountryid>216</coveragecountryid> <minlatitude>33.8232387</minlatitude> <maxlatitude>34.2312221</maxlatitude> <minlongitude>-85.3871842</minlongitude> <maxlongitude>-84.1028518</maxlongitude> </locationdata> <item> <id>C34928341</id> <name>Assault</name> <description><![CDATA[Assault with a Deadly Weapon, 2nd Degree<br>Time: 1:40 AM<br>Race: White<br>Sex: M<br>Born: Nov 24, 1979]]> </description> <categories> <categoryid>21</categoryid> </categories> <address>Baxter Ave And Tyler Park Dr</address> <city>Louisville</city> <state>KY</state> <date>2008-04-15</date> <detaillink>1</detaillink> </item> </dataset>
XOMGL Examples
XOMGL is the name of the XML formatted version of the OMGStandard data.Wikipedia
XOMGL Wikipedia ArticleXOMGL Spreadsheet Example
Download this spreadsheet to get started formatting your data by seeing a few examples.Empty XOMGL
All possible fields with commentsView XML
<?xml version="1.0" encoding="utf-8"?> <dataset name=""> <!-- * name:string - name of the dataset --> <metadata> <!-- * metadata about the dataset --> <sourcename><!-- * string: name of the source of the data --></sourcename> <sourceurl><!-- string: full path to the page that has the source data --></sourceurl> <sourcecity><!-- string: name of the originating city --></sourcecity> <sourcestateid><!-- int: id of the originating state --></sourcestateid> <sourcecountryid><!-- int: id of the originating country --></sourcecountryid> <description><!-- string: text description of source of the data --></description> <updatefrequency><!-- string: frequency of data updates, daily, monthly, weekly, etc --></updatefrequency> <lastupdated><!-- datetime: stamp of last update to this dataset --></lastupdated> <startdatetime><!-- datetime: earliest date of data coverage --></startdatetime> <enddatetime><!-- datetime: latest date of data coverage --></enddatetime> </metadata> <locationdata> <!-- * location based information --> <centercity><!-- string: name of the city of the data center --></centercity> <centerstateid><!-- int: id of the state of the data center --></centerstateid> <centercountryid><!-- int: id of the country of the data center --></centercountryid> <centerlatitude><!-- float: latitude of center --></centerlatitude> <centerlongitude><!-- float: longitude of center --></centerlongitude> <coveragecity><!-- string: city of the dataset coverage area. Leave out if coverage area is greater --></coveragecity> <coveragestateid><!-- int: id of the state of the dataset coverage area. Leave out if coverage area is greater --></coveragestateid> <coveragecountryid><!-- * int: id of the country of the dataset coverage area. --></coveragecountryid> <minlatitude><!-- float: based on data coverage area --></minlatitude> <maxlatitude><!-- float: based on data coverage area --></maxlatitude> <minlongitude><!-- float: based on data coverage area --></minlongitude> <maxlongitude><!-- float: based on data coverage area --></maxlongitude> </locationdata> <itemdata> <!-- data concerning all items --> <externalurl><!-- full path to more info with [ExternalID] in place of item->id in a url --></externalurl> </itemdata> <categories> <!-- category to id relationship for a dataset --> <category> <categoryid><!-- int: id of categorization of item. --></categoryid> <categoryname><!-- string: short name of category. --></categoryname> </category> <category> <categoryid><!-- int: id of categorization of item. --></categoryid> <categoryname><!-- string: short name of category. --></categoryname> </category> </categories> <item> <!-- * one item of data. there can be many of these --> <id><!-- * string: unique id of this item based on internal system id. used for updating data later --></id> <name><!-- * string: name of the item --></name> <description><!-- string: html formatted or plain text full description of item --></description> <categories> <!-- one or more categories the item can be classified as. id links to other xml formatted data that shows category name--> <categoryid><!-- int: id of categorization of item. --></categoryid> </categories> <!-- * Note: one of the following must be provided: (location) or (address, city, state) or (latitude, longitude) --> <location><!-- string: full text string of location. this can be used instead of address, city, state, zip--></location> <address><!-- string: street address with house number --></address> <address2><!-- string: more detailed street address info or location description --></address2> <city><!-- string: city of item location --></city> <state><!-- string: state of item location --></state> <zip><!-- string: zip code of item location --></zip> <date><!-- date: date associated with item --></date> <latitude><!-- float: latitude of item location --></latitude> <longitude><!-- float: longitude of item location --></longitude> <fullurl><!-- string: full path to more information about this item --></fullurl> <imagepath><!-- string: full path to location of image on a server --></imagepath> <detaillink><!-- boolean: if true, will us item->id to link to another XML dataset with full details --></detaillink> </item> </dataset>
Minimum XOMGL
Minimum required fields with commentsView XML
<?xml version="1.0" encoding="utf-8"?> <dataset name=""> <!-- * name:string - name of the dataset --> <metadata> <!-- * metadata about the dataset --> <sourcename><!-- * string: name of the source of the data --></sourcename> </metadata> <locationdata> <!-- * location based information --> <coveragecountryid><!-- * int: id of the country of the dataset coverage area. --></coveragecountryid> </locationdata> <item> <!-- * one item of data. there can be many of these --> <id><!-- * string: unique id of this item based on internal system id. used for updating data later --></id> <name><!-- * string: name of the item --></name> <!-- * Note: one of the following must be provided: (location) or (address, city, state) or (latitude, longitude) --> <location><!-- string: full text string of location. this can be used instead of address, city, state, zip--></location> </item> </dataset>
Example: Pollution Sources
An example of a pollution source datasetView XML
<?xml version="1.0" encoding="utf-8"?> <dataset name="EPA Toxic Pollution Sources"> <metadata> <sourcename>Environmental Protection Agency</sourcename> <sourceurl>http://www.epa.gov/tri/tridata/index.htm</sourceurl> <sourcecountryid>216</sourcecountryid> <description>All sources of toxic pollution from the EPA's monitoring and reporting system from 2006-2007</description> <updatefrequency>Yearly</updatefrequency> <lastupdated>2008-01-30 03:23:24.22</lastupdated> <startdatetime>2006-01-01 02:11:15.97</startdatetime> <enddatetime>2007-12-31 23:15:54.22</enddatetime> </metadata> <locationdata> <centerstateid>17</centerstateid> <centercountryid>216</centercountryid> <coveragecountryid>216</coveragecountryid> <minlatitude>15.82323</minlatitude> <maxlatitude>63.23122</maxlatitude> <minlongitude>-178.38718</minlongitude> <maxlongitude>-63.10285</maxlongitude> </locationdata> <itemdata> <externalurl>http://www.epa.gov/tri/tridata/datapoint/source.aspx?id=[ExternalID]</externalurl> </itemdata> <item> <id>439-B.3148253</id> <name>HEXION SPECIALTY CHEMICALS INC.</name> <description><![CDATA[Emissions (pounds): 12,500<br>Major Toxic Chemical: Hexaflorine]]> </description> <categories> <categoryid>3</categoryid> <categoryid>4</categoryid> </categories> <address>7635 National Turnpike Unit 130</address> <address2>Campmore Building</address2> <city>Kansas City</city> <state>KA</state> <latitude>45.39281332</latitude> <longitude>-152.34718334</longitude> </item> </dataset>