RFC: Integration with Wikidata

[brierjon]

In the OpenStates Slack People Channel I proposed adding Wikidata IDs to the People Data (to start).

This discussion is for scoping and feedback on the integration and maintenance between Wikidata and OpenStates and workflows to maintain OpenState to Wikidata and back entity resolution and data sync (if this is even desired). Syncing back from Wikidata would need checks on changes to avoid importing vandalism.

Wikidata is Public Domain which allows for data reciprocity, but a workflow to integrate in either direction should be discussed to keep ids in sync and monitor for value deviations. Wikidata affords queries and promoted values when a statement is the most complete or accurate compared to normal or deprecated in case there is a reoccurring error to prevent reintroduction or a conflict between statements with references.

There is an overlapping community of interest for keeping related information in linked open data via Wikiproject every politician specifically the United States portion of the project.

• Example Adam Hollier (Q76373561) for Adam Hollier

Initial scope:

1. Resolving IDs between Wikidata and OpenStates People Entity matching is collaborative and assisted by Mix'n'match, a Wikidata tool. It was selected to help match and maintain as generates reports for monitoring of conflicts and potential mismatches. Initially the 7997 people from the bulk dump are being resolved.
2. Adding of OpenStates ID to Wikidata - proposed Wikidata property ID: OpenStates People ID
3. Adding of Wikidata ID to OpenStates Field - Propose using "Wikidata" as the key and the "Q" from Wikidata as the value. Example: Wikidata: Q76373561
4. Adding of Wikidata IDs to people records. To be done following git pull request Contributing People Data

Focus of discussion:
5. Discussion needed - develop an ongoing workflow for new IDs being added to Wikidata and OpenStates resolving the entity IDs Wikidata / OpenStates. Wikidata's development plan is working on a REST API which may make a more direct approach as a future option. For this first run I grabbed a bulk dump and processed it with a python script to generate the computed ID for Wikidata and add headers for Mix'n'Match with the corresponding Wikidata fields. I'm new to OpenStates so I'm not sure what the update

Additional Discussion:

1. Discuss if Wikidata statements should be exported to OpenStates people automatically or with a human in the loop to review before importing as a pull request. For example, external identifiers.

Future considerations:

1. Expanding Wikidata resolving to include additional OpenStates entities where applicable. There are a limited number of bills in Wikidata. Committees may be a reasonable entity to match as well as legislative districts.
   https://www.wikidata.org/wiki/Wikidata:WikiProject_Govdirectory/United_States_of_America#Government_agencies
2. As mentioned in discussion IDs mapping to other unique identifiers #73 Wikidata might be a complementary commons where resolving of many fields of interest may fall for office holders and could integrate https://github.com/votesmart/political-id-match IDs where Wikidata Properties exist for those items. Votesmart, Ballotpedia, US Federal Election Commission ID, FEC Campaign Committee ID, OpenSecrets People ID, OpenSecrets Organization ID, and other properties.

Note:
Entity resolving with Wikidata has already lead to a few fixes and may be an additional QA tool as well as open up new opportunity to tap into the Linked Open Data Ecosystem:

• Handful of corrections in English Wikipedia entries for district numbers either both in text or just the Wikipedia structured data infobox.
• Identifying a few name issues in OpenStates data:
  • john-dr-bizon-1pQ5J6Tm1WO8Tg48vHVYaH name #637
  • mark-a-a-macdonald-4wlYddddiEuKhBUYOQVOhp name #640
  • linda-joy-j-sullivan-5pU9OxGOJVSzDyk0KnK4YV name #641
  • terri-lynn-l-williams-1qkUPxfDKLhv4PqxyMGqA1 name #642