Create Process/Scripts To Update Data in Portal

[seabre]

Whenever changes are made to bicycle_parking.geojson, a process/scripts should be in place so that the data on the portal is updated.

[aplannersguide]

1+

[junosuarez]

it would be cool to use travisci or something similar to do this. would also be cool if socrata supported it natively as a github postcommit hook.

[aplannersguide]

@seabre Is this something that I can change due to your latest commit? Also, does the readme file need updated with the build status or is that something to add later?

[seabre]

@aplannersguide There is a TravisCI badge you can add, but TravisCI needs to be configured, first.

[seabre]

@aplannersguide

I have a user on the data portal especially for TravisCI.

.travis.yml will need to be modified to include the needed encrypted credentials for this account.

You can add me to this repo and I can do it (I have to have access to the repo to generate encrypted credentials) or you can create a user on the data portal for the same purpose and I can show you how to add those credentials.

[aplannersguide]

@seabre added you as an admin for this repo. Let me know if you need anything else.

[seabre]

@jden @aplannersguide got it: https://travis-ci.org/cityofchattanooga/Bicycle-Parking/builds/29940516

[seabre]

I had to rerun the build a few times for tweaking permissions, but it looks good.

[seabre]

@jden @aplannersguide I turned off builds for pull requests.

Although this could be beneficial, travis.yml can be modified and is respected in a pull request.

I'm doing some custom stuff for deployment in travis.yml, which could be modified in the worst case, to redirect the dataset update to the portal to another dataset. Although, this should be pretty rare since the uploaded CSV's columns has to match the dataset being uploaded to, it could still happen.

Any workarounds you all could think of would be awesome.

[junosuarez]

I usually have my build script do different things depending on whether it's a pull request, eg test for a pull request, test + deploy for a merge / push. This what, you get the benefit of Travis giving feedback on whether the PR is going to be okay to merge.

[seabre]

Right, but I'm thinking of the case that a user either unknowingly/maliciously modifies .travis.yml in a pull request

As is now, if I had Travis building PRs, a user could send a PR that modifies the after_success hook to modify other datasets owned by the deployment user.

[seabre]

But yeah, I agree. The feedback from Travis is really really valuable.

I could strip-out the automated deployment and do automated updates another way. I'd like to keep both though, since it's a really really nice workflow.

[aplannersguide]

@seabre @jden Just checking to see where this is at right now? Is the portal updating when pull requests are merged in this repo?

[seabre]

@aplannersguide Yes.

If the test fails, the updated data will not deploy to the data portal. If the test passes, it will deploy. Both of these scenarios is what we want to happen, but we also want to test pull requests to help see if they are good for merging.

Due to the issue I noted though, I've turned testing pull requests off. I want to turn this back on, but a workaround for the issue needs to be found.

I might try out CircleCI and see if I can use them in a way that can work around it.