Skip to content
This repository has been archived by the owner on Oct 29, 2021. It is now read-only.

Catch-up from lacuna-tech/mds-core #249

Merged
merged 537 commits into from
May 21, 2020

Conversation

marie-x
Copy link
Collaborator

@marie-x marie-x commented Mar 3, 2020

Draft catch-up PR that will bring the original LADOT contribution up to date with the current deployed system.

Removes some outdated libraries and not-ready-for-contribution code (e.g. reports). Refactors Policy into Policy and Geography. Separate PR coming to mobility-data-specification for Geography specification.

avatarneil and others added 30 commits November 12, 2019 15:00
* Fix weird array construction bug in metrics API binning method

* Remove async modifier

* Cleanup and add state_snapshot test

* Cleanup
 - @container-images/env-inject@0.1.21
 - @container-images/mds-agency@0.1.22
 - @container-images/mds-audit@0.1.23
 - @container-images/mds-compliance@0.1.22
 - @container-images/mds-daily@0.1.22
 - @container-images/mds-metrics-sheet@0.1.23
 - @container-images/mds-metrics@0.0.4
 - @container-images/mds-native@0.0.20
 - @container-images/mds-policy-author@0.1.22
 - @container-images/mds-policy@0.1.22
 - @container-images/mds-provider@0.1.22
 - @mds-core/mds-agency@0.0.24
 - @mds-core/mds-api-authorizer@0.1.22
 - @mds-core/mds-api-helpers@0.1.22
 - @mds-core/mds-api-server@0.1.22
 - @mds-core/mds-audit@0.1.34
 - @mds-core/mds-cache@0.1.22
 - @mds-core/mds-compliance@0.1.23
 - @mds-core/mds-daily@0.0.23
 - @mds-core/mds-db@0.1.22
 - @mds-core/mds-logger@0.1.20
 - @mds-core/mds-metrics-sheet@0.0.23
 - @mds-core/mds-metrics@0.0.4
 - @mds-core/mds-native@0.0.20
 - @mds-core/mds-policy-author@0.0.19
 - @mds-core/mds-policy@0.0.23
 - @mds-core/mds-provider@1.0.22
 - @mds-core/mds-providers@0.1.22
 - @mds-core/mds-stream@0.1.22
 - @mds-core/mds-test-data@0.1.22
 - @mds-core/mds-types@0.1.19
 - @mds-core/mds-utils@0.1.22
 - @container-images/env-inject@0.1.22
 - @container-images/mds-agency@0.1.23
 - @container-images/mds-audit@0.1.24
 - @container-images/mds-compliance@0.1.23
 - @container-images/mds-daily@0.1.23
 - @container-images/mds-metrics-sheet@0.1.24
 - @container-images/mds-metrics@0.0.5
 - @container-images/mds-native@0.0.21
 - @container-images/mds-policy-author@0.1.23
 - @container-images/mds-policy@0.1.23
 - @container-images/mds-provider@0.1.23
 - @mds-core/mds-agency@0.0.25
 - @mds-core/mds-api-authorizer@0.1.23
 - @mds-core/mds-api-helpers@0.1.23
 - @mds-core/mds-api-server@0.1.23
 - @mds-core/mds-audit@0.1.35
 - @mds-core/mds-cache@0.1.23
 - @mds-core/mds-compliance@0.1.24
 - @mds-core/mds-daily@0.0.24
 - @mds-core/mds-db@0.1.23
 - @mds-core/mds-logger@0.1.21
 - @mds-core/mds-metrics-sheet@0.0.24
 - @mds-core/mds-metrics@0.0.5
 - @mds-core/mds-native@0.0.21
 - @mds-core/mds-policy-author@0.0.20
 - @mds-core/mds-policy@0.0.24
 - @mds-core/mds-provider@1.0.23
 - @mds-core/mds-providers@0.1.23
 - @mds-core/mds-stream@0.1.23
 - @mds-core/mds-test-data@0.1.23
 - @mds-core/mds-types@0.1.20
 - @mds-core/mds-utils@0.1.23
* Fix `editGeography`.

* ts assist

* Fixed `editGeography` with TS assist.
 - @container-images/env-inject@0.1.23
 - @container-images/mds-agency@0.1.24
 - @container-images/mds-audit@0.1.25
 - @container-images/mds-compliance@0.1.24
 - @container-images/mds-daily@0.1.24
 - @container-images/mds-metrics-sheet@0.1.25
 - @container-images/mds-metrics@0.0.6
 - @container-images/mds-native@0.0.22
 - @container-images/mds-policy-author@0.1.24
 - @container-images/mds-policy@0.1.24
 - @container-images/mds-provider@0.1.24
 - @mds-core/mds-agency@0.0.26
 - @mds-core/mds-api-authorizer@0.1.24
 - @mds-core/mds-api-helpers@0.1.24
 - @mds-core/mds-api-server@0.1.24
 - @mds-core/mds-audit@0.1.36
 - @mds-core/mds-cache@0.1.24
 - @mds-core/mds-compliance@0.1.25
 - @mds-core/mds-daily@0.0.25
 - @mds-core/mds-db@0.1.24
 - @mds-core/mds-logger@0.1.22
 - @mds-core/mds-metrics-sheet@0.0.25
 - @mds-core/mds-metrics@0.0.6
 - @mds-core/mds-native@0.0.22
 - @mds-core/mds-policy-author@0.0.21
 - @mds-core/mds-policy@0.0.25
 - @mds-core/mds-provider@1.0.24
 - @mds-core/mds-providers@0.1.24
 - @mds-core/mds-stream@0.1.24
 - @mds-core/mds-test-data@0.1.24
 - @mds-core/mds-types@0.1.21
 - @mds-core/mds-utils@0.1.24
* mdscl/istio: install (un)install via helm straight up

* mdsctl/istio: uninstallIstio cleanup
* Add new tables for audit attachments

* Integrate with audit attachments db tables

* New endpoints to add and delete attachments; return attachments with audits

* Lint

* Minor cleanups to s3 upload key and base url

* Move S3 config to env variables

* Let caller catch writeAttachment exception; type fixes

* Lint
…PI (#42)

* Add MDS vehicle status and location to get audit response

* Add test for provider info in audit response

* Capitalize DESC keyword

* Add provider_event_time to Get Audit response

* Remove references to dropped columns from schema

* Use enums

* Fix import

* Make ORDER BY configurable in readEventsWithTelemetry

* Fix errors when running `yarn image`

* Use optional chaining from TS 3.7
 - @container-images/env-inject@0.1.24
 - @container-images/mds-agency@0.1.25
 - @container-images/mds-audit@0.1.26
 - @container-images/mds-compliance@0.1.25
 - @container-images/mds-daily@0.1.25
 - @container-images/mds-metrics-sheet@0.1.26
 - @container-images/mds-metrics@0.0.7
 - @container-images/mds-native@0.0.23
 - @container-images/mds-policy-author@0.1.25
 - @container-images/mds-policy@0.1.25
 - @container-images/mds-provider@0.1.25
 - @mds-core/mds-agency@0.0.27
 - @mds-core/mds-api-authorizer@0.1.25
 - @mds-core/mds-api-helpers@0.1.25
 - @mds-core/mds-api-server@0.1.25
 - @mds-core/mds-audit@0.1.37
 - @mds-core/mds-cache@0.1.25
 - @mds-core/mds-compliance@0.1.26
 - @mds-core/mds-daily@0.0.26
 - @mds-core/mds-db@0.1.25
 - @mds-core/mds-logger@0.1.23
 - @mds-core/mds-metrics-sheet@0.0.26
 - @mds-core/mds-metrics@0.0.7
 - @mds-core/mds-native@0.0.23
 - @mds-core/mds-policy-author@0.0.22
 - @mds-core/mds-policy@0.0.26
 - @mds-core/mds-provider@1.0.25
 - @mds-core/mds-providers@0.1.25
 - @mds-core/mds-stream@0.1.25
 - @mds-core/mds-test-data@0.1.25
 - @mds-core/mds-types@0.1.22
 - @mds-core/mds-utils@0.1.25
…ary conditions & fluent syntax (#71)

* Overloading log with log level parameter to support ternary conditions & fluent syntax
…code into a new package. (#78)

* Consolidating validator/schema-related code in mds-policy to mds-utils

* Consolidating validator/schema-related code in mds-compliance to mds-utils

* Consolidating validator/schema-related code in mds-compliance to mds-utils

* extracting JSON Schema definitions and code into mds-json-schema

* fixing build errors caused by moving validation to new package

* renaming mds-json-schema -> mds-schema-validators and cleaning up build issues

* moving ValidationError back to mds-utils

* removing hapi/joi from mds-utils

* updating tsconfigs

* Referencing RULE_TYPES when constructing schemas

* Revert "Referencing RULE_TYPES when constructing schemas"

This reverts commit 87def6e.

* Referencing RULE_TYPES when constructing schemas
janedotx and others added 21 commits May 15, 2020 08:07
Co-authored-by: Mark Maxham <max@ellis-and-associates.com>
* Query for policy by rule_id.

* Adding ability to see Policies active at a particular time.

* Refactored snapshot logic.

* Adding test to ensure timestamp paramenter works for count endpoint

* minor fixes

* Making policy rules unique and refactoring compliance api logic to reflect that, and renaming `end_date` param to `timestamp`.

* repairing test cases that broke because rules cannot be repeated

* more consistent error handling and better active policies and rule uniqueness checks

* Versioning and fixing the definition of an active policy

* minor fixes

* minor fixes

Co-authored-by: Mark Maxham <max@ellis-and-associates.com>
…, improve error handling generally (#322)

* Add global error handling middleware, improve error handling generally

* Undo

* Remove TODOs

Co-authored-by: janedotx <jane.x.huang@gmail.com>
…th general v… (#331)

* Removing versiono param from agency payload to conform with general versioninig policy.

* Updating ApiResponse

* Better agency types.
…fy a publish_date (#328)

* Update geography schema to not allow unknown properties, update policy and geography validators to not allow unknown properties, add tests to verify POST and PUT that specify a publish_date are rejected

* Add more specific checks to the tests

Co-authored-by: janedotx <jane.x.huang@gmail.com>
…ions (#332)

* WIP

* Distil down versioning middlewares, add tests

* Clean up tests
…thor and geo services (#335)

* Adding data envelop and correcting mds-policy-author version

* Add data envelop to geo service and correcting version number.

* More data envelop for policy-author stuff.

* Version corrected and data envelopes added to mds-geography-author.
* Fix PR Template

* Minor correction to text in CONTRIBUTING

Co-authored-by: Mark Maxham <max@ellis-and-associates.com>
…334)

Co-authored-by: Mark Maxham <max@ellis-and-associates.com>
…that somehow got through CI/CD, ES2020->ES2019 downgrade (#337)

* Add missing data wrapper in geography-author that somehow got through CI/CD

* Revert from ES2020 -> ES2019

Co-authored-by: janedotx <jane.x.huang@gmail.com>
Allow entities to be configurable while maintaining tight subscription/filtering
* patch helm-init issue

* patch helm init
Co-authored-by: Mark Maxham <max@ellis-and-associates.com>
Copy link
Member

@schnuerle schnuerle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving this as a 0.1 beta release for the OMF for the dev branch. The code has not been tested by OMF with live MDS data feeds. The purpose of this release is to allow OMF members to test mds-core in a secure environment and provide technical, security, documentation, and feature feedback.

@schnuerle schnuerle changed the base branch from develop to dev May 21, 2020 16:26
@schnuerle schnuerle self-assigned this May 21, 2020
@schnuerle schnuerle added the beta a feature that is under development and testing label May 21, 2020
@schnuerle schnuerle added this to the OMF Contribution milestone May 21, 2020
@schnuerle schnuerle linked an issue May 21, 2020 that may be closed by this pull request
@schnuerle schnuerle merged commit f2e8a81 into openmobilityfoundation:dev May 21, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
beta a feature that is under development and testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PROPOSAL: mds-core transition to OMF City Services Working group