Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Synse Server v3 release #383

Merged
merged 125 commits into from
Mar 30, 2020
Merged
Show file tree
Hide file tree
Changes from 123 commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
6fa0995
run synse-server as a script from python packaging (#261)
edaniszewski Feb 4, 2019
e49993b
rename python package to 'synse_server' (#262)
edaniszewski Feb 4, 2019
602be42
use structured logging (#265)
edaniszewski Feb 4, 2019
05d13b4
add wrapper around application constructor (#266)
edaniszewski Feb 5, 2019
1812d5b
remove internal error id, simplify errors and error handler (#267)
edaniszewski Feb 5, 2019
ae281d4
simplify/update development flow (#268)
edaniszewski Feb 5, 2019
6127d0d
frame out the v3 API (#269)
edaniszewski Feb 5, 2019
c0e9f69
streamline docker build flow and image caching (#270)
edaniszewski Feb 7, 2019
487b409
various cleanup and reorganization work items (#271)
edaniszewski Feb 7, 2019
7d5497a
unify sanic log output with synse-server log output (#272)
edaniszewski Feb 7, 2019
be4e1ee
add favicon to application (#275)
edaniszewski Feb 8, 2019
431f54c
fix minor typo on websocket api proposal (#278)
Mar 1, 2019
0c0a35a
Fix proposal's websocket specs (#279)
Mar 7, 2019
31bd603
add ammendment-1 (device fqdn and aliases) to v3 proposals (#276)
edaniszewski Mar 8, 2019
5ac8fe1
Add some more minor fix to the proposal's websocket specs (#280)
Mar 11, 2019
29e3fa6
remove unit conversion capabilities from the server side (#281)
edaniszewski Mar 21, 2019
b329a28
cleanup / simplift api version definition and usage (#282)
edaniszewski Apr 3, 2019
f0c4940
remove the grpc client defined in synse server, use the new one defin…
edaniszewski Apr 4, 2019
b04c937
update internal modeling for plugins (#287)
edaniszewski Apr 5, 2019
73d04c8
update cache for tag based routing (#288)
edaniszewski Apr 5, 2019
d50d548
various incremental updates (#289)
edaniszewski Apr 5, 2019
8827451
various updates and cleanup (#290)
edaniszewski Apr 5, 2019
7e9194b
v3 HTTP API initial implementation (#291)
edaniszewski Apr 11, 2019
e81ddb1
v3 HTTP API development (#292)
edaniszewski Apr 12, 2019
7a63de7
add application monitoring capabilities, add example dashboard (#293)
edaniszewski Apr 18, 2019
24adb5b
update some error handling, various cleanup items (#294)
edaniszewski Apr 19, 2019
796f2ff
add support for ssl certs (#295)
edaniszewski Apr 19, 2019
9624f0e
update docker deployment example for v3 config (#299)
edaniszewski Apr 29, 2019
b280599
rebuild the unit test suite for synse server (#298)
edaniszewski Apr 29, 2019
39e550e
remove json logger, use kv logger as it is more human readable (#300)
edaniszewski Apr 29, 2019
b3f408a
update setup to render readme properly on pypi (#301)
edaniszewski Apr 29, 2019
4601ef2
add a cache for device alias lookups (#303)
edaniszewski May 2, 2019
21f2cda
Add WebSocket API (#304)
edaniszewski May 13, 2019
b60dc8d
update config for synse v3 (#305)
edaniszewski May 14, 2019
a1d149f
remove baked in emulator from docker image (#307)
edaniszewski May 16, 2019
f48b3df
fix background task registration (#308)
edaniszewski May 16, 2019
01c50d8
updates and adds tests to websocket api (#310)
edaniszewski May 20, 2019
ad251e1
remove documentation build + artifacts - moving to central docs repo …
edaniszewski May 20, 2019
f6d5c14
cleanup and organization (#312)
edaniszewski May 20, 2019
19304e0
remove proposals - migrated to synse repo (#313)
edaniszewski May 29, 2019
7eb31a2
add /device as an alias for /scan for api consistency (#314)
edaniszewski May 30, 2019
2e7ae7e
general cleanup (#315)
edaniszewski May 30, 2019
edcff96
fix typo
edaniszewski Jun 12, 2019
0a07005
update readme for v3 (#316)
edaniszewski Jun 13, 2019
f33648b
update for v3.0.0-alpha.1 prerelease (#319)
edaniszewski Jul 5, 2019
cc63808
fix build script to allow for prerelease image tags
edaniszewski Jul 5, 2019
0c43f74
run synse server as a non-root user (#320)
edaniszewski Jul 9, 2019
df0f55f
[v3.0.0-alpha.2] minor logging and cleanup updates
edaniszewski Jul 16, 2019
4d7c982
websocket streamed readings (#322)
edaniszewski Jul 26, 2019
5363a47
bump to v3.0.0-alpha.3
edaniszewski Jul 26, 2019
552aee2
fix bug in release script which prevented images from being pushed (#…
edaniszewski Aug 6, 2019
a9006e0
fix a typo in the build release script (#328)
edaniszewski Aug 6, 2019
2255041
update synse-grpc to 3.0.0-alpha.3 for proper b64 decode on grpc writ…
edaniszewski Aug 7, 2019
beb76cc
workaround for tox not invalidating cache when reqs file changes (#332)
edaniszewski Aug 8, 2019
7eab273
update tag query behavior to allow for multiple tag groups (#334)
edaniszewski Aug 9, 2019
137a709
bump to v3.0.0-alpha.6
edaniszewski Aug 9, 2019
4ea0665
bugfix: fix http streamed response chunking, update deps
edaniszewski Aug 19, 2019
a93e389
update unit tests for streamed response fix
edaniszewski Aug 19, 2019
0b8f290
bump to v3.0.0-alpha.7
edaniszewski Aug 19, 2019
6bb1353
add metadata to scan output (#335)
edaniszewski Aug 29, 2019
0007565
formatting for websocket payload for better logging
edaniszewski Sep 9, 2019
c321feb
various bug fixes related to websocket api
edaniszewski Oct 11, 2019
30c3e04
bump to 3.0.0-alpha.9
edaniszewski Oct 11, 2019
67d8c40
allow metrics export config to be set via env
edaniszewski Oct 18, 2019
dd7c93f
bump to 3.0.0-alpha.10
edaniszewski Oct 18, 2019
b5da3e4
update sanic-prometheus to use full url label
edaniszewski Oct 22, 2019
e75e818
initial implementation of custom application metrics handlers
edaniszewski Oct 24, 2019
b5106e7
add response bytes counter
edaniszewski Oct 24, 2019
9540619
add application metrics for websocket requests/responses
edaniszewski Oct 24, 2019
44790c6
add more metrics, update metric docs, fix ws error metric reporting
edaniszewski Oct 25, 2019
0be6eb6
remove unused import
edaniszewski Oct 25, 2019
a98fcad
update for basic grpc metrics
edaniszewski Oct 25, 2019
17e2139
fix tests
edaniszewski Oct 25, 2019
301f62f
exclude tests from the release package
edaniszewski Oct 25, 2019
85353bd
bump to 3.0.0-alpha.11
edaniszewski Oct 28, 2019
81fe652
fix bug in websocket close callback
edaniszewski Oct 31, 2019
ed808e4
bugfix: cannot get response size from body attr for streamed responses
edaniszewski Nov 7, 2019
508c76a
bump version to 3.0.0-alpha.12
edaniszewski Nov 8, 2019
39b912f
update and add logging messages
edaniszewski Nov 15, 2019
4ea3575
update i18n messages
edaniszewski Nov 15, 2019
290bdfd
add type hinting, update docstrings
edaniszewski Nov 18, 2019
d66e328
fix various bugs around tag processing
edaniszewski Nov 20, 2019
cdd0462
bump to 3.0.0-alpha.13
edaniszewski Nov 20, 2019
8e30c9c
update example dashboard - prometheus labels had changed
edaniszewski Dec 13, 2019
e1705ec
add http metric labels, update example dashboard
edaniszewski Dec 13, 2019
4314dc5
update example grafana dashboard
edaniszewski Dec 17, 2019
849b9ea
bump to 3.0.0-alpha.14
edaniszewski Jan 17, 2020
8b8b997
bugfix: update plugin refresh triggers, gate actions on plugin status…
edaniszewski Jan 28, 2020
1dd0f17
bump to 3.0.0-alpha.15
edaniszewski Jan 28, 2020
1bfa851
bugfix: already registered plugin was not being set to active on refresh
edaniszewski Jan 30, 2020
de84a39
rework logging for additional context and to include sanic logs
edaniszewski Jan 30, 2020
a1b404b
add a request ID for all logs generated during request handling
edaniszewski Jan 30, 2020
eb55d11
bump to 3.0.0-alpha.16
edaniszewski Jan 30, 2020
cab2a33
logging improvements around plugin registration
edaniszewski Feb 3, 2020
3c48805
bump to 3.0.0-alpha.17
edaniszewski Feb 3, 2020
4bdcb06
feat: make plugin refresh interval configurable
edaniszewski Feb 5, 2020
f334bfc
feat: add plugin manager util for checking all plugins active
edaniszewski Feb 5, 2020
a4251e4
bugfix: refresh plugins if inactive prior to device cache rebuild
edaniszewski Feb 5, 2020
8e4244f
bump: version 3.0.0-alpha.18
edaniszewski Feb 5, 2020
b4abc5b
bugfix: use a shared loop between sanic and asyncio components
edaniszewski Feb 20, 2020
d9febf9
bugfix: fix issues with txn cache ttl being set as well as rebuild cl…
edaniszewski Feb 21, 2020
8a6dbae
feat: reduce the amount of time the cache lock is active to improve r…
edaniszewski Feb 21, 2020
640c0c7
feat: prevent multiple plugin refreshes from running simultaneously
edaniszewski Feb 21, 2020
3a089bc
bump: version 3.0.0-alpha.19
edaniszewski Feb 21, 2020
6f7f37e
update: remove i18n, as it is unused and implemented inconsistently
edaniszewski Feb 29, 2020
90100d7
feat: update docker image to py38
edaniszewski Feb 29, 2020
675b1c6
feat: update request/reponse logging, disable sanic access log
edaniszewski Mar 2, 2020
bb2d277
feat: update to plugin reconnect mechanics
edaniszewski Mar 4, 2020
2b61d5d
pr review updates
edaniszewski Mar 9, 2020
d379c51
ci: updates to get CI running correctly
edaniszewski Mar 9, 2020
9516eb3
feat: allow reads to be filtered by originating plugin
edaniszewski Mar 10, 2020
46d9ff0
feat: omit an empty 'transaction' field from write/txn reflected context
edaniszewski Mar 10, 2020
a3909c1
test: cleaning up + normalizing test definitions
edaniszewski Mar 11, 2020
37e514f
feat: add metrics export for plugin monitoring, update dashboard
edaniszewski Mar 18, 2020
4fdf91f
cleanup: fix typos, fill out docs, fix warnings, normalize formatting
edaniszewski Mar 19, 2020
944aba0
deps: update project dependencies
edaniszewski Mar 19, 2020
a088e26
bugfix: iterate over plugin client generators within client context t…
edaniszewski Mar 20, 2020
0cd0f70
feat: update plugin refresh to also update active/inactive state
edaniszewski Mar 20, 2020
d2746d5
docs: update project readme
edaniszewski Mar 23, 2020
ae59be1
docs: update readme formatting
edaniszewski Mar 27, 2020
2e37ff3
deps: update project dependencies
edaniszewski Mar 27, 2020
0597d44
bump: version 3.0.0
edaniszewski Mar 27, 2020
60b5376
update: remove temporary workaround for version definition in makefile
edaniszewski Mar 27, 2020
c7bf39d
update: bump copyright date in NOTICE to 2020
edaniszewski Mar 30, 2020
9f0cc41
docs: add section on compatibility
edaniszewski Mar 30, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 5 additions & 17 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,3 @@
#
# .dockerignore.default
#
# The dockerignore file used for default Synse Server image builds.
# By default, Synse Server includes the emulator. For a version of
# Synse Server without the emulator, use the 'slim' build.
#

# JetBrains (PyCharm, WebStorm, etc.)
.idea
*.iws
Expand Down Expand Up @@ -40,32 +32,28 @@ build
dist

# Test Artifacts
results
htmlcov
.jenkins
.coverage
.pytest_cache
tests/testdata
tox.ini

# Logs
logs/
*.log

# Docker
dockerfile
Dockerfile.*
Dockerfile

# Docs
docs
**/*.md
**/*.rst
NOTICE
proposals/

# Scripts
bin/ci

# Other
**/*.png
assets
compose
deploy
**/*.svg
examples
63 changes: 43 additions & 20 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ __pycache__/

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
Expand All @@ -24,6 +23,7 @@ wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
Expand All @@ -35,10 +35,6 @@ wheels/
pip-log.txt
pip-delete-this-directory.txt

# Translations
*.mo
*.pot

# Unit test / coverage reports
htmlcov/
.tox/
Expand All @@ -47,42 +43,69 @@ htmlcov/
.cache
nosetests.xml
coverage.xml
*,cover
*.cover
.hypothesis/
results/
prof/
*.log
.pytest_cache/
tests/testdata/*
!tests/testdata/README.md

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# mkdocs documentation
/site

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule

# dotenv
.env
# SageMath parsed files
*.sage.py

# virtualenv
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

.vagrant
.idea
.DS_Store
# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# packages
*.rpm
*.deb
# mypy
.mypy_cache/

# IDEs / Editors
.idea/

# Other
.DS_Store
docs/build
130 changes: 31 additions & 99 deletions .jenkins
Original file line number Diff line number Diff line change
Expand Up @@ -12,122 +12,56 @@ pipeline {
checkout scm
}
}
// Lint the project source code.

stage('Lint') {
steps {
sh 'make lint'
}
post {
success {
script {
if (env.CHANGE_ID) {
pullRequest.createStatus(
status: 'success',
context: 'vio-build/lint',
description: 'Source code linting succeeded',
targetUrl: "${env.RUN_DISPLAY_URL}"
)
}
}
}
failure {
script {
if (env.CHANGE_ID) {
pullRequest.createStatus(
status: 'failure',
context: 'vio-build/lint',
description: 'Source code linting failed',
targetUrl: "${env.RUN_DISPLAY_URL}"
)
}
}
}
sh 'tox -e lint'
}
}
// Run unit and integration tests, generate and upload coverage report.

stage('Test') {
agent {
label 'python-3.8'
}
environment {
CODECOV_TOKEN = credentials('codecov-token')
}
steps {
sh 'make test-unit'
sh 'codecov'
sh 'make test-integration'
}
post {
success {
script {
if (env.CHANGE_ID) {
pullRequest.createStatus(
status: 'success',
context: 'vio-build/test',
description: 'Tests passed',
targetUrl: "${env.RUN_DISPLAY_URL}"
)
}
}
}
failure {
script {
if (env.CHANGE_ID) {
pullRequest.createStatus(
status: 'success',
context: 'vio-build/test',
description: 'Tests failed',
targetUrl: "${env.RUN_DISPLAY_URL}"
)
}
}
container('python') {
sh 'tox tests/unit'
}
}
}
// Verify that the Docker image builds successfully.
stage('Build Image') {

stage('Verify Image Build') {
steps {
sh 'make docker'
}
post {
success {
script {
if (env.CHANGE_ID) {
pullRequest.createStatus(
status: 'success',
context: 'vio-build/build-image',
description: 'Successfully built docker image',
targetUrl: "${env.RUN_DISPLAY_URL}"
)
}
}
}
failure {
script {
if (env.CHANGE_ID) {
pullRequest.createStatus(
status: 'success',
context: 'vio-build/build-image',
description: 'Failed to build docker image',
targetUrl: "${env.RUN_DISPLAY_URL}"
)
}
}
}
sh '''
docker build \
--label build_date=$(date -u +%Y-%m-%dT%T 2> /dev/null) \
--label version=$(python setup.py --version) \
--label commit=$(git rev-parse --short HEAD 2> /dev/null || true) \
-t ${IMAGE_NAME}:canary .
'''
}
}
// Build the image with the 'edge' tag and publish it to DockerHub. This
// should only be run on the master branch (e.g. PR merge)
stage('Publish Edge Image') {

stage('Publish "latest" Image (master)') {
when {
branch 'master'
}
steps {
withDockerRegistry(registry: [credentialsId: 'vio-docker-hub']) {
sh './bin/ci/build_and_publish.sh edge'
sh '''
docker build \
--label build_date=$(date -u +%Y-%m-%dT%T 2> /dev/null) \
--label version=$(python setup.py --version) \
--label commit=$(git rev-parse --short HEAD 2> /dev/null || true) \
-t ${IMAGE_NAME}:latest .
'''
}
}
}
// Generate a new release draft on GitHub for a tag matching a version string
// pattern. The release will include an auto-generated changelog and build
// artifacts.

stage('Draft GitHub Release') {
when {
buildingTag()
Expand All @@ -138,8 +72,8 @@ pipeline {
GITHUB_REPONAME = 'synse-server'
}
steps {
sh 'make ci-check-version'
sh 'make ci-package'
// Build package distribution
sh 'python setup.py sdist --formats=gztar,zip'

// Auto-generate a changelog for the release
sh './bin/ci/generate_changelog.sh'
Expand All @@ -149,19 +83,17 @@ pipeline {
sh 'docker run --rm -v $PWD:/repo edaniszewski/ghr -u ${GITHUB_USER} -r ${GITHUB_REPONAME} -t ${GITHUB_TOKEN} -b "$(cat ./CHANGELOG.md)" -replace -draft ${TAG_NAME} dist'
}
}
// Build the image with the release tags and publish it to Dockerhub. This
// should only be run on a git tag matching a version string pattern. This
// should only run if the "Draft GitHub Release" stage completed successfully.

stage('Publish Release Images') {
when {
buildingTag()
}
steps {
sh 'make ci-check-version'
withDockerRegistry(registry: [credentialsId: 'vio-docker-hub']) {
sh './bin/ci/build_and_publish.sh latest'
}
}
}

}
}
Loading