Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TNL-782 each request should have valid ID Token #2

Closed
wants to merge 65 commits into from
Closed
Show file tree
Hide file tree
Changes from 64 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
e237e94
Create app.
olmar Nov 12, 2014
96f8e2d
Add gitignore.
olmar Nov 12, 2014
95765d5
Use annotator store in views directly.
olmar Nov 14, 2014
6e9d2a7
Add empty licence and other.
olmar Nov 14, 2014
0029c07
Use cors and refactor.
olmar Nov 17, 2014
8b380ad
Add tests.
olmar Nov 18, 2014
06c069d
Use EdX Repository Skeleton.
olmar Nov 18, 2014
988a3e4
Add more tests.
olmar Nov 18, 2014
97d8ede
Add more tests.
olmar Nov 18, 2014
d9cfbc6
Add server test.
olmar Nov 19, 2014
0d7e901
Fix test.
olmar Nov 19, 2014
693a928
Remove unused import.
olmar Nov 19, 2014
c475d4f
Permission class for checking OAuth token
tymofij Nov 19, 2014
a849820
Add travis yml.
olmar Nov 19, 2014
c70bd5d
make things work (through refusing everyone) when DB is not available
tymofij Nov 20, 2014
df5f511
chmod a+x ./manage.py
tymofij Nov 20, 2014
0ef2b13
change token check to look for X-Annotator-Auth-Token
tymofij Nov 20, 2014
782ed10
Fix tests.
olmar Nov 20, 2014
0f77d84
Remove django_nose from common setting.
olmar Nov 20, 2014
b2bb6ca
Move test staff.
olmar Nov 20, 2014
78a1e92
Add debug setting to prod.
olmar Nov 20, 2014
6a032e8
Add test requirements to travis.
olmar Nov 21, 2014
976de54
Remove mysql-python.
olmar Nov 21, 2014
f977dfe
Add coverage for test run.
olmar Nov 21, 2014
3dd0846
Add ES setting to test.
olmar Nov 21, 2014
b834963
Allow changing ES host and index.
olmar Nov 24, 2014
a8aaab7
Add test.
olmar Nov 24, 2014
26854cd
Use own convert str.
olmar Nov 24, 2014
d97c1ab
Add coverage config.
olmar Nov 24, 2014
e83952d
Fix pep8 violations.
olmar Nov 24, 2014
f467f74
Add build and coveralls badges.
olmar Nov 24, 2014
44e717b
Add pylint.
olmar Nov 24, 2014
296d3a0
Fix travis file.
olmar Nov 24, 2014
03aa08c
Add diff coverage.
olmar Nov 25, 2014
97bc56c
Fix travis to work with diff-cover.
olmar Nov 25, 2014
4226868
Overwrite default annotator-store settings.
olmar Nov 25, 2014
a481a49
Add create index command.
olmar Nov 25, 2014
310973e
Add make command.
olmar Nov 25, 2014
7464dc8
Fix docstring.
olmar Nov 26, 2014
c89ad45
Rename convert.
olmar Nov 26, 2014
ed62505
Remove refresh parameter.
olmar Nov 26, 2014
6884926
Ignore docstrings in tests when fail.
olmar Nov 26, 2014
ccf6a6c
Return bad request when trying to update by create API.
olmar Nov 27, 2014
d9f39f8
Add more checking response statuses.
olmar Nov 27, 2014
ce8abdb
Add build dir.
olmar Nov 27, 2014
5d6d09f
Add develop target.
olmar Nov 27, 2014
3f736ef
Remove path dependency.
olmar Nov 27, 2014
9dc382f
Use validate.
olmar Nov 27, 2014
fdb3abe
Move annottaor-store overwrite to django settings.
olmar Nov 27, 2014
e7254a6
Add default and max size settings.
olmar Nov 28, 2014
70d57d9
Add logging.
olmar Dec 1, 2014
943234b
Add es to dev settings.
olmar Dec 1, 2014
75cad0b
Use slash in search endpoint.
olmar Dec 1, 2014
6a2ed52
Test search ordering.
olmar Dec 1, 2014
37024de
remove permission work from initial PR
tymofij Dec 1, 2014
9bbf32e
Update doc.
olmar Dec 2, 2014
e0eacd9
Add comment to secret key setting.
olmar Dec 2, 2014
ee1dec2
Use query params in list all notes.
olmar Dec 2, 2014
440e47f
Use request from instance.
olmar Dec 2, 2014
b75f00f
Check for valid ID Token
tymofij Dec 2, 2014
acce93e
pep8
tymofij Dec 3, 2014
aa3c503
set default permission class to HasAccessToken
tymofij Dec 3, 2014
164535b
improved docstring for HasAccessToken Permission class
tymofij Dec 3, 2014
54091d2
remove unused test mocks
tymofij Dec 3, 2014
e0c80ab
addressing comments
tymofij Dec 4, 2014
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .coveragerc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[run]
omit = notesserver/settings*
*wsgi.py
7 changes: 7 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Python artifacts
*.pyc

# Tests / Coverage reports
.coverage
.tox
coverage/
3 changes: 3 additions & 0 deletions .pep8
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[pep8]
ignore=E501
max_line_length=119
16 changes: 16 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
language: python
python: 2.7

services:
- elasticsearch

install:
- pip install -r requirements/test.txt
- git fetch origin master:refs/remotes/origin/master # https://github.com/edx/diff-cover#troubleshooting
- pip install coveralls

script:
- make validate

after_success:
- coveralls
1 change: 1 addition & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Oleg Marshev <oleg@edx.org>
9 changes: 9 additions & 0 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
How To Contribute
=================

Contributions are very welcome.

Please read `How To Contribute <https://github.com/edx/edx-platform/blob/master/CONTRIBUTING.rst>`_ for details.

Even though it was written with ``edx-platform`` in mind, the guidelines
should be followed for Open edX code in general.
671 changes: 671 additions & 0 deletions LICENSE.TXT

Large diffs are not rendered by default.

46 changes: 46 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
PACKAGES = notesserver notesapi
.PHONY: requirements

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this allows make requirement to work, otherwise it will just stop seeing requirements directory exists.


validate: test.requirements test coverage

test: clean
./manage.py test --settings=notesserver.settings.test --with-coverage --with-ignore-docstrings \
--exclude-dir=notesserver/settings --cover-inclusive --cover-branches \
--cover-html --cover-html-dir=build/coverage/html/ \
--cover-xml --cover-xml-file=build/coverage/coverage.xml \
$(foreach package,$(PACKAGES),--cover-package=$(package)) \
$(PACKAGES)

run:
./manage.py runserver 0.0.0.0:8042

shell:
./manage.py shell

clean:
coverage erase

quality:
pep8 --config=.pep8 $(PACKAGES)
pylint $(PACKAGES)

diff-coverage:
diff-cover build/coverage/coverage.xml --html-report build/coverage/diff_cover.html

diff-quality:
diff-quality --violations=pep8 --html-report build/coverage/diff_quality_pep8.html
diff-quality --violations=pylint --html-report build/coverage/diff_quality_pylint.html

coverage: diff-coverage diff-quality

create-index:
python manage.py create_index

requirements:
pip install -q -r requirements/base.txt --exists-action=w

test.requirements: requirements
pip install -q -r requirements/test.txt --exists-action=w

develop: test.requirements
pip install -q -r requirements/local.txt --exists-action=w
80 changes: 80 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
Part of `edX code`__.

__ http://code.edx.org/

edX Student Notes API |build-status| |coverage-status|
======================================================

This is a backend store for edX Student Notes.

Overview
--------

The edX Notes API is designed to be compatible with the
`Annotator <http://annotatorjs.org/>`__.

Getting Started
---------------

1. You'll need a recent version `ElasticSearch <http://elasticsearch.org>`__ (>=1.0.0)
installed.

2. Install the requirements:

::

$ make develop

3. Create index and put mapping:

::

$ make create-index

4. Run the server:

::

$ make run

Running Tests
-------------

Run ``make validate`` install the requirements, run the tests, and run
lint.

License
-------

The code in this repository is licensed under version 3 of the AGPL unless
otherwise noted.

Please see ``LICENSE.txt`` for details.

How To Contribute
-----------------

Contributions are very welcome.

Please read `How To Contribute <https://github.com/edx/edx-platform/blob/master/CONTRIBUTING.rst>`_ for details.

Even though it was written with ``edx-platform`` in mind, the guidelines
should be followed for Open edX code in general.

Reporting Security Issues
-------------------------

Please do not report security issues in public. Please email security@edx.org

Mailing List and IRC Channel
----------------------------

You can discuss this code on the `edx-code Google Group`__ or in the
``edx-code`` IRC channel on Freenode.

__ https://groups.google.com/forum/#!forum/edx-code

.. |build-status| image:: https://travis-ci.org/edx/edx-notes-api.svg?branch=master
:target: https://travis-ci.org/edx/edx-notes-api
.. |coverage-status| image:: https://coveralls.io/repos/edx/edx-notes-api/badge.png?branch=master
:target: https://coveralls.io/r/edx/edx-notes-api?branch=master
10 changes: 10 additions & 0 deletions manage.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env python
import os
import sys

if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "notesserver.settings.dev")

from django.core.management import execute_from_command_line

execute_from_command_line(sys.argv)
File renamed without changes.
Empty file added notesapi/management/__init__.py
Empty file.
Empty file.
9 changes: 9 additions & 0 deletions notesapi/management/commands/create_index.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
from django.core.management.base import BaseCommand
from annotator.annotation import Annotation


class Command(BaseCommand):
help = 'Creates the mapping in the index.'

def handle(self, *args, **options):
Annotation.create_all()
6 changes: 6 additions & 0 deletions notesapi/urls.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
from django.conf.urls import patterns, url, include

urlpatterns = patterns(
'',
url(r'^v1/', include('notesapi.v1.urls', namespace='v1')),
)
Empty file added notesapi/v1/__init__.py
Empty file.
58 changes: 58 additions & 0 deletions notesapi/v1/permissions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import jwt
import logging
from django.conf import settings
from rest_framework.permissions import BasePermission

logger = logging.getLogger(__name__)


class TokenWrongIssuer(Exception):
pass


class HasAccessToken(BasePermission):
"""
Allow requests having valid ID Token.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also leave a comment about raw token format?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added


https://tools.ietf.org/html/draft-ietf-oauth-json-web-token-31
Expected Token:
Header {
"alg": "HS256",
"typ": "JWT"
}
Claims {
"sub": "<USER_ID>",
"exp": <EXPIRATION TIMESTAMP>,
"iat": <ISSUED TIMESTAMP>,
"aud": "<CLIENT ID"
}
Should be signed with CLIENT_SECRET
"""
def has_permission(self, request, view):
if getattr(settings, 'DISABLE_TOKEN_CHECK', False):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this app does not use django.contrib.auth and does not even have sql database configured, so checking request.user is useless.

return True
token = request.META.get('HTTP_X_ANNOTATOR_AUTH_TOKEN', '')
if not token:
logger.debug("No token found in headers")
return False
try:
data = jwt.decode(token, settings.CLIENT_SECRET)
auth_user = data['sub']
if data['aud'] != settings.CLIENT_ID:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to raise ImproperlyConfigured exception if settings.CLIENT_ID , settings.CLIENT_SECRET were not defined for some reasons?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think so. They are expected to be always present, and their default values are good for developing the application.

raise TokenWrongIssuer
for request_field in ('GET', 'POST', 'DATA'):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we do not use django.contrib.auth here, we do not even have users or sessions, we do not set cookies. In fact, we do not have database at all.

if 'user' in getattr(request, request_field):
req_user = getattr(request, request_field)['user']
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to move this code in helper method:

def get_user(self, request):
  for request_field in ('GET', 'POST', 'DATA'):
    data = getattr(request, request_field)
    if 'user' in data:
      return data['user']
  return None

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that would requre returning from get_user not unly user but request_field where it was found, for better error log if it does not match. IMHO not worth it.

if req_user == auth_user:
return True
else:
logger.debug("Token user {auth_user} did not match {field} user {req_user}".format(
auth_user=auth_user, field=request_field, req_user=req_user
))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return False explicitly ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

logger.info("No user was present to compare in GET, POST or DATA")
except jwt.ExpiredSignature:
logger.exception("Token was expired: {}".format(token))
except jwt.DecodeError:
logger.exception("Could not decode token {}".format(token))
except TokenWrongIssuer:
logger.exception("Token has wrong issuer {}".format(token))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if these cases need to be logged at all, i'm not sure exception is the best level. any reason not to do logger.debug as you're doing in other places?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my line of thinking was that cases of wrong issue or bad token might be more attention-worthy than just user mismatch, but on a second thought they are not. changing to debug.

Empty file added notesapi/v1/tests/__init__.py
Empty file.
14 changes: 14 additions & 0 deletions notesapi/v1/tests/helpers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import jwt
from calendar import timegm
from datetime import datetime, timedelta
from django.conf import settings


def get_id_token(user):
now = datetime.utcnow()
return jwt.encode({
'aud': settings.CLIENT_ID,
'sub': user,
'iat': timegm(now.utctimetuple()),
'exp': timegm((now + timedelta(seconds=300)).utctimetuple()),
}, settings.CLIENT_SECRET)
Loading