Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds support for copying plios from personal workspace to organisational workspace #292

Merged
merged 58 commits into from
Jan 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
951ed82
DUMMY change
dalmia Jun 27, 2021
e22034f
Merge branch 'release' of https://github.com/avantifellows/plio-backe…
dalmia Jun 27, 2021
d9fdf59
Revert change done for deployment
dalmia Jun 27, 2021
655285f
Merge branch 'master' into release
dalmia Jun 30, 2021
c08aafa
Merge branch 'master' into release
dalmia Jul 1, 2021
4b11777
Merge branch 'master' into release
dalmia Jul 2, 2021
9e6350e
Merge branch 'master' into release
deepansh96 Jul 3, 2021
58f9ef5
Merge branch 'master' into release
deepansh96 Jul 16, 2021
8fcd7b1
FIX: active workspace info was not being passed during SSO call
deepansh96 Aug 1, 2021
c4a9901
Merge pull request #222 from avantifellows/sso_bug_fix
deepansh96 Aug 1, 2021
de5c74f
Merge branch 'master' into release
deepansh96 Aug 1, 2021
e143b6b
Merge branch 'master' of https://github.com/avantifellows/plio-backend
deepansh96 Aug 26, 2021
65860f2
Merge branch 'release' of https://github.com/avantifellows/plio-backe…
deepansh96 Aug 26, 2021
28c358a
Merge branch 'master' into release
deepansh96 Aug 26, 2021
13b6375
FIX: master and release were different
deepansh96 Aug 26, 2021
c66a7f7
Merge branch 'master' into release
dalmia Sep 3, 2021
f4dc5d1
Merge branch 'master' into release
dalmia Sep 7, 2021
7c66c87
Merge branch 'master' into release
dalmia Sep 7, 2021
e2a47f8
Merge branch 'master' into release
dalmia Sep 7, 2021
6ae64df
Merge branch 'master' into release
dalmia Sep 15, 2021
649507c
Merge branch 'master' into release
dalmia Sep 23, 2021
d92fbe0
Merge branch 'master' into release
dalmia Oct 21, 2021
079abe2
Merge branch 'master' into release
dalmia Oct 21, 2021
5bc48d6
Merge branch 'master' into release
dalmia Nov 7, 2021
01c5bd8
Merge branch 'master' into release
deepansh96 Nov 25, 2021
881c753
Merge branch 'master' into release
dalmia Dec 3, 2021
58b2ec9
Merge branch 'master' into release
deepansh96 Dec 6, 2021
ce4f922
Merge branch 'master' into release
dalmia Dec 7, 2021
0fba268
Merge branch 'master' into release
dalmia Dec 10, 2021
ebd0488
Merge branch 'master' into release
dalmia Dec 15, 2021
6801e7a
Merge branch 'master' into release
dalmia Dec 17, 2021
41e982c
Merge: branch master into release
dalmia Dec 21, 2021
ef0d9f5
Ghost commit for redeployment
deepansh96 Dec 31, 2021
9571336
Merge branch 'master' into release
dalmia Jan 5, 2022
bf9de2d
NEW: added endpoints for copying video and plio to another workspace
dalmia Jan 7, 2022
24622e6
FIX: ensure that the copied plio is in draft stage
dalmia Jan 10, 2022
d6337e1
NEW: support for copying items
dalmia Jan 10, 2022
7d7cca4
NEW: support for copying questions (except question with images)
dalmia Jan 10, 2022
690ba13
FIX: clear cache for plio upon creating items
dalmia Jan 11, 2022
e01526a
FIX: clear cache for plio upon creating items
dalmia Jan 11, 2022
ef774ac
NEW: support for copying images in questions too
dalmia Jan 11, 2022
06b260c
FIX: pep8 issues
dalmia Jan 11, 2022
38955e4
FIX: extra whitesapce
dalmia Jan 11, 2022
fa4c798
NEW: added tests for video
dalmia Jan 11, 2022
dc9798b
NEW: added tests for plio + fixed tests for video
dalmia Jan 11, 2022
d98d9d3
NEW: missing tests for plio
dalmia Jan 11, 2022
a029e91
NEW: added tests for items
dalmia Jan 11, 2022
9b54da9
NEW: added tests for questions
dalmia Jan 12, 2022
ed3c222
NEW: added tests for questions with image
dalmia Jan 12, 2022
08fdd94
FIX: tests for plio to fill missing coverage
dalmia Jan 12, 2022
f23ef4d
FIX: tests for items to fill missing coverage
dalmia Jan 12, 2022
40bbda0
FIX: tests for questions to fill missing coverage
dalmia Jan 12, 2022
e388c0b
FIX: failing tests
dalmia Jan 12, 2022
0f3bf52
FIX: only one api call for copying
dalmia Jan 14, 2022
c237d96
FIX: tests
dalmia Jan 14, 2022
60398b9
NEW: added test for missing coverage
dalmia Jan 14, 2022
5e9175e
DOC: docstring for set_tenant
dalmia Jan 14, 2022
3d2eeb0
FIX: PR feedback
dalmia Jan 19, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions plio/migrations/0029_auto_20220110_1044.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Generated by Django 3.1.1 on 2022-01-10 10:44

from django.db import migrations


class Migration(migrations.Migration):

dependencies = [
("plio", "0028_auto_20210902_1120"),
]

operations = [
migrations.AlterModelOptions(
name="item",
options={"ordering": ["time"]},
),
migrations.AlterModelOptions(
name="question",
options={"ordering": ["item__time"]},
),
]
2 changes: 2 additions & 0 deletions plio/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ class Item(SafeDeleteModel):

class Meta:
db_table = "item"
ordering = ["time"]
dalmia marked this conversation as resolved.
Show resolved Hide resolved

def __str__(self):
return "%d: %s - %s" % (self.id, self.plio.name, self.type)
Expand All @@ -140,3 +141,4 @@ class Question(SafeDeleteModel):

class Meta:
db_table = "question"
ordering = ["item__time"]
9 changes: 5 additions & 4 deletions plio/permissions.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,17 @@


class PlioPermission(permissions.BasePermission):
"""
Permission check for plios.
"""
"""Permission check for plios."""

def has_permission(self, request, view):
"""View-level permissions for plio. This determines whether the request can access plio instances or not."""
return True

def has_object_permission(self, request, view, obj):
"""Object-level permissions for plio/item/question. This determines whether the request can access a plio/item/question instance or not."""
"""
Object-level permissions for plio/item/question.
This determines whether the request can access a plio/item/question instance or not.
"""
if request.user.is_superuser:
return True

Expand Down
120 changes: 120 additions & 0 deletions plio/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
from entries.models import Session
from plio.views import StandardResultsSetPagination
from plio.cache import get_cache_key
from plio.serializers import ImageSerializer


class BaseTestCase(APITestCase):
Expand Down Expand Up @@ -541,6 +542,125 @@ def test_updating_plio_recreates_instance_cache(self):
self.assertEqual(len(cache.keys(cache_key_name)), 1)
self.assertEqual(cache.get(cache_key_name)["name"], new_name)

def test_copying_without_specifying_workspace_fails(self):
response = self.client.post(f"/api/v1/plios/{self.plio_1.uuid}/copy/")
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
dalmia marked this conversation as resolved.
Show resolved Hide resolved
self.assertEqual(response.data["detail"], "workspace is not provided")

def test_copying_to_non_existing_workspace_fails(self):
response = self.client.post(
f"/api/v1/plios/{self.plio_1.uuid}/copy/", {"workspace": "abcd"}
)
self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
self.assertEqual(response.data["detail"], "workspace does not exist")

def test_copying_to_workspace(self):
# create some items and questions
item_1 = Item.objects.create(type="question", plio=self.plio_1, time=1)
item_2 = Item.objects.create(type="question", plio=self.plio_1, time=10)
item_3 = Item.objects.create(type="question", plio=self.plio_1, time=20)

question_1 = Question.objects.create(type="checkbox", item=item_1)
# attach an image to one question
# upload a test image and retrieve the id
with open("plio/static/plio/test_image.jpeg", "rb") as img:
response = self.client.post(
reverse("images-list"), {"url": img, "alt_text": "test image"}
)
image_id = response.json()["id"]
question_2 = Question.objects.create(
type="subjective",
item=item_2,
image=Image.objects.filter(id=image_id).first(),
)
question_3 = Question.objects.create(type="checkbox", item=item_3)

response = self.client.post(
f"/api/v1/plios/{self.plio_1.uuid}/copy/",
{"workspace": self.organization.shortcode},
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
new_plio_id = response.data["id"]
new_plio_uuid = response.data["uuid"]
new_video_id = response.data["video"]["id"]
new_item_ids = []
new_question_ids = []

for item in response.data.get("items", []):
new_item_ids.append(item["id"])
new_question_ids.append(item["details"]["id"])

# check that the instances are actually created in the given workspace
connection.set_schema(self.organization.schema_name)

response = self.client.get(
f"/api/v1/videos/{new_video_id}/",
HTTP_ORGANIZATION=self.organization.shortcode,
)
self.assertEqual(response.status_code, status.HTTP_200_OK)

response = self.client.get(
f"/api/v1/plios/{new_plio_uuid}/",
HTTP_ORGANIZATION=self.organization.shortcode,
)
self.assertEqual(response.status_code, status.HTTP_200_OK)

for new_item_id, old_item in zip(new_item_ids, [item_1, item_2, item_3]):
response = self.client.get(
f"/api/v1/items/{new_item_id}/",
HTTP_ORGANIZATION=self.organization.shortcode,
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["plio"], new_plio_id)
self.assertEqual(response.data["type"], old_item.type)
self.assertEqual(response.data["time"], old_item.time)

for old_question, new_question_id, new_item_id in zip(
[question_1, question_2, question_3],
new_question_ids,
new_item_ids,
):
response = self.client.get(
f"/api/v1/questions/{new_question_id}/",
HTTP_ORGANIZATION=self.organization.shortcode,
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["item"], new_item_id)
self.assertEqual(response.data["type"], old_question.type)

if old_question.image is not None:
self.assertEqual(
ImageSerializer(old_question.image).data["url"],
response.data["image"]["url"],
)

# set db connection back to public (default) schema
connection.set_schema_to_public()

def test_copying_to_workspace_with_no_video(self):
plio = Plio.objects.create(
name="Plio 1", created_by=self.user, status="published"
)
response = self.client.post(
f"/api/v1/plios/{plio.uuid}/copy/",
{"workspace": self.organization.shortcode},
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
new_plio_uuid = response.data["uuid"]

# check that the instance is actually created in the given workspace
connection.set_schema(self.organization.schema_name)

response = self.client.get(
f"/api/v1/plios/{new_plio_uuid}/",
HTTP_ORGANIZATION=self.organization.shortcode,
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["video"]["url"], "")

# set db connection back to public (default) schema
connection.set_schema_to_public()


class PlioDownloadTestCase(BaseTestCase):
def setUp(self):
Expand Down
120 changes: 120 additions & 0 deletions plio/views.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
from django.db import connection
from django.db.models import Q
from django.http import FileResponse

from django_tenants.utils import get_tenant_model

import pandas as pd
from storages.backends.s3boto3 import S3Boto3Storage

Expand Down Expand Up @@ -42,6 +45,7 @@
)
from plio.permissions import PlioPermission
from plio.ordering import CustomOrderingFilter
from plio.cache import invalidate_cache_for_instance


class StandardResultsSetPagination(PageNumberPagination):
Expand Down Expand Up @@ -70,6 +74,23 @@ def get_paginated_response(self, params):
)


def set_tenant(workspace: str):
"""
Sets the current tenant to the given workspace if it exists

:param workspace: workspace shortcode to use as the current tenant
:type workspace: str
"""
tenant_model = get_tenant_model()
tenant = tenant_model.objects.filter(shortcode=workspace).first()

if not tenant:
return False

connection.set_tenant(tenant)
return True


class VideoViewSet(viewsets.ModelViewSet):
"""
Video ViewSet description
Expand All @@ -84,6 +105,7 @@ class VideoViewSet(viewsets.ModelViewSet):

queryset = Video.objects.all()
serializer_class = VideoSerializer
permission_classes = [IsAuthenticated]


class PlioViewSet(viewsets.ModelViewSet):
Expand Down Expand Up @@ -215,6 +237,104 @@ def duplicate(self, request, uuid):
plio.save()
return Response(self.get_serializer(plio).data)

@action(
methods=["post"],
detail=True,
permission_classes=[IsAuthenticated, PlioPermission],
)
def copy(self, request, uuid):
"""Copies the given plio to another workspace"""
if "workspace" not in request.data:
return Response(
{"detail": "workspace is not provided"},
status=status.HTTP_400_BAD_REQUEST,
)

# return 404 if user cannot access the object
# else fetch the object
plio = self.get_object()

if plio.video is not None:
video = Video.objects.filter(id=plio.video.id).first()
video.pk = None

items = list(Item.objects.filter(plio__id=plio.id))
questions = list(Question.objects.filter(item__plio__id=plio.id))

# will be needed to handle questions with images
question_indices_with_image = []
images = []

for index, question in enumerate(questions):
if question.image is not None:
question_indices_with_image.append(index)
images.append(question.image)
else:
video = None
items = []
questions = []

# django will auto-generate the key when the key is set to None
plio.pk = None
plio.uuid = None
plio.status = "draft"

# change workspace
workspace = request.data.get("workspace")
success = set_tenant(workspace)

if not success:
return Response(
{"detail": "workspace does not exist"},
status=status.HTTP_404_NOT_FOUND,
)
dalmia marked this conversation as resolved.
Show resolved Hide resolved

if video is not None:
video.save()
plio.video = video

plio.save()

if items:
# before creating the items in the given workspace, update the
# plio ids that they are linked to and reset the primary key;
# django will auto-generate the keys when they are set to None
for index, _ in enumerate(items):
items[index].plio = plio
items[index].pk = None

# create the items
items = Item.objects.bulk_create(items)

# before creating the questions in the given workspace, update the
# item ids that they are linked to and
# since we are ordering both items and questions by the item time,
# questions and items at the same index should be linked
for index, _ in enumerate(questions):
questions[index].item = items[index]
questions[index].pk = None

# if there are any questions with images, create instances of those images in the
# new workspace and link them to the question instances that need to be created
if images:
for index, _ in enumerate(images):
# reset the key - django will auto-generate the keys when they are set to None
images[index].pk = None

images = Image.objects.bulk_create(images)
for index, question_index in enumerate(question_indices_with_image):
questions[question_index].image = images[index]

# create the questions
questions = Question.objects.bulk_create(questions)

# clear the cache for the destination plio or else the items wouldn't show up
# when the plio is fetched; we need to trigger this manually as bulk_create
# does not call the post_save signal
invalidate_cache_for_instance(plio)

return Response(self.get_serializer(plio).data)

@action(
methods=["get"],
detail=True,
Expand Down