This project implements necessary services and API containing endpoints for providing marketing measurements and insights to the front-end application
This application relies on two technologies that need to be installed locally to be executed:
- Docker - Allows us to run the application and database containerized and within the docker network.
- Taskfile - Task runner that allows us to run commands in a more organized way, making it easier to interact with the application either for building it, executing it, running tests, etc.
To run the application, you need to execute the following command:
task runThis command will first build the application (which can be also done by running task build-project before task run) and then run it. The application will be available at http://localhost:8000.
ℹ️ You can also run the project detached by running
task run-detached, which will run the application in the background (you can check the containers withdocker ps).
⚠️ When running the project for the first time, execution might take a bit longer as it needs to download the necessary docker images and build the application, plus perform the initial database migrations (among which inserting the demo data records is included)
The service implements token authentication (for demonstration purposes), so in order to perform requests to the API, you need to obtain a token to consume the marketing endpoints.
Obtain a token by performing a request to the GET /api/auth/token/ (Basic Auth). The service implements sample credentials username: marketing_op and password: marketing_op_supersecret which you can use for this (base64 encoded).
curl --location --request GET 'http://localhost:8000/api/auth/token/' \
--header 'Authorization: Basic bWFya2V0aW5nX29wOm1hcmtldGluZ19vcF9zdXBlcnNlY3JldA=='The response will contain the token:
{
"data": {
"token": "<token>"
}
}Use the obtained token in previous step to perform requests to the marketing endpoints (Bearer Auth).
For example, to get the weekly sales per channel:
curl --location --request GET 'http://localhost:8000/api/marketing/stats/channel-weekly-sales/?channels=facebook&start_date=2020-01-01&end_date=2020-02-01' --header 'Authorization: Bearer <token>'Response:
{
"data": [
{
"channel": "facebook",
"year": 2020,
"week": 1,
"sales": 5333.516215408056
},
{
"channel": "facebook",
"year": 2020,
"week": 2,
"sales": 13225.911516370496
},
{
"channel": "facebook",
"year": 2020,
"week": 3,
"sales": 12036.479334585327
},
{
"channel": "facebook",
"year": 2020,
"week": 4,
"sales": 9708.314594927879
},
{
"channel": "facebook",
"year": 2020,
"week": 5,
"sales": 8593.207280276212
}
]
}To run all the tests (including the bdd scenario), you can execute the following command:
task testTo run all the test with contract validation (described later in Contract Testing section), you can execute the following command:
task test-contractThe application is built in Python, using Django-Ninja, which is a web framework for building APIs with Django and, as described in their documentation, "heavily inspired in FastAPI", sharing a lot of its concepts and structure (easy to use and intuitive, typed, high performant, based in standards, etc.) as you will be able to see in the source code.
The implementation is approached with a Service Layer pattern, where the business logic is separated from the Django models and the API endpoints implementation. This allows to encapsulate the business logic, providing a clear separation of concerns. It's basically an interface to the domain model, which allows to better test and maintain the code.
Application counts with two Django apps:
core: contains the models. In this case with just count with one additional application but, the reason to centralize models in this separate application is to allow re-usability and keep consistency in case we scale the project with more apps.api: contains the API endpoints and the service layer with all the logic.
marketing_op/
├─ api/
│ ├─ services/
│ │ ├─ marketing_measurements.py
│ │ ├─ marketing_stats.py
│ ├─ marketing_op_api.py
├─ core/
│ ├─ models.py
marketing_op_api.py contains the exposed endpoints implementation, which calls the service layer (api/services) modules to obtain the necessary data, without interacting directly with the database and simplifying the code.
The database used is a PostgreSQL database, containing the following tables:
Through the Django ORM, the models are defined in the core app, and the migrations are created and applied to the database. The database is initialized with some demo data, which is inserted through the migrations.
Also, queries for the different use cases have been implemented with the Django ORM, which allows to interact with the database in a more Pythonic way, but also used raw queries can be seen by using the query property of a Django query.
For example, the query used for obtaining the weekly sales per channel is translated as:
SELECT "channel"."name",
DATE_TRUNC(WEEK, "conversion"."date") AS "week",
SUM("conversion"."conversions") AS "net_sales"
FROM "conversion"
INNER JOIN "channel" ON ("conversion"."channel_id" = "channel"."id")
WHERE "channel"."name" IN (radio,
tv,
facebook,
instagram)
GROUP BY "channel"."name",
2
ORDER BY 2 ASC,
"channel"."name" ASC*Filtering for channels radio, tv, facebook, instagram.
The project uses the following 3rd party libraries, which are included also in later sections:
django-ninja: for building the API.pytest: for testing.pytest-django: for using pytest with django and facilitating features like database access.pytest-bdd: for implementing bdd-style tests.factory-boy: for creating test data.django-contract-tester: for implementing contract testing within the functional test suite.
The OpenAPI documentation (yaml) can be found in the openapi.yaml file. And there is also a corresponding html version generated within the same directory.
ℹ️ Even though Django-Ninja generates its own docs, I've decided to also manually create an OpenAPI documentation to customize it a bit and provide some additional context. The auto-generated docs can in any case be accessed in http://localhost:8000/api/docs
The API has four endpoints for the requested use cases, which details can be found in the previously mentioned OpenAPI documentation:
/api/token/: returns a token to authenticate the user in the rest of the endpoints./api/marketing/measurements/: returns the marketing measurements for a given date range and channel./api/marketing/stats/channels-sales-percentages: returns the net sales attributed to media channels./api/marketing/stats/channel-weekly-sales: returns the weekly sales per channel.
As briefly mentioned in the previous section, the API endpoints related to the marketing measurements require authentication, which is implemented for demonstration purposes. This is done through token-based authentication, where the user needs to obtain a token by providing a username and password (Basic Auth) to the /api/token/ endpoint.
Then the token can be used in the Authorization header (Bearer Auth) to authenticate the user in the rest of the endpoints.
ℹ️ Possible improvements here would be:
- Implementation of token expiration or refresh mechanism.
- Implementing a more secure authentication method (e.g. OAuth2, JWT).
The API endpoints implement a basic pagination, which you will be able to see in the OpenAPI schema, managed through the query parameters offset and page_size (default is 10).
ℹ️ Possible improvements here would be:
- Implementation of pagination information within Link header or in the response body.
There are some unit tests and also API tests included, separating the service functions tests for example (advantage of having a service layer). The tests can be run with the command task test.
For demo data creation (and as demonstration) I've used factoryboy to implement factories, which allow to easily create objects for testing purposes, and is also compatible with Django models.
Tests also implement contract testing for all the API tests, this is done using django-contract-tester library, which implements a client, that allow us to easily override our test clients and check in every existing tests, apart from the functional validations also if they match the written documentation/design.
This is a useful features for all APIs, but specially in public APIs, to keep consistency and make sure your API is always aligned with that the consumer expects.
The clients are instantiated as fixtures, running task test-contract will use the --contract flag which tells pytest to build the client to validate all tests against their OpenAPI schema. And how it works can be checked for example modifying the openapi schema (removing for example a response from an endpoint) and re-running the tests, they should fail stating an undocumented response error.
Behaviour driven development is a software development process which includes natural-language constructs and encourages collaboration among developers. Within the tests suite there is an example of using pytest-bdd to implement automation of tests, using Gherkin Language and facilitating behavioral driven development.
Scenario: Retrieve marketing measurements for specific channel
Given a set of existing conversions for marketing campaigns
When requesting marketing measurements through API for a specific channel
Then I should receive a list of measurements for that channel