Backend: All Main Features Complete
- API baseurl: http://bookstore.anrdhmshr.tech/api/v1
- Health Check: http://bookstore.anrdhmshr.tech/api/health
- API Docs: https://documenter.getpostman.com/view/19697822/2s9Y5Wxifq
Bonus — Frontend: Using Go and WASM. Incomplete (landing, login, signup, forgotpwd, basic catalog).
Demo Video: https://youtu.be/F_GDIHgObvU
Test Credentials to access Deployed Api (you can make your own account as well — will need a valid email):
{
"email": "testuser@anrdhmshr.com",
"password": "Testpwd@111"
}
- Deployed Links
- Table of Contents
- How to run locally
- Features
- Recommendation Engine
- Frontend
- Where I ran into issues (aka, 90% Gorm):
- Project Structure
- Clone the repo.
- Make sure you have docker/docker desktop installed, and a docker daemon running.
- Rename
sample.envto.env, and set JWT signing secret (API_SECRET) withopenssl rand -hex 32. - For MAILTRAP_API_TOKEN, obtain a free api token from https://mailtrap.io/, or run the app without it (confirmation mails, etc, will not be sent in that case).
make dev- Frontend will be served at http://0.0.0.0/ by default. Check http://0.0.0.0/api/health to see if everything is OK on serverside. API baseurl: http://0.0.0.0/api/v1.
- (Not on production. PG Admin only spinned up on dev) PG Admin will be served at http://0.0.0.0:5050/browser (only in development). Hostname:
postgres_db, Username and Password as in your env config. - API Docs at: https://documenter.getpostman.com/view/19697822/2s9Y5Wxifq
If you run via postman, make sure to set this in the Tests tab of the login route:
pm.collectionVariables.set("token", pm.response.json().token);This will automatically set the returned token when you login as a user so you won't need to set it in the collection variables manually.
JWT Auth. Role Based (Base User and Admin) access to resources. Hashed Password. Password Strength Check. After implementing my registration controllers I realised that a more secure way to do it would have been by asking users to first specify an email, confirm that email, and then ask them to set a password. This would guard against user enumeration attacks. Right now, I'm guarding against it by essentially lying — I say that the confirmation email got sent if you try making an account with an existing email. This could be confusing, however, if the user has genuinely forgotten if they had an account associated with a particular email or not. Alternatively, I could send a warning email in such cases.
There is router auth middleware which handles access for the 3 user types: base, admin and superadmin.
Require current and new password. On change, mail email associated with user about the change ("If this was not you, we request you to change your password via the Email-based forgot password option"). On failure to reset password due to incorrect current password, mail associated email about attempt (in case it was a malicious use trying to transfer ownership).
An OTP is sent to the registered email acount, and is valid for 3 minutes.
From: BOOKSTORE ADMIN test@anrdhmshr.tech Subject: Forgot Password.
Date: 28 August 2023 at 15:57
To: Anirudh Gray anirudhgray@gmail.com
A forgot password request was made for the email associated with your account. If this was not you, feel free to ignore this email. Otherwise, click on this link to post your new password: http://0.0.0.0:8000/v1/auth/set-forgotten-password? email=anirudh04mishra@gmail.com&otp=255610 . This link will be active for 3 minutes.
The user's profile is no longer accessible. User can request an account deletion, and will then recieve a confirmation email, with an OTP and a link to complete the process. The process must be completed within a set amount of time (3 minutes).
Admin users can add books to the catalog, as well as edit their details. In addition, they can delete reviews and books. They can also ban users (essentially, deactivate plus ban boolean). Deleting books should ordinarily NOT remove it from libraries of users who have already purchased it.
By default, if there are no admins in the database, the next user who signs up is set as the SuperAdmin (ie, intended to be used when deploying the applicatiod in prod for the first time). SuperAdmin can perform all normal admin functions, with the added permission of promoting base users to Admin, and vice verse.
Users can fuzzy search entire books catalog for title, author and/or category. They can filter by category, and sort by price.
Users can add any book not already purchased by them to their chopping cart. They can remove books from their cart as well. On checking out their cart, they "buy" all books in the cart, and those books get added to their library of bought books. A transaction record is created for admin audit purposes. The cart gets cleared of all books on a successful transaction.
Users need credits to buy books. They can purchase credits (a transaction record is created for this as well).
Library of books bought by a user. The user can download any of them as many times as they want.
Users can only review a book that they have bought (ie, which is in their library). Reviews have a comment, and a rating (which is used to calc avg rating for the book, and for generating recommendations).
Currently logging to a local rotating logfile. Logs are persisted on prod by mounting a docker volume for them (app_logs). To access (will copy logs to ./logs dir on host system):
docker cp <SERVER_CONTAINER_ID>:/app_logs ./logs
Check out . Backend at location /api and frontend static files at /.
I am protecting against attacks like SQLI, and dependency graph revealed no major vulnerabilities with known exploits.
Note: If you're trying this out with a new user, note that you will not get any recommendations. Review a few books, and then you'll be able to get recommendations — this is because of the "cold start" issue in my implementation (explained below).
I have implemented a simple collaborative filtering based recommendations engine. Ref: https://www.toptal.com/algorithms/predicting-likes-inside-a-simple-recommendation-engine.
Implementation at: Recommender Controller and Recommender Utils
The general idea is that we will not care about the specific attributes of books, and then use some sort of ML Algorithm to figure out what kind of books our user will like (using the user's existing library, reviews, etc etc). Instead, our system will look similarities between users.
We keep a record of each user's likes and dislikes (let's base this on review rating — maybe 4 or 5 is "like", everything else is "dislike"). These are two sets that exist for every user. We're going to use something called the Jaccard Coefficient to calculate how similar two such sets are. For example, two duplicate sets will be completely similar (coeff of 1) while two sets with nothing in common will have a coeff of 0 (no similarity or overlap between the sets).
J(A, B) = |A ∩ B| / |A ∪ B|
- Find user's current likes and dislikes.
- Get all users who have interacted with those items (both liked and unliked).
- For each of those users, calculate their similarity to our current user using the Jaccard Coefficient. (modified, on a -1 to +1 scale)
- Get a set of items the current user has not yet liked/disliked.
- For each of those items, calculate the probability that it should be recommended to the current user. How do we do this:
- Result = Numerator / Denominator
- Get other users who liked or disliked that item.
- Numerator is the sum of the similarity indices of all these users who liked it - sum of indices of users who disliked it.
- Denominator is simply the total number of users of liked or disliked the item.
- Now we can rank the items based on this calculated probability, and give X number of recommendations.
- Currently doing calculation in-memory. Like, user wants reccs => I do all of the above and return some reccs. Maybe a better way to handle this would be to use something like Redis which apparently has nice features to work with sets, and generate reccs periodically.
- Also, a major issue with collaborative filtering is that of "cold start" — since this method relies on other similar users, what do you do for the first few users? And you can't get any recommendations until you make a few likes/dislikes of your own, and books with no reviews cannot be recommended to any users. In such cases a hybrid approach involving this + a content based engine would be helpful.
- I am currently doing a basic like/dislike thing. However, since my ratings are on a scale of 1-5 I should make use of that (by giving more weight to a 5 than a 4, instead of treating both as equal likes).
Since I was through with the backend tasks, I wanted to try building a frontend for the bookstore while keeping in mind the "Golang only" restriction on the assignment. Which is why I used Go for both the backend and the frontend (compiled to WASM on the frontend). My frontend golang is at /frontend, and is deployed on the same host as the backend API. This webapp is a single page application.
- Frontend Site: http://bookstore.anrdhmshr.tech/
- Backend API Baseurl: http://bookstore.anrdhmshr.tech/api/v1
It is very limited: only the auth flows, and basic catalog viewing is implemented on the frontend. However, all routes and features are implemented in the backend API.
- Catalog View (after logging in)
- Mobile View (landing page)
- Login View (dark mode)
- Did not initially realise that gorm's auto-migrations do not, in fact, drop unused columns. While it does make sense as a default so that we don't lose data... well, anyway, spent some time trying to debug why my many2many join table had an unrelated column in it. Ended up dropping the table and then running migrations, will make sure to use my own migration scripts or a more full fledged library like goose.
- Needed to enter associations mode to delete properly, otherwise only the reference would be yeeted.
- The whole flow of user deletion. The culprit? Gorm, yet again.
- Overall, gorm seems like a pain. Unfortunately, I also made the poor design choice of having my data layer logic in my business logic controllers — using a "repository pattern" would have made it easier to switch out to something else. Maybe a sql query builder like Squirrel instead of an ORM.
- On the frontend: Generated wasm sizes are pretty massive. Some suggestions are to separate out the http package since it's pretty big, or to use some sort of compression middleware.
.
├── Dockerfile
├── Dockerfile-dev
├── Makefile
├── README.md
├── config
│ ├── config.go
│ ├── db.go
│ └── server.go
├── controllers
│ ├── admin.controller.go
│ ├── auth.controller.go
│ ├── books.controller.go
│ ├── cart.controller.go
│ ├── checkout.controller.go
│ ├── recommendations.controller.go
│ ├── review.controller.go
│ └── user.controller.go
├── docker-compose-dev.yml
├── docker-compose.yml
├── docs
│ ├── docs.go
│ ├── swagger.json
│ └── swagger.yaml
├── frontend
│ ├── Dockerfile
│ ├── go.mod
│ ├── go.sum
│ ├── main.go
│ ├── makefile
│ ├── src
│ │ ├── components
│ │ │ ├── footer.go
│ │ │ ├── layout.go
│ │ │ ├── navbar.go
│ │ │ └── title.go
│ │ └── pages
│ │ ├── catalog.go
│ │ ├── forgot.go
│ │ ├── landing.go
│ │ ├── login.go
│ │ ├── register.go
│ │ ├── setAfterForgot.go
│ │ └── verify.go
│ └── web
│ ├── app.wasm
│ ├── images
│ │ └── rickandmorty.jpeg
│ ├── js
│ │ └── themeToggle.js
│ ├── lottie
│ │ ├── themeToggle.json
│ │ └── themeToggleInverse.json
│ └── styles.css
├── go.mod
├── go.sum
├── go.work
├── go.work.sum
├── infra
│ ├── database
│ │ └── database.go
│ └── logger
│ └── logger.go
├── main.go
├── models
│ ├── book.model.go
│ ├── cart.model.go
│ ├── deletion_confirmation.model.go
│ ├── forgot_password.model.go
│ ├── review.model.go
│ ├── transaction.model.go
│ ├── user.model.go
│ ├── user_library.model.go
│ └── verification.model.go
├── nginx
│ ├── Dockerfile
│ └── nginx.conf
├── routers
│ ├── index.go
│ ├── middleware
│ │ ├── auth.middleware.go
│ │ └── cors.go
│ └── router.go
└── utils
├── auth
│ ├── auth.go
│ └── auth_test.go
├── email
│ └── email.go
├── recommender
│ ├── recommender.go
│ └── recommender_test.go
└── token
└── token.go
models/: The model structs for each table in my DB, along with their relations.controllers/: Combined handlers for each route, along with controllers for the business logic, as well as data accessing repository functions.routers/: API routes and auth+cors middleware.utils/: Reusable utility functions for business logic in controllers. Eg, for mailing, password validation, the recommender system etc.config/: Initial reading in config from .env, setting up server and DB configurations.
Note: This does not show 1:1 relations properly (shown as 1:N). Will update.
