Badge source
Key findings: The bug in the marketing campaign led to a loss of 32 potential subscribers to the business.
- Business problem
- Data source
- Methods
- Tech Stack
- Quick glance at the results
- Lessons learned and recommendation
- Limitation and what can be improved
- Run Locally
- Explore the notebook
- Deployment on streamlit
- App deployed on Streamlit
- Repository structure
- Contribution
- Blog post
- Project featuring
- License
In this notebook we analyze a marketing campaign for a subscription based product. We look at different factors to check the effectivesness of the campaign while also looking to find anormalities and improvements we can make to it.
- Exploratory data analysis
- Feature engineering
- Data visualization
- Summary statistics
- A/B Testing
- Python (refer to requirement.txt for the packages used in this project)
- pandas (data manipulation and analysis)
- numpy (mathematical operation on arrays)
- matplotlib (data visuaization)
-
Overall performance of the campaign: We can analyze the number of conversions, retention rates, and cancellations to understand how successful the campaign was. We can also calculate key performance indicators such as conversion rate, retention rate, and churn rate.
-
Performance by marketing channel: We can analyze how each marketing channel performed in terms of conversions, retention, and cancellations. We can also calculate conversion rates and retention rates for each channel.
-
Performance by variant: We can analyze how the personalization variant performed compared to the control group in terms of conversions, retention, and cancellations. We can also calculate conversion rates and retention rates for each variant.
-
Performance by language: We can analyze how each language performed in terms of conversions, retention, and cancellations. We can also calculate conversion rates and retention rates for each language.
-
Performance by age group: We can analyze how each age group performed in terms of conversions, retention, and cancellations. We can also calculate conversion rates and retention rates for each age group.
-
Performance by day of week: We can analyze how the campaign performed on each day of the week in terms of conversions, retention, and cancellations. We can also calculate conversion rates and retention rates for each day of the week.
-
Performance by subscribing channel: We can analyze how each subscribing channel performed in terms of conversions, retention, and cancellations. We can also calculate conversion rates and retention rates for each subscribing channel.
-
Correct language displayed: We can analyze how displaying the correct language affected conversions, retention, and cancellations. We can also calculate conversion rates and retention rates for the correct language displayed.
Line graph for number of user per day for the month of January.
Line graph for daily conversion rate.
Bar Graph showing the daily conversion rate grouped by language.
Top 3 models (with default parameters)
| Model | Recall score |
|---|---|
| Support vector machine | 88% |
| Gradient boosting | 90% |
| Adaboost | 79% |
-
The final model used for this project: Gradient boosting
-
Metrics used: Recall
-
Why choose recall as metrics: Since the objective of this problem is to minimize the risk of a credit default, the metrics to use depends on the current economic situation:
-
During a bull market (when the economy is expanding), people feel wealthy and are employed. Money is usually cheap, and the risk of default is low because of economic stability and low unemployment. The financial institution can handle the risk of default; therefore, it is not very strict about giving credit. The financial institution can handle some bad clients as long as most credit card owners are good clients (aka those who pay back their credit in time and in total).In this case, having a good recall (sensitivity) is ideal.
-
During a bear market (when the economy is contracting), people lose their jobs and money through the stock market and other investment venues. Many people struggle to meet their financial obligations. The financial institution, therefore, tends to be more conservative in giving out credit or loans. The financial institution can't afford to give out credit to many clients who won't be able to pay back their credit. The financial institution would rather have a smaller number of good clients, even if it means that some good clients are denied credit. In this case, having a good precision (specificity) is desirable.
Note: There is always a trade-off between precision and recall. Choosing the right metrics depends on the problem you are solving.
Conclusion: Since the time I worked on this project (beginning 2022), we were in the longest bull market (excluding March 2020 flash crash) ever recorded; we will use recall as our metric.
-
Lessons learned and recommendation
- Based on this project's analysis, income, family member headcount, and employment length are the three most predictive features in determining whether an applicant will be approved for a credit card. Other features like age and working employment status are also helpful. The least useful features are the type of dwelling and car ownership.
- The recommendation would be to focus more on the most predictive features when looking at the applicant profile and pay less attention to the least predictive features.
- Combine this model with with a regression model to predict how much of a credit limit an applicant will be approved for.
- Hyperparameter tuning with grid search or random search.
- Better interpretation of the chi-square test
- Retrain the model without the least predictive features
Initialize git
git initClone the project
git clone https://github.com/semasuka/Credit-card-approval-prediction-classification.gitenter the project directory
cd Credit-card-approval-prediction-classificationCreate a conda virtual environment and install all the packages from the environment.yml (recommended)
conda env create --prefix <env_name> --file assets/environment.ymlActivate the conda environment
conda activate <env_name>List all the packages installed
conda listStart the streamlit server locally
streamlit run cc_approval_pred.pyIf you are having issue with streamlit, please follow this tutorial on how to set up streamlit
To explore the notebook file here
To deploy this project on streamlit share, follow these steps:
- first, make sure you upload your files on Github, including a requirements.txt file
- go to streamlit share
- login with Github, Google, etc.
- click on new app button
- select the Github repo name, branch, python file with the streamlit codes
- click advanced settings, select python version 3.9 and add the secret keys if your model is stored on AWS or GCP bucket
- then save and deploy!
Video to gif tool
├── assets
│ ├── confusion_matrix.png <- confusion matrix image used in the README.
│ ├── gif_streamlit.gif <- gif file used in the README.
│ ├── heatmap.png <- heatmap image used in the README.
│ ├── Credit_card_approval_banner.png <- banner image used in the README.
│ ├── environment.yml <- list of all the dependencies with their versions(for conda environment).
│ ├── roc.png <- ROC image used in the README.
│
├── datasets
│ ├── application_record.csv <- the dataset with profile information (without the target variable).
│ ├── credit_records.csv <- the dataset with account credit records (used to derive the target variable).
│ ├── test.csv <- the test data (with target variable).
│ ├── train.csv <- the train data (with target variable).
│
│
├── pandas_profile_file
│ ├── credit_pred_profile.html <- exported panda profile html file.
│
│
├── .gitignore <- used to ignore certain folder and files that won't be commit to git.
│
│
├── Credit_card_approval_prediction.ipynb <- main python notebook where all the analysis and modeling are done.
│
│
├── LICENSE <- license file.
│
│
├── cc_approval_pred.py <- file with the model and streamlit component for rendering the interface.
│
│
├── README.md <- this readme file.
│
│
├── requirements.txt <- list of all the dependencies with their versions(used for Streamlit).
Pull requests are welcome! For major changes, please open an issue first to discuss what you would like to change or contribute.
MIT License
Copyright (c) 2022 Stern Semasuka
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Learn more about MIT license