dlookr_klasyfikacja.Rmd

---
title: "Projekt 1.2"
author: "Ziemowit Głowaczewski"
date: '`r Sys.Date()`'
output: html_document
---

# Pakiet: dlookr
# Zadanie: klasyfikacja


```{r}
#tutaj ładuje wszystkie potrzebne biblioteki
#library()
library(dlookr)
library(ggplot2)
library(dplyr)
library(forecast)
```

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
knitr::opts_chunk$set(warning = FALSE, message = FALSE)

# Regresja
n <- 500
# Niezr?wnowazone klasy
y <- sample(0:1, size=n, replace=TRUE, prob = c(0.3,0.7))
y1 <- sum(y==1)
y0 <- n-y1
# Nieskorelowana zmienna
x0 <- rnorm(n,3,5) + rgamma(n,1,1)
# Przycieta zmienna
x1 <- numeric(n)
x1[y==0] <- 3 + rnorm(y0,0,2)
x1[y==1] <- 5 + rnorm(y1, 0, 3)
x1[x1>=6.25] <- 6.25
x1[x1<=1.75] <- 1.75
# NULL i dyskretna
x2 <- numeric(n)
prob0 <- c(0.35, 0.30, 0.20, 0.15)
prob1 <- c(0.20, 0.10, 0.40, 0.30)
x2[y==0] <- sample(1:4,y0, replace=TRUE, prob=prob0)
x2[y==1] <- sample(1:4,y1, replace=TRUE, prob=prob1)
x2_NULL <- rbinom(n,1,prob=0.05)
x2[x2_NULL==1] <- NaN
# W pelni objasnia y
x3 <- numeric(n)
x3[y==0] <- 1.32
x3[y==1] <- 5.64
# Dwie mocno skorelowane
x4 <- numeric(n)
x4[y==0] <- rbeta(y0,3,1) 
x4[y==1] <- rbeta(y1,1,3)

x5 <- numeric(n)
x5 <- x4 + + rnorm(n, 0, 0.25)

dataRegression <- data.frame(x0,x1,x2,x3,x4,x5,y)


# Klasyfikacja
m <- 1500
y <- sample(0:1, size=m, replace=TRUE, prob = c(0.3,0.7))
# Niezr?wnowazone klasy
z <- sample(0:1,size=m, replace = TRUE,c(0.3,0.7))
z0 <- sum(z==0)
z1 <- sum(z==1)
# Klasa 1 rozdzielana przez 0
w1 <- numeric(m)
w1[z==0] <- rnorm(z0)
w1[z==1] <- sample(c(-1,1), z1, replace=TRUE)*rnorm(z1, 1.5, 1)
# Nieskorelowana zmienna
w2 <- rnorm(m,0,5) + rgamma(m,1,1)
# Nieznormalizowana + NULL
w3 <- numeric(m)
w3[z==0] <- rnorm(z0, 150, 20)
w3[z==1] <- rnorm(z1, 165, 15)
w3_NULL <- rbinom(m,1,prob=0.1)
w3[w3_NULL] <- NaN
# Duzo dyskretnych klas
classes <- 1:15
probs0 <- classes/sum(classes)
w4 <- numeric(m)
w4[z==0] <- sample(1:15, z0, replace=TRUE, prob = probs0)
probs1 <- 15:1/sum(15:1)
w4[z==1] <- sample(1:15, z1, replace=TRUE, prob = probs1)
# Duze odchylenie w jednej klasie
w5 <- numeric(m)
w5[z==0] <- rnorm(z0)
w5[z==1] <- rnorm(z1, 0.5, 5)
# NULL i dyskretna
w6 <- numeric(n)
prob0 <- c(0.35, 0.30, 0.20, 0.15)
prob1 <- c(0.20, 0.10, 0.40, 0.30)
w6[y==0] <- sample(1:4,y0, replace=TRUE, prob=prob0)
w6[y==1] <- sample(1:4,y1, replace=TRUE, prob=prob1)
w6_NULL <- rbinom(n,1,prob=0.05)
w6[x2_NULL==1] <- NaN
dataClassification <- data.frame(w1,w2,w3,w4,w5,w6,z)


```

## Overview of functionalities

### Dataset
For my analysis I've used my generated data containing some
basic problems - like strong correlation, NaNs etc.

```{r}

```

#### Variable types

```{r}
diagnose(dataClassification)
```

#### Dimensions

```{r}
describe(dataClassification)
```

#### Other info

```{r}
describe(dataClassification)
```

#### Compare datasets

```{r}
```

### Validity

#### Missing values 

```{r}
describe(dataClassification)

```

#### Redundant col.

```{r}
plot(correlate(dataClassification))
```

#### Outliers 

```{r}
diagnose_outlier(dataClassification)
plot_outlier(dataClassification)
```


#### Atypical values

```{r}
```


#### Level encoding 

```{r}
```

### Univar.

#### Descriptive stat.

```{r}
diagnose(dataClassification)
univar_numeric(dataClassification)
```

#### Histograms 

```{r}
plot_hist_numeric(dataClassification)
```

#### Other dist. plots

```{r}
plot_na_intersect(dataClassification)
```

#### Bar plots 

```{r}
plot_bar_category(dataClassification)
```

#### QQ plots

```{r}
plot_normality(dataClassification)
```

### Bivar.     

#### Descriptive stat.

```{r}
describe(dataClassification)
```

#### Correlation matrix 

```{r}
  plot_correlate(dataClassification)
```

#### 1 vs each corr.   

```{r}
dataClassification %>% 
  plot_correlate(w1)
```

#### Time-dependency  

```{r}
```

#### Bar plots by target  

```{r}

```

#### Num. plots by target 

```{r}
```

#### Scatter plots 

```{r}
by <- target_by(dataClassification, y)
m1 <- relate(by, w2)
plot(m1, 1)
```

#### Contigency tables   

```{r}
```

#### Other stats. (factor)

```{r}
skewness(dataClassification$w1)
```


### Multivar.  

#### PCA 

```{r}
```

#### Stat. models  

```{r}
#by2 <- target_by(dataClassification, y)
#m2 <- relate(by, w1)
#plot(m2, 1)
```

#### PCP        

```{r}
```


### Transform. 

#### Imputation
 imputate_na(dataClassification, w2)
 imputate_outlier(dataClassification, w4)

```{r}
```

#### Scaling 

```{r}

```

#### Skewness   

```{r}
skewness(dataClassification$z)
```

#### Outlier treatment  

```{r}
imputate_outlier(dataClassification, w4)
diagnose_outlier(dataClassification)
```

#### Binning  

```{r}
binning(dataClassification$w2)
```

#### Merging levels     

```{r}

```

### Reporting  

#### Reports

```{r}
#transformation_web_report(dataClassification)
#eda_web_report(dataClassification)
```

#### Saving outputs        

```{r}
```



## Summary

|  Task type | Task                  | Yes/No |
|:----------:|-----------------------|--------|
|   Dataset  | Variable types        |     Y  |
|            | Dimensions            |     Y  |
|            | Other info            |     Y  |
|            | Compare datasets      |     N  |
|  Validity  | Missing values        |     Y  |
|            | Redundant col.        |     ~  |
|            | Outliers              |     Y  |
|            | Atypical values       |     N  |
|            | Level encoding        |     N  |
|   Univar.  | Descriptive stat.     |     Y  |
|            | Histograms            |     Y  |
|            | Other dist. plots     |     Y  |
|            | Bar plots             |     Y  |
|            | QQ plots              |     Y  |      
|   Bivar.   | Descriptive stat.     |     Y  |       
|            | Correlation matrix    |     Y  |
|            | 1 vs each corr.       |     Y  |
|            | Time-dependency       |     N  |             
|            | Bar plots by target   |     N  |
|            | Num. plots by target  |     N  |
|            | Scatter plots         |     Y  |
|            | Contigency tables     |     N  |
|            | Other stats. (factor) |     Y  |
|  Multivar. | PCA                   |     N  |      
|            | Stat. models          |     N  |
|            | PCP                   |     N  |
| Transform. | Imputation            |     Y  |
|            | Scaling               |     N  |
|            | Skewness              |     Y  |
|            | Outlier treatment     |     Y  |
|            | Binning               |     Y  |
|            | Merging levels        |     N  |
|  Reporting | Reports               |     Y  |
|            | Saving outputs        |     N  |