autoGPT_ds.qmd

---
title: "chatGPT"
subtitle: "데이터 과학"
description: | 
  데이터 사이언스를 챗GPT로 풀어보자.
author:
  - name: 이광춘
    url: https://www.linkedin.com/in/kwangchunlee/
    affiliation: 한국 R 사용자회
    affiliation-url: https://github.com/bit2r
title-block-banner: true
#title-block-banner: "#562457"
format:
  html:
    css: css/quarto.css
    theme: flatly
    code-fold: true
    code-overflow: wrap
    toc: true
    toc-depth: 3
    toc-title: 목차
    number-sections: true
    highlight-style: github    
    self-contained: false
filters:
   - lightbox
lightbox: auto
link-citations: true
knitr:
  opts_chunk: 
    message: false
    warning: false
    collapse: false
editor_options: 
  chunk_output_type: console
editor: 
  markdown: 
    wrap: 72
---

# autoEDA

## `Hmisc::describe()`

[`Hmisc`](https://cran.r-project.org/web/packages/Hmisc/index.html) 패키지를 통해 과거 20년전 데이터 분석방법을 음미합니다.

```{r}
library(tidyverse)

penguins <- palmerpenguins::penguins %>%
  # 영어 변수명 한글 변환
  set_names(c("종명칭", "섬이름", "부리_길이", "부리_깊이", "물갈퀴_길이",
              "체중", "성별", "연도")) %>%
  # 결측값 제거
  # drop_na() %>%
  # 영어 값 한글 값으로 변환
  mutate(성별 = ifelse(성별 == "male", "수컷", "암컷"),
         섬이름 = case_when( str_detect(섬이름, "Biscoe") ~ "비스코",
                          str_detect(섬이름, "Dream") ~ "드림",
                          str_detect(섬이름, "Torgersen") ~ "토르거센"),
         종명칭 = case_when( str_detect(종명칭, "Adelie") ~ "아델리",
                          str_detect(종명칭, "Chinstrap") ~ "턱끈",
                          str_detect(종명칭, "Gentoo") ~ "젠투")
  ) %>%
  # 자료형 변환
  mutate(성별   = factor(성별, levels = c("수컷", "암컷")),
         섬이름 = factor(섬이름, levels = c("비스코", "드림", "토르거센")),
         종명칭 = factor(종명칭, levels = c("아델리", "턱끈", "젠투")),
         연도   = ordered(연도, levels = c(2007, 2008, 2009)))



Hmisc::describe(penguins)
```


## `skimr`

`skimr` 패키지를 사용하여 분석할 데이터와 친숙해진다.

```{r}
penguins %>% 
  skimr::skim()
```

## `dataxray`

[`dataxray`](https://github.com/agstn/dataxray) 패키지를 사용해서 데이터에 대한 이해를 더욱 높일 수 있다.

:::{.column-page}


```{r}
library(dataxray)

penguins %>% 
   make_xray() %>% 
   view_xray()
```

:::


## `dlookr`

[`dlookr`](https://cran.r-project.org/web/packages/dlookr/index.html) 패키지를 사용하여 분석할 데이터와 친숙해진다.

```{r}
library(kableExtra)
penguins %>% 
  dlookr::describe() %>% 
  kable(caption = "요약통계량") %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = F)  
```

## `DataExplorer `

[`DataExplorer `](https://cran.r-project.org/web/packages/DataExplorer/) 패키지를 사용하여 분석할 데이터와 친숙해진다.

```r
DataExplorer::create_report(penguins)
```

:::{.panel-tabset}
### 구조

```{r}
DataExplorer::plot_str(penguins)
```

### DF 요약

```{r}
DataExplorer::introduce(penguins)
```

### DF 요약 시각화

```{r}
DataExplorer::plot_intro(penguins)
```

### 결측값

```{r}
DataExplorer::plot_missing(penguins)
```

### 분포(범주형)

```{r}
DataExplorer::plot_bar(penguins)
```


### 분포(연속형)

```{r}
DataExplorer::plot_histogram(penguins)
```

### 상관관계

```{r}
penguins %>% select_if(is.numeric) %>% 
  # drop_na() %>% 
  DataExplorer::plot_correlation(cor_args = list("use" = "pairwise.complete.obs"))
```

### PCA

```{r}
penguins_pca <- penguins %>% select_if(is.numeric) %>% 
  drop_na() %>% prcomp(scale = TRUE)

summary(penguins_pca)$importance %>% as.data.frame() %>% 
  kable(caption = "PCA 요약") %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed", "responsive"), full_width = F)
```

:::

# 챗GPT EDA

[[Model: GPT-4, autoEDA and autoML](https://sharegpt.com/c/1lHtBnV)]{.aside}

챗GPT GPT-4 모형을 사용해서 생산성을 크게 향상시킬 수 있다. [@chatGPT2023ds]
먼저 역할을 지정하고 데이터셋이 크지 않은 경우 이를 프롬프트에 넣어 GPT-4모형이 작업할 수 있는 환경을 조성한다.

:::{.panel-tabset}

## 역할부여와 데이터 {.unnumbered}

:::{.callout-note collapse="false"}
### I want you to act as a data scientist using R programming language and code for me. I have a dataset of penguins as follows. If you understand what I instructed. print "OK" and wait for another instruction.

> species,island,culmen_length_mm,culmen_depth_mm,flipper_length_mm,body_mass_g,sex
> Adelie,Torgersen,39.1,18.7,181,3750,MALE
> Adelie,Torgersen,39.5,17.4,186,3800,FEMALE
> Adelie,Torgersen,40.3,18,195,3250,FEMALE
> Adelie,Torgersen,NA,NA,NA,NA,NA
> Adelie,Torgersen,36.7,19.3,193,3450,FEMALE
> Adelie,Torgersen,39.3,20.6,190,3650,MALE
> Adelie,Torgersen,38.9,17.8,181,3625,FEMALE
> Adelie,Torgersen,39.2,19.6,195,4675,MALE <br>
> ...

GPT-4 답변: **OK**
:::

## EDA 작업지시 {.unnumbered}

본격적으로 EDA 작업을 챗GPT에 부여하고 작업을 지시한다.

:::{.callout-note collapse="false"}
### Please write code for the exploratory data analysis (EDA) task and target variable is sex.

Here's an example of an R script that performs exploratory data analysis (EDA) on a dataset, with the target variable being 'sex'. The code uses the same dataset ('penguins.csv') as the previous answer.

```
# Load libraries
library(tidyverse)
library(tidymodels)
library(ggplot2)
library(readr)

# Import dataset
data <- read_csv("penguins.csv")

# Data preprocessing and cleaning
data_clean <- data %>%
  drop_na() %>%
  mutate(sex = as.factor(sex))

# Examine the structure of the dataset
print(str(data_clean))

# Summary statistics
print(summary(data_clean))

# Target variable distribution
print(table(data_clean$sex))

# Visualize target variable distribution
ggplot(data_clean, aes(x = sex)) +
  geom_bar() +
  ggtitle("Sex Distribution")

# Pairwise scatterplot of continuous variables
pairs_plot <- data_clean %>%
  select_if(is.numeric) %>%
  ggpairs()

print(pairs_plot)

# Boxplots for continuous variables by sex
data_clean %>%
  gather("variable", "value", -sex) %>%
  ggplot(aes(x = sex, y = value)) +
  geom_boxplot() +
  facet_wrap(~ variable, scales = "free") +
  ggtitle("Boxplots of Continuous Variables by Sex")

# Visualize categorical variables by sex
data_clean %>%
  ggplot(aes(x = island, fill = sex)) +
  geom_bar(position = "dodge") +
  ggtitle("Island Distribution by Sex")

# Correlation matrix
cor_matrix <- cor(data_clean %>% select_if(is.numeric))
print(cor_matrix)
```

This script includes several EDA techniques such as examining the dataset's structure, summarizing the dataset, visualizing the target variable distribution, creating pairwise scatterplots, boxplots, and bar charts, and calculating the correlation matrix for the continuous variables.

Make sure to replace the filename and adapt the code to your specific needs.

:::




## EDA 수행 {.unnumbered}

```{r}
# Load libraries
library(tidyverse)
library(palmerpenguins)
library(GGally)

# Import dataset
penguins <- palmerpenguins::penguins

# Data preprocessing and cleaning
penguins <- penguins %>%
  drop_na() %>%
  mutate(sex = as.factor(sex))

# Examine the structure of the dataset
print(str(penguins))

# Summary statistics
print(summary(penguins))

# Target variable distribution
print(table(penguins$sex))

# Visualize target variable distribution
ggplot(penguins, aes(x = sex)) +
  geom_bar() +
  ggtitle("Sex Distribution")

# Pairwise scatterplot of continuous variables
pairs_plot <- penguins %>%
  select_if(is.numeric) %>%
  ggpairs()

print(pairs_plot)

# Boxplots for continuous variables by sex
penguins %>%
  select(-species, -island) %>% 
  pivot_longer(cols = -sex, names_to = "variable", values_to = "value") %>%
  ggplot(aes(x = sex, y = value)) +
  geom_boxplot() +
  facet_wrap(~ variable, scales = "free") +
  ggtitle("Boxplots of Continuous Variables by Sex")

# Visualize categorical variables by sex
penguins %>%
  ggplot(aes(x = island, fill = sex)) +
  geom_bar(position = "dodge") +
  ggtitle("Island Distribution by Sex")

# Correlation matrix
cor_matrix <- cor(penguins %>% select_if(is.numeric))
print(cor_matrix)
```

:::