Skip to content

Latest commit

ย 

History

History
95 lines (66 loc) ยท 4.68 KB

colab.rst

File metadata and controls

95 lines (66 loc) ยท 4.68 KB

Google Colab์—์„œ ํŠœํ† ๋ฆฌ์–ผ ์‹คํ–‰ํ•˜๊ธฐ

Google Colab์—์„œ ํŠœํ† ๋ฆฌ์–ผ์„ ์‹คํ–‰ํ•  ๋•Œ, ํŠœํ† ๋ฆฌ์–ผ์ด ์ œ๋Œ€๋กœ ๋™์ž‘ํ•˜๊ธฐ ์œ„ํ•ด์„œ ์ถฉ์กฑํ•ด์•ผ ํ•˜๋Š” ์ถ”๊ฐ€์ ์ธ ์ถ”๊ฐ€ ์š”๊ตฌ ์‚ฌํ•ญ๊ณผ ์ข…์†์„ฑ(dependancy)์ด ์žˆ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด ์„น์…˜์—์„œ๋Š” Google Colab์—์„œ ํŒŒ์ดํ† ์น˜(PyTorch) ํŠœํ† ๋ฆฌ์–ผ์„ ์„ฑ๊ณต์ ์œผ๋กœ ์‹คํ–‰ํ•˜๊ธฐ ์œ„ํ•ด ๋‹ค์–‘ํ•œ ์„ค์ •์„ ๊ตฌ์„ฑํ•˜๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•ด ์„ค๋ช…ํ•ฉ๋‹ˆ๋‹ค.

Google Colab์˜ PyTorch ๋ฒ„์ „

๊ณต๊ฐœ(release)๋œ์ง€ ์–ผ๋งˆ๋˜์ง€ ์•Š์€ PyTorch ๋ฒ„์ „์„ ์‚ฌ์šฉํ•˜๋Š” ํŠœํ† ๋ฆฌ์–ผ์„ ์‹คํ–‰ํ•˜๋Š” ๊ฒฝ์šฐ, ํ•ด๋‹น ๋ฒ„์ „์ด ์•„์ง Google Colab์— ๋ฐ˜์˜๋˜์ง€ ์•Š์•˜์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ํ•„์š”ํ•œ torch ์™€ ํ˜ธํ™˜๋˜๋Š” ๋„๋ฉ”์ธ ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ(domain library)๊ฐ€ ์„ค์น˜๋˜์–ด ์žˆ๋Š”์ง€ ํ™•์ธํ•˜๋ ค๋ฉด !pip list ๋ฅผ ์‹คํ–‰ํ•˜์„ธ์š”.

๋งŒ์•ฝ ํ•„์š”ํ•œ PyTorch ๋ฒ„์ „๋ณด๋‹ค ๋‚ฎ์€ ๋ฒ„์ „์ด ์„ค์น˜๋˜์–ด ์žˆ๋Š” ๊ฒฝ์šฐ, ๋‹ค์Œ ๋ช…๋ น์–ด๋ฅผ ์‹คํ–‰ํ•˜์—ฌ ์ œ๊ฑฐํ•œ ๋’ค, ๋‹ค์‹œ ์„ค์น˜ํ•˜์„ธ์š”:

Colab์—์„œ Google Drive์˜ ํŠœํ† ๋ฆฌ์–ผ ๋ฐ์ดํ„ฐ ์‚ฌ์šฉํ•˜๊ธฐ

์‚ฌ์šฉ์ž๊ฐ€ Google Colab์—์„œ ํŠœํ† ๋ฆฌ์–ผ๊ณผ ๊ด€๋ จ๋œ ๋…ธํŠธ๋ถ์„ ์—ด ์ˆ˜ ์žˆ๋„๋ก ํ•˜๋Š” ์ƒˆ๋กœ์šด ๊ธฐ๋Šฅ์ด ํŠœํ† ๋ฆฌ์–ผ์— ์ถ”๊ฐ€๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ์ด ๋•Œ, ๋ณด๋‹ค ๋ณต์žกํ•œ ํŠœํ† ๋ฆฌ์–ผ์„ ์‹คํ–‰ํ•˜๋ ค๋ฉด ์‚ฌ์šฉ์ž์˜ Google Drive ๊ณ„์ •์— ๋ฐ์ดํ„ฐ๋ฅผ ๋ณต์‚ฌํ•ด์•ผ ํ•  ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค.

์ด ์˜ˆ์ œ์—์„œ๋Š” ์ฑ—๋ด‡(Chatbot) ํŠœํ† ๋ฆฌ์–ผ์„ Colab์—์„œ ๋™์ž‘ํ•˜๋„๋ก ๋ณ€๊ฒฝํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์„ค๋ช…ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ์ด๋ฅผ ์œ„ํ•ด์„œ, ๋จผ์ € Google Drive์— ๋กœ๊ทธ์ธ์ด ๋˜์–ด ์žˆ์–ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. (Colab์—์„œ ๋ฐ์ดํ„ฐ์— ์ ‘๊ทผํ•˜๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•œ ์ž์„ธํ•œ ์„ค๋ช…์€ ์—ฌ๊ธฐ ์—์„œ ์˜ˆ์ œ ๋…ธํŠธ๋ถ์„ ํ†ตํ•ด ๋ณผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.)

์‹œ์ž‘ํ•˜๊ธฐ ์ „์— ์ฑ—๋ด‡ ํŠœํ† ๋ฆฌ์–ผ ์„ ๋ธŒ๋ผ์šฐ์ €์— ๋„์›Œ์ฃผ์„ธ์š”.

ํŽ˜์ด์ง€ ์ƒ๋‹จ์— Run in Google Colab ์„ ํด๋ฆญํ•ฉ๋‹ˆ๋‹ค.

Colab์—์„œ ํŒŒ์ผ์ด ์—ด๋ฆฌ๊ฒŒ ๋ฉ๋‹ˆ๋‹ค.

Runtime ์„ ์„ ํƒํ•œ ๋’ค Run All ์„ ์„ ํƒํ•˜๋ฉด ํŒŒ์ผ์„ ์ฐพ์„ ์ˆ˜ ์—†๋‹ค(the file can't be found)๋Š” ์—๋Ÿฌ๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.

์ด๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด, ํ•„์š”ํ•œ ํŒŒ์ผ๋“ค์„ Google Drive์— ๋ณต์‚ฌํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

  1. Google Drive์— ๋กœ๊ทธ์ธํ•ฉ๋‹ˆ๋‹ค.
  2. Google Drive์—์„œ data ๋ผ๋Š” ์ด๋ฆ„์˜ ํด๋” ๋ฐ ์ด ์•„๋ž˜์— cornell ๋ผ๋Š” ํ•˜์œ„ ํด๋”๋„ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.
  3. Cornell Movie Dialogs Corpus์— ๋ฐฉ๋ฌธํ•˜์—ฌ movie-corpus ZIP ํŒŒ์ผ์„ ๋‚ด๋ ค๋ฐ›์Šต๋‹ˆ๋‹ค.
  4. ๋กœ์ปฌ ๋จธ์‹ ์— ์••์ถ•์„ ํ’‰๋‹ˆ๋‹ค.
  5. utterances.jsonl ํŒŒ์ผ์„ Google Drive์— ์ƒ์„ฑํ•œ data/cornell ํด๋” ์•ˆ์— ๋ณต์‚ฌํ•ฉ๋‹ˆ๋‹ค.

์ด์ œ Google Drive ์ƒ์˜ ํŒŒ์ผ์„ ๊ฐ€๋ฅดํ‚ค๋„๋ก Colab์˜ ํŒŒ์ผ์„ ํŽธ์ง‘ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

Colab์—์„œ corpus\_name ์œผ๋กœ ์‹œ์ž‘ํ•˜๋Š” ์ฝ”๋“œ ์„น์…˜์˜ ์œ— ๋ถ€๋ถ„์— ๋‹ค์Œ ๋‚ด์šฉ์„ ์ถ”๊ฐ€ํ•ฉ๋‹ˆ๋‹ค:

from google.colab import drive
drive.mount('/content/gdrive')

์ด์ œ ๋‹ค์Œ๊ณผ ๊ฐ™์ด 2์ค„์„ ๋ณ€๊ฒฝํ•˜์„ธ์š”:

  1. corpus\_name ๊ฐ’์„ "cornell" ๋กœ ๋ณ€๊ฒฝํ•ฉ๋‹ˆ๋‹ค.
  2. corpus ๋กœ ์‹œ์ž‘ํ•˜๋Š” ์ค„์„ ์•„๋ž˜์ฒ˜๋Ÿผ ๋ณ€๊ฒฝํ•ฉ๋‹ˆ๋‹ค:
corpus = os.path.join("/content/gdrive/My Drive/data", corpus_name)

์ด์ œ Google Drive์— ์—…๋กœ๋“œํ•œ ํŒŒ์ผ์„ ๊ฐ€๋ฆฌํ‚ค๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.

์ด์ œ ์ฝ”๋“œ ์„น์…˜์˜ Run cell ๋ฒ„ํŠผ์„ ํด๋ฆญํ•˜๊ฒŒ ๋˜๋ฉด Google Drive์— ์ธ์ฆํ•˜๋ผ๋Š” ๋ฉ”์‹œ์ง€๊ฐ€ ํ‘œ์‹œ๋˜๊ณ  ์ธ์ฆ ์ฝ”๋“œ๋ฅผ ๋ฐ›๊ฒŒ ๋ฉ๋‹ˆ๋‹ค. ์ธ์ฆ ์ฝ”๋“œ๋ฅผ Colab์— ๋ถ™์—ฌ๋„ฃ์œผ๋ฉด ์„ค์ •์ด ๋ฉ๋‹ˆ๋‹ค.

๋…ธํŠธ๋ถ์˜ Runtime / Run All ๋ฉ”๋‰ด ๋ช…๋ น์„ ๋‹ค์‹œ ์‹คํ–‰ํ•˜๋ฉด, ์ง„ํ–‰ ์ƒํ™ฉ์„ ๋ณผ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. (์ฑ—๋ด‡ ํŠœํ† ๋ฆฌ์–ผ์€ ์‹คํ–‰ํ•˜๋Š”๋ฐ ์‹œ๊ฐ„์ด ์˜ค๋ž˜ ๊ฑธ๋ฆฌ๋‹ˆ ์ฐธ๊ณ ํ•˜์„ธ์š”.)

์ด ์˜ˆ์ œ๊ฐ€ Coalb์—์„œ ๋ณด๋‹ค ๋ณต์žกํ•œ ํŠœํ† ๋ฆฌ์–ผ์„ ์‹คํ–‰ํ•˜๋Š”๋ฐ ์žˆ์–ด์„œ ์ข‹์€ ์‹œ์ž‘์ ์ด ๋˜๊ธธ ๋ฐ”๋ž๋‹ˆ๋‹ค. PyTorch ํŠœํ† ๋ฆฌ์–ผ ์‚ฌ์ดํŠธ์—์„œ Colab์„ ๋” ํ™œ์šฉํ•˜์—ฌ ์‚ฌ์šฉ์ž๋“ค์ด ๋” ์‰ฝ๊ฒŒ ์‚ฌ์šฉํ•  ์ˆ˜ ์žˆ๋Š” ๋ฐฉ๋ฒ•์„ ์ฐพ์•„๋ณด๊ฒ ์Šต๋‹ˆ๋‹ค.

CUDA ํ™œ์„ฑํ™”(Enabling CUDA)

์ผ๋ถ€ ํŠœํ† ๋ฆฌ์–ผ์€ CUDA-์ง€์› ์žฅ์น˜(NVIDIA GPU)๊ฐ€ ํ•„์š”ํ•˜๋ฉฐ, ์ด๋Ÿฐ ๊ฒฝ์šฐ ํŠœํ† ๋ฆฌ์–ผ์„ ์‹คํ–‰ํ•˜๊ธฐ ์ „ ๋Ÿฐํƒ€์ž„(Runtime) ์œ ํ˜•์„ ๋ณ€๊ฒฝํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. Google Colab์—์„œ ๋Ÿฐํƒ€์ž„์„ ๋ณ€๊ฒฝํ•˜๋ ค๋ฉด ์ƒ๋‹จ ๋“œ๋กญ๋‹ค์šด ๋ฉ”๋‰ด์—์„œ Runtime ์„ ์„ ํƒํ•œ ๋’ค, Change runtime type ์„ ์„ ํƒํ•˜์„ธ์š”. Hardware accelerator ์—์„œ T4 GPU ๋ฅผ ์„ ํƒํ•˜๊ณ  Save ๋ฅผ ํด๋ฆญํ•˜์„ธ์š”.