Session4_Chatbot29_3_2023.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fc67ae52",
   "metadata": {},
   "source": [
    "## A Chatbot using seq2seq model (Encoder-Decoder Architecture)\n",
    "##### Using Gated Recurrent Units for encoder-decoder and Luoung Attention mechanism.\n",
    "\n",
    "###  Sequence to Sequence (seq2seq) Models\n",
    "\n",
    "- Sequence-to-sequence learning (Seq2Seq) is about training models to convert sequences from one domain (e.g. sentences in English) to sequences in another domain (e.g. the same sentences translated to French).\n",
    "\n",
    "- Seq2seq models turn one input sequence into another sequence (output) (sequence transformation). That is why they are also called as <b>Transformers</b>.\n",
    "\n",
    "- In seq2seq models, the context for each item (word) in input is the output from the previous time step (previous word in the sentence).\n",
    "\n",
    "- These models can use recurrent neural network (RNN). Or variants of RNN like LSTM or GRU to avoid the problem of vanishing gradient. \n",
    "\n",
    "##### The primary components of a seq2seq model is an  encoder and  decoder network\n",
    "\n",
    "-  The encoder turns each item into a corresponding hidden vector containing the item and its context. The decoder reverses the process, turning the vector into an output item, using the previous output as the input context.\n",
    "\n",
    "- Seq2seq models use optimizations like attention mechanism, beam search, bucketing. \n",
    "\n",
    "    - Attention: The input to the decoder is a single vector which stores the entire context. Attention allows the decoder to look at the input sequence selectively.\n",
    "    \n",
    "- Training typically uses a cross-entropy loss function, whereby one output is penalized to the extent that the probability of the succeeding output is less than 1.\n",
    "\n",
    "<img src = 'seq2seq-teacher-forcing.png'>\n",
    "<center>Source:https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html </center>\n",
    "\n",
    "<img src = 'seq2seq-inference.png'>\n",
    "<center>Source:https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html </center>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2976604",
   "metadata": {},
   "source": [
    "### References\n",
    "#### This session uses the code from the excellent pytorch chatbot tutorial available here:  https://brsoff.github.io/tutorials/beginner/chatbot_tutorial.html \n",
    "\n",
    "1. https://brsoff.github.io/tutorials/beginner/chatbot_tutorial.html\n",
    "2. https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html\n",
    "3. https://blog.floydhub.com/attention-mechanism/\n",
    "4. Yuan-Kuei Wu’s pytorch-chatbot implementation: https://github.com/ywk991112/pytorch-chatbot\n",
    "5. https://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html\n",
    "6. https://www.kaggle.com/datasets/rajathmc/cornell-moviedialog-corpus"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e6da547",
   "metadata": {},
   "source": [
    "### Dataset used in this code: Cornell Movie-Dialogs Corpus\n",
    "Download URL:  https://www.kaggle.com/datasets/rajathmc/cornell-moviedialog-corpus\n",
    "\n",
    "- This dataset contains 220,579 conversational exchanges between 10,292 pairs of movie characters. It has 304,713 total utterances, 9035 characters from 617 movies. \n",
    "- movie metadata includes:\n",
    "    - genres\n",
    "    - release year\n",
    "    - IMDB rating\n",
    "    - number of IMDB votes\n",
    "    - IMDB rating\n",
    "- character metadata includes:\n",
    "    - gender (for 3,774 characters)\n",
    "    - position on movie credits (3,321 characters)\n",
    "- movie_conversations.txt has the following format: \n",
    "  - The pattern '+++\\$+++' is being used as a field separator in all the files within the corpus dataset.\n",
    "  - ID of the first character, ID of the second character, ID of the movie that this conversation occurred, and a list of line IDs. The character and movie information can be found in files movie_characters_metadata.txt and movie_titles_metadata.txt respectively.\n",
    "\n",
    "- Samples of conversations pairs from movie_conversations.txt \n",
    "\n",
    "        - u0 +++$+++ u2 +++$+++ m0 +++$+++ [‘L194’, ‘L195’, ‘L196’, ‘L197’]<br>\n",
    "        - u0 +++$+++ u2 +++$+++ m0 +++$+++ [‘L198’, ‘L199’]<br>\n",
    "        - u0 +++$+++ u2 +++$+++ m0 +++$+++ [‘L200’, ‘L201’, ‘L202’, ‘L203’]<br>\n",
    "        - u0 +++$+++ u2 +++$+++ m0 +++$+++ [‘L204’, ‘L205’, ‘L206’]<br>\n",
    "        - u0 +++$+++ u2 +++$+++ m0 +++$+++ [‘L207’, ‘L208’]      <br>  \n",
    "\n",
    "- File movie_lines.txt has the following format: \n",
    "\n",
    "      - ID of the conversation line, ID of the character who uttered this phase, ID of the movie, name of the character and the text of the line.<br>\n",
    "       - L901 +++$+++ u5 +++$+++ m0 +++$+++ KAT +++$+++ He said everyone was doing it. So I did it.<br>\n",
    "       - L900 +++$+++ u0 +++$+++ m0 +++$+++ BIANCA +++$+++ As in…<br>\n",
    "       - L899 +++$+++ u5 +++$+++ m0 +++$+++ KAT +++$+++ Now I do. Back then, was a different story.<br>\n",
    "       - L898 +++$+++ u0 +++$+++ m0 +++$+++ BIANCA +++$+++ But you hate Joey<br>\n",
    "       - L897 +++$+++ u5 +++$+++ m0 +++$+++ KAT +++$+++ He was, like, a total babe<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9f0ce29",
   "metadata": {},
   "source": [
    "#### Requried Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "33345734",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Requried Imports\n",
    "from __future__ import absolute_import\n",
    "from __future__ import division\n",
    "from __future__ import print_function\n",
    "from __future__ import unicode_literals\n",
    "\n",
    "import torch\n",
    "from torch.jit import script, trace\n",
    "import torch.nn as nn\n",
    "from torch import optim\n",
    "import torch.nn.functional as F\n",
    "import csv\n",
    "import random\n",
    "import re\n",
    "import os\n",
    "import unicodedata\n",
    "import codecs\n",
    "from io import open\n",
    "import itertools\n",
    "import math\n",
    "\n",
    "# If you have GPU and CUDA libraries, use GPU, else use CPU\n",
    "USE_CUDA = torch.cuda.is_available()\n",
    "device = torch.device(\"cuda\" if USE_CUDA else \"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8b6e299",
   "metadata": {},
   "source": [
    "### Utility Functions\n",
    "1. printLines() : to print lines of a text file\n",
    "2. loadLines() : Splits each line of the file into a dictionary of fields\n",
    "3. loadConversations() : Groups fields of lines read using loadLines() function into conversations based on the file *movie_conversations.txt*\n",
    "4. extractSentencePairs() : Extracts pairs of sentences from conversations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "29f25790",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b'L1045 +++$+++ u0 +++$+++ m0 +++$+++ BIANCA +++$+++ They do not!\\n'\n",
      "b'L1044 +++$+++ u2 +++$+++ m0 +++$+++ CAMERON +++$+++ They do to!\\n'\n",
      "b'L985 +++$+++ u0 +++$+++ m0 +++$+++ BIANCA +++$+++ I hope so.\\n'\n",
      "b'L984 +++$+++ u2 +++$+++ m0 +++$+++ CAMERON +++$+++ She okay?\\n'\n",
      "b\"L925 +++$+++ u0 +++$+++ m0 +++$+++ BIANCA +++$+++ Let's go.\\n\"\n",
      "b'L924 +++$+++ u2 +++$+++ m0 +++$+++ CAMERON +++$+++ Wow\\n'\n",
      "b\"L872 +++$+++ u0 +++$+++ m0 +++$+++ BIANCA +++$+++ Okay -- you're gonna need to learn how to lie.\\n\"\n",
      "b'L871 +++$+++ u2 +++$+++ m0 +++$+++ CAMERON +++$+++ No\\n'\n",
      "b'L870 +++$+++ u0 +++$+++ m0 +++$+++ BIANCA +++$+++ I\\'m kidding.  You know how sometimes you just become this \"persona\"?  And you don\\'t know how to quit?\\n'\n",
      "b'L869 +++$+++ u0 +++$+++ m0 +++$+++ BIANCA +++$+++ Like my fear of wearing pastels?\\n'\n"
     ]
    }
   ],
   "source": [
    "# Printing lines for file movie_lines.txt\n",
    "corpus_name = \"cornell movie-dialogs corpus\"\n",
    "corpus = os.path.join(\"data\", corpus_name)\n",
    "\n",
    "def printLines(file, n=10):\n",
    "    with open(file, 'rb') as datafile:\n",
    "        lines = datafile.readlines()\n",
    "    for line in lines[:n]:\n",
    "        print(line)\n",
    "\n",
    "printLines(os.path.join(corpus, \"movie_lines.txt\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e67e7296",
   "metadata": {},
   "source": [
    "#### 1. Reformat the  data file and load the data into structures suitable for executing with PyTorch.\n",
    " - The below code parses the movie_lines.txt file and the formats it such that each line contains a tab-separated query sentence and a response sentence pair.\n",
    "- The function loadLines() splits each line of the file into a dictionary of fields (lineID, characterID, movieID, character, text), \n",
    "- the function loadConversations() groups fields of lines from loadLines into conversations based on movie_conversations.txt, \n",
    "- function extractSentencePairs() extracts pairs of sentences from conversations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "17169242",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data pre-processing Functions\n",
    "# Splits each line of the file into a dictionary of fields\n",
    "def loadLines(fileName, fields):\n",
    "    lines = {}\n",
    "    with open(fileName, 'r', encoding='iso-8859-1') as f:\n",
    "        for line in f:\n",
    "            values = line.split(\" +++$+++ \")\n",
    "            # Extract fields\n",
    "            lineObj = {}\n",
    "            for i, field in enumerate(fields):\n",
    "                lineObj[field] = values[i]\n",
    "            lines[lineObj['lineID']] = lineObj\n",
    "    return lines\n",
    "\n",
    "\n",
    "# Groups fields of lines from `loadLines` into conversations based on *movie_conversations.txt*\n",
    "def loadConversations(fileName, lines, fields):\n",
    "    conversations = []\n",
    "    with open(fileName, 'r', encoding='iso-8859-1') as f:\n",
    "        for line in f:\n",
    "            values = line.split(\" +++$+++ \")\n",
    "            # Extract fields\n",
    "            convObj = {}\n",
    "            for i, field in enumerate(fields):\n",
    "                convObj[field] = values[i]\n",
    "            # Convert string to list (convObj[\"utteranceIDs\"] == \"['L598485', 'L598486', ...]\")\n",
    "            lineIds = eval(convObj[\"utteranceIDs\"])\n",
    "            # Reassemble lines\n",
    "            convObj[\"lines\"] = []\n",
    "            for lineId in lineIds:\n",
    "                convObj[\"lines\"].append(lines[lineId])\n",
    "            conversations.append(convObj)\n",
    "    return conversations\n",
    "\n",
    "\n",
    "# Extracts pairs of sentences from conversations\n",
    "def extractSentencePairs(conversations):\n",
    "    qa_pairs = []\n",
    "    for conversation in conversations:\n",
    "        # Iterate over all the lines of the conversation\n",
    "        for i in range(len(conversation[\"lines\"]) - 1):  # We ignore the last line (no answer for it)\n",
    "            inputLine = conversation[\"lines\"][i][\"text\"].strip()\n",
    "            targetLine = conversation[\"lines\"][i+1][\"text\"].strip()\n",
    "            # Filter wrong samples (if one of the lists is empty)\n",
    "            if inputLine and targetLine:\n",
    "                qa_pairs.append([inputLine, targetLine])\n",
    "    return qa_pairs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcfd1f19",
   "metadata": {},
   "source": [
    "#### 2. Creating the file formatted_movie_lines.txt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "08fb05d3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Processing corpus...\n",
      "\n",
      "Loading conversations...\n",
      "\n",
      "Writing newly formatted file...\n",
      "\n",
      "Sample lines from file:\n",
      "b\"Can we make this quick?  Roxanne Korrine and Andrew Barrett are having an incredibly horrendous public break- up on the quad.  Again.\\tWell, I thought we'd start with pronunciation, if that's okay with you.\\r\\n\"\n",
      "b\"Well, I thought we'd start with pronunciation, if that's okay with you.\\tNot the hacking and gagging and spitting part.  Please.\\r\\n\"\n",
      "b\"Not the hacking and gagging and spitting part.  Please.\\tOkay... then how 'bout we try out some French cuisine.  Saturday?  Night?\\r\\n\"\n",
      "b\"You're asking me out.  That's so cute. What's your name again?\\tForget it.\\r\\n\"\n",
      "b\"No, no, it's my fault -- we didn't have a proper introduction ---\\tCameron.\\r\\n\"\n",
      "b\"Cameron.\\tThe thing is, Cameron -- I'm at the mercy of a particularly hideous breed of loser.  My sister.  I can't date until she does.\\r\\n\"\n",
      "b\"The thing is, Cameron -- I'm at the mercy of a particularly hideous breed of loser.  My sister.  I can't date until she does.\\tSeems like she could get a date easy enough...\\r\\n\"\n",
      "b'Why?\\tUnsolved mystery.  She used to be really popular when she started high school, then it was just like she got sick of it or something.\\r\\n'\n",
      "b\"Unsolved mystery.  She used to be really popular when she started high school, then it was just like she got sick of it or something.\\tThat's a shame.\\r\\n\"\n",
      "b'Gosh, if only we could find Kat a boyfriend...\\tLet me see what I can do.\\r\\n'\n"
     ]
    }
   ],
   "source": [
    "# Define path to new file which will contain formatted movie text line by line\n",
    "datafile = os.path.join(corpus, \"formatted_movie_lines.txt\")\n",
    "\n",
    "delimiter = '\\t'\n",
    "# Unescape the delimiter\n",
    "delimiter = str(codecs.decode(delimiter, \"unicode_escape\"))\n",
    "\n",
    "# Initialize lines dict, conversations list, and field ids\n",
    "lines = {}\n",
    "conversations = []\n",
    "MOVIE_LINES_FIELDS = [\"lineID\", \"characterID\", \"movieID\", \"character\", \"text\"]\n",
    "MOVIE_CONVERSATIONS_FIELDS = [\"character1ID\", \"character2ID\", \"movieID\", \"utteranceIDs\"]\n",
    "\n",
    "# Load lines and process conversations\n",
    "print(\"\\nProcessing corpus...\")\n",
    "lines = loadLines(os.path.join(corpus, \"movie_lines.txt\"), MOVIE_LINES_FIELDS)\n",
    "print(\"\\nLoading conversations...\")\n",
    "conversations = loadConversations(os.path.join(corpus, \"movie_conversations.txt\"),\n",
    "                                  lines, MOVIE_CONVERSATIONS_FIELDS)\n",
    "\n",
    "# Write new csv file\n",
    "print(\"\\nWriting newly formatted file...\")\n",
    "with open(datafile, 'w', encoding='utf-8') as outputfile:\n",
    "    writer = csv.writer(outputfile, delimiter=delimiter, lineterminator='\\n')\n",
    "    for pair in extractSentencePairs(conversations):\n",
    "        writer.writerow(pair)\n",
    "\n",
    "# Print a sample of lines\n",
    "print(\"\\nSample lines from file:\")\n",
    "printLines(datafile)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "006f240a",
   "metadata": {},
   "source": [
    "#### 3. Load and trim data\n",
    "- Next we create a vocabulary and load query/response sentence pairs into memory.\n",
    "\n",
    "- We will also create a mapping for each unique word in dataset to a numerical index value. \n",
    "\n",
    "- For this, the code has defined a *Voc* class. *Voc* class keeps a mapping from words to indexes, a reverse mapping of indexes to words, a count of each word and a total word count. \n",
    "\n",
    "- Voc class provides following methods for manipulating the vocabulary: \n",
    "         - addWord(): for addign a word to the vocabulary\n",
    "         - addSentence(): for adding all words in a sentence to vocaculary\n",
    "         - trim(): for trimming (filtering out) infrequently seen words  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "fe315219",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Default word tokens\n",
    "PAD_token = 0  # Used for padding short sentences\n",
    "SOS_token = 1  # Start-of-sentence token\n",
    "EOS_token = 2  # End-of-sentence token\n",
    "\n",
    "class Voc:\n",
    "    def __init__(self, name):\n",
    "        self.name = name\n",
    "        self.trimmed = False\n",
    "        self.word2index = {}\n",
    "        self.word2count = {}\n",
    "        self.index2word = {PAD_token: \"PAD\", SOS_token: \"SOS\", EOS_token: \"EOS\"}\n",
    "        self.num_words = 3  # Count SOS, EOS, PAD\n",
    "\n",
    "    def addSentence(self, sentence):\n",
    "        for word in sentence.split(' '):\n",
    "            self.addWord(word)\n",
    "\n",
    "    def addWord(self, word):\n",
    "        if word not in self.word2index:\n",
    "            self.word2index[word] = self.num_words\n",
    "            self.word2count[word] = 1\n",
    "            self.index2word[self.num_words] = word\n",
    "            self.num_words += 1\n",
    "        else:\n",
    "            self.word2count[word] += 1\n",
    "\n",
    "    # Remove words below a certain count threshold\n",
    "    def trim(self, min_count):\n",
    "        if self.trimmed:\n",
    "            return\n",
    "        self.trimmed = True\n",
    "\n",
    "        keep_words = []\n",
    "\n",
    "        for k, v in self.word2count.items():\n",
    "            if v >= min_count:\n",
    "                keep_words.append(k)\n",
    "\n",
    "        print('keep_words {} / {} = {:.4f}'.format(\n",
    "            len(keep_words), len(self.word2index), len(keep_words) / len(self.word2index)\n",
    "        ))\n",
    "\n",
    "        # Reinitialize dictionaries\n",
    "        self.word2index = {}\n",
    "        self.word2count = {}\n",
    "        self.index2word = {PAD_token: \"PAD\", SOS_token: \"SOS\", EOS_token: \"EOS\"}\n",
    "        self.num_words = 3 # Count default tokens\n",
    "\n",
    "        for word in keep_words:\n",
    "            self.addWord(word)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f1c8c0d",
   "metadata": {},
   "source": [
    "#### 4. Preprocessing our vocabulary and query/response sentence pairs. \n",
    "\n",
    "We have following functions to preprocess vocabulary and query-response sentence pairs:\n",
    "\n",
    "    - Function unicodeToAscii() to convert the Unicode strings to ASCII. \n",
    "    - Function normalizeString() to convert all letters to lowercase and trim all non-letter characters.\n",
    "    - Function filterPairs() to filter out sentences with length greater than the MAX_LENGTH threshold."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "789d55ac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Start preparing training data ...\n",
      "Reading lines...\n",
      "Read 221282 sentence pairs\n",
      "Trimmed to 64271 sentence pairs\n",
      "Counting words...\n",
      "Counted words: 18008\n",
      "\n",
      "pairs:\n",
      "['there .', 'where ?']\n",
      "['you have my word . as a gentleman', 'you re sweet .']\n",
      "['hi .', 'looks like things worked out tonight huh ?']\n",
      "['you know chastity ?', 'i believe we share an art instructor']\n",
      "['have fun tonight ?', 'tons']\n",
      "['well no . . .', 'then that s all you had to say .']\n",
      "['then that s all you had to say .', 'but']\n",
      "['but', 'you always been this selfish ?']\n",
      "['do you listen to this crap ?', 'what crap ?']\n",
      "['what good stuff ?', 'the real you .']\n"
     ]
    }
   ],
   "source": [
    "MAX_LENGTH = 10  # Maximum sentence length to consider\n",
    "\n",
    "# Turn a Unicode string to plain ASCII, thanks to\n",
    "# http://stackoverflow.com/a/518232/2809427\n",
    "def unicodeToAscii(s):\n",
    "    return ''.join(\n",
    "        c for c in unicodedata.normalize('NFD', s)\n",
    "        if unicodedata.category(c) != 'Mn'\n",
    "    )\n",
    "\n",
    "# Lowercase, trim, and remove non-letter characters\n",
    "def normalizeString(s):\n",
    "    s = unicodeToAscii(s.lower().strip())\n",
    "    s = re.sub(r\"([.!?])\", r\" \\1\", s)\n",
    "    s = re.sub(r\"[^a-zA-Z.!?]+\", r\" \", s)\n",
    "    s = re.sub(r\"\\s+\", r\" \", s).strip()\n",
    "    return s\n",
    "\n",
    "# Read query/response pairs and return a voc object\n",
    "def readVocs(datafile, corpus_name):\n",
    "    print(\"Reading lines...\")\n",
    "    # Read the file and split into lines\n",
    "    lines = open(datafile, encoding='utf-8').\\\n",
    "        read().strip().split('\\n')\n",
    "    # Split every line into pairs and normalize\n",
    "    pairs = [[normalizeString(s) for s in l.split('\\t')] for l in lines]\n",
    "    voc = Voc(corpus_name)\n",
    "    return voc, pairs\n",
    "\n",
    "# Returns True iff both sentences in a pair 'p' are under the MAX_LENGTH threshold\n",
    "def filterPair(p):\n",
    "    # Input sequences need to preserve the last word for EOS token\n",
    "    return len(p[0].split(' ')) < MAX_LENGTH and len(p[1].split(' ')) < MAX_LENGTH\n",
    "\n",
    "# Filter pairs using filterPair condition\n",
    "def filterPairs(pairs):\n",
    "    return [pair for pair in pairs if filterPair(pair)]\n",
    "\n",
    "# Using the functions defined above, return a populated voc object and pairs list\n",
    "def loadPrepareData(corpus, corpus_name, datafile, save_dir):\n",
    "    print(\"Start preparing training data ...\")\n",
    "    voc, pairs = readVocs(datafile, corpus_name)\n",
    "    print(\"Read {!s} sentence pairs\".format(len(pairs)))\n",
    "    pairs = filterPairs(pairs)\n",
    "    print(\"Trimmed to {!s} sentence pairs\".format(len(pairs)))\n",
    "    print(\"Counting words...\")\n",
    "    for pair in pairs:\n",
    "        voc.addSentence(pair[0])\n",
    "        voc.addSentence(pair[1])\n",
    "    print(\"Counted words:\", voc.num_words)\n",
    "    return voc, pairs\n",
    "\n",
    "# Load/Assemble voc and pairs\n",
    "save_dir = os.path.join(\"data\", \"save\")\n",
    "voc, pairs = loadPrepareData(corpus, corpus_name, datafile, save_dir)\n",
    "# Print some pairs to validate\n",
    "print(\"\\npairs:\")\n",
    "for pair in pairs[:10]:\n",
    "    print(pair)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f6143c3",
   "metadata": {},
   "source": [
    "#### 5. Trinning rare words and sentence pairs with rare words.\n",
    "    - The function trimRareWords() calls the method voc.trim() to trim the words that are used less than a threshold number of times MIN_COUNT. \n",
    "    - It also filters out sentence pairs with trimmed words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "8ea2efed",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "keep_words 7823 / 18005 = 0.4345\n",
      "Trimmed from 64271 pairs to 53165, 0.8272 of total\n"
     ]
    }
   ],
   "source": [
    "MIN_COUNT = 3    # Minimum word count threshold for trimming\n",
    "\n",
    "def trimRareWords(voc, pairs, MIN_COUNT):\n",
    "    # Trim words used under the MIN_COUNT from the voc\n",
    "    voc.trim(MIN_COUNT)\n",
    "    # Filter out pairs with trimmed words\n",
    "    keep_pairs = []\n",
    "    for pair in pairs:\n",
    "        input_sentence = pair[0]\n",
    "        output_sentence = pair[1]\n",
    "        keep_input = True\n",
    "        keep_output = True\n",
    "        # Check input sentence\n",
    "        for word in input_sentence.split(' '):\n",
    "            if word not in voc.word2index:\n",
    "                keep_input = False\n",
    "                break\n",
    "        # Check output sentence\n",
    "        for word in output_sentence.split(' '):\n",
    "            if word not in voc.word2index:\n",
    "                keep_output = False\n",
    "                break\n",
    "\n",
    "        # Only keep pairs that do not contain trimmed word(s) in their input or output sentence\n",
    "        if keep_input and keep_output:\n",
    "            keep_pairs.append(pair)\n",
    "\n",
    "    print(\"Trimmed from {} pairs to {}, {:.4f} of total\".format(len(pairs), len(keep_pairs), len(keep_pairs) / len(pairs)))\n",
    "    return keep_pairs\n",
    "\n",
    "\n",
    "# Trim voc and pairs\n",
    "pairs = trimRareWords(voc, pairs, MIN_COUNT)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b9b2106",
   "metadata": {},
   "source": [
    "#### 5. Prepare data for models \n",
    "\n",
    "- Now we should convert our data (vocabulary object and list of sentence pairs) to torch tensors. \n",
    "\n",
    "- The function inputVar() converts sentences to a correct shaped, zero-padded tensor. It also returns a tensor of lengths for each of the sequences in the batch which will be passed to our decoder later.\n",
    "\n",
    "- The function outputVar() also converts target sequences to tensors, and it returns a binary mask tensor and a maximum target sentence length. The binary mask tensor has the same shape as the output target tensor, but every element that is a PAD_token is 0 and all others are 1.\n",
    "\n",
    "- The function batch2TrainData() takes in bunch of pairs of sequences and returns the input and target tensors using the inputVar() and outputVar() functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "1c7ecf42",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "input_variable: tensor([[ 840,  273,   25,   61, 2078],\n",
      "        [5354,   53,  132,  132,    4],\n",
      "        [ 410,  859,  250,  718,    2],\n",
      "        [ 380,   98,  421,  173,    0],\n",
      "        [ 169,    9, 2037,    4,    0],\n",
      "        [   7, 1264,    2,    2,    0],\n",
      "        [3761,    4,    0,    0,    0],\n",
      "        [   4,    2,    0,    0,    0],\n",
      "        [   2,    0,    0,    0,    0]])\n",
      "lengths: tensor([9, 8, 6, 6, 3])\n",
      "target_variable: tensor([[ 383,    7,  197,   71, 2076],\n",
      "        [   7,  557,  117,  122,    4],\n",
      "        [   4,    7,  669,  198,    2],\n",
      "        [   2,  132,   25,  111,    0],\n",
      "        [   0, 1419,  200,    6,    0],\n",
      "        [   0,  155,  274,    2,    0],\n",
      "        [   0,    6,    4,    0,    0],\n",
      "        [   0,    2,    2,    0,    0]])\n",
      "mask: tensor([[1, 1, 1, 1, 1],\n",
      "        [1, 1, 1, 1, 1],\n",
      "        [1, 1, 1, 1, 1],\n",
      "        [1, 1, 1, 1, 0],\n",
      "        [0, 1, 1, 1, 0],\n",
      "        [0, 1, 1, 1, 0],\n",
      "        [0, 1, 1, 0, 0],\n",
      "        [0, 1, 1, 0, 0]], dtype=torch.uint8)\n",
      "max_target_len: 8\n"
     ]
    }
   ],
   "source": [
    "def indexesFromSentence(voc, sentence):\n",
    "    return [voc.word2index[word] for word in sentence.split(' ')] + [EOS_token]\n",
    "\n",
    "def zeroPadding(l, fillvalue=PAD_token):\n",
    "    return list(itertools.zip_longest(*l, fillvalue=fillvalue))\n",
    "\n",
    "def binaryMatrix(l, value=PAD_token):\n",
    "    m = []\n",
    "    for i, seq in enumerate(l):\n",
    "        m.append([])\n",
    "        for token in seq:\n",
    "            if token == PAD_token:\n",
    "                m[i].append(0)\n",
    "            else:\n",
    "                m[i].append(1)\n",
    "    return m\n",
    "\n",
    "# Returns padded input sequence tensor and lengths\n",
    "def inputVar(l, voc):\n",
    "    indexes_batch = [indexesFromSentence(voc, sentence) for sentence in l]\n",
    "    lengths = torch.tensor([len(indexes) for indexes in indexes_batch])\n",
    "    padList = zeroPadding(indexes_batch)\n",
    "    padVar = torch.LongTensor(padList)\n",
    "    return padVar, lengths\n",
    "\n",
    "# Returns padded target sequence tensor, padding mask, and max target length\n",
    "def outputVar(l, voc):\n",
    "    indexes_batch = [indexesFromSentence(voc, sentence) for sentence in l]\n",
    "    max_target_len = max([len(indexes) for indexes in indexes_batch])\n",
    "    padList = zeroPadding(indexes_batch)\n",
    "    mask = binaryMatrix(padList)\n",
    "    mask = torch.ByteTensor(mask)\n",
    "    padVar = torch.LongTensor(padList)\n",
    "    return padVar, mask, max_target_len\n",
    "\n",
    "# Returns all items for a given batch of pairs\n",
    "def batch2TrainData(voc, pair_batch):\n",
    "    pair_batch.sort(key=lambda x: len(x[0].split(\" \")), reverse=True)\n",
    "    input_batch, output_batch = [], []\n",
    "    for pair in pair_batch:\n",
    "        input_batch.append(pair[0])\n",
    "        output_batch.append(pair[1])\n",
    "    inp, lengths = inputVar(input_batch, voc)\n",
    "    output, mask, max_target_len = outputVar(output_batch, voc)\n",
    "    return inp, lengths, output, mask, max_target_len\n",
    "\n",
    "\n",
    "# Example for validation\n",
    "small_batch_size = 5\n",
    "batches = batch2TrainData(voc, [random.choice(pairs) for _ in range(small_batch_size)])\n",
    "input_variable, lengths, target_variable, mask, max_target_len = batches\n",
    "\n",
    "print(\"input_variable:\", input_variable)\n",
    "print(\"lengths:\", lengths)\n",
    "print(\"target_variable:\", target_variable)\n",
    "print(\"mask:\", mask)\n",
    "print(\"max_target_len:\", max_target_len)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "162e1df0",
   "metadata": {},
   "source": [
    "#### 6. Define Model Architecture \n",
    "\n",
    "- The model in this tutorial(https://brsoff.github.io/tutorials/beginner/chatbot_tutorial.html) uses <b>encoder-decoder architecture</b>. \n",
    "\n",
    "- Encoder uses <b> bidirectional multi-layered Gated Recurrent Units</b>. There are essentially two independent RNNs: one that is fed the input sequence in normal sequential order, and one that is fed the input sequence in reverse order, so both the past and future context is encoded. \n",
    "\n",
    "- Encoder RNN iterates through the input sentence <b>one token (e.g. word) at a time</b>. \n",
    "\n",
    "- At each time step it outputs an “output” vector and a “hidden state” vector. \n",
    "- The hidden state vector is then passed to the next time step, while the output vector is recorded</b>. \n",
    "\n",
    "- This context vector (the final hidden layer of the RNN) contains semantic information about the query sentence (sentence entered by user interacting with the chatbot). Encoder RNN encodes a variable length input sequence to a fixed length context vector.\n",
    "\n",
    "- Thus the encoder transforms the context it saw at each point in the sequence into a set of points in a high-dimensional space, which the decoder will use to generate a meaningful output for the given task.\n",
    "\n",
    "##### Bidirectional RNN:\n",
    "\n",
    "<img src = 'RNN-bidirectional.png'>\n",
    "<center> Source: Image source: http://colah.github.io/posts/2015-09-NN-Types-FP/</center>\n",
    "\n",
    "- <b>An embedding layer is used to encode word indices in an arbitrarily sized feature space</b>. For this model, the embedding layer will map each word to a feature space of size <b>hidden_size</b>. When trained, these values encode semantic similarity between similar meaning words.\n",
    "\n",
    "- The functions torch.nn.utils.rnn.pack_padded_sequence() and torch.nn.utils.rnn.pad_packed_sequence() pack and unpack padding around the batch of sequences.\n",
    "\n",
    "- Decoder RNN takes an input word and the context vector and returns a guess for the next word in the sequence and a hidden state to use in the next iteration. \n",
    "\n",
    "#### Sequence of Operations:\n",
    "\n",
    "1. Convert word indexes to embeddings.\n",
    "2. Pack padded batch of sequences for RNN module.\n",
    "3. Forward pass through GRU.\n",
    "4. Unpack padding.\n",
    "5. Sum bidirectional GRU outputs.\n",
    "6. Return output and final hidden state."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "f0871f4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "class EncoderRNN(nn.Module):\n",
    "    def __init__(self, hidden_size, embedding, n_layers=1, dropout=0):\n",
    "        super(EncoderRNN, self).__init__()\n",
    "        self.n_layers = n_layers\n",
    "        self.hidden_size = hidden_size\n",
    "        self.embedding = embedding\n",
    "\n",
    "        # Initialize GRU; the input_size and hidden_size params are both set to 'hidden_size'\n",
    "        #   because our input size is a word embedding with number of features == hidden_size\n",
    "        self.gru = nn.GRU(hidden_size, hidden_size, n_layers,\n",
    "                          dropout=(0 if n_layers == 1 else dropout), bidirectional=True)\n",
    "\n",
    "    def forward(self, input_seq, input_lengths, hidden=None):\n",
    "        # Convert word indexes to embeddings\n",
    "        embedded = self.embedding(input_seq)\n",
    "        # Pack padded batch of sequences for RNN module\n",
    "        packed = torch.nn.utils.rnn.pack_padded_sequence(embedded, input_lengths)\n",
    "        # Forward pass through GRU\n",
    "        outputs, hidden = self.gru(packed, hidden)\n",
    "        # Unpack padding\n",
    "        outputs, _ = torch.nn.utils.rnn.pad_packed_sequence(outputs)\n",
    "        # Sum bidirectional GRU outputs\n",
    "        outputs = outputs[:, :, :self.hidden_size] + outputs[:, : ,self.hidden_size:]\n",
    "        # Return output and final hidden state\n",
    "        return outputs, hidden"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "637c6de3",
   "metadata": {},
   "source": [
    "#### 6. Model Architecture : Decoder with Attention Mechanism\n",
    "\n",
    "- The decoder RNN generates the response sentence in a token-by-token fashion. \n",
    "- It uses the encoder’s context vectors, and internal hidden states to generate the next word in the sequence. \n",
    "- It continues generating words until it outputs an <b>EOS_token</b>, representing the end of the sentence. \n",
    "\n",
    "- <b>Attention Mechanism </b>: Attention mechanism helps avoid information loss in long sequences. It allows the decoder to pay attention to certain parts of the input sequence, rather than using the entire fixed context at every step.\n",
    "\n",
    "- Attention weights are calculated using the decoder’s current hidden state and the encoder’s outputs. The output attention weights have the same shape as the input sequence, allowing us to multiply them by the encoder outputs, giving us a weighted sum which indicates the parts of encoder output to pay attention to. \n",
    "\n",
    "<img src = 'attn2.png'>\n",
    "<center> Source: https://brsoff.github.io/tutorials/beginner/chatbot_tutorial.html</center>\n",
    "\n",
    "- Global attention considers all of the encoder’s hidden states, as opposed to Local attention, which only considers the encoder’s hidden state from the current time step. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "6a0abf0b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Luong attention layer\n",
    "class Attn(torch.nn.Module):\n",
    "    def __init__(self, method, hidden_size):\n",
    "        super(Attn, self).__init__()\n",
    "        self.method = method\n",
    "        if self.method not in ['dot', 'general', 'concat']:\n",
    "            raise ValueError(self.method, \"is not an appropriate attention method.\")\n",
    "        self.hidden_size = hidden_size\n",
    "        if self.method == 'general':\n",
    "            self.attn = torch.nn.Linear(self.hidden_size, hidden_size)\n",
    "        elif self.method == 'concat':\n",
    "            self.attn = torch.nn.Linear(self.hidden_size * 2, hidden_size)\n",
    "            self.v = torch.nn.Parameter(torch.FloatTensor(hidden_size))\n",
    "\n",
    "    def dot_score(self, hidden, encoder_output):\n",
    "        return torch.sum(hidden * encoder_output, dim=2)\n",
    "\n",
    "    def general_score(self, hidden, encoder_output):\n",
    "        energy = self.attn(encoder_output)\n",
    "        return torch.sum(hidden * energy, dim=2)\n",
    "\n",
    "    def concat_score(self, hidden, encoder_output):\n",
    "        energy = self.attn(torch.cat((hidden.expand(encoder_output.size(0), -1, -1), encoder_output), 2)).tanh()\n",
    "        return torch.sum(self.v * energy, dim=2)\n",
    "\n",
    "    def forward(self, hidden, encoder_outputs):\n",
    "        # Calculate the attention weights (energies) based on the given method\n",
    "        if self.method == 'general':\n",
    "            attn_energies = self.general_score(hidden, encoder_outputs)\n",
    "        elif self.method == 'concat':\n",
    "            attn_energies = self.concat_score(hidden, encoder_outputs)\n",
    "        elif self.method == 'dot':\n",
    "            attn_energies = self.dot_score(hidden, encoder_outputs)\n",
    "\n",
    "        # Transpose max_length and batch_size dimensions\n",
    "        attn_energies = attn_energies.t()\n",
    "\n",
    "        # Return the softmax normalized probability scores (with added dimension)\n",
    "        return F.softmax(attn_energies, dim=1).unsqueeze(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac9ee3af",
   "metadata": {},
   "source": [
    "#### 6. Model Architecture: Implementation of decoder module. \n",
    "\n",
    "- For the decoder, we will manually feed our batch one time step at a time. This means that our embedded word tensor and GRU output will both have shape (1, batch_size, hidden_size).\n",
    "\n",
    "###### Sequence of Operations:\n",
    "\n",
    "1. Get embedding of current input word.\n",
    "2. Forward through unidirectional GRU.\n",
    "3. Calculate attention weights from the current GRU output from (2).\n",
    "4. Multiply attention weights to encoder outputs to get new “weighted sum” context vector.\n",
    "5. Concatenate weighted context vector and GRU output.\n",
    "6. Predict next word.\n",
    "7. Return output and final hidden state.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "517530a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "class LuongAttnDecoderRNN(nn.Module):\n",
    "    def __init__(self, attn_model, embedding, hidden_size, output_size, n_layers=1, dropout=0.1):\n",
    "        super(LuongAttnDecoderRNN, self).__init__()\n",
    "\n",
    "        # Keep for reference\n",
    "        self.attn_model = attn_model\n",
    "        self.hidden_size = hidden_size\n",
    "        self.output_size = output_size\n",
    "        self.n_layers = n_layers\n",
    "        self.dropout = dropout\n",
    "\n",
    "        # Define layers\n",
    "        self.embedding = embedding\n",
    "        self.embedding_dropout = nn.Dropout(dropout)\n",
    "        self.gru = nn.GRU(hidden_size, hidden_size, n_layers, dropout=(0 if n_layers == 1 else dropout))\n",
    "        self.concat = nn.Linear(hidden_size * 2, hidden_size)\n",
    "        self.out = nn.Linear(hidden_size, output_size)\n",
    "\n",
    "        self.attn = Attn(attn_model, hidden_size)\n",
    "\n",
    "    def forward(self, input_step, last_hidden, encoder_outputs):\n",
    "        # Note: we run this one step (word) at a time\n",
    "        # Get embedding of current input word\n",
    "        embedded = self.embedding(input_step)\n",
    "        embedded = self.embedding_dropout(embedded)\n",
    "        # Forward through unidirectional GRU\n",
    "        rnn_output, hidden = self.gru(embedded, last_hidden)\n",
    "        # Calculate attention weights from the current GRU output\n",
    "        attn_weights = self.attn(rnn_output, encoder_outputs)\n",
    "        # Multiply attention weights to encoder outputs to get new \"weighted sum\" context vector\n",
    "        context = attn_weights.bmm(encoder_outputs.transpose(0, 1))\n",
    "        # Concatenate weighted context vector and GRU output using Luong eq. 5\n",
    "        rnn_output = rnn_output.squeeze(0)\n",
    "        context = context.squeeze(1)\n",
    "        concat_input = torch.cat((rnn_output, context), 1)\n",
    "        concat_output = torch.tanh(self.concat(concat_input))\n",
    "        # Predict next word using Luong eq. 6\n",
    "        output = self.out(concat_output)\n",
    "        output = F.softmax(output, dim=1)\n",
    "        # Return output and final hidden state\n",
    "        return output, hidden"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "854039cb",
   "metadata": {},
   "source": [
    "#### 7. Training Procedure : masked loss function\n",
    "\n",
    "- maskNLLLoss() function calculates the average negative log likelihood of the elements that correspond to a 1 in the mask tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "d329b95f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def maskNLLLoss(inp, target, mask):\n",
    "    nTotal = mask.sum()\n",
    "    crossEntropy = -torch.log(torch.gather(inp, 1, target.view(-1, 1)))\n",
    "    loss = crossEntropy.masked_select(mask).mean()\n",
    "    loss = loss.to(device)\n",
    "    return loss, nTotal.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a9fa7c6",
   "metadata": {},
   "source": [
    "#### 7. Training Procedure:  Single training iteration\n",
    "\n",
    "- The train() function contains the algorithm for a single training iteration (a single batch of inputs).\n",
    "- To aid convergence, following tricks can be used: \n",
    "\n",
    "    1. Teacher forcing: This means that with some probability, set by teacher_forcing_ratio, we use the current target word as the decoder’s next input rather than using the decoder’s current guess. This technique helps in more efficient training but can lead to model instability during inference.\n",
    "    \n",
    "    2. Gradient Clipping: This is to counter the “exploding gradient” problem by clipping or thresholding gradients to a maximum value.\n",
    "\n",
    "##### Sequence of Operations:\n",
    "\n",
    "1. Forward pass entire input batch through encoder.\n",
    "2. Initialize decoder inputs as SOS_token, and hidden state as the encoder’s final hidden state.\n",
    "3. Forward input batch sequence through decoder one time step at a time.\n",
    "4. If teacher forcing: set next decoder input as the current target; else: set next decoder input as current decoder output.\n",
    "5. Calculate and accumulate loss.\n",
    "6. Perform backpropagation.\n",
    "7. Clip gradients.\n",
    "8. Update encoder and decoder model parameters.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "7f78e070",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(input_variable, lengths, target_variable, mask, max_target_len, encoder, decoder, embedding,\n",
    "          encoder_optimizer, decoder_optimizer, batch_size, clip, max_length=MAX_LENGTH):\n",
    "\n",
    "    # Zero gradients\n",
    "    encoder_optimizer.zero_grad()\n",
    "    decoder_optimizer.zero_grad()\n",
    "\n",
    "    # Set device options\n",
    "    input_variable = input_variable.to(device)\n",
    "    lengths = lengths.to(device)\n",
    "    target_variable = target_variable.to(device)\n",
    "    mask = mask.to(device)\n",
    "\n",
    "    # Initialize variables\n",
    "    loss = 0\n",
    "    print_losses = []\n",
    "    n_totals = 0\n",
    "\n",
    "    # Forward pass through encoder\n",
    "    encoder_outputs, encoder_hidden = encoder(input_variable, lengths)\n",
    "\n",
    "    # Create initial decoder input (start with SOS tokens for each sentence)\n",
    "    decoder_input = torch.LongTensor([[SOS_token for _ in range(batch_size)]])\n",
    "    decoder_input = decoder_input.to(device)\n",
    "\n",
    "    # Set initial decoder hidden state to the encoder's final hidden state\n",
    "    decoder_hidden = encoder_hidden[:decoder.n_layers]\n",
    "\n",
    "    # Determine if we are using teacher forcing this iteration\n",
    "    use_teacher_forcing = True if random.random() < teacher_forcing_ratio else False\n",
    "\n",
    "    # Forward batch of sequences through decoder one time step at a time\n",
    "    if use_teacher_forcing:\n",
    "        for t in range(max_target_len):\n",
    "            decoder_output, decoder_hidden = decoder(\n",
    "                decoder_input, decoder_hidden, encoder_outputs\n",
    "            )\n",
    "            # Teacher forcing: next input is current target\n",
    "            decoder_input = target_variable[t].view(1, -1)\n",
    "            # Calculate and accumulate loss\n",
    "            mask_loss, nTotal = maskNLLLoss(decoder_output, target_variable[t], mask[t])\n",
    "            loss += mask_loss\n",
    "            print_losses.append(mask_loss.item() * nTotal)\n",
    "            n_totals += nTotal\n",
    "    else:\n",
    "        for t in range(max_target_len):\n",
    "            decoder_output, decoder_hidden = decoder(\n",
    "                decoder_input, decoder_hidden, encoder_outputs\n",
    "            )\n",
    "            # No teacher forcing: next input is decoder's own current output\n",
    "            _, topi = decoder_output.topk(1)\n",
    "            decoder_input = torch.LongTensor([[topi[i][0] for i in range(batch_size)]])\n",
    "            decoder_input = decoder_input.to(device)\n",
    "            # Calculate and accumulate loss\n",
    "            mask_loss, nTotal = maskNLLLoss(decoder_output, target_variable[t], mask[t])\n",
    "            loss += mask_loss\n",
    "            print_losses.append(mask_loss.item() * nTotal)\n",
    "            n_totals += nTotal\n",
    "\n",
    "    # Perform backpropatation\n",
    "    loss.backward()\n",
    "\n",
    "    # Clip gradients: gradients are modified in place\n",
    "    _ = torch.nn.utils.clip_grad_norm_(encoder.parameters(), clip)\n",
    "    _ = torch.nn.utils.clip_grad_norm_(decoder.parameters(), clip)\n",
    "\n",
    "    # Adjust model weights\n",
    "    encoder_optimizer.step()\n",
    "    decoder_optimizer.step()\n",
    "\n",
    "    return sum(print_losses) / n_totals"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acea66dd",
   "metadata": {},
   "source": [
    "#### 7. Training Procedure:  Full training with many training iterations\n",
    "\n",
    "- For full training procedure with data,  trainIters() function runs for n_iterations of training.\n",
    "\n",
    "- Saving the model, saves a tarball containing the encoder and decoder state_dicts (parameters), the optimizers’ state_dicts, the loss, the iteration, etc. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "0277c728",
   "metadata": {},
   "outputs": [],
   "source": [
    "def trainIters(model_name, voc, pairs, encoder, decoder, encoder_optimizer, decoder_optimizer, embedding, encoder_n_layers, decoder_n_layers, save_dir, n_iteration, batch_size, print_every, save_every, clip, corpus_name, loadFilename):\n",
    "\n",
    "    # Load batches for each iteration\n",
    "    training_batches = [batch2TrainData(voc, [random.choice(pairs) for _ in range(batch_size)])\n",
    "                      for _ in range(n_iteration)]\n",
    "\n",
    "    # Initializations\n",
    "    print('Initializing ...')\n",
    "    start_iteration = 1\n",
    "    print_loss = 0\n",
    "    if loadFilename:\n",
    "        start_iteration = checkpoint['iteration'] + 1\n",
    "\n",
    "    # Training loop\n",
    "    print(\"Training...\")\n",
    "    for iteration in range(start_iteration, n_iteration + 1):\n",
    "        training_batch = training_batches[iteration - 1]\n",
    "        # Extract fields from batch\n",
    "        input_variable, lengths, target_variable, mask, max_target_len = training_batch\n",
    "\n",
    "        # Run a training iteration with batch\n",
    "        loss = train(input_variable, lengths, target_variable, mask, max_target_len, encoder,\n",
    "                     decoder, embedding, encoder_optimizer, decoder_optimizer, batch_size, clip)\n",
    "        print_loss += loss\n",
    "\n",
    "        # Print progress\n",
    "        if iteration % print_every == 0:\n",
    "            print_loss_avg = print_loss / print_every\n",
    "            print(\"Iteration: {}; Percent complete: {:.1f}%; Average loss: {:.4f}\".format(iteration, iteration / n_iteration * 100, print_loss_avg))\n",
    "            print_loss = 0\n",
    "\n",
    "        # Save checkpoint\n",
    "        if (iteration % save_every == 0):\n",
    "            directory = os.path.join(save_dir, model_name, corpus_name, '{}-{}_{}'.format(encoder_n_layers, decoder_n_layers, hidden_size))\n",
    "            if not os.path.exists(directory):\n",
    "                os.makedirs(directory)\n",
    "            torch.save({\n",
    "                'iteration': iteration,\n",
    "                'en': encoder.state_dict(),\n",
    "                'de': decoder.state_dict(),\n",
    "                'en_opt': encoder_optimizer.state_dict(),\n",
    "                'de_opt': decoder_optimizer.state_dict(),\n",
    "                'loss': loss,\n",
    "                'voc_dict': voc.__dict__,\n",
    "                'embedding': embedding.state_dict()\n",
    "            }, os.path.join(directory, '{}_{}.tar'.format(iteration, 'checkpoint')))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07e0ac40",
   "metadata": {},
   "source": [
    "#### 8. Define Evaluation Process for Encoded Input :  How the model decodes the encoded input\n",
    "\n",
    "##### GreedySearchDecoder class\n",
    "\n",
    "- The model decodes the encoded input using greedy decoding, i.e. without using teacher forcing. \n",
    "- For each time step, we <b>simply choose the word from decoder_output with the highest softmax value</b>. GreedySearchDecoder class facilites the greedy decoding operation. \n",
    "- When run, an object of this class takes an input sequence (input_seq) of shape (input_seq length, 1), a scalar input length (input_length) tensor, and a max_length to bound the response sentence length. \n",
    "\n",
    "\n",
    "The input sentence is evaluated using the following computational graph:\n",
    "##### Sequence of operations in evaluating input sentence:\n",
    "\n",
    "1. Forward input through encoder model.\n",
    "2. Prepare encoder’s final hidden layer to be first hidden input to the decoder.\n",
    "3. Initialize decoder’s first input as SOS_token.\n",
    "4. Initialize tensors to append decoded words to.\n",
    "5. Iteratively decode one word token at a time:\n",
    "        a. Forward pass through decoder.\n",
    "        b. Obtain most likely word token and its softmax score.\n",
    "        c. Record token and score.\n",
    "        d. Prepare current token to be next decoder input.\n",
    "        e. Return collections of word tokens and scores."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "6a8afadd",
   "metadata": {},
   "outputs": [],
   "source": [
    "class GreedySearchDecoder(nn.Module):\n",
    "    def __init__(self, encoder, decoder):\n",
    "        super(GreedySearchDecoder, self).__init__()\n",
    "        self.encoder = encoder\n",
    "        self.decoder = decoder\n",
    "\n",
    "    def forward(self, input_seq, input_length, max_length):\n",
    "        # Forward input through encoder model\n",
    "        encoder_outputs, encoder_hidden = self.encoder(input_seq, input_length)\n",
    "        # Prepare encoder's final hidden layer to be first hidden input to the decoder\n",
    "        decoder_hidden = encoder_hidden[:decoder.n_layers]\n",
    "        # Initialize decoder input with SOS_token\n",
    "        decoder_input = torch.ones(1, 1, device=device, dtype=torch.long) * SOS_token\n",
    "        # Initialize tensors to append decoded words to\n",
    "        all_tokens = torch.zeros([0], device=device, dtype=torch.long)\n",
    "        all_scores = torch.zeros([0], device=device)\n",
    "        # Iteratively decode one word token at a time\n",
    "        for _ in range(max_length):\n",
    "            # Forward pass through decoder\n",
    "            decoder_output, decoder_hidden = self.decoder(decoder_input, decoder_hidden, encoder_outputs)\n",
    "            # Obtain most likely word token and its softmax score\n",
    "            decoder_scores, decoder_input = torch.max(decoder_output, dim=1)\n",
    "            # Record token and score\n",
    "            all_tokens = torch.cat((all_tokens, decoder_input), dim=0)\n",
    "            all_scores = torch.cat((all_scores, decoder_scores), dim=0)\n",
    "            # Prepare current token to be next decoder input (add a dimension)\n",
    "            decoder_input = torch.unsqueeze(decoder_input, 0)\n",
    "        # Return collections of word tokens and scores\n",
    "        return all_tokens, all_scores"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f04a1e92",
   "metadata": {},
   "source": [
    "#### 9. Handling user Input in evaluate() function\n",
    "- The evaluate() function manages the low-level process of handling the input sentence.\n",
    "\n",
    "    1. It <b>formats the sentence as an input batch of word indexes</b> with batch_size==1 by converting the words of the sentence to their corresponding indexes, and transposing the dimensions to prepare the tensor for our models. \n",
    "    2. It <b>creates a lengths tensor </b>which contains the length of input sentence. In this case, lengths is scalar because we are only evaluating one sentence at a time (batch_size==1). \n",
    "    3. Next, we <b>obtain the decoded response sentence tensor </b>using our GreedySearchDecoder object (searcher). \n",
    "    4. Finally, we <b>convert the response’s indexes to words and return the list of decoded words<b>.\n",
    "\n",
    " - evaluateInput() function acts as the user interface for our chatbot. \n",
    "    - When called, an input text field is displayed in which used can enter query sentence. \n",
    "    - After typing our input sentence and pressing Enter, our text is normalized in the same way as our training data, and is ultimately fed to the evaluate function to obtain a decoded output sentence. \n",
    "    - We loop this process, so we can keep chatting with our bot until we enter either “q” or “quit”.\n",
    "    - Finally, if a sentence is entered that contains a word that is not in the vocabulary, we handle this gracefully by printing an error message and prompting the user to enter another sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "c1f25f45",
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(encoder, decoder, searcher, voc, sentence, max_length=MAX_LENGTH):\n",
    "    ### Format input sentence as a batch\n",
    "    # words -> indexes\n",
    "    indexes_batch = [indexesFromSentence(voc, sentence)]\n",
    "    # Create lengths tensor\n",
    "    lengths = torch.tensor([len(indexes) for indexes in indexes_batch])\n",
    "    # Transpose dimensions of batch to match models' expectations\n",
    "    input_batch = torch.LongTensor(indexes_batch).transpose(0, 1)\n",
    "    # Use appropriate device\n",
    "    input_batch = input_batch.to(device)\n",
    "    lengths = lengths.to(device)\n",
    "    # Decode sentence with searcher\n",
    "    tokens, scores = searcher(input_batch, lengths, max_length)\n",
    "    # indexes -> words\n",
    "    decoded_words = [voc.index2word[token.item()] for token in tokens]\n",
    "    return decoded_words\n",
    "\n",
    "\n",
    "def evaluateInput(encoder, decoder, searcher, voc):\n",
    "    input_sentence = ''\n",
    "    while(1):\n",
    "        try:\n",
    "            # Get input sentence\n",
    "            input_sentence = input('> ')\n",
    "            # Check if it is quit case\n",
    "            if input_sentence == 'q' or input_sentence == 'quit': break\n",
    "            # Normalize sentence\n",
    "            input_sentence = normalizeString(input_sentence)\n",
    "            # Evaluate sentence\n",
    "            output_words = evaluate(encoder, decoder, searcher, voc, input_sentence)\n",
    "            # Format and print response sentence\n",
    "            output_words[:] = [x for x in output_words if not (x == 'EOS' or x == 'PAD')]\n",
    "            print('Bot:', ' '.join(output_words))\n",
    "\n",
    "        except KeyError:\n",
    "            print(\"Error: Encountered unknown word.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd81b1e0",
   "metadata": {},
   "source": [
    "#### 10. Running the encoder and decoder models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "7104edb9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Building encoder and decoder ...\n",
      "Models built and ready to go!\n"
     ]
    }
   ],
   "source": [
    "# Configure models\n",
    "model_name = 'cb_model'\n",
    "attn_model = 'dot'\n",
    "#attn_model = 'general'\n",
    "#attn_model = 'concat'\n",
    "hidden_size = 500\n",
    "encoder_n_layers = 2\n",
    "decoder_n_layers = 2\n",
    "dropout = 0.1\n",
    "batch_size = 64\n",
    "\n",
    "# Set checkpoint to load from; set to None if starting from scratch\n",
    "loadFilename = None\n",
    "checkpoint_iter = 4000\n",
    "#loadFilename = os.path.join(save_dir, model_name, corpus_name,\n",
    "#                            '{}-{}_{}'.format(encoder_n_layers, decoder_n_layers, hidden_size),\n",
    "#                            '{}_checkpoint.tar'.format(checkpoint_iter))\n",
    "\n",
    "\n",
    "# Load model if a loadFilename is provided\n",
    "if loadFilename:\n",
    "    # If loading on same machine the model was trained on\n",
    "    checkpoint = torch.load(loadFilename)\n",
    "    # If loading a model trained on GPU to CPU\n",
    "    #checkpoint = torch.load(loadFilename, map_location=torch.device('cpu'))\n",
    "    encoder_sd = checkpoint['en']\n",
    "    decoder_sd = checkpoint['de']\n",
    "    encoder_optimizer_sd = checkpoint['en_opt']\n",
    "    decoder_optimizer_sd = checkpoint['de_opt']\n",
    "    embedding_sd = checkpoint['embedding']\n",
    "    voc.__dict__ = checkpoint['voc_dict']\n",
    "\n",
    "\n",
    "print('Building encoder and decoder ...')\n",
    "# Initialize word embeddings\n",
    "embedding = nn.Embedding(voc.num_words, hidden_size)\n",
    "if loadFilename:\n",
    "    embedding.load_state_dict(embedding_sd)\n",
    "# Initialize encoder & decoder models\n",
    "encoder = EncoderRNN(hidden_size, embedding, encoder_n_layers, dropout)\n",
    "decoder = LuongAttnDecoderRNN(attn_model, embedding, hidden_size, voc.num_words, decoder_n_layers, dropout)\n",
    "if loadFilename:\n",
    "    encoder.load_state_dict(encoder_sd)\n",
    "    decoder.load_state_dict(decoder_sd)\n",
    "# Use appropriate device\n",
    "encoder = encoder.to(device)\n",
    "decoder = decoder.to(device)\n",
    "print('Models built and ready to go!')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f797e376",
   "metadata": {},
   "source": [
    "#### 10.  Training the model\n",
    "First we set training parameters, then we initialize our optimizers, and finally we call the trainIters() function to run our training iterations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "7a89f383",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Building optimizers ...\n",
      "Starting Training!\n",
      "Initializing ...\n",
      "Training...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\Nimrita Koul\\AppData\\Local\\Temp\\ipykernel_11804\\2489183898.py:4: UserWarning: masked_select received a mask with dtype torch.uint8, this behavior is now deprecated,please use a mask with dtype torch.bool instead. (Triggered internally at C:\\actions-runner\\_work\\pytorch\\pytorch\\builder\\windows\\pytorch\\aten\\src\\ATen\\native\\TensorAdvancedIndexing.cpp:1856.)\n",
      "  loss = crossEntropy.masked_select(mask).mean()\n",
      "C:\\Anaconda3\\lib\\site-packages\\torch\\autograd\\__init__.py:200: UserWarning: masked_scatter_ received a mask with dtype torch.uint8, this behavior is now deprecated,please use a mask with dtype torch.bool instead. (Triggered internally at C:\\actions-runner\\_work\\pytorch\\pytorch\\builder\\windows\\pytorch\\aten\\src\\ATen\\native\\TensorAdvancedIndexing.cpp:2280.)\n",
      "  Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration: 1; Percent complete: 0.0%; Average loss: 8.9669\n",
      "Iteration: 2; Percent complete: 0.1%; Average loss: 8.7626\n",
      "Iteration: 3; Percent complete: 0.1%; Average loss: 8.4577\n",
      "Iteration: 4; Percent complete: 0.1%; Average loss: 8.1589\n",
      "Iteration: 5; Percent complete: 0.1%; Average loss: 7.5425\n",
      "Iteration: 6; Percent complete: 0.1%; Average loss: 7.0812\n",
      "Iteration: 7; Percent complete: 0.2%; Average loss: 6.6605\n",
      "Iteration: 8; Percent complete: 0.2%; Average loss: 6.7813\n",
      "Iteration: 9; Percent complete: 0.2%; Average loss: 6.6480\n",
      "Iteration: 10; Percent complete: 0.2%; Average loss: 6.6316\n",
      "Iteration: 11; Percent complete: 0.3%; Average loss: 6.1140\n",
      "Iteration: 12; Percent complete: 0.3%; Average loss: 5.8382\n",
      "Iteration: 13; Percent complete: 0.3%; Average loss: 5.6427\n",
      "Iteration: 14; Percent complete: 0.4%; Average loss: 5.2091\n",
      "Iteration: 15; Percent complete: 0.4%; Average loss: 4.9672\n",
      "Iteration: 16; Percent complete: 0.4%; Average loss: 4.8860\n",
      "Iteration: 17; Percent complete: 0.4%; Average loss: 4.5972\n",
      "Iteration: 18; Percent complete: 0.4%; Average loss: 4.4938\n",
      "Iteration: 19; Percent complete: 0.5%; Average loss: 4.5448\n",
      "Iteration: 20; Percent complete: 0.5%; Average loss: 4.6290\n",
      "Iteration: 21; Percent complete: 0.5%; Average loss: 4.2696\n",
      "Iteration: 22; Percent complete: 0.5%; Average loss: 4.2192\n",
      "Iteration: 23; Percent complete: 0.6%; Average loss: 4.0020\n",
      "Iteration: 24; Percent complete: 0.6%; Average loss: 4.3598\n",
      "Iteration: 25; Percent complete: 0.6%; Average loss: 4.5596\n",
      "Iteration: 26; Percent complete: 0.7%; Average loss: 4.1842\n",
      "Iteration: 27; Percent complete: 0.7%; Average loss: 4.4780\n",
      "Iteration: 28; Percent complete: 0.7%; Average loss: 4.3599\n",
      "Iteration: 29; Percent complete: 0.7%; Average loss: 4.3126\n",
      "Iteration: 30; Percent complete: 0.8%; Average loss: 4.3540\n",
      "Iteration: 31; Percent complete: 0.8%; Average loss: 4.3443\n",
      "Iteration: 32; Percent complete: 0.8%; Average loss: 4.0074\n",
      "Iteration: 33; Percent complete: 0.8%; Average loss: 4.0698\n",
      "Iteration: 34; Percent complete: 0.9%; Average loss: 4.1791\n",
      "Iteration: 35; Percent complete: 0.9%; Average loss: 4.0661\n",
      "Iteration: 36; Percent complete: 0.9%; Average loss: 4.1125\n",
      "Iteration: 37; Percent complete: 0.9%; Average loss: 4.0763\n",
      "Iteration: 38; Percent complete: 0.9%; Average loss: 4.1222\n",
      "Iteration: 39; Percent complete: 1.0%; Average loss: 4.0310\n",
      "Iteration: 40; Percent complete: 1.0%; Average loss: 3.8644\n",
      "Iteration: 41; Percent complete: 1.0%; Average loss: 4.0638\n",
      "Iteration: 42; Percent complete: 1.1%; Average loss: 3.9635\n",
      "Iteration: 43; Percent complete: 1.1%; Average loss: 4.0260\n",
      "Iteration: 44; Percent complete: 1.1%; Average loss: 3.9923\n",
      "Iteration: 45; Percent complete: 1.1%; Average loss: 4.0418\n",
      "Iteration: 46; Percent complete: 1.1%; Average loss: 4.1256\n",
      "Iteration: 47; Percent complete: 1.2%; Average loss: 4.0651\n",
      "Iteration: 48; Percent complete: 1.2%; Average loss: 3.9200\n",
      "Iteration: 49; Percent complete: 1.2%; Average loss: 3.8081\n",
      "Iteration: 50; Percent complete: 1.2%; Average loss: 3.9667\n",
      "Iteration: 51; Percent complete: 1.3%; Average loss: 3.7865\n",
      "Iteration: 52; Percent complete: 1.3%; Average loss: 4.1065\n",
      "Iteration: 53; Percent complete: 1.3%; Average loss: 4.0428\n",
      "Iteration: 54; Percent complete: 1.4%; Average loss: 3.9941\n",
      "Iteration: 55; Percent complete: 1.4%; Average loss: 3.8587\n",
      "Iteration: 56; Percent complete: 1.4%; Average loss: 3.9542\n",
      "Iteration: 57; Percent complete: 1.4%; Average loss: 4.0507\n",
      "Iteration: 58; Percent complete: 1.5%; Average loss: 4.0244\n",
      "Iteration: 59; Percent complete: 1.5%; Average loss: 4.0065\n",
      "Iteration: 60; Percent complete: 1.5%; Average loss: 4.1075\n",
      "Iteration: 61; Percent complete: 1.5%; Average loss: 3.9039\n",
      "Iteration: 62; Percent complete: 1.6%; Average loss: 3.8145\n",
      "Iteration: 63; Percent complete: 1.6%; Average loss: 4.0717\n",
      "Iteration: 64; Percent complete: 1.6%; Average loss: 3.9466\n",
      "Iteration: 65; Percent complete: 1.6%; Average loss: 3.9480\n",
      "Iteration: 66; Percent complete: 1.7%; Average loss: 3.9679\n",
      "Iteration: 67; Percent complete: 1.7%; Average loss: 4.1981\n",
      "Iteration: 68; Percent complete: 1.7%; Average loss: 3.7566\n",
      "Iteration: 69; Percent complete: 1.7%; Average loss: 3.8651\n",
      "Iteration: 70; Percent complete: 1.8%; Average loss: 3.9084\n",
      "Iteration: 71; Percent complete: 1.8%; Average loss: 3.8230\n",
      "Iteration: 72; Percent complete: 1.8%; Average loss: 3.9315\n",
      "Iteration: 73; Percent complete: 1.8%; Average loss: 3.6198\n",
      "Iteration: 74; Percent complete: 1.8%; Average loss: 3.9006\n",
      "Iteration: 75; Percent complete: 1.9%; Average loss: 3.5715\n",
      "Iteration: 76; Percent complete: 1.9%; Average loss: 4.0859\n",
      "Iteration: 77; Percent complete: 1.9%; Average loss: 3.6717\n",
      "Iteration: 78; Percent complete: 1.9%; Average loss: 3.8029\n",
      "Iteration: 79; Percent complete: 2.0%; Average loss: 3.8178\n",
      "Iteration: 80; Percent complete: 2.0%; Average loss: 3.9718\n",
      "Iteration: 81; Percent complete: 2.0%; Average loss: 3.7421\n",
      "Iteration: 82; Percent complete: 2.1%; Average loss: 3.7082\n",
      "Iteration: 83; Percent complete: 2.1%; Average loss: 3.3482\n",
      "Iteration: 84; Percent complete: 2.1%; Average loss: 3.6908\n",
      "Iteration: 85; Percent complete: 2.1%; Average loss: 3.6532\n",
      "Iteration: 86; Percent complete: 2.1%; Average loss: 3.8509\n",
      "Iteration: 87; Percent complete: 2.2%; Average loss: 4.1686\n",
      "Iteration: 88; Percent complete: 2.2%; Average loss: 3.8007\n",
      "Iteration: 89; Percent complete: 2.2%; Average loss: 3.8199\n",
      "Iteration: 90; Percent complete: 2.2%; Average loss: 3.6598\n",
      "Iteration: 91; Percent complete: 2.3%; Average loss: 3.9322\n",
      "Iteration: 92; Percent complete: 2.3%; Average loss: 3.8361\n",
      "Iteration: 93; Percent complete: 2.3%; Average loss: 3.8781\n",
      "Iteration: 94; Percent complete: 2.4%; Average loss: 3.6526\n",
      "Iteration: 95; Percent complete: 2.4%; Average loss: 3.6727\n",
      "Iteration: 96; Percent complete: 2.4%; Average loss: 3.8693\n",
      "Iteration: 97; Percent complete: 2.4%; Average loss: 3.6280\n",
      "Iteration: 98; Percent complete: 2.5%; Average loss: 3.6690\n",
      "Iteration: 99; Percent complete: 2.5%; Average loss: 3.7095\n",
      "Iteration: 100; Percent complete: 2.5%; Average loss: 3.8199\n",
      "Iteration: 101; Percent complete: 2.5%; Average loss: 3.6912\n",
      "Iteration: 102; Percent complete: 2.5%; Average loss: 3.8918\n",
      "Iteration: 103; Percent complete: 2.6%; Average loss: 3.6669\n",
      "Iteration: 104; Percent complete: 2.6%; Average loss: 3.8398\n",
      "Iteration: 105; Percent complete: 2.6%; Average loss: 3.6592\n",
      "Iteration: 106; Percent complete: 2.6%; Average loss: 3.5015\n",
      "Iteration: 107; Percent complete: 2.7%; Average loss: 3.4292\n",
      "Iteration: 108; Percent complete: 2.7%; Average loss: 3.7847\n",
      "Iteration: 109; Percent complete: 2.7%; Average loss: 3.4810\n",
      "Iteration: 110; Percent complete: 2.8%; Average loss: 3.9414\n",
      "Iteration: 111; Percent complete: 2.8%; Average loss: 3.4593\n",
      "Iteration: 112; Percent complete: 2.8%; Average loss: 3.6637\n",
      "Iteration: 113; Percent complete: 2.8%; Average loss: 3.7428\n",
      "Iteration: 114; Percent complete: 2.9%; Average loss: 4.0538\n",
      "Iteration: 115; Percent complete: 2.9%; Average loss: 3.6042\n",
      "Iteration: 116; Percent complete: 2.9%; Average loss: 3.7260\n",
      "Iteration: 117; Percent complete: 2.9%; Average loss: 3.8418\n",
      "Iteration: 118; Percent complete: 2.9%; Average loss: 3.7869\n",
      "Iteration: 119; Percent complete: 3.0%; Average loss: 3.7061\n",
      "Iteration: 120; Percent complete: 3.0%; Average loss: 3.8558\n",
      "Iteration: 121; Percent complete: 3.0%; Average loss: 3.6458\n",
      "Iteration: 122; Percent complete: 3.0%; Average loss: 3.8160\n",
      "Iteration: 123; Percent complete: 3.1%; Average loss: 3.6919\n",
      "Iteration: 124; Percent complete: 3.1%; Average loss: 3.8405\n",
      "Iteration: 125; Percent complete: 3.1%; Average loss: 3.8813\n",
      "Iteration: 126; Percent complete: 3.1%; Average loss: 3.6097\n",
      "Iteration: 127; Percent complete: 3.2%; Average loss: 3.7940\n",
      "Iteration: 128; Percent complete: 3.2%; Average loss: 3.9490\n",
      "Iteration: 129; Percent complete: 3.2%; Average loss: 3.8163\n",
      "Iteration: 130; Percent complete: 3.2%; Average loss: 3.7042\n",
      "Iteration: 131; Percent complete: 3.3%; Average loss: 3.6999\n",
      "Iteration: 132; Percent complete: 3.3%; Average loss: 3.5875\n",
      "Iteration: 133; Percent complete: 3.3%; Average loss: 3.6564\n",
      "Iteration: 134; Percent complete: 3.4%; Average loss: 3.5980\n",
      "Iteration: 135; Percent complete: 3.4%; Average loss: 3.5981\n",
      "Iteration: 136; Percent complete: 3.4%; Average loss: 3.6631\n",
      "Iteration: 137; Percent complete: 3.4%; Average loss: 3.7909\n",
      "Iteration: 138; Percent complete: 3.5%; Average loss: 3.7929\n",
      "Iteration: 139; Percent complete: 3.5%; Average loss: 3.4977\n",
      "Iteration: 140; Percent complete: 3.5%; Average loss: 3.5232\n",
      "Iteration: 141; Percent complete: 3.5%; Average loss: 3.7668\n",
      "Iteration: 142; Percent complete: 3.5%; Average loss: 3.5488\n",
      "Iteration: 143; Percent complete: 3.6%; Average loss: 3.6599\n",
      "Iteration: 144; Percent complete: 3.6%; Average loss: 3.8369\n",
      "Iteration: 145; Percent complete: 3.6%; Average loss: 3.4300\n",
      "Iteration: 146; Percent complete: 3.6%; Average loss: 3.4853\n",
      "Iteration: 147; Percent complete: 3.7%; Average loss: 3.5955\n",
      "Iteration: 148; Percent complete: 3.7%; Average loss: 3.6390\n",
      "Iteration: 149; Percent complete: 3.7%; Average loss: 3.3950\n",
      "Iteration: 150; Percent complete: 3.8%; Average loss: 3.4647\n",
      "Iteration: 151; Percent complete: 3.8%; Average loss: 3.6141\n",
      "Iteration: 152; Percent complete: 3.8%; Average loss: 3.6908\n",
      "Iteration: 153; Percent complete: 3.8%; Average loss: 3.5224\n",
      "Iteration: 154; Percent complete: 3.9%; Average loss: 3.6889\n",
      "Iteration: 155; Percent complete: 3.9%; Average loss: 3.4383\n",
      "Iteration: 156; Percent complete: 3.9%; Average loss: 3.6523\n",
      "Iteration: 157; Percent complete: 3.9%; Average loss: 3.6349\n",
      "Iteration: 158; Percent complete: 4.0%; Average loss: 3.5427\n",
      "Iteration: 159; Percent complete: 4.0%; Average loss: 3.3443\n",
      "Iteration: 160; Percent complete: 4.0%; Average loss: 3.7345\n",
      "Iteration: 161; Percent complete: 4.0%; Average loss: 3.3817\n",
      "Iteration: 162; Percent complete: 4.0%; Average loss: 3.4614\n",
      "Iteration: 163; Percent complete: 4.1%; Average loss: 3.5592\n",
      "Iteration: 164; Percent complete: 4.1%; Average loss: 3.4470\n",
      "Iteration: 165; Percent complete: 4.1%; Average loss: 3.6361\n",
      "Iteration: 166; Percent complete: 4.2%; Average loss: 3.7625\n",
      "Iteration: 167; Percent complete: 4.2%; Average loss: 3.7828\n",
      "Iteration: 168; Percent complete: 4.2%; Average loss: 3.2022\n",
      "Iteration: 169; Percent complete: 4.2%; Average loss: 3.8611\n",
      "Iteration: 170; Percent complete: 4.2%; Average loss: 3.5582\n",
      "Iteration: 171; Percent complete: 4.3%; Average loss: 3.3815\n",
      "Iteration: 172; Percent complete: 4.3%; Average loss: 3.6493\n",
      "Iteration: 173; Percent complete: 4.3%; Average loss: 3.6852\n",
      "Iteration: 174; Percent complete: 4.3%; Average loss: 3.5893\n",
      "Iteration: 175; Percent complete: 4.4%; Average loss: 3.3572\n",
      "Iteration: 176; Percent complete: 4.4%; Average loss: 3.4700\n",
      "Iteration: 177; Percent complete: 4.4%; Average loss: 3.5882\n",
      "Iteration: 178; Percent complete: 4.5%; Average loss: 3.3010\n",
      "Iteration: 179; Percent complete: 4.5%; Average loss: 3.6724\n",
      "Iteration: 180; Percent complete: 4.5%; Average loss: 3.3874\n",
      "Iteration: 181; Percent complete: 4.5%; Average loss: 3.4613\n",
      "Iteration: 182; Percent complete: 4.5%; Average loss: 3.6473\n",
      "Iteration: 183; Percent complete: 4.6%; Average loss: 3.3651\n",
      "Iteration: 184; Percent complete: 4.6%; Average loss: 3.7007\n",
      "Iteration: 185; Percent complete: 4.6%; Average loss: 3.3764\n",
      "Iteration: 186; Percent complete: 4.7%; Average loss: 3.7053\n",
      "Iteration: 187; Percent complete: 4.7%; Average loss: 3.4968\n",
      "Iteration: 188; Percent complete: 4.7%; Average loss: 3.5130\n",
      "Iteration: 189; Percent complete: 4.7%; Average loss: 3.6189\n",
      "Iteration: 190; Percent complete: 4.8%; Average loss: 3.2124\n",
      "Iteration: 191; Percent complete: 4.8%; Average loss: 3.6542\n",
      "Iteration: 192; Percent complete: 4.8%; Average loss: 3.6089\n",
      "Iteration: 193; Percent complete: 4.8%; Average loss: 3.6052\n",
      "Iteration: 194; Percent complete: 4.9%; Average loss: 3.4415\n",
      "Iteration: 195; Percent complete: 4.9%; Average loss: 3.5801\n",
      "Iteration: 196; Percent complete: 4.9%; Average loss: 3.1935\n",
      "Iteration: 197; Percent complete: 4.9%; Average loss: 3.2218\n",
      "Iteration: 198; Percent complete: 5.0%; Average loss: 3.5576\n",
      "Iteration: 199; Percent complete: 5.0%; Average loss: 3.6749\n",
      "Iteration: 200; Percent complete: 5.0%; Average loss: 3.3873\n",
      "Iteration: 201; Percent complete: 5.0%; Average loss: 3.5803\n",
      "Iteration: 202; Percent complete: 5.1%; Average loss: 3.3003\n",
      "Iteration: 203; Percent complete: 5.1%; Average loss: 3.6103\n",
      "Iteration: 204; Percent complete: 5.1%; Average loss: 3.5851\n",
      "Iteration: 205; Percent complete: 5.1%; Average loss: 3.2088\n",
      "Iteration: 206; Percent complete: 5.1%; Average loss: 3.4419\n",
      "Iteration: 207; Percent complete: 5.2%; Average loss: 3.3038\n",
      "Iteration: 208; Percent complete: 5.2%; Average loss: 3.1237\n",
      "Iteration: 209; Percent complete: 5.2%; Average loss: 3.2799\n",
      "Iteration: 210; Percent complete: 5.2%; Average loss: 3.2489\n",
      "Iteration: 211; Percent complete: 5.3%; Average loss: 3.4303\n",
      "Iteration: 212; Percent complete: 5.3%; Average loss: 3.1443\n",
      "Iteration: 213; Percent complete: 5.3%; Average loss: 3.2896\n",
      "Iteration: 214; Percent complete: 5.3%; Average loss: 3.4133\n",
      "Iteration: 215; Percent complete: 5.4%; Average loss: 3.4600\n",
      "Iteration: 216; Percent complete: 5.4%; Average loss: 3.3549\n",
      "Iteration: 217; Percent complete: 5.4%; Average loss: 3.5866\n",
      "Iteration: 218; Percent complete: 5.5%; Average loss: 3.4649\n",
      "Iteration: 219; Percent complete: 5.5%; Average loss: 3.1407\n",
      "Iteration: 220; Percent complete: 5.5%; Average loss: 3.3225\n",
      "Iteration: 221; Percent complete: 5.5%; Average loss: 3.3328\n",
      "Iteration: 222; Percent complete: 5.5%; Average loss: 3.5861\n",
      "Iteration: 223; Percent complete: 5.6%; Average loss: 3.2292\n",
      "Iteration: 224; Percent complete: 5.6%; Average loss: 3.5561\n",
      "Iteration: 225; Percent complete: 5.6%; Average loss: 3.5435\n",
      "Iteration: 226; Percent complete: 5.7%; Average loss: 3.2105\n",
      "Iteration: 227; Percent complete: 5.7%; Average loss: 3.4733\n",
      "Iteration: 228; Percent complete: 5.7%; Average loss: 3.2423\n",
      "Iteration: 229; Percent complete: 5.7%; Average loss: 3.1277\n",
      "Iteration: 230; Percent complete: 5.8%; Average loss: 3.4189\n",
      "Iteration: 231; Percent complete: 5.8%; Average loss: 3.5288\n",
      "Iteration: 232; Percent complete: 5.8%; Average loss: 3.7444\n",
      "Iteration: 233; Percent complete: 5.8%; Average loss: 3.6507\n",
      "Iteration: 234; Percent complete: 5.9%; Average loss: 3.4213\n",
      "Iteration: 235; Percent complete: 5.9%; Average loss: 3.5660\n",
      "Iteration: 236; Percent complete: 5.9%; Average loss: 3.1901\n",
      "Iteration: 237; Percent complete: 5.9%; Average loss: 3.4221\n",
      "Iteration: 238; Percent complete: 5.9%; Average loss: 3.4264\n",
      "Iteration: 239; Percent complete: 6.0%; Average loss: 3.5481\n",
      "Iteration: 240; Percent complete: 6.0%; Average loss: 3.2625\n",
      "Iteration: 241; Percent complete: 6.0%; Average loss: 3.3635\n",
      "Iteration: 242; Percent complete: 6.0%; Average loss: 3.4135\n",
      "Iteration: 243; Percent complete: 6.1%; Average loss: 3.5436\n",
      "Iteration: 244; Percent complete: 6.1%; Average loss: 3.3860\n",
      "Iteration: 245; Percent complete: 6.1%; Average loss: 3.3840\n",
      "Iteration: 246; Percent complete: 6.2%; Average loss: 3.5446\n",
      "Iteration: 247; Percent complete: 6.2%; Average loss: 3.2986\n",
      "Iteration: 248; Percent complete: 6.2%; Average loss: 3.5517\n",
      "Iteration: 249; Percent complete: 6.2%; Average loss: 3.1082\n",
      "Iteration: 250; Percent complete: 6.2%; Average loss: 3.5981\n",
      "Iteration: 251; Percent complete: 6.3%; Average loss: 3.1541\n",
      "Iteration: 252; Percent complete: 6.3%; Average loss: 3.1905\n",
      "Iteration: 253; Percent complete: 6.3%; Average loss: 3.2484\n",
      "Iteration: 254; Percent complete: 6.3%; Average loss: 3.5440\n",
      "Iteration: 255; Percent complete: 6.4%; Average loss: 3.4477\n",
      "Iteration: 256; Percent complete: 6.4%; Average loss: 3.3848\n",
      "Iteration: 257; Percent complete: 6.4%; Average loss: 3.4206\n",
      "Iteration: 258; Percent complete: 6.5%; Average loss: 3.4023\n",
      "Iteration: 259; Percent complete: 6.5%; Average loss: 3.2939\n",
      "Iteration: 260; Percent complete: 6.5%; Average loss: 3.2490\n",
      "Iteration: 261; Percent complete: 6.5%; Average loss: 3.3169\n",
      "Iteration: 262; Percent complete: 6.6%; Average loss: 3.1558\n",
      "Iteration: 263; Percent complete: 6.6%; Average loss: 3.1738\n",
      "Iteration: 264; Percent complete: 6.6%; Average loss: 3.3362\n",
      "Iteration: 265; Percent complete: 6.6%; Average loss: 3.3531\n",
      "Iteration: 266; Percent complete: 6.7%; Average loss: 3.2887\n",
      "Iteration: 267; Percent complete: 6.7%; Average loss: 3.2056\n",
      "Iteration: 268; Percent complete: 6.7%; Average loss: 3.1982\n",
      "Iteration: 269; Percent complete: 6.7%; Average loss: 3.7230\n",
      "Iteration: 270; Percent complete: 6.8%; Average loss: 3.4536\n",
      "Iteration: 271; Percent complete: 6.8%; Average loss: 3.6102\n",
      "Iteration: 272; Percent complete: 6.8%; Average loss: 3.3915\n",
      "Iteration: 273; Percent complete: 6.8%; Average loss: 3.2312\n",
      "Iteration: 274; Percent complete: 6.9%; Average loss: 3.4833\n",
      "Iteration: 275; Percent complete: 6.9%; Average loss: 3.1512\n",
      "Iteration: 276; Percent complete: 6.9%; Average loss: 3.2029\n",
      "Iteration: 277; Percent complete: 6.9%; Average loss: 3.3942\n",
      "Iteration: 278; Percent complete: 7.0%; Average loss: 3.5091\n",
      "Iteration: 279; Percent complete: 7.0%; Average loss: 3.4784\n",
      "Iteration: 280; Percent complete: 7.0%; Average loss: 3.2507\n",
      "Iteration: 281; Percent complete: 7.0%; Average loss: 3.3306\n",
      "Iteration: 282; Percent complete: 7.0%; Average loss: 3.1827\n",
      "Iteration: 283; Percent complete: 7.1%; Average loss: 3.4230\n",
      "Iteration: 284; Percent complete: 7.1%; Average loss: 3.3790\n",
      "Iteration: 285; Percent complete: 7.1%; Average loss: 3.2918\n",
      "Iteration: 286; Percent complete: 7.1%; Average loss: 3.1746\n",
      "Iteration: 287; Percent complete: 7.2%; Average loss: 3.3640\n",
      "Iteration: 288; Percent complete: 7.2%; Average loss: 3.2652\n",
      "Iteration: 289; Percent complete: 7.2%; Average loss: 3.4196\n",
      "Iteration: 290; Percent complete: 7.2%; Average loss: 3.3013\n",
      "Iteration: 291; Percent complete: 7.3%; Average loss: 3.3560\n",
      "Iteration: 292; Percent complete: 7.3%; Average loss: 3.5492\n",
      "Iteration: 293; Percent complete: 7.3%; Average loss: 3.3029\n",
      "Iteration: 294; Percent complete: 7.3%; Average loss: 3.3473\n",
      "Iteration: 295; Percent complete: 7.4%; Average loss: 3.3325\n",
      "Iteration: 296; Percent complete: 7.4%; Average loss: 3.5327\n",
      "Iteration: 297; Percent complete: 7.4%; Average loss: 3.0682\n",
      "Iteration: 298; Percent complete: 7.4%; Average loss: 3.6302\n",
      "Iteration: 299; Percent complete: 7.5%; Average loss: 3.0647\n",
      "Iteration: 300; Percent complete: 7.5%; Average loss: 3.0465\n",
      "Iteration: 301; Percent complete: 7.5%; Average loss: 3.4730\n",
      "Iteration: 302; Percent complete: 7.5%; Average loss: 3.2983\n",
      "Iteration: 303; Percent complete: 7.6%; Average loss: 3.3112\n",
      "Iteration: 304; Percent complete: 7.6%; Average loss: 3.3665\n",
      "Iteration: 305; Percent complete: 7.6%; Average loss: 3.5350\n",
      "Iteration: 306; Percent complete: 7.6%; Average loss: 3.3064\n",
      "Iteration: 307; Percent complete: 7.7%; Average loss: 3.0363\n",
      "Iteration: 308; Percent complete: 7.7%; Average loss: 3.2233\n",
      "Iteration: 309; Percent complete: 7.7%; Average loss: 3.5425\n",
      "Iteration: 310; Percent complete: 7.8%; Average loss: 3.3560\n",
      "Iteration: 311; Percent complete: 7.8%; Average loss: 3.3042\n",
      "Iteration: 312; Percent complete: 7.8%; Average loss: 3.2166\n",
      "Iteration: 313; Percent complete: 7.8%; Average loss: 3.0862\n",
      "Iteration: 314; Percent complete: 7.8%; Average loss: 3.1348\n",
      "Iteration: 315; Percent complete: 7.9%; Average loss: 3.2524\n",
      "Iteration: 316; Percent complete: 7.9%; Average loss: 3.3547\n",
      "Iteration: 317; Percent complete: 7.9%; Average loss: 3.1545\n",
      "Iteration: 318; Percent complete: 8.0%; Average loss: 3.1576\n",
      "Iteration: 319; Percent complete: 8.0%; Average loss: 3.2363\n",
      "Iteration: 320; Percent complete: 8.0%; Average loss: 3.2315\n",
      "Iteration: 321; Percent complete: 8.0%; Average loss: 3.0901\n",
      "Iteration: 322; Percent complete: 8.1%; Average loss: 3.5055\n",
      "Iteration: 323; Percent complete: 8.1%; Average loss: 3.0985\n",
      "Iteration: 324; Percent complete: 8.1%; Average loss: 3.1645\n",
      "Iteration: 325; Percent complete: 8.1%; Average loss: 3.3397\n",
      "Iteration: 326; Percent complete: 8.2%; Average loss: 3.3624\n",
      "Iteration: 327; Percent complete: 8.2%; Average loss: 3.0968\n",
      "Iteration: 328; Percent complete: 8.2%; Average loss: 3.2082\n",
      "Iteration: 329; Percent complete: 8.2%; Average loss: 3.0105\n",
      "Iteration: 330; Percent complete: 8.2%; Average loss: 3.3662\n",
      "Iteration: 331; Percent complete: 8.3%; Average loss: 3.3410\n",
      "Iteration: 332; Percent complete: 8.3%; Average loss: 3.3745\n",
      "Iteration: 333; Percent complete: 8.3%; Average loss: 3.1155\n",
      "Iteration: 334; Percent complete: 8.3%; Average loss: 3.3509\n",
      "Iteration: 335; Percent complete: 8.4%; Average loss: 3.3115\n",
      "Iteration: 336; Percent complete: 8.4%; Average loss: 3.3326\n",
      "Iteration: 337; Percent complete: 8.4%; Average loss: 3.1735\n",
      "Iteration: 338; Percent complete: 8.5%; Average loss: 3.5118\n",
      "Iteration: 339; Percent complete: 8.5%; Average loss: 3.2152\n",
      "Iteration: 340; Percent complete: 8.5%; Average loss: 3.3399\n",
      "Iteration: 341; Percent complete: 8.5%; Average loss: 3.1508\n",
      "Iteration: 342; Percent complete: 8.6%; Average loss: 3.3008\n",
      "Iteration: 343; Percent complete: 8.6%; Average loss: 3.2325\n",
      "Iteration: 344; Percent complete: 8.6%; Average loss: 3.2179\n",
      "Iteration: 345; Percent complete: 8.6%; Average loss: 3.4361\n",
      "Iteration: 346; Percent complete: 8.6%; Average loss: 3.3929\n",
      "Iteration: 347; Percent complete: 8.7%; Average loss: 3.1590\n",
      "Iteration: 348; Percent complete: 8.7%; Average loss: 3.3135\n",
      "Iteration: 349; Percent complete: 8.7%; Average loss: 3.2396\n",
      "Iteration: 350; Percent complete: 8.8%; Average loss: 3.3162\n",
      "Iteration: 351; Percent complete: 8.8%; Average loss: 3.3532\n",
      "Iteration: 352; Percent complete: 8.8%; Average loss: 3.2707\n",
      "Iteration: 353; Percent complete: 8.8%; Average loss: 3.1605\n",
      "Iteration: 354; Percent complete: 8.8%; Average loss: 3.4255\n",
      "Iteration: 355; Percent complete: 8.9%; Average loss: 3.2436\n",
      "Iteration: 356; Percent complete: 8.9%; Average loss: 3.1932\n",
      "Iteration: 357; Percent complete: 8.9%; Average loss: 3.2595\n",
      "Iteration: 358; Percent complete: 8.9%; Average loss: 3.0163\n",
      "Iteration: 359; Percent complete: 9.0%; Average loss: 3.2099\n",
      "Iteration: 360; Percent complete: 9.0%; Average loss: 3.4515\n",
      "Iteration: 361; Percent complete: 9.0%; Average loss: 3.1283\n",
      "Iteration: 362; Percent complete: 9.0%; Average loss: 3.2936\n",
      "Iteration: 363; Percent complete: 9.1%; Average loss: 3.3140\n",
      "Iteration: 364; Percent complete: 9.1%; Average loss: 3.2656\n",
      "Iteration: 365; Percent complete: 9.1%; Average loss: 3.2026\n",
      "Iteration: 366; Percent complete: 9.2%; Average loss: 3.5075\n",
      "Iteration: 367; Percent complete: 9.2%; Average loss: 3.3849\n",
      "Iteration: 368; Percent complete: 9.2%; Average loss: 3.4653\n",
      "Iteration: 369; Percent complete: 9.2%; Average loss: 3.0568\n",
      "Iteration: 370; Percent complete: 9.2%; Average loss: 3.2710\n",
      "Iteration: 371; Percent complete: 9.3%; Average loss: 3.2549\n",
      "Iteration: 372; Percent complete: 9.3%; Average loss: 3.4161\n",
      "Iteration: 373; Percent complete: 9.3%; Average loss: 3.1016\n",
      "Iteration: 374; Percent complete: 9.3%; Average loss: 3.4341\n",
      "Iteration: 375; Percent complete: 9.4%; Average loss: 3.1648\n",
      "Iteration: 376; Percent complete: 9.4%; Average loss: 3.2734\n",
      "Iteration: 377; Percent complete: 9.4%; Average loss: 3.2852\n",
      "Iteration: 378; Percent complete: 9.4%; Average loss: 3.2350\n",
      "Iteration: 379; Percent complete: 9.5%; Average loss: 3.3251\n",
      "Iteration: 380; Percent complete: 9.5%; Average loss: 3.2722\n",
      "Iteration: 381; Percent complete: 9.5%; Average loss: 3.2342\n",
      "Iteration: 382; Percent complete: 9.6%; Average loss: 3.1081\n",
      "Iteration: 383; Percent complete: 9.6%; Average loss: 3.1962\n",
      "Iteration: 384; Percent complete: 9.6%; Average loss: 3.5450\n",
      "Iteration: 385; Percent complete: 9.6%; Average loss: 3.3819\n",
      "Iteration: 386; Percent complete: 9.7%; Average loss: 3.4949\n",
      "Iteration: 387; Percent complete: 9.7%; Average loss: 3.1724\n",
      "Iteration: 388; Percent complete: 9.7%; Average loss: 3.0826\n",
      "Iteration: 389; Percent complete: 9.7%; Average loss: 3.0304\n",
      "Iteration: 390; Percent complete: 9.8%; Average loss: 3.2399\n",
      "Iteration: 391; Percent complete: 9.8%; Average loss: 3.0893\n",
      "Iteration: 392; Percent complete: 9.8%; Average loss: 3.1192\n",
      "Iteration: 393; Percent complete: 9.8%; Average loss: 3.2093\n",
      "Iteration: 394; Percent complete: 9.8%; Average loss: 2.9551\n",
      "Iteration: 395; Percent complete: 9.9%; Average loss: 3.3327\n",
      "Iteration: 396; Percent complete: 9.9%; Average loss: 3.2470\n",
      "Iteration: 397; Percent complete: 9.9%; Average loss: 3.0641\n",
      "Iteration: 398; Percent complete: 10.0%; Average loss: 3.2802\n",
      "Iteration: 399; Percent complete: 10.0%; Average loss: 3.1836\n",
      "Iteration: 400; Percent complete: 10.0%; Average loss: 3.2457\n",
      "Iteration: 401; Percent complete: 10.0%; Average loss: 3.2537\n",
      "Iteration: 402; Percent complete: 10.1%; Average loss: 3.1289\n",
      "Iteration: 403; Percent complete: 10.1%; Average loss: 3.1695\n",
      "Iteration: 404; Percent complete: 10.1%; Average loss: 3.0995\n",
      "Iteration: 405; Percent complete: 10.1%; Average loss: 3.4211\n",
      "Iteration: 406; Percent complete: 10.2%; Average loss: 3.6876\n",
      "Iteration: 407; Percent complete: 10.2%; Average loss: 3.1623\n",
      "Iteration: 408; Percent complete: 10.2%; Average loss: 3.3827\n",
      "Iteration: 409; Percent complete: 10.2%; Average loss: 3.0935\n",
      "Iteration: 410; Percent complete: 10.2%; Average loss: 3.0178\n",
      "Iteration: 411; Percent complete: 10.3%; Average loss: 3.0080\n",
      "Iteration: 412; Percent complete: 10.3%; Average loss: 3.3160\n",
      "Iteration: 413; Percent complete: 10.3%; Average loss: 3.3708\n",
      "Iteration: 414; Percent complete: 10.3%; Average loss: 3.2340\n",
      "Iteration: 415; Percent complete: 10.4%; Average loss: 3.2691\n",
      "Iteration: 416; Percent complete: 10.4%; Average loss: 3.1956\n",
      "Iteration: 417; Percent complete: 10.4%; Average loss: 3.1306\n",
      "Iteration: 418; Percent complete: 10.4%; Average loss: 3.3041\n",
      "Iteration: 419; Percent complete: 10.5%; Average loss: 3.2898\n",
      "Iteration: 420; Percent complete: 10.5%; Average loss: 3.4241\n",
      "Iteration: 421; Percent complete: 10.5%; Average loss: 3.3903\n",
      "Iteration: 422; Percent complete: 10.5%; Average loss: 3.1609\n",
      "Iteration: 423; Percent complete: 10.6%; Average loss: 3.2095\n",
      "Iteration: 424; Percent complete: 10.6%; Average loss: 2.7884\n",
      "Iteration: 425; Percent complete: 10.6%; Average loss: 3.1016\n",
      "Iteration: 426; Percent complete: 10.7%; Average loss: 3.2797\n",
      "Iteration: 427; Percent complete: 10.7%; Average loss: 3.3637\n",
      "Iteration: 428; Percent complete: 10.7%; Average loss: 3.4418\n",
      "Iteration: 429; Percent complete: 10.7%; Average loss: 3.2029\n",
      "Iteration: 430; Percent complete: 10.8%; Average loss: 3.3946\n",
      "Iteration: 431; Percent complete: 10.8%; Average loss: 3.2802\n",
      "Iteration: 432; Percent complete: 10.8%; Average loss: 3.1774\n",
      "Iteration: 433; Percent complete: 10.8%; Average loss: 3.1904\n",
      "Iteration: 434; Percent complete: 10.8%; Average loss: 3.2546\n",
      "Iteration: 435; Percent complete: 10.9%; Average loss: 3.2682\n",
      "Iteration: 436; Percent complete: 10.9%; Average loss: 3.2062\n",
      "Iteration: 437; Percent complete: 10.9%; Average loss: 3.2224\n",
      "Iteration: 438; Percent complete: 10.9%; Average loss: 3.2986\n",
      "Iteration: 439; Percent complete: 11.0%; Average loss: 3.0971\n",
      "Iteration: 440; Percent complete: 11.0%; Average loss: 2.9514\n",
      "Iteration: 441; Percent complete: 11.0%; Average loss: 3.3881\n",
      "Iteration: 442; Percent complete: 11.1%; Average loss: 3.1214\n",
      "Iteration: 443; Percent complete: 11.1%; Average loss: 3.0914\n",
      "Iteration: 444; Percent complete: 11.1%; Average loss: 2.9893\n",
      "Iteration: 445; Percent complete: 11.1%; Average loss: 3.1282\n",
      "Iteration: 446; Percent complete: 11.2%; Average loss: 3.1938\n",
      "Iteration: 447; Percent complete: 11.2%; Average loss: 3.2094\n",
      "Iteration: 448; Percent complete: 11.2%; Average loss: 3.4076\n",
      "Iteration: 449; Percent complete: 11.2%; Average loss: 3.2261\n",
      "Iteration: 450; Percent complete: 11.2%; Average loss: 3.2129\n",
      "Iteration: 451; Percent complete: 11.3%; Average loss: 3.2893\n",
      "Iteration: 452; Percent complete: 11.3%; Average loss: 2.8935\n",
      "Iteration: 453; Percent complete: 11.3%; Average loss: 3.5424\n",
      "Iteration: 454; Percent complete: 11.3%; Average loss: 3.1974\n",
      "Iteration: 455; Percent complete: 11.4%; Average loss: 3.0650\n",
      "Iteration: 456; Percent complete: 11.4%; Average loss: 3.3341\n",
      "Iteration: 457; Percent complete: 11.4%; Average loss: 3.3006\n",
      "Iteration: 458; Percent complete: 11.5%; Average loss: 3.3688\n",
      "Iteration: 459; Percent complete: 11.5%; Average loss: 3.2881\n",
      "Iteration: 460; Percent complete: 11.5%; Average loss: 3.3938\n",
      "Iteration: 461; Percent complete: 11.5%; Average loss: 3.2558\n",
      "Iteration: 462; Percent complete: 11.6%; Average loss: 3.1130\n",
      "Iteration: 463; Percent complete: 11.6%; Average loss: 3.3396\n",
      "Iteration: 464; Percent complete: 11.6%; Average loss: 3.4247\n",
      "Iteration: 465; Percent complete: 11.6%; Average loss: 3.1633\n",
      "Iteration: 466; Percent complete: 11.7%; Average loss: 3.4522\n",
      "Iteration: 467; Percent complete: 11.7%; Average loss: 3.2203\n",
      "Iteration: 468; Percent complete: 11.7%; Average loss: 3.2828\n",
      "Iteration: 469; Percent complete: 11.7%; Average loss: 3.1310\n",
      "Iteration: 470; Percent complete: 11.8%; Average loss: 3.1427\n",
      "Iteration: 471; Percent complete: 11.8%; Average loss: 3.2768\n",
      "Iteration: 472; Percent complete: 11.8%; Average loss: 3.4016\n",
      "Iteration: 473; Percent complete: 11.8%; Average loss: 3.1674\n",
      "Iteration: 474; Percent complete: 11.8%; Average loss: 3.2705\n",
      "Iteration: 475; Percent complete: 11.9%; Average loss: 2.9832\n",
      "Iteration: 476; Percent complete: 11.9%; Average loss: 2.9989\n",
      "Iteration: 477; Percent complete: 11.9%; Average loss: 2.9708\n",
      "Iteration: 478; Percent complete: 11.9%; Average loss: 3.1543\n",
      "Iteration: 479; Percent complete: 12.0%; Average loss: 2.9827\n",
      "Iteration: 480; Percent complete: 12.0%; Average loss: 3.1485\n",
      "Iteration: 481; Percent complete: 12.0%; Average loss: 2.7867\n",
      "Iteration: 482; Percent complete: 12.0%; Average loss: 3.2972\n",
      "Iteration: 483; Percent complete: 12.1%; Average loss: 3.1186\n",
      "Iteration: 484; Percent complete: 12.1%; Average loss: 2.9968\n",
      "Iteration: 485; Percent complete: 12.1%; Average loss: 3.1788\n",
      "Iteration: 486; Percent complete: 12.2%; Average loss: 2.9117\n",
      "Iteration: 487; Percent complete: 12.2%; Average loss: 2.9760\n",
      "Iteration: 488; Percent complete: 12.2%; Average loss: 3.0526\n",
      "Iteration: 489; Percent complete: 12.2%; Average loss: 3.2506\n",
      "Iteration: 490; Percent complete: 12.2%; Average loss: 3.0620\n",
      "Iteration: 491; Percent complete: 12.3%; Average loss: 3.4255\n",
      "Iteration: 492; Percent complete: 12.3%; Average loss: 3.1772\n",
      "Iteration: 493; Percent complete: 12.3%; Average loss: 2.9447\n",
      "Iteration: 494; Percent complete: 12.3%; Average loss: 2.9806\n",
      "Iteration: 495; Percent complete: 12.4%; Average loss: 3.3714\n",
      "Iteration: 496; Percent complete: 12.4%; Average loss: 3.1723\n",
      "Iteration: 497; Percent complete: 12.4%; Average loss: 3.2207\n",
      "Iteration: 498; Percent complete: 12.4%; Average loss: 3.0965\n",
      "Iteration: 499; Percent complete: 12.5%; Average loss: 3.2656\n",
      "Iteration: 500; Percent complete: 12.5%; Average loss: 2.7974\n",
      "Iteration: 501; Percent complete: 12.5%; Average loss: 3.4990\n",
      "Iteration: 502; Percent complete: 12.6%; Average loss: 3.0923\n",
      "Iteration: 503; Percent complete: 12.6%; Average loss: 3.1132\n",
      "Iteration: 504; Percent complete: 12.6%; Average loss: 3.1786\n",
      "Iteration: 505; Percent complete: 12.6%; Average loss: 2.9991\n",
      "Iteration: 506; Percent complete: 12.7%; Average loss: 3.0747\n",
      "Iteration: 507; Percent complete: 12.7%; Average loss: 3.3202\n",
      "Iteration: 508; Percent complete: 12.7%; Average loss: 3.1788\n",
      "Iteration: 509; Percent complete: 12.7%; Average loss: 3.2150\n",
      "Iteration: 510; Percent complete: 12.8%; Average loss: 3.1097\n",
      "Iteration: 511; Percent complete: 12.8%; Average loss: 3.2572\n",
      "Iteration: 512; Percent complete: 12.8%; Average loss: 3.0062\n",
      "Iteration: 513; Percent complete: 12.8%; Average loss: 3.1839\n",
      "Iteration: 514; Percent complete: 12.8%; Average loss: 3.2215\n",
      "Iteration: 515; Percent complete: 12.9%; Average loss: 3.2419\n",
      "Iteration: 516; Percent complete: 12.9%; Average loss: 2.9315\n",
      "Iteration: 517; Percent complete: 12.9%; Average loss: 3.4829\n",
      "Iteration: 518; Percent complete: 13.0%; Average loss: 3.3414\n",
      "Iteration: 519; Percent complete: 13.0%; Average loss: 3.4107\n",
      "Iteration: 520; Percent complete: 13.0%; Average loss: 3.3945\n",
      "Iteration: 521; Percent complete: 13.0%; Average loss: 3.2246\n",
      "Iteration: 522; Percent complete: 13.1%; Average loss: 3.1434\n",
      "Iteration: 523; Percent complete: 13.1%; Average loss: 3.1785\n",
      "Iteration: 524; Percent complete: 13.1%; Average loss: 3.0017\n",
      "Iteration: 525; Percent complete: 13.1%; Average loss: 3.0582\n",
      "Iteration: 526; Percent complete: 13.2%; Average loss: 2.7874\n",
      "Iteration: 527; Percent complete: 13.2%; Average loss: 3.0189\n",
      "Iteration: 528; Percent complete: 13.2%; Average loss: 3.0980\n",
      "Iteration: 529; Percent complete: 13.2%; Average loss: 3.0220\n",
      "Iteration: 530; Percent complete: 13.2%; Average loss: 2.9895\n",
      "Iteration: 531; Percent complete: 13.3%; Average loss: 3.3135\n",
      "Iteration: 532; Percent complete: 13.3%; Average loss: 3.0850\n",
      "Iteration: 533; Percent complete: 13.3%; Average loss: 3.0219\n",
      "Iteration: 534; Percent complete: 13.4%; Average loss: 3.2006\n",
      "Iteration: 535; Percent complete: 13.4%; Average loss: 3.2704\n",
      "Iteration: 536; Percent complete: 13.4%; Average loss: 3.0734\n",
      "Iteration: 537; Percent complete: 13.4%; Average loss: 2.9518\n",
      "Iteration: 538; Percent complete: 13.5%; Average loss: 3.2690\n",
      "Iteration: 539; Percent complete: 13.5%; Average loss: 3.1794\n",
      "Iteration: 540; Percent complete: 13.5%; Average loss: 3.2512\n",
      "Iteration: 541; Percent complete: 13.5%; Average loss: 3.2253\n",
      "Iteration: 542; Percent complete: 13.6%; Average loss: 3.2444\n",
      "Iteration: 543; Percent complete: 13.6%; Average loss: 3.1390\n",
      "Iteration: 544; Percent complete: 13.6%; Average loss: 3.1138\n",
      "Iteration: 545; Percent complete: 13.6%; Average loss: 2.9379\n",
      "Iteration: 546; Percent complete: 13.7%; Average loss: 3.2072\n",
      "Iteration: 547; Percent complete: 13.7%; Average loss: 3.1194\n",
      "Iteration: 548; Percent complete: 13.7%; Average loss: 3.2515\n",
      "Iteration: 549; Percent complete: 13.7%; Average loss: 3.1657\n",
      "Iteration: 550; Percent complete: 13.8%; Average loss: 3.2650\n",
      "Iteration: 551; Percent complete: 13.8%; Average loss: 3.0110\n",
      "Iteration: 552; Percent complete: 13.8%; Average loss: 3.2602\n",
      "Iteration: 553; Percent complete: 13.8%; Average loss: 2.9675\n",
      "Iteration: 554; Percent complete: 13.9%; Average loss: 3.1462\n",
      "Iteration: 555; Percent complete: 13.9%; Average loss: 2.9654\n",
      "Iteration: 556; Percent complete: 13.9%; Average loss: 3.2344\n",
      "Iteration: 557; Percent complete: 13.9%; Average loss: 3.2173\n",
      "Iteration: 558; Percent complete: 14.0%; Average loss: 3.0512\n",
      "Iteration: 559; Percent complete: 14.0%; Average loss: 3.3529\n",
      "Iteration: 560; Percent complete: 14.0%; Average loss: 3.2151\n",
      "Iteration: 561; Percent complete: 14.0%; Average loss: 3.0920\n",
      "Iteration: 562; Percent complete: 14.1%; Average loss: 3.0781\n",
      "Iteration: 563; Percent complete: 14.1%; Average loss: 2.8284\n",
      "Iteration: 564; Percent complete: 14.1%; Average loss: 2.8422\n",
      "Iteration: 565; Percent complete: 14.1%; Average loss: 3.2392\n",
      "Iteration: 566; Percent complete: 14.1%; Average loss: 3.2128\n",
      "Iteration: 567; Percent complete: 14.2%; Average loss: 3.2646\n",
      "Iteration: 568; Percent complete: 14.2%; Average loss: 3.2793\n",
      "Iteration: 569; Percent complete: 14.2%; Average loss: 3.0704\n",
      "Iteration: 570; Percent complete: 14.2%; Average loss: 3.0542\n",
      "Iteration: 571; Percent complete: 14.3%; Average loss: 2.9959\n",
      "Iteration: 572; Percent complete: 14.3%; Average loss: 3.2072\n",
      "Iteration: 573; Percent complete: 14.3%; Average loss: 3.0208\n",
      "Iteration: 574; Percent complete: 14.3%; Average loss: 2.9343\n",
      "Iteration: 575; Percent complete: 14.4%; Average loss: 3.0682\n",
      "Iteration: 576; Percent complete: 14.4%; Average loss: 3.1052\n",
      "Iteration: 577; Percent complete: 14.4%; Average loss: 3.1874\n",
      "Iteration: 578; Percent complete: 14.4%; Average loss: 2.9824\n",
      "Iteration: 579; Percent complete: 14.5%; Average loss: 3.0533\n",
      "Iteration: 580; Percent complete: 14.5%; Average loss: 3.2280\n",
      "Iteration: 581; Percent complete: 14.5%; Average loss: 3.1783\n",
      "Iteration: 582; Percent complete: 14.5%; Average loss: 3.0310\n",
      "Iteration: 583; Percent complete: 14.6%; Average loss: 2.8880\n",
      "Iteration: 584; Percent complete: 14.6%; Average loss: 3.3338\n",
      "Iteration: 585; Percent complete: 14.6%; Average loss: 3.1679\n",
      "Iteration: 586; Percent complete: 14.6%; Average loss: 3.0263\n",
      "Iteration: 587; Percent complete: 14.7%; Average loss: 3.2992\n",
      "Iteration: 588; Percent complete: 14.7%; Average loss: 3.2900\n",
      "Iteration: 589; Percent complete: 14.7%; Average loss: 3.2217\n",
      "Iteration: 590; Percent complete: 14.8%; Average loss: 3.0120\n",
      "Iteration: 591; Percent complete: 14.8%; Average loss: 3.0074\n",
      "Iteration: 592; Percent complete: 14.8%; Average loss: 2.8808\n",
      "Iteration: 593; Percent complete: 14.8%; Average loss: 3.0494\n",
      "Iteration: 594; Percent complete: 14.8%; Average loss: 3.0188\n",
      "Iteration: 595; Percent complete: 14.9%; Average loss: 2.9865\n",
      "Iteration: 596; Percent complete: 14.9%; Average loss: 3.2952\n",
      "Iteration: 597; Percent complete: 14.9%; Average loss: 3.4134\n",
      "Iteration: 598; Percent complete: 14.9%; Average loss: 3.0388\n",
      "Iteration: 599; Percent complete: 15.0%; Average loss: 3.1901\n",
      "Iteration: 600; Percent complete: 15.0%; Average loss: 3.0858\n",
      "Iteration: 601; Percent complete: 15.0%; Average loss: 3.1881\n",
      "Iteration: 602; Percent complete: 15.0%; Average loss: 3.6404\n",
      "Iteration: 603; Percent complete: 15.1%; Average loss: 3.1560\n",
      "Iteration: 604; Percent complete: 15.1%; Average loss: 3.1265\n",
      "Iteration: 605; Percent complete: 15.1%; Average loss: 3.0317\n",
      "Iteration: 606; Percent complete: 15.2%; Average loss: 3.3072\n",
      "Iteration: 607; Percent complete: 15.2%; Average loss: 2.9371\n",
      "Iteration: 608; Percent complete: 15.2%; Average loss: 2.9034\n",
      "Iteration: 609; Percent complete: 15.2%; Average loss: 3.0788\n",
      "Iteration: 610; Percent complete: 15.2%; Average loss: 3.1334\n",
      "Iteration: 611; Percent complete: 15.3%; Average loss: 2.9042\n",
      "Iteration: 612; Percent complete: 15.3%; Average loss: 3.0679\n",
      "Iteration: 613; Percent complete: 15.3%; Average loss: 3.1229\n",
      "Iteration: 614; Percent complete: 15.3%; Average loss: 2.8307\n",
      "Iteration: 615; Percent complete: 15.4%; Average loss: 3.1407\n",
      "Iteration: 616; Percent complete: 15.4%; Average loss: 3.4993\n",
      "Iteration: 617; Percent complete: 15.4%; Average loss: 3.3111\n",
      "Iteration: 618; Percent complete: 15.4%; Average loss: 3.0124\n",
      "Iteration: 619; Percent complete: 15.5%; Average loss: 2.8797\n",
      "Iteration: 620; Percent complete: 15.5%; Average loss: 3.0778\n",
      "Iteration: 621; Percent complete: 15.5%; Average loss: 2.9506\n",
      "Iteration: 622; Percent complete: 15.6%; Average loss: 3.0579\n",
      "Iteration: 623; Percent complete: 15.6%; Average loss: 3.0057\n",
      "Iteration: 624; Percent complete: 15.6%; Average loss: 2.9219\n",
      "Iteration: 625; Percent complete: 15.6%; Average loss: 3.1024\n",
      "Iteration: 626; Percent complete: 15.7%; Average loss: 3.1604\n",
      "Iteration: 627; Percent complete: 15.7%; Average loss: 2.8739\n",
      "Iteration: 628; Percent complete: 15.7%; Average loss: 3.3123\n",
      "Iteration: 629; Percent complete: 15.7%; Average loss: 3.0987\n",
      "Iteration: 630; Percent complete: 15.8%; Average loss: 3.0134\n",
      "Iteration: 631; Percent complete: 15.8%; Average loss: 3.0340\n",
      "Iteration: 632; Percent complete: 15.8%; Average loss: 3.1088\n",
      "Iteration: 633; Percent complete: 15.8%; Average loss: 3.1657\n",
      "Iteration: 634; Percent complete: 15.8%; Average loss: 3.3290\n",
      "Iteration: 635; Percent complete: 15.9%; Average loss: 3.0607\n",
      "Iteration: 636; Percent complete: 15.9%; Average loss: 3.0579\n",
      "Iteration: 637; Percent complete: 15.9%; Average loss: 3.1806\n",
      "Iteration: 638; Percent complete: 16.0%; Average loss: 3.2813\n",
      "Iteration: 639; Percent complete: 16.0%; Average loss: 3.0832\n",
      "Iteration: 640; Percent complete: 16.0%; Average loss: 3.0671\n",
      "Iteration: 641; Percent complete: 16.0%; Average loss: 3.1980\n",
      "Iteration: 642; Percent complete: 16.1%; Average loss: 3.1958\n",
      "Iteration: 643; Percent complete: 16.1%; Average loss: 2.9838\n",
      "Iteration: 644; Percent complete: 16.1%; Average loss: 3.0359\n",
      "Iteration: 645; Percent complete: 16.1%; Average loss: 3.1271\n",
      "Iteration: 646; Percent complete: 16.2%; Average loss: 3.1854\n",
      "Iteration: 647; Percent complete: 16.2%; Average loss: 2.8753\n",
      "Iteration: 648; Percent complete: 16.2%; Average loss: 3.1072\n",
      "Iteration: 649; Percent complete: 16.2%; Average loss: 3.1372\n",
      "Iteration: 650; Percent complete: 16.2%; Average loss: 2.8840\n",
      "Iteration: 651; Percent complete: 16.3%; Average loss: 3.1479\n",
      "Iteration: 652; Percent complete: 16.3%; Average loss: 2.8413\n",
      "Iteration: 653; Percent complete: 16.3%; Average loss: 3.2302\n",
      "Iteration: 654; Percent complete: 16.4%; Average loss: 3.0279\n",
      "Iteration: 655; Percent complete: 16.4%; Average loss: 3.3082\n",
      "Iteration: 656; Percent complete: 16.4%; Average loss: 2.9075\n",
      "Iteration: 657; Percent complete: 16.4%; Average loss: 3.3034\n",
      "Iteration: 658; Percent complete: 16.4%; Average loss: 3.1948\n",
      "Iteration: 659; Percent complete: 16.5%; Average loss: 2.9985\n",
      "Iteration: 660; Percent complete: 16.5%; Average loss: 3.0070\n",
      "Iteration: 661; Percent complete: 16.5%; Average loss: 3.2628\n",
      "Iteration: 662; Percent complete: 16.6%; Average loss: 2.8413\n",
      "Iteration: 663; Percent complete: 16.6%; Average loss: 2.8542\n",
      "Iteration: 664; Percent complete: 16.6%; Average loss: 3.0393\n",
      "Iteration: 665; Percent complete: 16.6%; Average loss: 2.9318\n",
      "Iteration: 666; Percent complete: 16.7%; Average loss: 2.9779\n",
      "Iteration: 667; Percent complete: 16.7%; Average loss: 3.0898\n",
      "Iteration: 668; Percent complete: 16.7%; Average loss: 2.7803\n",
      "Iteration: 669; Percent complete: 16.7%; Average loss: 2.9446\n",
      "Iteration: 670; Percent complete: 16.8%; Average loss: 2.9803\n",
      "Iteration: 671; Percent complete: 16.8%; Average loss: 3.3768\n",
      "Iteration: 672; Percent complete: 16.8%; Average loss: 2.9106\n",
      "Iteration: 673; Percent complete: 16.8%; Average loss: 3.3143\n",
      "Iteration: 674; Percent complete: 16.9%; Average loss: 3.0460\n",
      "Iteration: 675; Percent complete: 16.9%; Average loss: 3.1404\n",
      "Iteration: 676; Percent complete: 16.9%; Average loss: 3.1269\n",
      "Iteration: 677; Percent complete: 16.9%; Average loss: 3.0640\n",
      "Iteration: 678; Percent complete: 17.0%; Average loss: 2.8947\n",
      "Iteration: 679; Percent complete: 17.0%; Average loss: 2.9316\n",
      "Iteration: 680; Percent complete: 17.0%; Average loss: 3.2523\n",
      "Iteration: 681; Percent complete: 17.0%; Average loss: 2.7874\n",
      "Iteration: 682; Percent complete: 17.1%; Average loss: 2.9790\n",
      "Iteration: 683; Percent complete: 17.1%; Average loss: 2.9581\n",
      "Iteration: 684; Percent complete: 17.1%; Average loss: 3.2322\n",
      "Iteration: 685; Percent complete: 17.1%; Average loss: 2.9397\n",
      "Iteration: 686; Percent complete: 17.2%; Average loss: 3.3710\n",
      "Iteration: 687; Percent complete: 17.2%; Average loss: 2.9951\n",
      "Iteration: 688; Percent complete: 17.2%; Average loss: 2.9407\n",
      "Iteration: 689; Percent complete: 17.2%; Average loss: 3.1158\n",
      "Iteration: 690; Percent complete: 17.2%; Average loss: 3.1311\n",
      "Iteration: 691; Percent complete: 17.3%; Average loss: 2.6937\n",
      "Iteration: 692; Percent complete: 17.3%; Average loss: 3.1395\n",
      "Iteration: 693; Percent complete: 17.3%; Average loss: 3.1592\n",
      "Iteration: 694; Percent complete: 17.3%; Average loss: 3.0445\n",
      "Iteration: 695; Percent complete: 17.4%; Average loss: 2.8639\n",
      "Iteration: 696; Percent complete: 17.4%; Average loss: 3.2796\n",
      "Iteration: 697; Percent complete: 17.4%; Average loss: 3.0653\n",
      "Iteration: 698; Percent complete: 17.4%; Average loss: 3.1134\n",
      "Iteration: 699; Percent complete: 17.5%; Average loss: 3.2312\n",
      "Iteration: 700; Percent complete: 17.5%; Average loss: 3.0466\n",
      "Iteration: 701; Percent complete: 17.5%; Average loss: 3.1504\n",
      "Iteration: 702; Percent complete: 17.5%; Average loss: 3.3341\n",
      "Iteration: 703; Percent complete: 17.6%; Average loss: 3.1410\n",
      "Iteration: 704; Percent complete: 17.6%; Average loss: 2.9031\n",
      "Iteration: 705; Percent complete: 17.6%; Average loss: 3.0089\n",
      "Iteration: 706; Percent complete: 17.6%; Average loss: 2.7713\n",
      "Iteration: 707; Percent complete: 17.7%; Average loss: 3.1623\n",
      "Iteration: 708; Percent complete: 17.7%; Average loss: 2.7338\n",
      "Iteration: 709; Percent complete: 17.7%; Average loss: 2.9461\n",
      "Iteration: 710; Percent complete: 17.8%; Average loss: 3.0514\n",
      "Iteration: 711; Percent complete: 17.8%; Average loss: 3.1233\n",
      "Iteration: 712; Percent complete: 17.8%; Average loss: 3.2192\n",
      "Iteration: 713; Percent complete: 17.8%; Average loss: 3.2439\n",
      "Iteration: 714; Percent complete: 17.8%; Average loss: 2.8060\n",
      "Iteration: 715; Percent complete: 17.9%; Average loss: 3.1966\n",
      "Iteration: 716; Percent complete: 17.9%; Average loss: 3.1028\n",
      "Iteration: 717; Percent complete: 17.9%; Average loss: 2.9455\n",
      "Iteration: 718; Percent complete: 17.9%; Average loss: 2.7481\n",
      "Iteration: 719; Percent complete: 18.0%; Average loss: 2.9831\n",
      "Iteration: 720; Percent complete: 18.0%; Average loss: 3.0594\n",
      "Iteration: 721; Percent complete: 18.0%; Average loss: 2.9388\n",
      "Iteration: 722; Percent complete: 18.1%; Average loss: 3.2798\n",
      "Iteration: 723; Percent complete: 18.1%; Average loss: 3.0975\n",
      "Iteration: 724; Percent complete: 18.1%; Average loss: 3.2970\n",
      "Iteration: 725; Percent complete: 18.1%; Average loss: 3.1541\n",
      "Iteration: 726; Percent complete: 18.1%; Average loss: 2.7275\n",
      "Iteration: 727; Percent complete: 18.2%; Average loss: 2.9274\n",
      "Iteration: 728; Percent complete: 18.2%; Average loss: 3.4215\n",
      "Iteration: 729; Percent complete: 18.2%; Average loss: 3.1973\n",
      "Iteration: 730; Percent complete: 18.2%; Average loss: 3.1164\n",
      "Iteration: 731; Percent complete: 18.3%; Average loss: 2.9339\n",
      "Iteration: 732; Percent complete: 18.3%; Average loss: 3.1763\n",
      "Iteration: 733; Percent complete: 18.3%; Average loss: 3.0992\n",
      "Iteration: 734; Percent complete: 18.4%; Average loss: 3.1526\n",
      "Iteration: 735; Percent complete: 18.4%; Average loss: 2.9562\n",
      "Iteration: 736; Percent complete: 18.4%; Average loss: 2.9305\n",
      "Iteration: 737; Percent complete: 18.4%; Average loss: 2.9514\n",
      "Iteration: 738; Percent complete: 18.4%; Average loss: 3.0314\n",
      "Iteration: 739; Percent complete: 18.5%; Average loss: 3.2208\n",
      "Iteration: 740; Percent complete: 18.5%; Average loss: 3.0455\n",
      "Iteration: 741; Percent complete: 18.5%; Average loss: 3.0356\n",
      "Iteration: 742; Percent complete: 18.6%; Average loss: 3.1418\n",
      "Iteration: 743; Percent complete: 18.6%; Average loss: 2.9972\n",
      "Iteration: 744; Percent complete: 18.6%; Average loss: 3.0685\n",
      "Iteration: 745; Percent complete: 18.6%; Average loss: 2.8355\n",
      "Iteration: 746; Percent complete: 18.6%; Average loss: 2.9545\n",
      "Iteration: 747; Percent complete: 18.7%; Average loss: 2.8574\n",
      "Iteration: 748; Percent complete: 18.7%; Average loss: 2.9539\n",
      "Iteration: 749; Percent complete: 18.7%; Average loss: 3.2584\n",
      "Iteration: 750; Percent complete: 18.8%; Average loss: 3.0465\n",
      "Iteration: 751; Percent complete: 18.8%; Average loss: 2.8407\n",
      "Iteration: 752; Percent complete: 18.8%; Average loss: 3.1058\n",
      "Iteration: 753; Percent complete: 18.8%; Average loss: 3.0049\n",
      "Iteration: 754; Percent complete: 18.9%; Average loss: 2.8800\n",
      "Iteration: 755; Percent complete: 18.9%; Average loss: 2.9875\n",
      "Iteration: 756; Percent complete: 18.9%; Average loss: 3.0297\n",
      "Iteration: 757; Percent complete: 18.9%; Average loss: 3.0889\n",
      "Iteration: 758; Percent complete: 18.9%; Average loss: 3.1701\n",
      "Iteration: 759; Percent complete: 19.0%; Average loss: 2.7474\n",
      "Iteration: 760; Percent complete: 19.0%; Average loss: 3.0870\n",
      "Iteration: 761; Percent complete: 19.0%; Average loss: 3.1823\n",
      "Iteration: 762; Percent complete: 19.1%; Average loss: 3.0722\n",
      "Iteration: 763; Percent complete: 19.1%; Average loss: 3.1494\n",
      "Iteration: 764; Percent complete: 19.1%; Average loss: 2.7877\n",
      "Iteration: 765; Percent complete: 19.1%; Average loss: 2.9264\n",
      "Iteration: 766; Percent complete: 19.1%; Average loss: 3.1116\n",
      "Iteration: 767; Percent complete: 19.2%; Average loss: 3.0715\n",
      "Iteration: 768; Percent complete: 19.2%; Average loss: 3.1403\n",
      "Iteration: 769; Percent complete: 19.2%; Average loss: 3.1920\n",
      "Iteration: 770; Percent complete: 19.2%; Average loss: 3.2096\n",
      "Iteration: 771; Percent complete: 19.3%; Average loss: 3.0291\n",
      "Iteration: 772; Percent complete: 19.3%; Average loss: 3.0188\n",
      "Iteration: 773; Percent complete: 19.3%; Average loss: 2.9295\n",
      "Iteration: 774; Percent complete: 19.4%; Average loss: 3.1023\n",
      "Iteration: 775; Percent complete: 19.4%; Average loss: 2.9570\n",
      "Iteration: 776; Percent complete: 19.4%; Average loss: 3.0224\n",
      "Iteration: 777; Percent complete: 19.4%; Average loss: 3.1509\n",
      "Iteration: 778; Percent complete: 19.4%; Average loss: 2.9145\n",
      "Iteration: 779; Percent complete: 19.5%; Average loss: 3.1806\n",
      "Iteration: 780; Percent complete: 19.5%; Average loss: 2.8913\n",
      "Iteration: 781; Percent complete: 19.5%; Average loss: 3.2061\n",
      "Iteration: 782; Percent complete: 19.6%; Average loss: 2.9375\n",
      "Iteration: 783; Percent complete: 19.6%; Average loss: 2.9955\n",
      "Iteration: 784; Percent complete: 19.6%; Average loss: 3.0866\n",
      "Iteration: 785; Percent complete: 19.6%; Average loss: 2.7333\n",
      "Iteration: 786; Percent complete: 19.7%; Average loss: 3.1419\n",
      "Iteration: 787; Percent complete: 19.7%; Average loss: 3.0532\n",
      "Iteration: 788; Percent complete: 19.7%; Average loss: 2.9279\n",
      "Iteration: 789; Percent complete: 19.7%; Average loss: 3.0716\n",
      "Iteration: 790; Percent complete: 19.8%; Average loss: 2.9232\n",
      "Iteration: 791; Percent complete: 19.8%; Average loss: 2.8980\n",
      "Iteration: 792; Percent complete: 19.8%; Average loss: 3.0261\n",
      "Iteration: 793; Percent complete: 19.8%; Average loss: 2.8664\n",
      "Iteration: 794; Percent complete: 19.9%; Average loss: 3.2028\n",
      "Iteration: 795; Percent complete: 19.9%; Average loss: 2.9688\n",
      "Iteration: 796; Percent complete: 19.9%; Average loss: 3.0253\n",
      "Iteration: 797; Percent complete: 19.9%; Average loss: 3.2380\n",
      "Iteration: 798; Percent complete: 20.0%; Average loss: 3.0888\n",
      "Iteration: 799; Percent complete: 20.0%; Average loss: 3.3186\n",
      "Iteration: 800; Percent complete: 20.0%; Average loss: 3.1421\n",
      "Iteration: 801; Percent complete: 20.0%; Average loss: 3.0572\n",
      "Iteration: 802; Percent complete: 20.1%; Average loss: 3.0667\n",
      "Iteration: 803; Percent complete: 20.1%; Average loss: 2.9723\n",
      "Iteration: 804; Percent complete: 20.1%; Average loss: 3.1860\n",
      "Iteration: 805; Percent complete: 20.1%; Average loss: 3.0678\n",
      "Iteration: 806; Percent complete: 20.2%; Average loss: 2.8165\n",
      "Iteration: 807; Percent complete: 20.2%; Average loss: 3.1036\n",
      "Iteration: 808; Percent complete: 20.2%; Average loss: 2.9290\n",
      "Iteration: 809; Percent complete: 20.2%; Average loss: 2.9022\n",
      "Iteration: 810; Percent complete: 20.2%; Average loss: 3.3030\n",
      "Iteration: 811; Percent complete: 20.3%; Average loss: 3.1434\n",
      "Iteration: 812; Percent complete: 20.3%; Average loss: 3.0366\n",
      "Iteration: 813; Percent complete: 20.3%; Average loss: 2.9445\n",
      "Iteration: 814; Percent complete: 20.3%; Average loss: 3.0583\n",
      "Iteration: 815; Percent complete: 20.4%; Average loss: 2.8790\n",
      "Iteration: 816; Percent complete: 20.4%; Average loss: 2.9942\n",
      "Iteration: 817; Percent complete: 20.4%; Average loss: 3.0132\n",
      "Iteration: 818; Percent complete: 20.4%; Average loss: 2.8307\n",
      "Iteration: 819; Percent complete: 20.5%; Average loss: 2.9802\n",
      "Iteration: 820; Percent complete: 20.5%; Average loss: 3.0437\n",
      "Iteration: 821; Percent complete: 20.5%; Average loss: 2.8166\n",
      "Iteration: 822; Percent complete: 20.5%; Average loss: 2.7308\n",
      "Iteration: 823; Percent complete: 20.6%; Average loss: 2.8540\n",
      "Iteration: 824; Percent complete: 20.6%; Average loss: 3.0154\n",
      "Iteration: 825; Percent complete: 20.6%; Average loss: 2.7954\n",
      "Iteration: 826; Percent complete: 20.6%; Average loss: 2.8755\n",
      "Iteration: 827; Percent complete: 20.7%; Average loss: 3.0098\n",
      "Iteration: 828; Percent complete: 20.7%; Average loss: 3.0661\n",
      "Iteration: 829; Percent complete: 20.7%; Average loss: 3.2710\n",
      "Iteration: 830; Percent complete: 20.8%; Average loss: 3.1912\n",
      "Iteration: 831; Percent complete: 20.8%; Average loss: 2.9627\n",
      "Iteration: 832; Percent complete: 20.8%; Average loss: 2.9042\n",
      "Iteration: 833; Percent complete: 20.8%; Average loss: 2.9710\n",
      "Iteration: 834; Percent complete: 20.8%; Average loss: 3.1101\n",
      "Iteration: 835; Percent complete: 20.9%; Average loss: 2.9744\n",
      "Iteration: 836; Percent complete: 20.9%; Average loss: 2.8761\n",
      "Iteration: 837; Percent complete: 20.9%; Average loss: 3.2154\n",
      "Iteration: 838; Percent complete: 20.9%; Average loss: 3.0470\n",
      "Iteration: 839; Percent complete: 21.0%; Average loss: 3.0642\n",
      "Iteration: 840; Percent complete: 21.0%; Average loss: 3.0115\n",
      "Iteration: 841; Percent complete: 21.0%; Average loss: 2.9824\n",
      "Iteration: 842; Percent complete: 21.1%; Average loss: 2.9717\n",
      "Iteration: 843; Percent complete: 21.1%; Average loss: 3.1677\n",
      "Iteration: 844; Percent complete: 21.1%; Average loss: 3.0671\n",
      "Iteration: 845; Percent complete: 21.1%; Average loss: 2.9963\n",
      "Iteration: 846; Percent complete: 21.1%; Average loss: 3.2012\n",
      "Iteration: 847; Percent complete: 21.2%; Average loss: 2.9896\n",
      "Iteration: 848; Percent complete: 21.2%; Average loss: 3.1455\n",
      "Iteration: 849; Percent complete: 21.2%; Average loss: 2.9341\n",
      "Iteration: 850; Percent complete: 21.2%; Average loss: 2.9129\n",
      "Iteration: 851; Percent complete: 21.3%; Average loss: 3.1066\n",
      "Iteration: 852; Percent complete: 21.3%; Average loss: 2.8731\n",
      "Iteration: 853; Percent complete: 21.3%; Average loss: 3.1019\n",
      "Iteration: 854; Percent complete: 21.3%; Average loss: 3.1754\n",
      "Iteration: 855; Percent complete: 21.4%; Average loss: 3.2100\n",
      "Iteration: 856; Percent complete: 21.4%; Average loss: 3.1493\n",
      "Iteration: 857; Percent complete: 21.4%; Average loss: 3.0422\n",
      "Iteration: 858; Percent complete: 21.4%; Average loss: 2.9771\n",
      "Iteration: 859; Percent complete: 21.5%; Average loss: 3.1299\n",
      "Iteration: 860; Percent complete: 21.5%; Average loss: 2.7440\n",
      "Iteration: 861; Percent complete: 21.5%; Average loss: 2.8624\n",
      "Iteration: 862; Percent complete: 21.6%; Average loss: 3.0080\n",
      "Iteration: 863; Percent complete: 21.6%; Average loss: 3.0473\n",
      "Iteration: 864; Percent complete: 21.6%; Average loss: 3.2691\n",
      "Iteration: 865; Percent complete: 21.6%; Average loss: 2.8552\n",
      "Iteration: 866; Percent complete: 21.6%; Average loss: 2.9173\n",
      "Iteration: 867; Percent complete: 21.7%; Average loss: 3.0441\n",
      "Iteration: 868; Percent complete: 21.7%; Average loss: 2.7828\n",
      "Iteration: 869; Percent complete: 21.7%; Average loss: 3.0185\n",
      "Iteration: 870; Percent complete: 21.8%; Average loss: 3.0507\n",
      "Iteration: 871; Percent complete: 21.8%; Average loss: 3.0739\n",
      "Iteration: 872; Percent complete: 21.8%; Average loss: 2.6212\n",
      "Iteration: 873; Percent complete: 21.8%; Average loss: 3.0268\n",
      "Iteration: 874; Percent complete: 21.9%; Average loss: 2.8447\n",
      "Iteration: 875; Percent complete: 21.9%; Average loss: 2.7832\n",
      "Iteration: 876; Percent complete: 21.9%; Average loss: 3.4676\n",
      "Iteration: 877; Percent complete: 21.9%; Average loss: 2.8433\n",
      "Iteration: 878; Percent complete: 21.9%; Average loss: 2.9193\n",
      "Iteration: 879; Percent complete: 22.0%; Average loss: 2.9192\n",
      "Iteration: 880; Percent complete: 22.0%; Average loss: 3.2973\n",
      "Iteration: 881; Percent complete: 22.0%; Average loss: 3.0591\n",
      "Iteration: 882; Percent complete: 22.1%; Average loss: 2.8892\n",
      "Iteration: 883; Percent complete: 22.1%; Average loss: 3.1230\n",
      "Iteration: 884; Percent complete: 22.1%; Average loss: 3.0324\n",
      "Iteration: 885; Percent complete: 22.1%; Average loss: 3.1522\n",
      "Iteration: 886; Percent complete: 22.1%; Average loss: 3.0923\n",
      "Iteration: 887; Percent complete: 22.2%; Average loss: 3.0519\n",
      "Iteration: 888; Percent complete: 22.2%; Average loss: 2.9501\n",
      "Iteration: 889; Percent complete: 22.2%; Average loss: 2.9588\n",
      "Iteration: 890; Percent complete: 22.2%; Average loss: 3.3399\n",
      "Iteration: 891; Percent complete: 22.3%; Average loss: 3.0283\n",
      "Iteration: 892; Percent complete: 22.3%; Average loss: 2.9726\n",
      "Iteration: 893; Percent complete: 22.3%; Average loss: 3.0716\n",
      "Iteration: 894; Percent complete: 22.4%; Average loss: 3.1597\n",
      "Iteration: 895; Percent complete: 22.4%; Average loss: 3.0456\n",
      "Iteration: 896; Percent complete: 22.4%; Average loss: 3.0945\n",
      "Iteration: 897; Percent complete: 22.4%; Average loss: 2.9408\n",
      "Iteration: 898; Percent complete: 22.4%; Average loss: 2.9257\n",
      "Iteration: 899; Percent complete: 22.5%; Average loss: 2.9095\n",
      "Iteration: 900; Percent complete: 22.5%; Average loss: 2.9253\n",
      "Iteration: 901; Percent complete: 22.5%; Average loss: 2.8858\n",
      "Iteration: 902; Percent complete: 22.6%; Average loss: 2.6052\n",
      "Iteration: 903; Percent complete: 22.6%; Average loss: 2.6644\n",
      "Iteration: 904; Percent complete: 22.6%; Average loss: 2.9765\n",
      "Iteration: 905; Percent complete: 22.6%; Average loss: 3.0079\n",
      "Iteration: 906; Percent complete: 22.7%; Average loss: 3.1409\n",
      "Iteration: 907; Percent complete: 22.7%; Average loss: 2.7939\n",
      "Iteration: 908; Percent complete: 22.7%; Average loss: 2.8919\n",
      "Iteration: 909; Percent complete: 22.7%; Average loss: 3.1715\n",
      "Iteration: 910; Percent complete: 22.8%; Average loss: 3.0160\n",
      "Iteration: 911; Percent complete: 22.8%; Average loss: 2.8620\n",
      "Iteration: 912; Percent complete: 22.8%; Average loss: 2.8740\n",
      "Iteration: 913; Percent complete: 22.8%; Average loss: 2.8584\n",
      "Iteration: 914; Percent complete: 22.9%; Average loss: 2.9761\n",
      "Iteration: 915; Percent complete: 22.9%; Average loss: 2.8006\n",
      "Iteration: 916; Percent complete: 22.9%; Average loss: 2.8081\n",
      "Iteration: 917; Percent complete: 22.9%; Average loss: 3.1468\n",
      "Iteration: 918; Percent complete: 22.9%; Average loss: 3.1542\n",
      "Iteration: 919; Percent complete: 23.0%; Average loss: 2.7645\n",
      "Iteration: 920; Percent complete: 23.0%; Average loss: 2.8604\n",
      "Iteration: 921; Percent complete: 23.0%; Average loss: 3.0634\n",
      "Iteration: 922; Percent complete: 23.1%; Average loss: 2.9670\n",
      "Iteration: 923; Percent complete: 23.1%; Average loss: 3.0748\n",
      "Iteration: 924; Percent complete: 23.1%; Average loss: 2.8806\n",
      "Iteration: 925; Percent complete: 23.1%; Average loss: 2.9668\n",
      "Iteration: 926; Percent complete: 23.2%; Average loss: 3.0664\n",
      "Iteration: 927; Percent complete: 23.2%; Average loss: 2.7567\n",
      "Iteration: 928; Percent complete: 23.2%; Average loss: 3.0849\n",
      "Iteration: 929; Percent complete: 23.2%; Average loss: 2.9040\n",
      "Iteration: 930; Percent complete: 23.2%; Average loss: 2.6935\n",
      "Iteration: 931; Percent complete: 23.3%; Average loss: 3.0384\n",
      "Iteration: 932; Percent complete: 23.3%; Average loss: 2.9476\n",
      "Iteration: 933; Percent complete: 23.3%; Average loss: 3.0701\n",
      "Iteration: 934; Percent complete: 23.4%; Average loss: 3.4107\n",
      "Iteration: 935; Percent complete: 23.4%; Average loss: 2.9985\n",
      "Iteration: 936; Percent complete: 23.4%; Average loss: 2.8476\n",
      "Iteration: 937; Percent complete: 23.4%; Average loss: 2.9782\n",
      "Iteration: 938; Percent complete: 23.4%; Average loss: 2.9366\n",
      "Iteration: 939; Percent complete: 23.5%; Average loss: 2.8650\n",
      "Iteration: 940; Percent complete: 23.5%; Average loss: 3.1178\n",
      "Iteration: 941; Percent complete: 23.5%; Average loss: 2.9597\n",
      "Iteration: 942; Percent complete: 23.5%; Average loss: 2.9156\n",
      "Iteration: 943; Percent complete: 23.6%; Average loss: 2.8233\n",
      "Iteration: 944; Percent complete: 23.6%; Average loss: 2.6896\n",
      "Iteration: 945; Percent complete: 23.6%; Average loss: 2.9539\n",
      "Iteration: 946; Percent complete: 23.6%; Average loss: 2.6977\n",
      "Iteration: 947; Percent complete: 23.7%; Average loss: 3.0427\n",
      "Iteration: 948; Percent complete: 23.7%; Average loss: 2.9729\n",
      "Iteration: 949; Percent complete: 23.7%; Average loss: 2.9639\n",
      "Iteration: 950; Percent complete: 23.8%; Average loss: 2.9485\n",
      "Iteration: 951; Percent complete: 23.8%; Average loss: 3.1614\n",
      "Iteration: 952; Percent complete: 23.8%; Average loss: 3.0974\n",
      "Iteration: 953; Percent complete: 23.8%; Average loss: 2.8366\n",
      "Iteration: 954; Percent complete: 23.8%; Average loss: 2.8994\n",
      "Iteration: 955; Percent complete: 23.9%; Average loss: 2.9372\n",
      "Iteration: 956; Percent complete: 23.9%; Average loss: 3.2751\n",
      "Iteration: 957; Percent complete: 23.9%; Average loss: 2.8314\n",
      "Iteration: 958; Percent complete: 23.9%; Average loss: 3.1268\n",
      "Iteration: 959; Percent complete: 24.0%; Average loss: 3.2720\n",
      "Iteration: 960; Percent complete: 24.0%; Average loss: 2.8676\n",
      "Iteration: 961; Percent complete: 24.0%; Average loss: 3.0269\n",
      "Iteration: 962; Percent complete: 24.1%; Average loss: 2.9107\n",
      "Iteration: 963; Percent complete: 24.1%; Average loss: 2.8169\n",
      "Iteration: 964; Percent complete: 24.1%; Average loss: 3.1420\n",
      "Iteration: 965; Percent complete: 24.1%; Average loss: 2.7755\n",
      "Iteration: 966; Percent complete: 24.1%; Average loss: 3.4812\n",
      "Iteration: 967; Percent complete: 24.2%; Average loss: 2.7019\n",
      "Iteration: 968; Percent complete: 24.2%; Average loss: 2.7481\n",
      "Iteration: 969; Percent complete: 24.2%; Average loss: 3.0745\n",
      "Iteration: 970; Percent complete: 24.2%; Average loss: 2.8791\n",
      "Iteration: 971; Percent complete: 24.3%; Average loss: 2.7939\n",
      "Iteration: 972; Percent complete: 24.3%; Average loss: 2.7271\n",
      "Iteration: 973; Percent complete: 24.3%; Average loss: 2.9450\n",
      "Iteration: 974; Percent complete: 24.3%; Average loss: 2.9300\n",
      "Iteration: 975; Percent complete: 24.4%; Average loss: 2.8707\n",
      "Iteration: 976; Percent complete: 24.4%; Average loss: 2.9763\n",
      "Iteration: 977; Percent complete: 24.4%; Average loss: 2.8774\n",
      "Iteration: 978; Percent complete: 24.4%; Average loss: 2.9254\n",
      "Iteration: 979; Percent complete: 24.5%; Average loss: 2.8480\n",
      "Iteration: 980; Percent complete: 24.5%; Average loss: 2.9120\n",
      "Iteration: 981; Percent complete: 24.5%; Average loss: 3.0404\n",
      "Iteration: 982; Percent complete: 24.6%; Average loss: 2.7914\n",
      "Iteration: 983; Percent complete: 24.6%; Average loss: 2.8964\n",
      "Iteration: 984; Percent complete: 24.6%; Average loss: 2.9225\n",
      "Iteration: 985; Percent complete: 24.6%; Average loss: 2.9151\n",
      "Iteration: 986; Percent complete: 24.6%; Average loss: 2.8491\n",
      "Iteration: 987; Percent complete: 24.7%; Average loss: 2.8163\n",
      "Iteration: 988; Percent complete: 24.7%; Average loss: 2.8350\n",
      "Iteration: 989; Percent complete: 24.7%; Average loss: 3.0188\n",
      "Iteration: 990; Percent complete: 24.8%; Average loss: 3.0780\n",
      "Iteration: 991; Percent complete: 24.8%; Average loss: 2.9251\n",
      "Iteration: 992; Percent complete: 24.8%; Average loss: 3.2596\n",
      "Iteration: 993; Percent complete: 24.8%; Average loss: 2.7656\n",
      "Iteration: 994; Percent complete: 24.9%; Average loss: 3.2022\n",
      "Iteration: 995; Percent complete: 24.9%; Average loss: 2.8364\n",
      "Iteration: 996; Percent complete: 24.9%; Average loss: 3.0098\n",
      "Iteration: 997; Percent complete: 24.9%; Average loss: 2.8239\n",
      "Iteration: 998; Percent complete: 24.9%; Average loss: 2.6132\n",
      "Iteration: 999; Percent complete: 25.0%; Average loss: 3.0401\n",
      "Iteration: 1000; Percent complete: 25.0%; Average loss: 2.9290\n",
      "Iteration: 1001; Percent complete: 25.0%; Average loss: 2.8204\n",
      "Iteration: 1002; Percent complete: 25.1%; Average loss: 2.9810\n",
      "Iteration: 1003; Percent complete: 25.1%; Average loss: 2.8843\n",
      "Iteration: 1004; Percent complete: 25.1%; Average loss: 2.8044\n",
      "Iteration: 1005; Percent complete: 25.1%; Average loss: 2.7189\n",
      "Iteration: 1006; Percent complete: 25.1%; Average loss: 2.9512\n",
      "Iteration: 1007; Percent complete: 25.2%; Average loss: 3.1961\n",
      "Iteration: 1008; Percent complete: 25.2%; Average loss: 3.1215\n",
      "Iteration: 1009; Percent complete: 25.2%; Average loss: 3.0115\n",
      "Iteration: 1010; Percent complete: 25.2%; Average loss: 2.8976\n",
      "Iteration: 1011; Percent complete: 25.3%; Average loss: 2.8787\n",
      "Iteration: 1012; Percent complete: 25.3%; Average loss: 2.7330\n",
      "Iteration: 1013; Percent complete: 25.3%; Average loss: 2.9848\n",
      "Iteration: 1014; Percent complete: 25.4%; Average loss: 3.0947\n",
      "Iteration: 1015; Percent complete: 25.4%; Average loss: 2.9184\n",
      "Iteration: 1016; Percent complete: 25.4%; Average loss: 2.9496\n",
      "Iteration: 1017; Percent complete: 25.4%; Average loss: 3.0168\n",
      "Iteration: 1018; Percent complete: 25.4%; Average loss: 2.8741\n",
      "Iteration: 1019; Percent complete: 25.5%; Average loss: 2.7631\n",
      "Iteration: 1020; Percent complete: 25.5%; Average loss: 2.9508\n",
      "Iteration: 1021; Percent complete: 25.5%; Average loss: 3.0852\n",
      "Iteration: 1022; Percent complete: 25.6%; Average loss: 2.7424\n",
      "Iteration: 1023; Percent complete: 25.6%; Average loss: 2.9425\n",
      "Iteration: 1024; Percent complete: 25.6%; Average loss: 2.6058\n",
      "Iteration: 1025; Percent complete: 25.6%; Average loss: 2.9021\n",
      "Iteration: 1026; Percent complete: 25.7%; Average loss: 3.1542\n",
      "Iteration: 1027; Percent complete: 25.7%; Average loss: 2.8838\n",
      "Iteration: 1028; Percent complete: 25.7%; Average loss: 3.1120\n",
      "Iteration: 1029; Percent complete: 25.7%; Average loss: 3.0186\n",
      "Iteration: 1030; Percent complete: 25.8%; Average loss: 3.0819\n",
      "Iteration: 1031; Percent complete: 25.8%; Average loss: 2.9578\n",
      "Iteration: 1032; Percent complete: 25.8%; Average loss: 3.3832\n",
      "Iteration: 1033; Percent complete: 25.8%; Average loss: 2.9680\n",
      "Iteration: 1034; Percent complete: 25.9%; Average loss: 3.1412\n",
      "Iteration: 1035; Percent complete: 25.9%; Average loss: 3.0681\n",
      "Iteration: 1036; Percent complete: 25.9%; Average loss: 2.9287\n",
      "Iteration: 1037; Percent complete: 25.9%; Average loss: 2.9773\n",
      "Iteration: 1038; Percent complete: 25.9%; Average loss: 3.1298\n",
      "Iteration: 1039; Percent complete: 26.0%; Average loss: 2.8612\n",
      "Iteration: 1040; Percent complete: 26.0%; Average loss: 2.9867\n",
      "Iteration: 1041; Percent complete: 26.0%; Average loss: 3.2378\n",
      "Iteration: 1042; Percent complete: 26.1%; Average loss: 3.0262\n",
      "Iteration: 1043; Percent complete: 26.1%; Average loss: 2.8102\n",
      "Iteration: 1044; Percent complete: 26.1%; Average loss: 2.8278\n",
      "Iteration: 1045; Percent complete: 26.1%; Average loss: 2.8303\n",
      "Iteration: 1046; Percent complete: 26.2%; Average loss: 2.7614\n",
      "Iteration: 1047; Percent complete: 26.2%; Average loss: 2.9981\n",
      "Iteration: 1048; Percent complete: 26.2%; Average loss: 3.1723\n",
      "Iteration: 1049; Percent complete: 26.2%; Average loss: 2.9603\n",
      "Iteration: 1050; Percent complete: 26.2%; Average loss: 3.1282\n",
      "Iteration: 1051; Percent complete: 26.3%; Average loss: 3.0397\n",
      "Iteration: 1052; Percent complete: 26.3%; Average loss: 2.9681\n",
      "Iteration: 1053; Percent complete: 26.3%; Average loss: 3.0358\n",
      "Iteration: 1054; Percent complete: 26.4%; Average loss: 2.8854\n",
      "Iteration: 1055; Percent complete: 26.4%; Average loss: 2.8114\n",
      "Iteration: 1056; Percent complete: 26.4%; Average loss: 2.7530\n",
      "Iteration: 1057; Percent complete: 26.4%; Average loss: 2.7866\n",
      "Iteration: 1058; Percent complete: 26.5%; Average loss: 2.7466\n",
      "Iteration: 1059; Percent complete: 26.5%; Average loss: 3.1031\n",
      "Iteration: 1060; Percent complete: 26.5%; Average loss: 2.7946\n",
      "Iteration: 1061; Percent complete: 26.5%; Average loss: 2.6594\n",
      "Iteration: 1062; Percent complete: 26.6%; Average loss: 3.1021\n",
      "Iteration: 1063; Percent complete: 26.6%; Average loss: 3.1529\n",
      "Iteration: 1064; Percent complete: 26.6%; Average loss: 2.7255\n",
      "Iteration: 1065; Percent complete: 26.6%; Average loss: 3.0820\n",
      "Iteration: 1066; Percent complete: 26.7%; Average loss: 3.0192\n",
      "Iteration: 1067; Percent complete: 26.7%; Average loss: 2.9716\n",
      "Iteration: 1068; Percent complete: 26.7%; Average loss: 2.7939\n",
      "Iteration: 1069; Percent complete: 26.7%; Average loss: 3.0515\n",
      "Iteration: 1070; Percent complete: 26.8%; Average loss: 2.8846\n",
      "Iteration: 1071; Percent complete: 26.8%; Average loss: 3.0496\n",
      "Iteration: 1072; Percent complete: 26.8%; Average loss: 2.6700\n",
      "Iteration: 1073; Percent complete: 26.8%; Average loss: 2.7213\n",
      "Iteration: 1074; Percent complete: 26.9%; Average loss: 2.7685\n",
      "Iteration: 1075; Percent complete: 26.9%; Average loss: 2.6616\n",
      "Iteration: 1076; Percent complete: 26.9%; Average loss: 2.7812\n",
      "Iteration: 1077; Percent complete: 26.9%; Average loss: 2.9658\n",
      "Iteration: 1078; Percent complete: 27.0%; Average loss: 2.9761\n",
      "Iteration: 1079; Percent complete: 27.0%; Average loss: 3.0147\n",
      "Iteration: 1080; Percent complete: 27.0%; Average loss: 2.8599\n",
      "Iteration: 1081; Percent complete: 27.0%; Average loss: 3.0976\n",
      "Iteration: 1082; Percent complete: 27.1%; Average loss: 2.7588\n",
      "Iteration: 1083; Percent complete: 27.1%; Average loss: 2.9863\n",
      "Iteration: 1084; Percent complete: 27.1%; Average loss: 2.7943\n",
      "Iteration: 1085; Percent complete: 27.1%; Average loss: 2.9958\n",
      "Iteration: 1086; Percent complete: 27.2%; Average loss: 2.9573\n",
      "Iteration: 1087; Percent complete: 27.2%; Average loss: 2.8852\n",
      "Iteration: 1088; Percent complete: 27.2%; Average loss: 2.8722\n",
      "Iteration: 1089; Percent complete: 27.2%; Average loss: 3.0376\n",
      "Iteration: 1090; Percent complete: 27.3%; Average loss: 2.8939\n",
      "Iteration: 1091; Percent complete: 27.3%; Average loss: 3.1988\n",
      "Iteration: 1092; Percent complete: 27.3%; Average loss: 2.7274\n",
      "Iteration: 1093; Percent complete: 27.3%; Average loss: 2.8300\n",
      "Iteration: 1094; Percent complete: 27.4%; Average loss: 2.8234\n",
      "Iteration: 1095; Percent complete: 27.4%; Average loss: 2.8951\n",
      "Iteration: 1096; Percent complete: 27.4%; Average loss: 2.9268\n",
      "Iteration: 1097; Percent complete: 27.4%; Average loss: 3.0030\n",
      "Iteration: 1098; Percent complete: 27.5%; Average loss: 2.8910\n",
      "Iteration: 1099; Percent complete: 27.5%; Average loss: 2.8054\n",
      "Iteration: 1100; Percent complete: 27.5%; Average loss: 2.9586\n",
      "Iteration: 1101; Percent complete: 27.5%; Average loss: 2.9525\n",
      "Iteration: 1102; Percent complete: 27.6%; Average loss: 2.9723\n",
      "Iteration: 1103; Percent complete: 27.6%; Average loss: 2.7117\n",
      "Iteration: 1104; Percent complete: 27.6%; Average loss: 2.7962\n",
      "Iteration: 1105; Percent complete: 27.6%; Average loss: 3.0258\n",
      "Iteration: 1106; Percent complete: 27.7%; Average loss: 2.8765\n",
      "Iteration: 1107; Percent complete: 27.7%; Average loss: 2.7794\n",
      "Iteration: 1108; Percent complete: 27.7%; Average loss: 2.7496\n",
      "Iteration: 1109; Percent complete: 27.7%; Average loss: 2.5690\n",
      "Iteration: 1110; Percent complete: 27.8%; Average loss: 2.8829\n",
      "Iteration: 1111; Percent complete: 27.8%; Average loss: 2.7908\n",
      "Iteration: 1112; Percent complete: 27.8%; Average loss: 2.9758\n",
      "Iteration: 1113; Percent complete: 27.8%; Average loss: 2.9885\n",
      "Iteration: 1114; Percent complete: 27.9%; Average loss: 2.8579\n",
      "Iteration: 1115; Percent complete: 27.9%; Average loss: 3.0971\n",
      "Iteration: 1116; Percent complete: 27.9%; Average loss: 2.7044\n",
      "Iteration: 1117; Percent complete: 27.9%; Average loss: 2.7891\n",
      "Iteration: 1118; Percent complete: 28.0%; Average loss: 3.0741\n",
      "Iteration: 1119; Percent complete: 28.0%; Average loss: 3.1700\n",
      "Iteration: 1120; Percent complete: 28.0%; Average loss: 2.9684\n",
      "Iteration: 1121; Percent complete: 28.0%; Average loss: 2.8990\n",
      "Iteration: 1122; Percent complete: 28.1%; Average loss: 2.7844\n",
      "Iteration: 1123; Percent complete: 28.1%; Average loss: 2.8300\n",
      "Iteration: 1124; Percent complete: 28.1%; Average loss: 3.1742\n",
      "Iteration: 1125; Percent complete: 28.1%; Average loss: 2.9673\n",
      "Iteration: 1126; Percent complete: 28.1%; Average loss: 2.9055\n",
      "Iteration: 1127; Percent complete: 28.2%; Average loss: 2.7229\n",
      "Iteration: 1128; Percent complete: 28.2%; Average loss: 2.8747\n",
      "Iteration: 1129; Percent complete: 28.2%; Average loss: 2.8361\n",
      "Iteration: 1130; Percent complete: 28.2%; Average loss: 3.0647\n",
      "Iteration: 1131; Percent complete: 28.3%; Average loss: 3.0036\n",
      "Iteration: 1132; Percent complete: 28.3%; Average loss: 2.9020\n",
      "Iteration: 1133; Percent complete: 28.3%; Average loss: 2.9016\n",
      "Iteration: 1134; Percent complete: 28.3%; Average loss: 2.9659\n",
      "Iteration: 1135; Percent complete: 28.4%; Average loss: 2.6968\n",
      "Iteration: 1136; Percent complete: 28.4%; Average loss: 2.9997\n",
      "Iteration: 1137; Percent complete: 28.4%; Average loss: 3.1723\n",
      "Iteration: 1138; Percent complete: 28.4%; Average loss: 2.9605\n",
      "Iteration: 1139; Percent complete: 28.5%; Average loss: 2.6687\n",
      "Iteration: 1140; Percent complete: 28.5%; Average loss: 2.9608\n",
      "Iteration: 1141; Percent complete: 28.5%; Average loss: 3.2033\n",
      "Iteration: 1142; Percent complete: 28.5%; Average loss: 2.8245\n",
      "Iteration: 1143; Percent complete: 28.6%; Average loss: 2.9707\n",
      "Iteration: 1144; Percent complete: 28.6%; Average loss: 2.7237\n",
      "Iteration: 1145; Percent complete: 28.6%; Average loss: 2.6524\n",
      "Iteration: 1146; Percent complete: 28.6%; Average loss: 2.5082\n",
      "Iteration: 1147; Percent complete: 28.7%; Average loss: 2.5969\n",
      "Iteration: 1148; Percent complete: 28.7%; Average loss: 2.8255\n",
      "Iteration: 1149; Percent complete: 28.7%; Average loss: 2.8331\n",
      "Iteration: 1150; Percent complete: 28.7%; Average loss: 2.7922\n",
      "Iteration: 1151; Percent complete: 28.8%; Average loss: 3.0902\n",
      "Iteration: 1152; Percent complete: 28.8%; Average loss: 3.0287\n",
      "Iteration: 1153; Percent complete: 28.8%; Average loss: 3.1160\n",
      "Iteration: 1154; Percent complete: 28.8%; Average loss: 2.9285\n",
      "Iteration: 1155; Percent complete: 28.9%; Average loss: 2.9475\n",
      "Iteration: 1156; Percent complete: 28.9%; Average loss: 2.9411\n",
      "Iteration: 1157; Percent complete: 28.9%; Average loss: 2.6054\n",
      "Iteration: 1158; Percent complete: 28.9%; Average loss: 2.7797\n",
      "Iteration: 1159; Percent complete: 29.0%; Average loss: 3.0161\n",
      "Iteration: 1160; Percent complete: 29.0%; Average loss: 2.9111\n",
      "Iteration: 1161; Percent complete: 29.0%; Average loss: 3.0836\n",
      "Iteration: 1162; Percent complete: 29.0%; Average loss: 2.9088\n",
      "Iteration: 1163; Percent complete: 29.1%; Average loss: 2.8548\n",
      "Iteration: 1164; Percent complete: 29.1%; Average loss: 2.9361\n",
      "Iteration: 1165; Percent complete: 29.1%; Average loss: 2.8029\n",
      "Iteration: 1166; Percent complete: 29.1%; Average loss: 2.7882\n",
      "Iteration: 1167; Percent complete: 29.2%; Average loss: 3.0379\n",
      "Iteration: 1168; Percent complete: 29.2%; Average loss: 2.9559\n",
      "Iteration: 1169; Percent complete: 29.2%; Average loss: 2.9898\n",
      "Iteration: 1170; Percent complete: 29.2%; Average loss: 2.8478\n",
      "Iteration: 1171; Percent complete: 29.3%; Average loss: 2.8841\n",
      "Iteration: 1172; Percent complete: 29.3%; Average loss: 2.9979\n",
      "Iteration: 1173; Percent complete: 29.3%; Average loss: 2.8751\n",
      "Iteration: 1174; Percent complete: 29.3%; Average loss: 2.8543\n",
      "Iteration: 1175; Percent complete: 29.4%; Average loss: 3.0071\n",
      "Iteration: 1176; Percent complete: 29.4%; Average loss: 2.8575\n",
      "Iteration: 1177; Percent complete: 29.4%; Average loss: 3.1080\n",
      "Iteration: 1178; Percent complete: 29.4%; Average loss: 2.9310\n",
      "Iteration: 1179; Percent complete: 29.5%; Average loss: 2.8332\n",
      "Iteration: 1180; Percent complete: 29.5%; Average loss: 2.9888\n",
      "Iteration: 1181; Percent complete: 29.5%; Average loss: 2.7671\n",
      "Iteration: 1182; Percent complete: 29.5%; Average loss: 2.7474\n",
      "Iteration: 1183; Percent complete: 29.6%; Average loss: 2.7236\n",
      "Iteration: 1184; Percent complete: 29.6%; Average loss: 2.4556\n",
      "Iteration: 1185; Percent complete: 29.6%; Average loss: 2.6562\n",
      "Iteration: 1186; Percent complete: 29.6%; Average loss: 2.7429\n",
      "Iteration: 1187; Percent complete: 29.7%; Average loss: 2.9188\n",
      "Iteration: 1188; Percent complete: 29.7%; Average loss: 2.6018\n",
      "Iteration: 1189; Percent complete: 29.7%; Average loss: 2.9634\n",
      "Iteration: 1190; Percent complete: 29.8%; Average loss: 2.8748\n",
      "Iteration: 1191; Percent complete: 29.8%; Average loss: 2.6535\n",
      "Iteration: 1192; Percent complete: 29.8%; Average loss: 3.0482\n",
      "Iteration: 1193; Percent complete: 29.8%; Average loss: 3.1107\n",
      "Iteration: 1194; Percent complete: 29.8%; Average loss: 2.8470\n",
      "Iteration: 1195; Percent complete: 29.9%; Average loss: 2.8127\n",
      "Iteration: 1196; Percent complete: 29.9%; Average loss: 2.8516\n",
      "Iteration: 1197; Percent complete: 29.9%; Average loss: 2.7756\n",
      "Iteration: 1198; Percent complete: 29.9%; Average loss: 2.6610\n",
      "Iteration: 1199; Percent complete: 30.0%; Average loss: 2.8649\n",
      "Iteration: 1200; Percent complete: 30.0%; Average loss: 2.7215\n",
      "Iteration: 1201; Percent complete: 30.0%; Average loss: 2.8600\n",
      "Iteration: 1202; Percent complete: 30.0%; Average loss: 2.8763\n",
      "Iteration: 1203; Percent complete: 30.1%; Average loss: 2.4764\n",
      "Iteration: 1204; Percent complete: 30.1%; Average loss: 2.9319\n",
      "Iteration: 1205; Percent complete: 30.1%; Average loss: 3.0816\n",
      "Iteration: 1206; Percent complete: 30.1%; Average loss: 2.7062\n",
      "Iteration: 1207; Percent complete: 30.2%; Average loss: 2.7359\n",
      "Iteration: 1208; Percent complete: 30.2%; Average loss: 2.6808\n",
      "Iteration: 1209; Percent complete: 30.2%; Average loss: 2.7062\n",
      "Iteration: 1210; Percent complete: 30.2%; Average loss: 2.9172\n",
      "Iteration: 1211; Percent complete: 30.3%; Average loss: 2.8669\n",
      "Iteration: 1212; Percent complete: 30.3%; Average loss: 2.9656\n",
      "Iteration: 1213; Percent complete: 30.3%; Average loss: 3.1234\n",
      "Iteration: 1214; Percent complete: 30.3%; Average loss: 2.8269\n",
      "Iteration: 1215; Percent complete: 30.4%; Average loss: 2.7166\n",
      "Iteration: 1216; Percent complete: 30.4%; Average loss: 2.8077\n",
      "Iteration: 1217; Percent complete: 30.4%; Average loss: 2.8515\n",
      "Iteration: 1218; Percent complete: 30.4%; Average loss: 2.8431\n",
      "Iteration: 1219; Percent complete: 30.5%; Average loss: 2.4863\n",
      "Iteration: 1220; Percent complete: 30.5%; Average loss: 2.6651\n",
      "Iteration: 1221; Percent complete: 30.5%; Average loss: 3.2144\n",
      "Iteration: 1222; Percent complete: 30.6%; Average loss: 2.7068\n",
      "Iteration: 1223; Percent complete: 30.6%; Average loss: 3.0240\n",
      "Iteration: 1224; Percent complete: 30.6%; Average loss: 2.7507\n",
      "Iteration: 1225; Percent complete: 30.6%; Average loss: 2.9271\n",
      "Iteration: 1226; Percent complete: 30.6%; Average loss: 2.8633\n",
      "Iteration: 1227; Percent complete: 30.7%; Average loss: 3.1725\n",
      "Iteration: 1228; Percent complete: 30.7%; Average loss: 2.7752\n",
      "Iteration: 1229; Percent complete: 30.7%; Average loss: 2.6708\n",
      "Iteration: 1230; Percent complete: 30.8%; Average loss: 2.8361\n",
      "Iteration: 1231; Percent complete: 30.8%; Average loss: 2.9631\n",
      "Iteration: 1232; Percent complete: 30.8%; Average loss: 2.7405\n",
      "Iteration: 1233; Percent complete: 30.8%; Average loss: 2.7982\n",
      "Iteration: 1234; Percent complete: 30.9%; Average loss: 2.7475\n",
      "Iteration: 1235; Percent complete: 30.9%; Average loss: 2.6693\n",
      "Iteration: 1236; Percent complete: 30.9%; Average loss: 2.9288\n",
      "Iteration: 1237; Percent complete: 30.9%; Average loss: 3.0458\n",
      "Iteration: 1238; Percent complete: 30.9%; Average loss: 2.9079\n",
      "Iteration: 1239; Percent complete: 31.0%; Average loss: 2.9627\n",
      "Iteration: 1240; Percent complete: 31.0%; Average loss: 2.9518\n",
      "Iteration: 1241; Percent complete: 31.0%; Average loss: 2.7812\n",
      "Iteration: 1242; Percent complete: 31.1%; Average loss: 2.8957\n",
      "Iteration: 1243; Percent complete: 31.1%; Average loss: 3.1885\n",
      "Iteration: 1244; Percent complete: 31.1%; Average loss: 3.0055\n",
      "Iteration: 1245; Percent complete: 31.1%; Average loss: 2.8665\n",
      "Iteration: 1246; Percent complete: 31.1%; Average loss: 2.7265\n",
      "Iteration: 1247; Percent complete: 31.2%; Average loss: 2.9320\n",
      "Iteration: 1248; Percent complete: 31.2%; Average loss: 2.7817\n",
      "Iteration: 1249; Percent complete: 31.2%; Average loss: 3.0451\n",
      "Iteration: 1250; Percent complete: 31.2%; Average loss: 2.7861\n",
      "Iteration: 1251; Percent complete: 31.3%; Average loss: 2.5787\n",
      "Iteration: 1252; Percent complete: 31.3%; Average loss: 2.9374\n",
      "Iteration: 1253; Percent complete: 31.3%; Average loss: 2.9361\n",
      "Iteration: 1254; Percent complete: 31.4%; Average loss: 2.8955\n",
      "Iteration: 1255; Percent complete: 31.4%; Average loss: 2.9267\n",
      "Iteration: 1256; Percent complete: 31.4%; Average loss: 2.8792\n",
      "Iteration: 1257; Percent complete: 31.4%; Average loss: 2.8973\n",
      "Iteration: 1258; Percent complete: 31.4%; Average loss: 2.8809\n",
      "Iteration: 1259; Percent complete: 31.5%; Average loss: 2.8578\n",
      "Iteration: 1260; Percent complete: 31.5%; Average loss: 2.5902\n",
      "Iteration: 1261; Percent complete: 31.5%; Average loss: 2.9128\n",
      "Iteration: 1262; Percent complete: 31.6%; Average loss: 2.7483\n",
      "Iteration: 1263; Percent complete: 31.6%; Average loss: 3.1226\n",
      "Iteration: 1264; Percent complete: 31.6%; Average loss: 2.9398\n",
      "Iteration: 1265; Percent complete: 31.6%; Average loss: 2.7431\n",
      "Iteration: 1266; Percent complete: 31.6%; Average loss: 2.8342\n",
      "Iteration: 1267; Percent complete: 31.7%; Average loss: 2.8755\n",
      "Iteration: 1268; Percent complete: 31.7%; Average loss: 3.0369\n",
      "Iteration: 1269; Percent complete: 31.7%; Average loss: 2.7163\n",
      "Iteration: 1270; Percent complete: 31.8%; Average loss: 2.7990\n",
      "Iteration: 1271; Percent complete: 31.8%; Average loss: 2.6637\n",
      "Iteration: 1272; Percent complete: 31.8%; Average loss: 2.8143\n",
      "Iteration: 1273; Percent complete: 31.8%; Average loss: 2.9448\n",
      "Iteration: 1274; Percent complete: 31.9%; Average loss: 2.8613\n",
      "Iteration: 1275; Percent complete: 31.9%; Average loss: 2.8159\n",
      "Iteration: 1276; Percent complete: 31.9%; Average loss: 2.8881\n",
      "Iteration: 1277; Percent complete: 31.9%; Average loss: 2.6996\n",
      "Iteration: 1278; Percent complete: 31.9%; Average loss: 2.8832\n",
      "Iteration: 1279; Percent complete: 32.0%; Average loss: 2.9033\n",
      "Iteration: 1280; Percent complete: 32.0%; Average loss: 2.7816\n",
      "Iteration: 1281; Percent complete: 32.0%; Average loss: 2.8486\n",
      "Iteration: 1282; Percent complete: 32.0%; Average loss: 3.0337\n",
      "Iteration: 1283; Percent complete: 32.1%; Average loss: 2.9734\n",
      "Iteration: 1284; Percent complete: 32.1%; Average loss: 2.8945\n",
      "Iteration: 1285; Percent complete: 32.1%; Average loss: 2.9496\n",
      "Iteration: 1286; Percent complete: 32.1%; Average loss: 3.0189\n",
      "Iteration: 1287; Percent complete: 32.2%; Average loss: 2.7025\n",
      "Iteration: 1288; Percent complete: 32.2%; Average loss: 3.0493\n",
      "Iteration: 1289; Percent complete: 32.2%; Average loss: 2.6154\n",
      "Iteration: 1290; Percent complete: 32.2%; Average loss: 2.8171\n",
      "Iteration: 1291; Percent complete: 32.3%; Average loss: 3.0546\n",
      "Iteration: 1292; Percent complete: 32.3%; Average loss: 2.7452\n",
      "Iteration: 1293; Percent complete: 32.3%; Average loss: 2.7709\n",
      "Iteration: 1294; Percent complete: 32.4%; Average loss: 2.8342\n",
      "Iteration: 1295; Percent complete: 32.4%; Average loss: 2.8982\n",
      "Iteration: 1296; Percent complete: 32.4%; Average loss: 2.9116\n",
      "Iteration: 1297; Percent complete: 32.4%; Average loss: 2.7189\n",
      "Iteration: 1298; Percent complete: 32.5%; Average loss: 2.7806\n",
      "Iteration: 1299; Percent complete: 32.5%; Average loss: 2.7014\n",
      "Iteration: 1300; Percent complete: 32.5%; Average loss: 2.9046\n",
      "Iteration: 1301; Percent complete: 32.5%; Average loss: 2.5285\n",
      "Iteration: 1302; Percent complete: 32.6%; Average loss: 2.6679\n",
      "Iteration: 1303; Percent complete: 32.6%; Average loss: 2.5832\n",
      "Iteration: 1304; Percent complete: 32.6%; Average loss: 2.8424\n",
      "Iteration: 1305; Percent complete: 32.6%; Average loss: 2.8460\n",
      "Iteration: 1306; Percent complete: 32.6%; Average loss: 2.8187\n",
      "Iteration: 1307; Percent complete: 32.7%; Average loss: 3.0627\n",
      "Iteration: 1308; Percent complete: 32.7%; Average loss: 2.8176\n",
      "Iteration: 1309; Percent complete: 32.7%; Average loss: 2.7003\n",
      "Iteration: 1310; Percent complete: 32.8%; Average loss: 2.8257\n",
      "Iteration: 1311; Percent complete: 32.8%; Average loss: 2.8725\n",
      "Iteration: 1312; Percent complete: 32.8%; Average loss: 2.9966\n",
      "Iteration: 1313; Percent complete: 32.8%; Average loss: 2.9106\n",
      "Iteration: 1314; Percent complete: 32.9%; Average loss: 2.8319\n",
      "Iteration: 1315; Percent complete: 32.9%; Average loss: 2.9285\n",
      "Iteration: 1316; Percent complete: 32.9%; Average loss: 2.9566\n",
      "Iteration: 1317; Percent complete: 32.9%; Average loss: 2.9272\n",
      "Iteration: 1318; Percent complete: 33.0%; Average loss: 3.0213\n",
      "Iteration: 1319; Percent complete: 33.0%; Average loss: 3.0368\n",
      "Iteration: 1320; Percent complete: 33.0%; Average loss: 2.8934\n",
      "Iteration: 1321; Percent complete: 33.0%; Average loss: 2.9224\n",
      "Iteration: 1322; Percent complete: 33.1%; Average loss: 2.9418\n",
      "Iteration: 1323; Percent complete: 33.1%; Average loss: 2.6981\n",
      "Iteration: 1324; Percent complete: 33.1%; Average loss: 2.8765\n",
      "Iteration: 1325; Percent complete: 33.1%; Average loss: 2.9568\n",
      "Iteration: 1326; Percent complete: 33.1%; Average loss: 2.8394\n",
      "Iteration: 1327; Percent complete: 33.2%; Average loss: 2.5446\n",
      "Iteration: 1328; Percent complete: 33.2%; Average loss: 2.8110\n",
      "Iteration: 1329; Percent complete: 33.2%; Average loss: 3.0968\n",
      "Iteration: 1330; Percent complete: 33.2%; Average loss: 2.8094\n",
      "Iteration: 1331; Percent complete: 33.3%; Average loss: 2.9219\n",
      "Iteration: 1332; Percent complete: 33.3%; Average loss: 2.9020\n",
      "Iteration: 1333; Percent complete: 33.3%; Average loss: 2.8916\n",
      "Iteration: 1334; Percent complete: 33.4%; Average loss: 2.8869\n",
      "Iteration: 1335; Percent complete: 33.4%; Average loss: 3.0225\n",
      "Iteration: 1336; Percent complete: 33.4%; Average loss: 2.9314\n",
      "Iteration: 1337; Percent complete: 33.4%; Average loss: 2.7733\n",
      "Iteration: 1338; Percent complete: 33.5%; Average loss: 2.6941\n",
      "Iteration: 1339; Percent complete: 33.5%; Average loss: 3.0508\n",
      "Iteration: 1340; Percent complete: 33.5%; Average loss: 2.9045\n",
      "Iteration: 1341; Percent complete: 33.5%; Average loss: 2.7373\n",
      "Iteration: 1342; Percent complete: 33.6%; Average loss: 2.5510\n",
      "Iteration: 1343; Percent complete: 33.6%; Average loss: 3.1905\n",
      "Iteration: 1344; Percent complete: 33.6%; Average loss: 2.9293\n",
      "Iteration: 1345; Percent complete: 33.6%; Average loss: 2.7977\n",
      "Iteration: 1346; Percent complete: 33.7%; Average loss: 2.9309\n",
      "Iteration: 1347; Percent complete: 33.7%; Average loss: 2.6818\n",
      "Iteration: 1348; Percent complete: 33.7%; Average loss: 2.8786\n",
      "Iteration: 1349; Percent complete: 33.7%; Average loss: 2.7366\n",
      "Iteration: 1350; Percent complete: 33.8%; Average loss: 2.8185\n",
      "Iteration: 1351; Percent complete: 33.8%; Average loss: 2.6315\n",
      "Iteration: 1352; Percent complete: 33.8%; Average loss: 3.0157\n",
      "Iteration: 1353; Percent complete: 33.8%; Average loss: 2.9247\n",
      "Iteration: 1354; Percent complete: 33.9%; Average loss: 2.6108\n",
      "Iteration: 1355; Percent complete: 33.9%; Average loss: 2.9252\n",
      "Iteration: 1356; Percent complete: 33.9%; Average loss: 2.7901\n",
      "Iteration: 1357; Percent complete: 33.9%; Average loss: 3.0723\n",
      "Iteration: 1358; Percent complete: 34.0%; Average loss: 3.2788\n",
      "Iteration: 1359; Percent complete: 34.0%; Average loss: 3.0582\n",
      "Iteration: 1360; Percent complete: 34.0%; Average loss: 2.8292\n",
      "Iteration: 1361; Percent complete: 34.0%; Average loss: 2.9431\n",
      "Iteration: 1362; Percent complete: 34.1%; Average loss: 2.7405\n",
      "Iteration: 1363; Percent complete: 34.1%; Average loss: 2.8750\n",
      "Iteration: 1364; Percent complete: 34.1%; Average loss: 2.8462\n",
      "Iteration: 1365; Percent complete: 34.1%; Average loss: 2.8244\n",
      "Iteration: 1366; Percent complete: 34.2%; Average loss: 3.0120\n",
      "Iteration: 1367; Percent complete: 34.2%; Average loss: 2.7221\n",
      "Iteration: 1368; Percent complete: 34.2%; Average loss: 2.7429\n",
      "Iteration: 1369; Percent complete: 34.2%; Average loss: 2.7396\n",
      "Iteration: 1370; Percent complete: 34.2%; Average loss: 3.0861\n",
      "Iteration: 1371; Percent complete: 34.3%; Average loss: 2.8550\n",
      "Iteration: 1372; Percent complete: 34.3%; Average loss: 2.6230\n",
      "Iteration: 1373; Percent complete: 34.3%; Average loss: 2.8497\n",
      "Iteration: 1374; Percent complete: 34.4%; Average loss: 2.8772\n",
      "Iteration: 1375; Percent complete: 34.4%; Average loss: 2.8591\n",
      "Iteration: 1376; Percent complete: 34.4%; Average loss: 2.8903\n",
      "Iteration: 1377; Percent complete: 34.4%; Average loss: 2.7425\n",
      "Iteration: 1378; Percent complete: 34.4%; Average loss: 2.8901\n",
      "Iteration: 1379; Percent complete: 34.5%; Average loss: 2.9235\n",
      "Iteration: 1380; Percent complete: 34.5%; Average loss: 2.4336\n",
      "Iteration: 1381; Percent complete: 34.5%; Average loss: 2.7909\n",
      "Iteration: 1382; Percent complete: 34.5%; Average loss: 2.4798\n",
      "Iteration: 1383; Percent complete: 34.6%; Average loss: 2.6396\n",
      "Iteration: 1384; Percent complete: 34.6%; Average loss: 2.7760\n",
      "Iteration: 1385; Percent complete: 34.6%; Average loss: 2.7370\n",
      "Iteration: 1386; Percent complete: 34.6%; Average loss: 2.8801\n",
      "Iteration: 1387; Percent complete: 34.7%; Average loss: 2.9288\n",
      "Iteration: 1388; Percent complete: 34.7%; Average loss: 2.7906\n",
      "Iteration: 1389; Percent complete: 34.7%; Average loss: 2.6936\n",
      "Iteration: 1390; Percent complete: 34.8%; Average loss: 2.7933\n",
      "Iteration: 1391; Percent complete: 34.8%; Average loss: 2.9204\n",
      "Iteration: 1392; Percent complete: 34.8%; Average loss: 2.6548\n",
      "Iteration: 1393; Percent complete: 34.8%; Average loss: 2.8757\n",
      "Iteration: 1394; Percent complete: 34.8%; Average loss: 2.9360\n",
      "Iteration: 1395; Percent complete: 34.9%; Average loss: 2.7375\n",
      "Iteration: 1396; Percent complete: 34.9%; Average loss: 2.5489\n",
      "Iteration: 1397; Percent complete: 34.9%; Average loss: 2.9471\n",
      "Iteration: 1398; Percent complete: 34.9%; Average loss: 2.8371\n",
      "Iteration: 1399; Percent complete: 35.0%; Average loss: 2.9473\n",
      "Iteration: 1400; Percent complete: 35.0%; Average loss: 2.7475\n",
      "Iteration: 1401; Percent complete: 35.0%; Average loss: 2.8283\n",
      "Iteration: 1402; Percent complete: 35.0%; Average loss: 2.8614\n",
      "Iteration: 1403; Percent complete: 35.1%; Average loss: 2.8076\n",
      "Iteration: 1404; Percent complete: 35.1%; Average loss: 2.6860\n",
      "Iteration: 1405; Percent complete: 35.1%; Average loss: 2.8200\n",
      "Iteration: 1406; Percent complete: 35.1%; Average loss: 2.7472\n",
      "Iteration: 1407; Percent complete: 35.2%; Average loss: 2.7716\n",
      "Iteration: 1408; Percent complete: 35.2%; Average loss: 2.8697\n",
      "Iteration: 1409; Percent complete: 35.2%; Average loss: 3.0356\n",
      "Iteration: 1410; Percent complete: 35.2%; Average loss: 2.6720\n",
      "Iteration: 1411; Percent complete: 35.3%; Average loss: 2.8159\n",
      "Iteration: 1412; Percent complete: 35.3%; Average loss: 2.7068\n",
      "Iteration: 1413; Percent complete: 35.3%; Average loss: 3.0211\n",
      "Iteration: 1414; Percent complete: 35.4%; Average loss: 2.8274\n",
      "Iteration: 1415; Percent complete: 35.4%; Average loss: 2.8182\n",
      "Iteration: 1416; Percent complete: 35.4%; Average loss: 2.4388\n",
      "Iteration: 1417; Percent complete: 35.4%; Average loss: 2.7563\n",
      "Iteration: 1418; Percent complete: 35.4%; Average loss: 2.9334\n",
      "Iteration: 1419; Percent complete: 35.5%; Average loss: 2.6251\n",
      "Iteration: 1420; Percent complete: 35.5%; Average loss: 2.9165\n",
      "Iteration: 1421; Percent complete: 35.5%; Average loss: 2.6398\n",
      "Iteration: 1422; Percent complete: 35.5%; Average loss: 2.9015\n",
      "Iteration: 1423; Percent complete: 35.6%; Average loss: 2.6674\n",
      "Iteration: 1424; Percent complete: 35.6%; Average loss: 2.4795\n",
      "Iteration: 1425; Percent complete: 35.6%; Average loss: 2.8005\n",
      "Iteration: 1426; Percent complete: 35.6%; Average loss: 3.0611\n",
      "Iteration: 1427; Percent complete: 35.7%; Average loss: 2.8801\n",
      "Iteration: 1428; Percent complete: 35.7%; Average loss: 2.7243\n",
      "Iteration: 1429; Percent complete: 35.7%; Average loss: 2.9363\n",
      "Iteration: 1430; Percent complete: 35.8%; Average loss: 2.7905\n",
      "Iteration: 1431; Percent complete: 35.8%; Average loss: 2.6679\n",
      "Iteration: 1432; Percent complete: 35.8%; Average loss: 2.6760\n",
      "Iteration: 1433; Percent complete: 35.8%; Average loss: 2.7008\n",
      "Iteration: 1434; Percent complete: 35.9%; Average loss: 2.8721\n",
      "Iteration: 1435; Percent complete: 35.9%; Average loss: 2.5657\n",
      "Iteration: 1436; Percent complete: 35.9%; Average loss: 3.0374\n",
      "Iteration: 1437; Percent complete: 35.9%; Average loss: 2.8243\n",
      "Iteration: 1438; Percent complete: 35.9%; Average loss: 3.0255\n",
      "Iteration: 1439; Percent complete: 36.0%; Average loss: 2.7978\n",
      "Iteration: 1440; Percent complete: 36.0%; Average loss: 2.8379\n",
      "Iteration: 1441; Percent complete: 36.0%; Average loss: 3.0122\n",
      "Iteration: 1442; Percent complete: 36.0%; Average loss: 2.7569\n",
      "Iteration: 1443; Percent complete: 36.1%; Average loss: 2.6905\n",
      "Iteration: 1444; Percent complete: 36.1%; Average loss: 2.7081\n",
      "Iteration: 1445; Percent complete: 36.1%; Average loss: 2.9213\n",
      "Iteration: 1446; Percent complete: 36.1%; Average loss: 2.7864\n",
      "Iteration: 1447; Percent complete: 36.2%; Average loss: 2.8223\n",
      "Iteration: 1448; Percent complete: 36.2%; Average loss: 2.8251\n",
      "Iteration: 1449; Percent complete: 36.2%; Average loss: 2.7196\n",
      "Iteration: 1450; Percent complete: 36.2%; Average loss: 2.8937\n",
      "Iteration: 1451; Percent complete: 36.3%; Average loss: 2.5612\n",
      "Iteration: 1452; Percent complete: 36.3%; Average loss: 2.8848\n",
      "Iteration: 1453; Percent complete: 36.3%; Average loss: 2.5413\n",
      "Iteration: 1454; Percent complete: 36.4%; Average loss: 2.7592\n",
      "Iteration: 1455; Percent complete: 36.4%; Average loss: 3.0709\n",
      "Iteration: 1456; Percent complete: 36.4%; Average loss: 2.7139\n",
      "Iteration: 1457; Percent complete: 36.4%; Average loss: 2.8954\n",
      "Iteration: 1458; Percent complete: 36.4%; Average loss: 3.1442\n",
      "Iteration: 1459; Percent complete: 36.5%; Average loss: 2.5095\n",
      "Iteration: 1460; Percent complete: 36.5%; Average loss: 3.1622\n",
      "Iteration: 1461; Percent complete: 36.5%; Average loss: 2.9227\n",
      "Iteration: 1462; Percent complete: 36.5%; Average loss: 2.7870\n",
      "Iteration: 1463; Percent complete: 36.6%; Average loss: 2.6678\n",
      "Iteration: 1464; Percent complete: 36.6%; Average loss: 2.5543\n",
      "Iteration: 1465; Percent complete: 36.6%; Average loss: 2.9331\n",
      "Iteration: 1466; Percent complete: 36.6%; Average loss: 2.6537\n",
      "Iteration: 1467; Percent complete: 36.7%; Average loss: 2.5780\n",
      "Iteration: 1468; Percent complete: 36.7%; Average loss: 2.4646\n",
      "Iteration: 1469; Percent complete: 36.7%; Average loss: 2.6076\n",
      "Iteration: 1470; Percent complete: 36.8%; Average loss: 2.8412\n",
      "Iteration: 1471; Percent complete: 36.8%; Average loss: 2.8436\n",
      "Iteration: 1472; Percent complete: 36.8%; Average loss: 2.6097\n",
      "Iteration: 1473; Percent complete: 36.8%; Average loss: 3.1328\n",
      "Iteration: 1474; Percent complete: 36.9%; Average loss: 2.9006\n",
      "Iteration: 1475; Percent complete: 36.9%; Average loss: 2.8612\n",
      "Iteration: 1476; Percent complete: 36.9%; Average loss: 2.6463\n",
      "Iteration: 1477; Percent complete: 36.9%; Average loss: 2.9708\n",
      "Iteration: 1478; Percent complete: 37.0%; Average loss: 2.7330\n",
      "Iteration: 1479; Percent complete: 37.0%; Average loss: 3.0245\n",
      "Iteration: 1480; Percent complete: 37.0%; Average loss: 2.7215\n",
      "Iteration: 1481; Percent complete: 37.0%; Average loss: 2.7324\n",
      "Iteration: 1482; Percent complete: 37.0%; Average loss: 2.9200\n",
      "Iteration: 1483; Percent complete: 37.1%; Average loss: 2.7928\n",
      "Iteration: 1484; Percent complete: 37.1%; Average loss: 2.9445\n",
      "Iteration: 1485; Percent complete: 37.1%; Average loss: 2.8810\n",
      "Iteration: 1486; Percent complete: 37.1%; Average loss: 2.9392\n",
      "Iteration: 1487; Percent complete: 37.2%; Average loss: 2.7447\n",
      "Iteration: 1488; Percent complete: 37.2%; Average loss: 2.8709\n",
      "Iteration: 1489; Percent complete: 37.2%; Average loss: 2.7517\n",
      "Iteration: 1490; Percent complete: 37.2%; Average loss: 2.8953\n",
      "Iteration: 1491; Percent complete: 37.3%; Average loss: 2.6024\n",
      "Iteration: 1492; Percent complete: 37.3%; Average loss: 2.6128\n",
      "Iteration: 1493; Percent complete: 37.3%; Average loss: 2.9989\n",
      "Iteration: 1494; Percent complete: 37.4%; Average loss: 2.7294\n",
      "Iteration: 1495; Percent complete: 37.4%; Average loss: 2.8714\n",
      "Iteration: 1496; Percent complete: 37.4%; Average loss: 2.8767\n",
      "Iteration: 1497; Percent complete: 37.4%; Average loss: 2.7685\n",
      "Iteration: 1498; Percent complete: 37.5%; Average loss: 2.9929\n",
      "Iteration: 1499; Percent complete: 37.5%; Average loss: 3.0091\n",
      "Iteration: 1500; Percent complete: 37.5%; Average loss: 3.0218\n",
      "Iteration: 1501; Percent complete: 37.5%; Average loss: 2.9784\n",
      "Iteration: 1502; Percent complete: 37.5%; Average loss: 2.8275\n",
      "Iteration: 1503; Percent complete: 37.6%; Average loss: 2.7112\n",
      "Iteration: 1504; Percent complete: 37.6%; Average loss: 2.7146\n",
      "Iteration: 1505; Percent complete: 37.6%; Average loss: 2.8901\n",
      "Iteration: 1506; Percent complete: 37.6%; Average loss: 2.8787\n",
      "Iteration: 1507; Percent complete: 37.7%; Average loss: 2.8247\n",
      "Iteration: 1508; Percent complete: 37.7%; Average loss: 2.6641\n",
      "Iteration: 1509; Percent complete: 37.7%; Average loss: 2.8322\n",
      "Iteration: 1510; Percent complete: 37.8%; Average loss: 2.7462\n",
      "Iteration: 1511; Percent complete: 37.8%; Average loss: 2.6682\n",
      "Iteration: 1512; Percent complete: 37.8%; Average loss: 2.6226\n",
      "Iteration: 1513; Percent complete: 37.8%; Average loss: 2.6673\n",
      "Iteration: 1514; Percent complete: 37.9%; Average loss: 2.6076\n",
      "Iteration: 1515; Percent complete: 37.9%; Average loss: 3.0407\n",
      "Iteration: 1516; Percent complete: 37.9%; Average loss: 2.7877\n",
      "Iteration: 1517; Percent complete: 37.9%; Average loss: 2.6847\n",
      "Iteration: 1518; Percent complete: 38.0%; Average loss: 2.8105\n",
      "Iteration: 1519; Percent complete: 38.0%; Average loss: 2.7320\n",
      "Iteration: 1520; Percent complete: 38.0%; Average loss: 2.7592\n",
      "Iteration: 1521; Percent complete: 38.0%; Average loss: 2.9523\n",
      "Iteration: 1522; Percent complete: 38.0%; Average loss: 2.5449\n",
      "Iteration: 1523; Percent complete: 38.1%; Average loss: 3.0426\n",
      "Iteration: 1524; Percent complete: 38.1%; Average loss: 2.8518\n",
      "Iteration: 1525; Percent complete: 38.1%; Average loss: 2.5730\n",
      "Iteration: 1526; Percent complete: 38.1%; Average loss: 2.8397\n",
      "Iteration: 1527; Percent complete: 38.2%; Average loss: 2.7004\n",
      "Iteration: 1528; Percent complete: 38.2%; Average loss: 2.8220\n",
      "Iteration: 1529; Percent complete: 38.2%; Average loss: 3.0280\n",
      "Iteration: 1530; Percent complete: 38.2%; Average loss: 2.8156\n",
      "Iteration: 1531; Percent complete: 38.3%; Average loss: 2.8223\n",
      "Iteration: 1532; Percent complete: 38.3%; Average loss: 2.8711\n",
      "Iteration: 1533; Percent complete: 38.3%; Average loss: 2.7381\n",
      "Iteration: 1534; Percent complete: 38.4%; Average loss: 2.5236\n",
      "Iteration: 1535; Percent complete: 38.4%; Average loss: 2.8743\n",
      "Iteration: 1536; Percent complete: 38.4%; Average loss: 2.5814\n",
      "Iteration: 1537; Percent complete: 38.4%; Average loss: 2.8549\n",
      "Iteration: 1538; Percent complete: 38.5%; Average loss: 2.5995\n",
      "Iteration: 1539; Percent complete: 38.5%; Average loss: 2.7148\n",
      "Iteration: 1540; Percent complete: 38.5%; Average loss: 3.0185\n",
      "Iteration: 1541; Percent complete: 38.5%; Average loss: 2.8048\n",
      "Iteration: 1542; Percent complete: 38.6%; Average loss: 2.8730\n",
      "Iteration: 1543; Percent complete: 38.6%; Average loss: 2.7847\n",
      "Iteration: 1544; Percent complete: 38.6%; Average loss: 2.9425\n",
      "Iteration: 1545; Percent complete: 38.6%; Average loss: 2.8261\n",
      "Iteration: 1546; Percent complete: 38.6%; Average loss: 2.3494\n",
      "Iteration: 1547; Percent complete: 38.7%; Average loss: 3.0021\n",
      "Iteration: 1548; Percent complete: 38.7%; Average loss: 2.8383\n",
      "Iteration: 1549; Percent complete: 38.7%; Average loss: 2.7701\n",
      "Iteration: 1550; Percent complete: 38.8%; Average loss: 2.9387\n",
      "Iteration: 1551; Percent complete: 38.8%; Average loss: 2.7766\n",
      "Iteration: 1552; Percent complete: 38.8%; Average loss: 2.7429\n",
      "Iteration: 1553; Percent complete: 38.8%; Average loss: 2.7050\n",
      "Iteration: 1554; Percent complete: 38.9%; Average loss: 2.9247\n",
      "Iteration: 1555; Percent complete: 38.9%; Average loss: 2.9545\n",
      "Iteration: 1556; Percent complete: 38.9%; Average loss: 2.9594\n",
      "Iteration: 1557; Percent complete: 38.9%; Average loss: 2.7194\n",
      "Iteration: 1558; Percent complete: 39.0%; Average loss: 2.8906\n",
      "Iteration: 1559; Percent complete: 39.0%; Average loss: 2.9310\n",
      "Iteration: 1560; Percent complete: 39.0%; Average loss: 2.8036\n",
      "Iteration: 1561; Percent complete: 39.0%; Average loss: 2.7756\n",
      "Iteration: 1562; Percent complete: 39.1%; Average loss: 2.7002\n",
      "Iteration: 1563; Percent complete: 39.1%; Average loss: 2.9083\n",
      "Iteration: 1564; Percent complete: 39.1%; Average loss: 2.6236\n",
      "Iteration: 1565; Percent complete: 39.1%; Average loss: 2.8415\n",
      "Iteration: 1566; Percent complete: 39.1%; Average loss: 2.8351\n",
      "Iteration: 1567; Percent complete: 39.2%; Average loss: 2.8118\n",
      "Iteration: 1568; Percent complete: 39.2%; Average loss: 2.5873\n",
      "Iteration: 1569; Percent complete: 39.2%; Average loss: 2.7698\n",
      "Iteration: 1570; Percent complete: 39.2%; Average loss: 2.7573\n",
      "Iteration: 1571; Percent complete: 39.3%; Average loss: 2.8354\n",
      "Iteration: 1572; Percent complete: 39.3%; Average loss: 2.8622\n",
      "Iteration: 1573; Percent complete: 39.3%; Average loss: 2.8650\n",
      "Iteration: 1574; Percent complete: 39.4%; Average loss: 2.6577\n",
      "Iteration: 1575; Percent complete: 39.4%; Average loss: 2.7381\n",
      "Iteration: 1576; Percent complete: 39.4%; Average loss: 2.8132\n",
      "Iteration: 1577; Percent complete: 39.4%; Average loss: 2.7798\n",
      "Iteration: 1578; Percent complete: 39.5%; Average loss: 3.0088\n",
      "Iteration: 1579; Percent complete: 39.5%; Average loss: 2.6580\n",
      "Iteration: 1580; Percent complete: 39.5%; Average loss: 2.7170\n",
      "Iteration: 1581; Percent complete: 39.5%; Average loss: 2.8376\n",
      "Iteration: 1582; Percent complete: 39.6%; Average loss: 2.7704\n",
      "Iteration: 1583; Percent complete: 39.6%; Average loss: 2.7826\n",
      "Iteration: 1584; Percent complete: 39.6%; Average loss: 2.7263\n",
      "Iteration: 1585; Percent complete: 39.6%; Average loss: 2.7150\n",
      "Iteration: 1586; Percent complete: 39.6%; Average loss: 2.5938\n",
      "Iteration: 1587; Percent complete: 39.7%; Average loss: 2.7812\n",
      "Iteration: 1588; Percent complete: 39.7%; Average loss: 2.8639\n",
      "Iteration: 1589; Percent complete: 39.7%; Average loss: 2.6553\n",
      "Iteration: 1590; Percent complete: 39.8%; Average loss: 2.9161\n",
      "Iteration: 1591; Percent complete: 39.8%; Average loss: 2.7899\n",
      "Iteration: 1592; Percent complete: 39.8%; Average loss: 2.8427\n",
      "Iteration: 1593; Percent complete: 39.8%; Average loss: 2.7548\n",
      "Iteration: 1594; Percent complete: 39.9%; Average loss: 2.9698\n",
      "Iteration: 1595; Percent complete: 39.9%; Average loss: 2.8149\n",
      "Iteration: 1596; Percent complete: 39.9%; Average loss: 2.8885\n",
      "Iteration: 1597; Percent complete: 39.9%; Average loss: 2.5581\n",
      "Iteration: 1598; Percent complete: 40.0%; Average loss: 2.8276\n",
      "Iteration: 1599; Percent complete: 40.0%; Average loss: 2.6138\n",
      "Iteration: 1600; Percent complete: 40.0%; Average loss: 2.8045\n",
      "Iteration: 1601; Percent complete: 40.0%; Average loss: 2.7419\n",
      "Iteration: 1602; Percent complete: 40.1%; Average loss: 2.7908\n",
      "Iteration: 1603; Percent complete: 40.1%; Average loss: 2.9995\n",
      "Iteration: 1604; Percent complete: 40.1%; Average loss: 2.4693\n",
      "Iteration: 1605; Percent complete: 40.1%; Average loss: 2.6311\n",
      "Iteration: 1606; Percent complete: 40.2%; Average loss: 2.6358\n",
      "Iteration: 1607; Percent complete: 40.2%; Average loss: 2.8861\n",
      "Iteration: 1608; Percent complete: 40.2%; Average loss: 2.5274\n",
      "Iteration: 1609; Percent complete: 40.2%; Average loss: 2.7762\n",
      "Iteration: 1610; Percent complete: 40.2%; Average loss: 2.8592\n",
      "Iteration: 1611; Percent complete: 40.3%; Average loss: 2.7914\n",
      "Iteration: 1612; Percent complete: 40.3%; Average loss: 2.8798\n",
      "Iteration: 1613; Percent complete: 40.3%; Average loss: 2.6745\n",
      "Iteration: 1614; Percent complete: 40.4%; Average loss: 2.7229\n",
      "Iteration: 1615; Percent complete: 40.4%; Average loss: 2.7566\n",
      "Iteration: 1616; Percent complete: 40.4%; Average loss: 2.8150\n",
      "Iteration: 1617; Percent complete: 40.4%; Average loss: 2.6885\n",
      "Iteration: 1618; Percent complete: 40.5%; Average loss: 2.7994\n",
      "Iteration: 1619; Percent complete: 40.5%; Average loss: 3.0894\n",
      "Iteration: 1620; Percent complete: 40.5%; Average loss: 2.6950\n",
      "Iteration: 1621; Percent complete: 40.5%; Average loss: 2.7924\n",
      "Iteration: 1622; Percent complete: 40.6%; Average loss: 2.8700\n",
      "Iteration: 1623; Percent complete: 40.6%; Average loss: 2.7384\n",
      "Iteration: 1624; Percent complete: 40.6%; Average loss: 2.7819\n",
      "Iteration: 1625; Percent complete: 40.6%; Average loss: 2.9843\n",
      "Iteration: 1626; Percent complete: 40.6%; Average loss: 2.8586\n",
      "Iteration: 1627; Percent complete: 40.7%; Average loss: 2.8253\n",
      "Iteration: 1628; Percent complete: 40.7%; Average loss: 2.5935\n",
      "Iteration: 1629; Percent complete: 40.7%; Average loss: 2.7915\n",
      "Iteration: 1630; Percent complete: 40.8%; Average loss: 2.7987\n",
      "Iteration: 1631; Percent complete: 40.8%; Average loss: 2.9629\n",
      "Iteration: 1632; Percent complete: 40.8%; Average loss: 2.5791\n",
      "Iteration: 1633; Percent complete: 40.8%; Average loss: 2.8408\n",
      "Iteration: 1634; Percent complete: 40.8%; Average loss: 2.8693\n",
      "Iteration: 1635; Percent complete: 40.9%; Average loss: 2.8833\n",
      "Iteration: 1636; Percent complete: 40.9%; Average loss: 2.8862\n",
      "Iteration: 1637; Percent complete: 40.9%; Average loss: 2.8859\n",
      "Iteration: 1638; Percent complete: 40.9%; Average loss: 2.9824\n",
      "Iteration: 1639; Percent complete: 41.0%; Average loss: 2.7877\n",
      "Iteration: 1640; Percent complete: 41.0%; Average loss: 2.6557\n",
      "Iteration: 1641; Percent complete: 41.0%; Average loss: 2.7797\n",
      "Iteration: 1642; Percent complete: 41.0%; Average loss: 2.6716\n",
      "Iteration: 1643; Percent complete: 41.1%; Average loss: 2.5483\n",
      "Iteration: 1644; Percent complete: 41.1%; Average loss: 2.6061\n",
      "Iteration: 1645; Percent complete: 41.1%; Average loss: 2.6228\n",
      "Iteration: 1646; Percent complete: 41.1%; Average loss: 2.8351\n",
      "Iteration: 1647; Percent complete: 41.2%; Average loss: 2.6287\n",
      "Iteration: 1648; Percent complete: 41.2%; Average loss: 2.6281\n",
      "Iteration: 1649; Percent complete: 41.2%; Average loss: 2.6232\n",
      "Iteration: 1650; Percent complete: 41.2%; Average loss: 2.6815\n",
      "Iteration: 1651; Percent complete: 41.3%; Average loss: 2.7739\n",
      "Iteration: 1652; Percent complete: 41.3%; Average loss: 2.5249\n",
      "Iteration: 1653; Percent complete: 41.3%; Average loss: 2.7720\n",
      "Iteration: 1654; Percent complete: 41.3%; Average loss: 2.8579\n",
      "Iteration: 1655; Percent complete: 41.4%; Average loss: 2.5982\n",
      "Iteration: 1656; Percent complete: 41.4%; Average loss: 2.7262\n",
      "Iteration: 1657; Percent complete: 41.4%; Average loss: 2.8327\n",
      "Iteration: 1658; Percent complete: 41.4%; Average loss: 2.6882\n",
      "Iteration: 1659; Percent complete: 41.5%; Average loss: 2.5966\n",
      "Iteration: 1660; Percent complete: 41.5%; Average loss: 2.6457\n",
      "Iteration: 1661; Percent complete: 41.5%; Average loss: 2.6764\n",
      "Iteration: 1662; Percent complete: 41.5%; Average loss: 2.6302\n",
      "Iteration: 1663; Percent complete: 41.6%; Average loss: 2.6078\n",
      "Iteration: 1664; Percent complete: 41.6%; Average loss: 2.6865\n",
      "Iteration: 1665; Percent complete: 41.6%; Average loss: 3.0203\n",
      "Iteration: 1666; Percent complete: 41.6%; Average loss: 2.7456\n",
      "Iteration: 1667; Percent complete: 41.7%; Average loss: 2.8654\n",
      "Iteration: 1668; Percent complete: 41.7%; Average loss: 2.6685\n",
      "Iteration: 1669; Percent complete: 41.7%; Average loss: 2.7456\n",
      "Iteration: 1670; Percent complete: 41.8%; Average loss: 2.7371\n",
      "Iteration: 1671; Percent complete: 41.8%; Average loss: 2.8147\n",
      "Iteration: 1672; Percent complete: 41.8%; Average loss: 2.7942\n",
      "Iteration: 1673; Percent complete: 41.8%; Average loss: 2.5366\n",
      "Iteration: 1674; Percent complete: 41.9%; Average loss: 2.8216\n",
      "Iteration: 1675; Percent complete: 41.9%; Average loss: 2.7667\n",
      "Iteration: 1676; Percent complete: 41.9%; Average loss: 2.7226\n",
      "Iteration: 1677; Percent complete: 41.9%; Average loss: 2.6286\n",
      "Iteration: 1678; Percent complete: 41.9%; Average loss: 2.6900\n",
      "Iteration: 1679; Percent complete: 42.0%; Average loss: 2.7730\n",
      "Iteration: 1680; Percent complete: 42.0%; Average loss: 2.5974\n",
      "Iteration: 1681; Percent complete: 42.0%; Average loss: 2.8317\n",
      "Iteration: 1682; Percent complete: 42.0%; Average loss: 2.7276\n",
      "Iteration: 1683; Percent complete: 42.1%; Average loss: 2.6603\n",
      "Iteration: 1684; Percent complete: 42.1%; Average loss: 2.8859\n",
      "Iteration: 1685; Percent complete: 42.1%; Average loss: 2.6127\n",
      "Iteration: 1686; Percent complete: 42.1%; Average loss: 2.6965\n",
      "Iteration: 1687; Percent complete: 42.2%; Average loss: 2.6553\n",
      "Iteration: 1688; Percent complete: 42.2%; Average loss: 2.8029\n",
      "Iteration: 1689; Percent complete: 42.2%; Average loss: 2.6186\n",
      "Iteration: 1690; Percent complete: 42.2%; Average loss: 2.7933\n",
      "Iteration: 1691; Percent complete: 42.3%; Average loss: 2.8577\n",
      "Iteration: 1692; Percent complete: 42.3%; Average loss: 2.7721\n",
      "Iteration: 1693; Percent complete: 42.3%; Average loss: 2.7104\n",
      "Iteration: 1694; Percent complete: 42.4%; Average loss: 2.6700\n",
      "Iteration: 1695; Percent complete: 42.4%; Average loss: 2.8044\n",
      "Iteration: 1696; Percent complete: 42.4%; Average loss: 2.8364\n",
      "Iteration: 1697; Percent complete: 42.4%; Average loss: 2.6960\n",
      "Iteration: 1698; Percent complete: 42.4%; Average loss: 2.8055\n",
      "Iteration: 1699; Percent complete: 42.5%; Average loss: 3.0211\n",
      "Iteration: 1700; Percent complete: 42.5%; Average loss: 2.7755\n",
      "Iteration: 1701; Percent complete: 42.5%; Average loss: 2.7066\n",
      "Iteration: 1702; Percent complete: 42.5%; Average loss: 2.8745\n",
      "Iteration: 1703; Percent complete: 42.6%; Average loss: 2.6473\n",
      "Iteration: 1704; Percent complete: 42.6%; Average loss: 2.4821\n",
      "Iteration: 1705; Percent complete: 42.6%; Average loss: 2.5774\n",
      "Iteration: 1706; Percent complete: 42.6%; Average loss: 3.0268\n",
      "Iteration: 1707; Percent complete: 42.7%; Average loss: 2.9956\n",
      "Iteration: 1708; Percent complete: 42.7%; Average loss: 2.8310\n",
      "Iteration: 1709; Percent complete: 42.7%; Average loss: 2.7993\n",
      "Iteration: 1710; Percent complete: 42.8%; Average loss: 2.9469\n",
      "Iteration: 1711; Percent complete: 42.8%; Average loss: 2.7974\n",
      "Iteration: 1712; Percent complete: 42.8%; Average loss: 2.8201\n",
      "Iteration: 1713; Percent complete: 42.8%; Average loss: 2.7917\n",
      "Iteration: 1714; Percent complete: 42.9%; Average loss: 2.8028\n",
      "Iteration: 1715; Percent complete: 42.9%; Average loss: 2.5976\n",
      "Iteration: 1716; Percent complete: 42.9%; Average loss: 2.6769\n",
      "Iteration: 1717; Percent complete: 42.9%; Average loss: 2.8444\n",
      "Iteration: 1718; Percent complete: 43.0%; Average loss: 2.7329\n",
      "Iteration: 1719; Percent complete: 43.0%; Average loss: 2.5016\n",
      "Iteration: 1720; Percent complete: 43.0%; Average loss: 2.8825\n",
      "Iteration: 1721; Percent complete: 43.0%; Average loss: 2.6494\n",
      "Iteration: 1722; Percent complete: 43.0%; Average loss: 2.9552\n",
      "Iteration: 1723; Percent complete: 43.1%; Average loss: 2.7143\n",
      "Iteration: 1724; Percent complete: 43.1%; Average loss: 2.6059\n",
      "Iteration: 1725; Percent complete: 43.1%; Average loss: 2.6762\n",
      "Iteration: 1726; Percent complete: 43.1%; Average loss: 2.6370\n",
      "Iteration: 1727; Percent complete: 43.2%; Average loss: 2.8091\n",
      "Iteration: 1728; Percent complete: 43.2%; Average loss: 2.7632\n",
      "Iteration: 1729; Percent complete: 43.2%; Average loss: 2.7232\n",
      "Iteration: 1730; Percent complete: 43.2%; Average loss: 2.7563\n",
      "Iteration: 1731; Percent complete: 43.3%; Average loss: 3.0400\n",
      "Iteration: 1732; Percent complete: 43.3%; Average loss: 2.6914\n",
      "Iteration: 1733; Percent complete: 43.3%; Average loss: 2.8011\n",
      "Iteration: 1734; Percent complete: 43.4%; Average loss: 2.7753\n",
      "Iteration: 1735; Percent complete: 43.4%; Average loss: 2.7781\n",
      "Iteration: 1736; Percent complete: 43.4%; Average loss: 2.4497\n",
      "Iteration: 1737; Percent complete: 43.4%; Average loss: 2.8389\n",
      "Iteration: 1738; Percent complete: 43.5%; Average loss: 2.7759\n",
      "Iteration: 1739; Percent complete: 43.5%; Average loss: 2.8391\n",
      "Iteration: 1740; Percent complete: 43.5%; Average loss: 2.7197\n",
      "Iteration: 1741; Percent complete: 43.5%; Average loss: 2.7367\n",
      "Iteration: 1742; Percent complete: 43.5%; Average loss: 2.8815\n",
      "Iteration: 1743; Percent complete: 43.6%; Average loss: 2.7150\n",
      "Iteration: 1744; Percent complete: 43.6%; Average loss: 2.7960\n",
      "Iteration: 1745; Percent complete: 43.6%; Average loss: 2.7142\n",
      "Iteration: 1746; Percent complete: 43.6%; Average loss: 2.7712\n",
      "Iteration: 1747; Percent complete: 43.7%; Average loss: 2.5813\n",
      "Iteration: 1748; Percent complete: 43.7%; Average loss: 2.7158\n",
      "Iteration: 1749; Percent complete: 43.7%; Average loss: 2.8538\n",
      "Iteration: 1750; Percent complete: 43.8%; Average loss: 2.5473\n",
      "Iteration: 1751; Percent complete: 43.8%; Average loss: 2.8492\n",
      "Iteration: 1752; Percent complete: 43.8%; Average loss: 2.6327\n",
      "Iteration: 1753; Percent complete: 43.8%; Average loss: 2.5912\n",
      "Iteration: 1754; Percent complete: 43.9%; Average loss: 2.7761\n",
      "Iteration: 1755; Percent complete: 43.9%; Average loss: 2.6927\n",
      "Iteration: 1756; Percent complete: 43.9%; Average loss: 2.4613\n",
      "Iteration: 1757; Percent complete: 43.9%; Average loss: 2.7993\n",
      "Iteration: 1758; Percent complete: 44.0%; Average loss: 2.5982\n",
      "Iteration: 1759; Percent complete: 44.0%; Average loss: 2.7689\n",
      "Iteration: 1760; Percent complete: 44.0%; Average loss: 2.6308\n",
      "Iteration: 1761; Percent complete: 44.0%; Average loss: 2.7872\n",
      "Iteration: 1762; Percent complete: 44.0%; Average loss: 2.6594\n",
      "Iteration: 1763; Percent complete: 44.1%; Average loss: 2.5544\n",
      "Iteration: 1764; Percent complete: 44.1%; Average loss: 2.5778\n",
      "Iteration: 1765; Percent complete: 44.1%; Average loss: 2.7526\n",
      "Iteration: 1766; Percent complete: 44.1%; Average loss: 2.8857\n",
      "Iteration: 1767; Percent complete: 44.2%; Average loss: 2.7849\n",
      "Iteration: 1768; Percent complete: 44.2%; Average loss: 2.6507\n",
      "Iteration: 1769; Percent complete: 44.2%; Average loss: 2.5224\n",
      "Iteration: 1770; Percent complete: 44.2%; Average loss: 2.8471\n",
      "Iteration: 1771; Percent complete: 44.3%; Average loss: 2.6011\n",
      "Iteration: 1772; Percent complete: 44.3%; Average loss: 2.4849\n",
      "Iteration: 1773; Percent complete: 44.3%; Average loss: 2.6805\n",
      "Iteration: 1774; Percent complete: 44.4%; Average loss: 2.5579\n",
      "Iteration: 1775; Percent complete: 44.4%; Average loss: 2.7671\n",
      "Iteration: 1776; Percent complete: 44.4%; Average loss: 2.7432\n",
      "Iteration: 1777; Percent complete: 44.4%; Average loss: 2.6688\n",
      "Iteration: 1778; Percent complete: 44.5%; Average loss: 2.6945\n",
      "Iteration: 1779; Percent complete: 44.5%; Average loss: 2.6357\n",
      "Iteration: 1780; Percent complete: 44.5%; Average loss: 2.5199\n",
      "Iteration: 1781; Percent complete: 44.5%; Average loss: 2.5687\n",
      "Iteration: 1782; Percent complete: 44.5%; Average loss: 2.6426\n",
      "Iteration: 1783; Percent complete: 44.6%; Average loss: 2.8663\n",
      "Iteration: 1784; Percent complete: 44.6%; Average loss: 2.6015\n",
      "Iteration: 1785; Percent complete: 44.6%; Average loss: 2.6886\n",
      "Iteration: 1786; Percent complete: 44.6%; Average loss: 2.5908\n",
      "Iteration: 1787; Percent complete: 44.7%; Average loss: 2.5446\n",
      "Iteration: 1788; Percent complete: 44.7%; Average loss: 2.9645\n",
      "Iteration: 1789; Percent complete: 44.7%; Average loss: 2.7397\n",
      "Iteration: 1790; Percent complete: 44.8%; Average loss: 2.7686\n",
      "Iteration: 1791; Percent complete: 44.8%; Average loss: 2.6517\n",
      "Iteration: 1792; Percent complete: 44.8%; Average loss: 2.7891\n",
      "Iteration: 1793; Percent complete: 44.8%; Average loss: 2.8383\n",
      "Iteration: 1794; Percent complete: 44.9%; Average loss: 2.7886\n",
      "Iteration: 1795; Percent complete: 44.9%; Average loss: 2.4835\n",
      "Iteration: 1796; Percent complete: 44.9%; Average loss: 2.5502\n",
      "Iteration: 1797; Percent complete: 44.9%; Average loss: 2.8098\n",
      "Iteration: 1798; Percent complete: 45.0%; Average loss: 2.7306\n",
      "Iteration: 1799; Percent complete: 45.0%; Average loss: 3.0011\n",
      "Iteration: 1800; Percent complete: 45.0%; Average loss: 2.7181\n",
      "Iteration: 1801; Percent complete: 45.0%; Average loss: 2.7774\n",
      "Iteration: 1802; Percent complete: 45.1%; Average loss: 2.5475\n",
      "Iteration: 1803; Percent complete: 45.1%; Average loss: 2.5120\n",
      "Iteration: 1804; Percent complete: 45.1%; Average loss: 2.8563\n",
      "Iteration: 1805; Percent complete: 45.1%; Average loss: 2.6777\n",
      "Iteration: 1806; Percent complete: 45.1%; Average loss: 2.6881\n",
      "Iteration: 1807; Percent complete: 45.2%; Average loss: 2.6613\n",
      "Iteration: 1808; Percent complete: 45.2%; Average loss: 2.6845\n",
      "Iteration: 1809; Percent complete: 45.2%; Average loss: 2.7440\n",
      "Iteration: 1810; Percent complete: 45.2%; Average loss: 2.6963\n",
      "Iteration: 1811; Percent complete: 45.3%; Average loss: 2.7699\n",
      "Iteration: 1812; Percent complete: 45.3%; Average loss: 2.6403\n",
      "Iteration: 1813; Percent complete: 45.3%; Average loss: 2.5420\n",
      "Iteration: 1814; Percent complete: 45.4%; Average loss: 2.6596\n",
      "Iteration: 1815; Percent complete: 45.4%; Average loss: 2.6019\n",
      "Iteration: 1816; Percent complete: 45.4%; Average loss: 2.7018\n",
      "Iteration: 1817; Percent complete: 45.4%; Average loss: 2.5934\n",
      "Iteration: 1818; Percent complete: 45.5%; Average loss: 2.5838\n",
      "Iteration: 1819; Percent complete: 45.5%; Average loss: 2.5550\n",
      "Iteration: 1820; Percent complete: 45.5%; Average loss: 2.6009\n",
      "Iteration: 1821; Percent complete: 45.5%; Average loss: 2.7437\n",
      "Iteration: 1822; Percent complete: 45.6%; Average loss: 2.5568\n",
      "Iteration: 1823; Percent complete: 45.6%; Average loss: 2.9713\n",
      "Iteration: 1824; Percent complete: 45.6%; Average loss: 2.5937\n",
      "Iteration: 1825; Percent complete: 45.6%; Average loss: 2.5770\n",
      "Iteration: 1826; Percent complete: 45.6%; Average loss: 2.7359\n",
      "Iteration: 1827; Percent complete: 45.7%; Average loss: 2.7666\n",
      "Iteration: 1828; Percent complete: 45.7%; Average loss: 2.8756\n",
      "Iteration: 1829; Percent complete: 45.7%; Average loss: 2.5851\n",
      "Iteration: 1830; Percent complete: 45.8%; Average loss: 2.7530\n",
      "Iteration: 1831; Percent complete: 45.8%; Average loss: 2.8163\n",
      "Iteration: 1832; Percent complete: 45.8%; Average loss: 2.7504\n",
      "Iteration: 1833; Percent complete: 45.8%; Average loss: 2.5053\n",
      "Iteration: 1834; Percent complete: 45.9%; Average loss: 2.7180\n",
      "Iteration: 1835; Percent complete: 45.9%; Average loss: 2.6992\n",
      "Iteration: 1836; Percent complete: 45.9%; Average loss: 2.5763\n",
      "Iteration: 1837; Percent complete: 45.9%; Average loss: 2.6086\n",
      "Iteration: 1838; Percent complete: 46.0%; Average loss: 2.7049\n",
      "Iteration: 1839; Percent complete: 46.0%; Average loss: 2.7731\n",
      "Iteration: 1840; Percent complete: 46.0%; Average loss: 3.0048\n",
      "Iteration: 1841; Percent complete: 46.0%; Average loss: 2.6695\n",
      "Iteration: 1842; Percent complete: 46.1%; Average loss: 2.7062\n",
      "Iteration: 1843; Percent complete: 46.1%; Average loss: 2.6709\n",
      "Iteration: 1844; Percent complete: 46.1%; Average loss: 2.7083\n",
      "Iteration: 1845; Percent complete: 46.1%; Average loss: 2.5441\n",
      "Iteration: 1846; Percent complete: 46.2%; Average loss: 2.6073\n",
      "Iteration: 1847; Percent complete: 46.2%; Average loss: 2.7376\n",
      "Iteration: 1848; Percent complete: 46.2%; Average loss: 2.6248\n",
      "Iteration: 1849; Percent complete: 46.2%; Average loss: 2.8709\n",
      "Iteration: 1850; Percent complete: 46.2%; Average loss: 2.6800\n",
      "Iteration: 1851; Percent complete: 46.3%; Average loss: 2.5821\n",
      "Iteration: 1852; Percent complete: 46.3%; Average loss: 3.0263\n",
      "Iteration: 1853; Percent complete: 46.3%; Average loss: 2.9737\n",
      "Iteration: 1854; Percent complete: 46.4%; Average loss: 2.6045\n",
      "Iteration: 1855; Percent complete: 46.4%; Average loss: 2.7406\n",
      "Iteration: 1856; Percent complete: 46.4%; Average loss: 2.6005\n",
      "Iteration: 1857; Percent complete: 46.4%; Average loss: 2.5635\n",
      "Iteration: 1858; Percent complete: 46.5%; Average loss: 2.6795\n",
      "Iteration: 1859; Percent complete: 46.5%; Average loss: 2.6413\n",
      "Iteration: 1860; Percent complete: 46.5%; Average loss: 2.6521\n",
      "Iteration: 1861; Percent complete: 46.5%; Average loss: 2.8044\n",
      "Iteration: 1862; Percent complete: 46.6%; Average loss: 2.9316\n",
      "Iteration: 1863; Percent complete: 46.6%; Average loss: 2.6791\n",
      "Iteration: 1864; Percent complete: 46.6%; Average loss: 2.6747\n",
      "Iteration: 1865; Percent complete: 46.6%; Average loss: 2.5987\n",
      "Iteration: 1866; Percent complete: 46.7%; Average loss: 3.0071\n",
      "Iteration: 1867; Percent complete: 46.7%; Average loss: 2.7218\n",
      "Iteration: 1868; Percent complete: 46.7%; Average loss: 2.6711\n",
      "Iteration: 1869; Percent complete: 46.7%; Average loss: 2.7416\n",
      "Iteration: 1870; Percent complete: 46.8%; Average loss: 2.7418\n",
      "Iteration: 1871; Percent complete: 46.8%; Average loss: 2.5232\n",
      "Iteration: 1872; Percent complete: 46.8%; Average loss: 2.7824\n",
      "Iteration: 1873; Percent complete: 46.8%; Average loss: 2.7207\n",
      "Iteration: 1874; Percent complete: 46.9%; Average loss: 3.1237\n",
      "Iteration: 1875; Percent complete: 46.9%; Average loss: 2.5019\n",
      "Iteration: 1876; Percent complete: 46.9%; Average loss: 2.7407\n",
      "Iteration: 1877; Percent complete: 46.9%; Average loss: 2.3742\n",
      "Iteration: 1878; Percent complete: 46.9%; Average loss: 2.6404\n",
      "Iteration: 1879; Percent complete: 47.0%; Average loss: 2.7132\n",
      "Iteration: 1880; Percent complete: 47.0%; Average loss: 2.6183\n",
      "Iteration: 1881; Percent complete: 47.0%; Average loss: 2.6809\n",
      "Iteration: 1882; Percent complete: 47.0%; Average loss: 2.8625\n",
      "Iteration: 1883; Percent complete: 47.1%; Average loss: 2.7500\n",
      "Iteration: 1884; Percent complete: 47.1%; Average loss: 2.4920\n",
      "Iteration: 1885; Percent complete: 47.1%; Average loss: 2.6060\n",
      "Iteration: 1886; Percent complete: 47.1%; Average loss: 2.5442\n",
      "Iteration: 1887; Percent complete: 47.2%; Average loss: 2.6589\n",
      "Iteration: 1888; Percent complete: 47.2%; Average loss: 2.6763\n",
      "Iteration: 1889; Percent complete: 47.2%; Average loss: 2.4783\n",
      "Iteration: 1890; Percent complete: 47.2%; Average loss: 2.7054\n",
      "Iteration: 1891; Percent complete: 47.3%; Average loss: 2.6667\n",
      "Iteration: 1892; Percent complete: 47.3%; Average loss: 2.7993\n",
      "Iteration: 1893; Percent complete: 47.3%; Average loss: 2.6427\n",
      "Iteration: 1894; Percent complete: 47.3%; Average loss: 2.4500\n",
      "Iteration: 1895; Percent complete: 47.4%; Average loss: 2.4369\n",
      "Iteration: 1896; Percent complete: 47.4%; Average loss: 2.6965\n",
      "Iteration: 1897; Percent complete: 47.4%; Average loss: 2.7597\n",
      "Iteration: 1898; Percent complete: 47.4%; Average loss: 2.6258\n",
      "Iteration: 1899; Percent complete: 47.5%; Average loss: 2.6544\n",
      "Iteration: 1900; Percent complete: 47.5%; Average loss: 2.5412\n",
      "Iteration: 1901; Percent complete: 47.5%; Average loss: 2.6888\n",
      "Iteration: 1902; Percent complete: 47.5%; Average loss: 2.6314\n",
      "Iteration: 1903; Percent complete: 47.6%; Average loss: 2.8330\n",
      "Iteration: 1904; Percent complete: 47.6%; Average loss: 2.5659\n",
      "Iteration: 1905; Percent complete: 47.6%; Average loss: 2.7198\n",
      "Iteration: 1906; Percent complete: 47.6%; Average loss: 2.5350\n",
      "Iteration: 1907; Percent complete: 47.7%; Average loss: 2.6700\n",
      "Iteration: 1908; Percent complete: 47.7%; Average loss: 2.8054\n",
      "Iteration: 1909; Percent complete: 47.7%; Average loss: 2.7490\n",
      "Iteration: 1910; Percent complete: 47.8%; Average loss: 2.8153\n",
      "Iteration: 1911; Percent complete: 47.8%; Average loss: 2.7020\n",
      "Iteration: 1912; Percent complete: 47.8%; Average loss: 2.6576\n",
      "Iteration: 1913; Percent complete: 47.8%; Average loss: 2.6633\n",
      "Iteration: 1914; Percent complete: 47.9%; Average loss: 2.5891\n",
      "Iteration: 1915; Percent complete: 47.9%; Average loss: 2.5168\n",
      "Iteration: 1916; Percent complete: 47.9%; Average loss: 2.5700\n",
      "Iteration: 1917; Percent complete: 47.9%; Average loss: 2.7917\n",
      "Iteration: 1918; Percent complete: 47.9%; Average loss: 2.7566\n",
      "Iteration: 1919; Percent complete: 48.0%; Average loss: 2.6101\n",
      "Iteration: 1920; Percent complete: 48.0%; Average loss: 2.5919\n",
      "Iteration: 1921; Percent complete: 48.0%; Average loss: 2.3901\n",
      "Iteration: 1922; Percent complete: 48.0%; Average loss: 2.6636\n",
      "Iteration: 1923; Percent complete: 48.1%; Average loss: 2.7836\n",
      "Iteration: 1924; Percent complete: 48.1%; Average loss: 2.6675\n",
      "Iteration: 1925; Percent complete: 48.1%; Average loss: 2.5597\n",
      "Iteration: 1926; Percent complete: 48.1%; Average loss: 2.5227\n",
      "Iteration: 1927; Percent complete: 48.2%; Average loss: 2.7196\n",
      "Iteration: 1928; Percent complete: 48.2%; Average loss: 2.7448\n",
      "Iteration: 1929; Percent complete: 48.2%; Average loss: 2.7792\n",
      "Iteration: 1930; Percent complete: 48.2%; Average loss: 2.8196\n",
      "Iteration: 1931; Percent complete: 48.3%; Average loss: 2.8682\n",
      "Iteration: 1932; Percent complete: 48.3%; Average loss: 2.8829\n",
      "Iteration: 1933; Percent complete: 48.3%; Average loss: 2.8898\n",
      "Iteration: 1934; Percent complete: 48.4%; Average loss: 2.8875\n",
      "Iteration: 1935; Percent complete: 48.4%; Average loss: 2.4889\n",
      "Iteration: 1936; Percent complete: 48.4%; Average loss: 2.8439\n",
      "Iteration: 1937; Percent complete: 48.4%; Average loss: 2.8847\n",
      "Iteration: 1938; Percent complete: 48.4%; Average loss: 2.5680\n",
      "Iteration: 1939; Percent complete: 48.5%; Average loss: 2.4302\n",
      "Iteration: 1940; Percent complete: 48.5%; Average loss: 2.9420\n",
      "Iteration: 1941; Percent complete: 48.5%; Average loss: 2.6529\n",
      "Iteration: 1942; Percent complete: 48.5%; Average loss: 2.7099\n",
      "Iteration: 1943; Percent complete: 48.6%; Average loss: 2.6803\n",
      "Iteration: 1944; Percent complete: 48.6%; Average loss: 2.6241\n",
      "Iteration: 1945; Percent complete: 48.6%; Average loss: 2.5774\n",
      "Iteration: 1946; Percent complete: 48.6%; Average loss: 2.7770\n",
      "Iteration: 1947; Percent complete: 48.7%; Average loss: 2.9553\n",
      "Iteration: 1948; Percent complete: 48.7%; Average loss: 2.6933\n",
      "Iteration: 1949; Percent complete: 48.7%; Average loss: 2.6906\n",
      "Iteration: 1950; Percent complete: 48.8%; Average loss: 2.5023\n",
      "Iteration: 1951; Percent complete: 48.8%; Average loss: 2.8016\n",
      "Iteration: 1952; Percent complete: 48.8%; Average loss: 2.7809\n",
      "Iteration: 1953; Percent complete: 48.8%; Average loss: 2.6809\n",
      "Iteration: 1954; Percent complete: 48.9%; Average loss: 2.5198\n",
      "Iteration: 1955; Percent complete: 48.9%; Average loss: 2.6090\n",
      "Iteration: 1956; Percent complete: 48.9%; Average loss: 2.7248\n",
      "Iteration: 1957; Percent complete: 48.9%; Average loss: 2.6687\n",
      "Iteration: 1958; Percent complete: 48.9%; Average loss: 2.6701\n",
      "Iteration: 1959; Percent complete: 49.0%; Average loss: 2.5384\n",
      "Iteration: 1960; Percent complete: 49.0%; Average loss: 2.7450\n",
      "Iteration: 1961; Percent complete: 49.0%; Average loss: 2.3807\n",
      "Iteration: 1962; Percent complete: 49.0%; Average loss: 2.7091\n",
      "Iteration: 1963; Percent complete: 49.1%; Average loss: 2.4553\n",
      "Iteration: 1964; Percent complete: 49.1%; Average loss: 2.5190\n",
      "Iteration: 1965; Percent complete: 49.1%; Average loss: 2.8773\n",
      "Iteration: 1966; Percent complete: 49.1%; Average loss: 2.5100\n",
      "Iteration: 1967; Percent complete: 49.2%; Average loss: 2.5035\n",
      "Iteration: 1968; Percent complete: 49.2%; Average loss: 2.5312\n",
      "Iteration: 1969; Percent complete: 49.2%; Average loss: 2.7436\n",
      "Iteration: 1970; Percent complete: 49.2%; Average loss: 2.6694\n",
      "Iteration: 1971; Percent complete: 49.3%; Average loss: 2.7635\n",
      "Iteration: 1972; Percent complete: 49.3%; Average loss: 2.6685\n",
      "Iteration: 1973; Percent complete: 49.3%; Average loss: 2.7701\n",
      "Iteration: 1974; Percent complete: 49.4%; Average loss: 2.5970\n",
      "Iteration: 1975; Percent complete: 49.4%; Average loss: 2.7205\n",
      "Iteration: 1976; Percent complete: 49.4%; Average loss: 2.5903\n",
      "Iteration: 1977; Percent complete: 49.4%; Average loss: 2.6897\n",
      "Iteration: 1978; Percent complete: 49.5%; Average loss: 2.5648\n",
      "Iteration: 1979; Percent complete: 49.5%; Average loss: 2.8233\n",
      "Iteration: 1980; Percent complete: 49.5%; Average loss: 2.8060\n",
      "Iteration: 1981; Percent complete: 49.5%; Average loss: 2.5236\n",
      "Iteration: 1982; Percent complete: 49.5%; Average loss: 2.9207\n",
      "Iteration: 1983; Percent complete: 49.6%; Average loss: 2.6262\n",
      "Iteration: 1984; Percent complete: 49.6%; Average loss: 2.6362\n",
      "Iteration: 1985; Percent complete: 49.6%; Average loss: 2.8805\n",
      "Iteration: 1986; Percent complete: 49.6%; Average loss: 2.7542\n",
      "Iteration: 1987; Percent complete: 49.7%; Average loss: 2.6419\n",
      "Iteration: 1988; Percent complete: 49.7%; Average loss: 2.6764\n",
      "Iteration: 1989; Percent complete: 49.7%; Average loss: 2.7960\n",
      "Iteration: 1990; Percent complete: 49.8%; Average loss: 2.5851\n",
      "Iteration: 1991; Percent complete: 49.8%; Average loss: 2.8608\n",
      "Iteration: 1992; Percent complete: 49.8%; Average loss: 2.7548\n",
      "Iteration: 1993; Percent complete: 49.8%; Average loss: 3.0036\n",
      "Iteration: 1994; Percent complete: 49.9%; Average loss: 2.7997\n",
      "Iteration: 1995; Percent complete: 49.9%; Average loss: 2.5657\n",
      "Iteration: 1996; Percent complete: 49.9%; Average loss: 2.8608\n",
      "Iteration: 1997; Percent complete: 49.9%; Average loss: 2.4671\n",
      "Iteration: 1998; Percent complete: 50.0%; Average loss: 2.5049\n",
      "Iteration: 1999; Percent complete: 50.0%; Average loss: 2.6884\n",
      "Iteration: 2000; Percent complete: 50.0%; Average loss: 2.5505\n",
      "Iteration: 2001; Percent complete: 50.0%; Average loss: 2.8239\n",
      "Iteration: 2002; Percent complete: 50.0%; Average loss: 2.8584\n",
      "Iteration: 2003; Percent complete: 50.1%; Average loss: 2.4935\n",
      "Iteration: 2004; Percent complete: 50.1%; Average loss: 2.5844\n",
      "Iteration: 2005; Percent complete: 50.1%; Average loss: 2.5482\n",
      "Iteration: 2006; Percent complete: 50.1%; Average loss: 2.3925\n",
      "Iteration: 2007; Percent complete: 50.2%; Average loss: 2.5495\n",
      "Iteration: 2008; Percent complete: 50.2%; Average loss: 2.7126\n",
      "Iteration: 2009; Percent complete: 50.2%; Average loss: 2.7529\n",
      "Iteration: 2010; Percent complete: 50.2%; Average loss: 2.7393\n",
      "Iteration: 2011; Percent complete: 50.3%; Average loss: 2.5690\n",
      "Iteration: 2012; Percent complete: 50.3%; Average loss: 2.5347\n",
      "Iteration: 2013; Percent complete: 50.3%; Average loss: 2.6515\n",
      "Iteration: 2014; Percent complete: 50.3%; Average loss: 2.5112\n",
      "Iteration: 2015; Percent complete: 50.4%; Average loss: 2.6701\n",
      "Iteration: 2016; Percent complete: 50.4%; Average loss: 2.5768\n",
      "Iteration: 2017; Percent complete: 50.4%; Average loss: 2.5886\n",
      "Iteration: 2018; Percent complete: 50.4%; Average loss: 2.5661\n",
      "Iteration: 2019; Percent complete: 50.5%; Average loss: 2.5762\n",
      "Iteration: 2020; Percent complete: 50.5%; Average loss: 2.9255\n",
      "Iteration: 2021; Percent complete: 50.5%; Average loss: 2.5115\n",
      "Iteration: 2022; Percent complete: 50.5%; Average loss: 2.6108\n",
      "Iteration: 2023; Percent complete: 50.6%; Average loss: 2.5535\n",
      "Iteration: 2024; Percent complete: 50.6%; Average loss: 2.4686\n",
      "Iteration: 2025; Percent complete: 50.6%; Average loss: 2.6122\n",
      "Iteration: 2026; Percent complete: 50.6%; Average loss: 2.7386\n",
      "Iteration: 2027; Percent complete: 50.7%; Average loss: 2.7660\n",
      "Iteration: 2028; Percent complete: 50.7%; Average loss: 2.9665\n",
      "Iteration: 2029; Percent complete: 50.7%; Average loss: 2.6637\n",
      "Iteration: 2030; Percent complete: 50.7%; Average loss: 2.6448\n",
      "Iteration: 2031; Percent complete: 50.8%; Average loss: 2.2861\n",
      "Iteration: 2032; Percent complete: 50.8%; Average loss: 2.4724\n",
      "Iteration: 2033; Percent complete: 50.8%; Average loss: 2.5884\n",
      "Iteration: 2034; Percent complete: 50.8%; Average loss: 2.5586\n",
      "Iteration: 2035; Percent complete: 50.9%; Average loss: 2.7234\n",
      "Iteration: 2036; Percent complete: 50.9%; Average loss: 2.7759\n",
      "Iteration: 2037; Percent complete: 50.9%; Average loss: 2.4652\n",
      "Iteration: 2038; Percent complete: 50.9%; Average loss: 2.6171\n",
      "Iteration: 2039; Percent complete: 51.0%; Average loss: 2.4473\n",
      "Iteration: 2040; Percent complete: 51.0%; Average loss: 2.6890\n",
      "Iteration: 2041; Percent complete: 51.0%; Average loss: 2.7556\n",
      "Iteration: 2042; Percent complete: 51.0%; Average loss: 2.7416\n",
      "Iteration: 2043; Percent complete: 51.1%; Average loss: 2.4851\n",
      "Iteration: 2044; Percent complete: 51.1%; Average loss: 2.8933\n",
      "Iteration: 2045; Percent complete: 51.1%; Average loss: 2.7089\n",
      "Iteration: 2046; Percent complete: 51.1%; Average loss: 2.6747\n",
      "Iteration: 2047; Percent complete: 51.2%; Average loss: 2.7584\n",
      "Iteration: 2048; Percent complete: 51.2%; Average loss: 2.7473\n",
      "Iteration: 2049; Percent complete: 51.2%; Average loss: 2.7501\n",
      "Iteration: 2050; Percent complete: 51.2%; Average loss: 2.8148\n",
      "Iteration: 2051; Percent complete: 51.3%; Average loss: 2.6436\n",
      "Iteration: 2052; Percent complete: 51.3%; Average loss: 2.7221\n",
      "Iteration: 2053; Percent complete: 51.3%; Average loss: 2.4663\n",
      "Iteration: 2054; Percent complete: 51.3%; Average loss: 2.6384\n",
      "Iteration: 2055; Percent complete: 51.4%; Average loss: 2.3123\n",
      "Iteration: 2056; Percent complete: 51.4%; Average loss: 2.7388\n",
      "Iteration: 2057; Percent complete: 51.4%; Average loss: 2.6502\n",
      "Iteration: 2058; Percent complete: 51.4%; Average loss: 2.6573\n",
      "Iteration: 2059; Percent complete: 51.5%; Average loss: 2.4708\n",
      "Iteration: 2060; Percent complete: 51.5%; Average loss: 2.5609\n",
      "Iteration: 2061; Percent complete: 51.5%; Average loss: 2.6085\n",
      "Iteration: 2062; Percent complete: 51.5%; Average loss: 2.7226\n",
      "Iteration: 2063; Percent complete: 51.6%; Average loss: 2.5693\n",
      "Iteration: 2064; Percent complete: 51.6%; Average loss: 2.5146\n",
      "Iteration: 2065; Percent complete: 51.6%; Average loss: 2.5851\n",
      "Iteration: 2066; Percent complete: 51.6%; Average loss: 2.7656\n",
      "Iteration: 2067; Percent complete: 51.7%; Average loss: 2.5529\n",
      "Iteration: 2068; Percent complete: 51.7%; Average loss: 2.9287\n",
      "Iteration: 2069; Percent complete: 51.7%; Average loss: 2.3648\n",
      "Iteration: 2070; Percent complete: 51.7%; Average loss: 2.4458\n",
      "Iteration: 2071; Percent complete: 51.8%; Average loss: 2.6415\n",
      "Iteration: 2072; Percent complete: 51.8%; Average loss: 2.5018\n",
      "Iteration: 2073; Percent complete: 51.8%; Average loss: 2.6921\n",
      "Iteration: 2074; Percent complete: 51.8%; Average loss: 2.5239\n",
      "Iteration: 2075; Percent complete: 51.9%; Average loss: 2.8748\n",
      "Iteration: 2076; Percent complete: 51.9%; Average loss: 2.5876\n",
      "Iteration: 2077; Percent complete: 51.9%; Average loss: 2.6353\n",
      "Iteration: 2078; Percent complete: 51.9%; Average loss: 2.6922\n",
      "Iteration: 2079; Percent complete: 52.0%; Average loss: 2.6985\n",
      "Iteration: 2080; Percent complete: 52.0%; Average loss: 2.6632\n",
      "Iteration: 2081; Percent complete: 52.0%; Average loss: 2.8100\n",
      "Iteration: 2082; Percent complete: 52.0%; Average loss: 2.5113\n",
      "Iteration: 2083; Percent complete: 52.1%; Average loss: 2.6981\n",
      "Iteration: 2084; Percent complete: 52.1%; Average loss: 2.6490\n",
      "Iteration: 2085; Percent complete: 52.1%; Average loss: 2.6575\n",
      "Iteration: 2086; Percent complete: 52.1%; Average loss: 2.8735\n",
      "Iteration: 2087; Percent complete: 52.2%; Average loss: 2.5546\n",
      "Iteration: 2088; Percent complete: 52.2%; Average loss: 2.5417\n",
      "Iteration: 2089; Percent complete: 52.2%; Average loss: 2.4151\n",
      "Iteration: 2090; Percent complete: 52.2%; Average loss: 2.6332\n",
      "Iteration: 2091; Percent complete: 52.3%; Average loss: 2.4291\n",
      "Iteration: 2092; Percent complete: 52.3%; Average loss: 2.8208\n",
      "Iteration: 2093; Percent complete: 52.3%; Average loss: 2.6048\n",
      "Iteration: 2094; Percent complete: 52.3%; Average loss: 2.7021\n",
      "Iteration: 2095; Percent complete: 52.4%; Average loss: 2.5237\n",
      "Iteration: 2096; Percent complete: 52.4%; Average loss: 2.8807\n",
      "Iteration: 2097; Percent complete: 52.4%; Average loss: 2.8591\n",
      "Iteration: 2098; Percent complete: 52.4%; Average loss: 2.5951\n",
      "Iteration: 2099; Percent complete: 52.5%; Average loss: 2.6059\n",
      "Iteration: 2100; Percent complete: 52.5%; Average loss: 2.8321\n",
      "Iteration: 2101; Percent complete: 52.5%; Average loss: 2.7382\n",
      "Iteration: 2102; Percent complete: 52.5%; Average loss: 2.6146\n",
      "Iteration: 2103; Percent complete: 52.6%; Average loss: 2.6235\n",
      "Iteration: 2104; Percent complete: 52.6%; Average loss: 2.6981\n",
      "Iteration: 2105; Percent complete: 52.6%; Average loss: 2.6039\n",
      "Iteration: 2106; Percent complete: 52.6%; Average loss: 2.7141\n",
      "Iteration: 2107; Percent complete: 52.7%; Average loss: 3.0433\n",
      "Iteration: 2108; Percent complete: 52.7%; Average loss: 2.3491\n",
      "Iteration: 2109; Percent complete: 52.7%; Average loss: 2.4996\n",
      "Iteration: 2110; Percent complete: 52.8%; Average loss: 2.6029\n",
      "Iteration: 2111; Percent complete: 52.8%; Average loss: 2.6700\n",
      "Iteration: 2112; Percent complete: 52.8%; Average loss: 2.7506\n",
      "Iteration: 2113; Percent complete: 52.8%; Average loss: 2.2543\n",
      "Iteration: 2114; Percent complete: 52.8%; Average loss: 2.7666\n",
      "Iteration: 2115; Percent complete: 52.9%; Average loss: 2.5344\n",
      "Iteration: 2116; Percent complete: 52.9%; Average loss: 2.5008\n",
      "Iteration: 2117; Percent complete: 52.9%; Average loss: 2.7372\n",
      "Iteration: 2118; Percent complete: 52.9%; Average loss: 2.8403\n",
      "Iteration: 2119; Percent complete: 53.0%; Average loss: 2.6187\n",
      "Iteration: 2120; Percent complete: 53.0%; Average loss: 2.5994\n",
      "Iteration: 2121; Percent complete: 53.0%; Average loss: 2.8535\n",
      "Iteration: 2122; Percent complete: 53.0%; Average loss: 2.6604\n",
      "Iteration: 2123; Percent complete: 53.1%; Average loss: 2.7555\n",
      "Iteration: 2124; Percent complete: 53.1%; Average loss: 2.7141\n",
      "Iteration: 2125; Percent complete: 53.1%; Average loss: 2.7214\n",
      "Iteration: 2126; Percent complete: 53.1%; Average loss: 2.6554\n",
      "Iteration: 2127; Percent complete: 53.2%; Average loss: 2.3257\n",
      "Iteration: 2128; Percent complete: 53.2%; Average loss: 2.3380\n",
      "Iteration: 2129; Percent complete: 53.2%; Average loss: 2.8103\n",
      "Iteration: 2130; Percent complete: 53.2%; Average loss: 2.6277\n",
      "Iteration: 2131; Percent complete: 53.3%; Average loss: 2.7603\n",
      "Iteration: 2132; Percent complete: 53.3%; Average loss: 2.6114\n",
      "Iteration: 2133; Percent complete: 53.3%; Average loss: 2.5965\n",
      "Iteration: 2134; Percent complete: 53.3%; Average loss: 2.5149\n",
      "Iteration: 2135; Percent complete: 53.4%; Average loss: 2.2797\n",
      "Iteration: 2136; Percent complete: 53.4%; Average loss: 2.5604\n",
      "Iteration: 2137; Percent complete: 53.4%; Average loss: 2.6831\n",
      "Iteration: 2138; Percent complete: 53.4%; Average loss: 2.5540\n",
      "Iteration: 2139; Percent complete: 53.5%; Average loss: 2.4809\n",
      "Iteration: 2140; Percent complete: 53.5%; Average loss: 2.6943\n",
      "Iteration: 2141; Percent complete: 53.5%; Average loss: 2.6714\n",
      "Iteration: 2142; Percent complete: 53.5%; Average loss: 2.5650\n",
      "Iteration: 2143; Percent complete: 53.6%; Average loss: 2.4474\n",
      "Iteration: 2144; Percent complete: 53.6%; Average loss: 2.3348\n",
      "Iteration: 2145; Percent complete: 53.6%; Average loss: 2.5369\n",
      "Iteration: 2146; Percent complete: 53.6%; Average loss: 2.6513\n",
      "Iteration: 2147; Percent complete: 53.7%; Average loss: 2.5380\n",
      "Iteration: 2148; Percent complete: 53.7%; Average loss: 2.7708\n",
      "Iteration: 2149; Percent complete: 53.7%; Average loss: 2.5875\n",
      "Iteration: 2150; Percent complete: 53.8%; Average loss: 2.6892\n",
      "Iteration: 2151; Percent complete: 53.8%; Average loss: 2.5672\n",
      "Iteration: 2152; Percent complete: 53.8%; Average loss: 2.7707\n",
      "Iteration: 2153; Percent complete: 53.8%; Average loss: 2.7126\n",
      "Iteration: 2154; Percent complete: 53.8%; Average loss: 2.6340\n",
      "Iteration: 2155; Percent complete: 53.9%; Average loss: 2.5172\n",
      "Iteration: 2156; Percent complete: 53.9%; Average loss: 2.8353\n",
      "Iteration: 2157; Percent complete: 53.9%; Average loss: 2.3066\n",
      "Iteration: 2158; Percent complete: 53.9%; Average loss: 2.5316\n",
      "Iteration: 2159; Percent complete: 54.0%; Average loss: 2.5675\n",
      "Iteration: 2160; Percent complete: 54.0%; Average loss: 2.4676\n",
      "Iteration: 2161; Percent complete: 54.0%; Average loss: 2.5152\n",
      "Iteration: 2162; Percent complete: 54.0%; Average loss: 2.4814\n",
      "Iteration: 2163; Percent complete: 54.1%; Average loss: 2.8801\n",
      "Iteration: 2164; Percent complete: 54.1%; Average loss: 2.4908\n",
      "Iteration: 2165; Percent complete: 54.1%; Average loss: 2.2169\n",
      "Iteration: 2166; Percent complete: 54.1%; Average loss: 2.5760\n",
      "Iteration: 2167; Percent complete: 54.2%; Average loss: 2.6505\n",
      "Iteration: 2168; Percent complete: 54.2%; Average loss: 2.7880\n",
      "Iteration: 2169; Percent complete: 54.2%; Average loss: 2.6775\n",
      "Iteration: 2170; Percent complete: 54.2%; Average loss: 2.4932\n",
      "Iteration: 2171; Percent complete: 54.3%; Average loss: 2.5063\n",
      "Iteration: 2172; Percent complete: 54.3%; Average loss: 2.7739\n",
      "Iteration: 2173; Percent complete: 54.3%; Average loss: 2.3547\n",
      "Iteration: 2174; Percent complete: 54.4%; Average loss: 2.6166\n",
      "Iteration: 2175; Percent complete: 54.4%; Average loss: 2.7067\n",
      "Iteration: 2176; Percent complete: 54.4%; Average loss: 2.5671\n",
      "Iteration: 2177; Percent complete: 54.4%; Average loss: 2.7685\n",
      "Iteration: 2178; Percent complete: 54.4%; Average loss: 2.6182\n",
      "Iteration: 2179; Percent complete: 54.5%; Average loss: 2.6129\n",
      "Iteration: 2180; Percent complete: 54.5%; Average loss: 2.5225\n",
      "Iteration: 2181; Percent complete: 54.5%; Average loss: 2.5692\n",
      "Iteration: 2182; Percent complete: 54.5%; Average loss: 2.5920\n",
      "Iteration: 2183; Percent complete: 54.6%; Average loss: 2.6499\n",
      "Iteration: 2184; Percent complete: 54.6%; Average loss: 2.8116\n",
      "Iteration: 2185; Percent complete: 54.6%; Average loss: 2.5005\n",
      "Iteration: 2186; Percent complete: 54.6%; Average loss: 2.6235\n",
      "Iteration: 2187; Percent complete: 54.7%; Average loss: 2.5246\n",
      "Iteration: 2188; Percent complete: 54.7%; Average loss: 2.5631\n",
      "Iteration: 2189; Percent complete: 54.7%; Average loss: 2.6281\n",
      "Iteration: 2190; Percent complete: 54.8%; Average loss: 2.6601\n",
      "Iteration: 2191; Percent complete: 54.8%; Average loss: 2.6091\n",
      "Iteration: 2192; Percent complete: 54.8%; Average loss: 2.7380\n",
      "Iteration: 2193; Percent complete: 54.8%; Average loss: 2.7284\n",
      "Iteration: 2194; Percent complete: 54.9%; Average loss: 2.7436\n",
      "Iteration: 2195; Percent complete: 54.9%; Average loss: 2.3590\n",
      "Iteration: 2196; Percent complete: 54.9%; Average loss: 2.3614\n",
      "Iteration: 2197; Percent complete: 54.9%; Average loss: 2.6084\n",
      "Iteration: 2198; Percent complete: 54.9%; Average loss: 2.4968\n",
      "Iteration: 2199; Percent complete: 55.0%; Average loss: 2.8611\n",
      "Iteration: 2200; Percent complete: 55.0%; Average loss: 2.4462\n",
      "Iteration: 2201; Percent complete: 55.0%; Average loss: 2.6373\n",
      "Iteration: 2202; Percent complete: 55.0%; Average loss: 2.6784\n",
      "Iteration: 2203; Percent complete: 55.1%; Average loss: 2.5262\n",
      "Iteration: 2204; Percent complete: 55.1%; Average loss: 2.5509\n",
      "Iteration: 2205; Percent complete: 55.1%; Average loss: 2.6848\n",
      "Iteration: 2206; Percent complete: 55.1%; Average loss: 2.5624\n",
      "Iteration: 2207; Percent complete: 55.2%; Average loss: 2.6975\n",
      "Iteration: 2208; Percent complete: 55.2%; Average loss: 2.2100\n",
      "Iteration: 2209; Percent complete: 55.2%; Average loss: 2.5681\n",
      "Iteration: 2210; Percent complete: 55.2%; Average loss: 2.6272\n",
      "Iteration: 2211; Percent complete: 55.3%; Average loss: 2.7856\n",
      "Iteration: 2212; Percent complete: 55.3%; Average loss: 2.7710\n",
      "Iteration: 2213; Percent complete: 55.3%; Average loss: 2.6956\n",
      "Iteration: 2214; Percent complete: 55.4%; Average loss: 2.5873\n",
      "Iteration: 2215; Percent complete: 55.4%; Average loss: 2.7161\n",
      "Iteration: 2216; Percent complete: 55.4%; Average loss: 2.6408\n",
      "Iteration: 2217; Percent complete: 55.4%; Average loss: 2.4338\n",
      "Iteration: 2218; Percent complete: 55.5%; Average loss: 2.4774\n",
      "Iteration: 2219; Percent complete: 55.5%; Average loss: 2.5274\n",
      "Iteration: 2220; Percent complete: 55.5%; Average loss: 2.6815\n",
      "Iteration: 2221; Percent complete: 55.5%; Average loss: 2.5960\n",
      "Iteration: 2222; Percent complete: 55.5%; Average loss: 2.3310\n",
      "Iteration: 2223; Percent complete: 55.6%; Average loss: 2.6045\n",
      "Iteration: 2224; Percent complete: 55.6%; Average loss: 2.6658\n",
      "Iteration: 2225; Percent complete: 55.6%; Average loss: 2.5643\n",
      "Iteration: 2226; Percent complete: 55.6%; Average loss: 2.6633\n",
      "Iteration: 2227; Percent complete: 55.7%; Average loss: 2.7595\n",
      "Iteration: 2228; Percent complete: 55.7%; Average loss: 2.4921\n",
      "Iteration: 2229; Percent complete: 55.7%; Average loss: 2.7477\n",
      "Iteration: 2230; Percent complete: 55.8%; Average loss: 2.6765\n",
      "Iteration: 2231; Percent complete: 55.8%; Average loss: 2.7515\n",
      "Iteration: 2232; Percent complete: 55.8%; Average loss: 2.4968\n",
      "Iteration: 2233; Percent complete: 55.8%; Average loss: 2.6735\n",
      "Iteration: 2234; Percent complete: 55.9%; Average loss: 2.6110\n",
      "Iteration: 2235; Percent complete: 55.9%; Average loss: 2.5652\n",
      "Iteration: 2236; Percent complete: 55.9%; Average loss: 2.7525\n",
      "Iteration: 2237; Percent complete: 55.9%; Average loss: 2.4741\n",
      "Iteration: 2238; Percent complete: 56.0%; Average loss: 2.5637\n",
      "Iteration: 2239; Percent complete: 56.0%; Average loss: 2.5101\n",
      "Iteration: 2240; Percent complete: 56.0%; Average loss: 2.6305\n",
      "Iteration: 2241; Percent complete: 56.0%; Average loss: 2.5996\n",
      "Iteration: 2242; Percent complete: 56.0%; Average loss: 2.8310\n",
      "Iteration: 2243; Percent complete: 56.1%; Average loss: 2.7408\n",
      "Iteration: 2244; Percent complete: 56.1%; Average loss: 2.6540\n",
      "Iteration: 2245; Percent complete: 56.1%; Average loss: 2.1170\n",
      "Iteration: 2246; Percent complete: 56.1%; Average loss: 2.4910\n",
      "Iteration: 2247; Percent complete: 56.2%; Average loss: 2.6194\n",
      "Iteration: 2248; Percent complete: 56.2%; Average loss: 2.4581\n",
      "Iteration: 2249; Percent complete: 56.2%; Average loss: 2.7115\n",
      "Iteration: 2250; Percent complete: 56.2%; Average loss: 2.8656\n",
      "Iteration: 2251; Percent complete: 56.3%; Average loss: 2.5780\n",
      "Iteration: 2252; Percent complete: 56.3%; Average loss: 2.6377\n",
      "Iteration: 2253; Percent complete: 56.3%; Average loss: 2.4302\n",
      "Iteration: 2254; Percent complete: 56.4%; Average loss: 2.7672\n",
      "Iteration: 2255; Percent complete: 56.4%; Average loss: 2.5732\n",
      "Iteration: 2256; Percent complete: 56.4%; Average loss: 2.6685\n",
      "Iteration: 2257; Percent complete: 56.4%; Average loss: 2.4991\n",
      "Iteration: 2258; Percent complete: 56.5%; Average loss: 2.7417\n",
      "Iteration: 2259; Percent complete: 56.5%; Average loss: 2.2727\n",
      "Iteration: 2260; Percent complete: 56.5%; Average loss: 2.5177\n",
      "Iteration: 2261; Percent complete: 56.5%; Average loss: 2.4964\n",
      "Iteration: 2262; Percent complete: 56.5%; Average loss: 2.6846\n",
      "Iteration: 2263; Percent complete: 56.6%; Average loss: 2.5871\n",
      "Iteration: 2264; Percent complete: 56.6%; Average loss: 2.5958\n",
      "Iteration: 2265; Percent complete: 56.6%; Average loss: 2.7100\n",
      "Iteration: 2266; Percent complete: 56.6%; Average loss: 2.5629\n",
      "Iteration: 2267; Percent complete: 56.7%; Average loss: 2.3855\n",
      "Iteration: 2268; Percent complete: 56.7%; Average loss: 2.5249\n",
      "Iteration: 2269; Percent complete: 56.7%; Average loss: 2.6230\n",
      "Iteration: 2270; Percent complete: 56.8%; Average loss: 2.9467\n",
      "Iteration: 2271; Percent complete: 56.8%; Average loss: 2.8196\n",
      "Iteration: 2272; Percent complete: 56.8%; Average loss: 2.4479\n",
      "Iteration: 2273; Percent complete: 56.8%; Average loss: 2.6922\n",
      "Iteration: 2274; Percent complete: 56.9%; Average loss: 2.5364\n",
      "Iteration: 2275; Percent complete: 56.9%; Average loss: 2.4541\n",
      "Iteration: 2276; Percent complete: 56.9%; Average loss: 2.5701\n",
      "Iteration: 2277; Percent complete: 56.9%; Average loss: 2.4369\n",
      "Iteration: 2278; Percent complete: 57.0%; Average loss: 2.6319\n",
      "Iteration: 2279; Percent complete: 57.0%; Average loss: 2.5969\n",
      "Iteration: 2280; Percent complete: 57.0%; Average loss: 2.5527\n",
      "Iteration: 2281; Percent complete: 57.0%; Average loss: 2.9086\n",
      "Iteration: 2282; Percent complete: 57.0%; Average loss: 2.5347\n",
      "Iteration: 2283; Percent complete: 57.1%; Average loss: 2.7056\n",
      "Iteration: 2284; Percent complete: 57.1%; Average loss: 2.5984\n",
      "Iteration: 2285; Percent complete: 57.1%; Average loss: 2.6109\n",
      "Iteration: 2286; Percent complete: 57.1%; Average loss: 2.3572\n",
      "Iteration: 2287; Percent complete: 57.2%; Average loss: 2.5408\n",
      "Iteration: 2288; Percent complete: 57.2%; Average loss: 2.5053\n",
      "Iteration: 2289; Percent complete: 57.2%; Average loss: 2.5423\n",
      "Iteration: 2290; Percent complete: 57.2%; Average loss: 2.7383\n",
      "Iteration: 2291; Percent complete: 57.3%; Average loss: 2.7686\n",
      "Iteration: 2292; Percent complete: 57.3%; Average loss: 2.4245\n",
      "Iteration: 2293; Percent complete: 57.3%; Average loss: 2.6385\n",
      "Iteration: 2294; Percent complete: 57.4%; Average loss: 2.3799\n",
      "Iteration: 2295; Percent complete: 57.4%; Average loss: 2.6459\n",
      "Iteration: 2296; Percent complete: 57.4%; Average loss: 2.5633\n",
      "Iteration: 2297; Percent complete: 57.4%; Average loss: 2.5120\n",
      "Iteration: 2298; Percent complete: 57.5%; Average loss: 2.7321\n",
      "Iteration: 2299; Percent complete: 57.5%; Average loss: 2.6086\n",
      "Iteration: 2300; Percent complete: 57.5%; Average loss: 2.3978\n",
      "Iteration: 2301; Percent complete: 57.5%; Average loss: 2.4484\n",
      "Iteration: 2302; Percent complete: 57.6%; Average loss: 2.5926\n",
      "Iteration: 2303; Percent complete: 57.6%; Average loss: 2.8138\n",
      "Iteration: 2304; Percent complete: 57.6%; Average loss: 2.7697\n",
      "Iteration: 2305; Percent complete: 57.6%; Average loss: 2.8405\n",
      "Iteration: 2306; Percent complete: 57.6%; Average loss: 2.7304\n",
      "Iteration: 2307; Percent complete: 57.7%; Average loss: 2.6236\n",
      "Iteration: 2308; Percent complete: 57.7%; Average loss: 2.4268\n",
      "Iteration: 2309; Percent complete: 57.7%; Average loss: 2.6028\n",
      "Iteration: 2310; Percent complete: 57.8%; Average loss: 2.5927\n",
      "Iteration: 2311; Percent complete: 57.8%; Average loss: 2.5852\n",
      "Iteration: 2312; Percent complete: 57.8%; Average loss: 2.4320\n",
      "Iteration: 2313; Percent complete: 57.8%; Average loss: 2.4397\n",
      "Iteration: 2314; Percent complete: 57.9%; Average loss: 2.4795\n",
      "Iteration: 2315; Percent complete: 57.9%; Average loss: 2.7641\n",
      "Iteration: 2316; Percent complete: 57.9%; Average loss: 2.7612\n",
      "Iteration: 2317; Percent complete: 57.9%; Average loss: 2.5093\n",
      "Iteration: 2318; Percent complete: 58.0%; Average loss: 2.7882\n",
      "Iteration: 2319; Percent complete: 58.0%; Average loss: 2.5124\n",
      "Iteration: 2320; Percent complete: 58.0%; Average loss: 2.6084\n",
      "Iteration: 2321; Percent complete: 58.0%; Average loss: 2.8835\n",
      "Iteration: 2322; Percent complete: 58.1%; Average loss: 2.6323\n",
      "Iteration: 2323; Percent complete: 58.1%; Average loss: 2.4426\n",
      "Iteration: 2324; Percent complete: 58.1%; Average loss: 2.7655\n",
      "Iteration: 2325; Percent complete: 58.1%; Average loss: 2.5441\n",
      "Iteration: 2326; Percent complete: 58.1%; Average loss: 2.6448\n",
      "Iteration: 2327; Percent complete: 58.2%; Average loss: 2.6514\n",
      "Iteration: 2328; Percent complete: 58.2%; Average loss: 2.4676\n",
      "Iteration: 2329; Percent complete: 58.2%; Average loss: 2.7537\n",
      "Iteration: 2330; Percent complete: 58.2%; Average loss: 2.7492\n",
      "Iteration: 2331; Percent complete: 58.3%; Average loss: 2.6476\n",
      "Iteration: 2332; Percent complete: 58.3%; Average loss: 2.6691\n",
      "Iteration: 2333; Percent complete: 58.3%; Average loss: 2.4453\n",
      "Iteration: 2334; Percent complete: 58.4%; Average loss: 2.6840\n",
      "Iteration: 2335; Percent complete: 58.4%; Average loss: 2.6949\n",
      "Iteration: 2336; Percent complete: 58.4%; Average loss: 2.7962\n",
      "Iteration: 2337; Percent complete: 58.4%; Average loss: 2.6729\n",
      "Iteration: 2338; Percent complete: 58.5%; Average loss: 2.4767\n",
      "Iteration: 2339; Percent complete: 58.5%; Average loss: 2.3620\n",
      "Iteration: 2340; Percent complete: 58.5%; Average loss: 2.4774\n",
      "Iteration: 2341; Percent complete: 58.5%; Average loss: 2.4571\n",
      "Iteration: 2342; Percent complete: 58.6%; Average loss: 2.5181\n",
      "Iteration: 2343; Percent complete: 58.6%; Average loss: 2.5890\n",
      "Iteration: 2344; Percent complete: 58.6%; Average loss: 2.5169\n",
      "Iteration: 2345; Percent complete: 58.6%; Average loss: 2.5243\n",
      "Iteration: 2346; Percent complete: 58.7%; Average loss: 2.6290\n",
      "Iteration: 2347; Percent complete: 58.7%; Average loss: 2.5447\n",
      "Iteration: 2348; Percent complete: 58.7%; Average loss: 2.3966\n",
      "Iteration: 2349; Percent complete: 58.7%; Average loss: 2.3504\n",
      "Iteration: 2350; Percent complete: 58.8%; Average loss: 2.6798\n",
      "Iteration: 2351; Percent complete: 58.8%; Average loss: 2.4533\n",
      "Iteration: 2352; Percent complete: 58.8%; Average loss: 2.5503\n",
      "Iteration: 2353; Percent complete: 58.8%; Average loss: 2.6872\n",
      "Iteration: 2354; Percent complete: 58.9%; Average loss: 2.5045\n",
      "Iteration: 2355; Percent complete: 58.9%; Average loss: 2.5244\n",
      "Iteration: 2356; Percent complete: 58.9%; Average loss: 2.6601\n",
      "Iteration: 2357; Percent complete: 58.9%; Average loss: 2.5411\n",
      "Iteration: 2358; Percent complete: 59.0%; Average loss: 2.5679\n",
      "Iteration: 2359; Percent complete: 59.0%; Average loss: 2.5742\n",
      "Iteration: 2360; Percent complete: 59.0%; Average loss: 2.6472\n",
      "Iteration: 2361; Percent complete: 59.0%; Average loss: 2.3765\n",
      "Iteration: 2362; Percent complete: 59.1%; Average loss: 2.4749\n",
      "Iteration: 2363; Percent complete: 59.1%; Average loss: 2.6030\n",
      "Iteration: 2364; Percent complete: 59.1%; Average loss: 2.4981\n",
      "Iteration: 2365; Percent complete: 59.1%; Average loss: 2.5031\n",
      "Iteration: 2366; Percent complete: 59.2%; Average loss: 2.4874\n",
      "Iteration: 2367; Percent complete: 59.2%; Average loss: 2.5091\n",
      "Iteration: 2368; Percent complete: 59.2%; Average loss: 2.6443\n",
      "Iteration: 2369; Percent complete: 59.2%; Average loss: 2.7137\n",
      "Iteration: 2370; Percent complete: 59.2%; Average loss: 2.6317\n",
      "Iteration: 2371; Percent complete: 59.3%; Average loss: 2.4903\n",
      "Iteration: 2372; Percent complete: 59.3%; Average loss: 2.6684\n",
      "Iteration: 2373; Percent complete: 59.3%; Average loss: 2.3990\n",
      "Iteration: 2374; Percent complete: 59.4%; Average loss: 2.5339\n",
      "Iteration: 2375; Percent complete: 59.4%; Average loss: 2.4446\n",
      "Iteration: 2376; Percent complete: 59.4%; Average loss: 2.5212\n",
      "Iteration: 2377; Percent complete: 59.4%; Average loss: 2.6294\n",
      "Iteration: 2378; Percent complete: 59.5%; Average loss: 2.5688\n",
      "Iteration: 2379; Percent complete: 59.5%; Average loss: 2.6236\n",
      "Iteration: 2380; Percent complete: 59.5%; Average loss: 2.6116\n",
      "Iteration: 2381; Percent complete: 59.5%; Average loss: 2.3894\n",
      "Iteration: 2382; Percent complete: 59.6%; Average loss: 2.2960\n",
      "Iteration: 2383; Percent complete: 59.6%; Average loss: 2.3860\n",
      "Iteration: 2384; Percent complete: 59.6%; Average loss: 2.6092\n",
      "Iteration: 2385; Percent complete: 59.6%; Average loss: 2.8511\n",
      "Iteration: 2386; Percent complete: 59.7%; Average loss: 2.4955\n",
      "Iteration: 2387; Percent complete: 59.7%; Average loss: 2.4568\n",
      "Iteration: 2388; Percent complete: 59.7%; Average loss: 2.6173\n",
      "Iteration: 2389; Percent complete: 59.7%; Average loss: 2.3794\n",
      "Iteration: 2390; Percent complete: 59.8%; Average loss: 2.5485\n",
      "Iteration: 2391; Percent complete: 59.8%; Average loss: 2.6549\n",
      "Iteration: 2392; Percent complete: 59.8%; Average loss: 2.8288\n",
      "Iteration: 2393; Percent complete: 59.8%; Average loss: 2.6827\n",
      "Iteration: 2394; Percent complete: 59.9%; Average loss: 2.4781\n",
      "Iteration: 2395; Percent complete: 59.9%; Average loss: 2.6320\n",
      "Iteration: 2396; Percent complete: 59.9%; Average loss: 2.7438\n",
      "Iteration: 2397; Percent complete: 59.9%; Average loss: 2.6139\n",
      "Iteration: 2398; Percent complete: 60.0%; Average loss: 2.3497\n",
      "Iteration: 2399; Percent complete: 60.0%; Average loss: 2.6706\n",
      "Iteration: 2400; Percent complete: 60.0%; Average loss: 2.4702\n",
      "Iteration: 2401; Percent complete: 60.0%; Average loss: 2.6131\n",
      "Iteration: 2402; Percent complete: 60.1%; Average loss: 2.4382\n",
      "Iteration: 2403; Percent complete: 60.1%; Average loss: 2.6372\n",
      "Iteration: 2404; Percent complete: 60.1%; Average loss: 2.4560\n",
      "Iteration: 2405; Percent complete: 60.1%; Average loss: 2.5359\n",
      "Iteration: 2406; Percent complete: 60.2%; Average loss: 2.4229\n",
      "Iteration: 2407; Percent complete: 60.2%; Average loss: 2.5573\n",
      "Iteration: 2408; Percent complete: 60.2%; Average loss: 2.4007\n",
      "Iteration: 2409; Percent complete: 60.2%; Average loss: 2.5995\n",
      "Iteration: 2410; Percent complete: 60.2%; Average loss: 2.6405\n",
      "Iteration: 2411; Percent complete: 60.3%; Average loss: 2.4740\n",
      "Iteration: 2412; Percent complete: 60.3%; Average loss: 2.6385\n",
      "Iteration: 2413; Percent complete: 60.3%; Average loss: 2.3750\n",
      "Iteration: 2414; Percent complete: 60.4%; Average loss: 2.6965\n",
      "Iteration: 2415; Percent complete: 60.4%; Average loss: 2.4740\n",
      "Iteration: 2416; Percent complete: 60.4%; Average loss: 2.7096\n",
      "Iteration: 2417; Percent complete: 60.4%; Average loss: 2.6508\n",
      "Iteration: 2418; Percent complete: 60.5%; Average loss: 2.5852\n",
      "Iteration: 2419; Percent complete: 60.5%; Average loss: 2.5246\n",
      "Iteration: 2420; Percent complete: 60.5%; Average loss: 2.4285\n",
      "Iteration: 2421; Percent complete: 60.5%; Average loss: 2.4316\n",
      "Iteration: 2422; Percent complete: 60.6%; Average loss: 2.8128\n",
      "Iteration: 2423; Percent complete: 60.6%; Average loss: 2.5386\n",
      "Iteration: 2424; Percent complete: 60.6%; Average loss: 2.4404\n",
      "Iteration: 2425; Percent complete: 60.6%; Average loss: 2.9613\n",
      "Iteration: 2426; Percent complete: 60.7%; Average loss: 2.6121\n",
      "Iteration: 2427; Percent complete: 60.7%; Average loss: 2.5004\n",
      "Iteration: 2428; Percent complete: 60.7%; Average loss: 2.4172\n",
      "Iteration: 2429; Percent complete: 60.7%; Average loss: 2.7842\n",
      "Iteration: 2430; Percent complete: 60.8%; Average loss: 2.5641\n",
      "Iteration: 2431; Percent complete: 60.8%; Average loss: 2.5110\n",
      "Iteration: 2432; Percent complete: 60.8%; Average loss: 2.7747\n",
      "Iteration: 2433; Percent complete: 60.8%; Average loss: 2.7019\n",
      "Iteration: 2434; Percent complete: 60.9%; Average loss: 2.5774\n",
      "Iteration: 2435; Percent complete: 60.9%; Average loss: 2.4678\n",
      "Iteration: 2436; Percent complete: 60.9%; Average loss: 2.4894\n",
      "Iteration: 2437; Percent complete: 60.9%; Average loss: 2.5459\n",
      "Iteration: 2438; Percent complete: 61.0%; Average loss: 2.5157\n",
      "Iteration: 2439; Percent complete: 61.0%; Average loss: 2.6820\n",
      "Iteration: 2440; Percent complete: 61.0%; Average loss: 2.6258\n",
      "Iteration: 2441; Percent complete: 61.0%; Average loss: 2.3863\n",
      "Iteration: 2442; Percent complete: 61.1%; Average loss: 2.5208\n",
      "Iteration: 2443; Percent complete: 61.1%; Average loss: 2.4627\n",
      "Iteration: 2444; Percent complete: 61.1%; Average loss: 2.3718\n",
      "Iteration: 2445; Percent complete: 61.1%; Average loss: 2.6060\n",
      "Iteration: 2446; Percent complete: 61.2%; Average loss: 2.4170\n",
      "Iteration: 2447; Percent complete: 61.2%; Average loss: 2.5897\n",
      "Iteration: 2448; Percent complete: 61.2%; Average loss: 2.6066\n",
      "Iteration: 2449; Percent complete: 61.2%; Average loss: 2.5262\n",
      "Iteration: 2450; Percent complete: 61.3%; Average loss: 2.5550\n",
      "Iteration: 2451; Percent complete: 61.3%; Average loss: 2.3602\n",
      "Iteration: 2452; Percent complete: 61.3%; Average loss: 2.5092\n",
      "Iteration: 2453; Percent complete: 61.3%; Average loss: 2.4564\n",
      "Iteration: 2454; Percent complete: 61.4%; Average loss: 2.3748\n",
      "Iteration: 2455; Percent complete: 61.4%; Average loss: 2.3157\n",
      "Iteration: 2456; Percent complete: 61.4%; Average loss: 2.5705\n",
      "Iteration: 2457; Percent complete: 61.4%; Average loss: 2.7055\n",
      "Iteration: 2458; Percent complete: 61.5%; Average loss: 2.6601\n",
      "Iteration: 2459; Percent complete: 61.5%; Average loss: 2.6102\n",
      "Iteration: 2460; Percent complete: 61.5%; Average loss: 2.4543\n",
      "Iteration: 2461; Percent complete: 61.5%; Average loss: 2.4587\n",
      "Iteration: 2462; Percent complete: 61.6%; Average loss: 2.4734\n",
      "Iteration: 2463; Percent complete: 61.6%; Average loss: 2.5606\n",
      "Iteration: 2464; Percent complete: 61.6%; Average loss: 2.6500\n",
      "Iteration: 2465; Percent complete: 61.6%; Average loss: 2.6233\n",
      "Iteration: 2466; Percent complete: 61.7%; Average loss: 2.7554\n",
      "Iteration: 2467; Percent complete: 61.7%; Average loss: 2.3665\n",
      "Iteration: 2468; Percent complete: 61.7%; Average loss: 2.7559\n",
      "Iteration: 2469; Percent complete: 61.7%; Average loss: 2.4384\n",
      "Iteration: 2470; Percent complete: 61.8%; Average loss: 2.6715\n",
      "Iteration: 2471; Percent complete: 61.8%; Average loss: 2.4809\n",
      "Iteration: 2472; Percent complete: 61.8%; Average loss: 2.4295\n",
      "Iteration: 2473; Percent complete: 61.8%; Average loss: 2.7102\n",
      "Iteration: 2474; Percent complete: 61.9%; Average loss: 2.1516\n",
      "Iteration: 2475; Percent complete: 61.9%; Average loss: 2.8653\n",
      "Iteration: 2476; Percent complete: 61.9%; Average loss: 2.4762\n",
      "Iteration: 2477; Percent complete: 61.9%; Average loss: 2.3721\n",
      "Iteration: 2478; Percent complete: 62.0%; Average loss: 2.4331\n",
      "Iteration: 2479; Percent complete: 62.0%; Average loss: 2.5182\n",
      "Iteration: 2480; Percent complete: 62.0%; Average loss: 2.5322\n",
      "Iteration: 2481; Percent complete: 62.0%; Average loss: 2.4588\n",
      "Iteration: 2482; Percent complete: 62.1%; Average loss: 2.4751\n",
      "Iteration: 2483; Percent complete: 62.1%; Average loss: 2.4670\n",
      "Iteration: 2484; Percent complete: 62.1%; Average loss: 2.4059\n",
      "Iteration: 2485; Percent complete: 62.1%; Average loss: 2.5477\n",
      "Iteration: 2486; Percent complete: 62.2%; Average loss: 2.6029\n",
      "Iteration: 2487; Percent complete: 62.2%; Average loss: 2.9832\n",
      "Iteration: 2488; Percent complete: 62.2%; Average loss: 2.3619\n",
      "Iteration: 2489; Percent complete: 62.2%; Average loss: 2.3656\n",
      "Iteration: 2490; Percent complete: 62.3%; Average loss: 2.4841\n",
      "Iteration: 2491; Percent complete: 62.3%; Average loss: 2.4908\n",
      "Iteration: 2492; Percent complete: 62.3%; Average loss: 2.6813\n",
      "Iteration: 2493; Percent complete: 62.3%; Average loss: 2.4521\n",
      "Iteration: 2494; Percent complete: 62.4%; Average loss: 2.6780\n",
      "Iteration: 2495; Percent complete: 62.4%; Average loss: 2.7856\n",
      "Iteration: 2496; Percent complete: 62.4%; Average loss: 2.7716\n",
      "Iteration: 2497; Percent complete: 62.4%; Average loss: 2.5022\n",
      "Iteration: 2498; Percent complete: 62.5%; Average loss: 2.5002\n",
      "Iteration: 2499; Percent complete: 62.5%; Average loss: 2.5070\n",
      "Iteration: 2500; Percent complete: 62.5%; Average loss: 2.5365\n",
      "Iteration: 2501; Percent complete: 62.5%; Average loss: 2.6747\n",
      "Iteration: 2502; Percent complete: 62.5%; Average loss: 2.6074\n",
      "Iteration: 2503; Percent complete: 62.6%; Average loss: 2.4157\n",
      "Iteration: 2504; Percent complete: 62.6%; Average loss: 2.6970\n",
      "Iteration: 2505; Percent complete: 62.6%; Average loss: 2.4382\n",
      "Iteration: 2506; Percent complete: 62.6%; Average loss: 2.6890\n",
      "Iteration: 2507; Percent complete: 62.7%; Average loss: 2.6010\n",
      "Iteration: 2508; Percent complete: 62.7%; Average loss: 2.4503\n",
      "Iteration: 2509; Percent complete: 62.7%; Average loss: 2.6828\n",
      "Iteration: 2510; Percent complete: 62.7%; Average loss: 2.8012\n",
      "Iteration: 2511; Percent complete: 62.8%; Average loss: 2.4986\n",
      "Iteration: 2512; Percent complete: 62.8%; Average loss: 2.8058\n",
      "Iteration: 2513; Percent complete: 62.8%; Average loss: 2.6519\n",
      "Iteration: 2514; Percent complete: 62.8%; Average loss: 2.7910\n",
      "Iteration: 2515; Percent complete: 62.9%; Average loss: 2.5132\n",
      "Iteration: 2516; Percent complete: 62.9%; Average loss: 2.4974\n",
      "Iteration: 2517; Percent complete: 62.9%; Average loss: 2.5751\n",
      "Iteration: 2518; Percent complete: 62.9%; Average loss: 2.2877\n",
      "Iteration: 2519; Percent complete: 63.0%; Average loss: 2.3888\n",
      "Iteration: 2520; Percent complete: 63.0%; Average loss: 2.5278\n",
      "Iteration: 2521; Percent complete: 63.0%; Average loss: 2.4093\n",
      "Iteration: 2522; Percent complete: 63.0%; Average loss: 2.5581\n",
      "Iteration: 2523; Percent complete: 63.1%; Average loss: 2.2991\n",
      "Iteration: 2524; Percent complete: 63.1%; Average loss: 2.4771\n",
      "Iteration: 2525; Percent complete: 63.1%; Average loss: 2.3567\n",
      "Iteration: 2526; Percent complete: 63.1%; Average loss: 2.6963\n",
      "Iteration: 2527; Percent complete: 63.2%; Average loss: 2.6517\n",
      "Iteration: 2528; Percent complete: 63.2%; Average loss: 2.3783\n",
      "Iteration: 2529; Percent complete: 63.2%; Average loss: 2.5844\n",
      "Iteration: 2530; Percent complete: 63.2%; Average loss: 2.4690\n",
      "Iteration: 2531; Percent complete: 63.3%; Average loss: 2.4799\n",
      "Iteration: 2532; Percent complete: 63.3%; Average loss: 2.3206\n",
      "Iteration: 2533; Percent complete: 63.3%; Average loss: 2.6078\n",
      "Iteration: 2534; Percent complete: 63.3%; Average loss: 2.4748\n",
      "Iteration: 2535; Percent complete: 63.4%; Average loss: 2.2710\n",
      "Iteration: 2536; Percent complete: 63.4%; Average loss: 2.2891\n",
      "Iteration: 2537; Percent complete: 63.4%; Average loss: 2.5019\n",
      "Iteration: 2538; Percent complete: 63.4%; Average loss: 2.5647\n",
      "Iteration: 2539; Percent complete: 63.5%; Average loss: 2.5151\n",
      "Iteration: 2540; Percent complete: 63.5%; Average loss: 2.3887\n",
      "Iteration: 2541; Percent complete: 63.5%; Average loss: 2.5221\n",
      "Iteration: 2542; Percent complete: 63.5%; Average loss: 2.7797\n",
      "Iteration: 2543; Percent complete: 63.6%; Average loss: 2.5122\n",
      "Iteration: 2544; Percent complete: 63.6%; Average loss: 2.5116\n",
      "Iteration: 2545; Percent complete: 63.6%; Average loss: 2.5339\n",
      "Iteration: 2546; Percent complete: 63.6%; Average loss: 2.4949\n",
      "Iteration: 2547; Percent complete: 63.7%; Average loss: 2.4176\n",
      "Iteration: 2548; Percent complete: 63.7%; Average loss: 2.7817\n",
      "Iteration: 2549; Percent complete: 63.7%; Average loss: 2.3589\n",
      "Iteration: 2550; Percent complete: 63.7%; Average loss: 2.6291\n",
      "Iteration: 2551; Percent complete: 63.8%; Average loss: 2.6694\n",
      "Iteration: 2552; Percent complete: 63.8%; Average loss: 2.5364\n",
      "Iteration: 2553; Percent complete: 63.8%; Average loss: 2.4646\n",
      "Iteration: 2554; Percent complete: 63.8%; Average loss: 2.5223\n",
      "Iteration: 2555; Percent complete: 63.9%; Average loss: 2.4250\n",
      "Iteration: 2556; Percent complete: 63.9%; Average loss: 2.3466\n",
      "Iteration: 2557; Percent complete: 63.9%; Average loss: 2.4356\n",
      "Iteration: 2558; Percent complete: 63.9%; Average loss: 2.7001\n",
      "Iteration: 2559; Percent complete: 64.0%; Average loss: 2.5106\n",
      "Iteration: 2560; Percent complete: 64.0%; Average loss: 2.4988\n",
      "Iteration: 2561; Percent complete: 64.0%; Average loss: 2.3465\n",
      "Iteration: 2562; Percent complete: 64.0%; Average loss: 2.4751\n",
      "Iteration: 2563; Percent complete: 64.1%; Average loss: 2.4445\n",
      "Iteration: 2564; Percent complete: 64.1%; Average loss: 2.4687\n",
      "Iteration: 2565; Percent complete: 64.1%; Average loss: 2.6209\n",
      "Iteration: 2566; Percent complete: 64.1%; Average loss: 2.7589\n",
      "Iteration: 2567; Percent complete: 64.2%; Average loss: 2.8082\n",
      "Iteration: 2568; Percent complete: 64.2%; Average loss: 2.5805\n",
      "Iteration: 2569; Percent complete: 64.2%; Average loss: 2.6054\n",
      "Iteration: 2570; Percent complete: 64.2%; Average loss: 2.3904\n",
      "Iteration: 2571; Percent complete: 64.3%; Average loss: 2.5489\n",
      "Iteration: 2572; Percent complete: 64.3%; Average loss: 2.3892\n",
      "Iteration: 2573; Percent complete: 64.3%; Average loss: 2.5036\n",
      "Iteration: 2574; Percent complete: 64.3%; Average loss: 2.6005\n",
      "Iteration: 2575; Percent complete: 64.4%; Average loss: 2.5519\n",
      "Iteration: 2576; Percent complete: 64.4%; Average loss: 2.6551\n",
      "Iteration: 2577; Percent complete: 64.4%; Average loss: 2.3678\n",
      "Iteration: 2578; Percent complete: 64.5%; Average loss: 2.5819\n",
      "Iteration: 2579; Percent complete: 64.5%; Average loss: 2.3056\n",
      "Iteration: 2580; Percent complete: 64.5%; Average loss: 2.3707\n",
      "Iteration: 2581; Percent complete: 64.5%; Average loss: 2.4782\n",
      "Iteration: 2582; Percent complete: 64.5%; Average loss: 2.4078\n",
      "Iteration: 2583; Percent complete: 64.6%; Average loss: 2.4974\n",
      "Iteration: 2584; Percent complete: 64.6%; Average loss: 2.8296\n",
      "Iteration: 2585; Percent complete: 64.6%; Average loss: 2.6037\n",
      "Iteration: 2586; Percent complete: 64.6%; Average loss: 2.4815\n",
      "Iteration: 2587; Percent complete: 64.7%; Average loss: 2.4354\n",
      "Iteration: 2588; Percent complete: 64.7%; Average loss: 2.6612\n",
      "Iteration: 2589; Percent complete: 64.7%; Average loss: 2.3626\n",
      "Iteration: 2590; Percent complete: 64.8%; Average loss: 2.3121\n",
      "Iteration: 2591; Percent complete: 64.8%; Average loss: 2.4968\n",
      "Iteration: 2592; Percent complete: 64.8%; Average loss: 2.5521\n",
      "Iteration: 2593; Percent complete: 64.8%; Average loss: 2.4259\n",
      "Iteration: 2594; Percent complete: 64.8%; Average loss: 2.6325\n",
      "Iteration: 2595; Percent complete: 64.9%; Average loss: 2.5057\n",
      "Iteration: 2596; Percent complete: 64.9%; Average loss: 2.3875\n",
      "Iteration: 2597; Percent complete: 64.9%; Average loss: 2.5502\n",
      "Iteration: 2598; Percent complete: 65.0%; Average loss: 2.5050\n",
      "Iteration: 2599; Percent complete: 65.0%; Average loss: 2.6534\n",
      "Iteration: 2600; Percent complete: 65.0%; Average loss: 2.5290\n",
      "Iteration: 2601; Percent complete: 65.0%; Average loss: 2.4129\n",
      "Iteration: 2602; Percent complete: 65.0%; Average loss: 2.4835\n",
      "Iteration: 2603; Percent complete: 65.1%; Average loss: 2.2715\n",
      "Iteration: 2604; Percent complete: 65.1%; Average loss: 2.4231\n",
      "Iteration: 2605; Percent complete: 65.1%; Average loss: 2.4980\n",
      "Iteration: 2606; Percent complete: 65.1%; Average loss: 2.6000\n",
      "Iteration: 2607; Percent complete: 65.2%; Average loss: 2.5684\n",
      "Iteration: 2608; Percent complete: 65.2%; Average loss: 2.3561\n",
      "Iteration: 2609; Percent complete: 65.2%; Average loss: 2.5802\n",
      "Iteration: 2610; Percent complete: 65.2%; Average loss: 2.5321\n",
      "Iteration: 2611; Percent complete: 65.3%; Average loss: 2.6032\n",
      "Iteration: 2612; Percent complete: 65.3%; Average loss: 2.6159\n",
      "Iteration: 2613; Percent complete: 65.3%; Average loss: 2.5495\n",
      "Iteration: 2614; Percent complete: 65.3%; Average loss: 2.5628\n",
      "Iteration: 2615; Percent complete: 65.4%; Average loss: 2.4162\n",
      "Iteration: 2616; Percent complete: 65.4%; Average loss: 2.5794\n",
      "Iteration: 2617; Percent complete: 65.4%; Average loss: 2.5168\n",
      "Iteration: 2618; Percent complete: 65.5%; Average loss: 2.4768\n",
      "Iteration: 2619; Percent complete: 65.5%; Average loss: 2.7042\n",
      "Iteration: 2620; Percent complete: 65.5%; Average loss: 2.1932\n",
      "Iteration: 2621; Percent complete: 65.5%; Average loss: 2.3801\n",
      "Iteration: 2622; Percent complete: 65.5%; Average loss: 2.5650\n",
      "Iteration: 2623; Percent complete: 65.6%; Average loss: 2.4642\n",
      "Iteration: 2624; Percent complete: 65.6%; Average loss: 2.4370\n",
      "Iteration: 2625; Percent complete: 65.6%; Average loss: 2.4580\n",
      "Iteration: 2626; Percent complete: 65.6%; Average loss: 2.6788\n",
      "Iteration: 2627; Percent complete: 65.7%; Average loss: 2.6284\n",
      "Iteration: 2628; Percent complete: 65.7%; Average loss: 2.5297\n",
      "Iteration: 2629; Percent complete: 65.7%; Average loss: 2.6152\n",
      "Iteration: 2630; Percent complete: 65.8%; Average loss: 2.4356\n",
      "Iteration: 2631; Percent complete: 65.8%; Average loss: 2.6265\n",
      "Iteration: 2632; Percent complete: 65.8%; Average loss: 2.4782\n",
      "Iteration: 2633; Percent complete: 65.8%; Average loss: 2.3078\n",
      "Iteration: 2634; Percent complete: 65.8%; Average loss: 2.5134\n",
      "Iteration: 2635; Percent complete: 65.9%; Average loss: 2.5604\n",
      "Iteration: 2636; Percent complete: 65.9%; Average loss: 2.3101\n",
      "Iteration: 2637; Percent complete: 65.9%; Average loss: 2.5703\n",
      "Iteration: 2638; Percent complete: 66.0%; Average loss: 2.4501\n",
      "Iteration: 2639; Percent complete: 66.0%; Average loss: 2.5493\n",
      "Iteration: 2640; Percent complete: 66.0%; Average loss: 2.5467\n",
      "Iteration: 2641; Percent complete: 66.0%; Average loss: 2.6297\n",
      "Iteration: 2642; Percent complete: 66.0%; Average loss: 2.5336\n",
      "Iteration: 2643; Percent complete: 66.1%; Average loss: 2.7419\n",
      "Iteration: 2644; Percent complete: 66.1%; Average loss: 2.4884\n",
      "Iteration: 2645; Percent complete: 66.1%; Average loss: 2.4936\n",
      "Iteration: 2646; Percent complete: 66.1%; Average loss: 2.6046\n",
      "Iteration: 2647; Percent complete: 66.2%; Average loss: 2.2968\n",
      "Iteration: 2648; Percent complete: 66.2%; Average loss: 2.5315\n",
      "Iteration: 2649; Percent complete: 66.2%; Average loss: 2.5441\n",
      "Iteration: 2650; Percent complete: 66.2%; Average loss: 2.4372\n",
      "Iteration: 2651; Percent complete: 66.3%; Average loss: 2.5700\n",
      "Iteration: 2652; Percent complete: 66.3%; Average loss: 2.5496\n",
      "Iteration: 2653; Percent complete: 66.3%; Average loss: 2.6036\n",
      "Iteration: 2654; Percent complete: 66.3%; Average loss: 2.7960\n",
      "Iteration: 2655; Percent complete: 66.4%; Average loss: 2.5115\n",
      "Iteration: 2656; Percent complete: 66.4%; Average loss: 2.4448\n",
      "Iteration: 2657; Percent complete: 66.4%; Average loss: 2.5113\n",
      "Iteration: 2658; Percent complete: 66.5%; Average loss: 2.3765\n",
      "Iteration: 2659; Percent complete: 66.5%; Average loss: 2.6149\n",
      "Iteration: 2660; Percent complete: 66.5%; Average loss: 2.3251\n",
      "Iteration: 2661; Percent complete: 66.5%; Average loss: 2.6770\n",
      "Iteration: 2662; Percent complete: 66.5%; Average loss: 2.4819\n",
      "Iteration: 2663; Percent complete: 66.6%; Average loss: 2.4407\n",
      "Iteration: 2664; Percent complete: 66.6%; Average loss: 2.3231\n",
      "Iteration: 2665; Percent complete: 66.6%; Average loss: 2.6816\n",
      "Iteration: 2666; Percent complete: 66.6%; Average loss: 2.4853\n",
      "Iteration: 2667; Percent complete: 66.7%; Average loss: 2.5686\n",
      "Iteration: 2668; Percent complete: 66.7%; Average loss: 2.4842\n",
      "Iteration: 2669; Percent complete: 66.7%; Average loss: 2.5588\n",
      "Iteration: 2670; Percent complete: 66.8%; Average loss: 2.3370\n",
      "Iteration: 2671; Percent complete: 66.8%; Average loss: 2.4038\n",
      "Iteration: 2672; Percent complete: 66.8%; Average loss: 2.5408\n",
      "Iteration: 2673; Percent complete: 66.8%; Average loss: 2.6509\n",
      "Iteration: 2674; Percent complete: 66.8%; Average loss: 2.4250\n",
      "Iteration: 2675; Percent complete: 66.9%; Average loss: 2.5680\n",
      "Iteration: 2676; Percent complete: 66.9%; Average loss: 2.3724\n",
      "Iteration: 2677; Percent complete: 66.9%; Average loss: 2.2694\n",
      "Iteration: 2678; Percent complete: 67.0%; Average loss: 2.7754\n",
      "Iteration: 2679; Percent complete: 67.0%; Average loss: 2.5224\n",
      "Iteration: 2680; Percent complete: 67.0%; Average loss: 2.6127\n",
      "Iteration: 2681; Percent complete: 67.0%; Average loss: 2.6167\n",
      "Iteration: 2682; Percent complete: 67.0%; Average loss: 2.3847\n",
      "Iteration: 2683; Percent complete: 67.1%; Average loss: 2.5375\n",
      "Iteration: 2684; Percent complete: 67.1%; Average loss: 2.3037\n",
      "Iteration: 2685; Percent complete: 67.1%; Average loss: 2.7103\n",
      "Iteration: 2686; Percent complete: 67.2%; Average loss: 2.5347\n",
      "Iteration: 2687; Percent complete: 67.2%; Average loss: 2.6014\n",
      "Iteration: 2688; Percent complete: 67.2%; Average loss: 2.4643\n",
      "Iteration: 2689; Percent complete: 67.2%; Average loss: 2.6588\n",
      "Iteration: 2690; Percent complete: 67.2%; Average loss: 2.5925\n",
      "Iteration: 2691; Percent complete: 67.3%; Average loss: 2.4663\n",
      "Iteration: 2692; Percent complete: 67.3%; Average loss: 2.5376\n",
      "Iteration: 2693; Percent complete: 67.3%; Average loss: 2.5349\n",
      "Iteration: 2694; Percent complete: 67.3%; Average loss: 2.5722\n",
      "Iteration: 2695; Percent complete: 67.4%; Average loss: 2.7554\n",
      "Iteration: 2696; Percent complete: 67.4%; Average loss: 2.6392\n",
      "Iteration: 2697; Percent complete: 67.4%; Average loss: 2.4631\n",
      "Iteration: 2698; Percent complete: 67.5%; Average loss: 2.4195\n",
      "Iteration: 2699; Percent complete: 67.5%; Average loss: 2.5246\n",
      "Iteration: 2700; Percent complete: 67.5%; Average loss: 2.5193\n",
      "Iteration: 2701; Percent complete: 67.5%; Average loss: 2.5143\n",
      "Iteration: 2702; Percent complete: 67.5%; Average loss: 2.4112\n",
      "Iteration: 2703; Percent complete: 67.6%; Average loss: 2.4448\n",
      "Iteration: 2704; Percent complete: 67.6%; Average loss: 2.5805\n",
      "Iteration: 2705; Percent complete: 67.6%; Average loss: 2.7702\n",
      "Iteration: 2706; Percent complete: 67.7%; Average loss: 2.5948\n",
      "Iteration: 2707; Percent complete: 67.7%; Average loss: 2.4459\n",
      "Iteration: 2708; Percent complete: 67.7%; Average loss: 2.6010\n",
      "Iteration: 2709; Percent complete: 67.7%; Average loss: 2.3839\n",
      "Iteration: 2710; Percent complete: 67.8%; Average loss: 2.6649\n",
      "Iteration: 2711; Percent complete: 67.8%; Average loss: 2.6035\n",
      "Iteration: 2712; Percent complete: 67.8%; Average loss: 2.5010\n",
      "Iteration: 2713; Percent complete: 67.8%; Average loss: 2.5928\n",
      "Iteration: 2714; Percent complete: 67.8%; Average loss: 2.4286\n",
      "Iteration: 2715; Percent complete: 67.9%; Average loss: 2.2592\n",
      "Iteration: 2716; Percent complete: 67.9%; Average loss: 2.5682\n",
      "Iteration: 2717; Percent complete: 67.9%; Average loss: 2.5871\n",
      "Iteration: 2718; Percent complete: 68.0%; Average loss: 2.5996\n",
      "Iteration: 2719; Percent complete: 68.0%; Average loss: 2.3962\n",
      "Iteration: 2720; Percent complete: 68.0%; Average loss: 2.4401\n",
      "Iteration: 2721; Percent complete: 68.0%; Average loss: 2.4082\n",
      "Iteration: 2722; Percent complete: 68.0%; Average loss: 2.4101\n",
      "Iteration: 2723; Percent complete: 68.1%; Average loss: 2.5407\n",
      "Iteration: 2724; Percent complete: 68.1%; Average loss: 2.3938\n",
      "Iteration: 2725; Percent complete: 68.1%; Average loss: 2.3664\n",
      "Iteration: 2726; Percent complete: 68.2%; Average loss: 2.7640\n",
      "Iteration: 2727; Percent complete: 68.2%; Average loss: 2.4496\n",
      "Iteration: 2728; Percent complete: 68.2%; Average loss: 2.4264\n",
      "Iteration: 2729; Percent complete: 68.2%; Average loss: 2.4733\n",
      "Iteration: 2730; Percent complete: 68.2%; Average loss: 2.4320\n",
      "Iteration: 2731; Percent complete: 68.3%; Average loss: 2.6095\n",
      "Iteration: 2732; Percent complete: 68.3%; Average loss: 2.2928\n",
      "Iteration: 2733; Percent complete: 68.3%; Average loss: 2.7150\n",
      "Iteration: 2734; Percent complete: 68.3%; Average loss: 2.4372\n",
      "Iteration: 2735; Percent complete: 68.4%; Average loss: 2.1757\n",
      "Iteration: 2736; Percent complete: 68.4%; Average loss: 2.6282\n",
      "Iteration: 2737; Percent complete: 68.4%; Average loss: 2.4275\n",
      "Iteration: 2738; Percent complete: 68.5%; Average loss: 2.3382\n",
      "Iteration: 2739; Percent complete: 68.5%; Average loss: 2.3759\n",
      "Iteration: 2740; Percent complete: 68.5%; Average loss: 2.5889\n",
      "Iteration: 2741; Percent complete: 68.5%; Average loss: 2.4404\n",
      "Iteration: 2742; Percent complete: 68.5%; Average loss: 2.5907\n",
      "Iteration: 2743; Percent complete: 68.6%; Average loss: 2.2387\n",
      "Iteration: 2744; Percent complete: 68.6%; Average loss: 2.5589\n",
      "Iteration: 2745; Percent complete: 68.6%; Average loss: 2.6131\n",
      "Iteration: 2746; Percent complete: 68.7%; Average loss: 2.5646\n",
      "Iteration: 2747; Percent complete: 68.7%; Average loss: 2.7463\n",
      "Iteration: 2748; Percent complete: 68.7%; Average loss: 2.5601\n",
      "Iteration: 2749; Percent complete: 68.7%; Average loss: 2.5075\n",
      "Iteration: 2750; Percent complete: 68.8%; Average loss: 2.5786\n",
      "Iteration: 2751; Percent complete: 68.8%; Average loss: 2.4953\n",
      "Iteration: 2752; Percent complete: 68.8%; Average loss: 1.9790\n",
      "Iteration: 2753; Percent complete: 68.8%; Average loss: 2.6309\n",
      "Iteration: 2754; Percent complete: 68.8%; Average loss: 2.4467\n",
      "Iteration: 2755; Percent complete: 68.9%; Average loss: 2.4234\n",
      "Iteration: 2756; Percent complete: 68.9%; Average loss: 2.3006\n",
      "Iteration: 2757; Percent complete: 68.9%; Average loss: 2.3129\n",
      "Iteration: 2758; Percent complete: 69.0%; Average loss: 2.2849\n",
      "Iteration: 2759; Percent complete: 69.0%; Average loss: 2.4896\n",
      "Iteration: 2760; Percent complete: 69.0%; Average loss: 2.5277\n",
      "Iteration: 2761; Percent complete: 69.0%; Average loss: 2.5123\n",
      "Iteration: 2762; Percent complete: 69.0%; Average loss: 2.3957\n",
      "Iteration: 2763; Percent complete: 69.1%; Average loss: 2.4384\n",
      "Iteration: 2764; Percent complete: 69.1%; Average loss: 2.2962\n",
      "Iteration: 2765; Percent complete: 69.1%; Average loss: 2.4247\n",
      "Iteration: 2766; Percent complete: 69.2%; Average loss: 2.3771\n",
      "Iteration: 2767; Percent complete: 69.2%; Average loss: 2.5254\n",
      "Iteration: 2768; Percent complete: 69.2%; Average loss: 2.6148\n",
      "Iteration: 2769; Percent complete: 69.2%; Average loss: 2.4886\n",
      "Iteration: 2770; Percent complete: 69.2%; Average loss: 2.6224\n",
      "Iteration: 2771; Percent complete: 69.3%; Average loss: 2.6287\n",
      "Iteration: 2772; Percent complete: 69.3%; Average loss: 2.6056\n",
      "Iteration: 2773; Percent complete: 69.3%; Average loss: 2.2797\n",
      "Iteration: 2774; Percent complete: 69.3%; Average loss: 2.3573\n",
      "Iteration: 2775; Percent complete: 69.4%; Average loss: 2.4853\n",
      "Iteration: 2776; Percent complete: 69.4%; Average loss: 2.6424\n",
      "Iteration: 2777; Percent complete: 69.4%; Average loss: 2.2289\n",
      "Iteration: 2778; Percent complete: 69.5%; Average loss: 2.5418\n",
      "Iteration: 2779; Percent complete: 69.5%; Average loss: 2.5975\n",
      "Iteration: 2780; Percent complete: 69.5%; Average loss: 2.5066\n",
      "Iteration: 2781; Percent complete: 69.5%; Average loss: 2.5115\n",
      "Iteration: 2782; Percent complete: 69.5%; Average loss: 2.5695\n",
      "Iteration: 2783; Percent complete: 69.6%; Average loss: 2.4075\n",
      "Iteration: 2784; Percent complete: 69.6%; Average loss: 2.5485\n",
      "Iteration: 2785; Percent complete: 69.6%; Average loss: 2.6160\n",
      "Iteration: 2786; Percent complete: 69.7%; Average loss: 2.5099\n",
      "Iteration: 2787; Percent complete: 69.7%; Average loss: 2.4450\n",
      "Iteration: 2788; Percent complete: 69.7%; Average loss: 2.7150\n",
      "Iteration: 2789; Percent complete: 69.7%; Average loss: 2.8191\n",
      "Iteration: 2790; Percent complete: 69.8%; Average loss: 2.6108\n",
      "Iteration: 2791; Percent complete: 69.8%; Average loss: 2.4013\n",
      "Iteration: 2792; Percent complete: 69.8%; Average loss: 2.4238\n",
      "Iteration: 2793; Percent complete: 69.8%; Average loss: 2.3046\n",
      "Iteration: 2794; Percent complete: 69.8%; Average loss: 2.6444\n",
      "Iteration: 2795; Percent complete: 69.9%; Average loss: 2.4386\n",
      "Iteration: 2796; Percent complete: 69.9%; Average loss: 2.4499\n",
      "Iteration: 2797; Percent complete: 69.9%; Average loss: 2.3539\n",
      "Iteration: 2798; Percent complete: 70.0%; Average loss: 2.3994\n",
      "Iteration: 2799; Percent complete: 70.0%; Average loss: 2.4884\n",
      "Iteration: 2800; Percent complete: 70.0%; Average loss: 2.4068\n",
      "Iteration: 2801; Percent complete: 70.0%; Average loss: 2.4035\n",
      "Iteration: 2802; Percent complete: 70.0%; Average loss: 2.2431\n",
      "Iteration: 2803; Percent complete: 70.1%; Average loss: 2.4601\n",
      "Iteration: 2804; Percent complete: 70.1%; Average loss: 2.4862\n",
      "Iteration: 2805; Percent complete: 70.1%; Average loss: 2.4856\n",
      "Iteration: 2806; Percent complete: 70.2%; Average loss: 2.4377\n",
      "Iteration: 2807; Percent complete: 70.2%; Average loss: 2.3781\n",
      "Iteration: 2808; Percent complete: 70.2%; Average loss: 2.3883\n",
      "Iteration: 2809; Percent complete: 70.2%; Average loss: 2.3836\n",
      "Iteration: 2810; Percent complete: 70.2%; Average loss: 2.5210\n",
      "Iteration: 2811; Percent complete: 70.3%; Average loss: 2.5786\n",
      "Iteration: 2812; Percent complete: 70.3%; Average loss: 2.3493\n",
      "Iteration: 2813; Percent complete: 70.3%; Average loss: 2.4303\n",
      "Iteration: 2814; Percent complete: 70.3%; Average loss: 2.5375\n",
      "Iteration: 2815; Percent complete: 70.4%; Average loss: 2.3978\n",
      "Iteration: 2816; Percent complete: 70.4%; Average loss: 2.2339\n",
      "Iteration: 2817; Percent complete: 70.4%; Average loss: 2.4150\n",
      "Iteration: 2818; Percent complete: 70.5%; Average loss: 2.5624\n",
      "Iteration: 2819; Percent complete: 70.5%; Average loss: 2.5198\n",
      "Iteration: 2820; Percent complete: 70.5%; Average loss: 2.4467\n",
      "Iteration: 2821; Percent complete: 70.5%; Average loss: 2.3649\n",
      "Iteration: 2822; Percent complete: 70.5%; Average loss: 2.5887\n",
      "Iteration: 2823; Percent complete: 70.6%; Average loss: 2.5658\n",
      "Iteration: 2824; Percent complete: 70.6%; Average loss: 2.3322\n",
      "Iteration: 2825; Percent complete: 70.6%; Average loss: 2.4473\n",
      "Iteration: 2826; Percent complete: 70.7%; Average loss: 2.2337\n",
      "Iteration: 2827; Percent complete: 70.7%; Average loss: 2.4650\n",
      "Iteration: 2828; Percent complete: 70.7%; Average loss: 2.3984\n",
      "Iteration: 2829; Percent complete: 70.7%; Average loss: 2.5060\n",
      "Iteration: 2830; Percent complete: 70.8%; Average loss: 2.3146\n",
      "Iteration: 2831; Percent complete: 70.8%; Average loss: 2.4101\n",
      "Iteration: 2832; Percent complete: 70.8%; Average loss: 2.3717\n",
      "Iteration: 2833; Percent complete: 70.8%; Average loss: 2.3683\n",
      "Iteration: 2834; Percent complete: 70.9%; Average loss: 2.3116\n",
      "Iteration: 2835; Percent complete: 70.9%; Average loss: 2.4616\n",
      "Iteration: 2836; Percent complete: 70.9%; Average loss: 2.3232\n",
      "Iteration: 2837; Percent complete: 70.9%; Average loss: 2.2095\n",
      "Iteration: 2838; Percent complete: 71.0%; Average loss: 2.4722\n",
      "Iteration: 2839; Percent complete: 71.0%; Average loss: 2.4920\n",
      "Iteration: 2840; Percent complete: 71.0%; Average loss: 2.5381\n",
      "Iteration: 2841; Percent complete: 71.0%; Average loss: 2.3959\n",
      "Iteration: 2842; Percent complete: 71.0%; Average loss: 2.4117\n",
      "Iteration: 2843; Percent complete: 71.1%; Average loss: 2.4372\n",
      "Iteration: 2844; Percent complete: 71.1%; Average loss: 2.2825\n",
      "Iteration: 2845; Percent complete: 71.1%; Average loss: 2.6118\n",
      "Iteration: 2846; Percent complete: 71.2%; Average loss: 2.3936\n",
      "Iteration: 2847; Percent complete: 71.2%; Average loss: 2.2900\n",
      "Iteration: 2848; Percent complete: 71.2%; Average loss: 2.4415\n",
      "Iteration: 2849; Percent complete: 71.2%; Average loss: 2.5575\n",
      "Iteration: 2850; Percent complete: 71.2%; Average loss: 2.4620\n",
      "Iteration: 2851; Percent complete: 71.3%; Average loss: 2.1820\n",
      "Iteration: 2852; Percent complete: 71.3%; Average loss: 2.5051\n",
      "Iteration: 2853; Percent complete: 71.3%; Average loss: 2.4140\n",
      "Iteration: 2854; Percent complete: 71.4%; Average loss: 2.4892\n",
      "Iteration: 2855; Percent complete: 71.4%; Average loss: 2.2904\n",
      "Iteration: 2856; Percent complete: 71.4%; Average loss: 2.3338\n",
      "Iteration: 2857; Percent complete: 71.4%; Average loss: 2.4702\n",
      "Iteration: 2858; Percent complete: 71.5%; Average loss: 2.3072\n",
      "Iteration: 2859; Percent complete: 71.5%; Average loss: 2.3856\n",
      "Iteration: 2860; Percent complete: 71.5%; Average loss: 2.5252\n",
      "Iteration: 2861; Percent complete: 71.5%; Average loss: 2.5517\n",
      "Iteration: 2862; Percent complete: 71.5%; Average loss: 2.7473\n",
      "Iteration: 2863; Percent complete: 71.6%; Average loss: 2.1958\n",
      "Iteration: 2864; Percent complete: 71.6%; Average loss: 2.3448\n",
      "Iteration: 2865; Percent complete: 71.6%; Average loss: 2.3034\n",
      "Iteration: 2866; Percent complete: 71.7%; Average loss: 2.4630\n",
      "Iteration: 2867; Percent complete: 71.7%; Average loss: 2.3000\n",
      "Iteration: 2868; Percent complete: 71.7%; Average loss: 2.2352\n",
      "Iteration: 2869; Percent complete: 71.7%; Average loss: 2.4554\n",
      "Iteration: 2870; Percent complete: 71.8%; Average loss: 2.2813\n",
      "Iteration: 2871; Percent complete: 71.8%; Average loss: 2.3719\n",
      "Iteration: 2872; Percent complete: 71.8%; Average loss: 2.4976\n",
      "Iteration: 2873; Percent complete: 71.8%; Average loss: 2.3590\n",
      "Iteration: 2874; Percent complete: 71.9%; Average loss: 2.3753\n",
      "Iteration: 2875; Percent complete: 71.9%; Average loss: 2.2707\n",
      "Iteration: 2876; Percent complete: 71.9%; Average loss: 2.3606\n",
      "Iteration: 2877; Percent complete: 71.9%; Average loss: 2.3221\n",
      "Iteration: 2878; Percent complete: 72.0%; Average loss: 2.4113\n",
      "Iteration: 2879; Percent complete: 72.0%; Average loss: 2.5475\n",
      "Iteration: 2880; Percent complete: 72.0%; Average loss: 2.2879\n",
      "Iteration: 2881; Percent complete: 72.0%; Average loss: 2.3677\n",
      "Iteration: 2882; Percent complete: 72.0%; Average loss: 2.4682\n",
      "Iteration: 2883; Percent complete: 72.1%; Average loss: 2.5128\n",
      "Iteration: 2884; Percent complete: 72.1%; Average loss: 2.4914\n",
      "Iteration: 2885; Percent complete: 72.1%; Average loss: 2.4237\n",
      "Iteration: 2886; Percent complete: 72.2%; Average loss: 2.3746\n",
      "Iteration: 2887; Percent complete: 72.2%; Average loss: 2.4003\n",
      "Iteration: 2888; Percent complete: 72.2%; Average loss: 2.3550\n",
      "Iteration: 2889; Percent complete: 72.2%; Average loss: 2.4378\n",
      "Iteration: 2890; Percent complete: 72.2%; Average loss: 2.2759\n",
      "Iteration: 2891; Percent complete: 72.3%; Average loss: 2.4204\n",
      "Iteration: 2892; Percent complete: 72.3%; Average loss: 2.6122\n",
      "Iteration: 2893; Percent complete: 72.3%; Average loss: 2.4784\n",
      "Iteration: 2894; Percent complete: 72.4%; Average loss: 2.3524\n",
      "Iteration: 2895; Percent complete: 72.4%; Average loss: 2.3251\n",
      "Iteration: 2896; Percent complete: 72.4%; Average loss: 2.4863\n",
      "Iteration: 2897; Percent complete: 72.4%; Average loss: 2.4348\n",
      "Iteration: 2898; Percent complete: 72.5%; Average loss: 2.3841\n",
      "Iteration: 2899; Percent complete: 72.5%; Average loss: 2.4869\n",
      "Iteration: 2900; Percent complete: 72.5%; Average loss: 2.4968\n",
      "Iteration: 2901; Percent complete: 72.5%; Average loss: 2.3771\n",
      "Iteration: 2902; Percent complete: 72.5%; Average loss: 2.4599\n",
      "Iteration: 2903; Percent complete: 72.6%; Average loss: 2.6138\n",
      "Iteration: 2904; Percent complete: 72.6%; Average loss: 2.3533\n",
      "Iteration: 2905; Percent complete: 72.6%; Average loss: 2.5771\n",
      "Iteration: 2906; Percent complete: 72.7%; Average loss: 2.4765\n",
      "Iteration: 2907; Percent complete: 72.7%; Average loss: 2.4150\n",
      "Iteration: 2908; Percent complete: 72.7%; Average loss: 2.5059\n",
      "Iteration: 2909; Percent complete: 72.7%; Average loss: 2.3670\n",
      "Iteration: 2910; Percent complete: 72.8%; Average loss: 2.4566\n",
      "Iteration: 2911; Percent complete: 72.8%; Average loss: 2.4894\n",
      "Iteration: 2912; Percent complete: 72.8%; Average loss: 2.5618\n",
      "Iteration: 2913; Percent complete: 72.8%; Average loss: 2.5241\n",
      "Iteration: 2914; Percent complete: 72.9%; Average loss: 2.3443\n",
      "Iteration: 2915; Percent complete: 72.9%; Average loss: 2.3952\n",
      "Iteration: 2916; Percent complete: 72.9%; Average loss: 2.1446\n",
      "Iteration: 2917; Percent complete: 72.9%; Average loss: 2.3390\n",
      "Iteration: 2918; Percent complete: 73.0%; Average loss: 2.3843\n",
      "Iteration: 2919; Percent complete: 73.0%; Average loss: 2.3264\n",
      "Iteration: 2920; Percent complete: 73.0%; Average loss: 2.4290\n",
      "Iteration: 2921; Percent complete: 73.0%; Average loss: 2.4812\n",
      "Iteration: 2922; Percent complete: 73.0%; Average loss: 2.4785\n",
      "Iteration: 2923; Percent complete: 73.1%; Average loss: 2.3877\n",
      "Iteration: 2924; Percent complete: 73.1%; Average loss: 2.2975\n",
      "Iteration: 2925; Percent complete: 73.1%; Average loss: 2.4003\n",
      "Iteration: 2926; Percent complete: 73.2%; Average loss: 2.6156\n",
      "Iteration: 2927; Percent complete: 73.2%; Average loss: 2.4851\n",
      "Iteration: 2928; Percent complete: 73.2%; Average loss: 2.4594\n",
      "Iteration: 2929; Percent complete: 73.2%; Average loss: 2.4274\n",
      "Iteration: 2930; Percent complete: 73.2%; Average loss: 2.5954\n",
      "Iteration: 2931; Percent complete: 73.3%; Average loss: 2.5440\n",
      "Iteration: 2932; Percent complete: 73.3%; Average loss: 2.4459\n",
      "Iteration: 2933; Percent complete: 73.3%; Average loss: 2.1345\n",
      "Iteration: 2934; Percent complete: 73.4%; Average loss: 2.4253\n",
      "Iteration: 2935; Percent complete: 73.4%; Average loss: 2.2222\n",
      "Iteration: 2936; Percent complete: 73.4%; Average loss: 2.4577\n",
      "Iteration: 2937; Percent complete: 73.4%; Average loss: 2.5603\n",
      "Iteration: 2938; Percent complete: 73.5%; Average loss: 2.4709\n",
      "Iteration: 2939; Percent complete: 73.5%; Average loss: 2.7959\n",
      "Iteration: 2940; Percent complete: 73.5%; Average loss: 2.5867\n",
      "Iteration: 2941; Percent complete: 73.5%; Average loss: 2.6125\n",
      "Iteration: 2942; Percent complete: 73.6%; Average loss: 2.4202\n",
      "Iteration: 2943; Percent complete: 73.6%; Average loss: 2.3111\n",
      "Iteration: 2944; Percent complete: 73.6%; Average loss: 2.3373\n",
      "Iteration: 2945; Percent complete: 73.6%; Average loss: 2.4356\n",
      "Iteration: 2946; Percent complete: 73.7%; Average loss: 2.3278\n",
      "Iteration: 2947; Percent complete: 73.7%; Average loss: 2.4188\n",
      "Iteration: 2948; Percent complete: 73.7%; Average loss: 2.4633\n",
      "Iteration: 2949; Percent complete: 73.7%; Average loss: 2.4883\n",
      "Iteration: 2950; Percent complete: 73.8%; Average loss: 2.4017\n",
      "Iteration: 2951; Percent complete: 73.8%; Average loss: 2.4088\n",
      "Iteration: 2952; Percent complete: 73.8%; Average loss: 2.3238\n",
      "Iteration: 2953; Percent complete: 73.8%; Average loss: 2.3765\n",
      "Iteration: 2954; Percent complete: 73.9%; Average loss: 2.2914\n",
      "Iteration: 2955; Percent complete: 73.9%; Average loss: 2.4125\n",
      "Iteration: 2956; Percent complete: 73.9%; Average loss: 2.4349\n",
      "Iteration: 2957; Percent complete: 73.9%; Average loss: 2.5075\n",
      "Iteration: 2958; Percent complete: 74.0%; Average loss: 2.5776\n",
      "Iteration: 2959; Percent complete: 74.0%; Average loss: 2.4716\n",
      "Iteration: 2960; Percent complete: 74.0%; Average loss: 2.5589\n",
      "Iteration: 2961; Percent complete: 74.0%; Average loss: 2.3948\n",
      "Iteration: 2962; Percent complete: 74.1%; Average loss: 2.2845\n",
      "Iteration: 2963; Percent complete: 74.1%; Average loss: 2.2420\n",
      "Iteration: 2964; Percent complete: 74.1%; Average loss: 2.4641\n",
      "Iteration: 2965; Percent complete: 74.1%; Average loss: 2.3338\n",
      "Iteration: 2966; Percent complete: 74.2%; Average loss: 2.4690\n",
      "Iteration: 2967; Percent complete: 74.2%; Average loss: 2.5729\n",
      "Iteration: 2968; Percent complete: 74.2%; Average loss: 2.3053\n",
      "Iteration: 2969; Percent complete: 74.2%; Average loss: 2.2913\n",
      "Iteration: 2970; Percent complete: 74.2%; Average loss: 2.3195\n",
      "Iteration: 2971; Percent complete: 74.3%; Average loss: 2.5178\n",
      "Iteration: 2972; Percent complete: 74.3%; Average loss: 2.3094\n",
      "Iteration: 2973; Percent complete: 74.3%; Average loss: 2.3518\n",
      "Iteration: 2974; Percent complete: 74.4%; Average loss: 2.3413\n",
      "Iteration: 2975; Percent complete: 74.4%; Average loss: 2.3174\n",
      "Iteration: 2976; Percent complete: 74.4%; Average loss: 2.4852\n",
      "Iteration: 2977; Percent complete: 74.4%; Average loss: 2.5190\n",
      "Iteration: 2978; Percent complete: 74.5%; Average loss: 2.5077\n",
      "Iteration: 2979; Percent complete: 74.5%; Average loss: 2.2439\n",
      "Iteration: 2980; Percent complete: 74.5%; Average loss: 2.4148\n",
      "Iteration: 2981; Percent complete: 74.5%; Average loss: 2.6106\n",
      "Iteration: 2982; Percent complete: 74.6%; Average loss: 2.3708\n",
      "Iteration: 2983; Percent complete: 74.6%; Average loss: 2.5273\n",
      "Iteration: 2984; Percent complete: 74.6%; Average loss: 2.3532\n",
      "Iteration: 2985; Percent complete: 74.6%; Average loss: 2.4002\n",
      "Iteration: 2986; Percent complete: 74.7%; Average loss: 2.5732\n",
      "Iteration: 2987; Percent complete: 74.7%; Average loss: 2.4489\n",
      "Iteration: 2988; Percent complete: 74.7%; Average loss: 2.2810\n",
      "Iteration: 2989; Percent complete: 74.7%; Average loss: 2.2709\n",
      "Iteration: 2990; Percent complete: 74.8%; Average loss: 2.3858\n",
      "Iteration: 2991; Percent complete: 74.8%; Average loss: 2.2819\n",
      "Iteration: 2992; Percent complete: 74.8%; Average loss: 2.4181\n",
      "Iteration: 2993; Percent complete: 74.8%; Average loss: 2.4188\n",
      "Iteration: 2994; Percent complete: 74.9%; Average loss: 2.4411\n",
      "Iteration: 2995; Percent complete: 74.9%; Average loss: 2.3129\n",
      "Iteration: 2996; Percent complete: 74.9%; Average loss: 2.2977\n",
      "Iteration: 2997; Percent complete: 74.9%; Average loss: 2.2836\n",
      "Iteration: 2998; Percent complete: 75.0%; Average loss: 2.3752\n",
      "Iteration: 2999; Percent complete: 75.0%; Average loss: 2.4614\n",
      "Iteration: 3000; Percent complete: 75.0%; Average loss: 2.3629\n",
      "Iteration: 3001; Percent complete: 75.0%; Average loss: 2.4341\n",
      "Iteration: 3002; Percent complete: 75.0%; Average loss: 2.4929\n",
      "Iteration: 3003; Percent complete: 75.1%; Average loss: 2.4254\n",
      "Iteration: 3004; Percent complete: 75.1%; Average loss: 2.4394\n",
      "Iteration: 3005; Percent complete: 75.1%; Average loss: 2.2048\n",
      "Iteration: 3006; Percent complete: 75.1%; Average loss: 2.3981\n",
      "Iteration: 3007; Percent complete: 75.2%; Average loss: 2.3745\n",
      "Iteration: 3008; Percent complete: 75.2%; Average loss: 2.3102\n",
      "Iteration: 3009; Percent complete: 75.2%; Average loss: 2.4635\n",
      "Iteration: 3010; Percent complete: 75.2%; Average loss: 2.4202\n",
      "Iteration: 3011; Percent complete: 75.3%; Average loss: 2.7340\n",
      "Iteration: 3012; Percent complete: 75.3%; Average loss: 2.5004\n",
      "Iteration: 3013; Percent complete: 75.3%; Average loss: 2.2049\n",
      "Iteration: 3014; Percent complete: 75.3%; Average loss: 2.6334\n",
      "Iteration: 3015; Percent complete: 75.4%; Average loss: 2.3132\n",
      "Iteration: 3016; Percent complete: 75.4%; Average loss: 2.3680\n",
      "Iteration: 3017; Percent complete: 75.4%; Average loss: 2.3303\n",
      "Iteration: 3018; Percent complete: 75.4%; Average loss: 2.3160\n",
      "Iteration: 3019; Percent complete: 75.5%; Average loss: 2.3997\n",
      "Iteration: 3020; Percent complete: 75.5%; Average loss: 2.2819\n",
      "Iteration: 3021; Percent complete: 75.5%; Average loss: 2.3737\n",
      "Iteration: 3022; Percent complete: 75.5%; Average loss: 2.4880\n",
      "Iteration: 3023; Percent complete: 75.6%; Average loss: 2.2845\n",
      "Iteration: 3024; Percent complete: 75.6%; Average loss: 2.4679\n",
      "Iteration: 3025; Percent complete: 75.6%; Average loss: 2.5027\n",
      "Iteration: 3026; Percent complete: 75.6%; Average loss: 2.4450\n",
      "Iteration: 3027; Percent complete: 75.7%; Average loss: 2.5858\n",
      "Iteration: 3028; Percent complete: 75.7%; Average loss: 2.4162\n",
      "Iteration: 3029; Percent complete: 75.7%; Average loss: 2.2079\n",
      "Iteration: 3030; Percent complete: 75.8%; Average loss: 2.4701\n",
      "Iteration: 3031; Percent complete: 75.8%; Average loss: 2.3539\n",
      "Iteration: 3032; Percent complete: 75.8%; Average loss: 2.4067\n",
      "Iteration: 3033; Percent complete: 75.8%; Average loss: 2.2961\n",
      "Iteration: 3034; Percent complete: 75.8%; Average loss: 2.4235\n",
      "Iteration: 3035; Percent complete: 75.9%; Average loss: 2.4799\n",
      "Iteration: 3036; Percent complete: 75.9%; Average loss: 2.5585\n",
      "Iteration: 3037; Percent complete: 75.9%; Average loss: 2.3737\n",
      "Iteration: 3038; Percent complete: 75.9%; Average loss: 2.2055\n",
      "Iteration: 3039; Percent complete: 76.0%; Average loss: 2.4091\n",
      "Iteration: 3040; Percent complete: 76.0%; Average loss: 2.3315\n",
      "Iteration: 3041; Percent complete: 76.0%; Average loss: 2.4616\n",
      "Iteration: 3042; Percent complete: 76.0%; Average loss: 2.4286\n",
      "Iteration: 3043; Percent complete: 76.1%; Average loss: 2.3454\n",
      "Iteration: 3044; Percent complete: 76.1%; Average loss: 2.3162\n",
      "Iteration: 3045; Percent complete: 76.1%; Average loss: 2.4915\n",
      "Iteration: 3046; Percent complete: 76.1%; Average loss: 2.2812\n",
      "Iteration: 3047; Percent complete: 76.2%; Average loss: 2.5700\n",
      "Iteration: 3048; Percent complete: 76.2%; Average loss: 2.1788\n",
      "Iteration: 3049; Percent complete: 76.2%; Average loss: 2.3785\n",
      "Iteration: 3050; Percent complete: 76.2%; Average loss: 2.3489\n",
      "Iteration: 3051; Percent complete: 76.3%; Average loss: 2.5182\n",
      "Iteration: 3052; Percent complete: 76.3%; Average loss: 2.4065\n",
      "Iteration: 3053; Percent complete: 76.3%; Average loss: 2.4084\n",
      "Iteration: 3054; Percent complete: 76.3%; Average loss: 2.3146\n",
      "Iteration: 3055; Percent complete: 76.4%; Average loss: 2.3774\n",
      "Iteration: 3056; Percent complete: 76.4%; Average loss: 2.3213\n",
      "Iteration: 3057; Percent complete: 76.4%; Average loss: 2.0713\n",
      "Iteration: 3058; Percent complete: 76.4%; Average loss: 2.4294\n",
      "Iteration: 3059; Percent complete: 76.5%; Average loss: 2.5815\n",
      "Iteration: 3060; Percent complete: 76.5%; Average loss: 2.4753\n",
      "Iteration: 3061; Percent complete: 76.5%; Average loss: 2.0323\n",
      "Iteration: 3062; Percent complete: 76.5%; Average loss: 2.3204\n",
      "Iteration: 3063; Percent complete: 76.6%; Average loss: 2.3289\n",
      "Iteration: 3064; Percent complete: 76.6%; Average loss: 2.3421\n",
      "Iteration: 3065; Percent complete: 76.6%; Average loss: 2.2550\n",
      "Iteration: 3066; Percent complete: 76.6%; Average loss: 2.4254\n",
      "Iteration: 3067; Percent complete: 76.7%; Average loss: 2.3066\n",
      "Iteration: 3068; Percent complete: 76.7%; Average loss: 2.5035\n",
      "Iteration: 3069; Percent complete: 76.7%; Average loss: 2.4546\n",
      "Iteration: 3070; Percent complete: 76.8%; Average loss: 2.5453\n",
      "Iteration: 3071; Percent complete: 76.8%; Average loss: 2.4689\n",
      "Iteration: 3072; Percent complete: 76.8%; Average loss: 2.2828\n",
      "Iteration: 3073; Percent complete: 76.8%; Average loss: 2.3752\n",
      "Iteration: 3074; Percent complete: 76.8%; Average loss: 2.2953\n",
      "Iteration: 3075; Percent complete: 76.9%; Average loss: 2.2163\n",
      "Iteration: 3076; Percent complete: 76.9%; Average loss: 2.6176\n",
      "Iteration: 3077; Percent complete: 76.9%; Average loss: 2.3293\n",
      "Iteration: 3078; Percent complete: 77.0%; Average loss: 2.2459\n",
      "Iteration: 3079; Percent complete: 77.0%; Average loss: 2.4001\n",
      "Iteration: 3080; Percent complete: 77.0%; Average loss: 2.3139\n",
      "Iteration: 3081; Percent complete: 77.0%; Average loss: 2.2571\n",
      "Iteration: 3082; Percent complete: 77.0%; Average loss: 2.6576\n",
      "Iteration: 3083; Percent complete: 77.1%; Average loss: 2.4071\n",
      "Iteration: 3084; Percent complete: 77.1%; Average loss: 2.3559\n",
      "Iteration: 3085; Percent complete: 77.1%; Average loss: 2.3411\n",
      "Iteration: 3086; Percent complete: 77.1%; Average loss: 2.3119\n",
      "Iteration: 3087; Percent complete: 77.2%; Average loss: 2.4557\n",
      "Iteration: 3088; Percent complete: 77.2%; Average loss: 2.2367\n",
      "Iteration: 3089; Percent complete: 77.2%; Average loss: 2.1475\n",
      "Iteration: 3090; Percent complete: 77.2%; Average loss: 2.5017\n",
      "Iteration: 3091; Percent complete: 77.3%; Average loss: 2.4119\n",
      "Iteration: 3092; Percent complete: 77.3%; Average loss: 2.6174\n",
      "Iteration: 3093; Percent complete: 77.3%; Average loss: 2.4099\n",
      "Iteration: 3094; Percent complete: 77.3%; Average loss: 2.4919\n",
      "Iteration: 3095; Percent complete: 77.4%; Average loss: 2.4563\n",
      "Iteration: 3096; Percent complete: 77.4%; Average loss: 2.5906\n",
      "Iteration: 3097; Percent complete: 77.4%; Average loss: 2.2860\n",
      "Iteration: 3098; Percent complete: 77.5%; Average loss: 2.2328\n",
      "Iteration: 3099; Percent complete: 77.5%; Average loss: 2.4591\n",
      "Iteration: 3100; Percent complete: 77.5%; Average loss: 2.2994\n",
      "Iteration: 3101; Percent complete: 77.5%; Average loss: 2.6666\n",
      "Iteration: 3102; Percent complete: 77.5%; Average loss: 2.4052\n",
      "Iteration: 3103; Percent complete: 77.6%; Average loss: 2.3149\n",
      "Iteration: 3104; Percent complete: 77.6%; Average loss: 2.3792\n",
      "Iteration: 3105; Percent complete: 77.6%; Average loss: 2.4320\n",
      "Iteration: 3106; Percent complete: 77.6%; Average loss: 2.4925\n",
      "Iteration: 3107; Percent complete: 77.7%; Average loss: 2.3975\n",
      "Iteration: 3108; Percent complete: 77.7%; Average loss: 2.3171\n",
      "Iteration: 3109; Percent complete: 77.7%; Average loss: 2.4919\n",
      "Iteration: 3110; Percent complete: 77.8%; Average loss: 2.3390\n",
      "Iteration: 3111; Percent complete: 77.8%; Average loss: 2.4038\n",
      "Iteration: 3112; Percent complete: 77.8%; Average loss: 2.3388\n",
      "Iteration: 3113; Percent complete: 77.8%; Average loss: 2.2562\n",
      "Iteration: 3114; Percent complete: 77.8%; Average loss: 2.4617\n",
      "Iteration: 3115; Percent complete: 77.9%; Average loss: 2.3625\n",
      "Iteration: 3116; Percent complete: 77.9%; Average loss: 2.4658\n",
      "Iteration: 3117; Percent complete: 77.9%; Average loss: 2.3651\n",
      "Iteration: 3118; Percent complete: 78.0%; Average loss: 2.4336\n",
      "Iteration: 3119; Percent complete: 78.0%; Average loss: 2.4872\n",
      "Iteration: 3120; Percent complete: 78.0%; Average loss: 2.6690\n",
      "Iteration: 3121; Percent complete: 78.0%; Average loss: 2.6550\n",
      "Iteration: 3122; Percent complete: 78.0%; Average loss: 2.2853\n",
      "Iteration: 3123; Percent complete: 78.1%; Average loss: 2.6279\n",
      "Iteration: 3124; Percent complete: 78.1%; Average loss: 2.2030\n",
      "Iteration: 3125; Percent complete: 78.1%; Average loss: 2.5723\n",
      "Iteration: 3126; Percent complete: 78.1%; Average loss: 2.3321\n",
      "Iteration: 3127; Percent complete: 78.2%; Average loss: 2.2409\n",
      "Iteration: 3128; Percent complete: 78.2%; Average loss: 2.4011\n",
      "Iteration: 3129; Percent complete: 78.2%; Average loss: 2.4248\n",
      "Iteration: 3130; Percent complete: 78.2%; Average loss: 2.3834\n",
      "Iteration: 3131; Percent complete: 78.3%; Average loss: 2.3862\n",
      "Iteration: 3132; Percent complete: 78.3%; Average loss: 2.4637\n",
      "Iteration: 3133; Percent complete: 78.3%; Average loss: 2.3608\n",
      "Iteration: 3134; Percent complete: 78.3%; Average loss: 2.4319\n",
      "Iteration: 3135; Percent complete: 78.4%; Average loss: 2.3205\n",
      "Iteration: 3136; Percent complete: 78.4%; Average loss: 2.4853\n",
      "Iteration: 3137; Percent complete: 78.4%; Average loss: 2.4228\n",
      "Iteration: 3138; Percent complete: 78.5%; Average loss: 2.3482\n",
      "Iteration: 3139; Percent complete: 78.5%; Average loss: 2.3210\n",
      "Iteration: 3140; Percent complete: 78.5%; Average loss: 2.4424\n",
      "Iteration: 3141; Percent complete: 78.5%; Average loss: 2.3930\n",
      "Iteration: 3142; Percent complete: 78.5%; Average loss: 2.4569\n",
      "Iteration: 3143; Percent complete: 78.6%; Average loss: 2.2849\n",
      "Iteration: 3144; Percent complete: 78.6%; Average loss: 2.5924\n",
      "Iteration: 3145; Percent complete: 78.6%; Average loss: 2.4922\n",
      "Iteration: 3146; Percent complete: 78.6%; Average loss: 2.1701\n",
      "Iteration: 3147; Percent complete: 78.7%; Average loss: 2.3792\n",
      "Iteration: 3148; Percent complete: 78.7%; Average loss: 2.3345\n",
      "Iteration: 3149; Percent complete: 78.7%; Average loss: 2.3607\n",
      "Iteration: 3150; Percent complete: 78.8%; Average loss: 2.2852\n",
      "Iteration: 3151; Percent complete: 78.8%; Average loss: 2.4697\n",
      "Iteration: 3152; Percent complete: 78.8%; Average loss: 2.4500\n",
      "Iteration: 3153; Percent complete: 78.8%; Average loss: 2.4137\n",
      "Iteration: 3154; Percent complete: 78.8%; Average loss: 2.3916\n",
      "Iteration: 3155; Percent complete: 78.9%; Average loss: 2.2741\n",
      "Iteration: 3156; Percent complete: 78.9%; Average loss: 2.3595\n",
      "Iteration: 3157; Percent complete: 78.9%; Average loss: 2.4031\n",
      "Iteration: 3158; Percent complete: 79.0%; Average loss: 2.2868\n",
      "Iteration: 3159; Percent complete: 79.0%; Average loss: 2.4685\n",
      "Iteration: 3160; Percent complete: 79.0%; Average loss: 2.2453\n",
      "Iteration: 3161; Percent complete: 79.0%; Average loss: 2.3553\n",
      "Iteration: 3162; Percent complete: 79.0%; Average loss: 2.3132\n",
      "Iteration: 3163; Percent complete: 79.1%; Average loss: 2.4193\n",
      "Iteration: 3164; Percent complete: 79.1%; Average loss: 2.4255\n",
      "Iteration: 3165; Percent complete: 79.1%; Average loss: 2.1345\n",
      "Iteration: 3166; Percent complete: 79.1%; Average loss: 2.1560\n",
      "Iteration: 3167; Percent complete: 79.2%; Average loss: 2.4554\n",
      "Iteration: 3168; Percent complete: 79.2%; Average loss: 2.3374\n",
      "Iteration: 3169; Percent complete: 79.2%; Average loss: 2.3166\n",
      "Iteration: 3170; Percent complete: 79.2%; Average loss: 2.2040\n",
      "Iteration: 3171; Percent complete: 79.3%; Average loss: 2.4540\n",
      "Iteration: 3172; Percent complete: 79.3%; Average loss: 2.2661\n",
      "Iteration: 3173; Percent complete: 79.3%; Average loss: 2.3680\n",
      "Iteration: 3174; Percent complete: 79.3%; Average loss: 2.3869\n",
      "Iteration: 3175; Percent complete: 79.4%; Average loss: 2.2794\n",
      "Iteration: 3176; Percent complete: 79.4%; Average loss: 2.5722\n",
      "Iteration: 3177; Percent complete: 79.4%; Average loss: 2.3718\n",
      "Iteration: 3178; Percent complete: 79.5%; Average loss: 2.2758\n",
      "Iteration: 3179; Percent complete: 79.5%; Average loss: 2.4523\n",
      "Iteration: 3180; Percent complete: 79.5%; Average loss: 2.2656\n",
      "Iteration: 3181; Percent complete: 79.5%; Average loss: 2.3553\n",
      "Iteration: 3182; Percent complete: 79.5%; Average loss: 2.2892\n",
      "Iteration: 3183; Percent complete: 79.6%; Average loss: 2.2935\n",
      "Iteration: 3184; Percent complete: 79.6%; Average loss: 2.3070\n",
      "Iteration: 3185; Percent complete: 79.6%; Average loss: 2.5176\n",
      "Iteration: 3186; Percent complete: 79.7%; Average loss: 2.2035\n",
      "Iteration: 3187; Percent complete: 79.7%; Average loss: 2.3067\n",
      "Iteration: 3188; Percent complete: 79.7%; Average loss: 2.3130\n",
      "Iteration: 3189; Percent complete: 79.7%; Average loss: 2.5378\n",
      "Iteration: 3190; Percent complete: 79.8%; Average loss: 2.4452\n",
      "Iteration: 3191; Percent complete: 79.8%; Average loss: 2.6078\n",
      "Iteration: 3192; Percent complete: 79.8%; Average loss: 2.4692\n",
      "Iteration: 3193; Percent complete: 79.8%; Average loss: 2.2726\n",
      "Iteration: 3194; Percent complete: 79.8%; Average loss: 2.3372\n",
      "Iteration: 3195; Percent complete: 79.9%; Average loss: 2.3593\n",
      "Iteration: 3196; Percent complete: 79.9%; Average loss: 2.3622\n",
      "Iteration: 3197; Percent complete: 79.9%; Average loss: 2.4198\n",
      "Iteration: 3198; Percent complete: 80.0%; Average loss: 2.4267\n",
      "Iteration: 3199; Percent complete: 80.0%; Average loss: 2.2726\n",
      "Iteration: 3200; Percent complete: 80.0%; Average loss: 2.1862\n",
      "Iteration: 3201; Percent complete: 80.0%; Average loss: 2.2617\n",
      "Iteration: 3202; Percent complete: 80.0%; Average loss: 2.3397\n",
      "Iteration: 3203; Percent complete: 80.1%; Average loss: 2.3630\n",
      "Iteration: 3204; Percent complete: 80.1%; Average loss: 2.5527\n",
      "Iteration: 3205; Percent complete: 80.1%; Average loss: 2.2442\n",
      "Iteration: 3206; Percent complete: 80.2%; Average loss: 2.2809\n",
      "Iteration: 3207; Percent complete: 80.2%; Average loss: 2.3422\n",
      "Iteration: 3208; Percent complete: 80.2%; Average loss: 2.2541\n",
      "Iteration: 3209; Percent complete: 80.2%; Average loss: 2.2604\n",
      "Iteration: 3210; Percent complete: 80.2%; Average loss: 2.4588\n",
      "Iteration: 3211; Percent complete: 80.3%; Average loss: 2.5394\n",
      "Iteration: 3212; Percent complete: 80.3%; Average loss: 2.2200\n",
      "Iteration: 3213; Percent complete: 80.3%; Average loss: 2.3762\n",
      "Iteration: 3214; Percent complete: 80.3%; Average loss: 2.1300\n",
      "Iteration: 3215; Percent complete: 80.4%; Average loss: 2.4768\n",
      "Iteration: 3216; Percent complete: 80.4%; Average loss: 2.4178\n",
      "Iteration: 3217; Percent complete: 80.4%; Average loss: 2.1066\n",
      "Iteration: 3218; Percent complete: 80.5%; Average loss: 2.2653\n",
      "Iteration: 3219; Percent complete: 80.5%; Average loss: 2.6036\n",
      "Iteration: 3220; Percent complete: 80.5%; Average loss: 2.2544\n",
      "Iteration: 3221; Percent complete: 80.5%; Average loss: 2.2062\n",
      "Iteration: 3222; Percent complete: 80.5%; Average loss: 2.1834\n",
      "Iteration: 3223; Percent complete: 80.6%; Average loss: 2.3047\n",
      "Iteration: 3224; Percent complete: 80.6%; Average loss: 2.2688\n",
      "Iteration: 3225; Percent complete: 80.6%; Average loss: 2.2239\n",
      "Iteration: 3226; Percent complete: 80.7%; Average loss: 2.5700\n",
      "Iteration: 3227; Percent complete: 80.7%; Average loss: 2.4127\n",
      "Iteration: 3228; Percent complete: 80.7%; Average loss: 2.3859\n",
      "Iteration: 3229; Percent complete: 80.7%; Average loss: 2.4373\n",
      "Iteration: 3230; Percent complete: 80.8%; Average loss: 2.5209\n",
      "Iteration: 3231; Percent complete: 80.8%; Average loss: 2.4847\n",
      "Iteration: 3232; Percent complete: 80.8%; Average loss: 2.0924\n",
      "Iteration: 3233; Percent complete: 80.8%; Average loss: 2.1305\n",
      "Iteration: 3234; Percent complete: 80.8%; Average loss: 2.6180\n",
      "Iteration: 3235; Percent complete: 80.9%; Average loss: 2.3331\n",
      "Iteration: 3236; Percent complete: 80.9%; Average loss: 2.0504\n",
      "Iteration: 3237; Percent complete: 80.9%; Average loss: 2.1374\n",
      "Iteration: 3238; Percent complete: 81.0%; Average loss: 2.4937\n",
      "Iteration: 3239; Percent complete: 81.0%; Average loss: 2.0247\n",
      "Iteration: 3240; Percent complete: 81.0%; Average loss: 2.5212\n",
      "Iteration: 3241; Percent complete: 81.0%; Average loss: 2.4565\n",
      "Iteration: 3242; Percent complete: 81.0%; Average loss: 2.4155\n",
      "Iteration: 3243; Percent complete: 81.1%; Average loss: 2.3335\n",
      "Iteration: 3244; Percent complete: 81.1%; Average loss: 2.0207\n",
      "Iteration: 3245; Percent complete: 81.1%; Average loss: 2.4493\n",
      "Iteration: 3246; Percent complete: 81.2%; Average loss: 2.3629\n",
      "Iteration: 3247; Percent complete: 81.2%; Average loss: 2.4743\n",
      "Iteration: 3248; Percent complete: 81.2%; Average loss: 2.1957\n",
      "Iteration: 3249; Percent complete: 81.2%; Average loss: 2.3037\n",
      "Iteration: 3250; Percent complete: 81.2%; Average loss: 2.2718\n",
      "Iteration: 3251; Percent complete: 81.3%; Average loss: 2.2642\n",
      "Iteration: 3252; Percent complete: 81.3%; Average loss: 2.3181\n",
      "Iteration: 3253; Percent complete: 81.3%; Average loss: 2.4332\n",
      "Iteration: 3254; Percent complete: 81.3%; Average loss: 2.2812\n",
      "Iteration: 3255; Percent complete: 81.4%; Average loss: 2.4505\n",
      "Iteration: 3256; Percent complete: 81.4%; Average loss: 2.3131\n",
      "Iteration: 3257; Percent complete: 81.4%; Average loss: 2.4079\n",
      "Iteration: 3258; Percent complete: 81.5%; Average loss: 2.3300\n",
      "Iteration: 3259; Percent complete: 81.5%; Average loss: 2.1941\n",
      "Iteration: 3260; Percent complete: 81.5%; Average loss: 2.2307\n",
      "Iteration: 3261; Percent complete: 81.5%; Average loss: 2.3024\n",
      "Iteration: 3262; Percent complete: 81.5%; Average loss: 2.3955\n",
      "Iteration: 3263; Percent complete: 81.6%; Average loss: 2.2102\n",
      "Iteration: 3264; Percent complete: 81.6%; Average loss: 2.4996\n",
      "Iteration: 3265; Percent complete: 81.6%; Average loss: 2.2297\n",
      "Iteration: 3266; Percent complete: 81.7%; Average loss: 2.3088\n",
      "Iteration: 3267; Percent complete: 81.7%; Average loss: 2.3366\n",
      "Iteration: 3268; Percent complete: 81.7%; Average loss: 2.3400\n",
      "Iteration: 3269; Percent complete: 81.7%; Average loss: 2.3685\n",
      "Iteration: 3270; Percent complete: 81.8%; Average loss: 2.1725\n",
      "Iteration: 3271; Percent complete: 81.8%; Average loss: 2.2462\n",
      "Iteration: 3272; Percent complete: 81.8%; Average loss: 2.2682\n",
      "Iteration: 3273; Percent complete: 81.8%; Average loss: 2.3988\n",
      "Iteration: 3274; Percent complete: 81.8%; Average loss: 2.4124\n",
      "Iteration: 3275; Percent complete: 81.9%; Average loss: 2.1989\n",
      "Iteration: 3276; Percent complete: 81.9%; Average loss: 2.2848\n",
      "Iteration: 3277; Percent complete: 81.9%; Average loss: 2.1252\n",
      "Iteration: 3278; Percent complete: 82.0%; Average loss: 2.4123\n",
      "Iteration: 3279; Percent complete: 82.0%; Average loss: 2.2436\n",
      "Iteration: 3280; Percent complete: 82.0%; Average loss: 2.4513\n",
      "Iteration: 3281; Percent complete: 82.0%; Average loss: 2.1332\n",
      "Iteration: 3282; Percent complete: 82.0%; Average loss: 2.4817\n",
      "Iteration: 3283; Percent complete: 82.1%; Average loss: 2.3298\n",
      "Iteration: 3284; Percent complete: 82.1%; Average loss: 2.2313\n",
      "Iteration: 3285; Percent complete: 82.1%; Average loss: 2.3113\n",
      "Iteration: 3286; Percent complete: 82.2%; Average loss: 2.6404\n",
      "Iteration: 3287; Percent complete: 82.2%; Average loss: 2.6117\n",
      "Iteration: 3288; Percent complete: 82.2%; Average loss: 2.1777\n",
      "Iteration: 3289; Percent complete: 82.2%; Average loss: 2.2560\n",
      "Iteration: 3290; Percent complete: 82.2%; Average loss: 2.2762\n",
      "Iteration: 3291; Percent complete: 82.3%; Average loss: 2.1007\n",
      "Iteration: 3292; Percent complete: 82.3%; Average loss: 2.4012\n",
      "Iteration: 3293; Percent complete: 82.3%; Average loss: 2.3433\n",
      "Iteration: 3294; Percent complete: 82.3%; Average loss: 2.3242\n",
      "Iteration: 3295; Percent complete: 82.4%; Average loss: 2.4234\n",
      "Iteration: 3296; Percent complete: 82.4%; Average loss: 2.2718\n",
      "Iteration: 3297; Percent complete: 82.4%; Average loss: 2.4569\n",
      "Iteration: 3298; Percent complete: 82.5%; Average loss: 2.3727\n",
      "Iteration: 3299; Percent complete: 82.5%; Average loss: 2.4183\n",
      "Iteration: 3300; Percent complete: 82.5%; Average loss: 2.2995\n",
      "Iteration: 3301; Percent complete: 82.5%; Average loss: 2.2993\n",
      "Iteration: 3302; Percent complete: 82.5%; Average loss: 2.0904\n",
      "Iteration: 3303; Percent complete: 82.6%; Average loss: 2.3267\n",
      "Iteration: 3304; Percent complete: 82.6%; Average loss: 2.4300\n",
      "Iteration: 3305; Percent complete: 82.6%; Average loss: 2.4414\n",
      "Iteration: 3306; Percent complete: 82.7%; Average loss: 2.4147\n",
      "Iteration: 3307; Percent complete: 82.7%; Average loss: 2.2717\n",
      "Iteration: 3308; Percent complete: 82.7%; Average loss: 2.2431\n",
      "Iteration: 3309; Percent complete: 82.7%; Average loss: 2.2891\n",
      "Iteration: 3310; Percent complete: 82.8%; Average loss: 2.4009\n",
      "Iteration: 3311; Percent complete: 82.8%; Average loss: 2.4171\n",
      "Iteration: 3312; Percent complete: 82.8%; Average loss: 2.3831\n",
      "Iteration: 3313; Percent complete: 82.8%; Average loss: 2.2471\n",
      "Iteration: 3314; Percent complete: 82.8%; Average loss: 2.4091\n",
      "Iteration: 3315; Percent complete: 82.9%; Average loss: 2.5852\n",
      "Iteration: 3316; Percent complete: 82.9%; Average loss: 2.3029\n",
      "Iteration: 3317; Percent complete: 82.9%; Average loss: 2.2814\n",
      "Iteration: 3318; Percent complete: 83.0%; Average loss: 2.3870\n",
      "Iteration: 3319; Percent complete: 83.0%; Average loss: 2.5232\n",
      "Iteration: 3320; Percent complete: 83.0%; Average loss: 2.6202\n",
      "Iteration: 3321; Percent complete: 83.0%; Average loss: 2.2944\n",
      "Iteration: 3322; Percent complete: 83.0%; Average loss: 2.3918\n",
      "Iteration: 3323; Percent complete: 83.1%; Average loss: 2.2098\n",
      "Iteration: 3324; Percent complete: 83.1%; Average loss: 2.4574\n",
      "Iteration: 3325; Percent complete: 83.1%; Average loss: 2.3828\n",
      "Iteration: 3326; Percent complete: 83.2%; Average loss: 2.1716\n",
      "Iteration: 3327; Percent complete: 83.2%; Average loss: 2.2479\n",
      "Iteration: 3328; Percent complete: 83.2%; Average loss: 2.4925\n",
      "Iteration: 3329; Percent complete: 83.2%; Average loss: 2.3304\n",
      "Iteration: 3330; Percent complete: 83.2%; Average loss: 2.2597\n",
      "Iteration: 3331; Percent complete: 83.3%; Average loss: 2.3324\n",
      "Iteration: 3332; Percent complete: 83.3%; Average loss: 2.4267\n",
      "Iteration: 3333; Percent complete: 83.3%; Average loss: 2.3900\n",
      "Iteration: 3334; Percent complete: 83.4%; Average loss: 2.2987\n",
      "Iteration: 3335; Percent complete: 83.4%; Average loss: 2.2554\n",
      "Iteration: 3336; Percent complete: 83.4%; Average loss: 2.3834\n",
      "Iteration: 3337; Percent complete: 83.4%; Average loss: 2.2802\n",
      "Iteration: 3338; Percent complete: 83.5%; Average loss: 2.4122\n",
      "Iteration: 3339; Percent complete: 83.5%; Average loss: 2.2416\n",
      "Iteration: 3340; Percent complete: 83.5%; Average loss: 2.3363\n",
      "Iteration: 3341; Percent complete: 83.5%; Average loss: 2.1120\n",
      "Iteration: 3342; Percent complete: 83.5%; Average loss: 2.3403\n",
      "Iteration: 3343; Percent complete: 83.6%; Average loss: 2.3318\n",
      "Iteration: 3344; Percent complete: 83.6%; Average loss: 2.2246\n",
      "Iteration: 3345; Percent complete: 83.6%; Average loss: 2.3170\n",
      "Iteration: 3346; Percent complete: 83.7%; Average loss: 2.4509\n",
      "Iteration: 3347; Percent complete: 83.7%; Average loss: 2.3915\n",
      "Iteration: 3348; Percent complete: 83.7%; Average loss: 2.3031\n",
      "Iteration: 3349; Percent complete: 83.7%; Average loss: 2.2275\n",
      "Iteration: 3350; Percent complete: 83.8%; Average loss: 2.2496\n",
      "Iteration: 3351; Percent complete: 83.8%; Average loss: 2.2093\n",
      "Iteration: 3352; Percent complete: 83.8%; Average loss: 2.3611\n",
      "Iteration: 3353; Percent complete: 83.8%; Average loss: 2.3655\n",
      "Iteration: 3354; Percent complete: 83.9%; Average loss: 2.3462\n",
      "Iteration: 3355; Percent complete: 83.9%; Average loss: 2.3726\n",
      "Iteration: 3356; Percent complete: 83.9%; Average loss: 2.4771\n",
      "Iteration: 3357; Percent complete: 83.9%; Average loss: 2.5025\n",
      "Iteration: 3358; Percent complete: 84.0%; Average loss: 2.2316\n",
      "Iteration: 3359; Percent complete: 84.0%; Average loss: 2.1607\n",
      "Iteration: 3360; Percent complete: 84.0%; Average loss: 2.1163\n",
      "Iteration: 3361; Percent complete: 84.0%; Average loss: 2.3643\n",
      "Iteration: 3362; Percent complete: 84.0%; Average loss: 2.3946\n",
      "Iteration: 3363; Percent complete: 84.1%; Average loss: 2.3467\n",
      "Iteration: 3364; Percent complete: 84.1%; Average loss: 2.1776\n",
      "Iteration: 3365; Percent complete: 84.1%; Average loss: 2.1368\n",
      "Iteration: 3366; Percent complete: 84.2%; Average loss: 2.3174\n",
      "Iteration: 3367; Percent complete: 84.2%; Average loss: 2.3179\n",
      "Iteration: 3368; Percent complete: 84.2%; Average loss: 2.4724\n",
      "Iteration: 3369; Percent complete: 84.2%; Average loss: 2.2577\n",
      "Iteration: 3370; Percent complete: 84.2%; Average loss: 2.1197\n",
      "Iteration: 3371; Percent complete: 84.3%; Average loss: 2.3562\n",
      "Iteration: 3372; Percent complete: 84.3%; Average loss: 2.5713\n",
      "Iteration: 3373; Percent complete: 84.3%; Average loss: 2.1139\n",
      "Iteration: 3374; Percent complete: 84.4%; Average loss: 2.0721\n",
      "Iteration: 3375; Percent complete: 84.4%; Average loss: 2.1090\n",
      "Iteration: 3376; Percent complete: 84.4%; Average loss: 2.3887\n",
      "Iteration: 3377; Percent complete: 84.4%; Average loss: 2.2949\n",
      "Iteration: 3378; Percent complete: 84.5%; Average loss: 2.0853\n",
      "Iteration: 3379; Percent complete: 84.5%; Average loss: 2.2817\n",
      "Iteration: 3380; Percent complete: 84.5%; Average loss: 2.3261\n",
      "Iteration: 3381; Percent complete: 84.5%; Average loss: 2.6027\n",
      "Iteration: 3382; Percent complete: 84.5%; Average loss: 2.3119\n",
      "Iteration: 3383; Percent complete: 84.6%; Average loss: 2.2539\n",
      "Iteration: 3384; Percent complete: 84.6%; Average loss: 2.1931\n",
      "Iteration: 3385; Percent complete: 84.6%; Average loss: 2.3324\n",
      "Iteration: 3386; Percent complete: 84.7%; Average loss: 2.2930\n",
      "Iteration: 3387; Percent complete: 84.7%; Average loss: 2.2401\n",
      "Iteration: 3388; Percent complete: 84.7%; Average loss: 2.1529\n",
      "Iteration: 3389; Percent complete: 84.7%; Average loss: 2.4025\n",
      "Iteration: 3390; Percent complete: 84.8%; Average loss: 2.2571\n",
      "Iteration: 3391; Percent complete: 84.8%; Average loss: 2.3018\n",
      "Iteration: 3392; Percent complete: 84.8%; Average loss: 2.1556\n",
      "Iteration: 3393; Percent complete: 84.8%; Average loss: 2.3560\n",
      "Iteration: 3394; Percent complete: 84.9%; Average loss: 2.1468\n",
      "Iteration: 3395; Percent complete: 84.9%; Average loss: 2.4653\n",
      "Iteration: 3396; Percent complete: 84.9%; Average loss: 2.2871\n",
      "Iteration: 3397; Percent complete: 84.9%; Average loss: 2.4714\n",
      "Iteration: 3398; Percent complete: 85.0%; Average loss: 2.1397\n",
      "Iteration: 3399; Percent complete: 85.0%; Average loss: 2.2868\n",
      "Iteration: 3400; Percent complete: 85.0%; Average loss: 2.4530\n",
      "Iteration: 3401; Percent complete: 85.0%; Average loss: 2.4735\n",
      "Iteration: 3402; Percent complete: 85.0%; Average loss: 2.1676\n",
      "Iteration: 3403; Percent complete: 85.1%; Average loss: 2.2674\n",
      "Iteration: 3404; Percent complete: 85.1%; Average loss: 2.1786\n",
      "Iteration: 3405; Percent complete: 85.1%; Average loss: 2.3536\n",
      "Iteration: 3406; Percent complete: 85.2%; Average loss: 2.4145\n",
      "Iteration: 3407; Percent complete: 85.2%; Average loss: 2.2831\n",
      "Iteration: 3408; Percent complete: 85.2%; Average loss: 2.2504\n",
      "Iteration: 3409; Percent complete: 85.2%; Average loss: 2.2369\n",
      "Iteration: 3410; Percent complete: 85.2%; Average loss: 2.4236\n",
      "Iteration: 3411; Percent complete: 85.3%; Average loss: 2.4895\n",
      "Iteration: 3412; Percent complete: 85.3%; Average loss: 2.4202\n",
      "Iteration: 3413; Percent complete: 85.3%; Average loss: 2.1402\n",
      "Iteration: 3414; Percent complete: 85.4%; Average loss: 2.3751\n",
      "Iteration: 3415; Percent complete: 85.4%; Average loss: 2.4568\n",
      "Iteration: 3416; Percent complete: 85.4%; Average loss: 2.3938\n",
      "Iteration: 3417; Percent complete: 85.4%; Average loss: 2.1756\n",
      "Iteration: 3418; Percent complete: 85.5%; Average loss: 2.4002\n",
      "Iteration: 3419; Percent complete: 85.5%; Average loss: 2.2240\n",
      "Iteration: 3420; Percent complete: 85.5%; Average loss: 2.5609\n",
      "Iteration: 3421; Percent complete: 85.5%; Average loss: 2.3004\n",
      "Iteration: 3422; Percent complete: 85.5%; Average loss: 2.0807\n",
      "Iteration: 3423; Percent complete: 85.6%; Average loss: 2.1657\n",
      "Iteration: 3424; Percent complete: 85.6%; Average loss: 2.2843\n",
      "Iteration: 3425; Percent complete: 85.6%; Average loss: 2.5274\n",
      "Iteration: 3426; Percent complete: 85.7%; Average loss: 2.1616\n",
      "Iteration: 3427; Percent complete: 85.7%; Average loss: 2.1245\n",
      "Iteration: 3428; Percent complete: 85.7%; Average loss: 2.3439\n",
      "Iteration: 3429; Percent complete: 85.7%; Average loss: 2.2646\n",
      "Iteration: 3430; Percent complete: 85.8%; Average loss: 2.2877\n",
      "Iteration: 3431; Percent complete: 85.8%; Average loss: 2.2188\n",
      "Iteration: 3432; Percent complete: 85.8%; Average loss: 2.5348\n",
      "Iteration: 3433; Percent complete: 85.8%; Average loss: 2.2788\n",
      "Iteration: 3434; Percent complete: 85.9%; Average loss: 2.4702\n",
      "Iteration: 3435; Percent complete: 85.9%; Average loss: 2.3606\n",
      "Iteration: 3436; Percent complete: 85.9%; Average loss: 2.1909\n",
      "Iteration: 3437; Percent complete: 85.9%; Average loss: 2.2959\n",
      "Iteration: 3438; Percent complete: 86.0%; Average loss: 2.3474\n",
      "Iteration: 3439; Percent complete: 86.0%; Average loss: 2.3411\n",
      "Iteration: 3440; Percent complete: 86.0%; Average loss: 2.2816\n",
      "Iteration: 3441; Percent complete: 86.0%; Average loss: 2.2382\n",
      "Iteration: 3442; Percent complete: 86.1%; Average loss: 2.2829\n",
      "Iteration: 3443; Percent complete: 86.1%; Average loss: 2.4576\n",
      "Iteration: 3444; Percent complete: 86.1%; Average loss: 2.2710\n",
      "Iteration: 3445; Percent complete: 86.1%; Average loss: 2.4011\n",
      "Iteration: 3446; Percent complete: 86.2%; Average loss: 2.1137\n",
      "Iteration: 3447; Percent complete: 86.2%; Average loss: 2.2890\n",
      "Iteration: 3448; Percent complete: 86.2%; Average loss: 2.2278\n",
      "Iteration: 3449; Percent complete: 86.2%; Average loss: 2.2340\n",
      "Iteration: 3450; Percent complete: 86.2%; Average loss: 2.3921\n",
      "Iteration: 3451; Percent complete: 86.3%; Average loss: 2.3309\n",
      "Iteration: 3452; Percent complete: 86.3%; Average loss: 2.3513\n",
      "Iteration: 3453; Percent complete: 86.3%; Average loss: 2.3939\n",
      "Iteration: 3454; Percent complete: 86.4%; Average loss: 2.4174\n",
      "Iteration: 3455; Percent complete: 86.4%; Average loss: 2.1009\n",
      "Iteration: 3456; Percent complete: 86.4%; Average loss: 2.4179\n",
      "Iteration: 3457; Percent complete: 86.4%; Average loss: 2.2576\n",
      "Iteration: 3458; Percent complete: 86.5%; Average loss: 2.3513\n",
      "Iteration: 3459; Percent complete: 86.5%; Average loss: 2.2708\n",
      "Iteration: 3460; Percent complete: 86.5%; Average loss: 2.1670\n",
      "Iteration: 3461; Percent complete: 86.5%; Average loss: 2.2597\n",
      "Iteration: 3462; Percent complete: 86.6%; Average loss: 2.2728\n",
      "Iteration: 3463; Percent complete: 86.6%; Average loss: 2.3814\n",
      "Iteration: 3464; Percent complete: 86.6%; Average loss: 2.3570\n",
      "Iteration: 3465; Percent complete: 86.6%; Average loss: 2.4889\n",
      "Iteration: 3466; Percent complete: 86.7%; Average loss: 2.2552\n",
      "Iteration: 3467; Percent complete: 86.7%; Average loss: 2.2967\n",
      "Iteration: 3468; Percent complete: 86.7%; Average loss: 2.4404\n",
      "Iteration: 3469; Percent complete: 86.7%; Average loss: 2.2393\n",
      "Iteration: 3470; Percent complete: 86.8%; Average loss: 2.3479\n",
      "Iteration: 3471; Percent complete: 86.8%; Average loss: 2.2939\n",
      "Iteration: 3472; Percent complete: 86.8%; Average loss: 2.2208\n",
      "Iteration: 3473; Percent complete: 86.8%; Average loss: 2.1317\n",
      "Iteration: 3474; Percent complete: 86.9%; Average loss: 2.2780\n",
      "Iteration: 3475; Percent complete: 86.9%; Average loss: 2.3387\n",
      "Iteration: 3476; Percent complete: 86.9%; Average loss: 2.3537\n",
      "Iteration: 3477; Percent complete: 86.9%; Average loss: 2.4623\n",
      "Iteration: 3478; Percent complete: 87.0%; Average loss: 2.4924\n",
      "Iteration: 3479; Percent complete: 87.0%; Average loss: 2.0939\n",
      "Iteration: 3480; Percent complete: 87.0%; Average loss: 2.3756\n",
      "Iteration: 3481; Percent complete: 87.0%; Average loss: 2.2005\n",
      "Iteration: 3482; Percent complete: 87.1%; Average loss: 2.2795\n",
      "Iteration: 3483; Percent complete: 87.1%; Average loss: 2.2386\n",
      "Iteration: 3484; Percent complete: 87.1%; Average loss: 2.4178\n",
      "Iteration: 3485; Percent complete: 87.1%; Average loss: 2.2671\n",
      "Iteration: 3486; Percent complete: 87.2%; Average loss: 2.1088\n",
      "Iteration: 3487; Percent complete: 87.2%; Average loss: 2.4518\n",
      "Iteration: 3488; Percent complete: 87.2%; Average loss: 2.0444\n",
      "Iteration: 3489; Percent complete: 87.2%; Average loss: 2.0197\n",
      "Iteration: 3490; Percent complete: 87.2%; Average loss: 2.1840\n",
      "Iteration: 3491; Percent complete: 87.3%; Average loss: 2.1813\n",
      "Iteration: 3492; Percent complete: 87.3%; Average loss: 2.2260\n",
      "Iteration: 3493; Percent complete: 87.3%; Average loss: 2.2416\n",
      "Iteration: 3494; Percent complete: 87.4%; Average loss: 2.2683\n",
      "Iteration: 3495; Percent complete: 87.4%; Average loss: 2.1098\n",
      "Iteration: 3496; Percent complete: 87.4%; Average loss: 2.1908\n",
      "Iteration: 3497; Percent complete: 87.4%; Average loss: 2.1055\n",
      "Iteration: 3498; Percent complete: 87.5%; Average loss: 2.1046\n",
      "Iteration: 3499; Percent complete: 87.5%; Average loss: 2.4095\n",
      "Iteration: 3500; Percent complete: 87.5%; Average loss: 2.4667\n",
      "Iteration: 3501; Percent complete: 87.5%; Average loss: 2.2149\n",
      "Iteration: 3502; Percent complete: 87.5%; Average loss: 2.1512\n",
      "Iteration: 3503; Percent complete: 87.6%; Average loss: 2.3716\n",
      "Iteration: 3504; Percent complete: 87.6%; Average loss: 2.3644\n",
      "Iteration: 3505; Percent complete: 87.6%; Average loss: 2.4129\n",
      "Iteration: 3506; Percent complete: 87.6%; Average loss: 2.2970\n",
      "Iteration: 3507; Percent complete: 87.7%; Average loss: 2.4933\n",
      "Iteration: 3508; Percent complete: 87.7%; Average loss: 2.3114\n",
      "Iteration: 3509; Percent complete: 87.7%; Average loss: 2.1278\n",
      "Iteration: 3510; Percent complete: 87.8%; Average loss: 2.2647\n",
      "Iteration: 3511; Percent complete: 87.8%; Average loss: 2.4773\n",
      "Iteration: 3512; Percent complete: 87.8%; Average loss: 2.2653\n",
      "Iteration: 3513; Percent complete: 87.8%; Average loss: 2.2661\n",
      "Iteration: 3514; Percent complete: 87.8%; Average loss: 2.2427\n",
      "Iteration: 3515; Percent complete: 87.9%; Average loss: 2.0806\n",
      "Iteration: 3516; Percent complete: 87.9%; Average loss: 2.1442\n",
      "Iteration: 3517; Percent complete: 87.9%; Average loss: 2.4820\n",
      "Iteration: 3518; Percent complete: 87.9%; Average loss: 2.0714\n",
      "Iteration: 3519; Percent complete: 88.0%; Average loss: 2.3821\n",
      "Iteration: 3520; Percent complete: 88.0%; Average loss: 2.2013\n",
      "Iteration: 3521; Percent complete: 88.0%; Average loss: 2.2806\n",
      "Iteration: 3522; Percent complete: 88.0%; Average loss: 2.3663\n",
      "Iteration: 3523; Percent complete: 88.1%; Average loss: 2.1899\n",
      "Iteration: 3524; Percent complete: 88.1%; Average loss: 2.1858\n",
      "Iteration: 3525; Percent complete: 88.1%; Average loss: 2.3399\n",
      "Iteration: 3526; Percent complete: 88.1%; Average loss: 2.2242\n",
      "Iteration: 3527; Percent complete: 88.2%; Average loss: 2.2874\n",
      "Iteration: 3528; Percent complete: 88.2%; Average loss: 2.3282\n",
      "Iteration: 3529; Percent complete: 88.2%; Average loss: 2.1107\n",
      "Iteration: 3530; Percent complete: 88.2%; Average loss: 2.5698\n",
      "Iteration: 3531; Percent complete: 88.3%; Average loss: 2.4513\n",
      "Iteration: 3532; Percent complete: 88.3%; Average loss: 2.3881\n",
      "Iteration: 3533; Percent complete: 88.3%; Average loss: 2.2281\n",
      "Iteration: 3534; Percent complete: 88.3%; Average loss: 2.1717\n",
      "Iteration: 3535; Percent complete: 88.4%; Average loss: 2.3143\n",
      "Iteration: 3536; Percent complete: 88.4%; Average loss: 2.2692\n",
      "Iteration: 3537; Percent complete: 88.4%; Average loss: 2.4350\n",
      "Iteration: 3538; Percent complete: 88.4%; Average loss: 2.3400\n",
      "Iteration: 3539; Percent complete: 88.5%; Average loss: 2.3125\n",
      "Iteration: 3540; Percent complete: 88.5%; Average loss: 2.2479\n",
      "Iteration: 3541; Percent complete: 88.5%; Average loss: 2.1956\n",
      "Iteration: 3542; Percent complete: 88.5%; Average loss: 2.3055\n",
      "Iteration: 3543; Percent complete: 88.6%; Average loss: 2.4435\n",
      "Iteration: 3544; Percent complete: 88.6%; Average loss: 2.3167\n",
      "Iteration: 3545; Percent complete: 88.6%; Average loss: 2.4402\n",
      "Iteration: 3546; Percent complete: 88.6%; Average loss: 2.3629\n",
      "Iteration: 3547; Percent complete: 88.7%; Average loss: 2.3156\n",
      "Iteration: 3548; Percent complete: 88.7%; Average loss: 2.1385\n",
      "Iteration: 3549; Percent complete: 88.7%; Average loss: 2.2531\n",
      "Iteration: 3550; Percent complete: 88.8%; Average loss: 2.2667\n",
      "Iteration: 3551; Percent complete: 88.8%; Average loss: 2.1134\n",
      "Iteration: 3552; Percent complete: 88.8%; Average loss: 2.0870\n",
      "Iteration: 3553; Percent complete: 88.8%; Average loss: 2.2336\n",
      "Iteration: 3554; Percent complete: 88.8%; Average loss: 2.2138\n",
      "Iteration: 3555; Percent complete: 88.9%; Average loss: 2.3523\n",
      "Iteration: 3556; Percent complete: 88.9%; Average loss: 2.0896\n",
      "Iteration: 3557; Percent complete: 88.9%; Average loss: 2.1032\n",
      "Iteration: 3558; Percent complete: 88.9%; Average loss: 2.4160\n",
      "Iteration: 3559; Percent complete: 89.0%; Average loss: 2.4008\n",
      "Iteration: 3560; Percent complete: 89.0%; Average loss: 2.1820\n",
      "Iteration: 3561; Percent complete: 89.0%; Average loss: 2.0688\n",
      "Iteration: 3562; Percent complete: 89.0%; Average loss: 2.0889\n",
      "Iteration: 3563; Percent complete: 89.1%; Average loss: 2.5467\n",
      "Iteration: 3564; Percent complete: 89.1%; Average loss: 2.1573\n",
      "Iteration: 3565; Percent complete: 89.1%; Average loss: 2.2498\n",
      "Iteration: 3566; Percent complete: 89.1%; Average loss: 2.1254\n",
      "Iteration: 3567; Percent complete: 89.2%; Average loss: 2.3880\n",
      "Iteration: 3568; Percent complete: 89.2%; Average loss: 2.3179\n",
      "Iteration: 3569; Percent complete: 89.2%; Average loss: 2.1268\n",
      "Iteration: 3570; Percent complete: 89.2%; Average loss: 2.2009\n",
      "Iteration: 3571; Percent complete: 89.3%; Average loss: 2.5564\n",
      "Iteration: 3572; Percent complete: 89.3%; Average loss: 2.1475\n",
      "Iteration: 3573; Percent complete: 89.3%; Average loss: 2.4186\n",
      "Iteration: 3574; Percent complete: 89.3%; Average loss: 2.3463\n",
      "Iteration: 3575; Percent complete: 89.4%; Average loss: 2.2854\n",
      "Iteration: 3576; Percent complete: 89.4%; Average loss: 2.5548\n",
      "Iteration: 3577; Percent complete: 89.4%; Average loss: 2.3040\n",
      "Iteration: 3578; Percent complete: 89.5%; Average loss: 2.2619\n",
      "Iteration: 3579; Percent complete: 89.5%; Average loss: 2.1562\n",
      "Iteration: 3580; Percent complete: 89.5%; Average loss: 2.2169\n",
      "Iteration: 3581; Percent complete: 89.5%; Average loss: 2.5356\n",
      "Iteration: 3582; Percent complete: 89.5%; Average loss: 2.2413\n",
      "Iteration: 3583; Percent complete: 89.6%; Average loss: 2.1252\n",
      "Iteration: 3584; Percent complete: 89.6%; Average loss: 2.3151\n",
      "Iteration: 3585; Percent complete: 89.6%; Average loss: 2.1068\n",
      "Iteration: 3586; Percent complete: 89.6%; Average loss: 2.4037\n",
      "Iteration: 3587; Percent complete: 89.7%; Average loss: 2.3596\n",
      "Iteration: 3588; Percent complete: 89.7%; Average loss: 2.2419\n",
      "Iteration: 3589; Percent complete: 89.7%; Average loss: 2.2628\n",
      "Iteration: 3590; Percent complete: 89.8%; Average loss: 2.1492\n",
      "Iteration: 3591; Percent complete: 89.8%; Average loss: 2.0837\n",
      "Iteration: 3592; Percent complete: 89.8%; Average loss: 2.1765\n",
      "Iteration: 3593; Percent complete: 89.8%; Average loss: 2.4430\n",
      "Iteration: 3594; Percent complete: 89.8%; Average loss: 2.3424\n",
      "Iteration: 3595; Percent complete: 89.9%; Average loss: 2.2855\n",
      "Iteration: 3596; Percent complete: 89.9%; Average loss: 2.1777\n",
      "Iteration: 3597; Percent complete: 89.9%; Average loss: 2.3506\n",
      "Iteration: 3598; Percent complete: 90.0%; Average loss: 2.3229\n",
      "Iteration: 3599; Percent complete: 90.0%; Average loss: 2.2818\n",
      "Iteration: 3600; Percent complete: 90.0%; Average loss: 2.3150\n",
      "Iteration: 3601; Percent complete: 90.0%; Average loss: 2.1129\n",
      "Iteration: 3602; Percent complete: 90.0%; Average loss: 2.2261\n",
      "Iteration: 3603; Percent complete: 90.1%; Average loss: 2.2360\n",
      "Iteration: 3604; Percent complete: 90.1%; Average loss: 2.0980\n",
      "Iteration: 3605; Percent complete: 90.1%; Average loss: 2.2277\n",
      "Iteration: 3606; Percent complete: 90.1%; Average loss: 2.2233\n",
      "Iteration: 3607; Percent complete: 90.2%; Average loss: 2.2890\n",
      "Iteration: 3608; Percent complete: 90.2%; Average loss: 2.2071\n",
      "Iteration: 3609; Percent complete: 90.2%; Average loss: 1.9715\n",
      "Iteration: 3610; Percent complete: 90.2%; Average loss: 2.1173\n",
      "Iteration: 3611; Percent complete: 90.3%; Average loss: 2.3996\n",
      "Iteration: 3612; Percent complete: 90.3%; Average loss: 2.2942\n",
      "Iteration: 3613; Percent complete: 90.3%; Average loss: 2.1153\n",
      "Iteration: 3614; Percent complete: 90.3%; Average loss: 2.3670\n",
      "Iteration: 3615; Percent complete: 90.4%; Average loss: 2.3709\n",
      "Iteration: 3616; Percent complete: 90.4%; Average loss: 2.4354\n",
      "Iteration: 3617; Percent complete: 90.4%; Average loss: 2.1624\n",
      "Iteration: 3618; Percent complete: 90.5%; Average loss: 2.2107\n",
      "Iteration: 3619; Percent complete: 90.5%; Average loss: 2.1659\n",
      "Iteration: 3620; Percent complete: 90.5%; Average loss: 2.2660\n",
      "Iteration: 3621; Percent complete: 90.5%; Average loss: 2.1295\n",
      "Iteration: 3622; Percent complete: 90.5%; Average loss: 2.2195\n",
      "Iteration: 3623; Percent complete: 90.6%; Average loss: 2.2365\n",
      "Iteration: 3624; Percent complete: 90.6%; Average loss: 2.2543\n",
      "Iteration: 3625; Percent complete: 90.6%; Average loss: 2.1588\n",
      "Iteration: 3626; Percent complete: 90.6%; Average loss: 2.2340\n",
      "Iteration: 3627; Percent complete: 90.7%; Average loss: 2.1026\n",
      "Iteration: 3628; Percent complete: 90.7%; Average loss: 2.1777\n",
      "Iteration: 3629; Percent complete: 90.7%; Average loss: 2.1626\n",
      "Iteration: 3630; Percent complete: 90.8%; Average loss: 2.1889\n",
      "Iteration: 3631; Percent complete: 90.8%; Average loss: 2.2190\n",
      "Iteration: 3632; Percent complete: 90.8%; Average loss: 2.1883\n",
      "Iteration: 3633; Percent complete: 90.8%; Average loss: 2.1268\n",
      "Iteration: 3634; Percent complete: 90.8%; Average loss: 2.1069\n",
      "Iteration: 3635; Percent complete: 90.9%; Average loss: 2.3611\n",
      "Iteration: 3636; Percent complete: 90.9%; Average loss: 2.0742\n",
      "Iteration: 3637; Percent complete: 90.9%; Average loss: 2.3200\n",
      "Iteration: 3638; Percent complete: 91.0%; Average loss: 2.2424\n",
      "Iteration: 3639; Percent complete: 91.0%; Average loss: 2.1813\n",
      "Iteration: 3640; Percent complete: 91.0%; Average loss: 2.1504\n",
      "Iteration: 3641; Percent complete: 91.0%; Average loss: 1.9931\n",
      "Iteration: 3642; Percent complete: 91.0%; Average loss: 2.2920\n",
      "Iteration: 3643; Percent complete: 91.1%; Average loss: 2.1650\n",
      "Iteration: 3644; Percent complete: 91.1%; Average loss: 2.1130\n",
      "Iteration: 3645; Percent complete: 91.1%; Average loss: 2.2938\n",
      "Iteration: 3646; Percent complete: 91.1%; Average loss: 2.2403\n",
      "Iteration: 3647; Percent complete: 91.2%; Average loss: 2.0428\n",
      "Iteration: 3648; Percent complete: 91.2%; Average loss: 2.0862\n",
      "Iteration: 3649; Percent complete: 91.2%; Average loss: 2.1635\n",
      "Iteration: 3650; Percent complete: 91.2%; Average loss: 2.2999\n",
      "Iteration: 3651; Percent complete: 91.3%; Average loss: 2.2257\n",
      "Iteration: 3652; Percent complete: 91.3%; Average loss: 2.4005\n",
      "Iteration: 3653; Percent complete: 91.3%; Average loss: 2.0639\n",
      "Iteration: 3654; Percent complete: 91.3%; Average loss: 2.3050\n",
      "Iteration: 3655; Percent complete: 91.4%; Average loss: 2.4142\n",
      "Iteration: 3656; Percent complete: 91.4%; Average loss: 2.1499\n",
      "Iteration: 3657; Percent complete: 91.4%; Average loss: 2.0485\n",
      "Iteration: 3658; Percent complete: 91.5%; Average loss: 2.2580\n",
      "Iteration: 3659; Percent complete: 91.5%; Average loss: 2.1922\n",
      "Iteration: 3660; Percent complete: 91.5%; Average loss: 2.1781\n",
      "Iteration: 3661; Percent complete: 91.5%; Average loss: 2.3537\n",
      "Iteration: 3662; Percent complete: 91.5%; Average loss: 2.3246\n",
      "Iteration: 3663; Percent complete: 91.6%; Average loss: 2.0909\n",
      "Iteration: 3664; Percent complete: 91.6%; Average loss: 2.3030\n",
      "Iteration: 3665; Percent complete: 91.6%; Average loss: 2.5836\n",
      "Iteration: 3666; Percent complete: 91.6%; Average loss: 2.2000\n",
      "Iteration: 3667; Percent complete: 91.7%; Average loss: 2.3096\n",
      "Iteration: 3668; Percent complete: 91.7%; Average loss: 2.1954\n",
      "Iteration: 3669; Percent complete: 91.7%; Average loss: 2.0783\n",
      "Iteration: 3670; Percent complete: 91.8%; Average loss: 2.3938\n",
      "Iteration: 3671; Percent complete: 91.8%; Average loss: 1.8698\n",
      "Iteration: 3672; Percent complete: 91.8%; Average loss: 2.1266\n",
      "Iteration: 3673; Percent complete: 91.8%; Average loss: 2.1281\n",
      "Iteration: 3674; Percent complete: 91.8%; Average loss: 1.9391\n",
      "Iteration: 3675; Percent complete: 91.9%; Average loss: 2.3107\n",
      "Iteration: 3676; Percent complete: 91.9%; Average loss: 2.3173\n",
      "Iteration: 3677; Percent complete: 91.9%; Average loss: 2.4887\n",
      "Iteration: 3678; Percent complete: 92.0%; Average loss: 2.3036\n",
      "Iteration: 3679; Percent complete: 92.0%; Average loss: 2.1389\n",
      "Iteration: 3680; Percent complete: 92.0%; Average loss: 2.1884\n",
      "Iteration: 3681; Percent complete: 92.0%; Average loss: 2.2221\n",
      "Iteration: 3682; Percent complete: 92.0%; Average loss: 2.3177\n",
      "Iteration: 3683; Percent complete: 92.1%; Average loss: 2.1467\n",
      "Iteration: 3684; Percent complete: 92.1%; Average loss: 2.2164\n",
      "Iteration: 3685; Percent complete: 92.1%; Average loss: 2.3599\n",
      "Iteration: 3686; Percent complete: 92.2%; Average loss: 2.2759\n",
      "Iteration: 3687; Percent complete: 92.2%; Average loss: 2.4116\n",
      "Iteration: 3688; Percent complete: 92.2%; Average loss: 2.3109\n",
      "Iteration: 3689; Percent complete: 92.2%; Average loss: 2.2310\n",
      "Iteration: 3690; Percent complete: 92.2%; Average loss: 2.3324\n",
      "Iteration: 3691; Percent complete: 92.3%; Average loss: 2.1481\n",
      "Iteration: 3692; Percent complete: 92.3%; Average loss: 2.2620\n",
      "Iteration: 3693; Percent complete: 92.3%; Average loss: 2.2424\n",
      "Iteration: 3694; Percent complete: 92.3%; Average loss: 2.4192\n",
      "Iteration: 3695; Percent complete: 92.4%; Average loss: 2.2184\n",
      "Iteration: 3696; Percent complete: 92.4%; Average loss: 2.2439\n",
      "Iteration: 3697; Percent complete: 92.4%; Average loss: 2.2411\n",
      "Iteration: 3698; Percent complete: 92.5%; Average loss: 1.9904\n",
      "Iteration: 3699; Percent complete: 92.5%; Average loss: 2.2906\n",
      "Iteration: 3700; Percent complete: 92.5%; Average loss: 2.4089\n",
      "Iteration: 3701; Percent complete: 92.5%; Average loss: 2.2533\n",
      "Iteration: 3702; Percent complete: 92.5%; Average loss: 2.2529\n",
      "Iteration: 3703; Percent complete: 92.6%; Average loss: 2.2309\n",
      "Iteration: 3704; Percent complete: 92.6%; Average loss: 2.1207\n",
      "Iteration: 3705; Percent complete: 92.6%; Average loss: 2.1470\n",
      "Iteration: 3706; Percent complete: 92.7%; Average loss: 2.0963\n",
      "Iteration: 3707; Percent complete: 92.7%; Average loss: 2.2423\n",
      "Iteration: 3708; Percent complete: 92.7%; Average loss: 2.3899\n",
      "Iteration: 3709; Percent complete: 92.7%; Average loss: 2.2749\n",
      "Iteration: 3710; Percent complete: 92.8%; Average loss: 2.3624\n",
      "Iteration: 3711; Percent complete: 92.8%; Average loss: 1.8851\n",
      "Iteration: 3712; Percent complete: 92.8%; Average loss: 2.1645\n",
      "Iteration: 3713; Percent complete: 92.8%; Average loss: 2.0958\n",
      "Iteration: 3714; Percent complete: 92.8%; Average loss: 2.2633\n",
      "Iteration: 3715; Percent complete: 92.9%; Average loss: 2.2549\n",
      "Iteration: 3716; Percent complete: 92.9%; Average loss: 2.1594\n",
      "Iteration: 3717; Percent complete: 92.9%; Average loss: 2.3373\n",
      "Iteration: 3718; Percent complete: 93.0%; Average loss: 2.1260\n",
      "Iteration: 3719; Percent complete: 93.0%; Average loss: 2.2480\n",
      "Iteration: 3720; Percent complete: 93.0%; Average loss: 2.3388\n",
      "Iteration: 3721; Percent complete: 93.0%; Average loss: 2.2455\n",
      "Iteration: 3722; Percent complete: 93.0%; Average loss: 2.2484\n",
      "Iteration: 3723; Percent complete: 93.1%; Average loss: 2.4020\n",
      "Iteration: 3724; Percent complete: 93.1%; Average loss: 2.5286\n",
      "Iteration: 3725; Percent complete: 93.1%; Average loss: 2.1655\n",
      "Iteration: 3726; Percent complete: 93.2%; Average loss: 2.1046\n",
      "Iteration: 3727; Percent complete: 93.2%; Average loss: 2.3360\n",
      "Iteration: 3728; Percent complete: 93.2%; Average loss: 2.2249\n",
      "Iteration: 3729; Percent complete: 93.2%; Average loss: 2.2868\n",
      "Iteration: 3730; Percent complete: 93.2%; Average loss: 2.0651\n",
      "Iteration: 3731; Percent complete: 93.3%; Average loss: 1.9862\n",
      "Iteration: 3732; Percent complete: 93.3%; Average loss: 2.2213\n",
      "Iteration: 3733; Percent complete: 93.3%; Average loss: 2.0532\n",
      "Iteration: 3734; Percent complete: 93.3%; Average loss: 2.2808\n",
      "Iteration: 3735; Percent complete: 93.4%; Average loss: 2.1890\n",
      "Iteration: 3736; Percent complete: 93.4%; Average loss: 2.3902\n",
      "Iteration: 3737; Percent complete: 93.4%; Average loss: 2.0502\n",
      "Iteration: 3738; Percent complete: 93.5%; Average loss: 2.2050\n",
      "Iteration: 3739; Percent complete: 93.5%; Average loss: 2.2757\n",
      "Iteration: 3740; Percent complete: 93.5%; Average loss: 2.4383\n",
      "Iteration: 3741; Percent complete: 93.5%; Average loss: 2.1428\n",
      "Iteration: 3742; Percent complete: 93.5%; Average loss: 2.1952\n",
      "Iteration: 3743; Percent complete: 93.6%; Average loss: 2.3443\n",
      "Iteration: 3744; Percent complete: 93.6%; Average loss: 2.2195\n",
      "Iteration: 3745; Percent complete: 93.6%; Average loss: 2.3673\n",
      "Iteration: 3746; Percent complete: 93.7%; Average loss: 2.0252\n",
      "Iteration: 3747; Percent complete: 93.7%; Average loss: 2.4885\n",
      "Iteration: 3748; Percent complete: 93.7%; Average loss: 2.2358\n",
      "Iteration: 3749; Percent complete: 93.7%; Average loss: 2.1456\n",
      "Iteration: 3750; Percent complete: 93.8%; Average loss: 2.3124\n",
      "Iteration: 3751; Percent complete: 93.8%; Average loss: 2.0380\n",
      "Iteration: 3752; Percent complete: 93.8%; Average loss: 2.2845\n",
      "Iteration: 3753; Percent complete: 93.8%; Average loss: 2.2045\n",
      "Iteration: 3754; Percent complete: 93.8%; Average loss: 2.4443\n",
      "Iteration: 3755; Percent complete: 93.9%; Average loss: 2.1784\n",
      "Iteration: 3756; Percent complete: 93.9%; Average loss: 2.1473\n",
      "Iteration: 3757; Percent complete: 93.9%; Average loss: 2.0494\n",
      "Iteration: 3758; Percent complete: 94.0%; Average loss: 2.2377\n",
      "Iteration: 3759; Percent complete: 94.0%; Average loss: 2.1572\n",
      "Iteration: 3760; Percent complete: 94.0%; Average loss: 2.2032\n",
      "Iteration: 3761; Percent complete: 94.0%; Average loss: 2.3566\n",
      "Iteration: 3762; Percent complete: 94.0%; Average loss: 2.1852\n",
      "Iteration: 3763; Percent complete: 94.1%; Average loss: 2.0990\n",
      "Iteration: 3764; Percent complete: 94.1%; Average loss: 2.1603\n",
      "Iteration: 3765; Percent complete: 94.1%; Average loss: 2.1906\n",
      "Iteration: 3766; Percent complete: 94.2%; Average loss: 2.0236\n",
      "Iteration: 3767; Percent complete: 94.2%; Average loss: 2.4042\n",
      "Iteration: 3768; Percent complete: 94.2%; Average loss: 2.3140\n",
      "Iteration: 3769; Percent complete: 94.2%; Average loss: 2.0689\n",
      "Iteration: 3770; Percent complete: 94.2%; Average loss: 2.3324\n",
      "Iteration: 3771; Percent complete: 94.3%; Average loss: 2.0145\n",
      "Iteration: 3772; Percent complete: 94.3%; Average loss: 2.1159\n",
      "Iteration: 3773; Percent complete: 94.3%; Average loss: 2.3945\n",
      "Iteration: 3774; Percent complete: 94.3%; Average loss: 2.2664\n",
      "Iteration: 3775; Percent complete: 94.4%; Average loss: 2.1768\n",
      "Iteration: 3776; Percent complete: 94.4%; Average loss: 2.3250\n",
      "Iteration: 3777; Percent complete: 94.4%; Average loss: 2.1838\n",
      "Iteration: 3778; Percent complete: 94.5%; Average loss: 2.0857\n",
      "Iteration: 3779; Percent complete: 94.5%; Average loss: 2.0841\n",
      "Iteration: 3780; Percent complete: 94.5%; Average loss: 2.0837\n",
      "Iteration: 3781; Percent complete: 94.5%; Average loss: 2.1695\n",
      "Iteration: 3782; Percent complete: 94.5%; Average loss: 2.1427\n",
      "Iteration: 3783; Percent complete: 94.6%; Average loss: 2.2452\n",
      "Iteration: 3784; Percent complete: 94.6%; Average loss: 2.3640\n",
      "Iteration: 3785; Percent complete: 94.6%; Average loss: 2.4083\n",
      "Iteration: 3786; Percent complete: 94.7%; Average loss: 2.3264\n",
      "Iteration: 3787; Percent complete: 94.7%; Average loss: 2.1549\n",
      "Iteration: 3788; Percent complete: 94.7%; Average loss: 2.1812\n",
      "Iteration: 3789; Percent complete: 94.7%; Average loss: 2.1950\n",
      "Iteration: 3790; Percent complete: 94.8%; Average loss: 2.3478\n",
      "Iteration: 3791; Percent complete: 94.8%; Average loss: 2.2361\n",
      "Iteration: 3792; Percent complete: 94.8%; Average loss: 2.3344\n",
      "Iteration: 3793; Percent complete: 94.8%; Average loss: 2.1071\n",
      "Iteration: 3794; Percent complete: 94.8%; Average loss: 2.0500\n",
      "Iteration: 3795; Percent complete: 94.9%; Average loss: 2.2243\n",
      "Iteration: 3796; Percent complete: 94.9%; Average loss: 2.1813\n",
      "Iteration: 3797; Percent complete: 94.9%; Average loss: 2.4844\n",
      "Iteration: 3798; Percent complete: 95.0%; Average loss: 2.0647\n",
      "Iteration: 3799; Percent complete: 95.0%; Average loss: 2.1923\n",
      "Iteration: 3800; Percent complete: 95.0%; Average loss: 2.3126\n",
      "Iteration: 3801; Percent complete: 95.0%; Average loss: 2.3523\n",
      "Iteration: 3802; Percent complete: 95.0%; Average loss: 2.2503\n",
      "Iteration: 3803; Percent complete: 95.1%; Average loss: 2.1892\n",
      "Iteration: 3804; Percent complete: 95.1%; Average loss: 2.3969\n",
      "Iteration: 3805; Percent complete: 95.1%; Average loss: 2.3657\n",
      "Iteration: 3806; Percent complete: 95.2%; Average loss: 2.1987\n",
      "Iteration: 3807; Percent complete: 95.2%; Average loss: 2.0768\n",
      "Iteration: 3808; Percent complete: 95.2%; Average loss: 2.3724\n",
      "Iteration: 3809; Percent complete: 95.2%; Average loss: 2.2088\n",
      "Iteration: 3810; Percent complete: 95.2%; Average loss: 2.2691\n",
      "Iteration: 3811; Percent complete: 95.3%; Average loss: 2.1228\n",
      "Iteration: 3812; Percent complete: 95.3%; Average loss: 2.1535\n",
      "Iteration: 3813; Percent complete: 95.3%; Average loss: 2.2917\n",
      "Iteration: 3814; Percent complete: 95.3%; Average loss: 2.4158\n",
      "Iteration: 3815; Percent complete: 95.4%; Average loss: 2.1116\n",
      "Iteration: 3816; Percent complete: 95.4%; Average loss: 2.3106\n",
      "Iteration: 3817; Percent complete: 95.4%; Average loss: 2.1005\n",
      "Iteration: 3818; Percent complete: 95.5%; Average loss: 2.2220\n",
      "Iteration: 3819; Percent complete: 95.5%; Average loss: 2.1903\n",
      "Iteration: 3820; Percent complete: 95.5%; Average loss: 2.0636\n",
      "Iteration: 3821; Percent complete: 95.5%; Average loss: 2.1412\n",
      "Iteration: 3822; Percent complete: 95.5%; Average loss: 2.2172\n",
      "Iteration: 3823; Percent complete: 95.6%; Average loss: 2.2164\n",
      "Iteration: 3824; Percent complete: 95.6%; Average loss: 2.1064\n",
      "Iteration: 3825; Percent complete: 95.6%; Average loss: 2.3496\n",
      "Iteration: 3826; Percent complete: 95.7%; Average loss: 2.3898\n",
      "Iteration: 3827; Percent complete: 95.7%; Average loss: 2.2000\n",
      "Iteration: 3828; Percent complete: 95.7%; Average loss: 2.1719\n",
      "Iteration: 3829; Percent complete: 95.7%; Average loss: 2.1660\n",
      "Iteration: 3830; Percent complete: 95.8%; Average loss: 2.2210\n",
      "Iteration: 3831; Percent complete: 95.8%; Average loss: 2.2443\n",
      "Iteration: 3832; Percent complete: 95.8%; Average loss: 2.1755\n",
      "Iteration: 3833; Percent complete: 95.8%; Average loss: 2.2452\n",
      "Iteration: 3834; Percent complete: 95.9%; Average loss: 2.3394\n",
      "Iteration: 3835; Percent complete: 95.9%; Average loss: 2.0741\n",
      "Iteration: 3836; Percent complete: 95.9%; Average loss: 2.2857\n",
      "Iteration: 3837; Percent complete: 95.9%; Average loss: 2.3439\n",
      "Iteration: 3838; Percent complete: 96.0%; Average loss: 2.2805\n",
      "Iteration: 3839; Percent complete: 96.0%; Average loss: 2.0051\n",
      "Iteration: 3840; Percent complete: 96.0%; Average loss: 2.1255\n",
      "Iteration: 3841; Percent complete: 96.0%; Average loss: 2.1025\n",
      "Iteration: 3842; Percent complete: 96.0%; Average loss: 2.1905\n",
      "Iteration: 3843; Percent complete: 96.1%; Average loss: 2.0866\n",
      "Iteration: 3844; Percent complete: 96.1%; Average loss: 2.0927\n",
      "Iteration: 3845; Percent complete: 96.1%; Average loss: 2.3618\n",
      "Iteration: 3846; Percent complete: 96.2%; Average loss: 2.3366\n",
      "Iteration: 3847; Percent complete: 96.2%; Average loss: 2.3384\n",
      "Iteration: 3848; Percent complete: 96.2%; Average loss: 2.2649\n",
      "Iteration: 3849; Percent complete: 96.2%; Average loss: 2.0559\n",
      "Iteration: 3850; Percent complete: 96.2%; Average loss: 1.9719\n",
      "Iteration: 3851; Percent complete: 96.3%; Average loss: 2.2267\n",
      "Iteration: 3852; Percent complete: 96.3%; Average loss: 2.0565\n",
      "Iteration: 3853; Percent complete: 96.3%; Average loss: 2.1749\n",
      "Iteration: 3854; Percent complete: 96.4%; Average loss: 2.2929\n",
      "Iteration: 3855; Percent complete: 96.4%; Average loss: 2.1769\n",
      "Iteration: 3856; Percent complete: 96.4%; Average loss: 2.0825\n",
      "Iteration: 3857; Percent complete: 96.4%; Average loss: 2.2440\n",
      "Iteration: 3858; Percent complete: 96.5%; Average loss: 2.2680\n",
      "Iteration: 3859; Percent complete: 96.5%; Average loss: 2.2421\n",
      "Iteration: 3860; Percent complete: 96.5%; Average loss: 2.2363\n",
      "Iteration: 3861; Percent complete: 96.5%; Average loss: 2.1541\n",
      "Iteration: 3862; Percent complete: 96.5%; Average loss: 2.0924\n",
      "Iteration: 3863; Percent complete: 96.6%; Average loss: 2.4624\n",
      "Iteration: 3864; Percent complete: 96.6%; Average loss: 2.2182\n",
      "Iteration: 3865; Percent complete: 96.6%; Average loss: 2.1617\n",
      "Iteration: 3866; Percent complete: 96.7%; Average loss: 2.0998\n",
      "Iteration: 3867; Percent complete: 96.7%; Average loss: 2.3215\n",
      "Iteration: 3868; Percent complete: 96.7%; Average loss: 2.2761\n",
      "Iteration: 3869; Percent complete: 96.7%; Average loss: 2.2709\n",
      "Iteration: 3870; Percent complete: 96.8%; Average loss: 2.2685\n",
      "Iteration: 3871; Percent complete: 96.8%; Average loss: 2.1697\n",
      "Iteration: 3872; Percent complete: 96.8%; Average loss: 2.0531\n",
      "Iteration: 3873; Percent complete: 96.8%; Average loss: 2.2068\n",
      "Iteration: 3874; Percent complete: 96.9%; Average loss: 2.0877\n",
      "Iteration: 3875; Percent complete: 96.9%; Average loss: 2.2942\n",
      "Iteration: 3876; Percent complete: 96.9%; Average loss: 2.0476\n",
      "Iteration: 3877; Percent complete: 96.9%; Average loss: 2.2394\n",
      "Iteration: 3878; Percent complete: 97.0%; Average loss: 2.2084\n",
      "Iteration: 3879; Percent complete: 97.0%; Average loss: 2.1425\n",
      "Iteration: 3880; Percent complete: 97.0%; Average loss: 2.1669\n",
      "Iteration: 3881; Percent complete: 97.0%; Average loss: 1.9775\n",
      "Iteration: 3882; Percent complete: 97.0%; Average loss: 2.1064\n",
      "Iteration: 3883; Percent complete: 97.1%; Average loss: 2.0221\n",
      "Iteration: 3884; Percent complete: 97.1%; Average loss: 2.1942\n",
      "Iteration: 3885; Percent complete: 97.1%; Average loss: 2.3379\n",
      "Iteration: 3886; Percent complete: 97.2%; Average loss: 2.4347\n",
      "Iteration: 3887; Percent complete: 97.2%; Average loss: 2.1798\n",
      "Iteration: 3888; Percent complete: 97.2%; Average loss: 2.1985\n",
      "Iteration: 3889; Percent complete: 97.2%; Average loss: 2.1919\n",
      "Iteration: 3890; Percent complete: 97.2%; Average loss: 2.5197\n",
      "Iteration: 3891; Percent complete: 97.3%; Average loss: 2.1196\n",
      "Iteration: 3892; Percent complete: 97.3%; Average loss: 2.3647\n",
      "Iteration: 3893; Percent complete: 97.3%; Average loss: 1.9259\n",
      "Iteration: 3894; Percent complete: 97.4%; Average loss: 2.1220\n",
      "Iteration: 3895; Percent complete: 97.4%; Average loss: 2.2028\n",
      "Iteration: 3896; Percent complete: 97.4%; Average loss: 2.3324\n",
      "Iteration: 3897; Percent complete: 97.4%; Average loss: 2.0346\n",
      "Iteration: 3898; Percent complete: 97.5%; Average loss: 1.9585\n",
      "Iteration: 3899; Percent complete: 97.5%; Average loss: 2.2869\n",
      "Iteration: 3900; Percent complete: 97.5%; Average loss: 2.2494\n",
      "Iteration: 3901; Percent complete: 97.5%; Average loss: 2.1779\n",
      "Iteration: 3902; Percent complete: 97.5%; Average loss: 2.3664\n",
      "Iteration: 3903; Percent complete: 97.6%; Average loss: 2.1305\n",
      "Iteration: 3904; Percent complete: 97.6%; Average loss: 2.1186\n",
      "Iteration: 3905; Percent complete: 97.6%; Average loss: 2.1179\n",
      "Iteration: 3906; Percent complete: 97.7%; Average loss: 2.1489\n",
      "Iteration: 3907; Percent complete: 97.7%; Average loss: 2.0035\n",
      "Iteration: 3908; Percent complete: 97.7%; Average loss: 2.2522\n",
      "Iteration: 3909; Percent complete: 97.7%; Average loss: 2.0853\n",
      "Iteration: 3910; Percent complete: 97.8%; Average loss: 2.0895\n",
      "Iteration: 3911; Percent complete: 97.8%; Average loss: 2.2744\n",
      "Iteration: 3912; Percent complete: 97.8%; Average loss: 2.0135\n",
      "Iteration: 3913; Percent complete: 97.8%; Average loss: 2.0728\n",
      "Iteration: 3914; Percent complete: 97.9%; Average loss: 2.2537\n",
      "Iteration: 3915; Percent complete: 97.9%; Average loss: 2.1837\n",
      "Iteration: 3916; Percent complete: 97.9%; Average loss: 2.2531\n",
      "Iteration: 3917; Percent complete: 97.9%; Average loss: 2.2494\n",
      "Iteration: 3918; Percent complete: 98.0%; Average loss: 2.3787\n",
      "Iteration: 3919; Percent complete: 98.0%; Average loss: 2.1185\n",
      "Iteration: 3920; Percent complete: 98.0%; Average loss: 2.3221\n",
      "Iteration: 3921; Percent complete: 98.0%; Average loss: 2.1559\n",
      "Iteration: 3922; Percent complete: 98.0%; Average loss: 2.0255\n",
      "Iteration: 3923; Percent complete: 98.1%; Average loss: 2.0802\n",
      "Iteration: 3924; Percent complete: 98.1%; Average loss: 2.0826\n",
      "Iteration: 3925; Percent complete: 98.1%; Average loss: 2.1664\n",
      "Iteration: 3926; Percent complete: 98.2%; Average loss: 2.0451\n",
      "Iteration: 3927; Percent complete: 98.2%; Average loss: 2.2835\n",
      "Iteration: 3928; Percent complete: 98.2%; Average loss: 2.1401\n",
      "Iteration: 3929; Percent complete: 98.2%; Average loss: 2.2555\n",
      "Iteration: 3930; Percent complete: 98.2%; Average loss: 2.4479\n",
      "Iteration: 3931; Percent complete: 98.3%; Average loss: 2.2643\n",
      "Iteration: 3932; Percent complete: 98.3%; Average loss: 2.1169\n",
      "Iteration: 3933; Percent complete: 98.3%; Average loss: 2.3555\n",
      "Iteration: 3934; Percent complete: 98.4%; Average loss: 2.2045\n",
      "Iteration: 3935; Percent complete: 98.4%; Average loss: 2.2470\n",
      "Iteration: 3936; Percent complete: 98.4%; Average loss: 2.2070\n",
      "Iteration: 3937; Percent complete: 98.4%; Average loss: 2.4616\n",
      "Iteration: 3938; Percent complete: 98.5%; Average loss: 2.1713\n",
      "Iteration: 3939; Percent complete: 98.5%; Average loss: 2.1017\n",
      "Iteration: 3940; Percent complete: 98.5%; Average loss: 2.2522\n",
      "Iteration: 3941; Percent complete: 98.5%; Average loss: 2.2070\n",
      "Iteration: 3942; Percent complete: 98.6%; Average loss: 2.2108\n",
      "Iteration: 3943; Percent complete: 98.6%; Average loss: 2.1335\n",
      "Iteration: 3944; Percent complete: 98.6%; Average loss: 2.2753\n",
      "Iteration: 3945; Percent complete: 98.6%; Average loss: 2.3152\n",
      "Iteration: 3946; Percent complete: 98.7%; Average loss: 2.0817\n",
      "Iteration: 3947; Percent complete: 98.7%; Average loss: 2.1414\n",
      "Iteration: 3948; Percent complete: 98.7%; Average loss: 2.1436\n",
      "Iteration: 3949; Percent complete: 98.7%; Average loss: 2.2322\n",
      "Iteration: 3950; Percent complete: 98.8%; Average loss: 2.1147\n",
      "Iteration: 3951; Percent complete: 98.8%; Average loss: 2.1606\n",
      "Iteration: 3952; Percent complete: 98.8%; Average loss: 2.2004\n",
      "Iteration: 3953; Percent complete: 98.8%; Average loss: 2.2136\n",
      "Iteration: 3954; Percent complete: 98.9%; Average loss: 2.1029\n",
      "Iteration: 3955; Percent complete: 98.9%; Average loss: 2.0224\n",
      "Iteration: 3956; Percent complete: 98.9%; Average loss: 2.1740\n",
      "Iteration: 3957; Percent complete: 98.9%; Average loss: 2.1597\n",
      "Iteration: 3958; Percent complete: 99.0%; Average loss: 2.1926\n",
      "Iteration: 3959; Percent complete: 99.0%; Average loss: 1.9256\n",
      "Iteration: 3960; Percent complete: 99.0%; Average loss: 2.0570\n",
      "Iteration: 3961; Percent complete: 99.0%; Average loss: 2.2303\n",
      "Iteration: 3962; Percent complete: 99.1%; Average loss: 2.1153\n",
      "Iteration: 3963; Percent complete: 99.1%; Average loss: 1.9830\n",
      "Iteration: 3964; Percent complete: 99.1%; Average loss: 2.0997\n",
      "Iteration: 3965; Percent complete: 99.1%; Average loss: 2.1503\n",
      "Iteration: 3966; Percent complete: 99.2%; Average loss: 2.2105\n",
      "Iteration: 3967; Percent complete: 99.2%; Average loss: 2.2191\n",
      "Iteration: 3968; Percent complete: 99.2%; Average loss: 2.1964\n",
      "Iteration: 3969; Percent complete: 99.2%; Average loss: 2.1891\n",
      "Iteration: 3970; Percent complete: 99.2%; Average loss: 2.2875\n",
      "Iteration: 3971; Percent complete: 99.3%; Average loss: 2.1309\n",
      "Iteration: 3972; Percent complete: 99.3%; Average loss: 2.1236\n",
      "Iteration: 3973; Percent complete: 99.3%; Average loss: 2.3213\n",
      "Iteration: 3974; Percent complete: 99.4%; Average loss: 2.1667\n",
      "Iteration: 3975; Percent complete: 99.4%; Average loss: 1.9882\n",
      "Iteration: 3976; Percent complete: 99.4%; Average loss: 2.1444\n",
      "Iteration: 3977; Percent complete: 99.4%; Average loss: 2.2229\n",
      "Iteration: 3978; Percent complete: 99.5%; Average loss: 2.0809\n",
      "Iteration: 3979; Percent complete: 99.5%; Average loss: 2.2807\n",
      "Iteration: 3980; Percent complete: 99.5%; Average loss: 2.2640\n",
      "Iteration: 3981; Percent complete: 99.5%; Average loss: 2.2538\n",
      "Iteration: 3982; Percent complete: 99.6%; Average loss: 2.0666\n",
      "Iteration: 3983; Percent complete: 99.6%; Average loss: 2.4268\n",
      "Iteration: 3984; Percent complete: 99.6%; Average loss: 2.2176\n",
      "Iteration: 3985; Percent complete: 99.6%; Average loss: 2.1682\n",
      "Iteration: 3986; Percent complete: 99.7%; Average loss: 2.2234\n",
      "Iteration: 3987; Percent complete: 99.7%; Average loss: 2.2235\n",
      "Iteration: 3988; Percent complete: 99.7%; Average loss: 1.9837\n",
      "Iteration: 3989; Percent complete: 99.7%; Average loss: 2.0464\n",
      "Iteration: 3990; Percent complete: 99.8%; Average loss: 2.2615\n",
      "Iteration: 3991; Percent complete: 99.8%; Average loss: 2.1028\n",
      "Iteration: 3992; Percent complete: 99.8%; Average loss: 2.1903\n",
      "Iteration: 3993; Percent complete: 99.8%; Average loss: 2.0775\n",
      "Iteration: 3994; Percent complete: 99.9%; Average loss: 2.0734\n",
      "Iteration: 3995; Percent complete: 99.9%; Average loss: 2.3482\n",
      "Iteration: 3996; Percent complete: 99.9%; Average loss: 2.0710\n",
      "Iteration: 3997; Percent complete: 99.9%; Average loss: 2.2561\n",
      "Iteration: 3998; Percent complete: 100.0%; Average loss: 2.0815\n",
      "Iteration: 3999; Percent complete: 100.0%; Average loss: 2.2038\n",
      "Iteration: 4000; Percent complete: 100.0%; Average loss: 2.0396\n"
     ]
    }
   ],
   "source": [
    "# Configure training/optimization\n",
    "clip = 50.0\n",
    "teacher_forcing_ratio = 1.0\n",
    "learning_rate = 0.0001\n",
    "decoder_learning_ratio = 5.0\n",
    "n_iteration = 4000\n",
    "print_every = 1\n",
    "save_every = 500\n",
    "\n",
    "# Ensure dropout layers are in train mode\n",
    "encoder.train()\n",
    "decoder.train()\n",
    "\n",
    "# Initialize optimizers\n",
    "print('Building optimizers ...')\n",
    "encoder_optimizer = optim.Adam(encoder.parameters(), lr=learning_rate)\n",
    "decoder_optimizer = optim.Adam(decoder.parameters(), lr=learning_rate * decoder_learning_ratio)\n",
    "if loadFilename:\n",
    "    encoder_optimizer.load_state_dict(encoder_optimizer_sd)\n",
    "    decoder_optimizer.load_state_dict(decoder_optimizer_sd)\n",
    "\n",
    "# Run training iterations\n",
    "print(\"Starting Training!\")\n",
    "trainIters(model_name, voc, pairs, encoder, decoder, encoder_optimizer, decoder_optimizer,\n",
    "           embedding, encoder_n_layers, decoder_n_layers, save_dir, n_iteration, batch_size,\n",
    "           print_every, save_every, clip, corpus_name, loadFilename)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6a1e2e0",
   "metadata": {},
   "source": [
    "#### 10. Chat with your model : eval(), evalInput()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "47d3d5bd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "> hello how are you\n",
      "Bot: hello .\n",
      "> what is your name\n",
      "Bot: my name is travis .\n",
      "> where do you live\n",
      "Bot: i don t know .\n",
      "> tell me a story\n",
      "Bot: you re a compulsive man .\n",
      "> how much do you love\n",
      "Bot: i don t know .\n",
      "> what is the capital of India\n",
      "Bot: i m not going to kill you .\n",
      "> why would you kill me\n",
      "Bot: because i m not going to kill you\n",
      "> why do you speak like this\n",
      "Bot: because i m so sorry .\n",
      "> why are you sorry\n",
      "Bot: i m sorry i m sorry .\n",
      "> take a break\n",
      "Bot: a girl .\n",
      "> you are not good\n",
      "Bot: i m not\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "Interrupted by user",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "Input \u001b[1;32mIn [31]\u001b[0m, in \u001b[0;36m<cell line: 9>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      6\u001b[0m searcher \u001b[38;5;241m=\u001b[39m GreedySearchDecoder(encoder, decoder)\n\u001b[0;32m      8\u001b[0m \u001b[38;5;66;03m# Begin chatting (uncomment and run the following line to begin)\u001b[39;00m\n\u001b[1;32m----> 9\u001b[0m \u001b[43mevaluateInput\u001b[49m\u001b[43m(\u001b[49m\u001b[43mencoder\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mdecoder\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43msearcher\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mvoc\u001b[49m\u001b[43m)\u001b[49m\n",
      "Input \u001b[1;32mIn [30]\u001b[0m, in \u001b[0;36mevaluateInput\u001b[1;34m(encoder, decoder, searcher, voc)\u001b[0m\n\u001b[0;32m     21\u001b[0m \u001b[38;5;28;01mwhile\u001b[39;00m(\u001b[38;5;241m1\u001b[39m):\n\u001b[0;32m     22\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m     23\u001b[0m         \u001b[38;5;66;03m# Get input sentence\u001b[39;00m\n\u001b[1;32m---> 24\u001b[0m         input_sentence \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43minput\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43m> \u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m     25\u001b[0m         \u001b[38;5;66;03m# Check if it is quit case\u001b[39;00m\n\u001b[0;32m     26\u001b[0m         \u001b[38;5;28;01mif\u001b[39;00m input_sentence \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mq\u001b[39m\u001b[38;5;124m'\u001b[39m \u001b[38;5;129;01mor\u001b[39;00m input_sentence \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mquit\u001b[39m\u001b[38;5;124m'\u001b[39m: \u001b[38;5;28;01mbreak\u001b[39;00m\n",
      "File \u001b[1;32mC:\\Anaconda3\\lib\\site-packages\\ipykernel\\kernelbase.py:1075\u001b[0m, in \u001b[0;36mKernel.raw_input\u001b[1;34m(self, prompt)\u001b[0m\n\u001b[0;32m   1071\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_allow_stdin:\n\u001b[0;32m   1072\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m StdinNotImplementedError(\n\u001b[0;32m   1073\u001b[0m         \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mraw_input was called, but this frontend does not support input requests.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m   1074\u001b[0m     )\n\u001b[1;32m-> 1075\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_input_request\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m   1076\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;28;43mstr\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mprompt\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m   1077\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_parent_ident\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mshell\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m   1078\u001b[0m \u001b[43m    \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mget_parent\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mshell\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m   1079\u001b[0m \u001b[43m    \u001b[49m\u001b[43mpassword\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mFalse\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[0;32m   1080\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32mC:\\Anaconda3\\lib\\site-packages\\ipykernel\\kernelbase.py:1120\u001b[0m, in \u001b[0;36mKernel._input_request\u001b[1;34m(self, prompt, ident, parent, password)\u001b[0m\n\u001b[0;32m   1117\u001b[0m             \u001b[38;5;28;01mbreak\u001b[39;00m\n\u001b[0;32m   1118\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyboardInterrupt\u001b[39;00m:\n\u001b[0;32m   1119\u001b[0m     \u001b[38;5;66;03m# re-raise KeyboardInterrupt, to truncate traceback\u001b[39;00m\n\u001b[1;32m-> 1120\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mKeyboardInterrupt\u001b[39;00m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInterrupted by user\u001b[39m\u001b[38;5;124m\"\u001b[39m) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;28mNone\u001b[39m\n\u001b[0;32m   1121\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mException\u001b[39;00m:\n\u001b[0;32m   1122\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mlog\u001b[38;5;241m.\u001b[39mwarning(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInvalid Message:\u001b[39m\u001b[38;5;124m\"\u001b[39m, exc_info\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mTrue\u001b[39;00m)\n",
      "\u001b[1;31mKeyboardInterrupt\u001b[0m: Interrupted by user"
     ]
    }
   ],
   "source": [
    "# Set dropout layers to eval mode\n",
    "encoder.eval()\n",
    "decoder.eval()\n",
    "\n",
    "# Initialize search module\n",
    "searcher = GreedySearchDecoder(encoder, decoder)\n",
    "\n",
    "# Begin chatting (uncomment and run the following line to begin)\n",
    "evaluateInput(encoder, decoder, searcher, voc)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}