RandomProjectionNeuralNetwork/RandomProjectionNeuralNetwork.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training Neural Networks on High Dimensional Data using Random Projection\n",
    "- Implementation of the [paper](https://link.springer.com/content/pdf/10.1007/s10044-018-0697-0.pdf) with the same name."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**High Dimensional Data?**\n",
    "\n",
    "There is no specific condition under which data is classified as High Dimensional, but the dataset I am going to use, has 28395 features. That is very comfortably called as high dimensional data.In addition, this data doesnot have any spatial characteristic. So, this stops us from using convolutions and computer vision techniques.\n",
    "\n",
    "Such data is usually seen in Bioinformatics, Social Media, Finance etc.\n",
    "\n",
    "**Spatial Characteristics?**\n",
    "\n",
    "Images are generally considered as high dimensional because for each image we have #(width)*#(height)*#(numberofchannels) pixels as features. But Images exhibit something known as Spatial Characteristic, which we will familiarize with in the coming few lines.\n",
    "\n",
    "For example, suppose you are trying to classify images of shoes vs T-shirts from the Fashion MNIST [dataset](https://www.kaggle.com/zalando-research/fashionmnist).(As a side note MNIST images are not that big, but can be used to easily understand and experiment with concepts, on a basic level.) One of the first things we(humans), notice when asked to classify an image as a shoe or a T-shirt, is the shape. No matter, what color or size a shoe has, it has a distinctly different shape when compared to any T-shirt. This is called spatial Characteristic of the shoe. Even a neural network model tries to mimic our decision process i.e. using the shape to classify images. This is exactly what convolutions are used for i.e. identifying spatial characeristics of an image.\n",
    "\n",
    "Here is an image of how a well trained neural network, extracts spatial features(in this case shape) out of an Fashion MNIST image in the process of classification of the image. This image is taken from one of the lecture videos of Coursera's TensorFlow in AI specialistaion.\n",
    "\n",
    "![1](assets/1.jpeg)\n",
    "\n",
    "The above image depicts the output of 1st 2nd 3rd and 4th convolution layers respectively. Notice how the first convolution layer identifies the outline of a shoe and further convolution layers identify more resolved spatial characteristics. You can easily imagine this type of borderline for any shoe image taken at side viewing angles. So, this helps the neural network reduce number of features based on certain spatial characteristics guaranteed to be present in data like images.\n",
    "\n",
    "But this is not true in case of our data, as our data is not an image.\n",
    "\n",
    "**So Where is this data From?**\n",
    "\n",
    "This data consists of 28395 gene expression values for just 200 plants. That is, with each sample having 28395 features, we only have 200 such samples. We are required to predict a single phenotype value for each plant from this data. **As far as we(non-biologists) are concerned, we have 200 samples and 28395 features per sample.** You can find the data [here](https://github.com/achillesposiedon/PapersImplemented/tree/master/RandomProjectionNeuralNetwork/Data).\n",
    "\n",
    "**What is the problem with conventional regression Neural Networks?**\n",
    "\n",
    "We are aware of the fact that, we need a huge amount of training data, i.e. training samples to be able to train a neural network. How much data you exactly need, is a question almost impossible to answer, because the amount of Data required depends upon a lot of factors, including application, complexity of the function you are trying to map, neural network architecture (though some might argue architecture can to some extent depend upon the available data) and so on. It is a generally accepted [fact](https://machinelearningmastery.com/much-training-data-required-machine-learning/) that the number of training samples have to be some x% larger than number of features, where x is in tens.\n",
    "\n",
    "In our case we have a mere 200 samples against 28,395 features. Given that, our data does not look good to go for a direct regression neural network.\n",
    "\n",
    "**So What Else?**\n",
    "There are some 3-4 papers, which give us guidance as to how to overcome this task. I will be implementing one of those papers here.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**So what does the paper say?**\n",
    "\n",
    "This paper relies on a lemma and here is the relevant quote from the paper.\n",
    "\n",
    "![2](assets/2.jpeg)\n",
    "\n",
    "This in layman's terms implies that most of the structure of a very high dimensional data can be represented on a relatively very small dimensional space with very little loss, by multiplying the given data with a carefully designed Random Projection matrix.\n",
    "\n",
    "The paper cites several constructions of RP matrix, including Gaussian, Achlioptas’, Li’s, subsampled randomized Hadamard transform (SRHT) and Count Sketch-based constructions.\n",
    "\n",
    "The Achlioptas's construction is the one I am going to consider in this implementation."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Achlioptas's RP matrix is defined as follows:\n",
    "![3](assets/3.jpeg)\n",
    "\n",
    "where k is the dimension of the smaller dimension space we are going to project the input data onto."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have understood to some extent, how, this RP Neural Network is going to work, let us First see How a regular neural network performs in the given task."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda:0\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "\n",
    "#imports necessary to define a neural network \n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "\n",
    "#ensure you are using GPU.\n",
    "if torch.cuda.is_available():  \n",
    "  dev = \"cuda:0\" \n",
    "else:  \n",
    "  dev = \"cpu\"  \n",
    "\n",
    "device = torch.device(dev)\n",
    "print(device)\n",
    "\n",
    "dtype = torch.cuda.FloatTensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "trdt1=pd.read_csv('C:\\\\Users\\\\Ant Pc\\\\GitHub\\\\pROJ\\\\Genotype\\\\TrainingGenotype.csv')\n",
    "trdt=pd.read_csv('C:\\\\Users\\\\Ant Pc\\\\GitHub\\\\pROJ\\\\Geneexpression0\\\\traingeneexpression.csv',header=None)\n",
    "tstdt=pd.read_csv('C:\\\\Users\\\\Ant Pc\\\\GitHub\\\\pROJ\\\\Geneexpression0\\\\testgeneexpression.csv',header=None)\n",
    "\n",
    "tstdt=np.array(tstdt)\n",
    "tstdt=np.transpose(tstdt)\n",
    "tstdt=pd.DataFrame(tstdt)\n",
    "\n",
    "trdt=np.array(trdt)\n",
    "trdt=np.transpose(trdt)\n",
    "trdt=pd.DataFrame(trdt)\n",
    "\n",
    "results1=trdt1['Phenotype1']\n",
    "results2=trdt1['Phenotype2']\n",
    "\n",
    "testfeatures=tstdt\n",
    "\n",
    "X=np.random.randn(30,6)\n",
    "gole=tstdt\n",
    "gole=pd.DataFrame(gole)\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler=StandardScaler()\n",
    "trdt=scaler.fit_transform(trdt.values)\n",
    "trdt=pd.DataFrame(trdt)\n",
    "\n",
    "scaler1=StandardScaler()\n",
    "tstdt=scaler1.fit_transform(tstdt.values)\n",
    "tstdt=pd.DataFrame(tstdt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([200, 28395])\n",
      "torch.Size([200])\n"
     ]
    }
   ],
   "source": [
    "# convert matrices to pytorch tensors on gpu\n",
    "\n",
    "trdttensor=torch.from_numpy(trdt.values).type(dtype)\n",
    "results1tensor=torch.from_numpy(results1.values).type(dtype)\n",
    "\n",
    "print(trdttensor.shape)\n",
    "print(results1tensor.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Net(\n",
      "  (fc1): Linear(in_features=28395, out_features=3200, bias=True)\n",
      "  (fc3): Linear(in_features=3200, out_features=1, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "class Net(nn.Module):\n",
    "\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        \n",
    "        # an affine operation: y = Wx + b\n",
    "        \n",
    "        self.fc1 = nn.Linear(28395,3200).cuda()\n",
    "        self.fc3 = nn.Linear(3200,1).cuda()\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.fc1(x)).cuda()\n",
    "        x = self.fc3(x).cuda()\n",
    "        return x\n",
    "\n",
    "net = Net()\n",
    "\n",
    "#use gpu for all computations in model\n",
    "net.cuda()\n",
    "print(net)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\torch\\nn\\modules\\loss.py:432: UserWarning: Using a target size (torch.Size([20])) that is different to the input size (torch.Size([20, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.\n",
      "  return F.mse_loss(input, target, reduction=self.reduction)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(8750.3350, device='cuda:0', grad_fn=<MseLossBackward>) 0\n",
      "tensor(4182.0610, device='cuda:0', grad_fn=<MseLossBackward>) 1\n",
      "tensor(2963.4177, device='cuda:0', grad_fn=<MseLossBackward>) 2\n",
      "tensor(2426.8831, device='cuda:0', grad_fn=<MseLossBackward>) 3\n",
      "tensor(2015.3982, device='cuda:0', grad_fn=<MseLossBackward>) 4\n",
      "tensor(1665.8800, device='cuda:0', grad_fn=<MseLossBackward>) 5\n",
      "tensor(1371.6389, device='cuda:0', grad_fn=<MseLossBackward>) 6\n",
      "tensor(1137.2860, device='cuda:0', grad_fn=<MseLossBackward>) 7\n",
      "tensor(959.6178, device='cuda:0', grad_fn=<MseLossBackward>) 8\n",
      "tensor(825.8397, device='cuda:0', grad_fn=<MseLossBackward>) 9\n",
      "tensor(732.7545, device='cuda:0', grad_fn=<MseLossBackward>) 10\n",
      "tensor(660.8025, device='cuda:0', grad_fn=<MseLossBackward>) 11\n",
      "tensor(611.8418, device='cuda:0', grad_fn=<MseLossBackward>) 12\n",
      "tensor(571.9149, device='cuda:0', grad_fn=<MseLossBackward>) 13\n",
      "tensor(539.3962, device='cuda:0', grad_fn=<MseLossBackward>) 14\n",
      "tensor(515.4526, device='cuda:0', grad_fn=<MseLossBackward>) 15\n",
      "tensor(490.7573, device='cuda:0', grad_fn=<MseLossBackward>) 16\n",
      "tensor(470.3951, device='cuda:0', grad_fn=<MseLossBackward>) 17\n",
      "tensor(448.3788, device='cuda:0', grad_fn=<MseLossBackward>) 18\n",
      "tensor(431.1868, device='cuda:0', grad_fn=<MseLossBackward>) 19\n",
      "tensor(412.4607, device='cuda:0', grad_fn=<MseLossBackward>) 20\n",
      "tensor(396.7035, device='cuda:0', grad_fn=<MseLossBackward>) 21\n",
      "tensor(378.5137, device='cuda:0', grad_fn=<MseLossBackward>) 22\n",
      "tensor(367.0378, device='cuda:0', grad_fn=<MseLossBackward>) 23\n",
      "tensor(349.7375, device='cuda:0', grad_fn=<MseLossBackward>) 24\n",
      "tensor(336.2173, device='cuda:0', grad_fn=<MseLossBackward>) 25\n",
      "tensor(321.9388, device='cuda:0', grad_fn=<MseLossBackward>) 26\n",
      "tensor(311.6845, device='cuda:0', grad_fn=<MseLossBackward>) 27\n",
      "tensor(294.1494, device='cuda:0', grad_fn=<MseLossBackward>) 28\n",
      "tensor(288.4062, device='cuda:0', grad_fn=<MseLossBackward>) 29\n",
      "tensor(269.1255, device='cuda:0', grad_fn=<MseLossBackward>) 30\n",
      "tensor(270.7672, device='cuda:0', grad_fn=<MseLossBackward>) 31\n",
      "tensor(239.7666, device='cuda:0', grad_fn=<MseLossBackward>) 32\n",
      "tensor(261.2272, device='cuda:0', grad_fn=<MseLossBackward>) 33\n",
      "tensor(205.1661, device='cuda:0', grad_fn=<MseLossBackward>) 34\n",
      "tensor(281.8717, device='cuda:0', grad_fn=<MseLossBackward>) 35\n",
      "tensor(153.6567, device='cuda:0', grad_fn=<MseLossBackward>) 36\n",
      "tensor(483.5889, device='cuda:0', grad_fn=<MseLossBackward>) 37\n",
      "tensor(144.9245, device='cuda:0', grad_fn=<MseLossBackward>) 38\n",
      "tensor(745.3439, device='cuda:0', grad_fn=<MseLossBackward>) 39\n",
      "tensor(126.6699, device='cuda:0', grad_fn=<MseLossBackward>) 40\n",
      "tensor(233.6439, device='cuda:0', grad_fn=<MseLossBackward>) 41\n",
      "tensor(104.4481, device='cuda:0', grad_fn=<MseLossBackward>) 42\n",
      "tensor(157.3573, device='cuda:0', grad_fn=<MseLossBackward>) 43\n",
      "tensor(111.2050, device='cuda:0', grad_fn=<MseLossBackward>) 44\n",
      "tensor(132.2476, device='cuda:0', grad_fn=<MseLossBackward>) 45\n",
      "tensor(110.6943, device='cuda:0', grad_fn=<MseLossBackward>) 46\n",
      "tensor(118.8871, device='cuda:0', grad_fn=<MseLossBackward>) 47\n",
      "tensor(104.9705, device='cuda:0', grad_fn=<MseLossBackward>) 48\n",
      "tensor(108.9659, device='cuda:0', grad_fn=<MseLossBackward>) 49\n",
      "tensor(96.9397, device='cuda:0', grad_fn=<MseLossBackward>) 50\n",
      "tensor(99.7691, device='cuda:0', grad_fn=<MseLossBackward>) 51\n",
      "tensor(88.5560, device='cuda:0', grad_fn=<MseLossBackward>) 52\n",
      "tensor(90.8482, device='cuda:0', grad_fn=<MseLossBackward>) 53\n",
      "tensor(80.1662, device='cuda:0', grad_fn=<MseLossBackward>) 54\n",
      "tensor(83.9404, device='cuda:0', grad_fn=<MseLossBackward>) 55\n",
      "tensor(71.0506, device='cuda:0', grad_fn=<MseLossBackward>) 56\n",
      "tensor(80.2933, device='cuda:0', grad_fn=<MseLossBackward>) 57\n",
      "tensor(59.9114, device='cuda:0', grad_fn=<MseLossBackward>) 58\n",
      "tensor(85.9841, device='cuda:0', grad_fn=<MseLossBackward>) 59\n",
      "tensor(46.0885, device='cuda:0', grad_fn=<MseLossBackward>) 60\n",
      "tensor(148.4657, device='cuda:0', grad_fn=<MseLossBackward>) 61\n",
      "tensor(64.9901, device='cuda:0', grad_fn=<MseLossBackward>) 62\n",
      "tensor(484.1519, device='cuda:0', grad_fn=<MseLossBackward>) 63\n",
      "tensor(138.3507, device='cuda:0', grad_fn=<MseLossBackward>) 64\n",
      "tensor(262.6931, device='cuda:0', grad_fn=<MseLossBackward>) 65\n",
      "tensor(51.3570, device='cuda:0', grad_fn=<MseLossBackward>) 66\n",
      "tensor(57.5686, device='cuda:0', grad_fn=<MseLossBackward>) 67\n",
      "tensor(27.3437, device='cuda:0', grad_fn=<MseLossBackward>) 68\n",
      "tensor(42.0641, device='cuda:0', grad_fn=<MseLossBackward>) 69\n",
      "tensor(25.9077, device='cuda:0', grad_fn=<MseLossBackward>) 70\n",
      "tensor(36.4528, device='cuda:0', grad_fn=<MseLossBackward>) 71\n",
      "tensor(24.3903, device='cuda:0', grad_fn=<MseLossBackward>) 72\n",
      "tensor(33.3744, device='cuda:0', grad_fn=<MseLossBackward>) 73\n",
      "tensor(22.3363, device='cuda:0', grad_fn=<MseLossBackward>) 74\n",
      "tensor(31.3636, device='cuda:0', grad_fn=<MseLossBackward>) 75\n",
      "tensor(20.0819, device='cuda:0', grad_fn=<MseLossBackward>) 76\n",
      "tensor(29.5681, device='cuda:0', grad_fn=<MseLossBackward>) 77\n",
      "tensor(18.2494, device='cuda:0', grad_fn=<MseLossBackward>) 78\n",
      "tensor(24.7700, device='cuda:0', grad_fn=<MseLossBackward>) 79\n",
      "tensor(18.2422, device='cuda:0', grad_fn=<MseLossBackward>) 80\n",
      "tensor(16.3683, device='cuda:0', grad_fn=<MseLossBackward>) 81\n",
      "tensor(78.9165, device='cuda:0', grad_fn=<MseLossBackward>) 82\n",
      "tensor(71.3119, device='cuda:0', grad_fn=<MseLossBackward>) 83\n",
      "tensor(432.9660, device='cuda:0', grad_fn=<MseLossBackward>) 84\n",
      "tensor(52.5076, device='cuda:0', grad_fn=<MseLossBackward>) 85\n",
      "tensor(348.8090, device='cuda:0', grad_fn=<MseLossBackward>) 86\n",
      "tensor(36.1553, device='cuda:0', grad_fn=<MseLossBackward>) 87\n",
      "tensor(190.9792, device='cuda:0', grad_fn=<MseLossBackward>) 88\n",
      "tensor(93.1459, device='cuda:0', grad_fn=<MseLossBackward>) 89\n",
      "tensor(96.0883, device='cuda:0', grad_fn=<MseLossBackward>) 90\n",
      "tensor(104.8663, device='cuda:0', grad_fn=<MseLossBackward>) 91\n",
      "tensor(65.6370, device='cuda:0', grad_fn=<MseLossBackward>) 92\n",
      "tensor(107.9259, device='cuda:0', grad_fn=<MseLossBackward>) 93\n",
      "tensor(43.8318, device='cuda:0', grad_fn=<MseLossBackward>) 94\n",
      "tensor(110.2219, device='cuda:0', grad_fn=<MseLossBackward>) 95\n",
      "tensor(29.3547, device='cuda:0', grad_fn=<MseLossBackward>) 96\n",
      "tensor(104.6953, device='cuda:0', grad_fn=<MseLossBackward>) 97\n",
      "tensor(23.3575, device='cuda:0', grad_fn=<MseLossBackward>) 98\n",
      "tensor(93.1973, device='cuda:0', grad_fn=<MseLossBackward>) 99\n",
      "tensor(24.0356, device='cuda:0', grad_fn=<MseLossBackward>) 100\n",
      "tensor(78.9274, device='cuda:0', grad_fn=<MseLossBackward>) 101\n",
      "tensor(27.0885, device='cuda:0', grad_fn=<MseLossBackward>) 102\n",
      "tensor(61.5352, device='cuda:0', grad_fn=<MseLossBackward>) 103\n",
      "tensor(25.7997, device='cuda:0', grad_fn=<MseLossBackward>) 104\n",
      "tensor(52.9277, device='cuda:0', grad_fn=<MseLossBackward>) 105\n",
      "tensor(21.9498, device='cuda:0', grad_fn=<MseLossBackward>) 106\n",
      "tensor(46.4394, device='cuda:0', grad_fn=<MseLossBackward>) 107\n",
      "tensor(20.0145, device='cuda:0', grad_fn=<MseLossBackward>) 108\n",
      "tensor(39.5186, device='cuda:0', grad_fn=<MseLossBackward>) 109\n",
      "tensor(19.8987, device='cuda:0', grad_fn=<MseLossBackward>) 110\n",
      "tensor(32.5403, device='cuda:0', grad_fn=<MseLossBackward>) 111\n",
      "tensor(20.1310, device='cuda:0', grad_fn=<MseLossBackward>) 112\n",
      "tensor(26.9181, device='cuda:0', grad_fn=<MseLossBackward>) 113\n",
      "tensor(18.7004, device='cuda:0', grad_fn=<MseLossBackward>) 114\n",
      "tensor(22.5696, device='cuda:0', grad_fn=<MseLossBackward>) 115\n",
      "tensor(14.5073, device='cuda:0', grad_fn=<MseLossBackward>) 116\n",
      "tensor(17.7395, device='cuda:0', grad_fn=<MseLossBackward>) 117\n",
      "tensor(4.6065, device='cuda:0', grad_fn=<MseLossBackward>) 118\n",
      "tensor(6.4045, device='cuda:0', grad_fn=<MseLossBackward>) 119\n",
      "tensor(4.8052, device='cuda:0', grad_fn=<MseLossBackward>) 120\n",
      "tensor(7.0485, device='cuda:0', grad_fn=<MseLossBackward>) 121\n",
      "tensor(7.9018, device='cuda:0', grad_fn=<MseLossBackward>) 122\n",
      "tensor(3.4415, device='cuda:0', grad_fn=<MseLossBackward>) 123\n",
      "tensor(5.8700, device='cuda:0', grad_fn=<MseLossBackward>) 124\n",
      "tensor(11.2728, device='cuda:0', grad_fn=<MseLossBackward>) 125\n",
      "tensor(6.1587, device='cuda:0', grad_fn=<MseLossBackward>) 126\n",
      "tensor(8.1110, device='cuda:0', grad_fn=<MseLossBackward>) 127\n",
      "tensor(6.5716, device='cuda:0', grad_fn=<MseLossBackward>) 128\n",
      "tensor(4.2860, device='cuda:0', grad_fn=<MseLossBackward>) 129\n",
      "tensor(5.3221, device='cuda:0', grad_fn=<MseLossBackward>) 130\n",
      "tensor(5.2814, device='cuda:0', grad_fn=<MseLossBackward>) 131\n",
      "tensor(6.1860, device='cuda:0', grad_fn=<MseLossBackward>) 132\n",
      "tensor(7.4549, device='cuda:0', grad_fn=<MseLossBackward>) 133\n",
      "tensor(6.6937, device='cuda:0', grad_fn=<MseLossBackward>) 134\n",
      "tensor(5.9703, device='cuda:0', grad_fn=<MseLossBackward>) 135\n",
      "tensor(3.3363, device='cuda:0', grad_fn=<MseLossBackward>) 136\n",
      "tensor(4.9107, device='cuda:0', grad_fn=<MseLossBackward>) 137\n",
      "tensor(4.9522, device='cuda:0', grad_fn=<MseLossBackward>) 138\n",
      "tensor(5.1656, device='cuda:0', grad_fn=<MseLossBackward>) 139\n",
      "tensor(3.9103, device='cuda:0', grad_fn=<MseLossBackward>) 140\n",
      "tensor(8.7138, device='cuda:0', grad_fn=<MseLossBackward>) 141\n",
      "tensor(8.7998, device='cuda:0', grad_fn=<MseLossBackward>) 142\n",
      "tensor(6.1044, device='cuda:0', grad_fn=<MseLossBackward>) 143\n",
      "tensor(4.2044, device='cuda:0', grad_fn=<MseLossBackward>) 144\n",
      "tensor(13.5262, device='cuda:0', grad_fn=<MseLossBackward>) 145\n",
      "tensor(3.9224, device='cuda:0', grad_fn=<MseLossBackward>) 146\n",
      "tensor(4.5539, device='cuda:0', grad_fn=<MseLossBackward>) 147\n",
      "tensor(2.3667, device='cuda:0', grad_fn=<MseLossBackward>) 148\n",
      "tensor(2.1565, device='cuda:0', grad_fn=<MseLossBackward>) 149\n",
      "tensor(7.1787, device='cuda:0', grad_fn=<MseLossBackward>) 150\n",
      "tensor(2.8988, device='cuda:0', grad_fn=<MseLossBackward>) 151\n",
      "tensor(3.2190, device='cuda:0', grad_fn=<MseLossBackward>) 152\n",
      "tensor(3.4202, device='cuda:0', grad_fn=<MseLossBackward>) 153\n",
      "tensor(2.7029, device='cuda:0', grad_fn=<MseLossBackward>) 154\n",
      "tensor(3.1214, device='cuda:0', grad_fn=<MseLossBackward>) 155\n",
      "tensor(4.8918, device='cuda:0', grad_fn=<MseLossBackward>) 156\n",
      "tensor(2.1721, device='cuda:0', grad_fn=<MseLossBackward>) 157\n",
      "tensor(5.2877, device='cuda:0', grad_fn=<MseLossBackward>) 158\n",
      "tensor(2.0635, device='cuda:0', grad_fn=<MseLossBackward>) 159\n",
      "tensor(4.2122, device='cuda:0', grad_fn=<MseLossBackward>) 160\n",
      "tensor(2.3125, device='cuda:0', grad_fn=<MseLossBackward>) 161\n",
      "tensor(3.2099, device='cuda:0', grad_fn=<MseLossBackward>) 162\n",
      "tensor(2.8239, device='cuda:0', grad_fn=<MseLossBackward>) 163\n",
      "tensor(2.4983, device='cuda:0', grad_fn=<MseLossBackward>) 164\n",
      "tensor(2.6657, device='cuda:0', grad_fn=<MseLossBackward>) 165\n",
      "tensor(2.7813, device='cuda:0', grad_fn=<MseLossBackward>) 166\n",
      "tensor(2.4918, device='cuda:0', grad_fn=<MseLossBackward>) 167\n",
      "tensor(2.5699, device='cuda:0', grad_fn=<MseLossBackward>) 168\n",
      "tensor(2.6796, device='cuda:0', grad_fn=<MseLossBackward>) 169\n",
      "tensor(2.3116, device='cuda:0', grad_fn=<MseLossBackward>) 170\n",
      "tensor(2.2155, device='cuda:0', grad_fn=<MseLossBackward>) 171\n",
      "tensor(1.9269, device='cuda:0', grad_fn=<MseLossBackward>) 172\n",
      "tensor(2.9870, device='cuda:0', grad_fn=<MseLossBackward>) 173\n",
      "tensor(5.8946, device='cuda:0', grad_fn=<MseLossBackward>) 174\n",
      "tensor(3.9201, device='cuda:0', grad_fn=<MseLossBackward>) 175\n",
      "tensor(3.4659, device='cuda:0', grad_fn=<MseLossBackward>) 176\n",
      "tensor(6.6092, device='cuda:0', grad_fn=<MseLossBackward>) 177\n",
      "tensor(9.3251, device='cuda:0', grad_fn=<MseLossBackward>) 178\n",
      "tensor(4.3741, device='cuda:0', grad_fn=<MseLossBackward>) 179\n",
      "tensor(3.3394, device='cuda:0', grad_fn=<MseLossBackward>) 180\n",
      "tensor(2.3587, device='cuda:0', grad_fn=<MseLossBackward>) 181\n",
      "tensor(11.4269, device='cuda:0', grad_fn=<MseLossBackward>) 182\n",
      "tensor(4.7923, device='cuda:0', grad_fn=<MseLossBackward>) 183\n",
      "tensor(8.7219, device='cuda:0', grad_fn=<MseLossBackward>) 184\n",
      "tensor(6.7959, device='cuda:0', grad_fn=<MseLossBackward>) 185\n",
      "tensor(6.1643, device='cuda:0', grad_fn=<MseLossBackward>) 186\n",
      "tensor(3.4488, device='cuda:0', grad_fn=<MseLossBackward>) 187\n",
      "tensor(3.5980, device='cuda:0', grad_fn=<MseLossBackward>) 188\n",
      "tensor(8.1946, device='cuda:0', grad_fn=<MseLossBackward>) 189\n",
      "tensor(6.0971, device='cuda:0', grad_fn=<MseLossBackward>) 190\n",
      "tensor(6.6646, device='cuda:0', grad_fn=<MseLossBackward>) 191\n",
      "tensor(7.7882, device='cuda:0', grad_fn=<MseLossBackward>) 192\n",
      "tensor(5.3934, device='cuda:0', grad_fn=<MseLossBackward>) 193\n",
      "tensor(8.0998, device='cuda:0', grad_fn=<MseLossBackward>) 194\n",
      "tensor(4.1157, device='cuda:0', grad_fn=<MseLossBackward>) 195\n",
      "tensor(8.4967, device='cuda:0', grad_fn=<MseLossBackward>) 196\n",
      "tensor(3.0312, device='cuda:0', grad_fn=<MseLossBackward>) 197\n",
      "tensor(9.1918, device='cuda:0', grad_fn=<MseLossBackward>) 198\n",
      "tensor(2.0734, device='cuda:0', grad_fn=<MseLossBackward>) 199\n",
      "tensor(7.5016, device='cuda:0', grad_fn=<MseLossBackward>) 200\n",
      "tensor(1.0421, device='cuda:0', grad_fn=<MseLossBackward>) 201\n",
      "tensor(1.0877, device='cuda:0', grad_fn=<MseLossBackward>) 202\n",
      "tensor(1.5180, device='cuda:0', grad_fn=<MseLossBackward>) 203\n",
      "tensor(2.2666, device='cuda:0', grad_fn=<MseLossBackward>) 204\n",
      "tensor(0.8056, device='cuda:0', grad_fn=<MseLossBackward>) 205\n",
      "tensor(6.7930, device='cuda:0', grad_fn=<MseLossBackward>) 206\n",
      "tensor(1.6414, device='cuda:0', grad_fn=<MseLossBackward>) 207\n",
      "tensor(0.9091, device='cuda:0', grad_fn=<MseLossBackward>) 208\n",
      "tensor(9.0273, device='cuda:0', grad_fn=<MseLossBackward>) 209\n",
      "tensor(0.9397, device='cuda:0', grad_fn=<MseLossBackward>) 210\n",
      "tensor(3.6192, device='cuda:0', grad_fn=<MseLossBackward>) 211\n",
      "tensor(1.0381, device='cuda:0', grad_fn=<MseLossBackward>) 212\n",
      "tensor(5.5667, device='cuda:0', grad_fn=<MseLossBackward>) 213\n",
      "tensor(1.3388, device='cuda:0', grad_fn=<MseLossBackward>) 214\n",
      "tensor(1.6934, device='cuda:0', grad_fn=<MseLossBackward>) 215\n",
      "tensor(1.2563, device='cuda:0', grad_fn=<MseLossBackward>) 216\n",
      "tensor(2.3563, device='cuda:0', grad_fn=<MseLossBackward>) 217\n",
      "tensor(1.2012, device='cuda:0', grad_fn=<MseLossBackward>) 218\n",
      "tensor(1.1208, device='cuda:0', grad_fn=<MseLossBackward>) 219\n",
      "tensor(3.8162, device='cuda:0', grad_fn=<MseLossBackward>) 220\n",
      "tensor(0.9451, device='cuda:0', grad_fn=<MseLossBackward>) 221\n",
      "tensor(3.4769, device='cuda:0', grad_fn=<MseLossBackward>) 222\n",
      "tensor(1.8710, device='cuda:0', grad_fn=<MseLossBackward>) 223\n",
      "tensor(0.9169, device='cuda:0', grad_fn=<MseLossBackward>) 224\n",
      "tensor(2.4768, device='cuda:0', grad_fn=<MseLossBackward>) 225\n",
      "tensor(1.4488, device='cuda:0', grad_fn=<MseLossBackward>) 226\n",
      "tensor(2.1944, device='cuda:0', grad_fn=<MseLossBackward>) 227\n",
      "tensor(1.1774, device='cuda:0', grad_fn=<MseLossBackward>) 228\n",
      "tensor(2.8326, device='cuda:0', grad_fn=<MseLossBackward>) 229\n",
      "tensor(0.8933, device='cuda:0', grad_fn=<MseLossBackward>) 230\n",
      "tensor(2.0230, device='cuda:0', grad_fn=<MseLossBackward>) 231\n",
      "tensor(0.8402, device='cuda:0', grad_fn=<MseLossBackward>) 232\n",
      "tensor(2.6727, device='cuda:0', grad_fn=<MseLossBackward>) 233\n",
      "tensor(1.0826, device='cuda:0', grad_fn=<MseLossBackward>) 234\n",
      "tensor(2.0427, device='cuda:0', grad_fn=<MseLossBackward>) 235\n",
      "tensor(0.8849, device='cuda:0', grad_fn=<MseLossBackward>) 236\n",
      "tensor(1.5642, device='cuda:0', grad_fn=<MseLossBackward>) 237\n",
      "tensor(1.4562, device='cuda:0', grad_fn=<MseLossBackward>) 238\n",
      "tensor(1.0460, device='cuda:0', grad_fn=<MseLossBackward>) 239\n",
      "tensor(3.3643, device='cuda:0', grad_fn=<MseLossBackward>) 240\n",
      "tensor(1.0556, device='cuda:0', grad_fn=<MseLossBackward>) 241\n",
      "tensor(1.4397, device='cuda:0', grad_fn=<MseLossBackward>) 242\n",
      "tensor(0.8040, device='cuda:0', grad_fn=<MseLossBackward>) 243\n",
      "tensor(1.2700, device='cuda:0', grad_fn=<MseLossBackward>) 244\n",
      "tensor(0.9860, device='cuda:0', grad_fn=<MseLossBackward>) 245\n",
      "tensor(0.8798, device='cuda:0', grad_fn=<MseLossBackward>) 246\n",
      "tensor(1.0897, device='cuda:0', grad_fn=<MseLossBackward>) 247\n",
      "tensor(1.0867, device='cuda:0', grad_fn=<MseLossBackward>) 248\n",
      "tensor(1.1508, device='cuda:0', grad_fn=<MseLossBackward>) 249\n",
      "tensor(1.2187, device='cuda:0', grad_fn=<MseLossBackward>) 250\n",
      "tensor(1.1784, device='cuda:0', grad_fn=<MseLossBackward>) 251\n",
      "tensor(1.2005, device='cuda:0', grad_fn=<MseLossBackward>) 252\n",
      "tensor(1.1764, device='cuda:0', grad_fn=<MseLossBackward>) 253\n",
      "tensor(1.1482, device='cuda:0', grad_fn=<MseLossBackward>) 254\n",
      "tensor(1.0946, device='cuda:0', grad_fn=<MseLossBackward>) 255\n",
      "tensor(1.2247, device='cuda:0', grad_fn=<MseLossBackward>) 256\n",
      "tensor(1.0862, device='cuda:0', grad_fn=<MseLossBackward>) 257\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(1.0129, device='cuda:0', grad_fn=<MseLossBackward>) 258\n",
      "tensor(1.0918, device='cuda:0', grad_fn=<MseLossBackward>) 259\n",
      "tensor(1.1519, device='cuda:0', grad_fn=<MseLossBackward>) 260\n",
      "tensor(1.0812, device='cuda:0', grad_fn=<MseLossBackward>) 261\n",
      "tensor(1.1470, device='cuda:0', grad_fn=<MseLossBackward>) 262\n",
      "tensor(1.2006, device='cuda:0', grad_fn=<MseLossBackward>) 263\n",
      "tensor(1.2274, device='cuda:0', grad_fn=<MseLossBackward>) 264\n",
      "tensor(1.1613, device='cuda:0', grad_fn=<MseLossBackward>) 265\n",
      "tensor(1.0851, device='cuda:0', grad_fn=<MseLossBackward>) 266\n",
      "tensor(1.1403, device='cuda:0', grad_fn=<MseLossBackward>) 267\n",
      "tensor(1.0891, device='cuda:0', grad_fn=<MseLossBackward>) 268\n",
      "tensor(1.3024, device='cuda:0', grad_fn=<MseLossBackward>) 269\n",
      "tensor(1.2351, device='cuda:0', grad_fn=<MseLossBackward>) 270\n",
      "tensor(1.1051, device='cuda:0', grad_fn=<MseLossBackward>) 271\n",
      "tensor(0.9337, device='cuda:0', grad_fn=<MseLossBackward>) 272\n",
      "tensor(0.9385, device='cuda:0', grad_fn=<MseLossBackward>) 273\n",
      "tensor(0.9816, device='cuda:0', grad_fn=<MseLossBackward>) 274\n",
      "tensor(0.9763, device='cuda:0', grad_fn=<MseLossBackward>) 275\n",
      "tensor(0.9888, device='cuda:0', grad_fn=<MseLossBackward>) 276\n",
      "tensor(1.0037, device='cuda:0', grad_fn=<MseLossBackward>) 277\n",
      "tensor(0.9961, device='cuda:0', grad_fn=<MseLossBackward>) 278\n",
      "tensor(0.9751, device='cuda:0', grad_fn=<MseLossBackward>) 279\n",
      "tensor(0.9746, device='cuda:0', grad_fn=<MseLossBackward>) 280\n",
      "tensor(1.0009, device='cuda:0', grad_fn=<MseLossBackward>) 281\n",
      "tensor(0.9964, device='cuda:0', grad_fn=<MseLossBackward>) 282\n",
      "tensor(0.9991, device='cuda:0', grad_fn=<MseLossBackward>) 283\n",
      "tensor(1.0107, device='cuda:0', grad_fn=<MseLossBackward>) 284\n",
      "tensor(1.0022, device='cuda:0', grad_fn=<MseLossBackward>) 285\n",
      "tensor(0.9399, device='cuda:0', grad_fn=<MseLossBackward>) 286\n",
      "tensor(0.9996, device='cuda:0', grad_fn=<MseLossBackward>) 287\n",
      "tensor(0.9732, device='cuda:0', grad_fn=<MseLossBackward>) 288\n",
      "tensor(0.9920, device='cuda:0', grad_fn=<MseLossBackward>) 289\n",
      "tensor(0.9935, device='cuda:0', grad_fn=<MseLossBackward>) 290\n",
      "tensor(1.0008, device='cuda:0', grad_fn=<MseLossBackward>) 291\n",
      "tensor(0.9794, device='cuda:0', grad_fn=<MseLossBackward>) 292\n",
      "tensor(1.0217, device='cuda:0', grad_fn=<MseLossBackward>) 293\n",
      "tensor(0.9642, device='cuda:0', grad_fn=<MseLossBackward>) 294\n",
      "tensor(1.0117, device='cuda:0', grad_fn=<MseLossBackward>) 295\n",
      "tensor(1.0351, device='cuda:0', grad_fn=<MseLossBackward>) 296\n",
      "tensor(1.0066, device='cuda:0', grad_fn=<MseLossBackward>) 297\n",
      "tensor(1.0270, device='cuda:0', grad_fn=<MseLossBackward>) 298\n",
      "tensor(1.0054, device='cuda:0', grad_fn=<MseLossBackward>) 299\n",
      "tensor(1.0356, device='cuda:0', grad_fn=<MseLossBackward>) 300\n",
      "tensor(1.0351, device='cuda:0', grad_fn=<MseLossBackward>) 301\n",
      "tensor(1.0164, device='cuda:0', grad_fn=<MseLossBackward>) 302\n",
      "tensor(1.0415, device='cuda:0', grad_fn=<MseLossBackward>) 303\n",
      "tensor(1.0350, device='cuda:0', grad_fn=<MseLossBackward>) 304\n",
      "tensor(1.0489, device='cuda:0', grad_fn=<MseLossBackward>) 305\n",
      "tensor(1.0184, device='cuda:0', grad_fn=<MseLossBackward>) 306\n",
      "tensor(1.0526, device='cuda:0', grad_fn=<MseLossBackward>) 307\n",
      "tensor(1.0425, device='cuda:0', grad_fn=<MseLossBackward>) 308\n",
      "tensor(1.0512, device='cuda:0', grad_fn=<MseLossBackward>) 309\n",
      "tensor(1.0367, device='cuda:0', grad_fn=<MseLossBackward>) 310\n",
      "tensor(1.0496, device='cuda:0', grad_fn=<MseLossBackward>) 311\n",
      "tensor(1.0320, device='cuda:0', grad_fn=<MseLossBackward>) 312\n",
      "tensor(1.0446, device='cuda:0', grad_fn=<MseLossBackward>) 313\n",
      "tensor(1.0452, device='cuda:0', grad_fn=<MseLossBackward>) 314\n",
      "tensor(1.0517, device='cuda:0', grad_fn=<MseLossBackward>) 315\n",
      "tensor(1.0506, device='cuda:0', grad_fn=<MseLossBackward>) 316\n",
      "tensor(1.0494, device='cuda:0', grad_fn=<MseLossBackward>) 317\n",
      "tensor(1.0474, device='cuda:0', grad_fn=<MseLossBackward>) 318\n",
      "tensor(1.0576, device='cuda:0', grad_fn=<MseLossBackward>) 319\n",
      "tensor(1.0300, device='cuda:0', grad_fn=<MseLossBackward>) 320\n",
      "tensor(1.0599, device='cuda:0', grad_fn=<MseLossBackward>) 321\n",
      "tensor(1.0392, device='cuda:0', grad_fn=<MseLossBackward>) 322\n",
      "tensor(1.0577, device='cuda:0', grad_fn=<MseLossBackward>) 323\n",
      "tensor(1.0392, device='cuda:0', grad_fn=<MseLossBackward>) 324\n",
      "tensor(1.0739, device='cuda:0', grad_fn=<MseLossBackward>) 325\n",
      "tensor(1.0477, device='cuda:0', grad_fn=<MseLossBackward>) 326\n",
      "tensor(1.0677, device='cuda:0', grad_fn=<MseLossBackward>) 327\n",
      "tensor(1.0399, device='cuda:0', grad_fn=<MseLossBackward>) 328\n",
      "tensor(1.0680, device='cuda:0', grad_fn=<MseLossBackward>) 329\n",
      "tensor(1.0440, device='cuda:0', grad_fn=<MseLossBackward>) 330\n",
      "tensor(1.0651, device='cuda:0', grad_fn=<MseLossBackward>) 331\n",
      "tensor(1.0406, device='cuda:0', grad_fn=<MseLossBackward>) 332\n",
      "tensor(1.0687, device='cuda:0', grad_fn=<MseLossBackward>) 333\n",
      "tensor(1.0434, device='cuda:0', grad_fn=<MseLossBackward>) 334\n",
      "tensor(1.0709, device='cuda:0', grad_fn=<MseLossBackward>) 335\n",
      "tensor(1.0436, device='cuda:0', grad_fn=<MseLossBackward>) 336\n",
      "tensor(1.0785, device='cuda:0', grad_fn=<MseLossBackward>) 337\n",
      "tensor(1.0520, device='cuda:0', grad_fn=<MseLossBackward>) 338\n",
      "tensor(1.0759, device='cuda:0', grad_fn=<MseLossBackward>) 339\n",
      "tensor(1.0583, device='cuda:0', grad_fn=<MseLossBackward>) 340\n",
      "tensor(1.0652, device='cuda:0', grad_fn=<MseLossBackward>) 341\n",
      "tensor(1.0668, device='cuda:0', grad_fn=<MseLossBackward>) 342\n",
      "tensor(1.0593, device='cuda:0', grad_fn=<MseLossBackward>) 343\n",
      "tensor(1.0711, device='cuda:0', grad_fn=<MseLossBackward>) 344\n",
      "tensor(1.0616, device='cuda:0', grad_fn=<MseLossBackward>) 345\n",
      "tensor(1.0695, device='cuda:0', grad_fn=<MseLossBackward>) 346\n",
      "tensor(1.0602, device='cuda:0', grad_fn=<MseLossBackward>) 347\n",
      "tensor(1.0754, device='cuda:0', grad_fn=<MseLossBackward>) 348\n",
      "tensor(1.0576, device='cuda:0', grad_fn=<MseLossBackward>) 349\n",
      "tensor(1.0756, device='cuda:0', grad_fn=<MseLossBackward>) 350\n",
      "tensor(1.0572, device='cuda:0', grad_fn=<MseLossBackward>) 351\n",
      "tensor(1.0768, device='cuda:0', grad_fn=<MseLossBackward>) 352\n",
      "tensor(1.0633, device='cuda:0', grad_fn=<MseLossBackward>) 353\n",
      "tensor(1.0769, device='cuda:0', grad_fn=<MseLossBackward>) 354\n",
      "tensor(1.0655, device='cuda:0', grad_fn=<MseLossBackward>) 355\n",
      "tensor(1.0757, device='cuda:0', grad_fn=<MseLossBackward>) 356\n",
      "tensor(1.0695, device='cuda:0', grad_fn=<MseLossBackward>) 357\n",
      "tensor(1.0760, device='cuda:0', grad_fn=<MseLossBackward>) 358\n",
      "tensor(1.0713, device='cuda:0', grad_fn=<MseLossBackward>) 359\n",
      "tensor(1.0747, device='cuda:0', grad_fn=<MseLossBackward>) 360\n",
      "tensor(1.0729, device='cuda:0', grad_fn=<MseLossBackward>) 361\n",
      "tensor(1.0704, device='cuda:0', grad_fn=<MseLossBackward>) 362\n",
      "tensor(1.0743, device='cuda:0', grad_fn=<MseLossBackward>) 363\n",
      "tensor(1.0704, device='cuda:0', grad_fn=<MseLossBackward>) 364\n",
      "tensor(1.0761, device='cuda:0', grad_fn=<MseLossBackward>) 365\n",
      "tensor(1.0699, device='cuda:0', grad_fn=<MseLossBackward>) 366\n",
      "tensor(1.0762, device='cuda:0', grad_fn=<MseLossBackward>) 367\n",
      "tensor(1.0691, device='cuda:0', grad_fn=<MseLossBackward>) 368\n",
      "tensor(1.0746, device='cuda:0', grad_fn=<MseLossBackward>) 369\n",
      "tensor(1.0686, device='cuda:0', grad_fn=<MseLossBackward>) 370\n",
      "tensor(1.0718, device='cuda:0', grad_fn=<MseLossBackward>) 371\n",
      "tensor(1.0671, device='cuda:0', grad_fn=<MseLossBackward>) 372\n",
      "tensor(1.0658, device='cuda:0', grad_fn=<MseLossBackward>) 373\n",
      "tensor(1.0561, device='cuda:0', grad_fn=<MseLossBackward>) 374\n",
      "tensor(1.0558, device='cuda:0', grad_fn=<MseLossBackward>) 375\n",
      "tensor(1.0585, device='cuda:0', grad_fn=<MseLossBackward>) 376\n",
      "tensor(1.0545, device='cuda:0', grad_fn=<MseLossBackward>) 377\n",
      "tensor(1.0593, device='cuda:0', grad_fn=<MseLossBackward>) 378\n",
      "tensor(1.0539, device='cuda:0', grad_fn=<MseLossBackward>) 379\n",
      "tensor(1.0559, device='cuda:0', grad_fn=<MseLossBackward>) 380\n",
      "tensor(1.0592, device='cuda:0', grad_fn=<MseLossBackward>) 381\n",
      "tensor(1.0540, device='cuda:0', grad_fn=<MseLossBackward>) 382\n",
      "tensor(1.0540, device='cuda:0', grad_fn=<MseLossBackward>) 383\n",
      "tensor(1.0540, device='cuda:0', grad_fn=<MseLossBackward>) 384\n",
      "tensor(1.0550, device='cuda:0', grad_fn=<MseLossBackward>) 385\n",
      "tensor(1.0543, device='cuda:0', grad_fn=<MseLossBackward>) 386\n",
      "tensor(1.0558, device='cuda:0', grad_fn=<MseLossBackward>) 387\n",
      "tensor(1.0544, device='cuda:0', grad_fn=<MseLossBackward>) 388\n",
      "tensor(1.0564, device='cuda:0', grad_fn=<MseLossBackward>) 389\n",
      "tensor(1.0548, device='cuda:0', grad_fn=<MseLossBackward>) 390\n",
      "tensor(1.0567, device='cuda:0', grad_fn=<MseLossBackward>) 391\n",
      "tensor(1.0556, device='cuda:0', grad_fn=<MseLossBackward>) 392\n",
      "tensor(1.0568, device='cuda:0', grad_fn=<MseLossBackward>) 393\n",
      "tensor(1.0564, device='cuda:0', grad_fn=<MseLossBackward>) 394\n",
      "tensor(1.0566, device='cuda:0', grad_fn=<MseLossBackward>) 395\n",
      "tensor(1.0571, device='cuda:0', grad_fn=<MseLossBackward>) 396\n",
      "tensor(1.0566, device='cuda:0', grad_fn=<MseLossBackward>) 397\n",
      "tensor(1.0576, device='cuda:0', grad_fn=<MseLossBackward>) 398\n",
      "tensor(1.0566, device='cuda:0', grad_fn=<MseLossBackward>) 399\n",
      "tensor(1.0578, device='cuda:0', grad_fn=<MseLossBackward>) 400\n",
      "tensor(1.0570, device='cuda:0', grad_fn=<MseLossBackward>) 401\n",
      "tensor(1.0576, device='cuda:0', grad_fn=<MseLossBackward>) 402\n",
      "tensor(1.0573, device='cuda:0', grad_fn=<MseLossBackward>) 403\n",
      "tensor(1.0575, device='cuda:0', grad_fn=<MseLossBackward>) 404\n",
      "tensor(1.0573, device='cuda:0', grad_fn=<MseLossBackward>) 405\n",
      "tensor(1.0574, device='cuda:0', grad_fn=<MseLossBackward>) 406\n",
      "tensor(1.0574, device='cuda:0', grad_fn=<MseLossBackward>) 407\n",
      "tensor(1.0572, device='cuda:0', grad_fn=<MseLossBackward>) 408\n",
      "tensor(1.0573, device='cuda:0', grad_fn=<MseLossBackward>) 409\n",
      "tensor(1.0570, device='cuda:0', grad_fn=<MseLossBackward>) 410\n",
      "tensor(1.0570, device='cuda:0', grad_fn=<MseLossBackward>) 411\n",
      "tensor(1.0567, device='cuda:0', grad_fn=<MseLossBackward>) 412\n",
      "tensor(1.0568, device='cuda:0', grad_fn=<MseLossBackward>) 413\n",
      "tensor(1.0565, device='cuda:0', grad_fn=<MseLossBackward>) 414\n",
      "tensor(1.0563, device='cuda:0', grad_fn=<MseLossBackward>) 415\n",
      "tensor(1.0560, device='cuda:0', grad_fn=<MseLossBackward>) 416\n",
      "tensor(1.0558, device='cuda:0', grad_fn=<MseLossBackward>) 417\n",
      "tensor(1.0555, device='cuda:0', grad_fn=<MseLossBackward>) 418\n",
      "tensor(1.0552, device='cuda:0', grad_fn=<MseLossBackward>) 419\n",
      "tensor(1.0549, device='cuda:0', grad_fn=<MseLossBackward>) 420\n",
      "tensor(1.0547, device='cuda:0', grad_fn=<MseLossBackward>) 421\n",
      "tensor(1.0544, device='cuda:0', grad_fn=<MseLossBackward>) 422\n",
      "tensor(1.0541, device='cuda:0', grad_fn=<MseLossBackward>) 423\n",
      "tensor(1.0538, device='cuda:0', grad_fn=<MseLossBackward>) 424\n",
      "tensor(1.0535, device='cuda:0', grad_fn=<MseLossBackward>) 425\n",
      "tensor(1.0531, device='cuda:0', grad_fn=<MseLossBackward>) 426\n",
      "tensor(1.0528, device='cuda:0', grad_fn=<MseLossBackward>) 427\n",
      "tensor(1.0525, device='cuda:0', grad_fn=<MseLossBackward>) 428\n",
      "tensor(1.0522, device='cuda:0', grad_fn=<MseLossBackward>) 429\n",
      "tensor(1.0519, device='cuda:0', grad_fn=<MseLossBackward>) 430\n",
      "tensor(1.0516, device='cuda:0', grad_fn=<MseLossBackward>) 431\n",
      "tensor(1.0513, device='cuda:0', grad_fn=<MseLossBackward>) 432\n",
      "tensor(1.0510, device='cuda:0', grad_fn=<MseLossBackward>) 433\n",
      "tensor(1.0506, device='cuda:0', grad_fn=<MseLossBackward>) 434\n",
      "tensor(1.0503, device='cuda:0', grad_fn=<MseLossBackward>) 435\n",
      "tensor(1.0500, device='cuda:0', grad_fn=<MseLossBackward>) 436\n",
      "tensor(1.0498, device='cuda:0', grad_fn=<MseLossBackward>) 437\n",
      "tensor(1.0495, device='cuda:0', grad_fn=<MseLossBackward>) 438\n",
      "tensor(1.0492, device='cuda:0', grad_fn=<MseLossBackward>) 439\n",
      "tensor(1.0488, device='cuda:0', grad_fn=<MseLossBackward>) 440\n",
      "tensor(1.0485, device='cuda:0', grad_fn=<MseLossBackward>) 441\n",
      "tensor(1.0482, device='cuda:0', grad_fn=<MseLossBackward>) 442\n",
      "tensor(1.0479, device='cuda:0', grad_fn=<MseLossBackward>) 443\n",
      "tensor(1.0476, device='cuda:0', grad_fn=<MseLossBackward>) 444\n",
      "tensor(1.0473, device='cuda:0', grad_fn=<MseLossBackward>) 445\n",
      "tensor(1.0470, device='cuda:0', grad_fn=<MseLossBackward>) 446\n",
      "tensor(1.0467, device='cuda:0', grad_fn=<MseLossBackward>) 447\n",
      "tensor(1.0464, device='cuda:0', grad_fn=<MseLossBackward>) 448\n",
      "tensor(1.0461, device='cuda:0', grad_fn=<MseLossBackward>) 449\n",
      "tensor(1.0458, device='cuda:0', grad_fn=<MseLossBackward>) 450\n",
      "tensor(1.0455, device='cuda:0', grad_fn=<MseLossBackward>) 451\n",
      "tensor(1.0452, device='cuda:0', grad_fn=<MseLossBackward>) 452\n",
      "tensor(1.0449, device='cuda:0', grad_fn=<MseLossBackward>) 453\n",
      "tensor(1.0446, device='cuda:0', grad_fn=<MseLossBackward>) 454\n",
      "tensor(1.0443, device='cuda:0', grad_fn=<MseLossBackward>) 455\n",
      "tensor(1.0440, device='cuda:0', grad_fn=<MseLossBackward>) 456\n",
      "tensor(1.0437, device='cuda:0', grad_fn=<MseLossBackward>) 457\n",
      "tensor(1.0434, device='cuda:0', grad_fn=<MseLossBackward>) 458\n",
      "tensor(1.0432, device='cuda:0', grad_fn=<MseLossBackward>) 459\n",
      "tensor(1.0429, device='cuda:0', grad_fn=<MseLossBackward>) 460\n",
      "tensor(1.0426, device='cuda:0', grad_fn=<MseLossBackward>) 461\n",
      "tensor(1.0423, device='cuda:0', grad_fn=<MseLossBackward>) 462\n",
      "tensor(1.0421, device='cuda:0', grad_fn=<MseLossBackward>) 463\n",
      "tensor(1.0418, device='cuda:0', grad_fn=<MseLossBackward>) 464\n",
      "tensor(1.0415, device='cuda:0', grad_fn=<MseLossBackward>) 465\n",
      "tensor(1.0412, device='cuda:0', grad_fn=<MseLossBackward>) 466\n",
      "tensor(1.0410, device='cuda:0', grad_fn=<MseLossBackward>) 467\n",
      "tensor(1.0409, device='cuda:0', grad_fn=<MseLossBackward>) 468\n",
      "tensor(1.0406, device='cuda:0', grad_fn=<MseLossBackward>) 469\n",
      "tensor(1.0403, device='cuda:0', grad_fn=<MseLossBackward>) 470\n",
      "tensor(1.0401, device='cuda:0', grad_fn=<MseLossBackward>) 471\n",
      "tensor(1.0398, device='cuda:0', grad_fn=<MseLossBackward>) 472\n",
      "tensor(1.0395, device='cuda:0', grad_fn=<MseLossBackward>) 473\n",
      "tensor(1.0393, device='cuda:0', grad_fn=<MseLossBackward>) 474\n",
      "tensor(1.0390, device='cuda:0', grad_fn=<MseLossBackward>) 475\n",
      "tensor(1.0388, device='cuda:0', grad_fn=<MseLossBackward>) 476\n",
      "tensor(1.0385, device='cuda:0', grad_fn=<MseLossBackward>) 477\n",
      "tensor(1.0382, device='cuda:0', grad_fn=<MseLossBackward>) 478\n",
      "tensor(1.0379, device='cuda:0', grad_fn=<MseLossBackward>) 479\n",
      "tensor(1.0377, device='cuda:0', grad_fn=<MseLossBackward>) 480\n",
      "tensor(1.0374, device='cuda:0', grad_fn=<MseLossBackward>) 481\n",
      "tensor(1.0371, device='cuda:0', grad_fn=<MseLossBackward>) 482\n",
      "tensor(1.0369, device='cuda:0', grad_fn=<MseLossBackward>) 483\n",
      "tensor(1.0366, device='cuda:0', grad_fn=<MseLossBackward>) 484\n",
      "tensor(1.0364, device='cuda:0', grad_fn=<MseLossBackward>) 485\n",
      "tensor(1.0361, device='cuda:0', grad_fn=<MseLossBackward>) 486\n",
      "tensor(1.0358, device='cuda:0', grad_fn=<MseLossBackward>) 487\n",
      "tensor(1.0356, device='cuda:0', grad_fn=<MseLossBackward>) 488\n",
      "tensor(1.0353, device='cuda:0', grad_fn=<MseLossBackward>) 489\n",
      "tensor(1.0351, device='cuda:0', grad_fn=<MseLossBackward>) 490\n",
      "tensor(1.0348, device='cuda:0', grad_fn=<MseLossBackward>) 491\n",
      "tensor(1.0345, device='cuda:0', grad_fn=<MseLossBackward>) 492\n",
      "tensor(1.0343, device='cuda:0', grad_fn=<MseLossBackward>) 493\n",
      "tensor(1.0340, device='cuda:0', grad_fn=<MseLossBackward>) 494\n",
      "tensor(1.0338, device='cuda:0', grad_fn=<MseLossBackward>) 495\n",
      "tensor(1.0335, device='cuda:0', grad_fn=<MseLossBackward>) 496\n",
      "tensor(1.0333, device='cuda:0', grad_fn=<MseLossBackward>) 497\n",
      "tensor(1.0330, device='cuda:0', grad_fn=<MseLossBackward>) 498\n",
      "tensor(1.0328, device='cuda:0', grad_fn=<MseLossBackward>) 499\n",
      "tensor(1.0325, device='cuda:0', grad_fn=<MseLossBackward>) 500\n",
      "tensor(1.0323, device='cuda:0', grad_fn=<MseLossBackward>) 501\n",
      "tensor(1.0320, device='cuda:0', grad_fn=<MseLossBackward>) 502\n",
      "tensor(1.0317, device='cuda:0', grad_fn=<MseLossBackward>) 503\n",
      "tensor(1.0314, device='cuda:0', grad_fn=<MseLossBackward>) 504\n",
      "tensor(1.0311, device='cuda:0', grad_fn=<MseLossBackward>) 505\n",
      "tensor(1.0309, device='cuda:0', grad_fn=<MseLossBackward>) 506\n",
      "tensor(1.0307, device='cuda:0', grad_fn=<MseLossBackward>) 507\n",
      "tensor(1.0304, device='cuda:0', grad_fn=<MseLossBackward>) 508\n",
      "tensor(1.0302, device='cuda:0', grad_fn=<MseLossBackward>) 509\n",
      "tensor(1.0300, device='cuda:0', grad_fn=<MseLossBackward>) 510\n",
      "tensor(1.0297, device='cuda:0', grad_fn=<MseLossBackward>) 511\n",
      "tensor(1.0295, device='cuda:0', grad_fn=<MseLossBackward>) 512\n",
      "tensor(1.0293, device='cuda:0', grad_fn=<MseLossBackward>) 513\n",
      "tensor(1.0291, device='cuda:0', grad_fn=<MseLossBackward>) 514\n",
      "tensor(1.0289, device='cuda:0', grad_fn=<MseLossBackward>) 515\n",
      "tensor(1.0286, device='cuda:0', grad_fn=<MseLossBackward>) 516\n",
      "tensor(1.0284, device='cuda:0', grad_fn=<MseLossBackward>) 517\n",
      "tensor(1.0282, device='cuda:0', grad_fn=<MseLossBackward>) 518\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(1.0280, device='cuda:0', grad_fn=<MseLossBackward>) 519\n",
      "tensor(1.0278, device='cuda:0', grad_fn=<MseLossBackward>) 520\n",
      "tensor(1.0276, device='cuda:0', grad_fn=<MseLossBackward>) 521\n",
      "tensor(1.0274, device='cuda:0', grad_fn=<MseLossBackward>) 522\n",
      "tensor(1.0271, device='cuda:0', grad_fn=<MseLossBackward>) 523\n",
      "tensor(1.0269, device='cuda:0', grad_fn=<MseLossBackward>) 524\n",
      "tensor(1.0267, device='cuda:0', grad_fn=<MseLossBackward>) 525\n",
      "tensor(1.0265, device='cuda:0', grad_fn=<MseLossBackward>) 526\n",
      "tensor(1.0263, device='cuda:0', grad_fn=<MseLossBackward>) 527\n",
      "tensor(1.0260, device='cuda:0', grad_fn=<MseLossBackward>) 528\n",
      "tensor(1.0258, device='cuda:0', grad_fn=<MseLossBackward>) 529\n",
      "tensor(1.0256, device='cuda:0', grad_fn=<MseLossBackward>) 530\n",
      "tensor(1.0255, device='cuda:0', grad_fn=<MseLossBackward>) 531\n",
      "tensor(1.0252, device='cuda:0', grad_fn=<MseLossBackward>) 532\n",
      "tensor(1.0250, device='cuda:0', grad_fn=<MseLossBackward>) 533\n",
      "tensor(1.0248, device='cuda:0', grad_fn=<MseLossBackward>) 534\n",
      "tensor(1.0246, device='cuda:0', grad_fn=<MseLossBackward>) 535\n",
      "tensor(1.0244, device='cuda:0', grad_fn=<MseLossBackward>) 536\n",
      "tensor(1.0242, device='cuda:0', grad_fn=<MseLossBackward>) 537\n",
      "tensor(1.0241, device='cuda:0', grad_fn=<MseLossBackward>) 538\n",
      "tensor(1.0239, device='cuda:0', grad_fn=<MseLossBackward>) 539\n",
      "tensor(1.0237, device='cuda:0', grad_fn=<MseLossBackward>) 540\n",
      "tensor(1.0235, device='cuda:0', grad_fn=<MseLossBackward>) 541\n",
      "tensor(1.0233, device='cuda:0', grad_fn=<MseLossBackward>) 542\n",
      "tensor(1.0231, device='cuda:0', grad_fn=<MseLossBackward>) 543\n",
      "tensor(1.0230, device='cuda:0', grad_fn=<MseLossBackward>) 544\n",
      "tensor(1.0228, device='cuda:0', grad_fn=<MseLossBackward>) 545\n",
      "tensor(1.0226, device='cuda:0', grad_fn=<MseLossBackward>) 546\n",
      "tensor(1.0224, device='cuda:0', grad_fn=<MseLossBackward>) 547\n",
      "tensor(1.0222, device='cuda:0', grad_fn=<MseLossBackward>) 548\n",
      "tensor(1.0220, device='cuda:0', grad_fn=<MseLossBackward>) 549\n",
      "tensor(1.0220, device='cuda:0', grad_fn=<MseLossBackward>) 550\n",
      "tensor(1.0219, device='cuda:0', grad_fn=<MseLossBackward>) 551\n",
      "tensor(1.0218, device='cuda:0', grad_fn=<MseLossBackward>) 552\n",
      "tensor(1.0217, device='cuda:0', grad_fn=<MseLossBackward>) 553\n",
      "tensor(1.0216, device='cuda:0', grad_fn=<MseLossBackward>) 554\n",
      "tensor(1.0215, device='cuda:0', grad_fn=<MseLossBackward>) 555\n",
      "tensor(1.0215, device='cuda:0', grad_fn=<MseLossBackward>) 556\n",
      "tensor(1.0214, device='cuda:0', grad_fn=<MseLossBackward>) 557\n",
      "tensor(1.0213, device='cuda:0', grad_fn=<MseLossBackward>) 558\n",
      "tensor(1.0212, device='cuda:0', grad_fn=<MseLossBackward>) 559\n",
      "tensor(1.0211, device='cuda:0', grad_fn=<MseLossBackward>) 560\n",
      "tensor(1.0211, device='cuda:0', grad_fn=<MseLossBackward>) 561\n",
      "tensor(1.0209, device='cuda:0', grad_fn=<MseLossBackward>) 562\n",
      "tensor(1.0207, device='cuda:0', grad_fn=<MseLossBackward>) 563\n",
      "tensor(1.0206, device='cuda:0', grad_fn=<MseLossBackward>) 564\n",
      "tensor(1.0204, device='cuda:0', grad_fn=<MseLossBackward>) 565\n",
      "tensor(1.0203, device='cuda:0', grad_fn=<MseLossBackward>) 566\n",
      "tensor(1.0201, device='cuda:0', grad_fn=<MseLossBackward>) 567\n",
      "tensor(1.0200, device='cuda:0', grad_fn=<MseLossBackward>) 568\n",
      "tensor(1.0198, device='cuda:0', grad_fn=<MseLossBackward>) 569\n",
      "tensor(1.0197, device='cuda:0', grad_fn=<MseLossBackward>) 570\n",
      "tensor(1.0195, device='cuda:0', grad_fn=<MseLossBackward>) 571\n",
      "tensor(1.0196, device='cuda:0', grad_fn=<MseLossBackward>) 572\n",
      "tensor(1.0193, device='cuda:0', grad_fn=<MseLossBackward>) 573\n",
      "tensor(1.0191, device='cuda:0', grad_fn=<MseLossBackward>) 574\n",
      "tensor(1.0190, device='cuda:0', grad_fn=<MseLossBackward>) 575\n",
      "tensor(1.0188, device='cuda:0', grad_fn=<MseLossBackward>) 576\n",
      "tensor(1.0186, device='cuda:0', grad_fn=<MseLossBackward>) 577\n",
      "tensor(1.0185, device='cuda:0', grad_fn=<MseLossBackward>) 578\n",
      "tensor(1.0183, device='cuda:0', grad_fn=<MseLossBackward>) 579\n",
      "tensor(1.0182, device='cuda:0', grad_fn=<MseLossBackward>) 580\n",
      "tensor(1.0180, device='cuda:0', grad_fn=<MseLossBackward>) 581\n",
      "tensor(1.0179, device='cuda:0', grad_fn=<MseLossBackward>) 582\n",
      "tensor(1.0178, device='cuda:0', grad_fn=<MseLossBackward>) 583\n",
      "tensor(1.0176, device='cuda:0', grad_fn=<MseLossBackward>) 584\n",
      "tensor(1.0173, device='cuda:0', grad_fn=<MseLossBackward>) 585\n",
      "tensor(1.0170, device='cuda:0', grad_fn=<MseLossBackward>) 586\n",
      "tensor(1.0167, device='cuda:0', grad_fn=<MseLossBackward>) 587\n",
      "tensor(1.0163, device='cuda:0', grad_fn=<MseLossBackward>) 588\n",
      "tensor(1.0160, device='cuda:0', grad_fn=<MseLossBackward>) 589\n",
      "tensor(1.0157, device='cuda:0', grad_fn=<MseLossBackward>) 590\n",
      "tensor(1.0156, device='cuda:0', grad_fn=<MseLossBackward>) 591\n",
      "tensor(1.0154, device='cuda:0', grad_fn=<MseLossBackward>) 592\n",
      "tensor(1.0152, device='cuda:0', grad_fn=<MseLossBackward>) 593\n",
      "tensor(1.0151, device='cuda:0', grad_fn=<MseLossBackward>) 594\n",
      "tensor(1.0149, device='cuda:0', grad_fn=<MseLossBackward>) 595\n",
      "tensor(1.0148, device='cuda:0', grad_fn=<MseLossBackward>) 596\n",
      "tensor(1.0146, device='cuda:0', grad_fn=<MseLossBackward>) 597\n",
      "tensor(1.0145, device='cuda:0', grad_fn=<MseLossBackward>) 598\n",
      "tensor(1.0143, device='cuda:0', grad_fn=<MseLossBackward>) 599\n",
      "tensor(1.0142, device='cuda:0', grad_fn=<MseLossBackward>) 600\n",
      "tensor(1.0141, device='cuda:0', grad_fn=<MseLossBackward>) 601\n",
      "tensor(1.0141, device='cuda:0', grad_fn=<MseLossBackward>) 602\n",
      "tensor(1.0140, device='cuda:0', grad_fn=<MseLossBackward>) 603\n",
      "tensor(1.0140, device='cuda:0', grad_fn=<MseLossBackward>) 604\n",
      "tensor(1.0140, device='cuda:0', grad_fn=<MseLossBackward>) 605\n",
      "tensor(1.0139, device='cuda:0', grad_fn=<MseLossBackward>) 606\n",
      "tensor(1.0138, device='cuda:0', grad_fn=<MseLossBackward>) 607\n",
      "tensor(1.0138, device='cuda:0', grad_fn=<MseLossBackward>) 608\n",
      "tensor(1.0137, device='cuda:0', grad_fn=<MseLossBackward>) 609\n",
      "tensor(1.0137, device='cuda:0', grad_fn=<MseLossBackward>) 610\n",
      "tensor(1.0136, device='cuda:0', grad_fn=<MseLossBackward>) 611\n",
      "tensor(1.0136, device='cuda:0', grad_fn=<MseLossBackward>) 612\n",
      "tensor(1.0135, device='cuda:0', grad_fn=<MseLossBackward>) 613\n",
      "tensor(1.0134, device='cuda:0', grad_fn=<MseLossBackward>) 614\n",
      "tensor(1.0133, device='cuda:0', grad_fn=<MseLossBackward>) 615\n",
      "tensor(1.0133, device='cuda:0', grad_fn=<MseLossBackward>) 616\n",
      "tensor(1.0132, device='cuda:0', grad_fn=<MseLossBackward>) 617\n",
      "tensor(1.0131, device='cuda:0', grad_fn=<MseLossBackward>) 618\n",
      "tensor(1.0130, device='cuda:0', grad_fn=<MseLossBackward>) 619\n",
      "tensor(1.0130, device='cuda:0', grad_fn=<MseLossBackward>) 620\n",
      "tensor(1.0129, device='cuda:0', grad_fn=<MseLossBackward>) 621\n",
      "tensor(1.0127, device='cuda:0', grad_fn=<MseLossBackward>) 622\n",
      "tensor(1.0126, device='cuda:0', grad_fn=<MseLossBackward>) 623\n",
      "tensor(1.0125, device='cuda:0', grad_fn=<MseLossBackward>) 624\n",
      "tensor(1.0122, device='cuda:0', grad_fn=<MseLossBackward>) 625\n",
      "tensor(1.0120, device='cuda:0', grad_fn=<MseLossBackward>) 626\n",
      "tensor(1.0120, device='cuda:0', grad_fn=<MseLossBackward>) 627\n",
      "tensor(1.0118, device='cuda:0', grad_fn=<MseLossBackward>) 628\n",
      "tensor(1.0117, device='cuda:0', grad_fn=<MseLossBackward>) 629\n",
      "tensor(1.0116, device='cuda:0', grad_fn=<MseLossBackward>) 630\n",
      "tensor(1.0115, device='cuda:0', grad_fn=<MseLossBackward>) 631\n",
      "tensor(1.0114, device='cuda:0', grad_fn=<MseLossBackward>) 632\n",
      "tensor(1.0113, device='cuda:0', grad_fn=<MseLossBackward>) 633\n",
      "tensor(1.0112, device='cuda:0', grad_fn=<MseLossBackward>) 634\n",
      "tensor(1.0111, device='cuda:0', grad_fn=<MseLossBackward>) 635\n",
      "tensor(1.0109, device='cuda:0', grad_fn=<MseLossBackward>) 636\n",
      "tensor(1.0108, device='cuda:0', grad_fn=<MseLossBackward>) 637\n",
      "tensor(1.0107, device='cuda:0', grad_fn=<MseLossBackward>) 638\n",
      "tensor(1.0106, device='cuda:0', grad_fn=<MseLossBackward>) 639\n",
      "tensor(1.0104, device='cuda:0', grad_fn=<MseLossBackward>) 640\n",
      "tensor(1.0103, device='cuda:0', grad_fn=<MseLossBackward>) 641\n",
      "tensor(1.0102, device='cuda:0', grad_fn=<MseLossBackward>) 642\n",
      "tensor(1.0101, device='cuda:0', grad_fn=<MseLossBackward>) 643\n",
      "tensor(1.0100, device='cuda:0', grad_fn=<MseLossBackward>) 644\n",
      "tensor(1.0099, device='cuda:0', grad_fn=<MseLossBackward>) 645\n",
      "tensor(1.0098, device='cuda:0', grad_fn=<MseLossBackward>) 646\n",
      "tensor(1.0097, device='cuda:0', grad_fn=<MseLossBackward>) 647\n",
      "tensor(1.0094, device='cuda:0', grad_fn=<MseLossBackward>) 648\n",
      "tensor(1.0093, device='cuda:0', grad_fn=<MseLossBackward>) 649\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(1.0092, device='cuda:0', grad_fn=<MseLossBackward>) 650\n",
      "tensor(1.0091, device='cuda:0', grad_fn=<MseLossBackward>) 651\n",
      "tensor(1.0089, device='cuda:0', grad_fn=<MseLossBackward>) 652\n",
      "tensor(1.0089, device='cuda:0', grad_fn=<MseLossBackward>) 653\n",
      "tensor(1.0088, device='cuda:0', grad_fn=<MseLossBackward>) 654\n",
      "tensor(1.0088, device='cuda:0', grad_fn=<MseLossBackward>) 655\n",
      "tensor(1.0087, device='cuda:0', grad_fn=<MseLossBackward>) 656\n",
      "tensor(1.0087, device='cuda:0', grad_fn=<MseLossBackward>) 657\n",
      "tensor(1.0086, device='cuda:0', grad_fn=<MseLossBackward>) 658\n",
      "tensor(1.0086, device='cuda:0', grad_fn=<MseLossBackward>) 659\n",
      "tensor(1.0086, device='cuda:0', grad_fn=<MseLossBackward>) 660\n",
      "tensor(1.0086, device='cuda:0', grad_fn=<MseLossBackward>) 661\n",
      "tensor(1.0085, device='cuda:0', grad_fn=<MseLossBackward>) 662\n",
      "tensor(1.0085, device='cuda:0', grad_fn=<MseLossBackward>) 663\n",
      "tensor(1.0083, device='cuda:0', grad_fn=<MseLossBackward>) 664\n",
      "tensor(1.0082, device='cuda:0', grad_fn=<MseLossBackward>) 665\n",
      "tensor(1.0081, device='cuda:0', grad_fn=<MseLossBackward>) 666\n",
      "tensor(1.0080, device='cuda:0', grad_fn=<MseLossBackward>) 667\n",
      "tensor(1.0079, device='cuda:0', grad_fn=<MseLossBackward>) 668\n",
      "tensor(1.0079, device='cuda:0', grad_fn=<MseLossBackward>) 669\n",
      "tensor(1.0071, device='cuda:0', grad_fn=<MseLossBackward>) 670\n",
      "tensor(1.0067, device='cuda:0', grad_fn=<MseLossBackward>) 671\n",
      "tensor(1.0066, device='cuda:0', grad_fn=<MseLossBackward>) 672\n",
      "tensor(1.0064, device='cuda:0', grad_fn=<MseLossBackward>) 673\n",
      "tensor(1.0063, device='cuda:0', grad_fn=<MseLossBackward>) 674\n",
      "tensor(1.0062, device='cuda:0', grad_fn=<MseLossBackward>) 675\n",
      "tensor(1.0062, device='cuda:0', grad_fn=<MseLossBackward>) 676\n",
      "tensor(1.0058, device='cuda:0', grad_fn=<MseLossBackward>) 677\n",
      "tensor(1.0058, device='cuda:0', grad_fn=<MseLossBackward>) 678\n",
      "tensor(1.0057, device='cuda:0', grad_fn=<MseLossBackward>) 679\n",
      "tensor(1.0056, device='cuda:0', grad_fn=<MseLossBackward>) 680\n",
      "tensor(1.0055, device='cuda:0', grad_fn=<MseLossBackward>) 681\n",
      "tensor(1.0055, device='cuda:0', grad_fn=<MseLossBackward>) 682\n",
      "tensor(1.0054, device='cuda:0', grad_fn=<MseLossBackward>) 683\n",
      "tensor(1.0053, device='cuda:0', grad_fn=<MseLossBackward>) 684\n",
      "tensor(1.0052, device='cuda:0', grad_fn=<MseLossBackward>) 685\n",
      "tensor(1.0052, device='cuda:0', grad_fn=<MseLossBackward>) 686\n",
      "tensor(1.0050, device='cuda:0', grad_fn=<MseLossBackward>) 687\n",
      "tensor(1.0050, device='cuda:0', grad_fn=<MseLossBackward>) 688\n",
      "tensor(1.0049, device='cuda:0', grad_fn=<MseLossBackward>) 689\n",
      "tensor(1.0048, device='cuda:0', grad_fn=<MseLossBackward>) 690\n",
      "tensor(1.0047, device='cuda:0', grad_fn=<MseLossBackward>) 691\n",
      "tensor(1.0047, device='cuda:0', grad_fn=<MseLossBackward>) 692\n",
      "tensor(1.0045, device='cuda:0', grad_fn=<MseLossBackward>) 693\n",
      "tensor(1.0046, device='cuda:0', grad_fn=<MseLossBackward>) 694\n",
      "tensor(1.0038, device='cuda:0', grad_fn=<MseLossBackward>) 695\n",
      "tensor(1.0035, device='cuda:0', grad_fn=<MseLossBackward>) 696\n",
      "tensor(1.0033, device='cuda:0', grad_fn=<MseLossBackward>) 697\n",
      "tensor(1.0033, device='cuda:0', grad_fn=<MseLossBackward>) 698\n",
      "tensor(1.0032, device='cuda:0', grad_fn=<MseLossBackward>) 699\n",
      "tensor(1.0030, device='cuda:0', grad_fn=<MseLossBackward>) 700\n",
      "tensor(1.0028, device='cuda:0', grad_fn=<MseLossBackward>) 701\n",
      "tensor(1.0028, device='cuda:0', grad_fn=<MseLossBackward>) 702\n",
      "tensor(1.0025, device='cuda:0', grad_fn=<MseLossBackward>) 703\n",
      "tensor(1.0025, device='cuda:0', grad_fn=<MseLossBackward>) 704\n",
      "tensor(1.0022, device='cuda:0', grad_fn=<MseLossBackward>) 705\n",
      "tensor(1.0022, device='cuda:0', grad_fn=<MseLossBackward>) 706\n",
      "tensor(1.0019, device='cuda:0', grad_fn=<MseLossBackward>) 707\n",
      "tensor(1.0020, device='cuda:0', grad_fn=<MseLossBackward>) 708\n",
      "tensor(1.0016, device='cuda:0', grad_fn=<MseLossBackward>) 709\n",
      "tensor(1.0017, device='cuda:0', grad_fn=<MseLossBackward>) 710\n",
      "tensor(1.0015, device='cuda:0', grad_fn=<MseLossBackward>) 711\n",
      "tensor(1.0017, device='cuda:0', grad_fn=<MseLossBackward>) 712\n",
      "tensor(1.0014, device='cuda:0', grad_fn=<MseLossBackward>) 713\n",
      "tensor(1.0015, device='cuda:0', grad_fn=<MseLossBackward>) 714\n",
      "tensor(1.0010, device='cuda:0', grad_fn=<MseLossBackward>) 715\n",
      "tensor(1.0011, device='cuda:0', grad_fn=<MseLossBackward>) 716\n",
      "tensor(1.0008, device='cuda:0', grad_fn=<MseLossBackward>) 717\n",
      "tensor(1.0008, device='cuda:0', grad_fn=<MseLossBackward>) 718\n",
      "tensor(1.0003, device='cuda:0', grad_fn=<MseLossBackward>) 719\n",
      "tensor(1.0005, device='cuda:0', grad_fn=<MseLossBackward>) 720\n",
      "tensor(1.0000, device='cuda:0', grad_fn=<MseLossBackward>) 721\n",
      "tensor(1.0001, device='cuda:0', grad_fn=<MseLossBackward>) 722\n",
      "tensor(0.9998, device='cuda:0', grad_fn=<MseLossBackward>) 723\n",
      "tensor(1.0000, device='cuda:0', grad_fn=<MseLossBackward>) 724\n",
      "tensor(0.9996, device='cuda:0', grad_fn=<MseLossBackward>) 725\n",
      "tensor(0.9998, device='cuda:0', grad_fn=<MseLossBackward>) 726\n",
      "tensor(0.9994, device='cuda:0', grad_fn=<MseLossBackward>) 727\n",
      "tensor(0.9995, device='cuda:0', grad_fn=<MseLossBackward>) 728\n",
      "tensor(0.9991, device='cuda:0', grad_fn=<MseLossBackward>) 729\n",
      "tensor(0.9991, device='cuda:0', grad_fn=<MseLossBackward>) 730\n",
      "tensor(0.9987, device='cuda:0', grad_fn=<MseLossBackward>) 731\n",
      "tensor(0.9987, device='cuda:0', grad_fn=<MseLossBackward>) 732\n",
      "tensor(0.9984, device='cuda:0', grad_fn=<MseLossBackward>) 733\n",
      "tensor(0.9984, device='cuda:0', grad_fn=<MseLossBackward>) 734\n",
      "tensor(0.9980, device='cuda:0', grad_fn=<MseLossBackward>) 735\n",
      "tensor(0.9981, device='cuda:0', grad_fn=<MseLossBackward>) 736\n",
      "tensor(0.9977, device='cuda:0', grad_fn=<MseLossBackward>) 737\n",
      "tensor(0.9977, device='cuda:0', grad_fn=<MseLossBackward>) 738\n",
      "tensor(0.9973, device='cuda:0', grad_fn=<MseLossBackward>) 739\n",
      "tensor(0.9974, device='cuda:0', grad_fn=<MseLossBackward>) 740\n",
      "tensor(0.9969, device='cuda:0', grad_fn=<MseLossBackward>) 741\n",
      "tensor(0.9970, device='cuda:0', grad_fn=<MseLossBackward>) 742\n",
      "tensor(0.9966, device='cuda:0', grad_fn=<MseLossBackward>) 743\n",
      "tensor(0.9966, device='cuda:0', grad_fn=<MseLossBackward>) 744\n",
      "tensor(0.9962, device='cuda:0', grad_fn=<MseLossBackward>) 745\n",
      "tensor(0.9963, device='cuda:0', grad_fn=<MseLossBackward>) 746\n",
      "tensor(0.9958, device='cuda:0', grad_fn=<MseLossBackward>) 747\n",
      "tensor(0.9959, device='cuda:0', grad_fn=<MseLossBackward>) 748\n",
      "tensor(0.9955, device='cuda:0', grad_fn=<MseLossBackward>) 749\n",
      "tensor(0.9956, device='cuda:0', grad_fn=<MseLossBackward>) 750\n",
      "tensor(0.9951, device='cuda:0', grad_fn=<MseLossBackward>) 751\n",
      "tensor(0.9952, device='cuda:0', grad_fn=<MseLossBackward>) 752\n",
      "tensor(0.9947, device='cuda:0', grad_fn=<MseLossBackward>) 753\n",
      "tensor(0.9948, device='cuda:0', grad_fn=<MseLossBackward>) 754\n",
      "tensor(0.9943, device='cuda:0', grad_fn=<MseLossBackward>) 755\n",
      "tensor(0.9944, device='cuda:0', grad_fn=<MseLossBackward>) 756\n",
      "tensor(0.9939, device='cuda:0', grad_fn=<MseLossBackward>) 757\n",
      "tensor(0.9939, device='cuda:0', grad_fn=<MseLossBackward>) 758\n",
      "tensor(0.9935, device='cuda:0', grad_fn=<MseLossBackward>) 759\n",
      "tensor(0.9935, device='cuda:0', grad_fn=<MseLossBackward>) 760\n",
      "tensor(0.9930, device='cuda:0', grad_fn=<MseLossBackward>) 761\n",
      "tensor(0.9931, device='cuda:0', grad_fn=<MseLossBackward>) 762\n",
      "tensor(0.9926, device='cuda:0', grad_fn=<MseLossBackward>) 763\n",
      "tensor(0.9927, device='cuda:0', grad_fn=<MseLossBackward>) 764\n",
      "tensor(0.9922, device='cuda:0', grad_fn=<MseLossBackward>) 765\n",
      "tensor(0.9924, device='cuda:0', grad_fn=<MseLossBackward>) 766\n",
      "tensor(0.9918, device='cuda:0', grad_fn=<MseLossBackward>) 767\n",
      "tensor(0.9920, device='cuda:0', grad_fn=<MseLossBackward>) 768\n",
      "tensor(0.9915, device='cuda:0', grad_fn=<MseLossBackward>) 769\n",
      "tensor(0.9915, device='cuda:0', grad_fn=<MseLossBackward>) 770\n",
      "tensor(0.9910, device='cuda:0', grad_fn=<MseLossBackward>) 771\n",
      "tensor(0.9912, device='cuda:0', grad_fn=<MseLossBackward>) 772\n",
      "tensor(0.9906, device='cuda:0', grad_fn=<MseLossBackward>) 773\n",
      "tensor(0.9907, device='cuda:0', grad_fn=<MseLossBackward>) 774\n",
      "tensor(0.9903, device='cuda:0', grad_fn=<MseLossBackward>) 775\n",
      "tensor(0.9903, device='cuda:0', grad_fn=<MseLossBackward>) 776\n",
      "tensor(0.9898, device='cuda:0', grad_fn=<MseLossBackward>) 777\n",
      "tensor(0.9900, device='cuda:0', grad_fn=<MseLossBackward>) 778\n",
      "tensor(0.9894, device='cuda:0', grad_fn=<MseLossBackward>) 779\n",
      "tensor(0.9895, device='cuda:0', grad_fn=<MseLossBackward>) 780\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.9890, device='cuda:0', grad_fn=<MseLossBackward>) 781\n",
      "tensor(0.9891, device='cuda:0', grad_fn=<MseLossBackward>) 782\n",
      "tensor(0.9885, device='cuda:0', grad_fn=<MseLossBackward>) 783\n",
      "tensor(0.9887, device='cuda:0', grad_fn=<MseLossBackward>) 784\n",
      "tensor(0.9882, device='cuda:0', grad_fn=<MseLossBackward>) 785\n",
      "tensor(0.9882, device='cuda:0', grad_fn=<MseLossBackward>) 786\n",
      "tensor(0.9877, device='cuda:0', grad_fn=<MseLossBackward>) 787\n",
      "tensor(0.9879, device='cuda:0', grad_fn=<MseLossBackward>) 788\n",
      "tensor(0.9871, device='cuda:0', grad_fn=<MseLossBackward>) 789\n",
      "tensor(0.9873, device='cuda:0', grad_fn=<MseLossBackward>) 790\n",
      "tensor(0.9869, device='cuda:0', grad_fn=<MseLossBackward>) 791\n",
      "tensor(0.9868, device='cuda:0', grad_fn=<MseLossBackward>) 792\n",
      "tensor(0.9863, device='cuda:0', grad_fn=<MseLossBackward>) 793\n",
      "tensor(0.9864, device='cuda:0', grad_fn=<MseLossBackward>) 794\n",
      "tensor(0.9858, device='cuda:0', grad_fn=<MseLossBackward>) 795\n",
      "tensor(0.9854, device='cuda:0', grad_fn=<MseLossBackward>) 796\n",
      "tensor(0.9854, device='cuda:0', grad_fn=<MseLossBackward>) 797\n",
      "tensor(0.9850, device='cuda:0', grad_fn=<MseLossBackward>) 798\n",
      "tensor(0.9843, device='cuda:0', grad_fn=<MseLossBackward>) 799\n",
      "tensor(0.9845, device='cuda:0', grad_fn=<MseLossBackward>) 800\n",
      "tensor(0.9850, device='cuda:0', grad_fn=<MseLossBackward>) 801\n",
      "tensor(0.9843, device='cuda:0', grad_fn=<MseLossBackward>) 802\n",
      "tensor(0.9840, device='cuda:0', grad_fn=<MseLossBackward>) 803\n",
      "tensor(0.9842, device='cuda:0', grad_fn=<MseLossBackward>) 804\n",
      "tensor(0.9839, device='cuda:0', grad_fn=<MseLossBackward>) 805\n",
      "tensor(0.9848, device='cuda:0', grad_fn=<MseLossBackward>) 806\n",
      "tensor(0.9848, device='cuda:0', grad_fn=<MseLossBackward>) 807\n",
      "tensor(0.9855, device='cuda:0', grad_fn=<MseLossBackward>) 808\n",
      "tensor(0.9853, device='cuda:0', grad_fn=<MseLossBackward>) 809\n",
      "tensor(0.9853, device='cuda:0', grad_fn=<MseLossBackward>) 810\n",
      "tensor(0.9849, device='cuda:0', grad_fn=<MseLossBackward>) 811\n",
      "tensor(0.9847, device='cuda:0', grad_fn=<MseLossBackward>) 812\n",
      "tensor(0.9846, device='cuda:0', grad_fn=<MseLossBackward>) 813\n",
      "tensor(0.9840, device='cuda:0', grad_fn=<MseLossBackward>) 814\n",
      "tensor(0.9839, device='cuda:0', grad_fn=<MseLossBackward>) 815\n",
      "tensor(0.9830, device='cuda:0', grad_fn=<MseLossBackward>) 816\n",
      "tensor(0.9831, device='cuda:0', grad_fn=<MseLossBackward>) 817\n",
      "tensor(0.9825, device='cuda:0', grad_fn=<MseLossBackward>) 818\n",
      "tensor(0.9825, device='cuda:0', grad_fn=<MseLossBackward>) 819\n",
      "tensor(0.9820, device='cuda:0', grad_fn=<MseLossBackward>) 820\n",
      "tensor(0.9820, device='cuda:0', grad_fn=<MseLossBackward>) 821\n",
      "tensor(0.9817, device='cuda:0', grad_fn=<MseLossBackward>) 822\n",
      "tensor(0.9817, device='cuda:0', grad_fn=<MseLossBackward>) 823\n",
      "tensor(0.9814, device='cuda:0', grad_fn=<MseLossBackward>) 824\n",
      "tensor(0.9815, device='cuda:0', grad_fn=<MseLossBackward>) 825\n",
      "tensor(0.9812, device='cuda:0', grad_fn=<MseLossBackward>) 826\n",
      "tensor(0.9812, device='cuda:0', grad_fn=<MseLossBackward>) 827\n",
      "tensor(0.9809, device='cuda:0', grad_fn=<MseLossBackward>) 828\n",
      "tensor(0.9809, device='cuda:0', grad_fn=<MseLossBackward>) 829\n",
      "tensor(0.9806, device='cuda:0', grad_fn=<MseLossBackward>) 830\n",
      "tensor(0.9806, device='cuda:0', grad_fn=<MseLossBackward>) 831\n",
      "tensor(0.9803, device='cuda:0', grad_fn=<MseLossBackward>) 832\n",
      "tensor(0.9803, device='cuda:0', grad_fn=<MseLossBackward>) 833\n",
      "tensor(0.9801, device='cuda:0', grad_fn=<MseLossBackward>) 834\n",
      "tensor(0.9801, device='cuda:0', grad_fn=<MseLossBackward>) 835\n",
      "tensor(0.9799, device='cuda:0', grad_fn=<MseLossBackward>) 836\n",
      "tensor(0.9800, device='cuda:0', grad_fn=<MseLossBackward>) 837\n",
      "tensor(0.9798, device='cuda:0', grad_fn=<MseLossBackward>) 838\n",
      "tensor(0.9799, device='cuda:0', grad_fn=<MseLossBackward>) 839\n",
      "tensor(0.9797, device='cuda:0', grad_fn=<MseLossBackward>) 840\n",
      "tensor(0.9798, device='cuda:0', grad_fn=<MseLossBackward>) 841\n",
      "tensor(0.9797, device='cuda:0', grad_fn=<MseLossBackward>) 842\n",
      "tensor(0.9798, device='cuda:0', grad_fn=<MseLossBackward>) 843\n",
      "tensor(0.9796, device='cuda:0', grad_fn=<MseLossBackward>) 844\n",
      "tensor(0.9797, device='cuda:0', grad_fn=<MseLossBackward>) 845\n",
      "tensor(0.9796, device='cuda:0', grad_fn=<MseLossBackward>) 846\n",
      "tensor(0.9797, device='cuda:0', grad_fn=<MseLossBackward>) 847\n",
      "tensor(0.9796, device='cuda:0', grad_fn=<MseLossBackward>) 848\n",
      "tensor(0.9796, device='cuda:0', grad_fn=<MseLossBackward>) 849\n",
      "tensor(0.9795, device='cuda:0', grad_fn=<MseLossBackward>) 850\n",
      "tensor(0.9796, device='cuda:0', grad_fn=<MseLossBackward>) 851\n",
      "tensor(0.9795, device='cuda:0', grad_fn=<MseLossBackward>) 852\n",
      "tensor(0.9795, device='cuda:0', grad_fn=<MseLossBackward>) 853\n",
      "tensor(0.9795, device='cuda:0', grad_fn=<MseLossBackward>) 854\n",
      "tensor(0.9795, device='cuda:0', grad_fn=<MseLossBackward>) 855\n",
      "tensor(0.9794, device='cuda:0', grad_fn=<MseLossBackward>) 856\n",
      "tensor(0.9794, device='cuda:0', grad_fn=<MseLossBackward>) 857\n",
      "tensor(0.9794, device='cuda:0', grad_fn=<MseLossBackward>) 858\n",
      "tensor(0.9794, device='cuda:0', grad_fn=<MseLossBackward>) 859\n",
      "tensor(0.9794, device='cuda:0', grad_fn=<MseLossBackward>) 860\n",
      "tensor(0.9793, device='cuda:0', grad_fn=<MseLossBackward>) 861\n",
      "tensor(0.9793, device='cuda:0', grad_fn=<MseLossBackward>) 862\n",
      "tensor(0.9793, device='cuda:0', grad_fn=<MseLossBackward>) 863\n",
      "tensor(0.9793, device='cuda:0', grad_fn=<MseLossBackward>) 864\n",
      "tensor(0.9792, device='cuda:0', grad_fn=<MseLossBackward>) 865\n",
      "tensor(0.9792, device='cuda:0', grad_fn=<MseLossBackward>) 866\n",
      "tensor(0.9792, device='cuda:0', grad_fn=<MseLossBackward>) 867\n",
      "tensor(0.9792, device='cuda:0', grad_fn=<MseLossBackward>) 868\n",
      "tensor(0.9791, device='cuda:0', grad_fn=<MseLossBackward>) 869\n",
      "tensor(0.9791, device='cuda:0', grad_fn=<MseLossBackward>) 870\n",
      "tensor(0.9791, device='cuda:0', grad_fn=<MseLossBackward>) 871\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 872\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 873\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 874\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 875\n",
      "tensor(0.9789, device='cuda:0', grad_fn=<MseLossBackward>) 876\n",
      "tensor(0.9789, device='cuda:0', grad_fn=<MseLossBackward>) 877\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 878\n",
      "tensor(0.9792, device='cuda:0', grad_fn=<MseLossBackward>) 879\n",
      "tensor(0.9792, device='cuda:0', grad_fn=<MseLossBackward>) 880\n",
      "tensor(0.9792, device='cuda:0', grad_fn=<MseLossBackward>) 881\n",
      "tensor(0.9792, device='cuda:0', grad_fn=<MseLossBackward>) 882\n",
      "tensor(0.9791, device='cuda:0', grad_fn=<MseLossBackward>) 883\n",
      "tensor(0.9791, device='cuda:0', grad_fn=<MseLossBackward>) 884\n",
      "tensor(0.9791, device='cuda:0', grad_fn=<MseLossBackward>) 885\n",
      "tensor(0.9791, device='cuda:0', grad_fn=<MseLossBackward>) 886\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 887\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 888\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 889\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 890\n",
      "tensor(0.9789, device='cuda:0', grad_fn=<MseLossBackward>) 891\n",
      "tensor(0.9789, device='cuda:0', grad_fn=<MseLossBackward>) 892\n",
      "tensor(0.9789, device='cuda:0', grad_fn=<MseLossBackward>) 893\n",
      "tensor(0.9788, device='cuda:0', grad_fn=<MseLossBackward>) 894\n",
      "tensor(0.9788, device='cuda:0', grad_fn=<MseLossBackward>) 895\n",
      "tensor(0.9788, device='cuda:0', grad_fn=<MseLossBackward>) 896\n",
      "tensor(0.9787, device='cuda:0', grad_fn=<MseLossBackward>) 897\n",
      "tensor(0.9787, device='cuda:0', grad_fn=<MseLossBackward>) 898\n",
      "tensor(0.9787, device='cuda:0', grad_fn=<MseLossBackward>) 899\n",
      "tensor(0.9786, device='cuda:0', grad_fn=<MseLossBackward>) 900\n",
      "tensor(0.9786, device='cuda:0', grad_fn=<MseLossBackward>) 901\n",
      "tensor(0.9786, device='cuda:0', grad_fn=<MseLossBackward>) 902\n",
      "tensor(0.9785, device='cuda:0', grad_fn=<MseLossBackward>) 903\n",
      "tensor(0.9785, device='cuda:0', grad_fn=<MseLossBackward>) 904\n",
      "tensor(0.9785, device='cuda:0', grad_fn=<MseLossBackward>) 905\n",
      "tensor(0.9785, device='cuda:0', grad_fn=<MseLossBackward>) 906\n",
      "tensor(0.9784, device='cuda:0', grad_fn=<MseLossBackward>) 907\n",
      "tensor(0.9784, device='cuda:0', grad_fn=<MseLossBackward>) 908\n",
      "tensor(0.9784, device='cuda:0', grad_fn=<MseLossBackward>) 909\n",
      "tensor(0.9783, device='cuda:0', grad_fn=<MseLossBackward>) 910\n",
      "tensor(0.9783, device='cuda:0', grad_fn=<MseLossBackward>) 911\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.9782, device='cuda:0', grad_fn=<MseLossBackward>) 912\n",
      "tensor(0.9781, device='cuda:0', grad_fn=<MseLossBackward>) 913\n",
      "tensor(0.9780, device='cuda:0', grad_fn=<MseLossBackward>) 914\n",
      "tensor(0.9780, device='cuda:0', grad_fn=<MseLossBackward>) 915\n",
      "tensor(0.9779, device='cuda:0', grad_fn=<MseLossBackward>) 916\n",
      "tensor(0.9779, device='cuda:0', grad_fn=<MseLossBackward>) 917\n",
      "tensor(0.9778, device='cuda:0', grad_fn=<MseLossBackward>) 918\n",
      "tensor(0.9778, device='cuda:0', grad_fn=<MseLossBackward>) 919\n",
      "tensor(0.9777, device='cuda:0', grad_fn=<MseLossBackward>) 920\n",
      "tensor(0.9776, device='cuda:0', grad_fn=<MseLossBackward>) 921\n",
      "tensor(0.9776, device='cuda:0', grad_fn=<MseLossBackward>) 922\n",
      "tensor(0.9775, device='cuda:0', grad_fn=<MseLossBackward>) 923\n",
      "tensor(0.9775, device='cuda:0', grad_fn=<MseLossBackward>) 924\n",
      "tensor(0.9774, device='cuda:0', grad_fn=<MseLossBackward>) 925\n",
      "tensor(0.9774, device='cuda:0', grad_fn=<MseLossBackward>) 926\n",
      "tensor(0.9773, device='cuda:0', grad_fn=<MseLossBackward>) 927\n",
      "tensor(0.9773, device='cuda:0', grad_fn=<MseLossBackward>) 928\n",
      "tensor(0.9772, device='cuda:0', grad_fn=<MseLossBackward>) 929\n",
      "tensor(0.9771, device='cuda:0', grad_fn=<MseLossBackward>) 930\n",
      "tensor(0.9771, device='cuda:0', grad_fn=<MseLossBackward>) 931\n",
      "tensor(0.9770, device='cuda:0', grad_fn=<MseLossBackward>) 932\n",
      "tensor(0.9770, device='cuda:0', grad_fn=<MseLossBackward>) 933\n",
      "tensor(0.9769, device='cuda:0', grad_fn=<MseLossBackward>) 934\n",
      "tensor(0.9769, device='cuda:0', grad_fn=<MseLossBackward>) 935\n",
      "tensor(0.9768, device='cuda:0', grad_fn=<MseLossBackward>) 936\n",
      "tensor(0.9768, device='cuda:0', grad_fn=<MseLossBackward>) 937\n",
      "tensor(0.9767, device='cuda:0', grad_fn=<MseLossBackward>) 938\n",
      "tensor(0.9766, device='cuda:0', grad_fn=<MseLossBackward>) 939\n",
      "tensor(0.9766, device='cuda:0', grad_fn=<MseLossBackward>) 940\n",
      "tensor(0.9765, device='cuda:0', grad_fn=<MseLossBackward>) 941\n",
      "tensor(0.9765, device='cuda:0', grad_fn=<MseLossBackward>) 942\n",
      "tensor(0.9764, device='cuda:0', grad_fn=<MseLossBackward>) 943\n",
      "tensor(0.9763, device='cuda:0', grad_fn=<MseLossBackward>) 944\n",
      "tensor(0.9763, device='cuda:0', grad_fn=<MseLossBackward>) 945\n",
      "tensor(0.9762, device='cuda:0', grad_fn=<MseLossBackward>) 946\n",
      "tensor(0.9762, device='cuda:0', grad_fn=<MseLossBackward>) 947\n",
      "tensor(0.9761, device='cuda:0', grad_fn=<MseLossBackward>) 948\n",
      "tensor(0.9761, device='cuda:0', grad_fn=<MseLossBackward>) 949\n",
      "tensor(0.9760, device='cuda:0', grad_fn=<MseLossBackward>) 950\n",
      "tensor(0.9759, device='cuda:0', grad_fn=<MseLossBackward>) 951\n",
      "tensor(0.9759, device='cuda:0', grad_fn=<MseLossBackward>) 952\n",
      "tensor(0.9758, device='cuda:0', grad_fn=<MseLossBackward>) 953\n",
      "tensor(0.9758, device='cuda:0', grad_fn=<MseLossBackward>) 954\n",
      "tensor(0.9757, device='cuda:0', grad_fn=<MseLossBackward>) 955\n",
      "tensor(0.9757, device='cuda:0', grad_fn=<MseLossBackward>) 956\n",
      "tensor(0.9756, device='cuda:0', grad_fn=<MseLossBackward>) 957\n",
      "tensor(0.9755, device='cuda:0', grad_fn=<MseLossBackward>) 958\n",
      "tensor(0.9755, device='cuda:0', grad_fn=<MseLossBackward>) 959\n",
      "tensor(0.9754, device='cuda:0', grad_fn=<MseLossBackward>) 960\n",
      "tensor(0.9754, device='cuda:0', grad_fn=<MseLossBackward>) 961\n",
      "tensor(0.9753, device='cuda:0', grad_fn=<MseLossBackward>) 962\n",
      "tensor(0.9753, device='cuda:0', grad_fn=<MseLossBackward>) 963\n",
      "tensor(0.9752, device='cuda:0', grad_fn=<MseLossBackward>) 964\n",
      "tensor(0.9751, device='cuda:0', grad_fn=<MseLossBackward>) 965\n",
      "tensor(0.9751, device='cuda:0', grad_fn=<MseLossBackward>) 966\n",
      "tensor(0.9750, device='cuda:0', grad_fn=<MseLossBackward>) 967\n",
      "tensor(0.9750, device='cuda:0', grad_fn=<MseLossBackward>) 968\n",
      "tensor(0.9749, device='cuda:0', grad_fn=<MseLossBackward>) 969\n",
      "tensor(0.9749, device='cuda:0', grad_fn=<MseLossBackward>) 970\n",
      "tensor(0.9748, device='cuda:0', grad_fn=<MseLossBackward>) 971\n",
      "tensor(0.9747, device='cuda:0', grad_fn=<MseLossBackward>) 972\n",
      "tensor(0.9747, device='cuda:0', grad_fn=<MseLossBackward>) 973\n",
      "tensor(0.9746, device='cuda:0', grad_fn=<MseLossBackward>) 974\n",
      "tensor(0.9746, device='cuda:0', grad_fn=<MseLossBackward>) 975\n",
      "tensor(0.9745, device='cuda:0', grad_fn=<MseLossBackward>) 976\n",
      "tensor(0.9745, device='cuda:0', grad_fn=<MseLossBackward>) 977\n",
      "tensor(0.9744, device='cuda:0', grad_fn=<MseLossBackward>) 978\n",
      "tensor(0.9743, device='cuda:0', grad_fn=<MseLossBackward>) 979\n",
      "tensor(0.9743, device='cuda:0', grad_fn=<MseLossBackward>) 980\n",
      "tensor(0.9742, device='cuda:0', grad_fn=<MseLossBackward>) 981\n",
      "tensor(0.9742, device='cuda:0', grad_fn=<MseLossBackward>) 982\n",
      "tensor(0.9741, device='cuda:0', grad_fn=<MseLossBackward>) 983\n",
      "tensor(0.9741, device='cuda:0', grad_fn=<MseLossBackward>) 984\n",
      "tensor(0.9740, device='cuda:0', grad_fn=<MseLossBackward>) 985\n",
      "tensor(0.9740, device='cuda:0', grad_fn=<MseLossBackward>) 986\n",
      "tensor(0.9739, device='cuda:0', grad_fn=<MseLossBackward>) 987\n",
      "tensor(0.9739, device='cuda:0', grad_fn=<MseLossBackward>) 988\n",
      "tensor(0.9740, device='cuda:0', grad_fn=<MseLossBackward>) 989\n",
      "tensor(0.9740, device='cuda:0', grad_fn=<MseLossBackward>) 990\n",
      "tensor(0.9739, device='cuda:0', grad_fn=<MseLossBackward>) 991\n",
      "tensor(0.9738, device='cuda:0', grad_fn=<MseLossBackward>) 992\n",
      "tensor(0.9738, device='cuda:0', grad_fn=<MseLossBackward>) 993\n",
      "tensor(0.9737, device='cuda:0', grad_fn=<MseLossBackward>) 994\n",
      "tensor(0.9737, device='cuda:0', grad_fn=<MseLossBackward>) 995\n",
      "tensor(0.9735, device='cuda:0', grad_fn=<MseLossBackward>) 996\n",
      "tensor(0.9735, device='cuda:0', grad_fn=<MseLossBackward>) 997\n",
      "tensor(0.9734, device='cuda:0', grad_fn=<MseLossBackward>) 998\n",
      "tensor(0.9734, device='cuda:0', grad_fn=<MseLossBackward>) 999\n"
     ]
    }
   ],
   "source": [
    "#trainingloop\n",
    "\n",
    "import torch.optim as optim\n",
    "\n",
    "net=Net()\n",
    "\n",
    "lr = 0.01\n",
    "batchsize=20\n",
    "\n",
    "batches=len(trdttensor)/batchsize\n",
    "\n",
    "epochs=1000\n",
    "\n",
    "# create your optimizer\n",
    "optimizer = optim.Adagrad(net.parameters(), lr=0.01)\n",
    "\n",
    "\n",
    "for i in range(epochs):\n",
    "    \n",
    "    for j in range(int(batches)):\n",
    "        \n",
    "        #forward pass\n",
    "        out=net(trdttensor[j:j+batchsize,:].type(dtype))\n",
    "\n",
    "        #compute loss\n",
    "        criterion = nn.MSELoss()\n",
    "        loss = criterion(out,results1tensor[j:j+batchsize]).type(dtype)\n",
    "\n",
    "\n",
    "        #backprop loss i.e. find dloss/dparam for each parameter and store.\n",
    "        loss.backward(retain_graph=True)\n",
    "        \n",
    "        #clip gradients\n",
    "        torch.nn.utils.clip_grad_norm_(net.parameters(), 10.0)\n",
    "        \n",
    "        #use optimiser to update\n",
    "        optimizer.step()\n",
    "    c=nn.MSELoss()\n",
    "    print(c(torch.reshape(net(trdttensor),[200]),results1tensor),i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1.9931],\n",
      "        [2.0318],\n",
      "        [2.0399],\n",
      "        [2.0683],\n",
      "        [2.0839],\n",
      "        [2.0028],\n",
      "        [1.9945],\n",
      "        [2.0597],\n",
      "        [2.0555],\n",
      "        [2.0335],\n",
      "        [2.0362],\n",
      "        [2.0844],\n",
      "        [2.0534],\n",
      "        [2.0720],\n",
      "        [2.0386],\n",
      "        [2.0542],\n",
      "        [2.0577],\n",
      "        [2.0573],\n",
      "        [2.0499],\n",
      "        [2.0547],\n",
      "        [2.0124],\n",
      "        [2.0186],\n",
      "        [2.0459],\n",
      "        [2.0364],\n",
      "        [2.0416],\n",
      "        [2.0695],\n",
      "        [2.0415],\n",
      "        [2.0389],\n",
      "        [2.0271],\n",
      "        [1.1156],\n",
      "        [0.6907],\n",
      "        [1.3573],\n",
      "        [0.4286],\n",
      "        [1.6417],\n",
      "        [1.1683],\n",
      "        [0.5699],\n",
      "        [1.0049],\n",
      "        [0.6146],\n",
      "        [1.4106],\n",
      "        [0.3019],\n",
      "        [0.3761],\n",
      "        [2.1805],\n",
      "        [1.2070],\n",
      "        [1.0031],\n",
      "        [1.8537],\n",
      "        [0.8805],\n",
      "        [0.7997],\n",
      "        [0.2791],\n",
      "        [2.0448],\n",
      "        [2.0394],\n",
      "        [1.2948],\n",
      "        [0.7466],\n",
      "        [1.4898],\n",
      "        [1.6825],\n",
      "        [1.7262],\n",
      "        [2.8700],\n",
      "        [1.2822],\n",
      "        [2.1855],\n",
      "        [0.9176],\n",
      "        [2.1073],\n",
      "        [0.7826],\n",
      "        [1.9777],\n",
      "        [1.9857],\n",
      "        [1.0038],\n",
      "        [2.7551],\n",
      "        [1.2434],\n",
      "        [1.4851],\n",
      "        [0.6325],\n",
      "        [1.5366],\n",
      "        [2.6720],\n",
      "        [1.5854],\n",
      "        [0.7227],\n",
      "        [1.2520],\n",
      "        [1.2682],\n",
      "        [1.4102],\n",
      "        [3.1774],\n",
      "        [2.0086],\n",
      "        [1.4274],\n",
      "        [0.7139],\n",
      "        [0.7325],\n",
      "        [0.8882],\n",
      "        [0.2461],\n",
      "        [0.1943],\n",
      "        [0.7240],\n",
      "        [1.4340],\n",
      "        [0.7763],\n",
      "        [0.6980],\n",
      "        [1.5643],\n",
      "        [4.1004],\n",
      "        [1.3408],\n",
      "        [2.0195],\n",
      "        [0.3634],\n",
      "        [2.1293],\n",
      "        [1.1641],\n",
      "        [0.6932],\n",
      "        [0.9269],\n",
      "        [1.1350],\n",
      "        [2.7399],\n",
      "        [1.7879],\n",
      "        [1.4318],\n",
      "        [1.1873],\n",
      "        [1.7705],\n",
      "        [0.6764],\n",
      "        [1.8197],\n",
      "        [1.6694],\n",
      "        [1.7966],\n",
      "        [0.6556],\n",
      "        [1.9205],\n",
      "        [2.0202],\n",
      "        [1.3243],\n",
      "        [2.6473],\n",
      "        [1.3193],\n",
      "        [1.0890],\n",
      "        [1.7772],\n",
      "        [0.8428],\n",
      "        [0.8276],\n",
      "        [1.2878],\n",
      "        [0.9993],\n",
      "        [2.8449],\n",
      "        [0.4835],\n",
      "        [0.3983],\n",
      "        [1.3532],\n",
      "        [0.6834],\n",
      "        [1.1774],\n",
      "        [0.6191],\n",
      "        [1.8997],\n",
      "        [1.2783],\n",
      "        [1.4791],\n",
      "        [1.7846],\n",
      "        [1.5766],\n",
      "        [1.6945],\n",
      "        [1.8136],\n",
      "        [2.7338],\n",
      "        [1.4091],\n",
      "        [1.4498],\n",
      "        [2.6945],\n",
      "        [1.5709],\n",
      "        [1.7330],\n",
      "        [0.5533],\n",
      "        [1.0367],\n",
      "        [1.7297],\n",
      "        [0.3156],\n",
      "        [0.4289],\n",
      "        [1.2260],\n",
      "        [1.8508],\n",
      "        [0.2602],\n",
      "        [0.7686],\n",
      "        [1.3259],\n",
      "        [0.4148],\n",
      "        [1.8076],\n",
      "        [2.5752],\n",
      "        [1.0423],\n",
      "        [1.8007],\n",
      "        [1.7046],\n",
      "        [1.3415],\n",
      "        [2.0280],\n",
      "        [2.7781],\n",
      "        [1.6129],\n",
      "        [0.3256],\n",
      "        [1.3063],\n",
      "        [1.3189],\n",
      "        [2.6063],\n",
      "        [1.2821],\n",
      "        [1.6056],\n",
      "        [2.0467],\n",
      "        [0.8088],\n",
      "        [1.3462],\n",
      "        [1.2752],\n",
      "        [0.9696],\n",
      "        [2.3312],\n",
      "        [1.0938],\n",
      "        [1.3822],\n",
      "        [0.8882],\n",
      "        [1.5719],\n",
      "        [0.7408],\n",
      "        [0.8764],\n",
      "        [1.0107],\n",
      "        [0.4433],\n",
      "        [0.6171],\n",
      "        [0.3015],\n",
      "        [1.2105],\n",
      "        [1.8401],\n",
      "        [0.3219],\n",
      "        [1.1541],\n",
      "        [1.9901],\n",
      "        [0.7236],\n",
      "        [0.9937],\n",
      "        [0.8574],\n",
      "        [1.1518],\n",
      "        [1.1780],\n",
      "        [1.6090],\n",
      "        [0.6753],\n",
      "        [0.5989],\n",
      "        [3.9682],\n",
      "        [2.0380],\n",
      "        [1.5651],\n",
      "        [0.8328],\n",
      "        [1.9191],\n",
      "        [1.5739],\n",
      "        [1.2772]], device='cuda:0', grad_fn=<AddmmBackward>)\n"
     ]
    }
   ],
   "source": [
    "print(net(torch.from_numpy(trdt.values).type(dtype)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0894098549846931\n"
     ]
    }
   ],
   "source": [
    "#Evaluation\n",
    "\n",
    "import joblib\n",
    "#golden standard\n",
    "m=joblib.load('C:\\\\Users\\\\Ant Pc\\\\GitHub\\\\pROJ\\\\Geneexpression0\\\\12.pkl')\n",
    "\n",
    "pred=pd.DataFrame(m.predict(tstdt))\n",
    "\n",
    "from sklearn.metrics import mean_squared_error\n",
    "print(mean_squared_error(pred,net(torch.from_numpy(tstdt.values).type(dtype)).cpu().detach().numpy()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, we have seen that we get a mean squared error of 1.09, which is not very good, especially given that values are in the range of 1 to 2. That is almost 50% error, very high."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fixed Layer Random Projection Neural Network"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**So what is Fixed Layer Random Projection neural network?**\n",
    "\n",
    "![4](assets/4.jpeg)\n",
    "\n",
    "Here is an image to explain.\n",
    "\n",
    "It is basically almost like preprocessing the given data and here instead of just reducing the range of values in data, we map the high dimensional data onto a lower dimensional space with the help of a Random Projection Matrix. In this particular variation of the Random Projection Neural Network, we treat the values inside Random Projection Matrix as untrainable i.e. fixed. After mapping the given data onto a smaller dimensional space, we normalise the data and then feed our smaller dimensional data into a conventional regression neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# first fix the layer dimension after input layer\n",
    "import random\n",
    "import math\n",
    "\n",
    "input_dim=28395\n",
    "random_projection_layer_dim= 1000\n",
    "\n",
    "def generatelisrandommatrix(input_dim,random_projection_layer_dim):\n",
    "    \n",
    "    size_0=input_dim\n",
    "    size_1=random_projection_layer_dim\n",
    "    \n",
    "    matr=np.zeros([size_0,size_1])\n",
    "    \n",
    "    for i in range(size_0):\n",
    "        m=[]\n",
    "        for j in range(size_1):\n",
    "            x=random.uniform(0,1)\n",
    "            if x<(1/6):\n",
    "                matr[i,j]=(1*math.sqrt((3/size_1)))\n",
    "            \n",
    "            if x>(1/6):\n",
    "                if x<(1-1/6):\n",
    "                    matr[i,j]=(0)\n",
    "                \n",
    "            if x>((1)-(1/6)):\n",
    "                matr[i,j]=(-1*math.sqrt((3/size_1)))\n",
    "    \n",
    "    return matr\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Map the input into the smaller dimension of random_projection_layer using the generated Achilopta's  random projection matrix\n",
    "\n",
    "randmatr=generatelisrandommatrix(28395,1000)\n",
    "projectedtrdt=np.dot(trdt.values,randmatr)\n",
    "\n",
    "#Normalise the projected data, not batch normalisation because, we are not fine tuning random projection layer weights\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler=StandardScaler()\n",
    "normalisedprojection=scaler.fit_transform(projectedtrdt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Net(\n",
      "  (fc1): Linear(in_features=1000, out_features=320, bias=True)\n",
      "  (fc6): Linear(in_features=320, out_features=1, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "#convert this numpy array to tensor\n",
    "projectiontensor=torch.from_numpy(normalisedprojection).type(dtype)\n",
    "\n",
    "class Net(nn.Module):\n",
    "\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        \n",
    "        # an affine operation: y = Wx + b\n",
    "        \n",
    "        self.fc1 = nn.Linear(1000,320).cuda()\n",
    "        self.fc6 = nn.Linear(320,1).cuda()\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.fc1(x)).cuda()\n",
    "        x = self.fc6(x).cuda()\n",
    "        return x\n",
    "\n",
    "net = Net()\n",
    "\n",
    "#use gpu for all computations in model\n",
    "net.cuda()\n",
    "print(net)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\torch\\nn\\modules\\loss.py:432: UserWarning: Using a target size (torch.Size([20])) that is different to the input size (torch.Size([20, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.\n",
      "  return F.mse_loss(input, target, reduction=self.reduction)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(2.2253, device='cuda:0', grad_fn=<MseLossBackward>) 0\n",
      "tensor(3.2549, device='cuda:0', grad_fn=<MseLossBackward>) 1\n",
      "tensor(2.8043, device='cuda:0', grad_fn=<MseLossBackward>) 2\n",
      "tensor(2.1461, device='cuda:0', grad_fn=<MseLossBackward>) 3\n",
      "tensor(2.6343, device='cuda:0', grad_fn=<MseLossBackward>) 4\n",
      "tensor(1.6547, device='cuda:0', grad_fn=<MseLossBackward>) 5\n",
      "tensor(2.3205, device='cuda:0', grad_fn=<MseLossBackward>) 6\n",
      "tensor(1.9036, device='cuda:0', grad_fn=<MseLossBackward>) 7\n",
      "tensor(1.6818, device='cuda:0', grad_fn=<MseLossBackward>) 8\n",
      "tensor(2.0062, device='cuda:0', grad_fn=<MseLossBackward>) 9\n",
      "tensor(1.3060, device='cuda:0', grad_fn=<MseLossBackward>) 10\n",
      "tensor(1.9296, device='cuda:0', grad_fn=<MseLossBackward>) 11\n",
      "tensor(1.1499, device='cuda:0', grad_fn=<MseLossBackward>) 12\n",
      "tensor(1.7468, device='cuda:0', grad_fn=<MseLossBackward>) 13\n",
      "tensor(1.0742, device='cuda:0', grad_fn=<MseLossBackward>) 14\n",
      "tensor(1.5005, device='cuda:0', grad_fn=<MseLossBackward>) 15\n",
      "tensor(1.0512, device='cuda:0', grad_fn=<MseLossBackward>) 16\n",
      "tensor(1.4987, device='cuda:0', grad_fn=<MseLossBackward>) 17\n",
      "tensor(0.9510, device='cuda:0', grad_fn=<MseLossBackward>) 18\n",
      "tensor(1.4464, device='cuda:0', grad_fn=<MseLossBackward>) 19\n",
      "tensor(1.0404, device='cuda:0', grad_fn=<MseLossBackward>) 20\n",
      "tensor(1.2557, device='cuda:0', grad_fn=<MseLossBackward>) 21\n",
      "tensor(1.2566, device='cuda:0', grad_fn=<MseLossBackward>) 22\n",
      "tensor(1.3425, device='cuda:0', grad_fn=<MseLossBackward>) 23\n",
      "tensor(1.2707, device='cuda:0', grad_fn=<MseLossBackward>) 24\n",
      "tensor(1.1027, device='cuda:0', grad_fn=<MseLossBackward>) 25\n",
      "tensor(1.4858, device='cuda:0', grad_fn=<MseLossBackward>) 26\n",
      "tensor(0.9671, device='cuda:0', grad_fn=<MseLossBackward>) 27\n",
      "tensor(1.4291, device='cuda:0', grad_fn=<MseLossBackward>) 28\n",
      "tensor(1.0163, device='cuda:0', grad_fn=<MseLossBackward>) 29\n",
      "tensor(1.3402, device='cuda:0', grad_fn=<MseLossBackward>) 30\n",
      "tensor(1.0581, device='cuda:0', grad_fn=<MseLossBackward>) 31\n",
      "tensor(1.1377, device='cuda:0', grad_fn=<MseLossBackward>) 32\n",
      "tensor(1.4011, device='cuda:0', grad_fn=<MseLossBackward>) 33\n",
      "tensor(0.9532, device='cuda:0', grad_fn=<MseLossBackward>) 34\n",
      "tensor(1.3242, device='cuda:0', grad_fn=<MseLossBackward>) 35\n",
      "tensor(0.9913, device='cuda:0', grad_fn=<MseLossBackward>) 36\n",
      "tensor(1.2956, device='cuda:0', grad_fn=<MseLossBackward>) 37\n",
      "tensor(1.0390, device='cuda:0', grad_fn=<MseLossBackward>) 38\n",
      "tensor(1.0670, device='cuda:0', grad_fn=<MseLossBackward>) 39\n",
      "tensor(1.3743, device='cuda:0', grad_fn=<MseLossBackward>) 40\n",
      "tensor(0.9520, device='cuda:0', grad_fn=<MseLossBackward>) 41\n",
      "tensor(1.3074, device='cuda:0', grad_fn=<MseLossBackward>) 42\n",
      "tensor(0.9529, device='cuda:0', grad_fn=<MseLossBackward>) 43\n",
      "tensor(1.2322, device='cuda:0', grad_fn=<MseLossBackward>) 44\n",
      "tensor(1.0965, device='cuda:0', grad_fn=<MseLossBackward>) 45\n",
      "tensor(0.9893, device='cuda:0', grad_fn=<MseLossBackward>) 46\n",
      "tensor(1.3123, device='cuda:0', grad_fn=<MseLossBackward>) 47\n",
      "tensor(0.9372, device='cuda:0', grad_fn=<MseLossBackward>) 48\n",
      "tensor(1.2844, device='cuda:0', grad_fn=<MseLossBackward>) 49\n",
      "tensor(0.9678, device='cuda:0', grad_fn=<MseLossBackward>) 50\n",
      "tensor(1.1376, device='cuda:0', grad_fn=<MseLossBackward>) 51\n",
      "tensor(1.1937, device='cuda:0', grad_fn=<MseLossBackward>) 52\n",
      "tensor(0.9546, device='cuda:0', grad_fn=<MseLossBackward>) 53\n",
      "tensor(1.2717, device='cuda:0', grad_fn=<MseLossBackward>) 54\n",
      "tensor(0.9142, device='cuda:0', grad_fn=<MseLossBackward>) 55\n",
      "tensor(1.2343, device='cuda:0', grad_fn=<MseLossBackward>) 56\n",
      "tensor(1.0265, device='cuda:0', grad_fn=<MseLossBackward>) 57\n",
      "tensor(1.0188, device='cuda:0', grad_fn=<MseLossBackward>) 58\n",
      "tensor(1.2732, device='cuda:0', grad_fn=<MseLossBackward>) 59\n",
      "tensor(0.9268, device='cuda:0', grad_fn=<MseLossBackward>) 60\n",
      "tensor(1.2569, device='cuda:0', grad_fn=<MseLossBackward>) 61\n",
      "tensor(0.9576, device='cuda:0', grad_fn=<MseLossBackward>) 62\n",
      "tensor(1.1191, device='cuda:0', grad_fn=<MseLossBackward>) 63\n",
      "tensor(1.1866, device='cuda:0', grad_fn=<MseLossBackward>) 64\n",
      "tensor(0.9352, device='cuda:0', grad_fn=<MseLossBackward>) 65\n",
      "tensor(1.2546, device='cuda:0', grad_fn=<MseLossBackward>) 66\n",
      "tensor(0.9336, device='cuda:0', grad_fn=<MseLossBackward>) 67\n",
      "tensor(1.1878, device='cuda:0', grad_fn=<MseLossBackward>) 68\n",
      "tensor(1.0793, device='cuda:0', grad_fn=<MseLossBackward>) 69\n",
      "tensor(0.9683, device='cuda:0', grad_fn=<MseLossBackward>) 70\n",
      "tensor(1.2750, device='cuda:0', grad_fn=<MseLossBackward>) 71\n",
      "tensor(0.9260, device='cuda:0', grad_fn=<MseLossBackward>) 72\n",
      "tensor(1.2046, device='cuda:0', grad_fn=<MseLossBackward>) 73\n",
      "tensor(1.0247, device='cuda:0', grad_fn=<MseLossBackward>) 74\n",
      "tensor(1.0297, device='cuda:0', grad_fn=<MseLossBackward>) 75\n",
      "tensor(1.2585, device='cuda:0', grad_fn=<MseLossBackward>) 76\n",
      "tensor(0.9125, device='cuda:0', grad_fn=<MseLossBackward>) 77\n",
      "tensor(1.2035, device='cuda:0', grad_fn=<MseLossBackward>) 78\n",
      "tensor(1.0022, device='cuda:0', grad_fn=<MseLossBackward>) 79\n",
      "tensor(1.0640, device='cuda:0', grad_fn=<MseLossBackward>) 80\n",
      "tensor(1.2137, device='cuda:0', grad_fn=<MseLossBackward>) 81\n",
      "tensor(0.9095, device='cuda:0', grad_fn=<MseLossBackward>) 82\n",
      "tensor(1.2235, device='cuda:0', grad_fn=<MseLossBackward>) 83\n",
      "tensor(0.9912, device='cuda:0', grad_fn=<MseLossBackward>) 84\n",
      "tensor(1.0629, device='cuda:0', grad_fn=<MseLossBackward>) 85\n",
      "tensor(1.2036, device='cuda:0', grad_fn=<MseLossBackward>) 86\n",
      "tensor(0.9246, device='cuda:0', grad_fn=<MseLossBackward>) 87\n",
      "tensor(1.2295, device='cuda:0', grad_fn=<MseLossBackward>) 88\n",
      "tensor(0.9811, device='cuda:0', grad_fn=<MseLossBackward>) 89\n",
      "tensor(1.0735, device='cuda:0', grad_fn=<MseLossBackward>) 90\n",
      "tensor(1.1718, device='cuda:0', grad_fn=<MseLossBackward>) 91\n",
      "tensor(0.9274, device='cuda:0', grad_fn=<MseLossBackward>) 92\n",
      "tensor(1.1781, device='cuda:0', grad_fn=<MseLossBackward>) 93\n",
      "tensor(1.0764, device='cuda:0', grad_fn=<MseLossBackward>) 94\n",
      "tensor(1.0490, device='cuda:0', grad_fn=<MseLossBackward>) 95\n",
      "tensor(1.2078, device='cuda:0', grad_fn=<MseLossBackward>) 96\n",
      "tensor(1.1338, device='cuda:0', grad_fn=<MseLossBackward>) 97\n",
      "tensor(0.9193, device='cuda:0', grad_fn=<MseLossBackward>) 98\n",
      "tensor(1.3531, device='cuda:0', grad_fn=<MseLossBackward>) 99\n",
      "tensor(1.1074, device='cuda:0', grad_fn=<MseLossBackward>) 100\n",
      "tensor(1.0317, device='cuda:0', grad_fn=<MseLossBackward>) 101\n",
      "tensor(1.3683, device='cuda:0', grad_fn=<MseLossBackward>) 102\n",
      "tensor(1.0775, device='cuda:0', grad_fn=<MseLossBackward>) 103\n",
      "tensor(0.9802, device='cuda:0', grad_fn=<MseLossBackward>) 104\n",
      "tensor(0.8512, device='cuda:0', grad_fn=<MseLossBackward>) 105\n",
      "tensor(1.0596, device='cuda:0', grad_fn=<MseLossBackward>) 106\n",
      "tensor(0.8540, device='cuda:0', grad_fn=<MseLossBackward>) 107\n",
      "tensor(1.3554, device='cuda:0', grad_fn=<MseLossBackward>) 108\n",
      "tensor(1.0789, device='cuda:0', grad_fn=<MseLossBackward>) 109\n",
      "tensor(0.8220, device='cuda:0', grad_fn=<MseLossBackward>) 110\n",
      "tensor(1.1790, device='cuda:0', grad_fn=<MseLossBackward>) 111\n",
      "tensor(0.9790, device='cuda:0', grad_fn=<MseLossBackward>) 112\n",
      "tensor(0.9614, device='cuda:0', grad_fn=<MseLossBackward>) 113\n",
      "tensor(0.9243, device='cuda:0', grad_fn=<MseLossBackward>) 114\n",
      "tensor(0.8548, device='cuda:0', grad_fn=<MseLossBackward>) 115\n",
      "tensor(0.8663, device='cuda:0', grad_fn=<MseLossBackward>) 116\n",
      "tensor(1.2304, device='cuda:0', grad_fn=<MseLossBackward>) 117\n",
      "tensor(0.7740, device='cuda:0', grad_fn=<MseLossBackward>) 118\n",
      "tensor(1.0144, device='cuda:0', grad_fn=<MseLossBackward>) 119\n",
      "tensor(0.9453, device='cuda:0', grad_fn=<MseLossBackward>) 120\n",
      "tensor(0.8958, device='cuda:0', grad_fn=<MseLossBackward>) 121\n",
      "tensor(1.0246, device='cuda:0', grad_fn=<MseLossBackward>) 122\n",
      "tensor(0.8287, device='cuda:0', grad_fn=<MseLossBackward>) 123\n",
      "tensor(1.0780, device='cuda:0', grad_fn=<MseLossBackward>) 124\n",
      "tensor(0.8588, device='cuda:0', grad_fn=<MseLossBackward>) 125\n",
      "tensor(0.8811, device='cuda:0', grad_fn=<MseLossBackward>) 126\n",
      "tensor(1.1379, device='cuda:0', grad_fn=<MseLossBackward>) 127\n",
      "tensor(0.8175, device='cuda:0', grad_fn=<MseLossBackward>) 128\n",
      "tensor(0.9982, device='cuda:0', grad_fn=<MseLossBackward>) 129\n",
      "tensor(0.9383, device='cuda:0', grad_fn=<MseLossBackward>) 130\n",
      "tensor(0.8521, device='cuda:0', grad_fn=<MseLossBackward>) 131\n",
      "tensor(1.1032, device='cuda:0', grad_fn=<MseLossBackward>) 132\n",
      "tensor(0.8042, device='cuda:0', grad_fn=<MseLossBackward>) 133\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.9116, device='cuda:0', grad_fn=<MseLossBackward>) 134\n",
      "tensor(1.0755, device='cuda:0', grad_fn=<MseLossBackward>) 135\n",
      "tensor(0.8289, device='cuda:0', grad_fn=<MseLossBackward>) 136\n",
      "tensor(1.0033, device='cuda:0', grad_fn=<MseLossBackward>) 137\n",
      "tensor(0.8638, device='cuda:0', grad_fn=<MseLossBackward>) 138\n",
      "tensor(0.8791, device='cuda:0', grad_fn=<MseLossBackward>) 139\n",
      "tensor(1.1246, device='cuda:0', grad_fn=<MseLossBackward>) 140\n",
      "tensor(0.8029, device='cuda:0', grad_fn=<MseLossBackward>) 141\n",
      "tensor(0.9120, device='cuda:0', grad_fn=<MseLossBackward>) 142\n",
      "tensor(1.0555, device='cuda:0', grad_fn=<MseLossBackward>) 143\n",
      "tensor(0.8381, device='cuda:0', grad_fn=<MseLossBackward>) 144\n",
      "tensor(1.0441, device='cuda:0', grad_fn=<MseLossBackward>) 145\n",
      "tensor(0.8637, device='cuda:0', grad_fn=<MseLossBackward>) 146\n",
      "tensor(0.8239, device='cuda:0', grad_fn=<MseLossBackward>) 147\n",
      "tensor(1.1549, device='cuda:0', grad_fn=<MseLossBackward>) 148\n",
      "tensor(0.8729, device='cuda:0', grad_fn=<MseLossBackward>) 149\n",
      "tensor(0.8767, device='cuda:0', grad_fn=<MseLossBackward>) 150\n",
      "tensor(1.0593, device='cuda:0', grad_fn=<MseLossBackward>) 151\n",
      "tensor(0.8662, device='cuda:0', grad_fn=<MseLossBackward>) 152\n",
      "tensor(1.0151, device='cuda:0', grad_fn=<MseLossBackward>) 153\n",
      "tensor(0.8859, device='cuda:0', grad_fn=<MseLossBackward>) 154\n",
      "tensor(0.8552, device='cuda:0', grad_fn=<MseLossBackward>) 155\n",
      "tensor(1.0838, device='cuda:0', grad_fn=<MseLossBackward>) 156\n",
      "tensor(0.8334, device='cuda:0', grad_fn=<MseLossBackward>) 157\n",
      "tensor(0.9302, device='cuda:0', grad_fn=<MseLossBackward>) 158\n",
      "tensor(1.0238, device='cuda:0', grad_fn=<MseLossBackward>) 159\n",
      "tensor(0.8065, device='cuda:0', grad_fn=<MseLossBackward>) 160\n",
      "tensor(1.0111, device='cuda:0', grad_fn=<MseLossBackward>) 161\n",
      "tensor(0.8761, device='cuda:0', grad_fn=<MseLossBackward>) 162\n",
      "tensor(0.8425, device='cuda:0', grad_fn=<MseLossBackward>) 163\n",
      "tensor(1.0711, device='cuda:0', grad_fn=<MseLossBackward>) 164\n",
      "tensor(0.8316, device='cuda:0', grad_fn=<MseLossBackward>) 165\n",
      "tensor(0.9157, device='cuda:0', grad_fn=<MseLossBackward>) 166\n",
      "tensor(0.9845, device='cuda:0', grad_fn=<MseLossBackward>) 167\n",
      "tensor(0.8294, device='cuda:0', grad_fn=<MseLossBackward>) 168\n",
      "tensor(1.0151, device='cuda:0', grad_fn=<MseLossBackward>) 169\n",
      "tensor(0.8487, device='cuda:0', grad_fn=<MseLossBackward>) 170\n",
      "tensor(0.8587, device='cuda:0', grad_fn=<MseLossBackward>) 171\n",
      "tensor(1.0754, device='cuda:0', grad_fn=<MseLossBackward>) 172\n",
      "tensor(0.8269, device='cuda:0', grad_fn=<MseLossBackward>) 173\n",
      "tensor(0.9225, device='cuda:0', grad_fn=<MseLossBackward>) 174\n",
      "tensor(0.9719, device='cuda:0', grad_fn=<MseLossBackward>) 175\n",
      "tensor(0.8160, device='cuda:0', grad_fn=<MseLossBackward>) 176\n",
      "tensor(1.0289, device='cuda:0', grad_fn=<MseLossBackward>) 177\n",
      "tensor(0.8612, device='cuda:0', grad_fn=<MseLossBackward>) 178\n",
      "tensor(0.8280, device='cuda:0', grad_fn=<MseLossBackward>) 179\n",
      "tensor(1.0714, device='cuda:0', grad_fn=<MseLossBackward>) 180\n",
      "tensor(0.8599, device='cuda:0', grad_fn=<MseLossBackward>) 181\n",
      "tensor(0.9176, device='cuda:0', grad_fn=<MseLossBackward>) 182\n",
      "tensor(0.9686, device='cuda:0', grad_fn=<MseLossBackward>) 183\n",
      "tensor(0.8152, device='cuda:0', grad_fn=<MseLossBackward>) 184\n",
      "tensor(1.0148, device='cuda:0', grad_fn=<MseLossBackward>) 185\n",
      "tensor(0.8763, device='cuda:0', grad_fn=<MseLossBackward>) 186\n",
      "tensor(0.8248, device='cuda:0', grad_fn=<MseLossBackward>) 187\n",
      "tensor(1.0313, device='cuda:0', grad_fn=<MseLossBackward>) 188\n",
      "tensor(0.8473, device='cuda:0', grad_fn=<MseLossBackward>) 189\n",
      "tensor(0.9160, device='cuda:0', grad_fn=<MseLossBackward>) 190\n",
      "tensor(0.9642, device='cuda:0', grad_fn=<MseLossBackward>) 191\n",
      "tensor(0.7858, device='cuda:0', grad_fn=<MseLossBackward>) 192\n",
      "tensor(0.9817, device='cuda:0', grad_fn=<MseLossBackward>) 193\n",
      "tensor(0.8974, device='cuda:0', grad_fn=<MseLossBackward>) 194\n",
      "tensor(0.8124, device='cuda:0', grad_fn=<MseLossBackward>) 195\n",
      "tensor(1.0008, device='cuda:0', grad_fn=<MseLossBackward>) 196\n",
      "tensor(0.8437, device='cuda:0', grad_fn=<MseLossBackward>) 197\n",
      "tensor(0.9120, device='cuda:0', grad_fn=<MseLossBackward>) 198\n",
      "tensor(0.9664, device='cuda:0', grad_fn=<MseLossBackward>) 199\n",
      "tensor(0.7851, device='cuda:0', grad_fn=<MseLossBackward>) 200\n",
      "tensor(0.9543, device='cuda:0', grad_fn=<MseLossBackward>) 201\n",
      "tensor(0.8897, device='cuda:0', grad_fn=<MseLossBackward>) 202\n",
      "tensor(0.8233, device='cuda:0', grad_fn=<MseLossBackward>) 203\n",
      "tensor(0.9991, device='cuda:0', grad_fn=<MseLossBackward>) 204\n",
      "tensor(0.8213, device='cuda:0', grad_fn=<MseLossBackward>) 205\n",
      "tensor(0.8942, device='cuda:0', grad_fn=<MseLossBackward>) 206\n",
      "tensor(0.9863, device='cuda:0', grad_fn=<MseLossBackward>) 207\n",
      "tensor(0.7888, device='cuda:0', grad_fn=<MseLossBackward>) 208\n",
      "tensor(0.9368, device='cuda:0', grad_fn=<MseLossBackward>) 209\n",
      "tensor(0.8911, device='cuda:0', grad_fn=<MseLossBackward>) 210\n",
      "tensor(0.8275, device='cuda:0', grad_fn=<MseLossBackward>) 211\n",
      "tensor(1.0113, device='cuda:0', grad_fn=<MseLossBackward>) 212\n",
      "tensor(0.8203, device='cuda:0', grad_fn=<MseLossBackward>) 213\n",
      "tensor(0.8694, device='cuda:0', grad_fn=<MseLossBackward>) 214\n",
      "tensor(1.0059, device='cuda:0', grad_fn=<MseLossBackward>) 215\n",
      "tensor(0.8072, device='cuda:0', grad_fn=<MseLossBackward>) 216\n",
      "tensor(0.9359, device='cuda:0', grad_fn=<MseLossBackward>) 217\n",
      "tensor(0.8970, device='cuda:0', grad_fn=<MseLossBackward>) 218\n",
      "tensor(0.8065, device='cuda:0', grad_fn=<MseLossBackward>) 219\n",
      "tensor(1.0215, device='cuda:0', grad_fn=<MseLossBackward>) 220\n",
      "tensor(0.8481, device='cuda:0', grad_fn=<MseLossBackward>) 221\n",
      "tensor(0.8395, device='cuda:0', grad_fn=<MseLossBackward>) 222\n",
      "tensor(1.0121, device='cuda:0', grad_fn=<MseLossBackward>) 223\n",
      "tensor(0.8285, device='cuda:0', grad_fn=<MseLossBackward>) 224\n",
      "tensor(0.9227, device='cuda:0', grad_fn=<MseLossBackward>) 225\n",
      "tensor(0.9179, device='cuda:0', grad_fn=<MseLossBackward>) 226\n",
      "tensor(0.7853, device='cuda:0', grad_fn=<MseLossBackward>) 227\n",
      "tensor(0.9840, device='cuda:0', grad_fn=<MseLossBackward>) 228\n",
      "tensor(0.8765, device='cuda:0', grad_fn=<MseLossBackward>) 229\n",
      "tensor(0.8169, device='cuda:0', grad_fn=<MseLossBackward>) 230\n",
      "tensor(0.9833, device='cuda:0', grad_fn=<MseLossBackward>) 231\n",
      "tensor(0.8270, device='cuda:0', grad_fn=<MseLossBackward>) 232\n",
      "tensor(0.8986, device='cuda:0', grad_fn=<MseLossBackward>) 233\n",
      "tensor(0.9499, device='cuda:0', grad_fn=<MseLossBackward>) 234\n",
      "tensor(0.7687, device='cuda:0', grad_fn=<MseLossBackward>) 235\n",
      "tensor(0.9238, device='cuda:0', grad_fn=<MseLossBackward>) 236\n",
      "tensor(0.8992, device='cuda:0', grad_fn=<MseLossBackward>) 237\n",
      "tensor(0.8031, device='cuda:0', grad_fn=<MseLossBackward>) 238\n",
      "tensor(0.9686, device='cuda:0', grad_fn=<MseLossBackward>) 239\n",
      "tensor(0.8247, device='cuda:0', grad_fn=<MseLossBackward>) 240\n",
      "tensor(0.8571, device='cuda:0', grad_fn=<MseLossBackward>) 241\n",
      "tensor(0.9952, device='cuda:0', grad_fn=<MseLossBackward>) 242\n",
      "tensor(0.7870, device='cuda:0', grad_fn=<MseLossBackward>) 243\n",
      "tensor(0.8842, device='cuda:0', grad_fn=<MseLossBackward>) 244\n",
      "tensor(0.9339, device='cuda:0', grad_fn=<MseLossBackward>) 245\n",
      "tensor(0.8019, device='cuda:0', grad_fn=<MseLossBackward>) 246\n",
      "tensor(0.9794, device='cuda:0', grad_fn=<MseLossBackward>) 247\n",
      "tensor(0.8535, device='cuda:0', grad_fn=<MseLossBackward>) 248\n",
      "tensor(0.8029, device='cuda:0', grad_fn=<MseLossBackward>) 249\n",
      "tensor(1.0225, device='cuda:0', grad_fn=<MseLossBackward>) 250\n",
      "tensor(0.8501, device='cuda:0', grad_fn=<MseLossBackward>) 251\n",
      "tensor(0.8490, device='cuda:0', grad_fn=<MseLossBackward>) 252\n",
      "tensor(0.9571, device='cuda:0', grad_fn=<MseLossBackward>) 253\n",
      "tensor(0.7987, device='cuda:0', grad_fn=<MseLossBackward>) 254\n",
      "tensor(0.9266, device='cuda:0', grad_fn=<MseLossBackward>) 255\n",
      "tensor(0.9023, device='cuda:0', grad_fn=<MseLossBackward>) 256\n",
      "tensor(0.7630, device='cuda:0', grad_fn=<MseLossBackward>) 257\n",
      "tensor(0.9371, device='cuda:0', grad_fn=<MseLossBackward>) 258\n",
      "tensor(0.8748, device='cuda:0', grad_fn=<MseLossBackward>) 259\n",
      "tensor(0.8154, device='cuda:0', grad_fn=<MseLossBackward>) 260\n",
      "tensor(0.9690, device='cuda:0', grad_fn=<MseLossBackward>) 261\n",
      "tensor(0.8048, device='cuda:0', grad_fn=<MseLossBackward>) 262\n",
      "tensor(0.9025, device='cuda:0', grad_fn=<MseLossBackward>) 263\n",
      "tensor(0.8712, device='cuda:0', grad_fn=<MseLossBackward>) 264\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.7879, device='cuda:0', grad_fn=<MseLossBackward>) 265\n",
      "tensor(0.9595, device='cuda:0', grad_fn=<MseLossBackward>) 266\n",
      "tensor(1.0049, device='cuda:0', grad_fn=<MseLossBackward>) 267\n",
      "tensor(0.8988, device='cuda:0', grad_fn=<MseLossBackward>) 268\n",
      "tensor(0.8570, device='cuda:0', grad_fn=<MseLossBackward>) 269\n",
      "tensor(1.0338, device='cuda:0', grad_fn=<MseLossBackward>) 270\n",
      "tensor(0.8014, device='cuda:0', grad_fn=<MseLossBackward>) 271\n",
      "tensor(0.8299, device='cuda:0', grad_fn=<MseLossBackward>) 272\n",
      "tensor(1.0213, device='cuda:0', grad_fn=<MseLossBackward>) 273\n",
      "tensor(0.8698, device='cuda:0', grad_fn=<MseLossBackward>) 274\n",
      "tensor(0.9215, device='cuda:0', grad_fn=<MseLossBackward>) 275\n",
      "tensor(0.8928, device='cuda:0', grad_fn=<MseLossBackward>) 276\n",
      "tensor(0.8030, device='cuda:0', grad_fn=<MseLossBackward>) 277\n",
      "tensor(0.9751, device='cuda:0', grad_fn=<MseLossBackward>) 278\n",
      "tensor(1.0221, device='cuda:0', grad_fn=<MseLossBackward>) 279\n",
      "tensor(0.9420, device='cuda:0', grad_fn=<MseLossBackward>) 280\n",
      "tensor(0.8913, device='cuda:0', grad_fn=<MseLossBackward>) 281\n",
      "tensor(0.7524, device='cuda:0', grad_fn=<MseLossBackward>) 282\n",
      "tensor(0.8797, device='cuda:0', grad_fn=<MseLossBackward>) 283\n",
      "tensor(0.8168, device='cuda:0', grad_fn=<MseLossBackward>) 284\n",
      "tensor(0.8262, device='cuda:0', grad_fn=<MseLossBackward>) 285\n",
      "tensor(1.0412, device='cuda:0', grad_fn=<MseLossBackward>) 286\n",
      "tensor(0.8328, device='cuda:0', grad_fn=<MseLossBackward>) 287\n",
      "tensor(1.0134, device='cuda:0', grad_fn=<MseLossBackward>) 288\n",
      "tensor(0.8439, device='cuda:0', grad_fn=<MseLossBackward>) 289\n",
      "tensor(0.8191, device='cuda:0', grad_fn=<MseLossBackward>) 290\n",
      "tensor(0.8155, device='cuda:0', grad_fn=<MseLossBackward>) 291\n",
      "tensor(0.8383, device='cuda:0', grad_fn=<MseLossBackward>) 292\n",
      "tensor(0.7993, device='cuda:0', grad_fn=<MseLossBackward>) 293\n",
      "tensor(0.6747, device='cuda:0', grad_fn=<MseLossBackward>) 294\n",
      "tensor(0.8819, device='cuda:0', grad_fn=<MseLossBackward>) 295\n",
      "tensor(0.7216, device='cuda:0', grad_fn=<MseLossBackward>) 296\n",
      "tensor(1.1074, device='cuda:0', grad_fn=<MseLossBackward>) 297\n",
      "tensor(0.7283, device='cuda:0', grad_fn=<MseLossBackward>) 298\n",
      "tensor(1.0542, device='cuda:0', grad_fn=<MseLossBackward>) 299\n",
      "tensor(0.9432, device='cuda:0', grad_fn=<MseLossBackward>) 300\n",
      "tensor(1.4298, device='cuda:0', grad_fn=<MseLossBackward>) 301\n",
      "tensor(0.8232, device='cuda:0', grad_fn=<MseLossBackward>) 302\n",
      "tensor(1.1485, device='cuda:0', grad_fn=<MseLossBackward>) 303\n",
      "tensor(0.7579, device='cuda:0', grad_fn=<MseLossBackward>) 304\n",
      "tensor(0.7699, device='cuda:0', grad_fn=<MseLossBackward>) 305\n",
      "tensor(0.8072, device='cuda:0', grad_fn=<MseLossBackward>) 306\n",
      "tensor(0.9371, device='cuda:0', grad_fn=<MseLossBackward>) 307\n",
      "tensor(0.9870, device='cuda:0', grad_fn=<MseLossBackward>) 308\n",
      "tensor(1.1404, device='cuda:0', grad_fn=<MseLossBackward>) 309\n",
      "tensor(1.0931, device='cuda:0', grad_fn=<MseLossBackward>) 310\n",
      "tensor(0.8164, device='cuda:0', grad_fn=<MseLossBackward>) 311\n",
      "tensor(0.8822, device='cuda:0', grad_fn=<MseLossBackward>) 312\n",
      "tensor(0.9726, device='cuda:0', grad_fn=<MseLossBackward>) 313\n",
      "tensor(0.7649, device='cuda:0', grad_fn=<MseLossBackward>) 314\n",
      "tensor(0.9656, device='cuda:0', grad_fn=<MseLossBackward>) 315\n",
      "tensor(0.7826, device='cuda:0', grad_fn=<MseLossBackward>) 316\n",
      "tensor(1.0671, device='cuda:0', grad_fn=<MseLossBackward>) 317\n",
      "tensor(0.7829, device='cuda:0', grad_fn=<MseLossBackward>) 318\n",
      "tensor(0.8810, device='cuda:0', grad_fn=<MseLossBackward>) 319\n",
      "tensor(1.0504, device='cuda:0', grad_fn=<MseLossBackward>) 320\n",
      "tensor(1.2950, device='cuda:0', grad_fn=<MseLossBackward>) 321\n",
      "tensor(0.9237, device='cuda:0', grad_fn=<MseLossBackward>) 322\n",
      "tensor(1.0100, device='cuda:0', grad_fn=<MseLossBackward>) 323\n",
      "tensor(0.8199, device='cuda:0', grad_fn=<MseLossBackward>) 324\n",
      "tensor(0.8356, device='cuda:0', grad_fn=<MseLossBackward>) 325\n",
      "tensor(1.0516, device='cuda:0', grad_fn=<MseLossBackward>) 326\n",
      "tensor(0.8110, device='cuda:0', grad_fn=<MseLossBackward>) 327\n",
      "tensor(0.7451, device='cuda:0', grad_fn=<MseLossBackward>) 328\n",
      "tensor(1.0196, device='cuda:0', grad_fn=<MseLossBackward>) 329\n",
      "tensor(0.7562, device='cuda:0', grad_fn=<MseLossBackward>) 330\n",
      "tensor(0.7921, device='cuda:0', grad_fn=<MseLossBackward>) 331\n",
      "tensor(1.0358, device='cuda:0', grad_fn=<MseLossBackward>) 332\n",
      "tensor(0.7895, device='cuda:0', grad_fn=<MseLossBackward>) 333\n",
      "tensor(0.9005, device='cuda:0', grad_fn=<MseLossBackward>) 334\n",
      "tensor(0.9020, device='cuda:0', grad_fn=<MseLossBackward>) 335\n",
      "tensor(0.7582, device='cuda:0', grad_fn=<MseLossBackward>) 336\n",
      "tensor(0.9018, device='cuda:0', grad_fn=<MseLossBackward>) 337\n",
      "tensor(0.8331, device='cuda:0', grad_fn=<MseLossBackward>) 338\n",
      "tensor(0.7696, device='cuda:0', grad_fn=<MseLossBackward>) 339\n",
      "tensor(0.9216, device='cuda:0', grad_fn=<MseLossBackward>) 340\n",
      "tensor(0.7948, device='cuda:0', grad_fn=<MseLossBackward>) 341\n",
      "tensor(0.8198, device='cuda:0', grad_fn=<MseLossBackward>) 342\n",
      "tensor(0.9381, device='cuda:0', grad_fn=<MseLossBackward>) 343\n",
      "tensor(0.7766, device='cuda:0', grad_fn=<MseLossBackward>) 344\n",
      "tensor(0.8660, device='cuda:0', grad_fn=<MseLossBackward>) 345\n",
      "tensor(0.9074, device='cuda:0', grad_fn=<MseLossBackward>) 346\n",
      "tensor(0.7691, device='cuda:0', grad_fn=<MseLossBackward>) 347\n",
      "tensor(0.9060, device='cuda:0', grad_fn=<MseLossBackward>) 348\n",
      "tensor(0.8511, device='cuda:0', grad_fn=<MseLossBackward>) 349\n",
      "tensor(0.7669, device='cuda:0', grad_fn=<MseLossBackward>) 350\n",
      "tensor(0.9294, device='cuda:0', grad_fn=<MseLossBackward>) 351\n",
      "tensor(0.8123, device='cuda:0', grad_fn=<MseLossBackward>) 352\n",
      "tensor(0.8024, device='cuda:0', grad_fn=<MseLossBackward>) 353\n",
      "tensor(0.9381, device='cuda:0', grad_fn=<MseLossBackward>) 354\n",
      "tensor(0.7872, device='cuda:0', grad_fn=<MseLossBackward>) 355\n",
      "tensor(0.8503, device='cuda:0', grad_fn=<MseLossBackward>) 356\n",
      "tensor(0.9197, device='cuda:0', grad_fn=<MseLossBackward>) 357\n",
      "tensor(0.7653, device='cuda:0', grad_fn=<MseLossBackward>) 358\n",
      "tensor(0.8816, device='cuda:0', grad_fn=<MseLossBackward>) 359\n",
      "tensor(0.8730, device='cuda:0', grad_fn=<MseLossBackward>) 360\n",
      "tensor(0.7553, device='cuda:0', grad_fn=<MseLossBackward>) 361\n",
      "tensor(0.9024, device='cuda:0', grad_fn=<MseLossBackward>) 362\n",
      "tensor(0.8230, device='cuda:0', grad_fn=<MseLossBackward>) 363\n",
      "tensor(0.7722, device='cuda:0', grad_fn=<MseLossBackward>) 364\n",
      "tensor(0.9280, device='cuda:0', grad_fn=<MseLossBackward>) 365\n",
      "tensor(0.7923, device='cuda:0', grad_fn=<MseLossBackward>) 366\n",
      "tensor(0.8124, device='cuda:0', grad_fn=<MseLossBackward>) 367\n",
      "tensor(0.9346, device='cuda:0', grad_fn=<MseLossBackward>) 368\n",
      "tensor(0.7782, device='cuda:0', grad_fn=<MseLossBackward>) 369\n",
      "tensor(0.8550, device='cuda:0', grad_fn=<MseLossBackward>) 370\n",
      "tensor(0.9006, device='cuda:0', grad_fn=<MseLossBackward>) 371\n",
      "tensor(0.7543, device='cuda:0', grad_fn=<MseLossBackward>) 372\n",
      "tensor(0.8799, device='cuda:0', grad_fn=<MseLossBackward>) 373\n",
      "tensor(0.8527, device='cuda:0', grad_fn=<MseLossBackward>) 374\n",
      "tensor(0.7432, device='cuda:0', grad_fn=<MseLossBackward>) 375\n",
      "tensor(0.8887, device='cuda:0', grad_fn=<MseLossBackward>) 376\n",
      "tensor(0.8076, device='cuda:0', grad_fn=<MseLossBackward>) 377\n",
      "tensor(0.7640, device='cuda:0', grad_fn=<MseLossBackward>) 378\n",
      "tensor(0.9129, device='cuda:0', grad_fn=<MseLossBackward>) 379\n",
      "tensor(0.7768, device='cuda:0', grad_fn=<MseLossBackward>) 380\n",
      "tensor(0.7994, device='cuda:0', grad_fn=<MseLossBackward>) 381\n",
      "tensor(0.9226, device='cuda:0', grad_fn=<MseLossBackward>) 382\n",
      "tensor(0.7668, device='cuda:0', grad_fn=<MseLossBackward>) 383\n",
      "tensor(0.8369, device='cuda:0', grad_fn=<MseLossBackward>) 384\n",
      "tensor(0.8881, device='cuda:0', grad_fn=<MseLossBackward>) 385\n",
      "tensor(0.7424, device='cuda:0', grad_fn=<MseLossBackward>) 386\n",
      "tensor(0.8604, device='cuda:0', grad_fn=<MseLossBackward>) 387\n",
      "tensor(0.8429, device='cuda:0', grad_fn=<MseLossBackward>) 388\n",
      "tensor(0.7242, device='cuda:0', grad_fn=<MseLossBackward>) 389\n",
      "tensor(0.8630, device='cuda:0', grad_fn=<MseLossBackward>) 390\n",
      "tensor(0.8016, device='cuda:0', grad_fn=<MseLossBackward>) 391\n",
      "tensor(0.7393, device='cuda:0', grad_fn=<MseLossBackward>) 392\n",
      "tensor(0.8868, device='cuda:0', grad_fn=<MseLossBackward>) 393\n",
      "tensor(0.7679, device='cuda:0', grad_fn=<MseLossBackward>) 394\n",
      "tensor(0.7681, device='cuda:0', grad_fn=<MseLossBackward>) 395\n",
      "tensor(0.9092, device='cuda:0', grad_fn=<MseLossBackward>) 396\n",
      "tensor(0.7593, device='cuda:0', grad_fn=<MseLossBackward>) 397\n",
      "tensor(0.8032, device='cuda:0', grad_fn=<MseLossBackward>) 398\n",
      "tensor(0.8868, device='cuda:0', grad_fn=<MseLossBackward>) 399\n",
      "tensor(0.7382, device='cuda:0', grad_fn=<MseLossBackward>) 400\n",
      "tensor(0.8289, device='cuda:0', grad_fn=<MseLossBackward>) 401\n",
      "tensor(0.8487, device='cuda:0', grad_fn=<MseLossBackward>) 402\n",
      "tensor(0.7098, device='cuda:0', grad_fn=<MseLossBackward>) 403\n",
      "tensor(0.8296, device='cuda:0', grad_fn=<MseLossBackward>) 404\n",
      "tensor(0.8113, device='cuda:0', grad_fn=<MseLossBackward>) 405\n",
      "tensor(0.7089, device='cuda:0', grad_fn=<MseLossBackward>) 406\n",
      "tensor(0.8456, device='cuda:0', grad_fn=<MseLossBackward>) 407\n",
      "tensor(0.7751, device='cuda:0', grad_fn=<MseLossBackward>) 408\n",
      "tensor(0.7250, device='cuda:0', grad_fn=<MseLossBackward>) 409\n",
      "tensor(0.8805, device='cuda:0', grad_fn=<MseLossBackward>) 410\n",
      "tensor(0.7571, device='cuda:0', grad_fn=<MseLossBackward>) 411\n",
      "tensor(0.7511, device='cuda:0', grad_fn=<MseLossBackward>) 412\n",
      "tensor(0.8838, device='cuda:0', grad_fn=<MseLossBackward>) 413\n",
      "tensor(0.7460, device='cuda:0', grad_fn=<MseLossBackward>) 414\n",
      "tensor(0.7810, device='cuda:0', grad_fn=<MseLossBackward>) 415\n",
      "tensor(0.8611, device='cuda:0', grad_fn=<MseLossBackward>) 416\n",
      "tensor(0.7130, device='cuda:0', grad_fn=<MseLossBackward>) 417\n",
      "tensor(0.7921, device='cuda:0', grad_fn=<MseLossBackward>) 418\n",
      "tensor(0.8383, device='cuda:0', grad_fn=<MseLossBackward>) 419\n",
      "tensor(0.6969, device='cuda:0', grad_fn=<MseLossBackward>) 420\n",
      "tensor(0.8006, device='cuda:0', grad_fn=<MseLossBackward>) 421\n",
      "tensor(0.8066, device='cuda:0', grad_fn=<MseLossBackward>) 422\n",
      "tensor(0.6941, device='cuda:0', grad_fn=<MseLossBackward>) 423\n",
      "tensor(0.8256, device='cuda:0', grad_fn=<MseLossBackward>) 424\n",
      "tensor(0.7755, device='cuda:0', grad_fn=<MseLossBackward>) 425\n",
      "tensor(0.6946, device='cuda:0', grad_fn=<MseLossBackward>) 426\n",
      "tensor(0.8413, device='cuda:0', grad_fn=<MseLossBackward>) 427\n",
      "tensor(0.7541, device='cuda:0', grad_fn=<MseLossBackward>) 428\n",
      "tensor(0.7115, device='cuda:0', grad_fn=<MseLossBackward>) 429\n",
      "tensor(0.8493, device='cuda:0', grad_fn=<MseLossBackward>) 430\n",
      "tensor(0.7332, device='cuda:0', grad_fn=<MseLossBackward>) 431\n",
      "tensor(0.7344, device='cuda:0', grad_fn=<MseLossBackward>) 432\n",
      "tensor(0.8563, device='cuda:0', grad_fn=<MseLossBackward>) 433\n",
      "tensor(0.7169, device='cuda:0', grad_fn=<MseLossBackward>) 434\n",
      "tensor(0.7544, device='cuda:0', grad_fn=<MseLossBackward>) 435\n",
      "tensor(0.8475, device='cuda:0', grad_fn=<MseLossBackward>) 436\n",
      "tensor(0.7046, device='cuda:0', grad_fn=<MseLossBackward>) 437\n",
      "tensor(0.7718, device='cuda:0', grad_fn=<MseLossBackward>) 438\n",
      "tensor(0.8236, device='cuda:0', grad_fn=<MseLossBackward>) 439\n",
      "tensor(0.6844, device='cuda:0', grad_fn=<MseLossBackward>) 440\n",
      "tensor(0.7781, device='cuda:0', grad_fn=<MseLossBackward>) 441\n",
      "tensor(0.7989, device='cuda:0', grad_fn=<MseLossBackward>) 442\n",
      "tensor(0.6731, device='cuda:0', grad_fn=<MseLossBackward>) 443\n",
      "tensor(0.7846, device='cuda:0', grad_fn=<MseLossBackward>) 444\n",
      "tensor(0.7748, device='cuda:0', grad_fn=<MseLossBackward>) 445\n",
      "tensor(0.6722, device='cuda:0', grad_fn=<MseLossBackward>) 446\n",
      "tensor(0.8025, device='cuda:0', grad_fn=<MseLossBackward>) 447\n",
      "tensor(0.7537, device='cuda:0', grad_fn=<MseLossBackward>) 448\n",
      "tensor(0.6760, device='cuda:0', grad_fn=<MseLossBackward>) 449\n",
      "tensor(0.8172, device='cuda:0', grad_fn=<MseLossBackward>) 450\n",
      "tensor(0.7373, device='cuda:0', grad_fn=<MseLossBackward>) 451\n",
      "tensor(0.6875, device='cuda:0', grad_fn=<MseLossBackward>) 452\n",
      "tensor(0.8280, device='cuda:0', grad_fn=<MseLossBackward>) 453\n",
      "tensor(0.7258, device='cuda:0', grad_fn=<MseLossBackward>) 454\n",
      "tensor(0.7023, device='cuda:0', grad_fn=<MseLossBackward>) 455\n",
      "tensor(0.8349, device='cuda:0', grad_fn=<MseLossBackward>) 456\n",
      "tensor(0.7136, device='cuda:0', grad_fn=<MseLossBackward>) 457\n",
      "tensor(0.7151, device='cuda:0', grad_fn=<MseLossBackward>) 458\n",
      "tensor(0.8392, device='cuda:0', grad_fn=<MseLossBackward>) 459\n",
      "tensor(0.7064, device='cuda:0', grad_fn=<MseLossBackward>) 460\n",
      "tensor(0.7289, device='cuda:0', grad_fn=<MseLossBackward>) 461\n",
      "tensor(0.8367, device='cuda:0', grad_fn=<MseLossBackward>) 462\n",
      "tensor(0.6980, device='cuda:0', grad_fn=<MseLossBackward>) 463\n",
      "tensor(0.7385, device='cuda:0', grad_fn=<MseLossBackward>) 464\n",
      "tensor(0.8281, device='cuda:0', grad_fn=<MseLossBackward>) 465\n",
      "tensor(0.6866, device='cuda:0', grad_fn=<MseLossBackward>) 466\n",
      "tensor(0.7439, device='cuda:0', grad_fn=<MseLossBackward>) 467\n",
      "tensor(0.8191, device='cuda:0', grad_fn=<MseLossBackward>) 468\n",
      "tensor(0.6791, device='cuda:0', grad_fn=<MseLossBackward>) 469\n",
      "tensor(0.7497, device='cuda:0', grad_fn=<MseLossBackward>) 470\n",
      "tensor(0.8076, device='cuda:0', grad_fn=<MseLossBackward>) 471\n",
      "tensor(0.6714, device='cuda:0', grad_fn=<MseLossBackward>) 472\n",
      "tensor(0.7543, device='cuda:0', grad_fn=<MseLossBackward>) 473\n",
      "tensor(0.7959, device='cuda:0', grad_fn=<MseLossBackward>) 474\n",
      "tensor(0.6639, device='cuda:0', grad_fn=<MseLossBackward>) 475\n",
      "tensor(0.7569, device='cuda:0', grad_fn=<MseLossBackward>) 476\n",
      "tensor(0.7851, device='cuda:0', grad_fn=<MseLossBackward>) 477\n",
      "tensor(0.6601, device='cuda:0', grad_fn=<MseLossBackward>) 478\n",
      "tensor(0.7624, device='cuda:0', grad_fn=<MseLossBackward>) 479\n",
      "tensor(0.7745, device='cuda:0', grad_fn=<MseLossBackward>) 480\n",
      "tensor(0.6559, device='cuda:0', grad_fn=<MseLossBackward>) 481\n",
      "tensor(0.7669, device='cuda:0', grad_fn=<MseLossBackward>) 482\n",
      "tensor(0.7650, device='cuda:0', grad_fn=<MseLossBackward>) 483\n",
      "tensor(0.6536, device='cuda:0', grad_fn=<MseLossBackward>) 484\n",
      "tensor(0.7710, device='cuda:0', grad_fn=<MseLossBackward>) 485\n",
      "tensor(0.7567, device='cuda:0', grad_fn=<MseLossBackward>) 486\n",
      "tensor(0.6541, device='cuda:0', grad_fn=<MseLossBackward>) 487\n",
      "tensor(0.7785, device='cuda:0', grad_fn=<MseLossBackward>) 488\n",
      "tensor(0.7490, device='cuda:0', grad_fn=<MseLossBackward>) 489\n",
      "tensor(0.6544, device='cuda:0', grad_fn=<MseLossBackward>) 490\n",
      "tensor(0.7845, device='cuda:0', grad_fn=<MseLossBackward>) 491\n",
      "tensor(0.7431, device='cuda:0', grad_fn=<MseLossBackward>) 492\n",
      "tensor(0.6569, device='cuda:0', grad_fn=<MseLossBackward>) 493\n",
      "tensor(0.7898, device='cuda:0', grad_fn=<MseLossBackward>) 494\n",
      "tensor(0.7357, device='cuda:0', grad_fn=<MseLossBackward>) 495\n",
      "tensor(0.6600, device='cuda:0', grad_fn=<MseLossBackward>) 496\n",
      "tensor(0.8007, device='cuda:0', grad_fn=<MseLossBackward>) 497\n",
      "tensor(0.7329, device='cuda:0', grad_fn=<MseLossBackward>) 498\n",
      "tensor(0.6673, device='cuda:0', grad_fn=<MseLossBackward>) 499\n",
      "tensor(0.8092, device='cuda:0', grad_fn=<MseLossBackward>) 500\n",
      "tensor(0.7299, device='cuda:0', grad_fn=<MseLossBackward>) 501\n",
      "tensor(0.6735, device='cuda:0', grad_fn=<MseLossBackward>) 502\n",
      "tensor(0.8143, device='cuda:0', grad_fn=<MseLossBackward>) 503\n",
      "tensor(0.7245, device='cuda:0', grad_fn=<MseLossBackward>) 504\n",
      "tensor(0.6796, device='cuda:0', grad_fn=<MseLossBackward>) 505\n",
      "tensor(0.8213, device='cuda:0', grad_fn=<MseLossBackward>) 506\n",
      "tensor(0.7220, device='cuda:0', grad_fn=<MseLossBackward>) 507\n",
      "tensor(0.6852, device='cuda:0', grad_fn=<MseLossBackward>) 508\n",
      "tensor(0.8228, device='cuda:0', grad_fn=<MseLossBackward>) 509\n",
      "tensor(0.7177, device='cuda:0', grad_fn=<MseLossBackward>) 510\n",
      "tensor(0.6902, device='cuda:0', grad_fn=<MseLossBackward>) 511\n",
      "tensor(0.8263, device='cuda:0', grad_fn=<MseLossBackward>) 512\n",
      "tensor(0.7149, device='cuda:0', grad_fn=<MseLossBackward>) 513\n",
      "tensor(0.6949, device='cuda:0', grad_fn=<MseLossBackward>) 514\n",
      "tensor(0.8300, device='cuda:0', grad_fn=<MseLossBackward>) 515\n",
      "tensor(0.7154, device='cuda:0', grad_fn=<MseLossBackward>) 516\n",
      "tensor(0.6986, device='cuda:0', grad_fn=<MseLossBackward>) 517\n",
      "tensor(0.8270, device='cuda:0', grad_fn=<MseLossBackward>) 518\n",
      "tensor(0.7093, device='cuda:0', grad_fn=<MseLossBackward>) 519\n",
      "tensor(0.6986, device='cuda:0', grad_fn=<MseLossBackward>) 520\n",
      "tensor(0.8284, device='cuda:0', grad_fn=<MseLossBackward>) 521\n",
      "tensor(0.7085, device='cuda:0', grad_fn=<MseLossBackward>) 522\n",
      "tensor(0.7010, device='cuda:0', grad_fn=<MseLossBackward>) 523\n",
      "tensor(0.8288, device='cuda:0', grad_fn=<MseLossBackward>) 524\n",
      "tensor(0.7082, device='cuda:0', grad_fn=<MseLossBackward>) 525\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.7018, device='cuda:0', grad_fn=<MseLossBackward>) 526\n",
      "tensor(0.8273, device='cuda:0', grad_fn=<MseLossBackward>) 527\n",
      "tensor(0.7063, device='cuda:0', grad_fn=<MseLossBackward>) 528\n",
      "tensor(0.7015, device='cuda:0', grad_fn=<MseLossBackward>) 529\n",
      "tensor(0.8277, device='cuda:0', grad_fn=<MseLossBackward>) 530\n",
      "tensor(0.7059, device='cuda:0', grad_fn=<MseLossBackward>) 531\n",
      "tensor(0.7012, device='cuda:0', grad_fn=<MseLossBackward>) 532\n",
      "tensor(0.8268, device='cuda:0', grad_fn=<MseLossBackward>) 533\n",
      "tensor(0.7063, device='cuda:0', grad_fn=<MseLossBackward>) 534\n",
      "tensor(0.7009, device='cuda:0', grad_fn=<MseLossBackward>) 535\n",
      "tensor(0.8262, device='cuda:0', grad_fn=<MseLossBackward>) 536\n",
      "tensor(0.7054, device='cuda:0', grad_fn=<MseLossBackward>) 537\n",
      "tensor(0.6984, device='cuda:0', grad_fn=<MseLossBackward>) 538\n",
      "tensor(0.8267, device='cuda:0', grad_fn=<MseLossBackward>) 539\n",
      "tensor(0.7087, device='cuda:0', grad_fn=<MseLossBackward>) 540\n",
      "tensor(0.6970, device='cuda:0', grad_fn=<MseLossBackward>) 541\n",
      "tensor(0.8234, device='cuda:0', grad_fn=<MseLossBackward>) 542\n",
      "tensor(0.7080, device='cuda:0', grad_fn=<MseLossBackward>) 543\n",
      "tensor(0.6936, device='cuda:0', grad_fn=<MseLossBackward>) 544\n",
      "tensor(0.8228, device='cuda:0', grad_fn=<MseLossBackward>) 545\n",
      "tensor(0.7097, device='cuda:0', grad_fn=<MseLossBackward>) 546\n",
      "tensor(0.6917, device='cuda:0', grad_fn=<MseLossBackward>) 547\n",
      "tensor(0.8231, device='cuda:0', grad_fn=<MseLossBackward>) 548\n",
      "tensor(0.7135, device='cuda:0', grad_fn=<MseLossBackward>) 549\n",
      "tensor(0.6898, device='cuda:0', grad_fn=<MseLossBackward>) 550\n",
      "tensor(0.8201, device='cuda:0', grad_fn=<MseLossBackward>) 551\n",
      "tensor(0.7135, device='cuda:0', grad_fn=<MseLossBackward>) 552\n",
      "tensor(0.6911, device='cuda:0', grad_fn=<MseLossBackward>) 553\n",
      "tensor(0.8300, device='cuda:0', grad_fn=<MseLossBackward>) 554\n",
      "tensor(0.7241, device='cuda:0', grad_fn=<MseLossBackward>) 555\n",
      "tensor(0.6916, device='cuda:0', grad_fn=<MseLossBackward>) 556\n",
      "tensor(0.8280, device='cuda:0', grad_fn=<MseLossBackward>) 557\n",
      "tensor(0.7267, device='cuda:0', grad_fn=<MseLossBackward>) 558\n",
      "tensor(0.6886, device='cuda:0', grad_fn=<MseLossBackward>) 559\n",
      "tensor(0.8253, device='cuda:0', grad_fn=<MseLossBackward>) 560\n",
      "tensor(0.7285, device='cuda:0', grad_fn=<MseLossBackward>) 561\n",
      "tensor(0.6855, device='cuda:0', grad_fn=<MseLossBackward>) 562\n",
      "tensor(0.8236, device='cuda:0', grad_fn=<MseLossBackward>) 563\n",
      "tensor(0.7317, device='cuda:0', grad_fn=<MseLossBackward>) 564\n",
      "tensor(0.6818, device='cuda:0', grad_fn=<MseLossBackward>) 565\n",
      "tensor(0.8190, device='cuda:0', grad_fn=<MseLossBackward>) 566\n",
      "tensor(0.7337, device='cuda:0', grad_fn=<MseLossBackward>) 567\n",
      "tensor(0.6782, device='cuda:0', grad_fn=<MseLossBackward>) 568\n",
      "tensor(0.8168, device='cuda:0', grad_fn=<MseLossBackward>) 569\n",
      "tensor(0.7380, device='cuda:0', grad_fn=<MseLossBackward>) 570\n",
      "tensor(0.6748, device='cuda:0', grad_fn=<MseLossBackward>) 571\n",
      "tensor(0.8125, device='cuda:0', grad_fn=<MseLossBackward>) 572\n",
      "tensor(0.7420, device='cuda:0', grad_fn=<MseLossBackward>) 573\n",
      "tensor(0.6707, device='cuda:0', grad_fn=<MseLossBackward>) 574\n",
      "tensor(0.8071, device='cuda:0', grad_fn=<MseLossBackward>) 575\n",
      "tensor(0.7465, device='cuda:0', grad_fn=<MseLossBackward>) 576\n",
      "tensor(0.6675, device='cuda:0', grad_fn=<MseLossBackward>) 577\n",
      "tensor(0.8025, device='cuda:0', grad_fn=<MseLossBackward>) 578\n",
      "tensor(0.7522, device='cuda:0', grad_fn=<MseLossBackward>) 579\n",
      "tensor(0.6638, device='cuda:0', grad_fn=<MseLossBackward>) 580\n",
      "tensor(0.7954, device='cuda:0', grad_fn=<MseLossBackward>) 581\n",
      "tensor(0.7588, device='cuda:0', grad_fn=<MseLossBackward>) 582\n",
      "tensor(0.6613, device='cuda:0', grad_fn=<MseLossBackward>) 583\n",
      "tensor(0.7890, device='cuda:0', grad_fn=<MseLossBackward>) 584\n",
      "tensor(0.7614, device='cuda:0', grad_fn=<MseLossBackward>) 585\n",
      "tensor(0.6604, device='cuda:0', grad_fn=<MseLossBackward>) 586\n",
      "tensor(0.9063, device='cuda:0', grad_fn=<MseLossBackward>) 587\n",
      "tensor(0.9205, device='cuda:0', grad_fn=<MseLossBackward>) 588\n",
      "tensor(1.0823, device='cuda:0', grad_fn=<MseLossBackward>) 589\n",
      "tensor(0.7810, device='cuda:0', grad_fn=<MseLossBackward>) 590\n",
      "tensor(0.9904, device='cuda:0', grad_fn=<MseLossBackward>) 591\n",
      "tensor(1.4317, device='cuda:0', grad_fn=<MseLossBackward>) 592\n",
      "tensor(0.8710, device='cuda:0', grad_fn=<MseLossBackward>) 593\n",
      "tensor(1.1405, device='cuda:0', grad_fn=<MseLossBackward>) 594\n",
      "tensor(0.6980, device='cuda:0', grad_fn=<MseLossBackward>) 595\n",
      "tensor(1.4231, device='cuda:0', grad_fn=<MseLossBackward>) 596\n",
      "tensor(0.8679, device='cuda:0', grad_fn=<MseLossBackward>) 597\n",
      "tensor(1.0632, device='cuda:0', grad_fn=<MseLossBackward>) 598\n",
      "tensor(0.9241, device='cuda:0', grad_fn=<MseLossBackward>) 599\n",
      "tensor(1.1120, device='cuda:0', grad_fn=<MseLossBackward>) 600\n",
      "tensor(1.0377, device='cuda:0', grad_fn=<MseLossBackward>) 601\n",
      "tensor(0.7040, device='cuda:0', grad_fn=<MseLossBackward>) 602\n",
      "tensor(0.9177, device='cuda:0', grad_fn=<MseLossBackward>) 603\n",
      "tensor(0.9619, device='cuda:0', grad_fn=<MseLossBackward>) 604\n",
      "tensor(1.4143, device='cuda:0', grad_fn=<MseLossBackward>) 605\n",
      "tensor(1.7412, device='cuda:0', grad_fn=<MseLossBackward>) 606\n",
      "tensor(1.0363, device='cuda:0', grad_fn=<MseLossBackward>) 607\n",
      "tensor(0.9450, device='cuda:0', grad_fn=<MseLossBackward>) 608\n",
      "tensor(0.9490, device='cuda:0', grad_fn=<MseLossBackward>) 609\n",
      "tensor(0.9998, device='cuda:0', grad_fn=<MseLossBackward>) 610\n",
      "tensor(0.8544, device='cuda:0', grad_fn=<MseLossBackward>) 611\n",
      "tensor(0.8652, device='cuda:0', grad_fn=<MseLossBackward>) 612\n",
      "tensor(1.0613, device='cuda:0', grad_fn=<MseLossBackward>) 613\n",
      "tensor(0.7944, device='cuda:0', grad_fn=<MseLossBackward>) 614\n",
      "tensor(1.1112, device='cuda:0', grad_fn=<MseLossBackward>) 615\n",
      "tensor(0.8871, device='cuda:0', grad_fn=<MseLossBackward>) 616\n",
      "tensor(1.4494, device='cuda:0', grad_fn=<MseLossBackward>) 617\n",
      "tensor(1.3105, device='cuda:0', grad_fn=<MseLossBackward>) 618\n",
      "tensor(1.1794, device='cuda:0', grad_fn=<MseLossBackward>) 619\n",
      "tensor(1.0522, device='cuda:0', grad_fn=<MseLossBackward>) 620\n",
      "tensor(0.7689, device='cuda:0', grad_fn=<MseLossBackward>) 621\n",
      "tensor(0.6974, device='cuda:0', grad_fn=<MseLossBackward>) 622\n",
      "tensor(0.9096, device='cuda:0', grad_fn=<MseLossBackward>) 623\n",
      "tensor(0.7558, device='cuda:0', grad_fn=<MseLossBackward>) 624\n",
      "tensor(0.8683, device='cuda:0', grad_fn=<MseLossBackward>) 625\n",
      "tensor(0.8788, device='cuda:0', grad_fn=<MseLossBackward>) 626\n",
      "tensor(0.6938, device='cuda:0', grad_fn=<MseLossBackward>) 627\n",
      "tensor(0.9968, device='cuda:0', grad_fn=<MseLossBackward>) 628\n",
      "tensor(0.8946, device='cuda:0', grad_fn=<MseLossBackward>) 629\n",
      "tensor(1.0605, device='cuda:0', grad_fn=<MseLossBackward>) 630\n",
      "tensor(1.1100, device='cuda:0', grad_fn=<MseLossBackward>) 631\n",
      "tensor(0.7946, device='cuda:0', grad_fn=<MseLossBackward>) 632\n",
      "tensor(0.6159, device='cuda:0', grad_fn=<MseLossBackward>) 633\n",
      "tensor(0.6141, device='cuda:0', grad_fn=<MseLossBackward>) 634\n",
      "tensor(0.8500, device='cuda:0', grad_fn=<MseLossBackward>) 635\n",
      "tensor(0.8758, device='cuda:0', grad_fn=<MseLossBackward>) 636\n",
      "tensor(0.9536, device='cuda:0', grad_fn=<MseLossBackward>) 637\n",
      "tensor(0.8593, device='cuda:0', grad_fn=<MseLossBackward>) 638\n",
      "tensor(0.8228, device='cuda:0', grad_fn=<MseLossBackward>) 639\n",
      "tensor(1.0014, device='cuda:0', grad_fn=<MseLossBackward>) 640\n",
      "tensor(0.8109, device='cuda:0', grad_fn=<MseLossBackward>) 641\n",
      "tensor(0.6991, device='cuda:0', grad_fn=<MseLossBackward>) 642\n",
      "tensor(0.9079, device='cuda:0', grad_fn=<MseLossBackward>) 643\n",
      "tensor(0.8396, device='cuda:0', grad_fn=<MseLossBackward>) 644\n",
      "tensor(0.7029, device='cuda:0', grad_fn=<MseLossBackward>) 645\n",
      "tensor(0.8382, device='cuda:0', grad_fn=<MseLossBackward>) 646\n",
      "tensor(0.8332, device='cuda:0', grad_fn=<MseLossBackward>) 647\n",
      "tensor(0.7215, device='cuda:0', grad_fn=<MseLossBackward>) 648\n",
      "tensor(0.8597, device='cuda:0', grad_fn=<MseLossBackward>) 649\n",
      "tensor(0.8322, device='cuda:0', grad_fn=<MseLossBackward>) 650\n",
      "tensor(0.6946, device='cuda:0', grad_fn=<MseLossBackward>) 651\n",
      "tensor(0.8205, device='cuda:0', grad_fn=<MseLossBackward>) 652\n",
      "tensor(0.8462, device='cuda:0', grad_fn=<MseLossBackward>) 653\n",
      "tensor(0.7081, device='cuda:0', grad_fn=<MseLossBackward>) 654\n",
      "tensor(0.8169, device='cuda:0', grad_fn=<MseLossBackward>) 655\n",
      "tensor(0.8497, device='cuda:0', grad_fn=<MseLossBackward>) 656\n",
      "tensor(0.7095, device='cuda:0', grad_fn=<MseLossBackward>) 657\n",
      "tensor(0.8155, device='cuda:0', grad_fn=<MseLossBackward>) 658\n",
      "tensor(0.8642, device='cuda:0', grad_fn=<MseLossBackward>) 659\n",
      "tensor(0.7080, device='cuda:0', grad_fn=<MseLossBackward>) 660\n",
      "tensor(0.7928, device='cuda:0', grad_fn=<MseLossBackward>) 661\n",
      "tensor(0.8717, device='cuda:0', grad_fn=<MseLossBackward>) 662\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.7273, device='cuda:0', grad_fn=<MseLossBackward>) 663\n",
      "tensor(0.7968, device='cuda:0', grad_fn=<MseLossBackward>) 664\n",
      "tensor(0.8734, device='cuda:0', grad_fn=<MseLossBackward>) 665\n",
      "tensor(0.7214, device='cuda:0', grad_fn=<MseLossBackward>) 666\n",
      "tensor(0.7825, device='cuda:0', grad_fn=<MseLossBackward>) 667\n",
      "tensor(0.8864, device='cuda:0', grad_fn=<MseLossBackward>) 668\n",
      "tensor(0.7350, device='cuda:0', grad_fn=<MseLossBackward>) 669\n",
      "tensor(0.7748, device='cuda:0', grad_fn=<MseLossBackward>) 670\n",
      "tensor(0.8848, device='cuda:0', grad_fn=<MseLossBackward>) 671\n",
      "tensor(0.7427, device='cuda:0', grad_fn=<MseLossBackward>) 672\n",
      "tensor(0.7688, device='cuda:0', grad_fn=<MseLossBackward>) 673\n",
      "tensor(0.8870, device='cuda:0', grad_fn=<MseLossBackward>) 674\n",
      "tensor(0.7441, device='cuda:0', grad_fn=<MseLossBackward>) 675\n",
      "tensor(0.7563, device='cuda:0', grad_fn=<MseLossBackward>) 676\n",
      "tensor(0.8891, device='cuda:0', grad_fn=<MseLossBackward>) 677\n",
      "tensor(0.7568, device='cuda:0', grad_fn=<MseLossBackward>) 678\n",
      "tensor(0.7498, device='cuda:0', grad_fn=<MseLossBackward>) 679\n",
      "tensor(0.8806, device='cuda:0', grad_fn=<MseLossBackward>) 680\n",
      "tensor(0.7570, device='cuda:0', grad_fn=<MseLossBackward>) 681\n",
      "tensor(0.7388, device='cuda:0', grad_fn=<MseLossBackward>) 682\n",
      "tensor(0.8800, device='cuda:0', grad_fn=<MseLossBackward>) 683\n",
      "tensor(0.7637, device='cuda:0', grad_fn=<MseLossBackward>) 684\n",
      "tensor(0.7285, device='cuda:0', grad_fn=<MseLossBackward>) 685\n",
      "tensor(0.8701, device='cuda:0', grad_fn=<MseLossBackward>) 686\n",
      "tensor(0.7702, device='cuda:0', grad_fn=<MseLossBackward>) 687\n",
      "tensor(0.7202, device='cuda:0', grad_fn=<MseLossBackward>) 688\n",
      "tensor(0.8608, device='cuda:0', grad_fn=<MseLossBackward>) 689\n",
      "tensor(0.7731, device='cuda:0', grad_fn=<MseLossBackward>) 690\n",
      "tensor(0.7098, device='cuda:0', grad_fn=<MseLossBackward>) 691\n",
      "tensor(0.8525, device='cuda:0', grad_fn=<MseLossBackward>) 692\n",
      "tensor(0.7821, device='cuda:0', grad_fn=<MseLossBackward>) 693\n",
      "tensor(0.7008, device='cuda:0', grad_fn=<MseLossBackward>) 694\n",
      "tensor(0.8360, device='cuda:0', grad_fn=<MseLossBackward>) 695\n",
      "tensor(0.7885, device='cuda:0', grad_fn=<MseLossBackward>) 696\n",
      "tensor(0.6939, device='cuda:0', grad_fn=<MseLossBackward>) 697\n",
      "tensor(0.8240, device='cuda:0', grad_fn=<MseLossBackward>) 698\n",
      "tensor(0.7984, device='cuda:0', grad_fn=<MseLossBackward>) 699\n",
      "tensor(0.6885, device='cuda:0', grad_fn=<MseLossBackward>) 700\n",
      "tensor(0.8084, device='cuda:0', grad_fn=<MseLossBackward>) 701\n",
      "tensor(0.8108, device='cuda:0', grad_fn=<MseLossBackward>) 702\n",
      "tensor(0.6886, device='cuda:0', grad_fn=<MseLossBackward>) 703\n",
      "tensor(0.7949, device='cuda:0', grad_fn=<MseLossBackward>) 704\n",
      "tensor(0.8237, device='cuda:0', grad_fn=<MseLossBackward>) 705\n",
      "tensor(0.6929, device='cuda:0', grad_fn=<MseLossBackward>) 706\n",
      "tensor(0.7840, device='cuda:0', grad_fn=<MseLossBackward>) 707\n",
      "tensor(0.8381, device='cuda:0', grad_fn=<MseLossBackward>) 708\n",
      "tensor(0.7003, device='cuda:0', grad_fn=<MseLossBackward>) 709\n",
      "tensor(0.7721, device='cuda:0', grad_fn=<MseLossBackward>) 710\n",
      "tensor(0.8503, device='cuda:0', grad_fn=<MseLossBackward>) 711\n",
      "tensor(0.7120, device='cuda:0', grad_fn=<MseLossBackward>) 712\n",
      "tensor(0.7622, device='cuda:0', grad_fn=<MseLossBackward>) 713\n",
      "tensor(0.8588, device='cuda:0', grad_fn=<MseLossBackward>) 714\n",
      "tensor(0.7225, device='cuda:0', grad_fn=<MseLossBackward>) 715\n",
      "tensor(0.7503, device='cuda:0', grad_fn=<MseLossBackward>) 716\n",
      "tensor(0.8635, device='cuda:0', grad_fn=<MseLossBackward>) 717\n",
      "tensor(0.7339, device='cuda:0', grad_fn=<MseLossBackward>) 718\n",
      "tensor(0.7381, device='cuda:0', grad_fn=<MseLossBackward>) 719\n",
      "tensor(0.8615, device='cuda:0', grad_fn=<MseLossBackward>) 720\n",
      "tensor(0.7427, device='cuda:0', grad_fn=<MseLossBackward>) 721\n",
      "tensor(0.7244, device='cuda:0', grad_fn=<MseLossBackward>) 722\n",
      "tensor(0.8545, device='cuda:0', grad_fn=<MseLossBackward>) 723\n",
      "tensor(0.7493, device='cuda:0', grad_fn=<MseLossBackward>) 724\n",
      "tensor(0.7099, device='cuda:0', grad_fn=<MseLossBackward>) 725\n",
      "tensor(0.8433, device='cuda:0', grad_fn=<MseLossBackward>) 726\n",
      "tensor(0.7559, device='cuda:0', grad_fn=<MseLossBackward>) 727\n",
      "tensor(0.6951, device='cuda:0', grad_fn=<MseLossBackward>) 728\n",
      "tensor(0.8263, device='cuda:0', grad_fn=<MseLossBackward>) 729\n",
      "tensor(0.7605, device='cuda:0', grad_fn=<MseLossBackward>) 730\n",
      "tensor(0.6803, device='cuda:0', grad_fn=<MseLossBackward>) 731\n",
      "tensor(0.8064, device='cuda:0', grad_fn=<MseLossBackward>) 732\n",
      "tensor(0.7671, device='cuda:0', grad_fn=<MseLossBackward>) 733\n",
      "tensor(0.6681, device='cuda:0', grad_fn=<MseLossBackward>) 734\n",
      "tensor(0.7841, device='cuda:0', grad_fn=<MseLossBackward>) 735\n",
      "tensor(0.7764, device='cuda:0', grad_fn=<MseLossBackward>) 736\n",
      "tensor(0.6625, device='cuda:0', grad_fn=<MseLossBackward>) 737\n",
      "tensor(0.7643, device='cuda:0', grad_fn=<MseLossBackward>) 738\n",
      "tensor(0.7889, device='cuda:0', grad_fn=<MseLossBackward>) 739\n",
      "tensor(0.6648, device='cuda:0', grad_fn=<MseLossBackward>) 740\n",
      "tensor(0.7500, device='cuda:0', grad_fn=<MseLossBackward>) 741\n",
      "tensor(0.8052, device='cuda:0', grad_fn=<MseLossBackward>) 742\n",
      "tensor(0.6743, device='cuda:0', grad_fn=<MseLossBackward>) 743\n",
      "tensor(0.7365, device='cuda:0', grad_fn=<MseLossBackward>) 744\n",
      "tensor(0.8179, device='cuda:0', grad_fn=<MseLossBackward>) 745\n",
      "tensor(0.6868, device='cuda:0', grad_fn=<MseLossBackward>) 746\n",
      "tensor(0.7229, device='cuda:0', grad_fn=<MseLossBackward>) 747\n",
      "tensor(0.8248, device='cuda:0', grad_fn=<MseLossBackward>) 748\n",
      "tensor(0.6991, device='cuda:0', grad_fn=<MseLossBackward>) 749\n",
      "tensor(0.7080, device='cuda:0', grad_fn=<MseLossBackward>) 750\n",
      "tensor(0.8246, device='cuda:0', grad_fn=<MseLossBackward>) 751\n",
      "tensor(0.7101, device='cuda:0', grad_fn=<MseLossBackward>) 752\n",
      "tensor(0.6920, device='cuda:0', grad_fn=<MseLossBackward>) 753\n",
      "tensor(0.8168, device='cuda:0', grad_fn=<MseLossBackward>) 754\n",
      "tensor(0.7183, device='cuda:0', grad_fn=<MseLossBackward>) 755\n",
      "tensor(0.6748, device='cuda:0', grad_fn=<MseLossBackward>) 756\n",
      "tensor(0.8025, device='cuda:0', grad_fn=<MseLossBackward>) 757\n",
      "tensor(0.7253, device='cuda:0', grad_fn=<MseLossBackward>) 758\n",
      "tensor(0.6575, device='cuda:0', grad_fn=<MseLossBackward>) 759\n",
      "tensor(0.7817, device='cuda:0', grad_fn=<MseLossBackward>) 760\n",
      "tensor(0.7322, device='cuda:0', grad_fn=<MseLossBackward>) 761\n",
      "tensor(0.6423, device='cuda:0', grad_fn=<MseLossBackward>) 762\n",
      "tensor(0.7573, device='cuda:0', grad_fn=<MseLossBackward>) 763\n",
      "tensor(0.7415, device='cuda:0', grad_fn=<MseLossBackward>) 764\n",
      "tensor(0.6338, device='cuda:0', grad_fn=<MseLossBackward>) 765\n",
      "tensor(0.7340, device='cuda:0', grad_fn=<MseLossBackward>) 766\n",
      "tensor(0.7554, device='cuda:0', grad_fn=<MseLossBackward>) 767\n",
      "tensor(0.6355, device='cuda:0', grad_fn=<MseLossBackward>) 768\n",
      "tensor(0.7156, device='cuda:0', grad_fn=<MseLossBackward>) 769\n",
      "tensor(0.7713, device='cuda:0', grad_fn=<MseLossBackward>) 770\n",
      "tensor(0.6461, device='cuda:0', grad_fn=<MseLossBackward>) 771\n",
      "tensor(0.7010, device='cuda:0', grad_fn=<MseLossBackward>) 772\n",
      "tensor(0.7846, device='cuda:0', grad_fn=<MseLossBackward>) 773\n",
      "tensor(0.6604, device='cuda:0', grad_fn=<MseLossBackward>) 774\n",
      "tensor(0.6863, device='cuda:0', grad_fn=<MseLossBackward>) 775\n",
      "tensor(0.7909, device='cuda:0', grad_fn=<MseLossBackward>) 776\n",
      "tensor(0.6745, device='cuda:0', grad_fn=<MseLossBackward>) 777\n",
      "tensor(0.6699, device='cuda:0', grad_fn=<MseLossBackward>) 778\n",
      "tensor(0.7874, device='cuda:0', grad_fn=<MseLossBackward>) 779\n",
      "tensor(0.6857, device='cuda:0', grad_fn=<MseLossBackward>) 780\n",
      "tensor(0.6515, device='cuda:0', grad_fn=<MseLossBackward>) 781\n",
      "tensor(0.7744, device='cuda:0', grad_fn=<MseLossBackward>) 782\n",
      "tensor(0.6942, device='cuda:0', grad_fn=<MseLossBackward>) 783\n",
      "tensor(0.6324, device='cuda:0', grad_fn=<MseLossBackward>) 784\n",
      "tensor(0.7532, device='cuda:0', grad_fn=<MseLossBackward>) 785\n",
      "tensor(0.7024, device='cuda:0', grad_fn=<MseLossBackward>) 786\n",
      "tensor(0.6158, device='cuda:0', grad_fn=<MseLossBackward>) 787\n",
      "tensor(0.7272, device='cuda:0', grad_fn=<MseLossBackward>) 788\n",
      "tensor(0.7136, device='cuda:0', grad_fn=<MseLossBackward>) 789\n",
      "tensor(0.6075, device='cuda:0', grad_fn=<MseLossBackward>) 790\n",
      "tensor(0.7025, device='cuda:0', grad_fn=<MseLossBackward>) 791\n",
      "tensor(0.7295, device='cuda:0', grad_fn=<MseLossBackward>) 792\n",
      "tensor(0.6119, device='cuda:0', grad_fn=<MseLossBackward>) 793\n",
      "tensor(0.6840, device='cuda:0', grad_fn=<MseLossBackward>) 794\n",
      "tensor(0.7473, device='cuda:0', grad_fn=<MseLossBackward>) 795\n",
      "tensor(0.6260, device='cuda:0', grad_fn=<MseLossBackward>) 796\n",
      "tensor(0.6693, device='cuda:0', grad_fn=<MseLossBackward>) 797\n",
      "tensor(0.7601, device='cuda:0', grad_fn=<MseLossBackward>) 798\n",
      "tensor(0.6428, device='cuda:0', grad_fn=<MseLossBackward>) 799\n",
      "tensor(0.6539, device='cuda:0', grad_fn=<MseLossBackward>) 800\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.7634, device='cuda:0', grad_fn=<MseLossBackward>) 801\n",
      "tensor(0.6574, device='cuda:0', grad_fn=<MseLossBackward>) 802\n",
      "tensor(0.6354, device='cuda:0', grad_fn=<MseLossBackward>) 803\n",
      "tensor(0.7604, device='cuda:0', grad_fn=<MseLossBackward>) 804\n",
      "tensor(0.6760, device='cuda:0', grad_fn=<MseLossBackward>) 805\n",
      "tensor(0.6214, device='cuda:0', grad_fn=<MseLossBackward>) 806\n",
      "tensor(0.7424, device='cuda:0', grad_fn=<MseLossBackward>) 807\n",
      "tensor(0.6840, device='cuda:0', grad_fn=<MseLossBackward>) 808\n",
      "tensor(0.6006, device='cuda:0', grad_fn=<MseLossBackward>) 809\n",
      "tensor(0.7137, device='cuda:0', grad_fn=<MseLossBackward>) 810\n",
      "tensor(0.6965, device='cuda:0', grad_fn=<MseLossBackward>) 811\n",
      "tensor(0.5917, device='cuda:0', grad_fn=<MseLossBackward>) 812\n",
      "tensor(0.6876, device='cuda:0', grad_fn=<MseLossBackward>) 813\n",
      "tensor(0.7155, device='cuda:0', grad_fn=<MseLossBackward>) 814\n",
      "tensor(0.5982, device='cuda:0', grad_fn=<MseLossBackward>) 815\n",
      "tensor(0.6690, device='cuda:0', grad_fn=<MseLossBackward>) 816\n",
      "tensor(0.7361, device='cuda:0', grad_fn=<MseLossBackward>) 817\n",
      "tensor(0.6157, device='cuda:0', grad_fn=<MseLossBackward>) 818\n",
      "tensor(0.6545, device='cuda:0', grad_fn=<MseLossBackward>) 819\n",
      "tensor(0.7499, device='cuda:0', grad_fn=<MseLossBackward>) 820\n",
      "tensor(0.6356, device='cuda:0', grad_fn=<MseLossBackward>) 821\n",
      "tensor(0.6387, device='cuda:0', grad_fn=<MseLossBackward>) 822\n",
      "tensor(0.7509, device='cuda:0', grad_fn=<MseLossBackward>) 823\n",
      "tensor(0.6511, device='cuda:0', grad_fn=<MseLossBackward>) 824\n",
      "tensor(0.6186, device='cuda:0', grad_fn=<MseLossBackward>) 825\n",
      "tensor(0.7379, device='cuda:0', grad_fn=<MseLossBackward>) 826\n",
      "tensor(0.6634, device='cuda:0', grad_fn=<MseLossBackward>) 827\n",
      "tensor(0.5970, device='cuda:0', grad_fn=<MseLossBackward>) 828\n",
      "tensor(0.7118, device='cuda:0', grad_fn=<MseLossBackward>) 829\n",
      "tensor(0.6737, device='cuda:0', grad_fn=<MseLossBackward>) 830\n",
      "tensor(0.5796, device='cuda:0', grad_fn=<MseLossBackward>) 831\n",
      "tensor(0.6803, device='cuda:0', grad_fn=<MseLossBackward>) 832\n",
      "tensor(0.6896, device='cuda:0', grad_fn=<MseLossBackward>) 833\n",
      "tensor(0.5783, device='cuda:0', grad_fn=<MseLossBackward>) 834\n",
      "tensor(0.6554, device='cuda:0', grad_fn=<MseLossBackward>) 835\n",
      "tensor(0.7101, device='cuda:0', grad_fn=<MseLossBackward>) 836\n",
      "tensor(0.5935, device='cuda:0', grad_fn=<MseLossBackward>) 837\n",
      "tensor(0.6385, device='cuda:0', grad_fn=<MseLossBackward>) 838\n",
      "tensor(0.7264, device='cuda:0', grad_fn=<MseLossBackward>) 839\n",
      "tensor(0.6146, device='cuda:0', grad_fn=<MseLossBackward>) 840\n",
      "tensor(0.6222, device='cuda:0', grad_fn=<MseLossBackward>) 841\n",
      "tensor(0.7309, device='cuda:0', grad_fn=<MseLossBackward>) 842\n",
      "tensor(0.6337, device='cuda:0', grad_fn=<MseLossBackward>) 843\n",
      "tensor(0.6025, device='cuda:0', grad_fn=<MseLossBackward>) 844\n",
      "tensor(0.7181, device='cuda:0', grad_fn=<MseLossBackward>) 845\n",
      "tensor(0.6469, device='cuda:0', grad_fn=<MseLossBackward>) 846\n",
      "tensor(0.5803, device='cuda:0', grad_fn=<MseLossBackward>) 847\n",
      "tensor(0.6915, device='cuda:0', grad_fn=<MseLossBackward>) 848\n",
      "tensor(0.6590, device='cuda:0', grad_fn=<MseLossBackward>) 849\n",
      "tensor(0.5639, device='cuda:0', grad_fn=<MseLossBackward>) 850\n",
      "tensor(0.6593, device='cuda:0', grad_fn=<MseLossBackward>) 851\n",
      "tensor(0.6767, device='cuda:0', grad_fn=<MseLossBackward>) 852\n",
      "tensor(0.5658, device='cuda:0', grad_fn=<MseLossBackward>) 853\n",
      "tensor(0.6349, device='cuda:0', grad_fn=<MseLossBackward>) 854\n",
      "tensor(0.6979, device='cuda:0', grad_fn=<MseLossBackward>) 855\n",
      "tensor(0.5845, device='cuda:0', grad_fn=<MseLossBackward>) 856\n",
      "tensor(0.6184, device='cuda:0', grad_fn=<MseLossBackward>) 857\n",
      "tensor(0.7126, device='cuda:0', grad_fn=<MseLossBackward>) 858\n",
      "tensor(0.6074, device='cuda:0', grad_fn=<MseLossBackward>) 859\n",
      "tensor(0.6009, device='cuda:0', grad_fn=<MseLossBackward>) 860\n",
      "tensor(0.7118, device='cuda:0', grad_fn=<MseLossBackward>) 861\n",
      "tensor(0.6258, device='cuda:0', grad_fn=<MseLossBackward>) 862\n",
      "tensor(0.5791, device='cuda:0', grad_fn=<MseLossBackward>) 863\n",
      "tensor(0.6917, device='cuda:0', grad_fn=<MseLossBackward>) 864\n",
      "tensor(0.6386, device='cuda:0', grad_fn=<MseLossBackward>) 865\n",
      "tensor(0.5576, device='cuda:0', grad_fn=<MseLossBackward>) 866\n",
      "tensor(0.6595, device='cuda:0', grad_fn=<MseLossBackward>) 867\n",
      "tensor(0.6539, device='cuda:0', grad_fn=<MseLossBackward>) 868\n",
      "tensor(0.5498, device='cuda:0', grad_fn=<MseLossBackward>) 869\n",
      "tensor(0.6292, device='cuda:0', grad_fn=<MseLossBackward>) 870\n",
      "tensor(0.6757, device='cuda:0', grad_fn=<MseLossBackward>) 871\n",
      "tensor(0.5640, device='cuda:0', grad_fn=<MseLossBackward>) 872\n",
      "tensor(0.6104, device='cuda:0', grad_fn=<MseLossBackward>) 873\n",
      "tensor(0.6951, device='cuda:0', grad_fn=<MseLossBackward>) 874\n",
      "tensor(0.5883, device='cuda:0', grad_fn=<MseLossBackward>) 875\n",
      "tensor(0.5931, device='cuda:0', grad_fn=<MseLossBackward>) 876\n",
      "tensor(0.6996, device='cuda:0', grad_fn=<MseLossBackward>) 877\n",
      "tensor(0.6098, device='cuda:0', grad_fn=<MseLossBackward>) 878\n",
      "tensor(0.5727, device='cuda:0', grad_fn=<MseLossBackward>) 879\n",
      "tensor(0.6845, device='cuda:0', grad_fn=<MseLossBackward>) 880\n",
      "tensor(0.6251, device='cuda:0', grad_fn=<MseLossBackward>) 881\n",
      "tensor(0.5500, device='cuda:0', grad_fn=<MseLossBackward>) 882\n",
      "tensor(0.6524, device='cuda:0', grad_fn=<MseLossBackward>) 883\n",
      "tensor(0.6399, device='cuda:0', grad_fn=<MseLossBackward>) 884\n",
      "tensor(0.5387, device='cuda:0', grad_fn=<MseLossBackward>) 885\n",
      "tensor(0.6193, device='cuda:0', grad_fn=<MseLossBackward>) 886\n",
      "tensor(0.6616, device='cuda:0', grad_fn=<MseLossBackward>) 887\n",
      "tensor(0.5517, device='cuda:0', grad_fn=<MseLossBackward>) 888\n",
      "tensor(0.5990, device='cuda:0', grad_fn=<MseLossBackward>) 889\n",
      "tensor(0.6825, device='cuda:0', grad_fn=<MseLossBackward>) 890\n",
      "tensor(0.5773, device='cuda:0', grad_fn=<MseLossBackward>) 891\n",
      "tensor(0.5815, device='cuda:0', grad_fn=<MseLossBackward>) 892\n",
      "tensor(0.6877, device='cuda:0', grad_fn=<MseLossBackward>) 893\n",
      "tensor(0.6007, device='cuda:0', grad_fn=<MseLossBackward>) 894\n",
      "tensor(0.5605, device='cuda:0', grad_fn=<MseLossBackward>) 895\n",
      "tensor(0.6708, device='cuda:0', grad_fn=<MseLossBackward>) 896\n",
      "tensor(0.6170, device='cuda:0', grad_fn=<MseLossBackward>) 897\n",
      "tensor(0.5384, device='cuda:0', grad_fn=<MseLossBackward>) 898\n",
      "tensor(0.6370, device='cuda:0', grad_fn=<MseLossBackward>) 899\n",
      "tensor(0.6344, device='cuda:0', grad_fn=<MseLossBackward>) 900\n",
      "tensor(0.5314, device='cuda:0', grad_fn=<MseLossBackward>) 901\n",
      "tensor(0.6053, device='cuda:0', grad_fn=<MseLossBackward>) 902\n",
      "tensor(0.6587, device='cuda:0', grad_fn=<MseLossBackward>) 903\n",
      "tensor(0.5504, device='cuda:0', grad_fn=<MseLossBackward>) 904\n",
      "tensor(0.5868, device='cuda:0', grad_fn=<MseLossBackward>) 905\n",
      "tensor(0.6780, device='cuda:0', grad_fn=<MseLossBackward>) 906\n",
      "tensor(0.5781, device='cuda:0', grad_fn=<MseLossBackward>) 907\n",
      "tensor(0.5682, device='cuda:0', grad_fn=<MseLossBackward>) 908\n",
      "tensor(0.6771, device='cuda:0', grad_fn=<MseLossBackward>) 909\n",
      "tensor(0.6007, device='cuda:0', grad_fn=<MseLossBackward>) 910\n",
      "tensor(0.5460, device='cuda:0', grad_fn=<MseLossBackward>) 911\n",
      "tensor(0.6526, device='cuda:0', grad_fn=<MseLossBackward>) 912\n",
      "tensor(0.6172, device='cuda:0', grad_fn=<MseLossBackward>) 913\n",
      "tensor(0.5269, device='cuda:0', grad_fn=<MseLossBackward>) 914\n",
      "tensor(0.6156, device='cuda:0', grad_fn=<MseLossBackward>) 915\n",
      "tensor(0.6387, device='cuda:0', grad_fn=<MseLossBackward>) 916\n",
      "tensor(0.5320, device='cuda:0', grad_fn=<MseLossBackward>) 917\n",
      "tensor(0.5895, device='cuda:0', grad_fn=<MseLossBackward>) 918\n",
      "tensor(0.6640, device='cuda:0', grad_fn=<MseLossBackward>) 919\n",
      "tensor(0.5595, device='cuda:0', grad_fn=<MseLossBackward>) 920\n",
      "tensor(0.5725, device='cuda:0', grad_fn=<MseLossBackward>) 921\n",
      "tensor(0.6756, device='cuda:0', grad_fn=<MseLossBackward>) 922\n",
      "tensor(0.5873, device='cuda:0', grad_fn=<MseLossBackward>) 923\n",
      "tensor(0.5514, device='cuda:0', grad_fn=<MseLossBackward>) 924\n",
      "tensor(0.6609, device='cuda:0', grad_fn=<MseLossBackward>) 925\n",
      "tensor(0.6068, device='cuda:0', grad_fn=<MseLossBackward>) 926\n",
      "tensor(0.5278, device='cuda:0', grad_fn=<MseLossBackward>) 927\n",
      "tensor(0.6265, device='cuda:0', grad_fn=<MseLossBackward>) 928\n",
      "tensor(0.6296, device='cuda:0', grad_fn=<MseLossBackward>) 929\n",
      "tensor(0.5254, device='cuda:0', grad_fn=<MseLossBackward>) 930\n",
      "tensor(0.5938, device='cuda:0', grad_fn=<MseLossBackward>) 931\n",
      "tensor(0.6526, device='cuda:0', grad_fn=<MseLossBackward>) 932\n",
      "tensor(0.5455, device='cuda:0', grad_fn=<MseLossBackward>) 933\n",
      "tensor(0.5717, device='cuda:0', grad_fn=<MseLossBackward>) 934\n",
      "tensor(0.6692, device='cuda:0', grad_fn=<MseLossBackward>) 935\n",
      "tensor(0.5766, device='cuda:0', grad_fn=<MseLossBackward>) 936\n",
      "tensor(0.5533, device='cuda:0', grad_fn=<MseLossBackward>) 937\n",
      "tensor(0.6635, device='cuda:0', grad_fn=<MseLossBackward>) 938\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.6007, device='cuda:0', grad_fn=<MseLossBackward>) 939\n",
      "tensor(0.5301, device='cuda:0', grad_fn=<MseLossBackward>) 940\n",
      "tensor(0.6300, device='cuda:0', grad_fn=<MseLossBackward>) 941\n",
      "tensor(0.6202, device='cuda:0', grad_fn=<MseLossBackward>) 942\n",
      "tensor(0.5192, device='cuda:0', grad_fn=<MseLossBackward>) 943\n",
      "tensor(0.5925, device='cuda:0', grad_fn=<MseLossBackward>) 944\n",
      "tensor(0.6464, device='cuda:0', grad_fn=<MseLossBackward>) 945\n",
      "tensor(0.5407, device='cuda:0', grad_fn=<MseLossBackward>) 946\n",
      "tensor(0.5717, device='cuda:0', grad_fn=<MseLossBackward>) 947\n",
      "tensor(0.6658, device='cuda:0', grad_fn=<MseLossBackward>) 948\n",
      "tensor(0.5720, device='cuda:0', grad_fn=<MseLossBackward>) 949\n",
      "tensor(0.5511, device='cuda:0', grad_fn=<MseLossBackward>) 950\n",
      "tensor(0.6595, device='cuda:0', grad_fn=<MseLossBackward>) 951\n",
      "tensor(0.5965, device='cuda:0', grad_fn=<MseLossBackward>) 952\n",
      "tensor(0.5273, device='cuda:0', grad_fn=<MseLossBackward>) 953\n",
      "tensor(0.6256, device='cuda:0', grad_fn=<MseLossBackward>) 954\n",
      "tensor(0.6165, device='cuda:0', grad_fn=<MseLossBackward>) 955\n",
      "tensor(0.5162, device='cuda:0', grad_fn=<MseLossBackward>) 956\n",
      "tensor(0.5873, device='cuda:0', grad_fn=<MseLossBackward>) 957\n",
      "tensor(0.6433, device='cuda:0', grad_fn=<MseLossBackward>) 958\n",
      "tensor(0.5394, device='cuda:0', grad_fn=<MseLossBackward>) 959\n",
      "tensor(0.5663, device='cuda:0', grad_fn=<MseLossBackward>) 960\n",
      "tensor(0.6615, device='cuda:0', grad_fn=<MseLossBackward>) 961\n",
      "tensor(0.5715, device='cuda:0', grad_fn=<MseLossBackward>) 962\n",
      "tensor(0.5456, device='cuda:0', grad_fn=<MseLossBackward>) 963\n",
      "tensor(0.6530, device='cuda:0', grad_fn=<MseLossBackward>) 964\n",
      "tensor(0.5957, device='cuda:0', grad_fn=<MseLossBackward>) 965\n",
      "tensor(0.5222, device='cuda:0', grad_fn=<MseLossBackward>) 966\n",
      "tensor(0.6261, device='cuda:0', grad_fn=<MseLossBackward>) 967\n",
      "tensor(0.6299, device='cuda:0', grad_fn=<MseLossBackward>) 968\n",
      "tensor(0.5241, device='cuda:0', grad_fn=<MseLossBackward>) 969\n",
      "tensor(0.5882, device='cuda:0', grad_fn=<MseLossBackward>) 970\n",
      "tensor(0.6531, device='cuda:0', grad_fn=<MseLossBackward>) 971\n",
      "tensor(0.5487, device='cuda:0', grad_fn=<MseLossBackward>) 972\n",
      "tensor(0.5647, device='cuda:0', grad_fn=<MseLossBackward>) 973\n",
      "tensor(0.6657, device='cuda:0', grad_fn=<MseLossBackward>) 974\n",
      "tensor(0.5803, device='cuda:0', grad_fn=<MseLossBackward>) 975\n",
      "tensor(0.5407, device='cuda:0', grad_fn=<MseLossBackward>) 976\n",
      "tensor(0.6472, device='cuda:0', grad_fn=<MseLossBackward>) 977\n",
      "tensor(0.6047, device='cuda:0', grad_fn=<MseLossBackward>) 978\n",
      "tensor(0.5184, device='cuda:0', grad_fn=<MseLossBackward>) 979\n",
      "tensor(0.6043, device='cuda:0', grad_fn=<MseLossBackward>) 980\n",
      "tensor(0.6284, device='cuda:0', grad_fn=<MseLossBackward>) 981\n",
      "tensor(0.5240, device='cuda:0', grad_fn=<MseLossBackward>) 982\n",
      "tensor(0.5719, device='cuda:0', grad_fn=<MseLossBackward>) 983\n",
      "tensor(0.6535, device='cuda:0', grad_fn=<MseLossBackward>) 984\n",
      "tensor(0.5581, device='cuda:0', grad_fn=<MseLossBackward>) 985\n",
      "tensor(0.5522, device='cuda:0', grad_fn=<MseLossBackward>) 986\n",
      "tensor(0.6564, device='cuda:0', grad_fn=<MseLossBackward>) 987\n",
      "tensor(0.5881, device='cuda:0', grad_fn=<MseLossBackward>) 988\n",
      "tensor(0.5277, device='cuda:0', grad_fn=<MseLossBackward>) 989\n",
      "tensor(0.6257, device='cuda:0', grad_fn=<MseLossBackward>) 990\n",
      "tensor(0.6109, device='cuda:0', grad_fn=<MseLossBackward>) 991\n",
      "tensor(0.5130, device='cuda:0', grad_fn=<MseLossBackward>) 992\n",
      "tensor(0.5815, device='cuda:0', grad_fn=<MseLossBackward>) 993\n",
      "tensor(0.6373, device='cuda:0', grad_fn=<MseLossBackward>) 994\n",
      "tensor(0.5376, device='cuda:0', grad_fn=<MseLossBackward>) 995\n",
      "tensor(0.5604, device='cuda:0', grad_fn=<MseLossBackward>) 996\n",
      "tensor(0.6561, device='cuda:0', grad_fn=<MseLossBackward>) 997\n",
      "tensor(0.5728, device='cuda:0', grad_fn=<MseLossBackward>) 998\n",
      "tensor(0.5370, device='cuda:0', grad_fn=<MseLossBackward>) 999\n"
     ]
    }
   ],
   "source": [
    "#trainingloop\n",
    "\n",
    "import torch.optim as optim\n",
    "\n",
    "net1=Net()\n",
    "\n",
    "lr = 0.001\n",
    "batchsize=20\n",
    "\n",
    "batches=len(projectiontensor)/batchsize\n",
    "\n",
    "epochs=1000\n",
    "\n",
    "# create your optimizer\n",
    "optimizer = optim.Adam(net1.parameters(), lr)\n",
    "\n",
    "\n",
    "for i in range(epochs):\n",
    "    \n",
    "    for j in range(int(batches)):\n",
    "        \n",
    "        #forward pass\n",
    "        out=net1(projectiontensor[j:j+batchsize,:].type(dtype))\n",
    "\n",
    "        #compute loss\n",
    "        criterion = nn.MSELoss()\n",
    "        loss = criterion(out,results1tensor[j:j+batchsize]).type(dtype)\n",
    "\n",
    "\n",
    "        #backprop loss i.e. find dloss/dparam for each parameter and store.\n",
    "        loss.backward(retain_graph=True)\n",
    "\n",
    "        #clip gradients\n",
    "        torch.nn.utils.clip_grad_norm_(net1.parameters(), 10.0)\n",
    "        \n",
    "        #use optimiser to update\n",
    "        optimizer.step()\n",
    "    c=nn.MSELoss()\n",
    "    print(c(torch.reshape(net1(projectiontensor),[200]),results1tensor),i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[2.1695],\n",
      "        [2.0072],\n",
      "        [2.1011],\n",
      "        [1.9959],\n",
      "        [2.2116],\n",
      "        [2.0176],\n",
      "        [2.0215],\n",
      "        [2.2107],\n",
      "        [2.0146],\n",
      "        [2.1682],\n",
      "        [2.1837],\n",
      "        [2.2179],\n",
      "        [2.0253],\n",
      "        [2.0163],\n",
      "        [2.1467],\n",
      "        [2.0081],\n",
      "        [2.1118],\n",
      "        [2.2233],\n",
      "        [2.0302],\n",
      "        [2.2133],\n",
      "        [2.1381],\n",
      "        [2.0445],\n",
      "        [2.2174],\n",
      "        [2.1869],\n",
      "        [2.0158],\n",
      "        [2.2267],\n",
      "        [2.0471],\n",
      "        [2.1156],\n",
      "        [2.0044],\n",
      "        [1.4764],\n",
      "        [1.7115],\n",
      "        [1.2514],\n",
      "        [1.0006],\n",
      "        [1.5163],\n",
      "        [1.5072],\n",
      "        [0.8886],\n",
      "        [1.1226],\n",
      "        [1.2752],\n",
      "        [1.8468],\n",
      "        [0.9757],\n",
      "        [0.8897],\n",
      "        [1.9588],\n",
      "        [0.9279],\n",
      "        [1.2430],\n",
      "        [1.8887],\n",
      "        [1.1929],\n",
      "        [1.4623],\n",
      "        [1.4297],\n",
      "        [1.4935],\n",
      "        [1.7694],\n",
      "        [1.5851],\n",
      "        [1.1844],\n",
      "        [1.8678],\n",
      "        [1.8757],\n",
      "        [1.6009],\n",
      "        [2.3161],\n",
      "        [1.0575],\n",
      "        [1.9003],\n",
      "        [1.3151],\n",
      "        [1.5922],\n",
      "        [1.3366],\n",
      "        [1.5877],\n",
      "        [1.8975],\n",
      "        [1.4315],\n",
      "        [2.6238],\n",
      "        [1.5335],\n",
      "        [2.0026],\n",
      "        [1.2779],\n",
      "        [2.1052],\n",
      "        [2.8071],\n",
      "        [1.6231],\n",
      "        [0.9799],\n",
      "        [1.3616],\n",
      "        [1.8914],\n",
      "        [1.4490],\n",
      "        [2.6419],\n",
      "        [1.3677],\n",
      "        [1.3099],\n",
      "        [1.0370],\n",
      "        [1.0012],\n",
      "        [1.0417],\n",
      "        [0.9444],\n",
      "        [1.1082],\n",
      "        [1.1261],\n",
      "        [1.9583],\n",
      "        [1.2181],\n",
      "        [1.3351],\n",
      "        [1.5374],\n",
      "        [1.9620],\n",
      "        [1.6189],\n",
      "        [2.0770],\n",
      "        [1.1008],\n",
      "        [1.6192],\n",
      "        [1.6306],\n",
      "        [1.0343],\n",
      "        [1.3728],\n",
      "        [1.3521],\n",
      "        [2.6338],\n",
      "        [1.6372],\n",
      "        [1.6935],\n",
      "        [1.7124],\n",
      "        [1.8685],\n",
      "        [1.4968],\n",
      "        [1.5876],\n",
      "        [0.9693],\n",
      "        [1.7192],\n",
      "        [1.2854],\n",
      "        [1.6739],\n",
      "        [1.9576],\n",
      "        [1.7553],\n",
      "        [2.1428],\n",
      "        [1.9599],\n",
      "        [0.9565],\n",
      "        [2.0138],\n",
      "        [0.9507],\n",
      "        [0.9556],\n",
      "        [1.6474],\n",
      "        [1.3614],\n",
      "        [2.4546],\n",
      "        [0.9666],\n",
      "        [0.9701],\n",
      "        [1.9271],\n",
      "        [1.2552],\n",
      "        [1.5321],\n",
      "        [1.1178],\n",
      "        [1.8086],\n",
      "        [1.5778],\n",
      "        [1.4047],\n",
      "        [1.9746],\n",
      "        [1.2510],\n",
      "        [1.9782],\n",
      "        [1.8419],\n",
      "        [2.4324],\n",
      "        [1.4455],\n",
      "        [1.5815],\n",
      "        [1.7356],\n",
      "        [1.6929],\n",
      "        [0.8510],\n",
      "        [1.0828],\n",
      "        [1.1363],\n",
      "        [1.5510],\n",
      "        [1.0585],\n",
      "        [0.9049],\n",
      "        [1.4830],\n",
      "        [2.0003],\n",
      "        [0.9237],\n",
      "        [1.0004],\n",
      "        [1.4549],\n",
      "        [0.8678],\n",
      "        [1.4111],\n",
      "        [1.6967],\n",
      "        [1.1448],\n",
      "        [2.1219],\n",
      "        [1.6255],\n",
      "        [1.9374],\n",
      "        [1.3020],\n",
      "        [2.2967],\n",
      "        [1.3392],\n",
      "        [1.0065],\n",
      "        [1.1026],\n",
      "        [1.2186],\n",
      "        [2.3866],\n",
      "        [1.5887],\n",
      "        [1.5922],\n",
      "        [2.0195],\n",
      "        [1.2749],\n",
      "        [1.5565],\n",
      "        [1.0893],\n",
      "        [1.2727],\n",
      "        [2.0122],\n",
      "        [1.9117],\n",
      "        [1.8355],\n",
      "        [1.3366],\n",
      "        [1.7148],\n",
      "        [1.2614],\n",
      "        [0.9596],\n",
      "        [1.2262],\n",
      "        [1.3620],\n",
      "        [1.0369],\n",
      "        [1.1436],\n",
      "        [1.5011],\n",
      "        [1.7748],\n",
      "        [1.1537],\n",
      "        [1.9558],\n",
      "        [2.0356],\n",
      "        [1.1425],\n",
      "        [1.2652],\n",
      "        [1.3957],\n",
      "        [1.2508],\n",
      "        [1.4451],\n",
      "        [1.7095],\n",
      "        [1.6882],\n",
      "        [1.1524],\n",
      "        [2.3535],\n",
      "        [1.7248],\n",
      "        [1.0999],\n",
      "        [1.0826],\n",
      "        [2.1740],\n",
      "        [1.5535],\n",
      "        [1.2018]], device='cuda:0', grad_fn=<AddmmBackward>)\n"
     ]
    }
   ],
   "source": [
    "print(net1(projectiontensor).type(dtype))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1.8908],\n",
      "        [1.8004],\n",
      "        [1.3634],\n",
      "        [1.2090],\n",
      "        [1.3241],\n",
      "        [1.5034],\n",
      "        [1.1810],\n",
      "        [2.2858],\n",
      "        [1.6984],\n",
      "        [1.7172],\n",
      "        [1.4761],\n",
      "        [1.0720],\n",
      "        [1.2300],\n",
      "        [1.3050],\n",
      "        [2.3853],\n",
      "        [1.3023],\n",
      "        [1.1797],\n",
      "        [1.7256],\n",
      "        [1.2844],\n",
      "        [1.7006],\n",
      "        [0.9631],\n",
      "        [1.1747],\n",
      "        [1.4469],\n",
      "        [0.8785],\n",
      "        [1.5071],\n",
      "        [1.7533],\n",
      "        [1.8542],\n",
      "        [1.6746],\n",
      "        [2.4079],\n",
      "        [0.8707]], device='cuda:0', grad_fn=<AddmmBackward>)\n"
     ]
    }
   ],
   "source": [
    "#Map the test data also into the smaller dimension of random_projection_layer using the same \n",
    "#generated Achliopta's  random projection matrix\n",
    "\n",
    "projectedtstdt=np.dot(tstdt.values,randmatr)\n",
    "\n",
    "#Normalise the projected data, not batch normalisation because, we are not fine tuning random projection layer weights\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "scaler=StandardScaler()\n",
    "normalisedprojectiontest=scaler.fit_transform(projectedtstdt)\n",
    "\n",
    "projectiontesttensor=torch.from_numpy(normalisedprojectiontest).type(dtype)\n",
    "print(net1(projectiontesttensor).type(dtype))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.5250404650424876\n"
     ]
    }
   ],
   "source": [
    "#Evaluation\n",
    "\n",
    "import joblib\n",
    "#Gold Standard\n",
    "m=joblib.load('C:\\\\Users\\\\Ant Pc\\\\GitHub\\\\pROJ\\\\Geneexpression0\\\\12.pkl')\n",
    "\n",
    "pred=pd.DataFrame(m.predict(tstdt))\n",
    "\n",
    "from sklearn.metrics import mean_squared_error\n",
    "print(mean_squared_error(pred,net1(projectiontesttensor).cpu().detach().numpy()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can already see MSE became almost half of the previous MSE, even with just 320 hidden neurons as opposed to 3200 neurons in the previous conventional regression neural network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fine Tuned Random Projection Neural Network"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![5](assets/5.jpeg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here Random matrix is taken as weights between input layer and next layer, and only non-zero weights are updated(to save computational power).One more difference is batch normalisation is used after the random projection instead of whole data normalisation as was done in the fixed weights case, because random projection matrix is changing on the go."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# first fix the layer dimension after input layer\n",
    "import random\n",
    "import math\n",
    "\n",
    "input_dim=28395\n",
    "random_projection_layer_dim= 1000\n",
    "\n",
    "def generatelisrandommatrix(input_dim,random_projection_layer_dim):\n",
    "    \n",
    "    size_0=input_dim\n",
    "    size_1=random_projection_layer_dim\n",
    "    \n",
    "    matr=np.zeros([size_0,size_1])\n",
    "    \n",
    "    for i in range(size_0):\n",
    "        m=[]\n",
    "        for j in range(size_1):\n",
    "            x=random.uniform(0,1)\n",
    "            if x<(1/6):\n",
    "                matr[i,j]=(1*math.sqrt((3/size_1)))\n",
    "            \n",
    "            if x>(1/6):\n",
    "                if x<(1-1/6):\n",
    "                    matr[i,j]=(0)\n",
    "                \n",
    "            if x>((1)-(1/6)):\n",
    "                matr[i,j]=(-1*math.sqrt((3/size_1)))\n",
    "    \n",
    "    return matr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "randmatr=generatelisrandommatrix(28395,1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Net(\n",
      "  (rand): Linear(in_features=28395, out_features=1000, bias=True)\n",
      "  (batchnorm): BatchNorm1d(1000, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
      "  (fc1): Linear(in_features=1000, out_features=320, bias=True)\n",
      "  (fc6): Linear(in_features=320, out_features=1, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "class Net(nn.Module):\n",
    "\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        \n",
    "        # an affine operation: y = Wx + b\n",
    "        \n",
    "        self.rand=nn.Linear(28395,1000).cuda()\n",
    "        self.batchnorm=nn.BatchNorm1d(1000).cuda()\n",
    "        self.fc1=nn.Linear(1000,320).cuda()\n",
    "        self.fc6 = nn.Linear(320,1).cuda()\n",
    "        \n",
    "        #set weights as random projection matrix\n",
    "        self.rand.weight.data = torch.transpose(torch.from_numpy(randmatr),0,1).type(dtype)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = self.rand(x).cuda()\n",
    "        x = self.batchnorm(x).cuda()\n",
    "        x = F.relu(x).cuda()\n",
    "        x = F.relu(self.fc1(x)).cuda()        \n",
    "        x = self.fc6(x).cuda()\n",
    "        return x\n",
    "\n",
    "net = Net()\n",
    "\n",
    "#use gpu for all computations in model\n",
    "net.cuda()\n",
    "print(net)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\torch\\nn\\modules\\loss.py:432: UserWarning: Using a target size (torch.Size([20])) that is different to the input size (torch.Size([20, 1])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.\n",
      "  return F.mse_loss(input, target, reduction=self.reduction)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(1.2223, device='cuda:0', grad_fn=<MseLossBackward>) 0\n",
      "tensor(2.3697, device='cuda:0', grad_fn=<MseLossBackward>) 1\n",
      "tensor(1.0614, device='cuda:0', grad_fn=<MseLossBackward>) 2\n",
      "tensor(1.2771, device='cuda:0', grad_fn=<MseLossBackward>) 3\n",
      "tensor(0.9971, device='cuda:0', grad_fn=<MseLossBackward>) 4\n",
      "tensor(0.8913, device='cuda:0', grad_fn=<MseLossBackward>) 5\n",
      "tensor(1.1316, device='cuda:0', grad_fn=<MseLossBackward>) 6\n",
      "tensor(0.8832, device='cuda:0', grad_fn=<MseLossBackward>) 7\n",
      "tensor(0.8639, device='cuda:0', grad_fn=<MseLossBackward>) 8\n",
      "tensor(1.4735, device='cuda:0', grad_fn=<MseLossBackward>) 9\n",
      "tensor(0.9262, device='cuda:0', grad_fn=<MseLossBackward>) 10\n",
      "tensor(1.0696, device='cuda:0', grad_fn=<MseLossBackward>) 11\n",
      "tensor(1.0549, device='cuda:0', grad_fn=<MseLossBackward>) 12\n",
      "tensor(1.0780, device='cuda:0', grad_fn=<MseLossBackward>) 13\n",
      "tensor(1.0891, device='cuda:0', grad_fn=<MseLossBackward>) 14\n",
      "tensor(0.8573, device='cuda:0', grad_fn=<MseLossBackward>) 15\n",
      "tensor(0.8681, device='cuda:0', grad_fn=<MseLossBackward>) 16\n",
      "tensor(0.8634, device='cuda:0', grad_fn=<MseLossBackward>) 17\n",
      "tensor(0.9618, device='cuda:0', grad_fn=<MseLossBackward>) 18\n",
      "tensor(0.8296, device='cuda:0', grad_fn=<MseLossBackward>) 19\n",
      "tensor(0.8782, device='cuda:0', grad_fn=<MseLossBackward>) 20\n",
      "tensor(0.8471, device='cuda:0', grad_fn=<MseLossBackward>) 21\n",
      "tensor(0.8204, device='cuda:0', grad_fn=<MseLossBackward>) 22\n",
      "tensor(0.9648, device='cuda:0', grad_fn=<MseLossBackward>) 23\n",
      "tensor(0.8433, device='cuda:0', grad_fn=<MseLossBackward>) 24\n",
      "tensor(0.8121, device='cuda:0', grad_fn=<MseLossBackward>) 25\n",
      "tensor(0.8686, device='cuda:0', grad_fn=<MseLossBackward>) 26\n",
      "tensor(0.8396, device='cuda:0', grad_fn=<MseLossBackward>) 27\n",
      "tensor(0.9830, device='cuda:0', grad_fn=<MseLossBackward>) 28\n",
      "tensor(0.8205, device='cuda:0', grad_fn=<MseLossBackward>) 29\n",
      "tensor(0.8042, device='cuda:0', grad_fn=<MseLossBackward>) 30\n",
      "tensor(1.0012, device='cuda:0', grad_fn=<MseLossBackward>) 31\n",
      "tensor(0.8527, device='cuda:0', grad_fn=<MseLossBackward>) 32\n",
      "tensor(0.8493, device='cuda:0', grad_fn=<MseLossBackward>) 33\n",
      "tensor(0.8744, device='cuda:0', grad_fn=<MseLossBackward>) 34\n",
      "tensor(0.8497, device='cuda:0', grad_fn=<MseLossBackward>) 35\n",
      "tensor(1.0060, device='cuda:0', grad_fn=<MseLossBackward>) 36\n",
      "tensor(0.8124, device='cuda:0', grad_fn=<MseLossBackward>) 37\n",
      "tensor(0.8165, device='cuda:0', grad_fn=<MseLossBackward>) 38\n",
      "tensor(1.0310, device='cuda:0', grad_fn=<MseLossBackward>) 39\n",
      "tensor(0.8392, device='cuda:0', grad_fn=<MseLossBackward>) 40\n",
      "tensor(0.8220, device='cuda:0', grad_fn=<MseLossBackward>) 41\n",
      "tensor(0.9052, device='cuda:0', grad_fn=<MseLossBackward>) 42\n",
      "tensor(0.8744, device='cuda:0', grad_fn=<MseLossBackward>) 43\n",
      "tensor(0.9815, device='cuda:0', grad_fn=<MseLossBackward>) 44\n",
      "tensor(0.8116, device='cuda:0', grad_fn=<MseLossBackward>) 45\n",
      "tensor(0.9401, device='cuda:0', grad_fn=<MseLossBackward>) 46\n",
      "tensor(0.9867, device='cuda:0', grad_fn=<MseLossBackward>) 47\n",
      "tensor(0.8371, device='cuda:0', grad_fn=<MseLossBackward>) 48\n",
      "tensor(1.4063, device='cuda:0', grad_fn=<MseLossBackward>) 49\n",
      "tensor(0.8207, device='cuda:0', grad_fn=<MseLossBackward>) 50\n",
      "tensor(0.8251, device='cuda:0', grad_fn=<MseLossBackward>) 51\n",
      "tensor(1.0602, device='cuda:0', grad_fn=<MseLossBackward>) 52\n",
      "tensor(0.8536, device='cuda:0', grad_fn=<MseLossBackward>) 53\n",
      "tensor(0.8105, device='cuda:0', grad_fn=<MseLossBackward>) 54\n",
      "tensor(0.8988, device='cuda:0', grad_fn=<MseLossBackward>) 55\n",
      "tensor(0.8636, device='cuda:0', grad_fn=<MseLossBackward>) 56\n",
      "tensor(0.9724, device='cuda:0', grad_fn=<MseLossBackward>) 57\n",
      "tensor(0.8357, device='cuda:0', grad_fn=<MseLossBackward>) 58\n",
      "tensor(0.8481, device='cuda:0', grad_fn=<MseLossBackward>) 59\n",
      "tensor(1.0029, device='cuda:0', grad_fn=<MseLossBackward>) 60\n",
      "tensor(0.8411, device='cuda:0', grad_fn=<MseLossBackward>) 61\n",
      "tensor(0.8455, device='cuda:0', grad_fn=<MseLossBackward>) 62\n",
      "tensor(0.9081, device='cuda:0', grad_fn=<MseLossBackward>) 63\n",
      "tensor(0.8637, device='cuda:0', grad_fn=<MseLossBackward>) 64\n",
      "tensor(0.9501, device='cuda:0', grad_fn=<MseLossBackward>) 65\n",
      "tensor(0.8682, device='cuda:0', grad_fn=<MseLossBackward>) 66\n",
      "tensor(0.8539, device='cuda:0', grad_fn=<MseLossBackward>) 67\n",
      "tensor(0.9994, device='cuda:0', grad_fn=<MseLossBackward>) 68\n",
      "tensor(0.8578, device='cuda:0', grad_fn=<MseLossBackward>) 69\n",
      "tensor(0.8527, device='cuda:0', grad_fn=<MseLossBackward>) 70\n",
      "tensor(0.9869, device='cuda:0', grad_fn=<MseLossBackward>) 71\n",
      "tensor(0.8330, device='cuda:0', grad_fn=<MseLossBackward>) 72\n",
      "tensor(0.8436, device='cuda:0', grad_fn=<MseLossBackward>) 73\n",
      "tensor(0.9183, device='cuda:0', grad_fn=<MseLossBackward>) 74\n",
      "tensor(0.8388, device='cuda:0', grad_fn=<MseLossBackward>) 75\n",
      "tensor(0.8689, device='cuda:0', grad_fn=<MseLossBackward>) 76\n",
      "tensor(0.8552, device='cuda:0', grad_fn=<MseLossBackward>) 77\n",
      "tensor(0.8454, device='cuda:0', grad_fn=<MseLossBackward>) 78\n",
      "tensor(0.9687, device='cuda:0', grad_fn=<MseLossBackward>) 79\n",
      "tensor(0.8149, device='cuda:0', grad_fn=<MseLossBackward>) 80\n",
      "tensor(0.8679, device='cuda:0', grad_fn=<MseLossBackward>) 81\n",
      "tensor(0.9700, device='cuda:0', grad_fn=<MseLossBackward>) 82\n",
      "tensor(0.7942, device='cuda:0', grad_fn=<MseLossBackward>) 83\n",
      "tensor(0.9592, device='cuda:0', grad_fn=<MseLossBackward>) 84\n",
      "tensor(0.9359, device='cuda:0', grad_fn=<MseLossBackward>) 85\n",
      "tensor(0.8297, device='cuda:0', grad_fn=<MseLossBackward>) 86\n",
      "tensor(0.9667, device='cuda:0', grad_fn=<MseLossBackward>) 87\n",
      "tensor(0.9392, device='cuda:0', grad_fn=<MseLossBackward>) 88\n",
      "tensor(0.8173, device='cuda:0', grad_fn=<MseLossBackward>) 89\n",
      "tensor(0.8409, device='cuda:0', grad_fn=<MseLossBackward>) 90\n",
      "tensor(0.8920, device='cuda:0', grad_fn=<MseLossBackward>) 91\n",
      "tensor(0.8232, device='cuda:0', grad_fn=<MseLossBackward>) 92\n",
      "tensor(0.8967, device='cuda:0', grad_fn=<MseLossBackward>) 93\n",
      "tensor(0.8558, device='cuda:0', grad_fn=<MseLossBackward>) 94\n",
      "tensor(0.8270, device='cuda:0', grad_fn=<MseLossBackward>) 95\n",
      "tensor(0.9357, device='cuda:0', grad_fn=<MseLossBackward>) 96\n",
      "tensor(0.8357, device='cuda:0', grad_fn=<MseLossBackward>) 97\n",
      "tensor(0.8228, device='cuda:0', grad_fn=<MseLossBackward>) 98\n",
      "tensor(0.9485, device='cuda:0', grad_fn=<MseLossBackward>) 99\n",
      "tensor(0.8327, device='cuda:0', grad_fn=<MseLossBackward>) 100\n",
      "tensor(0.8227, device='cuda:0', grad_fn=<MseLossBackward>) 101\n",
      "tensor(0.9563, device='cuda:0', grad_fn=<MseLossBackward>) 102\n",
      "tensor(0.8529, device='cuda:0', grad_fn=<MseLossBackward>) 103\n",
      "tensor(0.8281, device='cuda:0', grad_fn=<MseLossBackward>) 104\n",
      "tensor(0.9564, device='cuda:0', grad_fn=<MseLossBackward>) 105\n",
      "tensor(0.9049, device='cuda:0', grad_fn=<MseLossBackward>) 106\n",
      "tensor(0.8313, device='cuda:0', grad_fn=<MseLossBackward>) 107\n",
      "tensor(0.9291, device='cuda:0', grad_fn=<MseLossBackward>) 108\n",
      "tensor(0.9289, device='cuda:0', grad_fn=<MseLossBackward>) 109\n",
      "tensor(0.8295, device='cuda:0', grad_fn=<MseLossBackward>) 110\n",
      "tensor(0.8435, device='cuda:0', grad_fn=<MseLossBackward>) 111\n",
      "tensor(0.8880, device='cuda:0', grad_fn=<MseLossBackward>) 112\n",
      "tensor(0.8258, device='cuda:0', grad_fn=<MseLossBackward>) 113\n",
      "tensor(0.8306, device='cuda:0', grad_fn=<MseLossBackward>) 114\n",
      "tensor(0.8726, device='cuda:0', grad_fn=<MseLossBackward>) 115\n",
      "tensor(0.8252, device='cuda:0', grad_fn=<MseLossBackward>) 116\n",
      "tensor(0.8455, device='cuda:0', grad_fn=<MseLossBackward>) 117\n",
      "tensor(0.8642, device='cuda:0', grad_fn=<MseLossBackward>) 118\n",
      "tensor(0.8245, device='cuda:0', grad_fn=<MseLossBackward>) 119\n",
      "tensor(0.8539, device='cuda:0', grad_fn=<MseLossBackward>) 120\n",
      "tensor(0.8631, device='cuda:0', grad_fn=<MseLossBackward>) 121\n",
      "tensor(0.8275, device='cuda:0', grad_fn=<MseLossBackward>) 122\n",
      "tensor(0.8472, device='cuda:0', grad_fn=<MseLossBackward>) 123\n",
      "tensor(0.8678, device='cuda:0', grad_fn=<MseLossBackward>) 124\n",
      "tensor(0.8331, device='cuda:0', grad_fn=<MseLossBackward>) 125\n",
      "tensor(0.8371, device='cuda:0', grad_fn=<MseLossBackward>) 126\n",
      "tensor(0.8782, device='cuda:0', grad_fn=<MseLossBackward>) 127\n",
      "tensor(0.8334, device='cuda:0', grad_fn=<MseLossBackward>) 128\n",
      "tensor(0.8597, device='cuda:0', grad_fn=<MseLossBackward>) 129\n",
      "tensor(0.8918, device='cuda:0', grad_fn=<MseLossBackward>) 130\n",
      "tensor(0.8024, device='cuda:0', grad_fn=<MseLossBackward>) 131\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.9533, device='cuda:0', grad_fn=<MseLossBackward>) 132\n",
      "tensor(0.8812, device='cuda:0', grad_fn=<MseLossBackward>) 133\n",
      "tensor(0.7767, device='cuda:0', grad_fn=<MseLossBackward>) 134\n",
      "tensor(1.0100, device='cuda:0', grad_fn=<MseLossBackward>) 135\n",
      "tensor(0.8673, device='cuda:0', grad_fn=<MseLossBackward>) 136\n",
      "tensor(0.7690, device='cuda:0', grad_fn=<MseLossBackward>) 137\n",
      "tensor(0.8353, device='cuda:0', grad_fn=<MseLossBackward>) 138\n",
      "tensor(0.9151, device='cuda:0', grad_fn=<MseLossBackward>) 139\n",
      "tensor(0.7618, device='cuda:0', grad_fn=<MseLossBackward>) 140\n",
      "tensor(0.7551, device='cuda:0', grad_fn=<MseLossBackward>) 141\n",
      "tensor(0.8635, device='cuda:0', grad_fn=<MseLossBackward>) 142\n",
      "tensor(0.7994, device='cuda:0', grad_fn=<MseLossBackward>) 143\n",
      "tensor(0.7858, device='cuda:0', grad_fn=<MseLossBackward>) 144\n",
      "tensor(0.8511, device='cuda:0', grad_fn=<MseLossBackward>) 145\n",
      "tensor(0.7850, device='cuda:0', grad_fn=<MseLossBackward>) 146\n",
      "tensor(0.7870, device='cuda:0', grad_fn=<MseLossBackward>) 147\n",
      "tensor(0.8962, device='cuda:0', grad_fn=<MseLossBackward>) 148\n",
      "tensor(0.7757, device='cuda:0', grad_fn=<MseLossBackward>) 149\n",
      "tensor(0.7655, device='cuda:0', grad_fn=<MseLossBackward>) 150\n",
      "tensor(0.9024, device='cuda:0', grad_fn=<MseLossBackward>) 151\n",
      "tensor(0.7802, device='cuda:0', grad_fn=<MseLossBackward>) 152\n",
      "tensor(0.7662, device='cuda:0', grad_fn=<MseLossBackward>) 153\n",
      "tensor(0.8920, device='cuda:0', grad_fn=<MseLossBackward>) 154\n",
      "tensor(0.7748, device='cuda:0', grad_fn=<MseLossBackward>) 155\n",
      "tensor(0.7700, device='cuda:0', grad_fn=<MseLossBackward>) 156\n",
      "tensor(0.8901, device='cuda:0', grad_fn=<MseLossBackward>) 157\n",
      "tensor(0.7686, device='cuda:0', grad_fn=<MseLossBackward>) 158\n",
      "tensor(0.7726, device='cuda:0', grad_fn=<MseLossBackward>) 159\n",
      "tensor(0.8853, device='cuda:0', grad_fn=<MseLossBackward>) 160\n",
      "tensor(0.7730, device='cuda:0', grad_fn=<MseLossBackward>) 161\n",
      "tensor(0.7771, device='cuda:0', grad_fn=<MseLossBackward>) 162\n",
      "tensor(0.8768, device='cuda:0', grad_fn=<MseLossBackward>) 163\n",
      "tensor(0.8089, device='cuda:0', grad_fn=<MseLossBackward>) 164\n",
      "tensor(0.7828, device='cuda:0', grad_fn=<MseLossBackward>) 165\n",
      "tensor(0.8133, device='cuda:0', grad_fn=<MseLossBackward>) 166\n",
      "tensor(0.8389, device='cuda:0', grad_fn=<MseLossBackward>) 167\n",
      "tensor(0.7813, device='cuda:0', grad_fn=<MseLossBackward>) 168\n",
      "tensor(0.7757, device='cuda:0', grad_fn=<MseLossBackward>) 169\n",
      "tensor(0.8852, device='cuda:0', grad_fn=<MseLossBackward>) 170\n",
      "tensor(0.7665, device='cuda:0', grad_fn=<MseLossBackward>) 171\n",
      "tensor(0.7605, device='cuda:0', grad_fn=<MseLossBackward>) 172\n",
      "tensor(0.9145, device='cuda:0', grad_fn=<MseLossBackward>) 173\n",
      "tensor(0.7612, device='cuda:0', grad_fn=<MseLossBackward>) 174\n",
      "tensor(0.7661, device='cuda:0', grad_fn=<MseLossBackward>) 175\n",
      "tensor(0.8926, device='cuda:0', grad_fn=<MseLossBackward>) 176\n",
      "tensor(0.8068, device='cuda:0', grad_fn=<MseLossBackward>) 177\n",
      "tensor(0.7694, device='cuda:0', grad_fn=<MseLossBackward>) 178\n",
      "tensor(0.8502, device='cuda:0', grad_fn=<MseLossBackward>) 179\n",
      "tensor(0.9485, device='cuda:0', grad_fn=<MseLossBackward>) 180\n",
      "tensor(0.7685, device='cuda:0', grad_fn=<MseLossBackward>) 181\n",
      "tensor(0.8140, device='cuda:0', grad_fn=<MseLossBackward>) 182\n",
      "tensor(1.0172, device='cuda:0', grad_fn=<MseLossBackward>) 183\n",
      "tensor(0.8650, device='cuda:0', grad_fn=<MseLossBackward>) 184\n",
      "tensor(0.7394, device='cuda:0', grad_fn=<MseLossBackward>) 185\n",
      "tensor(0.8527, device='cuda:0', grad_fn=<MseLossBackward>) 186\n",
      "tensor(0.8651, device='cuda:0', grad_fn=<MseLossBackward>) 187\n",
      "tensor(0.7259, device='cuda:0', grad_fn=<MseLossBackward>) 188\n",
      "tensor(0.7912, device='cuda:0', grad_fn=<MseLossBackward>) 189\n",
      "tensor(0.8274, device='cuda:0', grad_fn=<MseLossBackward>) 190\n",
      "tensor(0.7563, device='cuda:0', grad_fn=<MseLossBackward>) 191\n",
      "tensor(0.7794, device='cuda:0', grad_fn=<MseLossBackward>) 192\n",
      "tensor(0.7915, device='cuda:0', grad_fn=<MseLossBackward>) 193\n",
      "tensor(0.7546, device='cuda:0', grad_fn=<MseLossBackward>) 194\n",
      "tensor(0.7483, device='cuda:0', grad_fn=<MseLossBackward>) 195\n",
      "tensor(0.8113, device='cuda:0', grad_fn=<MseLossBackward>) 196\n",
      "tensor(0.7385, device='cuda:0', grad_fn=<MseLossBackward>) 197\n",
      "tensor(0.7373, device='cuda:0', grad_fn=<MseLossBackward>) 198\n",
      "tensor(0.8380, device='cuda:0', grad_fn=<MseLossBackward>) 199\n",
      "tensor(0.7306, device='cuda:0', grad_fn=<MseLossBackward>) 200\n",
      "tensor(0.7370, device='cuda:0', grad_fn=<MseLossBackward>) 201\n",
      "tensor(0.8385, device='cuda:0', grad_fn=<MseLossBackward>) 202\n",
      "tensor(0.7360, device='cuda:0', grad_fn=<MseLossBackward>) 203\n",
      "tensor(0.7433, device='cuda:0', grad_fn=<MseLossBackward>) 204\n",
      "tensor(0.8091, device='cuda:0', grad_fn=<MseLossBackward>) 205\n",
      "tensor(0.7820, device='cuda:0', grad_fn=<MseLossBackward>) 206\n",
      "tensor(0.7445, device='cuda:0', grad_fn=<MseLossBackward>) 207\n",
      "tensor(0.7441, device='cuda:0', grad_fn=<MseLossBackward>) 208\n",
      "tensor(0.8292, device='cuda:0', grad_fn=<MseLossBackward>) 209\n",
      "tensor(0.7285, device='cuda:0', grad_fn=<MseLossBackward>) 210\n",
      "tensor(0.7338, device='cuda:0', grad_fn=<MseLossBackward>) 211\n",
      "tensor(0.8391, device='cuda:0', grad_fn=<MseLossBackward>) 212\n",
      "tensor(0.7261, device='cuda:0', grad_fn=<MseLossBackward>) 213\n",
      "tensor(0.7429, device='cuda:0', grad_fn=<MseLossBackward>) 214\n",
      "tensor(0.8147, device='cuda:0', grad_fn=<MseLossBackward>) 215\n",
      "tensor(0.7684, device='cuda:0', grad_fn=<MseLossBackward>) 216\n",
      "tensor(0.7426, device='cuda:0', grad_fn=<MseLossBackward>) 217\n",
      "tensor(0.7575, device='cuda:0', grad_fn=<MseLossBackward>) 218\n",
      "tensor(0.8210, device='cuda:0', grad_fn=<MseLossBackward>) 219\n",
      "tensor(0.7095, device='cuda:0', grad_fn=<MseLossBackward>) 220\n",
      "tensor(0.7644, device='cuda:0', grad_fn=<MseLossBackward>) 221\n",
      "tensor(0.8303, device='cuda:0', grad_fn=<MseLossBackward>) 222\n",
      "tensor(0.7079, device='cuda:0', grad_fn=<MseLossBackward>) 223\n",
      "tensor(0.7611, device='cuda:0', grad_fn=<MseLossBackward>) 224\n",
      "tensor(0.8574, device='cuda:0', grad_fn=<MseLossBackward>) 225\n",
      "tensor(0.7551, device='cuda:0', grad_fn=<MseLossBackward>) 226\n",
      "tensor(0.7087, device='cuda:0', grad_fn=<MseLossBackward>) 227\n",
      "tensor(0.8384, device='cuda:0', grad_fn=<MseLossBackward>) 228\n",
      "tensor(0.7898, device='cuda:0', grad_fn=<MseLossBackward>) 229\n",
      "tensor(0.6874, device='cuda:0', grad_fn=<MseLossBackward>) 230\n",
      "tensor(0.7407, device='cuda:0', grad_fn=<MseLossBackward>) 231\n",
      "tensor(0.8569, device='cuda:0', grad_fn=<MseLossBackward>) 232\n",
      "tensor(0.7161, device='cuda:0', grad_fn=<MseLossBackward>) 233\n",
      "tensor(0.6957, device='cuda:0', grad_fn=<MseLossBackward>) 234\n",
      "tensor(0.7611, device='cuda:0', grad_fn=<MseLossBackward>) 235\n",
      "tensor(0.7827, device='cuda:0', grad_fn=<MseLossBackward>) 236\n",
      "tensor(0.6893, device='cuda:0', grad_fn=<MseLossBackward>) 237\n",
      "tensor(0.7027, device='cuda:0', grad_fn=<MseLossBackward>) 238\n",
      "tensor(0.7934, device='cuda:0', grad_fn=<MseLossBackward>) 239\n",
      "tensor(0.7037, device='cuda:0', grad_fn=<MseLossBackward>) 240\n",
      "tensor(0.6942, device='cuda:0', grad_fn=<MseLossBackward>) 241\n",
      "tensor(0.7316, device='cuda:0', grad_fn=<MseLossBackward>) 242\n",
      "tensor(0.7606, device='cuda:0', grad_fn=<MseLossBackward>) 243\n",
      "tensor(0.7011, device='cuda:0', grad_fn=<MseLossBackward>) 244\n",
      "tensor(0.6838, device='cuda:0', grad_fn=<MseLossBackward>) 245\n",
      "tensor(0.7845, device='cuda:0', grad_fn=<MseLossBackward>) 246\n",
      "tensor(0.7047, device='cuda:0', grad_fn=<MseLossBackward>) 247\n",
      "tensor(0.6969, device='cuda:0', grad_fn=<MseLossBackward>) 248\n",
      "tensor(0.7221, device='cuda:0', grad_fn=<MseLossBackward>) 249\n",
      "tensor(0.7601, device='cuda:0', grad_fn=<MseLossBackward>) 250\n",
      "tensor(0.6978, device='cuda:0', grad_fn=<MseLossBackward>) 251\n",
      "tensor(0.6832, device='cuda:0', grad_fn=<MseLossBackward>) 252\n",
      "tensor(0.7792, device='cuda:0', grad_fn=<MseLossBackward>) 253\n",
      "tensor(0.7009, device='cuda:0', grad_fn=<MseLossBackward>) 254\n",
      "tensor(0.6943, device='cuda:0', grad_fn=<MseLossBackward>) 255\n",
      "tensor(0.7167, device='cuda:0', grad_fn=<MseLossBackward>) 256\n",
      "tensor(0.7541, device='cuda:0', grad_fn=<MseLossBackward>) 257\n",
      "tensor(0.6925, device='cuda:0', grad_fn=<MseLossBackward>) 258\n",
      "tensor(0.6803, device='cuda:0', grad_fn=<MseLossBackward>) 259\n",
      "tensor(0.7740, device='cuda:0', grad_fn=<MseLossBackward>) 260\n",
      "tensor(0.6979, device='cuda:0', grad_fn=<MseLossBackward>) 261\n",
      "tensor(0.6893, device='cuda:0', grad_fn=<MseLossBackward>) 262\n",
      "tensor(0.7109, device='cuda:0', grad_fn=<MseLossBackward>) 263\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.7501, device='cuda:0', grad_fn=<MseLossBackward>) 264\n",
      "tensor(0.6873, device='cuda:0', grad_fn=<MseLossBackward>) 265\n",
      "tensor(0.6760, device='cuda:0', grad_fn=<MseLossBackward>) 266\n",
      "tensor(0.7666, device='cuda:0', grad_fn=<MseLossBackward>) 267\n",
      "tensor(0.6937, device='cuda:0', grad_fn=<MseLossBackward>) 268\n",
      "tensor(0.6831, device='cuda:0', grad_fn=<MseLossBackward>) 269\n",
      "tensor(0.7059, device='cuda:0', grad_fn=<MseLossBackward>) 270\n",
      "tensor(0.7425, device='cuda:0', grad_fn=<MseLossBackward>) 271\n",
      "tensor(0.6808, device='cuda:0', grad_fn=<MseLossBackward>) 272\n",
      "tensor(0.6726, device='cuda:0', grad_fn=<MseLossBackward>) 273\n",
      "tensor(0.7584, device='cuda:0', grad_fn=<MseLossBackward>) 274\n",
      "tensor(0.6895, device='cuda:0', grad_fn=<MseLossBackward>) 275\n",
      "tensor(0.6765, device='cuda:0', grad_fn=<MseLossBackward>) 276\n",
      "tensor(0.7085, device='cuda:0', grad_fn=<MseLossBackward>) 277\n",
      "tensor(0.7327, device='cuda:0', grad_fn=<MseLossBackward>) 278\n",
      "tensor(0.6755, device='cuda:0', grad_fn=<MseLossBackward>) 279\n",
      "tensor(0.6811, device='cuda:0', grad_fn=<MseLossBackward>) 280\n",
      "tensor(0.7385, device='cuda:0', grad_fn=<MseLossBackward>) 281\n",
      "tensor(0.6933, device='cuda:0', grad_fn=<MseLossBackward>) 282\n",
      "tensor(0.6844, device='cuda:0', grad_fn=<MseLossBackward>) 283\n",
      "tensor(0.7511, device='cuda:0', grad_fn=<MseLossBackward>) 284\n",
      "tensor(0.6894, device='cuda:0', grad_fn=<MseLossBackward>) 285\n",
      "tensor(0.6649, device='cuda:0', grad_fn=<MseLossBackward>) 286\n",
      "tensor(0.7498, device='cuda:0', grad_fn=<MseLossBackward>) 287\n",
      "tensor(0.7141, device='cuda:0', grad_fn=<MseLossBackward>) 288\n",
      "tensor(0.6351, device='cuda:0', grad_fn=<MseLossBackward>) 289\n",
      "tensor(0.6888, device='cuda:0', grad_fn=<MseLossBackward>) 290\n",
      "tensor(0.8226, device='cuda:0', grad_fn=<MseLossBackward>) 291\n",
      "tensor(0.6615, device='cuda:0', grad_fn=<MseLossBackward>) 292\n",
      "tensor(0.6124, device='cuda:0', grad_fn=<MseLossBackward>) 293\n",
      "tensor(0.6832, device='cuda:0', grad_fn=<MseLossBackward>) 294\n",
      "tensor(0.7673, device='cuda:0', grad_fn=<MseLossBackward>) 295\n",
      "tensor(0.6435, device='cuda:0', grad_fn=<MseLossBackward>) 296\n",
      "tensor(0.6293, device='cuda:0', grad_fn=<MseLossBackward>) 297\n",
      "tensor(0.6446, device='cuda:0', grad_fn=<MseLossBackward>) 298\n",
      "tensor(0.7247, device='cuda:0', grad_fn=<MseLossBackward>) 299\n",
      "tensor(0.6463, device='cuda:0', grad_fn=<MseLossBackward>) 300\n",
      "tensor(0.6404, device='cuda:0', grad_fn=<MseLossBackward>) 301\n",
      "tensor(0.6455, device='cuda:0', grad_fn=<MseLossBackward>) 302\n",
      "tensor(0.7022, device='cuda:0', grad_fn=<MseLossBackward>) 303\n",
      "tensor(0.6409, device='cuda:0', grad_fn=<MseLossBackward>) 304\n",
      "tensor(0.6364, device='cuda:0', grad_fn=<MseLossBackward>) 305\n",
      "tensor(0.6760, device='cuda:0', grad_fn=<MseLossBackward>) 306\n",
      "tensor(0.6839, device='cuda:0', grad_fn=<MseLossBackward>) 307\n",
      "tensor(0.6401, device='cuda:0', grad_fn=<MseLossBackward>) 308\n",
      "tensor(0.6318, device='cuda:0', grad_fn=<MseLossBackward>) 309\n",
      "tensor(0.6958, device='cuda:0', grad_fn=<MseLossBackward>) 310\n",
      "tensor(0.6562, device='cuda:0', grad_fn=<MseLossBackward>) 311\n",
      "tensor(0.6409, device='cuda:0', grad_fn=<MseLossBackward>) 312\n",
      "tensor(0.6326, device='cuda:0', grad_fn=<MseLossBackward>) 313\n",
      "tensor(0.7075, device='cuda:0', grad_fn=<MseLossBackward>) 314\n",
      "tensor(0.6405, device='cuda:0', grad_fn=<MseLossBackward>) 315\n",
      "tensor(0.6386, device='cuda:0', grad_fn=<MseLossBackward>) 316\n",
      "tensor(0.6400, device='cuda:0', grad_fn=<MseLossBackward>) 317\n",
      "tensor(0.7007, device='cuda:0', grad_fn=<MseLossBackward>) 318\n",
      "tensor(0.6312, device='cuda:0', grad_fn=<MseLossBackward>) 319\n",
      "tensor(0.6338, device='cuda:0', grad_fn=<MseLossBackward>) 320\n",
      "tensor(0.6537, device='cuda:0', grad_fn=<MseLossBackward>) 321\n",
      "tensor(0.6915, device='cuda:0', grad_fn=<MseLossBackward>) 322\n",
      "tensor(0.6287, device='cuda:0', grad_fn=<MseLossBackward>) 323\n",
      "tensor(0.6278, device='cuda:0', grad_fn=<MseLossBackward>) 324\n",
      "tensor(0.6736, device='cuda:0', grad_fn=<MseLossBackward>) 325\n",
      "tensor(0.6774, device='cuda:0', grad_fn=<MseLossBackward>) 326\n",
      "tensor(0.6298, device='cuda:0', grad_fn=<MseLossBackward>) 327\n",
      "tensor(0.6230, device='cuda:0', grad_fn=<MseLossBackward>) 328\n",
      "tensor(0.6855, device='cuda:0', grad_fn=<MseLossBackward>) 329\n",
      "tensor(0.6587, device='cuda:0', grad_fn=<MseLossBackward>) 330\n",
      "tensor(0.6306, device='cuda:0', grad_fn=<MseLossBackward>) 331\n",
      "tensor(0.6209, device='cuda:0', grad_fn=<MseLossBackward>) 332\n",
      "tensor(0.6999, device='cuda:0', grad_fn=<MseLossBackward>) 333\n",
      "tensor(0.6531, device='cuda:0', grad_fn=<MseLossBackward>) 334\n",
      "tensor(0.6273, device='cuda:0', grad_fn=<MseLossBackward>) 335\n",
      "tensor(0.6191, device='cuda:0', grad_fn=<MseLossBackward>) 336\n",
      "tensor(0.7063, device='cuda:0', grad_fn=<MseLossBackward>) 337\n",
      "tensor(0.6614, device='cuda:0', grad_fn=<MseLossBackward>) 338\n",
      "tensor(0.6177, device='cuda:0', grad_fn=<MseLossBackward>) 339\n",
      "tensor(0.6088, device='cuda:0', grad_fn=<MseLossBackward>) 340\n",
      "tensor(0.7089, device='cuda:0', grad_fn=<MseLossBackward>) 341\n",
      "tensor(0.6918, device='cuda:0', grad_fn=<MseLossBackward>) 342\n",
      "tensor(0.6105, device='cuda:0', grad_fn=<MseLossBackward>) 343\n",
      "tensor(0.5980, device='cuda:0', grad_fn=<MseLossBackward>) 344\n",
      "tensor(0.7111, device='cuda:0', grad_fn=<MseLossBackward>) 345\n",
      "tensor(0.7497, device='cuda:0', grad_fn=<MseLossBackward>) 346\n",
      "tensor(0.6098, device='cuda:0', grad_fn=<MseLossBackward>) 347\n",
      "tensor(0.5945, device='cuda:0', grad_fn=<MseLossBackward>) 348\n",
      "tensor(0.6789, device='cuda:0', grad_fn=<MseLossBackward>) 349\n",
      "tensor(0.7451, device='cuda:0', grad_fn=<MseLossBackward>) 350\n",
      "tensor(0.6047, device='cuda:0', grad_fn=<MseLossBackward>) 351\n",
      "tensor(0.5804, device='cuda:0', grad_fn=<MseLossBackward>) 352\n",
      "tensor(0.6452, device='cuda:0', grad_fn=<MseLossBackward>) 353\n",
      "tensor(0.6587, device='cuda:0', grad_fn=<MseLossBackward>) 354\n",
      "tensor(0.5986, device='cuda:0', grad_fn=<MseLossBackward>) 355\n",
      "tensor(0.5873, device='cuda:0', grad_fn=<MseLossBackward>) 356\n",
      "tensor(0.6335, device='cuda:0', grad_fn=<MseLossBackward>) 357\n",
      "tensor(0.6053, device='cuda:0', grad_fn=<MseLossBackward>) 358\n",
      "tensor(0.5906, device='cuda:0', grad_fn=<MseLossBackward>) 359\n",
      "tensor(0.6144, device='cuda:0', grad_fn=<MseLossBackward>) 360\n",
      "tensor(0.6185, device='cuda:0', grad_fn=<MseLossBackward>) 361\n",
      "tensor(0.5839, device='cuda:0', grad_fn=<MseLossBackward>) 362\n",
      "tensor(0.5850, device='cuda:0', grad_fn=<MseLossBackward>) 363\n",
      "tensor(0.6258, device='cuda:0', grad_fn=<MseLossBackward>) 364\n",
      "tensor(0.6046, device='cuda:0', grad_fn=<MseLossBackward>) 365\n",
      "tensor(0.5817, device='cuda:0', grad_fn=<MseLossBackward>) 366\n",
      "tensor(0.5821, device='cuda:0', grad_fn=<MseLossBackward>) 367\n",
      "tensor(0.6289, device='cuda:0', grad_fn=<MseLossBackward>) 368\n",
      "tensor(0.6010, device='cuda:0', grad_fn=<MseLossBackward>) 369\n",
      "tensor(0.5814, device='cuda:0', grad_fn=<MseLossBackward>) 370\n",
      "tensor(0.5796, device='cuda:0', grad_fn=<MseLossBackward>) 371\n",
      "tensor(0.6291, device='cuda:0', grad_fn=<MseLossBackward>) 372\n",
      "tensor(0.5991, device='cuda:0', grad_fn=<MseLossBackward>) 373\n",
      "tensor(0.5808, device='cuda:0', grad_fn=<MseLossBackward>) 374\n",
      "tensor(0.5777, device='cuda:0', grad_fn=<MseLossBackward>) 375\n",
      "tensor(0.6292, device='cuda:0', grad_fn=<MseLossBackward>) 376\n",
      "tensor(0.5966, device='cuda:0', grad_fn=<MseLossBackward>) 377\n",
      "tensor(0.5798, device='cuda:0', grad_fn=<MseLossBackward>) 378\n",
      "tensor(0.5761, device='cuda:0', grad_fn=<MseLossBackward>) 379\n",
      "tensor(0.6286, device='cuda:0', grad_fn=<MseLossBackward>) 380\n",
      "tensor(0.5938, device='cuda:0', grad_fn=<MseLossBackward>) 381\n",
      "tensor(0.5785, device='cuda:0', grad_fn=<MseLossBackward>) 382\n",
      "tensor(0.5750, device='cuda:0', grad_fn=<MseLossBackward>) 383\n",
      "tensor(0.6280, device='cuda:0', grad_fn=<MseLossBackward>) 384\n",
      "tensor(0.5905, device='cuda:0', grad_fn=<MseLossBackward>) 385\n",
      "tensor(0.5769, device='cuda:0', grad_fn=<MseLossBackward>) 386\n",
      "tensor(0.5742, device='cuda:0', grad_fn=<MseLossBackward>) 387\n",
      "tensor(0.6274, device='cuda:0', grad_fn=<MseLossBackward>) 388\n",
      "tensor(0.5868, device='cuda:0', grad_fn=<MseLossBackward>) 389\n",
      "tensor(0.5750, device='cuda:0', grad_fn=<MseLossBackward>) 390\n",
      "tensor(0.5742, device='cuda:0', grad_fn=<MseLossBackward>) 391\n",
      "tensor(0.6268, device='cuda:0', grad_fn=<MseLossBackward>) 392\n",
      "tensor(0.5832, device='cuda:0', grad_fn=<MseLossBackward>) 393\n",
      "tensor(0.5725, device='cuda:0', grad_fn=<MseLossBackward>) 394\n",
      "tensor(0.5758, device='cuda:0', grad_fn=<MseLossBackward>) 395\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.6261, device='cuda:0', grad_fn=<MseLossBackward>) 396\n",
      "tensor(0.5798, device='cuda:0', grad_fn=<MseLossBackward>) 397\n",
      "tensor(0.5684, device='cuda:0', grad_fn=<MseLossBackward>) 398\n",
      "tensor(0.5805, device='cuda:0', grad_fn=<MseLossBackward>) 399\n",
      "tensor(0.6238, device='cuda:0', grad_fn=<MseLossBackward>) 400\n",
      "tensor(0.5751, device='cuda:0', grad_fn=<MseLossBackward>) 401\n",
      "tensor(0.5583, device='cuda:0', grad_fn=<MseLossBackward>) 402\n",
      "tensor(0.5864, device='cuda:0', grad_fn=<MseLossBackward>) 403\n",
      "tensor(0.6260, device='cuda:0', grad_fn=<MseLossBackward>) 404\n",
      "tensor(0.5683, device='cuda:0', grad_fn=<MseLossBackward>) 405\n",
      "tensor(0.5429, device='cuda:0', grad_fn=<MseLossBackward>) 406\n",
      "tensor(0.5851, device='cuda:0', grad_fn=<MseLossBackward>) 407\n",
      "tensor(0.6274, device='cuda:0', grad_fn=<MseLossBackward>) 408\n",
      "tensor(0.5613, device='cuda:0', grad_fn=<MseLossBackward>) 409\n",
      "tensor(0.5304, device='cuda:0', grad_fn=<MseLossBackward>) 410\n",
      "tensor(0.5811, device='cuda:0', grad_fn=<MseLossBackward>) 411\n",
      "tensor(0.6256, device='cuda:0', grad_fn=<MseLossBackward>) 412\n",
      "tensor(0.5602, device='cuda:0', grad_fn=<MseLossBackward>) 413\n",
      "tensor(0.5247, device='cuda:0', grad_fn=<MseLossBackward>) 414\n",
      "tensor(0.5811, device='cuda:0', grad_fn=<MseLossBackward>) 415\n",
      "tensor(0.6278, device='cuda:0', grad_fn=<MseLossBackward>) 416\n",
      "tensor(0.5605, device='cuda:0', grad_fn=<MseLossBackward>) 417\n",
      "tensor(0.5269, device='cuda:0', grad_fn=<MseLossBackward>) 418\n",
      "tensor(0.5817, device='cuda:0', grad_fn=<MseLossBackward>) 419\n",
      "tensor(0.6409, device='cuda:0', grad_fn=<MseLossBackward>) 420\n",
      "tensor(0.5482, device='cuda:0', grad_fn=<MseLossBackward>) 421\n",
      "tensor(0.5293, device='cuda:0', grad_fn=<MseLossBackward>) 422\n",
      "tensor(0.5610, device='cuda:0', grad_fn=<MseLossBackward>) 423\n",
      "tensor(0.6251, device='cuda:0', grad_fn=<MseLossBackward>) 424\n",
      "tensor(0.5434, device='cuda:0', grad_fn=<MseLossBackward>) 425\n",
      "tensor(0.5280, device='cuda:0', grad_fn=<MseLossBackward>) 426\n",
      "tensor(0.5373, device='cuda:0', grad_fn=<MseLossBackward>) 427\n",
      "tensor(0.5869, device='cuda:0', grad_fn=<MseLossBackward>) 428\n",
      "tensor(0.5545, device='cuda:0', grad_fn=<MseLossBackward>) 429\n",
      "tensor(0.5287, device='cuda:0', grad_fn=<MseLossBackward>) 430\n",
      "tensor(0.5292, device='cuda:0', grad_fn=<MseLossBackward>) 431\n",
      "tensor(0.5663, device='cuda:0', grad_fn=<MseLossBackward>) 432\n",
      "tensor(0.5606, device='cuda:0', grad_fn=<MseLossBackward>) 433\n",
      "tensor(0.5290, device='cuda:0', grad_fn=<MseLossBackward>) 434\n",
      "tensor(0.5268, device='cuda:0', grad_fn=<MseLossBackward>) 435\n",
      "tensor(0.5564, device='cuda:0', grad_fn=<MseLossBackward>) 436\n",
      "tensor(0.5641, device='cuda:0', grad_fn=<MseLossBackward>) 437\n",
      "tensor(0.5268, device='cuda:0', grad_fn=<MseLossBackward>) 438\n",
      "tensor(0.5267, device='cuda:0', grad_fn=<MseLossBackward>) 439\n",
      "tensor(0.5474, device='cuda:0', grad_fn=<MseLossBackward>) 440\n",
      "tensor(0.5703, device='cuda:0', grad_fn=<MseLossBackward>) 441\n",
      "tensor(0.5236, device='cuda:0', grad_fn=<MseLossBackward>) 442\n",
      "tensor(0.5284, device='cuda:0', grad_fn=<MseLossBackward>) 443\n",
      "tensor(0.5343, device='cuda:0', grad_fn=<MseLossBackward>) 444\n",
      "tensor(0.5731, device='cuda:0', grad_fn=<MseLossBackward>) 445\n",
      "tensor(0.5221, device='cuda:0', grad_fn=<MseLossBackward>) 446\n",
      "tensor(0.5293, device='cuda:0', grad_fn=<MseLossBackward>) 447\n",
      "tensor(0.5253, device='cuda:0', grad_fn=<MseLossBackward>) 448\n",
      "tensor(0.5767, device='cuda:0', grad_fn=<MseLossBackward>) 449\n",
      "tensor(0.5238, device='cuda:0', grad_fn=<MseLossBackward>) 450\n",
      "tensor(0.5286, device='cuda:0', grad_fn=<MseLossBackward>) 451\n",
      "tensor(0.5190, device='cuda:0', grad_fn=<MseLossBackward>) 452\n",
      "tensor(0.5781, device='cuda:0', grad_fn=<MseLossBackward>) 453\n",
      "tensor(0.5299, device='cuda:0', grad_fn=<MseLossBackward>) 454\n",
      "tensor(0.5263, device='cuda:0', grad_fn=<MseLossBackward>) 455\n",
      "tensor(0.5145, device='cuda:0', grad_fn=<MseLossBackward>) 456\n",
      "tensor(0.5704, device='cuda:0', grad_fn=<MseLossBackward>) 457\n",
      "tensor(0.5381, device='cuda:0', grad_fn=<MseLossBackward>) 458\n",
      "tensor(0.5231, device='cuda:0', grad_fn=<MseLossBackward>) 459\n",
      "tensor(0.5118, device='cuda:0', grad_fn=<MseLossBackward>) 460\n",
      "tensor(0.5559, device='cuda:0', grad_fn=<MseLossBackward>) 461\n",
      "tensor(0.5468, device='cuda:0', grad_fn=<MseLossBackward>) 462\n",
      "tensor(0.5199, device='cuda:0', grad_fn=<MseLossBackward>) 463\n",
      "tensor(0.5105, device='cuda:0', grad_fn=<MseLossBackward>) 464\n",
      "tensor(0.5400, device='cuda:0', grad_fn=<MseLossBackward>) 465\n",
      "tensor(0.5627, device='cuda:0', grad_fn=<MseLossBackward>) 466\n",
      "tensor(0.5178, device='cuda:0', grad_fn=<MseLossBackward>) 467\n",
      "tensor(0.5104, device='cuda:0', grad_fn=<MseLossBackward>) 468\n",
      "tensor(0.5190, device='cuda:0', grad_fn=<MseLossBackward>) 469\n",
      "tensor(0.5801, device='cuda:0', grad_fn=<MseLossBackward>) 470\n",
      "tensor(0.5271, device='cuda:0', grad_fn=<MseLossBackward>) 471\n",
      "tensor(0.5125, device='cuda:0', grad_fn=<MseLossBackward>) 472\n",
      "tensor(0.4990, device='cuda:0', grad_fn=<MseLossBackward>) 473\n",
      "tensor(0.5871, device='cuda:0', grad_fn=<MseLossBackward>) 474\n",
      "tensor(0.5632, device='cuda:0', grad_fn=<MseLossBackward>) 475\n",
      "tensor(0.5147, device='cuda:0', grad_fn=<MseLossBackward>) 476\n",
      "tensor(0.4965, device='cuda:0', grad_fn=<MseLossBackward>) 477\n",
      "tensor(0.5713, device='cuda:0', grad_fn=<MseLossBackward>) 478\n",
      "tensor(0.6318, device='cuda:0', grad_fn=<MseLossBackward>) 479\n",
      "tensor(0.5305, device='cuda:0', grad_fn=<MseLossBackward>) 480\n",
      "tensor(0.5032, device='cuda:0', grad_fn=<MseLossBackward>) 481\n",
      "tensor(0.6248, device='cuda:0', grad_fn=<MseLossBackward>) 482\n",
      "tensor(0.5366, device='cuda:0', grad_fn=<MseLossBackward>) 483\n",
      "tensor(0.5176, device='cuda:0', grad_fn=<MseLossBackward>) 484\n",
      "tensor(0.4904, device='cuda:0', grad_fn=<MseLossBackward>) 485\n",
      "tensor(0.5310, device='cuda:0', grad_fn=<MseLossBackward>) 486\n",
      "tensor(0.5337, device='cuda:0', grad_fn=<MseLossBackward>) 487\n",
      "tensor(0.5129, device='cuda:0', grad_fn=<MseLossBackward>) 488\n",
      "tensor(0.4955, device='cuda:0', grad_fn=<MseLossBackward>) 489\n",
      "tensor(0.5135, device='cuda:0', grad_fn=<MseLossBackward>) 490\n",
      "tensor(0.5427, device='cuda:0', grad_fn=<MseLossBackward>) 491\n",
      "tensor(0.5015, device='cuda:0', grad_fn=<MseLossBackward>) 492\n",
      "tensor(0.5007, device='cuda:0', grad_fn=<MseLossBackward>) 493\n",
      "tensor(0.4940, device='cuda:0', grad_fn=<MseLossBackward>) 494\n",
      "tensor(0.5449, device='cuda:0', grad_fn=<MseLossBackward>) 495\n",
      "tensor(0.5024, device='cuda:0', grad_fn=<MseLossBackward>) 496\n",
      "tensor(0.5005, device='cuda:0', grad_fn=<MseLossBackward>) 497\n",
      "tensor(0.4855, device='cuda:0', grad_fn=<MseLossBackward>) 498\n",
      "tensor(0.5370, device='cuda:0', grad_fn=<MseLossBackward>) 499\n",
      "tensor(0.5153, device='cuda:0', grad_fn=<MseLossBackward>) 500\n",
      "tensor(0.4946, device='cuda:0', grad_fn=<MseLossBackward>) 501\n",
      "tensor(0.4838, device='cuda:0', grad_fn=<MseLossBackward>) 502\n",
      "tensor(0.5046, device='cuda:0', grad_fn=<MseLossBackward>) 503\n",
      "tensor(0.5378, device='cuda:0', grad_fn=<MseLossBackward>) 504\n",
      "tensor(0.4913, device='cuda:0', grad_fn=<MseLossBackward>) 505\n",
      "tensor(0.4873, device='cuda:0', grad_fn=<MseLossBackward>) 506\n",
      "tensor(0.4767, device='cuda:0', grad_fn=<MseLossBackward>) 507\n",
      "tensor(0.5366, device='cuda:0', grad_fn=<MseLossBackward>) 508\n",
      "tensor(0.5021, device='cuda:0', grad_fn=<MseLossBackward>) 509\n",
      "tensor(0.4892, device='cuda:0', grad_fn=<MseLossBackward>) 510\n",
      "tensor(0.4681, device='cuda:0', grad_fn=<MseLossBackward>) 511\n",
      "tensor(0.5161, device='cuda:0', grad_fn=<MseLossBackward>) 512\n",
      "tensor(0.5169, device='cuda:0', grad_fn=<MseLossBackward>) 513\n",
      "tensor(0.4887, device='cuda:0', grad_fn=<MseLossBackward>) 514\n",
      "tensor(0.4694, device='cuda:0', grad_fn=<MseLossBackward>) 515\n",
      "tensor(0.4837, device='cuda:0', grad_fn=<MseLossBackward>) 516\n",
      "tensor(0.5334, device='cuda:0', grad_fn=<MseLossBackward>) 517\n",
      "tensor(0.4921, device='cuda:0', grad_fn=<MseLossBackward>) 518\n",
      "tensor(0.4738, device='cuda:0', grad_fn=<MseLossBackward>) 519\n",
      "tensor(0.4598, device='cuda:0', grad_fn=<MseLossBackward>) 520\n",
      "tensor(0.5147, device='cuda:0', grad_fn=<MseLossBackward>) 521\n",
      "tensor(0.4982, device='cuda:0', grad_fn=<MseLossBackward>) 522\n",
      "tensor(0.4758, device='cuda:0', grad_fn=<MseLossBackward>) 523\n",
      "tensor(0.4525, device='cuda:0', grad_fn=<MseLossBackward>) 524\n",
      "tensor(0.4919, device='cuda:0', grad_fn=<MseLossBackward>) 525\n",
      "tensor(0.4966, device='cuda:0', grad_fn=<MseLossBackward>) 526\n",
      "tensor(0.4730, device='cuda:0', grad_fn=<MseLossBackward>) 527\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.4550, device='cuda:0', grad_fn=<MseLossBackward>) 528\n",
      "tensor(0.4662, device='cuda:0', grad_fn=<MseLossBackward>) 529\n",
      "tensor(0.4268, device='cuda:0', grad_fn=<MseLossBackward>) 530\n",
      "tensor(0.5704, device='cuda:0', grad_fn=<MseLossBackward>) 531\n",
      "tensor(0.5233, device='cuda:0', grad_fn=<MseLossBackward>) 532\n",
      "tensor(0.4709, device='cuda:0', grad_fn=<MseLossBackward>) 533\n",
      "tensor(0.4741, device='cuda:0', grad_fn=<MseLossBackward>) 534\n",
      "tensor(0.4729, device='cuda:0', grad_fn=<MseLossBackward>) 535\n",
      "tensor(0.4585, device='cuda:0', grad_fn=<MseLossBackward>) 536\n",
      "tensor(0.4189, device='cuda:0', grad_fn=<MseLossBackward>) 537\n",
      "tensor(0.4394, device='cuda:0', grad_fn=<MseLossBackward>) 538\n",
      "tensor(0.4720, device='cuda:0', grad_fn=<MseLossBackward>) 539\n",
      "tensor(0.4513, device='cuda:0', grad_fn=<MseLossBackward>) 540\n",
      "tensor(0.4197, device='cuda:0', grad_fn=<MseLossBackward>) 541\n",
      "tensor(0.4074, device='cuda:0', grad_fn=<MseLossBackward>) 542\n",
      "tensor(0.4473, device='cuda:0', grad_fn=<MseLossBackward>) 543\n",
      "tensor(0.4277, device='cuda:0', grad_fn=<MseLossBackward>) 544\n",
      "tensor(0.4332, device='cuda:0', grad_fn=<MseLossBackward>) 545\n",
      "tensor(0.4108, device='cuda:0', grad_fn=<MseLossBackward>) 546\n",
      "tensor(0.4482, device='cuda:0', grad_fn=<MseLossBackward>) 547\n",
      "tensor(0.4253, device='cuda:0', grad_fn=<MseLossBackward>) 548\n",
      "tensor(0.4244, device='cuda:0', grad_fn=<MseLossBackward>) 549\n",
      "tensor(0.4172, device='cuda:0', grad_fn=<MseLossBackward>) 550\n",
      "tensor(0.4283, device='cuda:0', grad_fn=<MseLossBackward>) 551\n",
      "tensor(0.4432, device='cuda:0', grad_fn=<MseLossBackward>) 552\n",
      "tensor(0.4117, device='cuda:0', grad_fn=<MseLossBackward>) 553\n",
      "tensor(0.4248, device='cuda:0', grad_fn=<MseLossBackward>) 554\n",
      "tensor(0.4114, device='cuda:0', grad_fn=<MseLossBackward>) 555\n",
      "tensor(0.4502, device='cuda:0', grad_fn=<MseLossBackward>) 556\n",
      "tensor(0.4167, device='cuda:0', grad_fn=<MseLossBackward>) 557\n",
      "tensor(0.4222, device='cuda:0', grad_fn=<MseLossBackward>) 558\n",
      "tensor(0.4128, device='cuda:0', grad_fn=<MseLossBackward>) 559\n",
      "tensor(0.4318, device='cuda:0', grad_fn=<MseLossBackward>) 560\n",
      "tensor(0.4336, device='cuda:0', grad_fn=<MseLossBackward>) 561\n",
      "tensor(0.4099, device='cuda:0', grad_fn=<MseLossBackward>) 562\n",
      "tensor(0.4199, device='cuda:0', grad_fn=<MseLossBackward>) 563\n",
      "tensor(0.4116, device='cuda:0', grad_fn=<MseLossBackward>) 564\n",
      "tensor(0.4472, device='cuda:0', grad_fn=<MseLossBackward>) 565\n",
      "tensor(0.4096, device='cuda:0', grad_fn=<MseLossBackward>) 566\n",
      "tensor(0.4221, device='cuda:0', grad_fn=<MseLossBackward>) 567\n",
      "tensor(0.4085, device='cuda:0', grad_fn=<MseLossBackward>) 568\n",
      "tensor(0.4371, device='cuda:0', grad_fn=<MseLossBackward>) 569\n",
      "tensor(0.4265, device='cuda:0', grad_fn=<MseLossBackward>) 570\n",
      "tensor(0.4093, device='cuda:0', grad_fn=<MseLossBackward>) 571\n",
      "tensor(0.4157, device='cuda:0', grad_fn=<MseLossBackward>) 572\n",
      "tensor(0.4119, device='cuda:0', grad_fn=<MseLossBackward>) 573\n",
      "tensor(0.4440, device='cuda:0', grad_fn=<MseLossBackward>) 574\n",
      "tensor(0.4050, device='cuda:0', grad_fn=<MseLossBackward>) 575\n",
      "tensor(0.4206, device='cuda:0', grad_fn=<MseLossBackward>) 576\n",
      "tensor(0.4051, device='cuda:0', grad_fn=<MseLossBackward>) 577\n",
      "tensor(0.4380, device='cuda:0', grad_fn=<MseLossBackward>) 578\n",
      "tensor(0.4213, device='cuda:0', grad_fn=<MseLossBackward>) 579\n",
      "tensor(0.4076, device='cuda:0', grad_fn=<MseLossBackward>) 580\n",
      "tensor(0.4128, device='cuda:0', grad_fn=<MseLossBackward>) 581\n",
      "tensor(0.4104, device='cuda:0', grad_fn=<MseLossBackward>) 582\n",
      "tensor(0.4419, device='cuda:0', grad_fn=<MseLossBackward>) 583\n",
      "tensor(0.4011, device='cuda:0', grad_fn=<MseLossBackward>) 584\n",
      "tensor(0.4182, device='cuda:0', grad_fn=<MseLossBackward>) 585\n",
      "tensor(0.4030, device='cuda:0', grad_fn=<MseLossBackward>) 586\n",
      "tensor(0.4353, device='cuda:0', grad_fn=<MseLossBackward>) 587\n",
      "tensor(0.4179, device='cuda:0', grad_fn=<MseLossBackward>) 588\n",
      "tensor(0.4028, device='cuda:0', grad_fn=<MseLossBackward>) 589\n",
      "tensor(0.4115, device='cuda:0', grad_fn=<MseLossBackward>) 590\n",
      "tensor(0.4078, device='cuda:0', grad_fn=<MseLossBackward>) 591\n",
      "tensor(0.4382, device='cuda:0', grad_fn=<MseLossBackward>) 592\n",
      "tensor(0.3954, device='cuda:0', grad_fn=<MseLossBackward>) 593\n",
      "tensor(0.4149, device='cuda:0', grad_fn=<MseLossBackward>) 594\n",
      "tensor(0.4040, device='cuda:0', grad_fn=<MseLossBackward>) 595\n",
      "tensor(0.4329, device='cuda:0', grad_fn=<MseLossBackward>) 596\n",
      "tensor(0.4117, device='cuda:0', grad_fn=<MseLossBackward>) 597\n",
      "tensor(0.3947, device='cuda:0', grad_fn=<MseLossBackward>) 598\n",
      "tensor(0.4144, device='cuda:0', grad_fn=<MseLossBackward>) 599\n",
      "tensor(0.4136, device='cuda:0', grad_fn=<MseLossBackward>) 600\n",
      "tensor(0.4312, device='cuda:0', grad_fn=<MseLossBackward>) 601\n",
      "tensor(0.3842, device='cuda:0', grad_fn=<MseLossBackward>) 602\n",
      "tensor(0.4115, device='cuda:0', grad_fn=<MseLossBackward>) 603\n",
      "tensor(0.4246, device='cuda:0', grad_fn=<MseLossBackward>) 604\n",
      "tensor(0.4409, device='cuda:0', grad_fn=<MseLossBackward>) 605\n",
      "tensor(0.3996, device='cuda:0', grad_fn=<MseLossBackward>) 606\n",
      "tensor(0.3811, device='cuda:0', grad_fn=<MseLossBackward>) 607\n",
      "tensor(0.4310, device='cuda:0', grad_fn=<MseLossBackward>) 608\n",
      "tensor(0.4712, device='cuda:0', grad_fn=<MseLossBackward>) 609\n",
      "tensor(0.4179, device='cuda:0', grad_fn=<MseLossBackward>) 610\n",
      "tensor(0.3821, device='cuda:0', grad_fn=<MseLossBackward>) 611\n",
      "tensor(0.3892, device='cuda:0', grad_fn=<MseLossBackward>) 612\n",
      "tensor(0.5439, device='cuda:0', grad_fn=<MseLossBackward>) 613\n",
      "tensor(0.4810, device='cuda:0', grad_fn=<MseLossBackward>) 614\n",
      "tensor(0.4768, device='cuda:0', grad_fn=<MseLossBackward>) 615\n",
      "tensor(0.3896, device='cuda:0', grad_fn=<MseLossBackward>) 616\n",
      "tensor(0.4647, device='cuda:0', grad_fn=<MseLossBackward>) 617\n",
      "tensor(0.4080, device='cuda:0', grad_fn=<MseLossBackward>) 618\n",
      "tensor(0.3978, device='cuda:0', grad_fn=<MseLossBackward>) 619\n",
      "tensor(0.3818, device='cuda:0', grad_fn=<MseLossBackward>) 620\n",
      "tensor(0.3873, device='cuda:0', grad_fn=<MseLossBackward>) 621\n",
      "tensor(0.3967, device='cuda:0', grad_fn=<MseLossBackward>) 622\n",
      "tensor(0.4195, device='cuda:0', grad_fn=<MseLossBackward>) 623\n",
      "tensor(0.3744, device='cuda:0', grad_fn=<MseLossBackward>) 624\n",
      "tensor(0.3924, device='cuda:0', grad_fn=<MseLossBackward>) 625\n",
      "tensor(0.3824, device='cuda:0', grad_fn=<MseLossBackward>) 626\n",
      "tensor(0.4003, device='cuda:0', grad_fn=<MseLossBackward>) 627\n",
      "tensor(0.4004, device='cuda:0', grad_fn=<MseLossBackward>) 628\n",
      "tensor(0.3718, device='cuda:0', grad_fn=<MseLossBackward>) 629\n",
      "tensor(0.3898, device='cuda:0', grad_fn=<MseLossBackward>) 630\n",
      "tensor(0.3801, device='cuda:0', grad_fn=<MseLossBackward>) 631\n",
      "tensor(0.4081, device='cuda:0', grad_fn=<MseLossBackward>) 632\n",
      "tensor(0.3817, device='cuda:0', grad_fn=<MseLossBackward>) 633\n",
      "tensor(0.3736, device='cuda:0', grad_fn=<MseLossBackward>) 634\n",
      "tensor(0.3830, device='cuda:0', grad_fn=<MseLossBackward>) 635\n",
      "tensor(0.3841, device='cuda:0', grad_fn=<MseLossBackward>) 636\n",
      "tensor(0.3994, device='cuda:0', grad_fn=<MseLossBackward>) 637\n",
      "tensor(0.3618, device='cuda:0', grad_fn=<MseLossBackward>) 638\n",
      "tensor(0.3676, device='cuda:0', grad_fn=<MseLossBackward>) 639\n",
      "tensor(0.4371, device='cuda:0', grad_fn=<MseLossBackward>) 640\n",
      "tensor(0.4199, device='cuda:0', grad_fn=<MseLossBackward>) 641\n",
      "tensor(0.3852, device='cuda:0', grad_fn=<MseLossBackward>) 642\n",
      "tensor(0.3852, device='cuda:0', grad_fn=<MseLossBackward>) 643\n",
      "tensor(0.3899, device='cuda:0', grad_fn=<MseLossBackward>) 644\n",
      "tensor(0.3598, device='cuda:0', grad_fn=<MseLossBackward>) 645\n",
      "tensor(0.3804, device='cuda:0', grad_fn=<MseLossBackward>) 646\n",
      "tensor(0.3878, device='cuda:0', grad_fn=<MseLossBackward>) 647\n",
      "tensor(0.4026, device='cuda:0', grad_fn=<MseLossBackward>) 648\n",
      "tensor(0.3613, device='cuda:0', grad_fn=<MseLossBackward>) 649\n",
      "tensor(0.3536, device='cuda:0', grad_fn=<MseLossBackward>) 650\n",
      "tensor(0.3702, device='cuda:0', grad_fn=<MseLossBackward>) 651\n",
      "tensor(0.3795, device='cuda:0', grad_fn=<MseLossBackward>) 652\n",
      "tensor(0.3973, device='cuda:0', grad_fn=<MseLossBackward>) 653\n",
      "tensor(0.3479, device='cuda:0', grad_fn=<MseLossBackward>) 654\n",
      "tensor(0.3559, device='cuda:0', grad_fn=<MseLossBackward>) 655\n",
      "tensor(0.3610, device='cuda:0', grad_fn=<MseLossBackward>) 656\n",
      "tensor(0.3655, device='cuda:0', grad_fn=<MseLossBackward>) 657\n",
      "tensor(0.3889, device='cuda:0', grad_fn=<MseLossBackward>) 658\n",
      "tensor(0.3475, device='cuda:0', grad_fn=<MseLossBackward>) 659\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.3603, device='cuda:0', grad_fn=<MseLossBackward>) 660\n",
      "tensor(0.3544, device='cuda:0', grad_fn=<MseLossBackward>) 661\n",
      "tensor(0.3603, device='cuda:0', grad_fn=<MseLossBackward>) 662\n",
      "tensor(0.3842, device='cuda:0', grad_fn=<MseLossBackward>) 663\n",
      "tensor(0.3470, device='cuda:0', grad_fn=<MseLossBackward>) 664\n",
      "tensor(0.3617, device='cuda:0', grad_fn=<MseLossBackward>) 665\n",
      "tensor(0.3505, device='cuda:0', grad_fn=<MseLossBackward>) 666\n",
      "tensor(0.3579, device='cuda:0', grad_fn=<MseLossBackward>) 667\n",
      "tensor(0.3787, device='cuda:0', grad_fn=<MseLossBackward>) 668\n",
      "tensor(0.3444, device='cuda:0', grad_fn=<MseLossBackward>) 669\n",
      "tensor(0.3612, device='cuda:0', grad_fn=<MseLossBackward>) 670\n",
      "tensor(0.3470, device='cuda:0', grad_fn=<MseLossBackward>) 671\n",
      "tensor(0.3568, device='cuda:0', grad_fn=<MseLossBackward>) 672\n",
      "tensor(0.3738, device='cuda:0', grad_fn=<MseLossBackward>) 673\n",
      "tensor(0.3413, device='cuda:0', grad_fn=<MseLossBackward>) 674\n",
      "tensor(0.3601, device='cuda:0', grad_fn=<MseLossBackward>) 675\n",
      "tensor(0.3439, device='cuda:0', grad_fn=<MseLossBackward>) 676\n",
      "tensor(0.3554, device='cuda:0', grad_fn=<MseLossBackward>) 677\n",
      "tensor(0.3704, device='cuda:0', grad_fn=<MseLossBackward>) 678\n",
      "tensor(0.3385, device='cuda:0', grad_fn=<MseLossBackward>) 679\n",
      "tensor(0.3581, device='cuda:0', grad_fn=<MseLossBackward>) 680\n",
      "tensor(0.3411, device='cuda:0', grad_fn=<MseLossBackward>) 681\n",
      "tensor(0.3531, device='cuda:0', grad_fn=<MseLossBackward>) 682\n",
      "tensor(0.3682, device='cuda:0', grad_fn=<MseLossBackward>) 683\n",
      "tensor(0.3358, device='cuda:0', grad_fn=<MseLossBackward>) 684\n",
      "tensor(0.3557, device='cuda:0', grad_fn=<MseLossBackward>) 685\n",
      "tensor(0.3387, device='cuda:0', grad_fn=<MseLossBackward>) 686\n",
      "tensor(0.3496, device='cuda:0', grad_fn=<MseLossBackward>) 687\n",
      "tensor(0.3674, device='cuda:0', grad_fn=<MseLossBackward>) 688\n",
      "tensor(0.3334, device='cuda:0', grad_fn=<MseLossBackward>) 689\n",
      "tensor(0.3527, device='cuda:0', grad_fn=<MseLossBackward>) 690\n",
      "tensor(0.3366, device='cuda:0', grad_fn=<MseLossBackward>) 691\n",
      "tensor(0.3445, device='cuda:0', grad_fn=<MseLossBackward>) 692\n",
      "tensor(0.3682, device='cuda:0', grad_fn=<MseLossBackward>) 693\n",
      "tensor(0.3321, device='cuda:0', grad_fn=<MseLossBackward>) 694\n",
      "tensor(0.3488, device='cuda:0', grad_fn=<MseLossBackward>) 695\n",
      "tensor(0.3348, device='cuda:0', grad_fn=<MseLossBackward>) 696\n",
      "tensor(0.3383, device='cuda:0', grad_fn=<MseLossBackward>) 697\n",
      "tensor(0.3711, device='cuda:0', grad_fn=<MseLossBackward>) 698\n",
      "tensor(0.3334, device='cuda:0', grad_fn=<MseLossBackward>) 699\n",
      "tensor(0.3441, device='cuda:0', grad_fn=<MseLossBackward>) 700\n",
      "tensor(0.3348, device='cuda:0', grad_fn=<MseLossBackward>) 701\n",
      "tensor(0.3300, device='cuda:0', grad_fn=<MseLossBackward>) 702\n",
      "tensor(0.3719, device='cuda:0', grad_fn=<MseLossBackward>) 703\n",
      "tensor(0.3441, device='cuda:0', grad_fn=<MseLossBackward>) 704\n",
      "tensor(0.3191, device='cuda:0', grad_fn=<MseLossBackward>) 705\n",
      "tensor(0.3331, device='cuda:0', grad_fn=<MseLossBackward>) 706\n",
      "tensor(0.3486, device='cuda:0', grad_fn=<MseLossBackward>) 707\n",
      "tensor(0.3344, device='cuda:0', grad_fn=<MseLossBackward>) 708\n",
      "tensor(0.3410, device='cuda:0', grad_fn=<MseLossBackward>) 709\n",
      "tensor(0.3171, device='cuda:0', grad_fn=<MseLossBackward>) 710\n",
      "tensor(0.3472, device='cuda:0', grad_fn=<MseLossBackward>) 711\n",
      "tensor(0.3418, device='cuda:0', grad_fn=<MseLossBackward>) 712\n",
      "tensor(0.3950, device='cuda:0', grad_fn=<MseLossBackward>) 713\n",
      "tensor(0.3452, device='cuda:0', grad_fn=<MseLossBackward>) 714\n",
      "tensor(0.3195, device='cuda:0', grad_fn=<MseLossBackward>) 715\n",
      "tensor(0.3290, device='cuda:0', grad_fn=<MseLossBackward>) 716\n",
      "tensor(0.3112, device='cuda:0', grad_fn=<MseLossBackward>) 717\n",
      "tensor(0.3446, device='cuda:0', grad_fn=<MseLossBackward>) 718\n",
      "tensor(0.3443, device='cuda:0', grad_fn=<MseLossBackward>) 719\n",
      "tensor(0.3251, device='cuda:0', grad_fn=<MseLossBackward>) 720\n",
      "tensor(0.3258, device='cuda:0', grad_fn=<MseLossBackward>) 721\n",
      "tensor(0.3076, device='cuda:0', grad_fn=<MseLossBackward>) 722\n",
      "tensor(0.3273, device='cuda:0', grad_fn=<MseLossBackward>) 723\n",
      "tensor(0.3152, device='cuda:0', grad_fn=<MseLossBackward>) 724\n",
      "tensor(0.3039, device='cuda:0', grad_fn=<MseLossBackward>) 725\n",
      "tensor(0.3229, device='cuda:0', grad_fn=<MseLossBackward>) 726\n",
      "tensor(0.3105, device='cuda:0', grad_fn=<MseLossBackward>) 727\n",
      "tensor(0.3196, device='cuda:0', grad_fn=<MseLossBackward>) 728\n",
      "tensor(0.3035, device='cuda:0', grad_fn=<MseLossBackward>) 729\n",
      "tensor(0.2813, device='cuda:0', grad_fn=<MseLossBackward>) 730\n",
      "tensor(0.3095, device='cuda:0', grad_fn=<MseLossBackward>) 731\n",
      "tensor(0.3229, device='cuda:0', grad_fn=<MseLossBackward>) 732\n",
      "tensor(0.3352, device='cuda:0', grad_fn=<MseLossBackward>) 733\n",
      "tensor(0.3105, device='cuda:0', grad_fn=<MseLossBackward>) 734\n",
      "tensor(0.2793, device='cuda:0', grad_fn=<MseLossBackward>) 735\n",
      "tensor(0.2886, device='cuda:0', grad_fn=<MseLossBackward>) 736\n",
      "tensor(0.3149, device='cuda:0', grad_fn=<MseLossBackward>) 737\n",
      "tensor(0.3349, device='cuda:0', grad_fn=<MseLossBackward>) 738\n",
      "tensor(0.3317, device='cuda:0', grad_fn=<MseLossBackward>) 739\n",
      "tensor(0.2775, device='cuda:0', grad_fn=<MseLossBackward>) 740\n",
      "tensor(0.2732, device='cuda:0', grad_fn=<MseLossBackward>) 741\n",
      "tensor(0.2988, device='cuda:0', grad_fn=<MseLossBackward>) 742\n",
      "tensor(0.2775, device='cuda:0', grad_fn=<MseLossBackward>) 743\n",
      "tensor(0.3012, device='cuda:0', grad_fn=<MseLossBackward>) 744\n",
      "tensor(0.2973, device='cuda:0', grad_fn=<MseLossBackward>) 745\n",
      "tensor(0.2713, device='cuda:0', grad_fn=<MseLossBackward>) 746\n",
      "tensor(0.2917, device='cuda:0', grad_fn=<MseLossBackward>) 747\n",
      "tensor(0.2799, device='cuda:0', grad_fn=<MseLossBackward>) 748\n",
      "tensor(0.2699, device='cuda:0', grad_fn=<MseLossBackward>) 749\n",
      "tensor(0.3005, device='cuda:0', grad_fn=<MseLossBackward>) 750\n",
      "tensor(0.2793, device='cuda:0', grad_fn=<MseLossBackward>) 751\n",
      "tensor(0.2781, device='cuda:0', grad_fn=<MseLossBackward>) 752\n",
      "tensor(0.2894, device='cuda:0', grad_fn=<MseLossBackward>) 753\n",
      "tensor(0.2671, device='cuda:0', grad_fn=<MseLossBackward>) 754\n",
      "tensor(0.2819, device='cuda:0', grad_fn=<MseLossBackward>) 755\n",
      "tensor(0.2903, device='cuda:0', grad_fn=<MseLossBackward>) 756\n",
      "tensor(0.2681, device='cuda:0', grad_fn=<MseLossBackward>) 757\n",
      "tensor(0.2874, device='cuda:0', grad_fn=<MseLossBackward>) 758\n",
      "tensor(0.2775, device='cuda:0', grad_fn=<MseLossBackward>) 759\n",
      "tensor(0.2624, device='cuda:0', grad_fn=<MseLossBackward>) 760\n",
      "tensor(0.2924, device='cuda:0', grad_fn=<MseLossBackward>) 761\n",
      "tensor(0.2747, device='cuda:0', grad_fn=<MseLossBackward>) 762\n",
      "tensor(0.2678, device='cuda:0', grad_fn=<MseLossBackward>) 763\n",
      "tensor(0.2854, device='cuda:0', grad_fn=<MseLossBackward>) 764\n",
      "tensor(0.2640, device='cuda:0', grad_fn=<MseLossBackward>) 765\n",
      "tensor(0.2684, device='cuda:0', grad_fn=<MseLossBackward>) 766\n",
      "tensor(0.2884, device='cuda:0', grad_fn=<MseLossBackward>) 767\n",
      "tensor(0.2624, device='cuda:0', grad_fn=<MseLossBackward>) 768\n",
      "tensor(0.2750, device='cuda:0', grad_fn=<MseLossBackward>) 769\n",
      "tensor(0.2772, device='cuda:0', grad_fn=<MseLossBackward>) 770\n",
      "tensor(0.2553, device='cuda:0', grad_fn=<MseLossBackward>) 771\n",
      "tensor(0.2799, device='cuda:0', grad_fn=<MseLossBackward>) 772\n",
      "tensor(0.2784, device='cuda:0', grad_fn=<MseLossBackward>) 773\n",
      "tensor(0.2573, device='cuda:0', grad_fn=<MseLossBackward>) 774\n",
      "tensor(0.2799, device='cuda:0', grad_fn=<MseLossBackward>) 775\n",
      "tensor(0.2675, device='cuda:0', grad_fn=<MseLossBackward>) 776\n",
      "tensor(0.2509, device='cuda:0', grad_fn=<MseLossBackward>) 777\n",
      "tensor(0.2832, device='cuda:0', grad_fn=<MseLossBackward>) 778\n",
      "tensor(0.2681, device='cuda:0', grad_fn=<MseLossBackward>) 779\n",
      "tensor(0.2568, device='cuda:0', grad_fn=<MseLossBackward>) 780\n",
      "tensor(0.2764, device='cuda:0', grad_fn=<MseLossBackward>) 781\n",
      "tensor(0.3411, device='cuda:0', grad_fn=<MseLossBackward>) 782\n",
      "tensor(0.2494, device='cuda:0', grad_fn=<MseLossBackward>) 783\n",
      "tensor(0.2587, device='cuda:0', grad_fn=<MseLossBackward>) 784\n",
      "tensor(0.2700, device='cuda:0', grad_fn=<MseLossBackward>) 785\n",
      "tensor(0.2624, device='cuda:0', grad_fn=<MseLossBackward>) 786\n",
      "tensor(0.2608, device='cuda:0', grad_fn=<MseLossBackward>) 787\n",
      "tensor(0.2885, device='cuda:0', grad_fn=<MseLossBackward>) 788\n",
      "tensor(0.2591, device='cuda:0', grad_fn=<MseLossBackward>) 789\n",
      "tensor(0.2478, device='cuda:0', grad_fn=<MseLossBackward>) 790\n",
      "tensor(0.2576, device='cuda:0', grad_fn=<MseLossBackward>) 791\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.2502, device='cuda:0', grad_fn=<MseLossBackward>) 792\n",
      "tensor(0.2817, device='cuda:0', grad_fn=<MseLossBackward>) 793\n",
      "tensor(0.2476, device='cuda:0', grad_fn=<MseLossBackward>) 794\n",
      "tensor(0.2578, device='cuda:0', grad_fn=<MseLossBackward>) 795\n",
      "tensor(0.2576, device='cuda:0', grad_fn=<MseLossBackward>) 796\n",
      "tensor(0.2502, device='cuda:0', grad_fn=<MseLossBackward>) 797\n",
      "tensor(0.2283, device='cuda:0', grad_fn=<MseLossBackward>) 798\n",
      "tensor(0.2445, device='cuda:0', grad_fn=<MseLossBackward>) 799\n",
      "tensor(0.2596, device='cuda:0', grad_fn=<MseLossBackward>) 800\n",
      "tensor(0.2618, device='cuda:0', grad_fn=<MseLossBackward>) 801\n",
      "tensor(0.2526, device='cuda:0', grad_fn=<MseLossBackward>) 802\n",
      "tensor(0.2773, device='cuda:0', grad_fn=<MseLossBackward>) 803\n",
      "tensor(0.2649, device='cuda:0', grad_fn=<MseLossBackward>) 804\n",
      "tensor(0.3176, device='cuda:0', grad_fn=<MseLossBackward>) 805\n",
      "tensor(0.2399, device='cuda:0', grad_fn=<MseLossBackward>) 806\n",
      "tensor(0.2301, device='cuda:0', grad_fn=<MseLossBackward>) 807\n",
      "tensor(0.2496, device='cuda:0', grad_fn=<MseLossBackward>) 808\n",
      "tensor(0.2359, device='cuda:0', grad_fn=<MseLossBackward>) 809\n",
      "tensor(0.2531, device='cuda:0', grad_fn=<MseLossBackward>) 810\n",
      "tensor(0.2146, device='cuda:0', grad_fn=<MseLossBackward>) 811\n",
      "tensor(0.2357, device='cuda:0', grad_fn=<MseLossBackward>) 812\n",
      "tensor(0.2362, device='cuda:0', grad_fn=<MseLossBackward>) 813\n",
      "tensor(0.2075, device='cuda:0', grad_fn=<MseLossBackward>) 814\n",
      "tensor(0.2255, device='cuda:0', grad_fn=<MseLossBackward>) 815\n",
      "tensor(0.1970, device='cuda:0', grad_fn=<MseLossBackward>) 816\n",
      "tensor(0.2094, device='cuda:0', grad_fn=<MseLossBackward>) 817\n",
      "tensor(0.2077, device='cuda:0', grad_fn=<MseLossBackward>) 818\n",
      "tensor(0.2407, device='cuda:0', grad_fn=<MseLossBackward>) 819\n",
      "tensor(0.2285, device='cuda:0', grad_fn=<MseLossBackward>) 820\n",
      "tensor(0.1999, device='cuda:0', grad_fn=<MseLossBackward>) 821\n",
      "tensor(0.2109, device='cuda:0', grad_fn=<MseLossBackward>) 822\n",
      "tensor(0.1948, device='cuda:0', grad_fn=<MseLossBackward>) 823\n",
      "tensor(0.1809, device='cuda:0', grad_fn=<MseLossBackward>) 824\n",
      "tensor(0.1980, device='cuda:0', grad_fn=<MseLossBackward>) 825\n",
      "tensor(0.1817, device='cuda:0', grad_fn=<MseLossBackward>) 826\n",
      "tensor(0.1747, device='cuda:0', grad_fn=<MseLossBackward>) 827\n",
      "tensor(0.1616, device='cuda:0', grad_fn=<MseLossBackward>) 828\n",
      "tensor(0.1649, device='cuda:0', grad_fn=<MseLossBackward>) 829\n",
      "tensor(0.1778, device='cuda:0', grad_fn=<MseLossBackward>) 830\n",
      "tensor(0.1726, device='cuda:0', grad_fn=<MseLossBackward>) 831\n",
      "tensor(0.1988, device='cuda:0', grad_fn=<MseLossBackward>) 832\n",
      "tensor(0.1763, device='cuda:0', grad_fn=<MseLossBackward>) 833\n",
      "tensor(0.1954, device='cuda:0', grad_fn=<MseLossBackward>) 834\n",
      "tensor(0.1451, device='cuda:0', grad_fn=<MseLossBackward>) 835\n",
      "tensor(0.1571, device='cuda:0', grad_fn=<MseLossBackward>) 836\n",
      "tensor(0.1403, device='cuda:0', grad_fn=<MseLossBackward>) 837\n",
      "tensor(0.1528, device='cuda:0', grad_fn=<MseLossBackward>) 838\n",
      "tensor(0.1265, device='cuda:0', grad_fn=<MseLossBackward>) 839\n",
      "tensor(0.1217, device='cuda:0', grad_fn=<MseLossBackward>) 840\n",
      "tensor(0.1180, device='cuda:0', grad_fn=<MseLossBackward>) 841\n",
      "tensor(0.1160, device='cuda:0', grad_fn=<MseLossBackward>) 842\n",
      "tensor(0.1345, device='cuda:0', grad_fn=<MseLossBackward>) 843\n",
      "tensor(0.1260, device='cuda:0', grad_fn=<MseLossBackward>) 844\n",
      "tensor(0.1122, device='cuda:0', grad_fn=<MseLossBackward>) 845\n",
      "tensor(0.1190, device='cuda:0', grad_fn=<MseLossBackward>) 846\n",
      "tensor(0.1234, device='cuda:0', grad_fn=<MseLossBackward>) 847\n",
      "tensor(0.1123, device='cuda:0', grad_fn=<MseLossBackward>) 848\n",
      "tensor(0.1178, device='cuda:0', grad_fn=<MseLossBackward>) 849\n",
      "tensor(0.1469, device='cuda:0', grad_fn=<MseLossBackward>) 850\n",
      "tensor(0.1873, device='cuda:0', grad_fn=<MseLossBackward>) 851\n",
      "tensor(0.2198, device='cuda:0', grad_fn=<MseLossBackward>) 852\n",
      "tensor(0.2438, device='cuda:0', grad_fn=<MseLossBackward>) 853\n",
      "tensor(0.2327, device='cuda:0', grad_fn=<MseLossBackward>) 854\n",
      "tensor(0.2087, device='cuda:0', grad_fn=<MseLossBackward>) 855\n",
      "tensor(0.1906, device='cuda:0', grad_fn=<MseLossBackward>) 856\n",
      "tensor(0.1772, device='cuda:0', grad_fn=<MseLossBackward>) 857\n",
      "tensor(0.1683, device='cuda:0', grad_fn=<MseLossBackward>) 858\n",
      "tensor(0.2174, device='cuda:0', grad_fn=<MseLossBackward>) 859\n",
      "tensor(0.3690, device='cuda:0', grad_fn=<MseLossBackward>) 860\n",
      "tensor(0.3707, device='cuda:0', grad_fn=<MseLossBackward>) 861\n",
      "tensor(0.1859, device='cuda:0', grad_fn=<MseLossBackward>) 862\n",
      "tensor(0.1214, device='cuda:0', grad_fn=<MseLossBackward>) 863\n",
      "tensor(0.1498, device='cuda:0', grad_fn=<MseLossBackward>) 864\n",
      "tensor(0.1923, device='cuda:0', grad_fn=<MseLossBackward>) 865\n",
      "tensor(0.1566, device='cuda:0', grad_fn=<MseLossBackward>) 866\n",
      "tensor(0.1271, device='cuda:0', grad_fn=<MseLossBackward>) 867\n",
      "tensor(0.1230, device='cuda:0', grad_fn=<MseLossBackward>) 868\n",
      "tensor(0.1330, device='cuda:0', grad_fn=<MseLossBackward>) 869\n",
      "tensor(0.1424, device='cuda:0', grad_fn=<MseLossBackward>) 870\n",
      "tensor(0.1431, device='cuda:0', grad_fn=<MseLossBackward>) 871\n",
      "tensor(0.1360, device='cuda:0', grad_fn=<MseLossBackward>) 872\n",
      "tensor(0.1288, device='cuda:0', grad_fn=<MseLossBackward>) 873\n",
      "tensor(0.1251, device='cuda:0', grad_fn=<MseLossBackward>) 874\n",
      "tensor(0.1249, device='cuda:0', grad_fn=<MseLossBackward>) 875\n",
      "tensor(0.1282, device='cuda:0', grad_fn=<MseLossBackward>) 876\n",
      "tensor(0.1352, device='cuda:0', grad_fn=<MseLossBackward>) 877\n",
      "tensor(0.1452, device='cuda:0', grad_fn=<MseLossBackward>) 878\n",
      "tensor(0.1563, device='cuda:0', grad_fn=<MseLossBackward>) 879\n",
      "tensor(0.1639, device='cuda:0', grad_fn=<MseLossBackward>) 880\n",
      "tensor(0.1640, device='cuda:0', grad_fn=<MseLossBackward>) 881\n",
      "tensor(0.1559, device='cuda:0', grad_fn=<MseLossBackward>) 882\n",
      "tensor(0.1438, device='cuda:0', grad_fn=<MseLossBackward>) 883\n",
      "tensor(0.1309, device='cuda:0', grad_fn=<MseLossBackward>) 884\n",
      "tensor(0.1210, device='cuda:0', grad_fn=<MseLossBackward>) 885\n",
      "tensor(0.1148, device='cuda:0', grad_fn=<MseLossBackward>) 886\n",
      "tensor(0.1123, device='cuda:0', grad_fn=<MseLossBackward>) 887\n",
      "tensor(0.1118, device='cuda:0', grad_fn=<MseLossBackward>) 888\n",
      "tensor(0.1105, device='cuda:0', grad_fn=<MseLossBackward>) 889\n",
      "tensor(0.1088, device='cuda:0', grad_fn=<MseLossBackward>) 890\n",
      "tensor(0.1107, device='cuda:0', grad_fn=<MseLossBackward>) 891\n",
      "tensor(0.1189, device='cuda:0', grad_fn=<MseLossBackward>) 892\n",
      "tensor(0.1273, device='cuda:0', grad_fn=<MseLossBackward>) 893\n",
      "tensor(0.1218, device='cuda:0', grad_fn=<MseLossBackward>) 894\n",
      "tensor(0.1159, device='cuda:0', grad_fn=<MseLossBackward>) 895\n",
      "tensor(0.1174, device='cuda:0', grad_fn=<MseLossBackward>) 896\n",
      "tensor(0.1216, device='cuda:0', grad_fn=<MseLossBackward>) 897\n",
      "tensor(0.1243, device='cuda:0', grad_fn=<MseLossBackward>) 898\n",
      "tensor(0.1240, device='cuda:0', grad_fn=<MseLossBackward>) 899\n",
      "tensor(0.1224, device='cuda:0', grad_fn=<MseLossBackward>) 900\n",
      "tensor(0.1246, device='cuda:0', grad_fn=<MseLossBackward>) 901\n",
      "tensor(0.1331, device='cuda:0', grad_fn=<MseLossBackward>) 902\n",
      "tensor(0.1453, device='cuda:0', grad_fn=<MseLossBackward>) 903\n",
      "tensor(0.1606, device='cuda:0', grad_fn=<MseLossBackward>) 904\n",
      "tensor(0.1690, device='cuda:0', grad_fn=<MseLossBackward>) 905\n",
      "tensor(0.1631, device='cuda:0', grad_fn=<MseLossBackward>) 906\n",
      "tensor(0.1501, device='cuda:0', grad_fn=<MseLossBackward>) 907\n",
      "tensor(0.1392, device='cuda:0', grad_fn=<MseLossBackward>) 908\n",
      "tensor(0.1326, device='cuda:0', grad_fn=<MseLossBackward>) 909\n",
      "tensor(0.1301, device='cuda:0', grad_fn=<MseLossBackward>) 910\n",
      "tensor(0.1305, device='cuda:0', grad_fn=<MseLossBackward>) 911\n",
      "tensor(0.1323, device='cuda:0', grad_fn=<MseLossBackward>) 912\n",
      "tensor(0.1341, device='cuda:0', grad_fn=<MseLossBackward>) 913\n",
      "tensor(0.1351, device='cuda:0', grad_fn=<MseLossBackward>) 914\n",
      "tensor(0.1347, device='cuda:0', grad_fn=<MseLossBackward>) 915\n",
      "tensor(0.1333, device='cuda:0', grad_fn=<MseLossBackward>) 916\n",
      "tensor(0.1320, device='cuda:0', grad_fn=<MseLossBackward>) 917\n",
      "tensor(0.1317, device='cuda:0', grad_fn=<MseLossBackward>) 918\n",
      "tensor(0.1327, device='cuda:0', grad_fn=<MseLossBackward>) 919\n",
      "tensor(0.1351, device='cuda:0', grad_fn=<MseLossBackward>) 920\n",
      "tensor(0.1389, device='cuda:0', grad_fn=<MseLossBackward>) 921\n",
      "tensor(0.1439, device='cuda:0', grad_fn=<MseLossBackward>) 922\n",
      "tensor(0.1497, device='cuda:0', grad_fn=<MseLossBackward>) 923\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.1558, device='cuda:0', grad_fn=<MseLossBackward>) 924\n",
      "tensor(0.1614, device='cuda:0', grad_fn=<MseLossBackward>) 925\n",
      "tensor(0.1657, device='cuda:0', grad_fn=<MseLossBackward>) 926\n",
      "tensor(0.1680, device='cuda:0', grad_fn=<MseLossBackward>) 927\n",
      "tensor(0.1681, device='cuda:0', grad_fn=<MseLossBackward>) 928\n",
      "tensor(0.1657, device='cuda:0', grad_fn=<MseLossBackward>) 929\n",
      "tensor(0.1612, device='cuda:0', grad_fn=<MseLossBackward>) 930\n",
      "tensor(0.1554, device='cuda:0', grad_fn=<MseLossBackward>) 931\n",
      "tensor(0.1493, device='cuda:0', grad_fn=<MseLossBackward>) 932\n",
      "tensor(0.1439, device='cuda:0', grad_fn=<MseLossBackward>) 933\n",
      "tensor(0.1396, device='cuda:0', grad_fn=<MseLossBackward>) 934\n",
      "tensor(0.1364, device='cuda:0', grad_fn=<MseLossBackward>) 935\n",
      "tensor(0.1344, device='cuda:0', grad_fn=<MseLossBackward>) 936\n",
      "tensor(0.1333, device='cuda:0', grad_fn=<MseLossBackward>) 937\n",
      "tensor(0.1332, device='cuda:0', grad_fn=<MseLossBackward>) 938\n",
      "tensor(0.1336, device='cuda:0', grad_fn=<MseLossBackward>) 939\n",
      "tensor(0.1343, device='cuda:0', grad_fn=<MseLossBackward>) 940\n",
      "tensor(0.1350, device='cuda:0', grad_fn=<MseLossBackward>) 941\n",
      "tensor(0.1355, device='cuda:0', grad_fn=<MseLossBackward>) 942\n",
      "tensor(0.1355, device='cuda:0', grad_fn=<MseLossBackward>) 943\n",
      "tensor(0.1350, device='cuda:0', grad_fn=<MseLossBackward>) 944\n",
      "tensor(0.1344, device='cuda:0', grad_fn=<MseLossBackward>) 945\n",
      "tensor(0.1340, device='cuda:0', grad_fn=<MseLossBackward>) 946\n",
      "tensor(0.1339, device='cuda:0', grad_fn=<MseLossBackward>) 947\n",
      "tensor(0.1343, device='cuda:0', grad_fn=<MseLossBackward>) 948\n",
      "tensor(0.1356, device='cuda:0', grad_fn=<MseLossBackward>) 949\n",
      "tensor(0.1377, device='cuda:0', grad_fn=<MseLossBackward>) 950\n",
      "tensor(0.1406, device='cuda:0', grad_fn=<MseLossBackward>) 951\n",
      "tensor(0.1442, device='cuda:0', grad_fn=<MseLossBackward>) 952\n",
      "tensor(0.1483, device='cuda:0', grad_fn=<MseLossBackward>) 953\n",
      "tensor(0.1525, device='cuda:0', grad_fn=<MseLossBackward>) 954\n",
      "tensor(0.1566, device='cuda:0', grad_fn=<MseLossBackward>) 955\n",
      "tensor(0.1601, device='cuda:0', grad_fn=<MseLossBackward>) 956\n",
      "tensor(0.1627, device='cuda:0', grad_fn=<MseLossBackward>) 957\n",
      "tensor(0.1640, device='cuda:0', grad_fn=<MseLossBackward>) 958\n",
      "tensor(0.1639, device='cuda:0', grad_fn=<MseLossBackward>) 959\n",
      "tensor(0.1622, device='cuda:0', grad_fn=<MseLossBackward>) 960\n",
      "tensor(0.1595, device='cuda:0', grad_fn=<MseLossBackward>) 961\n",
      "tensor(0.1563, device='cuda:0', grad_fn=<MseLossBackward>) 962\n",
      "tensor(0.1527, device='cuda:0', grad_fn=<MseLossBackward>) 963\n",
      "tensor(0.1475, device='cuda:0', grad_fn=<MseLossBackward>) 964\n",
      "tensor(0.1410, device='cuda:0', grad_fn=<MseLossBackward>) 965\n",
      "tensor(0.1359, device='cuda:0', grad_fn=<MseLossBackward>) 966\n",
      "tensor(0.1327, device='cuda:0', grad_fn=<MseLossBackward>) 967\n",
      "tensor(0.1306, device='cuda:0', grad_fn=<MseLossBackward>) 968\n",
      "tensor(0.1296, device='cuda:0', grad_fn=<MseLossBackward>) 969\n",
      "tensor(0.1291, device='cuda:0', grad_fn=<MseLossBackward>) 970\n",
      "tensor(0.1288, device='cuda:0', grad_fn=<MseLossBackward>) 971\n",
      "tensor(0.1289, device='cuda:0', grad_fn=<MseLossBackward>) 972\n",
      "tensor(0.1291, device='cuda:0', grad_fn=<MseLossBackward>) 973\n",
      "tensor(0.1295, device='cuda:0', grad_fn=<MseLossBackward>) 974\n",
      "tensor(0.1299, device='cuda:0', grad_fn=<MseLossBackward>) 975\n",
      "tensor(0.1301, device='cuda:0', grad_fn=<MseLossBackward>) 976\n",
      "tensor(0.1302, device='cuda:0', grad_fn=<MseLossBackward>) 977\n",
      "tensor(0.1302, device='cuda:0', grad_fn=<MseLossBackward>) 978\n",
      "tensor(0.1302, device='cuda:0', grad_fn=<MseLossBackward>) 979\n",
      "tensor(0.1301, device='cuda:0', grad_fn=<MseLossBackward>) 980\n",
      "tensor(0.1305, device='cuda:0', grad_fn=<MseLossBackward>) 981\n",
      "tensor(0.1314, device='cuda:0', grad_fn=<MseLossBackward>) 982\n",
      "tensor(0.1330, device='cuda:0', grad_fn=<MseLossBackward>) 983\n",
      "tensor(0.1352, device='cuda:0', grad_fn=<MseLossBackward>) 984\n",
      "tensor(0.1379, device='cuda:0', grad_fn=<MseLossBackward>) 985\n",
      "tensor(0.1411, device='cuda:0', grad_fn=<MseLossBackward>) 986\n",
      "tensor(0.1447, device='cuda:0', grad_fn=<MseLossBackward>) 987\n",
      "tensor(0.1484, device='cuda:0', grad_fn=<MseLossBackward>) 988\n",
      "tensor(0.1521, device='cuda:0', grad_fn=<MseLossBackward>) 989\n",
      "tensor(0.1554, device='cuda:0', grad_fn=<MseLossBackward>) 990\n",
      "tensor(0.1580, device='cuda:0', grad_fn=<MseLossBackward>) 991\n",
      "tensor(0.1598, device='cuda:0', grad_fn=<MseLossBackward>) 992\n",
      "tensor(0.1604, device='cuda:0', grad_fn=<MseLossBackward>) 993\n",
      "tensor(0.1600, device='cuda:0', grad_fn=<MseLossBackward>) 994\n",
      "tensor(0.1583, device='cuda:0', grad_fn=<MseLossBackward>) 995\n",
      "tensor(0.1556, device='cuda:0', grad_fn=<MseLossBackward>) 996\n",
      "tensor(0.1520, device='cuda:0', grad_fn=<MseLossBackward>) 997\n",
      "tensor(0.1480, device='cuda:0', grad_fn=<MseLossBackward>) 998\n",
      "tensor(0.1439, device='cuda:0', grad_fn=<MseLossBackward>) 999\n"
     ]
    }
   ],
   "source": [
    "#trainingloop\n",
    "\n",
    "import torch.optim as optim\n",
    "\n",
    "net2=Net()\n",
    "\n",
    "lr = 0.001\n",
    "batchsize=20\n",
    "\n",
    "batches=len(trdttensor)/batchsize\n",
    "\n",
    "epochs=1000\n",
    "\n",
    "# create your optimizer\n",
    "optimizer = optim.Adam(net2.parameters(), lr)\n",
    "\n",
    "\n",
    "for i in range(epochs):\n",
    "    \n",
    "    for j in range(int(batches)):\n",
    "        \n",
    "        #forward pass\n",
    "        out=net2(trdttensor[j:j+batchsize,:].type(dtype))\n",
    "\n",
    "        #compute loss\n",
    "        criterion = nn.MSELoss()\n",
    "        loss = criterion(out,results1tensor[j:j+batchsize]).type(dtype)\n",
    "\n",
    "\n",
    "        #backprop loss i.e. find dloss/dparam for each parameter and store.\n",
    "        loss.backward(retain_graph=True)\n",
    "\n",
    "        #clip gradients\n",
    "        torch.nn.utils.clip_grad_norm_(net2.parameters(), 10.0)\n",
    "        \n",
    "        #use optimiser to update\n",
    "        optimizer.step()\n",
    "    c=nn.MSELoss()\n",
    "    print(c(torch.reshape(net2(trdttensor),[200]),results1tensor),i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.03190409490732618\n"
     ]
    }
   ],
   "source": [
    "#Evaluation\n",
    "\n",
    "import joblib\n",
    "\n",
    "m=joblib.load('C:\\\\Users\\\\Ant Pc\\\\GitHub\\\\pROJ\\\\Geneexpression0\\\\12.pkl')\n",
    "\n",
    "pred=pd.DataFrame(m.predict(tstdt))\n",
    "\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "tstdttensor=torch.from_numpy(tstdt.values).type(dtype).cuda()\n",
    "print(mean_squared_error(pred,net2(tstdttensor).cpu().detach().numpy()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Whoa! This almost (1/30)th of the MSE of Conventional Regression Neural Network. So, this conncludes my implementation of this paper. Further, you can try changing the RP matrix and also experimenting with different values of smaller dimensional space.\n",
    "\n",
    "So, next time you come across High dimensional non-spatial data, you know where to come.\n",
    "\n",
    "See you next time. Cheers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}