2022-11-18T20:35:02.3591222Z Requested labels: windows-latest 2022-11-18T20:35:02.3591322Z Job defined at: pytorch/serve/.github/workflows/regression_tests_cpu.yml@refs/heads/upgrade_pytorch_113 2022-11-18T20:35:02.3591347Z Waiting for a runner to pick up this job... 2022-11-18T20:35:03.9592204Z Job is waiting for a hosted runner to come online. 2022-11-18T20:35:17.9535067Z Job is about to start running on the hosted runner: GitHub Actions 133 (hosted) 2022-11-18T20:35:24.0146107Z Current runner version: '2.299.1' 2022-11-18T20:35:24.0171539Z ##[group]Operating System 2022-11-18T20:35:24.0172169Z Microsoft Windows Server 2022 2022-11-18T20:35:24.0172509Z 10.0.20348 2022-11-18T20:35:24.0172785Z Datacenter 2022-11-18T20:35:24.0173019Z ##[endgroup] 2022-11-18T20:35:24.0173319Z ##[group]Runner Image 2022-11-18T20:35:24.0173628Z Image: windows-2022 2022-11-18T20:35:24.0173875Z Version: 20221027.1 2022-11-18T20:35:24.0174389Z Included Software: https://github.com/actions/runner-images/blob/win22/20221027.1/images/win/Windows2022-Readme.md 2022-11-18T20:35:24.0174976Z Image Release: https://github.com/actions/runner-images/releases/tag/win22%2F20221027.1 2022-11-18T20:35:24.0175372Z ##[endgroup] 2022-11-18T20:35:24.0175683Z ##[group]Runner Image Provisioner 2022-11-18T20:35:24.0175984Z 2.0.91.1 2022-11-18T20:35:24.0176221Z ##[endgroup] 2022-11-18T20:35:24.0177143Z ##[group]GITHUB_TOKEN Permissions 2022-11-18T20:35:24.0177673Z Actions: write 2022-11-18T20:35:24.0178106Z Checks: write 2022-11-18T20:35:24.0178425Z Contents: write 2022-11-18T20:35:24.0178731Z Deployments: write 2022-11-18T20:35:24.0179029Z Discussions: write 2022-11-18T20:35:24.0179276Z Issues: write 2022-11-18T20:35:24.0179576Z Metadata: read 2022-11-18T20:35:24.0179869Z Packages: write 2022-11-18T20:35:24.0180112Z Pages: write 2022-11-18T20:35:24.0180399Z PullRequests: write 2022-11-18T20:35:24.0180721Z RepositoryProjects: write 2022-11-18T20:35:24.0181068Z SecurityEvents: write 2022-11-18T20:35:24.0181375Z Statuses: write 2022-11-18T20:35:24.0181653Z ##[endgroup] 2022-11-18T20:35:24.0185267Z Secret source: Actions 2022-11-18T20:35:24.0185728Z Prepare workflow directory 2022-11-18T20:35:24.0998810Z Prepare all required actions 2022-11-18T20:35:24.1180501Z Getting action download info 2022-11-18T20:35:24.3733885Z Download action repository 'actions/setup-python@v3' (SHA:98f2ad02fd48d057ee3b4d4f66525b231c3e52b6) 2022-11-18T20:35:24.7402736Z Download action repository 'actions/setup-java@v3' (SHA:de1bb2b0c5634f0fc4438d7aa9944e68f9bf86cc) 2022-11-18T20:35:25.0704800Z Download action repository 'actions/checkout@v3' (SHA:93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8) 2022-11-18T20:35:25.7092715Z ##[group]Run actions/setup-python@v3 2022-11-18T20:35:25.7093185Z with: 2022-11-18T20:35:25.7093441Z python-version: 3.8 2022-11-18T20:35:25.7093713Z architecture: x64 2022-11-18T20:35:25.7094245Z token: *** 2022-11-18T20:35:25.7094470Z ##[endgroup] 2022-11-18T20:35:32.0794733Z ##[warning]The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ 2022-11-18T20:35:32.0805175Z Successfully setup CPython (3.8.10) 2022-11-18T20:35:32.1257696Z ##[group]Run actions/setup-java@v3 2022-11-18T20:35:32.1258134Z with: 2022-11-18T20:35:32.1258425Z distribution: zulu 2022-11-18T20:35:32.1258710Z java-version: 17 2022-11-18T20:35:32.1259049Z java-package: jdk 2022-11-18T20:35:32.1259323Z check-latest: false 2022-11-18T20:35:32.1259629Z server-id: github 2022-11-18T20:35:32.1259922Z server-username: GITHUB_ACTOR 2022-11-18T20:35:32.1260228Z server-password: GITHUB_TOKEN 2022-11-18T20:35:32.1260561Z overwrite-settings: true 2022-11-18T20:35:32.1260879Z job-status: success 2022-11-18T20:35:32.1261350Z token: *** 2022-11-18T20:35:32.1261631Z env: 2022-11-18T20:35:32.1261994Z pythonLocation: C:\hostedtoolcache\windows\Python\3.8.10\x64 2022-11-18T20:35:32.1262278Z ##[endgroup] 2022-11-18T20:35:32.4338399Z ##[group]Installed distributions 2022-11-18T20:35:32.4397902Z Trying to resolve the latest version from remote 2022-11-18T20:35:32.9476398Z Resolved latest version as 17.0.5+8 2022-11-18T20:35:32.9477260Z Trying to download... 2022-11-18T20:35:32.9478237Z Downloading Java 17.0.5+8 (Zulu) from https://cdn.azul.com/zulu/bin/zulu17.38.21-ca-jdk17.0.5-win_x64.zip ... 2022-11-18T20:35:34.3922925Z Extracting Java archive... 2022-11-18T20:35:34.5359600Z [command]C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoLogo -Sta -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command "$ErrorActionPreference = 'Stop' ; try { Add-Type -AssemblyName System.IO.Compression.FileSystem } catch { } ; [System.IO.Compression.ZipFile]::ExtractToDirectory('D:\a\_temp\56ab4802-dce6-48f0-a29e-b4545e6d93b0', 'D:\a\_temp\bf5556d2-34e7-4c03-8d58-f7f9e958189f')" 2022-11-18T20:35:42.4727371Z Java 17.0.5+8 was downloaded 2022-11-18T20:35:42.4731830Z Setting Java 17.0.5+8 as the default 2022-11-18T20:35:42.4757291Z Creating toolchains.xml for JDK version 17 from zulu 2022-11-18T20:35:42.4881923Z Writing to C:\Users\runneradmin\.m2\toolchains.xml 2022-11-18T20:35:42.4889132Z 2022-11-18T20:35:42.4892751Z Java configuration: 2022-11-18T20:35:42.4893487Z Distribution: zulu 2022-11-18T20:35:42.4893829Z Version: 17.0.5+8 2022-11-18T20:35:42.4895284Z Path: C:\hostedtoolcache\windows\Java_Zulu_jdk\17.0.5-8\x64 2022-11-18T20:35:42.4895696Z 2022-11-18T20:35:42.4897303Z ##[endgroup] 2022-11-18T20:35:42.4915156Z Creating settings.xml with server-id: github 2022-11-18T20:35:42.4915790Z Writing to C:\Users\runneradmin\.m2\settings.xml 2022-11-18T20:35:42.5206761Z ##[group]Run actions/checkout@v3 2022-11-18T20:35:42.5207167Z with: 2022-11-18T20:35:42.5207484Z repository: pytorch/serve 2022-11-18T20:35:42.5208003Z token: *** 2022-11-18T20:35:42.5208282Z ssh-strict: true 2022-11-18T20:35:42.5208572Z persist-credentials: true 2022-11-18T20:35:42.5208824Z clean: true 2022-11-18T20:35:42.5209113Z fetch-depth: 1 2022-11-18T20:35:42.5209370Z lfs: false 2022-11-18T20:35:42.5209575Z submodules: false 2022-11-18T20:35:42.5209861Z set-safe-directory: true 2022-11-18T20:35:42.5210137Z env: 2022-11-18T20:35:42.5210422Z pythonLocation: C:\hostedtoolcache\windows\Python\3.8.10\x64 2022-11-18T20:35:42.5210830Z JAVA_HOME: C:\hostedtoolcache\windows\Java_Zulu_jdk\17.0.5-8\x64 2022-11-18T20:35:42.5211236Z JAVA_HOME_17_X64: C:\hostedtoolcache\windows\Java_Zulu_jdk\17.0.5-8\x64 2022-11-18T20:35:42.5211580Z ##[endgroup] 2022-11-18T20:35:45.8942176Z Syncing repository: pytorch/serve 2022-11-18T20:35:45.8944290Z ##[group]Getting Git version info 2022-11-18T20:35:45.8945615Z Working directory is 'D:\a\serve\serve' 2022-11-18T20:35:45.9232991Z [command]"C:\Program Files\Git\bin\git.exe" version 2022-11-18T20:35:46.9541621Z git version 2.38.1.windows.1 2022-11-18T20:35:46.9583205Z ##[endgroup] 2022-11-18T20:35:46.9607852Z Temporarily overriding HOME='D:\a\_temp\67f7f747-932b-4af0-8643-c9056ef33592' before making global git config changes 2022-11-18T20:35:46.9608670Z Adding repository directory to the temporary git global config as a safe directory 2022-11-18T20:35:46.9616015Z [command]"C:\Program Files\Git\bin\git.exe" config --global --add safe.directory D:\a\serve\serve 2022-11-18T20:35:47.0378967Z Deleting the contents of 'D:\a\serve\serve' 2022-11-18T20:35:47.0387684Z ##[group]Initializing the repository 2022-11-18T20:35:47.0395053Z [command]"C:\Program Files\Git\bin\git.exe" init D:\a\serve\serve 2022-11-18T20:35:47.1411495Z Initialized empty Git repository in D:/a/serve/serve/.git/ 2022-11-18T20:35:47.1447306Z [command]"C:\Program Files\Git\bin\git.exe" remote add origin https://github.com/pytorch/serve 2022-11-18T20:35:47.1949129Z ##[endgroup] 2022-11-18T20:35:47.1950370Z ##[group]Disabling automatic garbage collection 2022-11-18T20:35:47.1956722Z [command]"C:\Program Files\Git\bin\git.exe" config --local gc.auto 0 2022-11-18T20:35:47.2197248Z ##[endgroup] 2022-11-18T20:35:47.2198058Z ##[group]Setting up auth 2022-11-18T20:35:47.2212048Z [command]"C:\Program Files\Git\bin\git.exe" config --local --name-only --get-regexp core\.sshCommand 2022-11-18T20:35:47.2452112Z [command]"C:\Program Files\Git\bin\git.exe" submodule foreach --recursive "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2022-11-18T20:35:49.4612561Z [command]"C:\Program Files\Git\bin\git.exe" config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2022-11-18T20:35:49.4854259Z [command]"C:\Program Files\Git\bin\git.exe" submodule foreach --recursive "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2022-11-18T20:35:49.9563505Z [command]"C:\Program Files\Git\bin\git.exe" config --local http.https://github.com/.extraheader "AUTHORIZATION: basic ***" 2022-11-18T20:35:49.9815599Z ##[endgroup] 2022-11-18T20:35:49.9816375Z ##[group]Fetching the repository 2022-11-18T20:35:49.9828568Z [command]"C:\Program Files\Git\bin\git.exe" -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +a6e30a23223298fac8fa80737306bf32d6b55b1a:refs/remotes/origin/upgrade_pytorch_113 2022-11-18T20:35:51.7524126Z remote: Enumerating objects: 1291, done. 2022-11-18T20:35:51.7524817Z remote: Counting objects: 0% (1/1291) 2022-11-18T20:35:51.7525410Z remote: Counting objects: 1% (13/1291) 2022-11-18T20:35:51.7525810Z remote: Counting objects: 2% (26/1291) 2022-11-18T20:35:51.7529706Z remote: Counting objects: 3% (39/1291) 2022-11-18T20:35:51.7530422Z remote: Counting objects: 4% (52/1291) 2022-11-18T20:35:51.7530844Z remote: Counting objects: 5% (65/1291) 2022-11-18T20:35:51.7531347Z remote: Counting objects: 6% (78/1291) 2022-11-18T20:35:51.7531812Z remote: Counting objects: 7% (91/1291) 2022-11-18T20:35:51.7532299Z remote: Counting objects: 8% (104/1291) 2022-11-18T20:35:51.7532706Z remote: Counting objects: 9% (117/1291) 2022-11-18T20:35:51.7533176Z remote: Counting objects: 10% (130/1291) 2022-11-18T20:35:51.7533622Z remote: Counting objects: 11% (143/1291) 2022-11-18T20:35:51.7534062Z remote: Counting objects: 12% (155/1291) 2022-11-18T20:35:51.7534524Z remote: Counting objects: 13% (168/1291) 2022-11-18T20:35:51.7534977Z remote: Counting objects: 14% (181/1291) 2022-11-18T20:35:51.7535422Z remote: Counting objects: 15% (194/1291) 2022-11-18T20:35:51.7535866Z remote: Counting objects: 16% (207/1291) 2022-11-18T20:35:51.7536250Z remote: Counting objects: 17% (220/1291) 2022-11-18T20:35:51.7536684Z remote: Counting objects: 18% (233/1291) 2022-11-18T20:35:51.7537140Z remote: Counting objects: 19% (246/1291) 2022-11-18T20:35:51.7537574Z remote: Counting objects: 20% (259/1291) 2022-11-18T20:35:51.7537967Z remote: Counting objects: 21% (272/1291) 2022-11-18T20:35:51.7538404Z remote: Counting objects: 22% (285/1291) 2022-11-18T20:35:51.7538849Z remote: Counting objects: 23% (297/1291) 2022-11-18T20:35:51.7539284Z remote: Counting objects: 24% (310/1291) 2022-11-18T20:35:51.7539668Z remote: Counting objects: 25% (323/1291) 2022-11-18T20:35:51.7540100Z remote: Counting objects: 26% (336/1291) 2022-11-18T20:35:51.7540626Z remote: Counting objects: 27% (349/1291) 2022-11-18T20:35:51.7541073Z remote: Counting objects: 28% (362/1291) 2022-11-18T20:35:51.7541459Z remote: Counting objects: 29% (375/1291) 2022-11-18T20:35:51.7541888Z remote: Counting objects: 30% (388/1291) 2022-11-18T20:35:51.7542320Z remote: Counting objects: 31% (401/1291) 2022-11-18T20:35:51.7542710Z remote: Counting objects: 32% (414/1291) 2022-11-18T20:35:51.7543262Z remote: Counting objects: 33% (427/1291) 2022-11-18T20:35:51.7543647Z remote: Counting objects: 34% (439/1291) 2022-11-18T20:35:51.7544031Z remote: Counting objects: 35% (452/1291) 2022-11-18T20:35:51.7544364Z remote: Counting objects: 36% (465/1291) 2022-11-18T20:35:51.7544752Z remote: Counting objects: 37% (478/1291) 2022-11-18T20:35:51.7545137Z remote: Counting objects: 38% (491/1291) 2022-11-18T20:35:51.7545672Z remote: Counting objects: 39% (504/1291) 2022-11-18T20:35:51.7546017Z remote: Counting objects: 40% (517/1291) 2022-11-18T20:35:51.7546406Z remote: Counting objects: 41% (530/1291) 2022-11-18T20:35:51.7546803Z remote: Counting objects: 42% (543/1291) 2022-11-18T20:35:51.7547183Z remote: Counting objects: 43% (556/1291) 2022-11-18T20:35:51.7547520Z remote: Counting objects: 44% (569/1291) 2022-11-18T20:35:51.7547909Z remote: Counting objects: 45% (581/1291) 2022-11-18T20:35:51.7548340Z remote: Counting objects: 46% (594/1291) 2022-11-18T20:35:51.7548703Z remote: Counting objects: 47% (607/1291) 2022-11-18T20:35:51.7549028Z remote: Counting objects: 48% (620/1291) 2022-11-18T20:35:51.7549302Z remote: Counting objects: 49% (633/1291) 2022-11-18T20:35:51.7549627Z remote: Counting objects: 50% (646/1291) 2022-11-18T20:35:51.7549945Z remote: Counting objects: 51% (659/1291) 2022-11-18T20:35:51.7550221Z remote: Counting objects: 52% (672/1291) 2022-11-18T20:35:51.7550550Z remote: Counting objects: 53% (685/1291) 2022-11-18T20:35:51.7551457Z remote: Counting objects: 54% (698/1291) 2022-11-18T20:35:51.7551816Z remote: Counting objects: 55% (711/1291) 2022-11-18T20:35:51.7552095Z remote: Counting objects: 56% (723/1291) 2022-11-18T20:35:51.7552414Z remote: Counting objects: 57% (736/1291) 2022-11-18T20:35:51.7552728Z remote: Counting objects: 58% (749/1291) 2022-11-18T20:35:51.7553001Z remote: Counting objects: 59% (762/1291) 2022-11-18T20:35:51.7553330Z remote: Counting objects: 60% (775/1291) 2022-11-18T20:35:51.7553646Z remote: Counting objects: 61% (788/1291) 2022-11-18T20:35:51.7553960Z remote: Counting objects: 62% (801/1291) 2022-11-18T20:35:51.7554236Z remote: Counting objects: 63% (814/1291) 2022-11-18T20:35:51.7554676Z remote: Counting objects: 64% (827/1291) 2022-11-18T20:35:51.7555001Z remote: Counting objects: 65% (840/1291) 2022-11-18T20:35:51.7555318Z remote: Counting objects: 66% (853/1291) 2022-11-18T20:35:51.7555596Z remote: Counting objects: 67% (865/1291) 2022-11-18T20:35:51.7555908Z remote: Counting objects: 68% (878/1291) 2022-11-18T20:35:51.7556230Z remote: Counting objects: 69% (891/1291) 2022-11-18T20:35:51.7556504Z remote: Counting objects: 70% (904/1291) 2022-11-18T20:35:51.7556817Z remote: Counting objects: 71% (917/1291) 2022-11-18T20:35:51.7557126Z remote: Counting objects: 72% (930/1291) 2022-11-18T20:35:51.7557457Z remote: Counting objects: 73% (943/1291) 2022-11-18T20:35:51.7557735Z remote: Counting objects: 74% (956/1291) 2022-11-18T20:35:51.7558049Z remote: Counting objects: 75% (969/1291) 2022-11-18T20:35:51.7558362Z remote: Counting objects: 76% (982/1291) 2022-11-18T20:35:51.7558638Z remote: Counting objects: 77% (995/1291) 2022-11-18T20:35:51.7558968Z remote: Counting objects: 78% (1007/1291) 2022-11-18T20:35:51.7559317Z remote: Counting objects: 79% (1020/1291) 2022-11-18T20:35:51.7559646Z remote: Counting objects: 80% (1033/1291) 2022-11-18T20:35:51.7559930Z remote: Counting objects: 81% (1046/1291) 2022-11-18T20:35:51.7560258Z remote: Counting objects: 82% (1059/1291) 2022-11-18T20:35:51.7560592Z remote: Counting objects: 83% (1072/1291) 2022-11-18T20:35:51.7843604Z remote: Counting objects: 84% (1085/1291) 2022-11-18T20:35:51.7847748Z remote: Counting objects: 85% (1098/1291) 2022-11-18T20:35:51.7848147Z remote: Counting objects: 86% (1111/1291) 2022-11-18T20:35:51.7848540Z remote: Counting objects: 87% (1124/1291) 2022-11-18T20:35:51.7848874Z remote: Counting objects: 88% (1137/1291) 2022-11-18T20:35:51.7849227Z remote: Counting objects: 89% (1149/1291) 2022-11-18T20:35:51.7849726Z remote: Counting objects: 90% (1162/1291) 2022-11-18T20:35:51.7850075Z remote: Counting objects: 91% (1175/1291) 2022-11-18T20:35:51.7850433Z remote: Counting objects: 92% (1188/1291) 2022-11-18T20:35:51.7850711Z remote: Counting objects: 93% (1201/1291) 2022-11-18T20:35:51.7851039Z remote: Counting objects: 94% (1214/1291) 2022-11-18T20:35:51.7851406Z remote: Counting objects: 95% (1227/1291) 2022-11-18T20:35:51.7851732Z remote: Counting objects: 96% (1240/1291) 2022-11-18T20:35:51.7852013Z remote: Counting objects: 97% (1253/1291) 2022-11-18T20:35:51.7852341Z remote: Counting objects: 98% (1266/1291) 2022-11-18T20:35:51.7852665Z remote: Counting objects: 99% (1279/1291) 2022-11-18T20:35:51.7852964Z remote: Counting objects: 100% (1291/1291) 2022-11-18T20:35:51.7853300Z remote: Counting objects: 100% (1291/1291), done. 2022-11-18T20:35:51.7853652Z remote: Compressing objects: 0% (1/1080) 2022-11-18T20:35:51.7853958Z remote: Compressing objects: 1% (11/1080) 2022-11-18T20:35:51.7854331Z remote: Compressing objects: 2% (22/1080) 2022-11-18T20:35:51.7854758Z remote: Compressing objects: 3% (33/1080) 2022-11-18T20:35:51.7855105Z remote: Compressing objects: 4% (44/1080) 2022-11-18T20:35:51.7855988Z remote: Compressing objects: 5% (54/1080) 2022-11-18T20:35:51.7856557Z remote: Compressing objects: 6% (65/1080) 2022-11-18T20:35:51.7856935Z remote: Compressing objects: 7% (76/1080) 2022-11-18T20:35:51.7857230Z remote: Compressing objects: 8% (87/1080) 2022-11-18T20:35:51.7857574Z remote: Compressing objects: 9% (98/1080) 2022-11-18T20:35:51.7857912Z remote: Compressing objects: 10% (108/1080) 2022-11-18T20:35:51.7858267Z remote: Compressing objects: 11% (119/1080) 2022-11-18T20:35:51.7858589Z remote: Compressing objects: 12% (130/1080) 2022-11-18T20:35:51.7858939Z remote: Compressing objects: 13% (141/1080) 2022-11-18T20:35:51.7859275Z remote: Compressing objects: 14% (152/1080) 2022-11-18T20:35:51.7859568Z remote: Compressing objects: 15% (162/1080) 2022-11-18T20:35:51.7859917Z remote: Compressing objects: 16% (173/1080) 2022-11-18T20:35:51.7860250Z remote: Compressing objects: 17% (184/1080) 2022-11-18T20:35:51.7926491Z remote: Compressing objects: 18% (195/1080) 2022-11-18T20:35:51.7926816Z remote: Compressing objects: 19% (206/1080) 2022-11-18T20:35:51.7927094Z remote: Compressing objects: 20% (216/1080) 2022-11-18T20:35:51.7928858Z remote: Compressing objects: 21% (227/1080) 2022-11-18T20:35:51.7929199Z remote: Compressing objects: 22% (238/1080) 2022-11-18T20:35:51.7929495Z remote: Compressing objects: 23% (249/1080) 2022-11-18T20:35:51.7929757Z remote: Compressing objects: 24% (260/1080) 2022-11-18T20:35:51.7930061Z remote: Compressing objects: 25% (270/1080) 2022-11-18T20:35:51.7930337Z remote: Compressing objects: 26% (281/1080) 2022-11-18T20:35:51.7930604Z remote: Compressing objects: 27% (292/1080) 2022-11-18T20:35:51.7930892Z remote: Compressing objects: 28% (303/1080) 2022-11-18T20:35:51.7931172Z remote: Compressing objects: 29% (314/1080) 2022-11-18T20:35:51.7931434Z remote: Compressing objects: 30% (324/1080) 2022-11-18T20:35:51.7931717Z remote: Compressing objects: 31% (335/1080) 2022-11-18T20:35:51.7931989Z remote: Compressing objects: 32% (346/1080) 2022-11-18T20:35:51.7932260Z remote: Compressing objects: 33% (357/1080) 2022-11-18T20:35:51.8410768Z remote: Compressing objects: 34% (368/1080) 2022-11-18T20:35:51.8411150Z remote: Compressing objects: 35% (378/1080) 2022-11-18T20:35:51.8962521Z remote: Compressing objects: 36% (389/1080) 2022-11-18T20:35:51.8979866Z remote: Compressing objects: 37% (400/1080) 2022-11-18T20:35:51.8988826Z remote: Compressing objects: 38% (411/1080) 2022-11-18T20:35:51.8998399Z remote: Compressing objects: 39% (422/1080) 2022-11-18T20:35:51.9001124Z remote: Compressing objects: 40% (432/1080) 2022-11-18T20:35:51.9004835Z remote: Compressing objects: 41% (443/1080) 2022-11-18T20:35:51.9010403Z remote: Compressing objects: 42% (454/1080) 2022-11-18T20:35:51.9010956Z remote: Compressing objects: 43% (465/1080) 2022-11-18T20:35:51.9017845Z remote: Compressing objects: 44% (476/1080) 2022-11-18T20:35:51.9018211Z remote: Compressing objects: 45% (486/1080) 2022-11-18T20:35:51.9018524Z remote: Compressing objects: 46% (497/1080) 2022-11-18T20:35:51.9018869Z remote: Compressing objects: 47% (508/1080) 2022-11-18T20:35:51.9019189Z remote: Compressing objects: 48% (519/1080) 2022-11-18T20:35:51.9019493Z remote: Compressing objects: 49% (530/1080) 2022-11-18T20:35:51.9019824Z remote: Compressing objects: 50% (540/1080) 2022-11-18T20:35:51.9025541Z remote: Compressing objects: 51% (551/1080) 2022-11-18T20:35:51.9034248Z remote: Compressing objects: 52% (562/1080) 2022-11-18T20:35:51.9036440Z remote: Compressing objects: 53% (573/1080) 2022-11-18T20:35:51.9057337Z remote: Compressing objects: 54% (584/1080) 2022-11-18T20:35:51.9078049Z remote: Compressing objects: 55% (594/1080) 2022-11-18T20:35:51.9078373Z remote: Compressing objects: 56% (605/1080) 2022-11-18T20:35:51.9084706Z remote: Compressing objects: 57% (616/1080) 2022-11-18T20:35:51.9121035Z remote: Compressing objects: 58% (627/1080) 2022-11-18T20:35:51.9121545Z remote: Compressing objects: 59% (638/1080) 2022-11-18T20:35:51.9168818Z remote: Compressing objects: 60% (648/1080) 2022-11-18T20:35:51.9175758Z remote: Compressing objects: 61% (659/1080) 2022-11-18T20:35:51.9192272Z remote: Compressing objects: 62% (670/1080) 2022-11-18T20:35:51.9211969Z remote: Compressing objects: 63% (681/1080) 2022-11-18T20:35:51.9248618Z remote: Compressing objects: 64% (692/1080) 2022-11-18T20:35:51.9271740Z remote: Compressing objects: 65% (702/1080) 2022-11-18T20:35:51.9272083Z remote: Compressing objects: 66% (713/1080) 2022-11-18T20:35:51.9296189Z remote: Compressing objects: 67% (724/1080) 2022-11-18T20:35:51.9297060Z remote: Compressing objects: 68% (735/1080) 2022-11-18T20:35:51.9341009Z remote: Compressing objects: 69% (746/1080) 2022-11-18T20:35:51.9353598Z remote: Compressing objects: 70% (756/1080) 2022-11-18T20:35:51.9384347Z remote: Compressing objects: 71% (767/1080) 2022-11-18T20:35:51.9441289Z remote: Compressing objects: 72% (778/1080) 2022-11-18T20:35:51.9460880Z remote: Compressing objects: 73% (789/1080) 2022-11-18T20:35:51.9490022Z remote: Compressing objects: 74% (800/1080) 2022-11-18T20:35:51.9528504Z remote: Compressing objects: 75% (810/1080) 2022-11-18T20:35:51.9578128Z remote: Compressing objects: 76% (821/1080) 2022-11-18T20:35:51.9588074Z remote: Compressing objects: 77% (832/1080) 2022-11-18T20:35:51.9631040Z remote: Compressing objects: 78% (843/1080) 2022-11-18T20:35:51.9643807Z remote: Compressing objects: 79% (854/1080) 2022-11-18T20:35:51.9644101Z remote: Compressing objects: 80% (864/1080) 2022-11-18T20:35:51.9644375Z remote: Compressing objects: 81% (875/1080) 2022-11-18T20:35:51.9644679Z remote: Compressing objects: 82% (886/1080) 2022-11-18T20:35:51.9646574Z remote: Compressing objects: 83% (897/1080) 2022-11-18T20:35:51.9647723Z remote: Compressing objects: 84% (908/1080) 2022-11-18T20:35:51.9648465Z remote: Compressing objects: 85% (918/1080) 2022-11-18T20:35:51.9649320Z remote: Compressing objects: 86% (929/1080) 2022-11-18T20:35:51.9651619Z remote: Compressing objects: 87% (940/1080) 2022-11-18T20:35:51.9652220Z remote: Compressing objects: 88% (951/1080) 2022-11-18T20:35:51.9652682Z remote: Compressing objects: 89% (962/1080) 2022-11-18T20:35:51.9652983Z remote: Compressing objects: 90% (972/1080) 2022-11-18T20:35:51.9653284Z remote: Compressing objects: 91% (983/1080) 2022-11-18T20:35:51.9653571Z remote: Compressing objects: 92% (994/1080) 2022-11-18T20:35:51.9653844Z remote: Compressing objects: 93% (1005/1080) 2022-11-18T20:35:51.9655637Z remote: Compressing objects: 94% (1016/1080) 2022-11-18T20:35:51.9657810Z remote: Compressing objects: 95% (1026/1080) 2022-11-18T20:35:51.9659251Z remote: Compressing objects: 96% (1037/1080) 2022-11-18T20:35:51.9659933Z remote: Compressing objects: 97% (1048/1080) 2022-11-18T20:35:51.9660252Z remote: Compressing objects: 98% (1059/1080) 2022-11-18T20:35:51.9660556Z remote: Compressing objects: 99% (1070/1080) 2022-11-18T20:35:51.9660830Z remote: Compressing objects: 100% (1080/1080) 2022-11-18T20:35:51.9661986Z remote: Compressing objects: 100% (1080/1080), done. 2022-11-18T20:35:52.0510203Z Receiving objects: 0% (1/1291) 2022-11-18T20:35:52.0519594Z Receiving objects: 1% (13/1291) 2022-11-18T20:35:52.0534634Z Receiving objects: 2% (26/1291) 2022-11-18T20:35:52.0872027Z Receiving objects: 3% (39/1291) 2022-11-18T20:35:52.1005684Z Receiving objects: 4% (52/1291) 2022-11-18T20:35:52.1018193Z Receiving objects: 5% (65/1291) 2022-11-18T20:35:52.1058162Z Receiving objects: 6% (78/1291) 2022-11-18T20:35:52.1069051Z Receiving objects: 7% (91/1291) 2022-11-18T20:35:52.1069525Z Receiving objects: 8% (104/1291) 2022-11-18T20:35:52.1069907Z Receiving objects: 9% (117/1291) 2022-11-18T20:35:52.1073426Z Receiving objects: 10% (130/1291) 2022-11-18T20:35:52.1073905Z Receiving objects: 11% (143/1291) 2022-11-18T20:35:52.1114423Z Receiving objects: 12% (155/1291) 2022-11-18T20:35:52.1344983Z Receiving objects: 13% (168/1291) 2022-11-18T20:35:52.1362382Z Receiving objects: 14% (181/1291) 2022-11-18T20:35:52.1372078Z Receiving objects: 15% (194/1291) 2022-11-18T20:35:52.1376442Z Receiving objects: 16% (207/1291) 2022-11-18T20:35:52.1382323Z Receiving objects: 17% (220/1291) 2022-11-18T20:35:52.1382893Z Receiving objects: 18% (233/1291) 2022-11-18T20:35:52.1392909Z Receiving objects: 19% (246/1291) 2022-11-18T20:35:52.1417281Z Receiving objects: 20% (259/1291) 2022-11-18T20:35:52.1419878Z Receiving objects: 21% (272/1291) 2022-11-18T20:35:52.1441083Z Receiving objects: 22% (285/1291) 2022-11-18T20:35:52.1450167Z Receiving objects: 23% (297/1291) 2022-11-18T20:35:52.1467917Z Receiving objects: 24% (310/1291) 2022-11-18T20:35:52.1469519Z Receiving objects: 25% (323/1291) 2022-11-18T20:35:52.2379658Z Receiving objects: 26% (336/1291) 2022-11-18T20:35:52.2416078Z Receiving objects: 27% (349/1291) 2022-11-18T20:35:52.2430624Z Receiving objects: 28% (362/1291) 2022-11-18T20:35:52.2463067Z Receiving objects: 29% (375/1291) 2022-11-18T20:35:52.3006477Z Receiving objects: 30% (388/1291) 2022-11-18T20:35:52.3067575Z Receiving objects: 31% (401/1291) 2022-11-18T20:35:52.3068882Z Receiving objects: 32% (414/1291) 2022-11-18T20:35:52.3107417Z Receiving objects: 33% (427/1291) 2022-11-18T20:35:52.3109068Z Receiving objects: 34% (439/1291) 2022-11-18T20:35:52.3130389Z Receiving objects: 35% (452/1291) 2022-11-18T20:35:52.3133792Z Receiving objects: 36% (465/1291) 2022-11-18T20:35:52.3139385Z Receiving objects: 37% (478/1291) 2022-11-18T20:35:52.3139902Z Receiving objects: 38% (491/1291) 2022-11-18T20:35:52.3143281Z Receiving objects: 39% (504/1291) 2022-11-18T20:35:52.3146647Z Receiving objects: 40% (517/1291) 2022-11-18T20:35:52.3196531Z Receiving objects: 41% (530/1291) 2022-11-18T20:35:52.9534435Z Receiving objects: 42% (543/1291) 2022-11-18T20:35:52.9535104Z Receiving objects: 43% (556/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9535632Z Receiving objects: 44% (569/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9574434Z Receiving objects: 45% (581/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9578180Z Receiving objects: 46% (594/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9582803Z Receiving objects: 47% (607/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9587534Z Receiving objects: 48% (620/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9589774Z Receiving objects: 49% (633/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9594954Z Receiving objects: 50% (646/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9599566Z Receiving objects: 51% (659/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9603510Z Receiving objects: 52% (672/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9613407Z Receiving objects: 53% (685/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9627140Z Receiving objects: 54% (698/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9627673Z Receiving objects: 55% (711/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9633201Z Receiving objects: 56% (723/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9635623Z Receiving objects: 57% (736/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9649524Z Receiving objects: 58% (749/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9655520Z Receiving objects: 59% (762/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9655978Z Receiving objects: 60% (775/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9659861Z Receiving objects: 61% (788/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9663650Z Receiving objects: 62% (801/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9823590Z Receiving objects: 63% (814/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9824063Z Receiving objects: 64% (827/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9832897Z Receiving objects: 65% (840/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9836783Z Receiving objects: 66% (853/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9841543Z Receiving objects: 67% (865/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9844384Z Receiving objects: 68% (878/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9848294Z Receiving objects: 69% (891/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9848709Z Receiving objects: 70% (904/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9862843Z Receiving objects: 71% (917/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9864561Z Receiving objects: 72% (930/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9872903Z Receiving objects: 73% (943/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9873347Z Receiving objects: 74% (956/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9875216Z Receiving objects: 75% (969/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9876332Z Receiving objects: 76% (982/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9877476Z Receiving objects: 77% (995/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9889225Z Receiving objects: 78% (1007/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9889634Z Receiving objects: 79% (1020/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9929712Z Receiving objects: 80% (1033/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9930196Z Receiving objects: 81% (1046/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9934811Z Receiving objects: 82% (1059/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9935244Z Receiving objects: 83% (1072/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9938807Z Receiving objects: 84% (1085/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9947735Z Receiving objects: 85% (1098/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9952642Z Receiving objects: 86% (1111/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:52.9959266Z Receiving objects: 87% (1124/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0040188Z Receiving objects: 88% (1137/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0040844Z Receiving objects: 89% (1149/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0045539Z Receiving objects: 90% (1162/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0050857Z Receiving objects: 91% (1175/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0054177Z Receiving objects: 92% (1188/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0058016Z Receiving objects: 93% (1201/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0061797Z Receiving objects: 94% (1214/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0064354Z Receiving objects: 95% (1227/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0086748Z Receiving objects: 96% (1240/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0088780Z Receiving objects: 97% (1253/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0089846Z Receiving objects: 98% (1266/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0090452Z Receiving objects: 99% (1279/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0091810Z remote: Total 1291 (delta 205), reused 704 (delta 105), pack-reused 0 2022-11-18T20:35:53.0097306Z Receiving objects: 100% (1291/1291), 18.53 MiB | 37.05 MiB/s 2022-11-18T20:35:53.0122294Z Receiving objects: 100% (1291/1291), 27.49 MiB | 27.73 MiB/s, done. 2022-11-18T20:35:53.0206533Z Resolving deltas: 0% (0/205) 2022-11-18T20:35:53.0206989Z Resolving deltas: 1% (3/205) 2022-11-18T20:35:53.0207331Z Resolving deltas: 2% (5/205) 2022-11-18T20:35:53.0207641Z Resolving deltas: 3% (7/205) 2022-11-18T20:35:53.0207966Z Resolving deltas: 4% (9/205) 2022-11-18T20:35:53.0208287Z Resolving deltas: 5% (11/205) 2022-11-18T20:35:53.0208610Z Resolving deltas: 6% (13/205) 2022-11-18T20:35:53.0208912Z Resolving deltas: 7% (15/205) 2022-11-18T20:35:53.0209228Z Resolving deltas: 8% (17/205) 2022-11-18T20:35:53.0209546Z Resolving deltas: 9% (19/205) 2022-11-18T20:35:53.0209853Z Resolving deltas: 10% (21/205) 2022-11-18T20:35:53.0210171Z Resolving deltas: 11% (23/205) 2022-11-18T20:35:53.0210498Z Resolving deltas: 12% (25/205) 2022-11-18T20:35:53.0210809Z Resolving deltas: 13% (27/205) 2022-11-18T20:35:53.0211106Z Resolving deltas: 14% (29/205) 2022-11-18T20:35:53.0211430Z Resolving deltas: 15% (31/205) 2022-11-18T20:35:53.0211744Z Resolving deltas: 16% (33/205) 2022-11-18T20:35:53.0212059Z Resolving deltas: 17% (35/205) 2022-11-18T20:35:53.0213062Z Resolving deltas: 18% (37/205) 2022-11-18T20:35:53.0213523Z Resolving deltas: 19% (39/205) 2022-11-18T20:35:53.0213852Z Resolving deltas: 20% (41/205) 2022-11-18T20:35:53.0214170Z Resolving deltas: 21% (44/205) 2022-11-18T20:35:53.0214479Z Resolving deltas: 22% (46/205) 2022-11-18T20:35:53.0214798Z Resolving deltas: 23% (48/205) 2022-11-18T20:35:53.0215258Z Resolving deltas: 24% (50/205) 2022-11-18T20:35:53.0215560Z Resolving deltas: 25% (52/205) 2022-11-18T20:35:53.0215888Z Resolving deltas: 26% (54/205) 2022-11-18T20:35:53.0216210Z Resolving deltas: 27% (56/205) 2022-11-18T20:35:53.0216522Z Resolving deltas: 28% (58/205) 2022-11-18T20:35:53.0216833Z Resolving deltas: 29% (60/205) 2022-11-18T20:35:53.0217278Z Resolving deltas: 30% (62/205) 2022-11-18T20:35:53.0218030Z Resolving deltas: 31% (64/205) 2022-11-18T20:35:53.0219226Z Resolving deltas: 32% (66/205) 2022-11-18T20:35:53.0219562Z Resolving deltas: 33% (68/205) 2022-11-18T20:35:53.0219878Z Resolving deltas: 34% (70/205) 2022-11-18T20:35:53.0220176Z Resolving deltas: 35% (72/205) 2022-11-18T20:35:53.0220495Z Resolving deltas: 36% (74/205) 2022-11-18T20:35:53.0220811Z Resolving deltas: 37% (76/205) 2022-11-18T20:35:53.0221126Z Resolving deltas: 38% (78/205) 2022-11-18T20:35:53.0221415Z Resolving deltas: 39% (80/205) 2022-11-18T20:35:53.0221726Z Resolving deltas: 40% (82/205) 2022-11-18T20:35:53.0222041Z Resolving deltas: 41% (85/205) 2022-11-18T20:35:53.0222354Z Resolving deltas: 42% (87/205) 2022-11-18T20:35:53.0222671Z Resolving deltas: 43% (89/205) 2022-11-18T20:35:53.0222984Z Resolving deltas: 44% (91/205) 2022-11-18T20:35:53.0224241Z Resolving deltas: 45% (93/205) 2022-11-18T20:35:53.0224542Z Resolving deltas: 46% (95/205) 2022-11-18T20:35:53.0224855Z Resolving deltas: 47% (97/205) 2022-11-18T20:35:53.0225166Z Resolving deltas: 48% (99/205) 2022-11-18T20:35:53.0225480Z Resolving deltas: 49% (101/205) 2022-11-18T20:35:53.0225784Z Resolving deltas: 50% (103/205) 2022-11-18T20:35:53.0226092Z Resolving deltas: 51% (105/205) 2022-11-18T20:35:53.0226410Z Resolving deltas: 52% (107/205) 2022-11-18T20:35:53.0226723Z Resolving deltas: 53% (109/205) 2022-11-18T20:35:53.0227096Z Resolving deltas: 54% (111/205) 2022-11-18T20:35:53.0227332Z Resolving deltas: 55% (113/205) 2022-11-18T20:35:53.0227561Z Resolving deltas: 56% (115/205) 2022-11-18T20:35:53.0227776Z Resolving deltas: 57% (117/205) 2022-11-18T20:35:53.0228008Z Resolving deltas: 58% (119/205) 2022-11-18T20:35:53.0228233Z Resolving deltas: 59% (121/205) 2022-11-18T20:35:53.0228458Z Resolving deltas: 60% (123/205) 2022-11-18T20:35:53.0228676Z Resolving deltas: 61% (126/205) 2022-11-18T20:35:53.0228897Z Resolving deltas: 62% (128/205) 2022-11-18T20:35:53.0229123Z Resolving deltas: 63% (130/205) 2022-11-18T20:35:53.0229442Z Resolving deltas: 64% (132/205) 2022-11-18T20:35:53.0229706Z Resolving deltas: 65% (134/205) 2022-11-18T20:35:53.0229930Z Resolving deltas: 66% (136/205) 2022-11-18T20:35:53.0230142Z Resolving deltas: 67% (138/205) 2022-11-18T20:35:53.0230404Z Resolving deltas: 68% (140/205) 2022-11-18T20:35:53.0230617Z Resolving deltas: 69% (142/205) 2022-11-18T20:35:53.0230841Z Resolving deltas: 70% (144/205) 2022-11-18T20:35:53.0231065Z Resolving deltas: 71% (146/205) 2022-11-18T20:35:53.0231275Z Resolving deltas: 72% (148/205) 2022-11-18T20:35:53.0231503Z Resolving deltas: 73% (150/205) 2022-11-18T20:35:53.0231727Z Resolving deltas: 74% (152/205) 2022-11-18T20:35:53.0231942Z Resolving deltas: 75% (154/205) 2022-11-18T20:35:53.0232172Z Resolving deltas: 76% (156/205) 2022-11-18T20:35:53.0232403Z Resolving deltas: 77% (158/205) 2022-11-18T20:35:53.0232620Z Resolving deltas: 78% (160/205) 2022-11-18T20:35:53.0232849Z Resolving deltas: 79% (162/205) 2022-11-18T20:35:53.0233072Z Resolving deltas: 80% (164/205) 2022-11-18T20:35:53.0233309Z Resolving deltas: 81% (167/205) 2022-11-18T20:35:53.0233519Z Resolving deltas: 82% (169/205) 2022-11-18T20:35:53.0233740Z Resolving deltas: 83% (171/205) 2022-11-18T20:35:53.0233961Z Resolving deltas: 84% (173/205) 2022-11-18T20:35:53.0234171Z Resolving deltas: 85% (175/205) 2022-11-18T20:35:53.0234612Z Resolving deltas: 86% (177/205) 2022-11-18T20:35:53.0235194Z Resolving deltas: 87% (179/205) 2022-11-18T20:35:53.0235481Z Resolving deltas: 88% (181/205) 2022-11-18T20:35:53.0235735Z Resolving deltas: 89% (183/205) 2022-11-18T20:35:53.0235979Z Resolving deltas: 90% (185/205) 2022-11-18T20:35:53.0236226Z Resolving deltas: 91% (187/205) 2022-11-18T20:35:53.0236449Z Resolving deltas: 92% (189/205) 2022-11-18T20:35:53.0236687Z Resolving deltas: 93% (191/205) 2022-11-18T20:35:53.0236945Z Resolving deltas: 94% (193/205) 2022-11-18T20:35:53.0237162Z Resolving deltas: 95% (195/205) 2022-11-18T20:35:53.0237390Z Resolving deltas: 96% (197/205) 2022-11-18T20:35:53.0237621Z Resolving deltas: 97% (199/205) 2022-11-18T20:35:53.0237837Z Resolving deltas: 98% (201/205) 2022-11-18T20:35:53.0238065Z Resolving deltas: 99% (203/205) 2022-11-18T20:35:53.0238297Z Resolving deltas: 100% (205/205) 2022-11-18T20:35:53.0238523Z Resolving deltas: 100% (205/205), done. 2022-11-18T20:35:53.1427632Z From https://github.com/pytorch/serve 2022-11-18T20:35:53.1428769Z * [new ref] a6e30a23223298fac8fa80737306bf32d6b55b1a -> origin/upgrade_pytorch_113 2022-11-18T20:35:53.2546204Z ##[endgroup] 2022-11-18T20:35:53.2546881Z ##[group]Determining the checkout info 2022-11-18T20:35:53.2548495Z ##[endgroup] 2022-11-18T20:35:53.2548983Z ##[group]Checking out the ref 2022-11-18T20:35:53.2558725Z [command]"C:\Program Files\Git\bin\git.exe" checkout --progress --force -B upgrade_pytorch_113 refs/remotes/origin/upgrade_pytorch_113 2022-11-18T20:35:53.7556997Z branch 'upgrade_pytorch_113' set up to track 'origin/upgrade_pytorch_113'. 2022-11-18T20:35:53.7558232Z Switched to a new branch 'upgrade_pytorch_113' 2022-11-18T20:35:53.7627103Z ##[endgroup] 2022-11-18T20:35:53.7890302Z [command]"C:\Program Files\Git\bin\git.exe" log -1 --format='%H' 2022-11-18T20:35:53.8096629Z 'a6e30a23223298fac8fa80737306bf32d6b55b1a' 2022-11-18T20:35:53.8481857Z ##[group]Run python ts_scripts/install_dependencies.py --environment=dev 2022-11-18T20:35:53.8482408Z python ts_scripts/install_dependencies.py --environment=dev 2022-11-18T20:35:53.8583211Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'" 2022-11-18T20:35:53.8583514Z env: 2022-11-18T20:35:53.8583783Z pythonLocation: C:\hostedtoolcache\windows\Python\3.8.10\x64 2022-11-18T20:35:53.8584144Z JAVA_HOME: C:\hostedtoolcache\windows\Java_Zulu_jdk\17.0.5-8\x64 2022-11-18T20:35:53.8584510Z JAVA_HOME_17_X64: C:\hostedtoolcache\windows\Java_Zulu_jdk\17.0.5-8\x64 2022-11-18T20:35:53.8584792Z ##[endgroup] 2022-11-18T20:37:07.4347264Z npm WARN deprecated source-map-url@0.4.1: See https://github.com/lydell/source-map-url#deprecated 2022-11-18T20:37:08.9867094Z npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated 2022-11-18T20:37:09.7301444Z npm WARN deprecated gulp-header@1.8.12: Removed event-stream from gulp-header 2022-11-18T20:37:10.5768994Z npm WARN deprecated har-validator@5.1.5: this library is no longer supported 2022-11-18T20:37:11.4976048Z npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated 2022-11-18T20:37:11.5889334Z npm WARN deprecated source-map-resolve@0.5.3: See https://github.com/lydell/source-map-resolve#deprecated 2022-11-18T20:37:16.0728384Z npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. 2022-11-18T20:37:16.0808764Z npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. 2022-11-18T20:38:25.3077988Z 2022-11-18T20:38:25.3080592Z added 250 packages, changed 111 packages, and audited 362 packages in 2m 2022-11-18T20:38:25.3081528Z 2022-11-18T20:38:25.3081952Z 23 packages are looking for funding 2022-11-18T20:38:25.3084372Z run `npm fund` for details 2022-11-18T20:38:25.3084730Z 2022-11-18T20:38:25.3084946Z found 0 vulnerabilities 2022-11-18T20:38:25.8663265Z INFO: Could not find files for the given pattern(s). 2022-11-18T20:38:33.5043605Z Collecting wheel 2022-11-18T20:38:33.5352340Z Downloading wheel-0.38.4-py3-none-any.whl (36 kB) 2022-11-18T20:38:33.6862248Z Collecting torch==1.13.0 2022-11-18T20:38:33.7064207Z Downloading torch-1.13.0-cp38-cp38-win_amd64.whl (167.3 MB) 2022-11-18T20:38:37.0981194Z -------------------------------------- 167.3/167.3 MB 4.5 MB/s eta 0:00:00 2022-11-18T20:38:37.8596957Z Collecting torchvision==0.14.0 2022-11-18T20:38:37.8876332Z Downloading torchvision-0.14.0-cp38-cp38-win_amd64.whl (1.1 MB) 2022-11-18T20:38:37.9419750Z ---------------------------------------- 1.1/1.1 MB 17.3 MB/s eta 0:00:00 2022-11-18T20:38:38.0041476Z Collecting torchtext==0.14.0 2022-11-18T20:38:38.0165605Z Downloading torchtext-0.14.0-cp38-cp38-win_amd64.whl (1.9 MB) 2022-11-18T20:38:38.0704438Z ---------------------------------------- 1.9/1.9 MB 39.3 MB/s eta 0:00:00 2022-11-18T20:38:38.1146500Z Collecting torchaudio==0.13.0 2022-11-18T20:38:38.1340087Z Downloading torchaudio-0.13.0-cp38-cp38-win_amd64.whl (2.1 MB) 2022-11-18T20:38:38.1868375Z ---------------------------------------- 2.1/2.1 MB 43.6 MB/s eta 0:00:00 2022-11-18T20:38:38.3917599Z Collecting pillow==9.0.1 2022-11-18T20:38:38.4141116Z Downloading Pillow-9.0.1-cp38-cp38-win_amd64.whl (3.2 MB) 2022-11-18T20:38:38.4782746Z ---------------------------------------- 3.2/3.2 MB 52.0 MB/s eta 0:00:00 2022-11-18T20:38:38.5184626Z Collecting typing-extensions 2022-11-18T20:38:38.5245242Z Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB) 2022-11-18T20:38:38.5924311Z Collecting requests 2022-11-18T20:38:38.5984493Z Downloading requests-2.28.1-py3-none-any.whl (62 kB) 2022-11-18T20:38:38.6164813Z ---------------------------------------- 62.8/62.8 kB 3.3 MB/s eta 0:00:00 2022-11-18T20:38:38.8361235Z Collecting numpy 2022-11-18T20:38:38.8441881Z Downloading numpy-1.23.4-cp38-cp38-win_amd64.whl (14.7 MB) 2022-11-18T20:38:39.1857430Z --------------------------------------- 14.7/14.7 MB 34.4 MB/s eta 0:00:00 2022-11-18T20:38:39.3345030Z Collecting tqdm 2022-11-18T20:38:39.3402100Z Downloading tqdm-4.64.1-py2.py3-none-any.whl (78 kB) 2022-11-18T20:38:39.3556119Z ---------------------------------------- 78.5/78.5 kB 4.3 MB/s eta 0:00:00 2022-11-18T20:38:39.4152549Z Collecting certifi>=2017.4.17 2022-11-18T20:38:39.4206847Z Downloading certifi-2022.9.24-py3-none-any.whl (161 kB) 2022-11-18T20:38:39.4382887Z ------------------------------------- 161.1/161.1 kB 10.1 MB/s eta 0:00:00 2022-11-18T20:38:39.5091586Z Collecting charset-normalizer<3,>=2 2022-11-18T20:38:39.5147107Z Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB) 2022-11-18T20:38:39.5815594Z Collecting urllib3<1.27,>=1.21.1 2022-11-18T20:38:39.5873904Z Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB) 2022-11-18T20:38:39.6057364Z -------------------------------------- 140.4/140.4 kB 8.1 MB/s eta 0:00:00 2022-11-18T20:38:39.6312769Z Collecting idna<4,>=2.5 2022-11-18T20:38:39.6364309Z Downloading idna-3.4-py3-none-any.whl (61 kB) 2022-11-18T20:38:39.6650661Z ---------------------------------------- 61.5/61.5 kB 3.2 MB/s eta 0:00:00 2022-11-18T20:38:39.7322750Z Collecting colorama 2022-11-18T20:38:39.7376523Z Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB) 2022-11-18T20:38:39.9699845Z Installing collected packages: wheel, urllib3, typing-extensions, pillow, numpy, idna, colorama, charset-normalizer, certifi, tqdm, torch, requests, torchvision, torchtext, torchaudio 2022-11-18T20:39:42.7597468Z Successfully installed certifi-2022.9.24 charset-normalizer-2.1.1 colorama-0.4.6 idna-3.4 numpy-1.23.4 pillow-9.0.1 requests-2.28.1 torch-1.13.0 torchaudio-0.13.0 torchtext-0.14.0 torchvision-0.14.0 tqdm-4.64.1 typing-extensions-4.4.0 urllib3-1.26.12 wheel-0.38.4 2022-11-18T20:39:43.3935905Z 2022-11-18T20:39:43.3937092Z [notice] A new release of pip available: 22.3 -> 22.3.1 2022-11-18T20:39:43.3937726Z [notice] To update, run: python.exe -m pip install --upgrade pip 2022-11-18T20:39:44.6247759Z Requirement already satisfied: pip in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (22.3) 2022-11-18T20:39:44.7265979Z Collecting pip 2022-11-18T20:39:44.7568233Z Downloading pip-22.3.1-py3-none-any.whl (2.1 MB) 2022-11-18T20:39:44.8129296Z ---------------------------------------- 2.1/2.1 MB 32.9 MB/s eta 0:00:00 2022-11-18T20:39:44.8265113Z Requirement already satisfied: setuptools in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (56.0.0) 2022-11-18T20:39:45.0492819Z Collecting setuptools 2022-11-18T20:39:45.0550116Z Downloading setuptools-65.5.1-py3-none-any.whl (1.2 MB) 2022-11-18T20:39:45.0953829Z ---------------------------------------- 1.2/1.2 MB 26.6 MB/s eta 0:00:00 2022-11-18T20:39:45.2453971Z Installing collected packages: setuptools, pip 2022-11-18T20:39:45.2455501Z Attempting uninstall: setuptools 2022-11-18T20:39:45.2464987Z Found existing installation: setuptools 56.0.0 2022-11-18T20:39:46.2534631Z Uninstalling setuptools-56.0.0: 2022-11-18T20:39:46.5117940Z Successfully uninstalled setuptools-56.0.0 2022-11-18T20:39:48.6453515Z Attempting uninstall: pip 2022-11-18T20:39:48.6463030Z Found existing installation: pip 22.3 2022-11-18T20:39:49.1978817Z Uninstalling pip-22.3: 2022-11-18T20:39:49.2225179Z Successfully uninstalled pip-22.3 2022-11-18T20:39:51.7160257Z Successfully installed pip-22.3.1 setuptools-65.5.1 2022-11-18T20:39:53.4039840Z Ignoring numpy: markers 'sys_platform != "win32"' don't match your environment 2022-11-18T20:39:53.4042262Z Ignoring nvgpu: markers 'sys_platform != "win32"' don't match your environment 2022-11-18T20:39:53.4047654Z Ignoring intel_extension_for_pytorch: markers 'sys_platform != "win32" and sys_platform != "darwin"' don't match your environment 2022-11-18T20:39:53.4569035Z Collecting future 2022-11-18T20:39:53.4879536Z Downloading future-0.18.2.tar.gz (829 kB) 2022-11-18T20:39:53.5324834Z ------------------------------------- 829.2/829.2 kB 17.4 MB/s eta 0:00:00 2022-11-18T20:39:53.9319377Z Preparing metadata (setup.py): started 2022-11-18T20:39:54.6872489Z Preparing metadata (setup.py): finished with status 'done' 2022-11-18T20:39:54.8095510Z Collecting psutil 2022-11-18T20:39:54.8230511Z Downloading psutil-5.9.4-cp36-abi3-win_amd64.whl (252 kB) 2022-11-18T20:39:54.8430569Z ------------------------------------- 252.5/252.5 kB 16.1 MB/s eta 0:00:00 2022-11-18T20:39:55.0441801Z Collecting cython 2022-11-18T20:39:55.0544329Z Downloading Cython-0.29.32-py2.py3-none-any.whl (986 kB) 2022-11-18T20:39:55.0996617Z ------------------------------------- 986.3/986.3 kB 20.8 MB/s eta 0:00:00 2022-11-18T20:39:55.1083138Z Requirement already satisfied: wheel in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from -r requirements\common.txt (line 4)) (0.38.4) 2022-11-18T20:39:55.1474694Z Requirement already satisfied: requests in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from -r requirements\common.txt (line 5)) (2.28.1) 2022-11-18T20:39:55.1950284Z Requirement already satisfied: pillow==9.0.1 in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from -r requirements\common.txt (line 6)) (9.0.1) 2022-11-18T20:39:55.4005638Z Collecting captum 2022-11-18T20:39:55.4108629Z Downloading captum-0.5.0-py3-none-any.whl (1.4 MB) 2022-11-18T20:39:55.4517441Z ---------------------------------------- 1.4/1.4 MB 45.7 MB/s eta 0:00:00 2022-11-18T20:39:55.4930213Z Collecting packaging 2022-11-18T20:39:55.5013447Z Downloading packaging-21.3-py3-none-any.whl (40 kB) 2022-11-18T20:39:55.5472161Z -------------------------------------- 40.8/40.8 kB 953.4 kB/s eta 0:00:00 2022-11-18T20:39:55.7578547Z Collecting numpy==1.19.3 2022-11-18T20:39:55.7653792Z Downloading numpy-1.19.3-cp38-cp38-win_amd64.whl (13.3 MB) 2022-11-18T20:39:55.9521530Z --------------------------------------- 13.3/13.3 MB 65.5 MB/s eta 0:00:00 2022-11-18T20:39:56.0174356Z Collecting nvgpu==0.8.0 2022-11-18T20:39:56.0242620Z Downloading nvgpu-0.8.0-py2.py3-none-any.whl (9.6 kB) 2022-11-18T20:39:56.0630634Z Collecting mock 2022-11-18T20:39:56.0966974Z Downloading mock-4.0.3-py3-none-any.whl (28 kB) 2022-11-18T20:39:56.1871074Z Collecting pytest 2022-11-18T20:39:56.1931298Z Downloading pytest-7.2.0-py3-none-any.whl (316 kB) 2022-11-18T20:39:56.2168736Z ------------------------------------- 316.8/316.8 kB 20.4 MB/s eta 0:00:00 2022-11-18T20:39:56.2829985Z Collecting pylint==2.6.0 2022-11-18T20:39:56.2903382Z Downloading pylint-2.6.0-py3-none-any.whl (325 kB) 2022-11-18T20:39:56.3189626Z ------------------------------------- 325.7/325.7 kB 21.0 MB/s eta 0:00:00 2022-11-18T20:39:56.3552540Z Collecting pytest-mock 2022-11-18T20:39:56.3605110Z Downloading pytest_mock-3.10.0-py3-none-any.whl (9.3 kB) 2022-11-18T20:39:56.4004777Z Collecting pytest-cov 2022-11-18T20:39:56.4056193Z Downloading pytest_cov-4.0.0-py3-none-any.whl (21 kB) 2022-11-18T20:39:56.8475616Z Collecting grpcio 2022-11-18T20:39:56.8663710Z Downloading grpcio-1.50.0-cp38-cp38-win_amd64.whl (3.7 MB) 2022-11-18T20:39:56.9432563Z ---------------------------------------- 3.7/3.7 MB 46.8 MB/s eta 0:00:00 2022-11-18T20:39:57.1145648Z Collecting protobuf 2022-11-18T20:39:57.1282507Z Downloading protobuf-4.21.9-cp38-cp38-win_amd64.whl (525 kB) 2022-11-18T20:39:57.1566448Z ------------------------------------- 525.5/525.5 kB 32.2 MB/s eta 0:00:00 2022-11-18T20:39:57.5864619Z Collecting grpcio-tools 2022-11-18T20:39:57.5970909Z Downloading grpcio_tools-1.50.0-cp38-cp38-win_amd64.whl (1.8 MB) 2022-11-18T20:39:57.6430018Z ---------------------------------------- 1.8/1.8 MB 39.0 MB/s eta 0:00:00 2022-11-18T20:39:57.7041926Z Collecting transformers==4.11.0 2022-11-18T20:39:57.7219592Z Downloading transformers-4.11.0-py3-none-any.whl (2.9 MB) 2022-11-18T20:39:57.8201062Z ---------------------------------------- 2.9/2.9 MB 30.4 MB/s eta 0:00:00 2022-11-18T20:39:57.8608290Z Collecting pyspelling 2022-11-18T20:39:57.8677780Z Downloading pyspelling-2.8.1-py3-none-any.whl (45 kB) 2022-11-18T20:39:57.8811350Z ---------------------------------------- 45.3/45.3 kB 2.3 MB/s eta 0:00:00 2022-11-18T20:39:57.9599679Z Collecting pygit2==1.6.1 2022-11-18T20:39:57.9731294Z Downloading pygit2-1.6.1-cp38-cp38-win_amd64.whl (1.2 MB) 2022-11-18T20:39:58.0085974Z ---------------------------------------- 1.2/1.2 MB 36.8 MB/s eta 0:00:00 2022-11-18T20:39:58.0781425Z Collecting pre-commit 2022-11-18T20:39:58.0850632Z Downloading pre_commit-2.20.0-py2.py3-none-any.whl (199 kB) 2022-11-18T20:39:58.1033529Z ------------------------------------- 199.5/199.5 kB 11.8 MB/s eta 0:00:00 2022-11-18T20:39:58.1484532Z Collecting twine 2022-11-18T20:39:58.1539598Z Downloading twine-4.0.1-py3-none-any.whl (36 kB) 2022-11-18T20:39:58.2597332Z Collecting tabulate 2022-11-18T20:39:58.3867882Z Downloading tabulate-0.9.0-py3-none-any.whl (35 kB) 2022-11-18T20:39:58.4492772Z Collecting flask-restful 2022-11-18T20:39:58.4579976Z Downloading Flask_RESTful-0.3.9-py2.py3-none-any.whl (25 kB) 2022-11-18T20:39:58.4950271Z Collecting pynvml 2022-11-18T20:39:58.5014381Z Downloading pynvml-11.4.1-py3-none-any.whl (46 kB) 2022-11-18T20:39:58.5183375Z ---------------------------------------- 47.0/47.0 kB 2.4 MB/s eta 0:00:00 2022-11-18T20:39:58.5571909Z Collecting arrow 2022-11-18T20:39:58.5623549Z Downloading arrow-1.2.3-py3-none-any.whl (66 kB) 2022-11-18T20:39:58.5751221Z ---------------------------------------- 66.4/66.4 kB 3.5 MB/s eta 0:00:00 2022-11-18T20:39:58.6111331Z Collecting six 2022-11-18T20:39:58.6163418Z Downloading six-1.16.0-py2.py3-none-any.whl (11 kB) 2022-11-18T20:39:58.7581364Z Collecting pandas 2022-11-18T20:39:58.7650519Z Downloading pandas-1.5.1-cp38-cp38-win_amd64.whl (11.0 MB) 2022-11-18T20:39:58.9818798Z --------------------------------------- 11.0/11.0 MB 46.7 MB/s eta 0:00:00 2022-11-18T20:39:59.0534297Z Collecting ansi2html 2022-11-18T20:39:59.0604453Z Downloading ansi2html-1.8.0-py3-none-any.whl (16 kB) 2022-11-18T20:39:59.0928672Z Collecting termcolor 2022-11-18T20:39:59.1027225Z Downloading termcolor-2.1.0-py3-none-any.whl (5.8 kB) 2022-11-18T20:39:59.1423648Z Collecting flask 2022-11-18T20:39:59.1477869Z Downloading Flask-2.2.2-py3-none-any.whl (101 kB) 2022-11-18T20:39:59.1722353Z -------------------------------------- 101.5/101.5 kB 2.9 MB/s eta 0:00:00 2022-11-18T20:39:59.2073716Z Collecting mccabe<0.7,>=0.6 2022-11-18T20:39:59.2131512Z Downloading mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB) 2022-11-18T20:39:59.2880200Z Collecting isort<6,>=4.2.5 2022-11-18T20:39:59.2933441Z Downloading isort-5.10.1-py3-none-any.whl (103 kB) 2022-11-18T20:39:59.3132147Z -------------------------------------- 103.4/103.4 kB 5.8 MB/s eta 0:00:00 2022-11-18T20:39:59.3726824Z Collecting astroid<=2.5,>=2.4.0 2022-11-18T20:39:59.3853384Z Downloading astroid-2.5-py3-none-any.whl (220 kB) 2022-11-18T20:39:59.4046053Z ------------------------------------- 220.3/220.3 kB 13.1 MB/s eta 0:00:00 2022-11-18T20:39:59.4277958Z Collecting toml>=0.7.1 2022-11-18T20:39:59.4352254Z Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB) 2022-11-18T20:39:59.4477816Z Requirement already satisfied: colorama in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from pylint==2.6.0->-r requirements/developer.txt (line 4)) (0.4.6) 2022-11-18T20:39:59.7208110Z Collecting pyyaml>=5.1 2022-11-18T20:39:59.7292908Z Downloading PyYAML-6.0-cp38-cp38-win_amd64.whl (155 kB) 2022-11-18T20:39:59.7448261Z -------------------------------------- 155.4/155.4 kB 9.1 MB/s eta 0:00:00 2022-11-18T20:39:59.7486830Z Requirement already satisfied: tqdm>=4.27 in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from transformers==4.11.0->-r requirements/developer.txt (line 10)) (4.64.1) 2022-11-18T20:39:59.7943316Z Collecting huggingface-hub>=0.0.17 2022-11-18T20:39:59.8015698Z Downloading huggingface_hub-0.11.0-py3-none-any.whl (182 kB) 2022-11-18T20:39:59.8189994Z ------------------------------------- 182.1/182.1 kB 10.7 MB/s eta 0:00:00 2022-11-18T20:40:00.2456269Z Collecting regex!=2019.12.17 2022-11-18T20:40:00.2543433Z Downloading regex-2022.10.31-cp38-cp38-win_amd64.whl (267 kB) 2022-11-18T20:40:00.2728500Z ------------------------------------- 267.7/267.7 kB 16.1 MB/s eta 0:00:00 2022-11-18T20:40:00.3042831Z Collecting filelock 2022-11-18T20:40:00.3105685Z Downloading filelock-3.8.0-py3-none-any.whl (10 kB) 2022-11-18T20:40:00.4537120Z Collecting tokenizers<0.11,>=0.10.1 2022-11-18T20:40:00.4708718Z Downloading tokenizers-0.10.3-cp38-cp38-win_amd64.whl (2.0 MB) 2022-11-18T20:40:00.5184829Z ---------------------------------------- 2.0/2.0 MB 42.3 MB/s eta 0:00:00 2022-11-18T20:40:00.5482159Z Collecting sacremoses 2022-11-18T20:40:00.5601960Z Downloading sacremoses-0.0.53.tar.gz (880 kB) 2022-11-18T20:40:00.5926312Z ------------------------------------- 880.6/880.6 kB 28.1 MB/s eta 0:00:00 2022-11-18T20:40:00.6978243Z Preparing metadata (setup.py): started 2022-11-18T20:40:01.1027558Z Preparing metadata (setup.py): finished with status 'done' 2022-11-18T20:40:01.2327816Z Collecting cffi>=1.4.0 2022-11-18T20:40:01.2409990Z Downloading cffi-1.15.1-cp38-cp38-win_amd64.whl (178 kB) 2022-11-18T20:40:01.2596659Z ------------------------------------- 178.8/178.8 kB 11.2 MB/s eta 0:00:00 2022-11-18T20:40:01.2866884Z Requirement already satisfied: certifi>=2017.4.17 in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from requests->-r requirements\common.txt (line 5)) (2022.9.24) 2022-11-18T20:40:01.2874595Z Requirement already satisfied: idna<4,>=2.5 in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from requests->-r requirements\common.txt (line 5)) (3.4) 2022-11-18T20:40:01.2882123Z Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from requests->-r requirements\common.txt (line 5)) (1.26.12) 2022-11-18T20:40:01.2889867Z Requirement already satisfied: charset-normalizer<3,>=2 in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from requests->-r requirements\common.txt (line 5)) (2.1.1) 2022-11-18T20:40:01.4338145Z Collecting matplotlib 2022-11-18T20:40:01.4490164Z Downloading matplotlib-3.6.2-cp38-cp38-win_amd64.whl (7.2 MB) 2022-11-18T20:40:01.5731022Z ---------------------------------------- 7.2/7.2 MB 57.7 MB/s eta 0:00:00 2022-11-18T20:40:01.6045699Z Requirement already satisfied: torch>=1.6 in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from captum->-r requirements\common.txt (line 7)) (1.13.0) 2022-11-18T20:40:01.6566374Z Collecting pyparsing!=3.0.5,>=2.0.2 2022-11-18T20:40:01.6620192Z Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB) 2022-11-18T20:40:01.6824489Z ---------------------------------------- 98.3/98.3 kB 5.9 MB/s eta 0:00:00 2022-11-18T20:40:01.7265194Z Collecting iniconfig 2022-11-18T20:40:01.7320869Z Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB) 2022-11-18T20:40:01.7699302Z Collecting exceptiongroup>=1.0.0rc8 2022-11-18T20:40:01.7751283Z Downloading exceptiongroup-1.0.4-py3-none-any.whl (14 kB) 2022-11-18T20:40:01.8137607Z Collecting tomli>=1.0.0 2022-11-18T20:40:01.8221176Z Downloading tomli-2.0.1-py3-none-any.whl (12 kB) 2022-11-18T20:40:01.8743793Z Collecting attrs>=19.2.0 2022-11-18T20:40:01.8801070Z Downloading attrs-22.1.0-py2.py3-none-any.whl (58 kB) 2022-11-18T20:40:01.8962516Z ---------------------------------------- 58.8/58.8 kB 3.0 MB/s eta 0:00:00 2022-11-18T20:40:01.9276939Z Collecting pluggy<2.0,>=0.12 2022-11-18T20:40:01.9337666Z Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB) 2022-11-18T20:40:02.1926174Z Collecting coverage[toml]>=5.2.1 2022-11-18T20:40:02.2001278Z Downloading coverage-6.5.0-cp38-cp38-win_amd64.whl (188 kB) 2022-11-18T20:40:02.2201758Z ------------------------------------- 188.5/188.5 kB 11.9 MB/s eta 0:00:00 2022-11-18T20:40:02.2741586Z Requirement already satisfied: setuptools in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from grpcio-tools->-r requirements/developer.txt (line 9)) (65.5.1) 2022-11-18T20:40:02.4529625Z Collecting markdown 2022-11-18T20:40:02.4585047Z Downloading Markdown-3.4.1-py3-none-any.whl (93 kB) 2022-11-18T20:40:02.4825927Z ---------------------------------------- 93.3/93.3 kB 2.7 MB/s eta 0:00:00 2022-11-18T20:40:02.5199370Z Collecting wcmatch>=6.0.3 2022-11-18T20:40:02.5259983Z Downloading wcmatch-8.4.1-py3-none-any.whl (39 kB) 2022-11-18T20:40:02.8800444Z Collecting lxml 2022-11-18T20:40:02.9216744Z Downloading lxml-4.9.1-cp38-cp38-win_amd64.whl (3.6 MB) 2022-11-18T20:40:03.0038732Z ---------------------------------------- 3.6/3.6 MB 46.3 MB/s eta 0:00:00 2022-11-18T20:40:03.0394850Z Collecting html5lib 2022-11-18T20:40:03.0450145Z Downloading html5lib-1.1-py2.py3-none-any.whl (112 kB) 2022-11-18T20:40:03.0652842Z -------------------------------------- 112.2/112.2 kB 6.4 MB/s eta 0:00:00 2022-11-18T20:40:03.0944170Z Collecting beautifulsoup4 2022-11-18T20:40:03.1022625Z Downloading beautifulsoup4-4.11.1-py3-none-any.whl (128 kB) 2022-11-18T20:40:03.1196475Z -------------------------------------- 128.2/128.2 kB 7.4 MB/s eta 0:00:00 2022-11-18T20:40:03.1587445Z Collecting soupsieve>=1.8 2022-11-18T20:40:03.1712108Z Downloading soupsieve-2.3.2.post1-py3-none-any.whl (37 kB) 2022-11-18T20:40:03.2149920Z Collecting cfgv>=2.0.0 2022-11-18T20:40:03.2223435Z Downloading cfgv-3.3.1-py2.py3-none-any.whl (7.3 kB) 2022-11-18T20:40:03.3205190Z Collecting identify>=1.0.0 2022-11-18T20:40:03.3275693Z Downloading identify-2.5.9-py2.py3-none-any.whl (98 kB) 2022-11-18T20:40:03.3441297Z ---------------------------------------- 98.8/98.8 kB ? eta 0:00:00 2022-11-18T20:40:03.4583311Z Collecting virtualenv>=20.0.8 2022-11-18T20:40:03.4653944Z Downloading virtualenv-20.16.7-py3-none-any.whl (8.8 MB) 2022-11-18T20:40:03.6120197Z ---------------------------------------- 8.8/8.8 MB 70.7 MB/s eta 0:00:00 2022-11-18T20:40:03.6730039Z Collecting nodeenv>=0.11.1 2022-11-18T20:40:03.6783830Z Downloading nodeenv-1.7.0-py2.py3-none-any.whl (21 kB) 2022-11-18T20:40:03.7677410Z Collecting requests-toolbelt!=0.9.0,>=0.8.0 2022-11-18T20:40:03.7735103Z Downloading requests_toolbelt-0.10.1-py2.py3-none-any.whl (54 kB) 2022-11-18T20:40:03.7877988Z ---------------------------------------- 54.5/54.5 kB 2.8 MB/s eta 0:00:00 2022-11-18T20:40:03.8116498Z Collecting rfc3986>=1.4.0 2022-11-18T20:40:03.8168604Z Downloading rfc3986-2.0.0-py2.py3-none-any.whl (31 kB) 2022-11-18T20:40:03.8994910Z Collecting keyring>=15.1 2022-11-18T20:40:03.9093598Z Downloading keyring-23.11.0-py3-none-any.whl (36 kB) 2022-11-18T20:40:03.9922576Z Collecting importlib-metadata>=3.6 2022-11-18T20:40:03.9976795Z Downloading importlib_metadata-5.0.0-py3-none-any.whl (21 kB) 2022-11-18T20:40:04.1085098Z Collecting rich>=12.0.0 2022-11-18T20:40:04.1170666Z Downloading rich-12.6.0-py3-none-any.whl (237 kB) 2022-11-18T20:40:04.1337437Z ------------------------------------- 237.5/237.5 kB 15.2 MB/s eta 0:00:00 2022-11-18T20:40:04.1711941Z Collecting readme-renderer>=35.0 2022-11-18T20:40:04.1765214Z Downloading readme_renderer-37.3-py3-none-any.whl (14 kB) 2022-11-18T20:40:04.2146485Z Collecting pkginfo>=1.8.1 2022-11-18T20:40:04.2205421Z Downloading pkginfo-1.8.3-py2.py3-none-any.whl (26 kB) 2022-11-18T20:40:04.3135713Z Collecting wrapt<1.13,>=1.11 2022-11-18T20:40:04.3326154Z Downloading wrapt-1.12.1.tar.gz (27 kB) 2022-11-18T20:40:04.3588375Z Preparing metadata (setup.py): started 2022-11-18T20:40:04.7278172Z Preparing metadata (setup.py): finished with status 'done' 2022-11-18T20:40:04.7923693Z Collecting lazy-object-proxy>=1.4.0 2022-11-18T20:40:04.7980276Z Downloading lazy_object_proxy-1.8.0-cp38-cp38-win_amd64.whl (22 kB) 2022-11-18T20:40:04.8682864Z Collecting pycparser 2022-11-18T20:40:04.8732939Z Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB) 2022-11-18T20:40:04.8935881Z -------------------------------------- 118.7/118.7 kB 7.2 MB/s eta 0:00:00 2022-11-18T20:40:04.9746905Z Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages (from huggingface-hub>=0.0.17->transformers==4.11.0->-r requirements/developer.txt (line 10)) (4.4.0) 2022-11-18T20:40:05.0382959Z Collecting zipp>=0.5 2022-11-18T20:40:05.0440747Z Downloading zipp-3.10.0-py3-none-any.whl (6.2 kB) 2022-11-18T20:40:05.1259684Z Collecting jaraco.classes 2022-11-18T20:40:05.1358697Z Downloading jaraco.classes-3.2.3-py3-none-any.whl (6.0 kB) 2022-11-18T20:40:05.1693617Z Collecting pywin32-ctypes!=0.1.0,!=0.1.1 2022-11-18T20:40:05.1746282Z Downloading pywin32_ctypes-0.2.0-py2.py3-none-any.whl (28 kB) 2022-11-18T20:40:05.2802057Z Collecting bleach>=2.1.0 2022-11-18T20:40:05.3011609Z Downloading bleach-5.0.1-py3-none-any.whl (160 kB) 2022-11-18T20:40:05.3194695Z -------------------------------------- 160.9/160.9 kB 9.4 MB/s eta 0:00:00 2022-11-18T20:40:05.3678438Z Collecting Pygments>=2.5.1 2022-11-18T20:40:05.3734060Z Downloading Pygments-2.13.0-py3-none-any.whl (1.1 MB) 2022-11-18T20:40:05.4144863Z ---------------------------------------- 1.1/1.1 MB 23.2 MB/s eta 0:00:00 2022-11-18T20:40:05.4532341Z Collecting docutils>=0.13.1 2022-11-18T20:40:05.4620867Z Downloading docutils-0.19-py3-none-any.whl (570 kB) 2022-11-18T20:40:05.4871982Z ------------------------------------- 570.5/570.5 kB 35.0 MB/s eta 0:00:00 2022-11-18T20:40:05.5673435Z Collecting commonmark<0.10.0,>=0.9.0 2022-11-18T20:40:05.5756871Z Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB) 2022-11-18T20:40:05.5931207Z ---------------------------------------- 51.1/51.1 kB 2.7 MB/s eta 0:00:00 2022-11-18T20:40:05.7199136Z Collecting distlib<1,>=0.3.6 2022-11-18T20:40:05.7257323Z Downloading distlib-0.3.6-py2.py3-none-any.whl (468 kB) 2022-11-18T20:40:05.7488931Z ------------------------------------- 468.5/468.5 kB 28.6 MB/s eta 0:00:00 2022-11-18T20:40:05.7871314Z Collecting platformdirs<3,>=2.4 2022-11-18T20:40:05.8009441Z Downloading platformdirs-2.5.4-py3-none-any.whl (14 kB) 2022-11-18T20:40:05.8615616Z Collecting bracex>=2.1.1 2022-11-18T20:40:06.7519043Z Downloading bracex-2.3.post1-py3-none-any.whl (12 kB) 2022-11-18T20:40:06.7519588Z Collecting python-dateutil>=2.7.0 2022-11-18T20:40:06.7520052Z Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) 2022-11-18T20:40:06.7520618Z -------------------------------------- 247.7/247.7 kB 7.7 MB/s eta 0:00:00 2022-11-18T20:40:06.7520963Z Collecting Jinja2>=3.0 2022-11-18T20:40:06.7521400Z Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB) 2022-11-18T20:40:06.7521889Z -------------------------------------- 133.1/133.1 kB 7.7 MB/s eta 0:00:00 2022-11-18T20:40:06.7522223Z Collecting click>=8.0 2022-11-18T20:40:06.7522618Z Downloading click-8.1.3-py3-none-any.whl (96 kB) 2022-11-18T20:40:06.7523092Z ---------------------------------------- 96.6/96.6 kB 5.8 MB/s eta 0:00:00 2022-11-18T20:40:06.7523443Z Collecting itsdangerous>=2.0 2022-11-18T20:40:06.7523885Z Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB) 2022-11-18T20:40:06.7524912Z Collecting Werkzeug>=2.2.2 2022-11-18T20:40:06.7525355Z Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB) 2022-11-18T20:40:06.7525863Z ------------------------------------- 232.7/232.7 kB 14.8 MB/s eta 0:00:00 2022-11-18T20:40:06.7526182Z Collecting pytz 2022-11-18T20:40:06.7526570Z Downloading pytz-2022.6-py2.py3-none-any.whl (498 kB) 2022-11-18T20:40:06.7527174Z ------------------------------------- 498.1/498.1 kB 15.7 MB/s eta 0:00:00 2022-11-18T20:40:06.7527515Z Collecting aniso8601>=0.82 2022-11-18T20:40:06.7527923Z Downloading aniso8601-9.0.1-py2.py3-none-any.whl (52 kB) 2022-11-18T20:40:06.7528426Z ---------------------------------------- 52.8/52.8 kB 2.8 MB/s eta 0:00:00 2022-11-18T20:40:06.7528757Z Collecting webencodings 2022-11-18T20:40:06.7529186Z Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB) 2022-11-18T20:40:06.7529550Z Collecting kiwisolver>=1.0.1 2022-11-18T20:40:06.7529999Z Downloading kiwisolver-1.4.4-cp38-cp38-win_amd64.whl (55 kB) 2022-11-18T20:40:06.7530513Z ---------------------------------------- 55.4/55.4 kB 2.8 MB/s eta 0:00:00 2022-11-18T20:40:06.8255817Z Collecting contourpy>=1.0.1 2022-11-18T20:40:06.8332291Z Downloading contourpy-1.0.6-cp38-cp38-win_amd64.whl (163 kB) 2022-11-18T20:40:06.8520630Z -------------------------------------- 163.5/163.5 kB 9.6 MB/s eta 0:00:00 2022-11-18T20:40:06.8712147Z Collecting cycler>=0.10 2022-11-18T20:40:06.8762501Z Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB) 2022-11-18T20:40:06.9574689Z Collecting fonttools>=4.22.0 2022-11-18T20:40:06.9635744Z Downloading fonttools-4.38.0-py3-none-any.whl (965 kB) 2022-11-18T20:40:06.9996255Z ------------------------------------- 965.4/965.4 kB 29.8 MB/s eta 0:00:00 2022-11-18T20:40:07.0431154Z Collecting pandas 2022-11-18T20:40:07.0615689Z Downloading pandas-1.5.0-cp38-cp38-win_amd64.whl (11.0 MB) 2022-11-18T20:40:07.2495855Z --------------------------------------- 11.0/11.0 MB 59.5 MB/s eta 0:00:00 2022-11-18T20:40:07.3164929Z Downloading pandas-1.4.4-cp38-cp38-win_amd64.whl (10.6 MB) 2022-11-18T20:40:07.4861476Z --------------------------------------- 10.6/10.6 MB 65.6 MB/s eta 0:00:00 2022-11-18T20:40:07.6139900Z Collecting joblib 2022-11-18T20:40:07.6341538Z Downloading joblib-1.2.0-py3-none-any.whl (297 kB) 2022-11-18T20:40:07.6579713Z ------------------------------------- 298.0/298.0 kB 18.0 MB/s eta 0:00:00 2022-11-18T20:40:07.9781491Z Collecting MarkupSafe>=2.0 2022-11-18T20:40:07.9843982Z Downloading MarkupSafe-2.1.1-cp38-cp38-win_amd64.whl (17 kB) 2022-11-18T20:40:08.2174143Z Collecting more-itertools 2022-11-18T20:40:08.2227755Z Downloading more_itertools-9.0.0-py3-none-any.whl (52 kB) 2022-11-18T20:40:08.2362605Z ---------------------------------------- 52.8/52.8 kB 2.7 MB/s eta 0:00:00 2022-11-18T20:40:08.3548851Z Building wheels for collected packages: future, sacremoses, wrapt 2022-11-18T20:40:08.3566266Z Building wheel for future (setup.py): started 2022-11-18T20:40:10.3343764Z Building wheel for future (setup.py): finished with status 'done' 2022-11-18T20:40:10.3379180Z Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491086 sha256=6bb4fca9adfe273a308e2ac08cc90372d7fee78eb28a53bdf9352926855f89a7 2022-11-18T20:40:10.3381108Z Stored in directory: c:\users\runneradmin\appdata\local\pip\cache\wheels\01\49\0c\4e0a697824c7bd6516afb22e1af9d51427ccd36c74b05a297e 2022-11-18T20:40:10.3430921Z Building wheel for sacremoses (setup.py): started 2022-11-18T20:40:11.1751413Z Building wheel for sacremoses (setup.py): finished with status 'done' 2022-11-18T20:40:11.1797574Z Created wheel for sacremoses: filename=sacremoses-0.0.53-py3-none-any.whl size=895252 sha256=8428fda71e1f1632d376620caede75de8f693677c2e64c9c4961b918776cecd9 2022-11-18T20:40:11.1799477Z Stored in directory: c:\users\runneradmin\appdata\local\pip\cache\wheels\64\a3\ff\01dc060d7fc51176b3ce7cf1561466a12e658164b594747547 2022-11-18T20:40:11.1839567Z Building wheel for wrapt (setup.py): started 2022-11-18T20:40:43.4216626Z Building wheel for wrapt (setup.py): finished with status 'done' 2022-11-18T20:40:43.4237290Z Created wheel for wrapt: filename=wrapt-1.12.1-cp38-cp38-win_amd64.whl size=33031 sha256=a64cc6190bb27869e1d9b794db870d335b70a909de9a7d58c54a511dbd51006a 2022-11-18T20:40:43.4239588Z Stored in directory: c:\users\runneradmin\appdata\local\pip\cache\wheels\67\a1\46\13226bbf258069e74ce1598cd4077014852a5e63434be9f1c6 2022-11-18T20:40:43.4281978Z Successfully built future sacremoses wrapt 2022-11-18T20:40:44.1550470Z Installing collected packages: wrapt, webencodings, tokenizers, pywin32-ctypes, pytz, mccabe, iniconfig, distlib, commonmark, aniso8601, zipp, tomli, toml, termcolor, tabulate, soupsieve, six, rfc3986, regex, pyyaml, pyparsing, pynvml, Pygments, pycparser, psutil, protobuf, pluggy, platformdirs, pkginfo, numpy, nodeenv, more-itertools, mock, MarkupSafe, lxml, lazy-object-proxy, kiwisolver, joblib, itsdangerous, isort, identify, future, fonttools, filelock, exceptiongroup, docutils, cython, cycler, coverage, click, cfgv, bracex, attrs, ansi2html, Werkzeug, wcmatch, virtualenv, sacremoses, rich, requests-toolbelt, python-dateutil, packaging, Jinja2, jaraco.classes, importlib-metadata, html5lib, grpcio, contourpy, cffi, bleach, beautifulsoup4, astroid, readme-renderer, pytest, pylint, pygit2, pre-commit, pandas, matplotlib, markdown, keyring, huggingface-hub, grpcio-tools, flask, arrow, twine, transformers, pytest-mock, pytest-cov, pyspelling, flask-restful, captum, nvgpu 2022-11-18T20:40:58.7295487Z Attempting uninstall: numpy 2022-11-18T20:40:58.7306370Z Found existing installation: numpy 1.23.4 2022-11-18T20:40:59.2834426Z Uninstalling numpy-1.23.4: 2022-11-18T20:41:00.1877702Z Successfully uninstalled numpy-1.23.4 2022-11-18T20:42:05.6051714Z Successfully installed Jinja2-3.1.2 MarkupSafe-2.1.1 Pygments-2.13.0 Werkzeug-2.2.2 aniso8601-9.0.1 ansi2html-1.8.0 arrow-1.2.3 astroid-2.5 attrs-22.1.0 beautifulsoup4-4.11.1 bleach-5.0.1 bracex-2.3.post1 captum-0.5.0 cffi-1.15.1 cfgv-3.3.1 click-8.1.3 commonmark-0.9.1 contourpy-1.0.6 coverage-6.5.0 cycler-0.11.0 cython-0.29.32 distlib-0.3.6 docutils-0.19 exceptiongroup-1.0.4 filelock-3.8.0 flask-2.2.2 flask-restful-0.3.9 fonttools-4.38.0 future-0.18.2 grpcio-1.50.0 grpcio-tools-1.50.0 html5lib-1.1 huggingface-hub-0.11.0 identify-2.5.9 importlib-metadata-5.0.0 iniconfig-1.1.1 isort-5.10.1 itsdangerous-2.1.2 jaraco.classes-3.2.3 joblib-1.2.0 keyring-23.11.0 kiwisolver-1.4.4 lazy-object-proxy-1.8.0 lxml-4.9.1 markdown-3.4.1 matplotlib-3.6.2 mccabe-0.6.1 mock-4.0.3 more-itertools-9.0.0 nodeenv-1.7.0 numpy-1.19.3 nvgpu-0.8.0 packaging-21.3 pandas-1.4.4 pkginfo-1.8.3 platformdirs-2.5.4 pluggy-1.0.0 pre-commit-2.20.0 protobuf-4.21.9 psutil-5.9.4 pycparser-2.21 pygit2-1.6.1 pylint-2.6.0 pynvml-11.4.1 pyparsing-3.0.9 pyspelling-2.8.1 pytest-7.2.0 pytest-cov-4.0.0 pytest-mock-3.10.0 python-dateutil-2.8.2 pytz-2022.6 pywin32-ctypes-0.2.0 pyyaml-6.0 readme-renderer-37.3 regex-2022.10.31 requests-toolbelt-0.10.1 rfc3986-2.0.0 rich-12.6.0 sacremoses-0.0.53 six-1.16.0 soupsieve-2.3.2.post1 tabulate-0.9.0 termcolor-2.1.0 tokenizers-0.10.3 toml-0.10.2 tomli-2.0.1 transformers-4.11.0 twine-4.0.1 virtualenv-20.16.7 wcmatch-8.4.1 webencodings-0.5.1 wrapt-1.12.1 zipp-3.10.0 2022-11-18T20:42:07.9524284Z ##[group]Run python test/regression_tests.py 2022-11-18T20:42:07.9524784Z python test/regression_tests.py 2022-11-18T20:42:07.9571743Z shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'" 2022-11-18T20:42:07.9572061Z env: 2022-11-18T20:42:07.9572341Z pythonLocation: C:\hostedtoolcache\windows\Python\3.8.10\x64 2022-11-18T20:42:07.9572729Z JAVA_HOME: C:\hostedtoolcache\windows\Java_Zulu_jdk\17.0.5-8\x64 2022-11-18T20:42:07.9573114Z JAVA_HOME_17_X64: C:\hostedtoolcache\windows\Java_Zulu_jdk\17.0.5-8\x64 2022-11-18T20:42:07.9573395Z ##[endgroup] 2022-11-18T20:42:40.6586808Z Processing d:\a\serve\serve 2022-11-18T20:42:40.6605835Z Preparing metadata (setup.py): started 2022-11-18T20:42:41.1808550Z Preparing metadata (setup.py): finished with status 'done' 2022-11-18T20:42:41.4030320Z Collecting Pillow 2022-11-18T20:42:41.4585143Z Downloading Pillow-9.3.0-cp38-cp38-win_amd64.whl (2.5 MB) 2022-11-18T20:42:41.5209133Z ---------------------------------------- 2.5/2.5 MB 40.0 MB/s eta 0:00:00 2022-11-18T20:42:41.6378734Z Collecting psutil 2022-11-18T20:42:41.6412078Z Using cached psutil-5.9.4-cp36-abi3-win_amd64.whl (252 kB) 2022-11-18T20:42:41.6654485Z Collecting future 2022-11-18T20:42:41.6656037Z Using cached future-0.18.2-py3-none-any.whl 2022-11-18T20:42:41.8639058Z Collecting packaging 2022-11-18T20:42:41.8666918Z Using cached packaging-21.3-py3-none-any.whl (40 kB) 2022-11-18T20:42:41.9097364Z Collecting wheel 2022-11-18T20:42:41.9126588Z Using cached wheel-0.38.4-py3-none-any.whl (36 kB) 2022-11-18T20:42:41.9655316Z Collecting pyparsing!=3.0.5,>=2.0.2 2022-11-18T20:42:41.9685867Z Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB) 2022-11-18T20:42:42.0020890Z Building wheels for collected packages: torchserve 2022-11-18T20:42:42.0034691Z Building wheel for torchserve (setup.py): started 2022-11-18T20:43:25.5374050Z Building wheel for torchserve (setup.py): finished with status 'done' 2022-11-18T20:43:25.6006705Z Created wheel for torchserve: filename=torchserve-0.6.1b20221118-py3-none-any.whl size=19582107 sha256=4a24a0d120d537d6706ea12e53114dc24fd5ed56e62606e1f2f6afde27c4a5f7 2022-11-18T20:43:25.6008109Z Stored in directory: C:\Users\runneradmin\AppData\Local\Temp\pip-ephem-wheel-cache-bgkuxipv\wheels\a3\1e\5e\b868eb1c8523553c6a764472d4baef9c65a14a123d5d0fab12 2022-11-18T20:43:25.9599509Z Successfully built torchserve 2022-11-18T20:43:26.7135132Z Installing collected packages: wheel, pyparsing, psutil, Pillow, future, packaging, torchserve 2022-11-18T20:43:26.7136906Z Attempting uninstall: wheel 2022-11-18T20:43:26.7147728Z Found existing installation: wheel 0.38.4 2022-11-18T20:43:26.7360733Z Uninstalling wheel-0.38.4: 2022-11-18T20:43:26.7417596Z Successfully uninstalled wheel-0.38.4 2022-11-18T20:43:31.8366180Z Attempting uninstall: pyparsing 2022-11-18T20:43:31.8366864Z Found existing installation: pyparsing 3.0.9 2022-11-18T20:43:31.8367696Z Uninstalling pyparsing-3.0.9: 2022-11-18T20:43:31.8434650Z Successfully uninstalled pyparsing-3.0.9 2022-11-18T20:43:33.6164424Z Attempting uninstall: psutil 2022-11-18T20:43:33.6180830Z Found existing installation: psutil 5.9.4 2022-11-18T20:43:33.6486494Z Uninstalling psutil-5.9.4: 2022-11-18T20:43:33.6664325Z Successfully uninstalled psutil-5.9.4 2022-11-18T20:43:35.7688093Z Attempting uninstall: Pillow 2022-11-18T20:43:35.7703712Z Found existing installation: Pillow 9.0.1 2022-11-18T20:43:35.8633668Z Uninstalling Pillow-9.0.1: 2022-11-18T20:43:35.8699159Z Successfully uninstalled Pillow-9.0.1 2022-11-18T20:43:43.9089528Z Attempting uninstall: future 2022-11-18T20:43:43.9090318Z Found existing installation: future 0.18.2 2022-11-18T20:43:44.0769705Z Uninstalling future-0.18.2: 2022-11-18T20:43:44.1390856Z Successfully uninstalled future-0.18.2 2022-11-18T20:43:54.8629212Z Attempting uninstall: packaging 2022-11-18T20:43:54.8646718Z Found existing installation: packaging 21.3 2022-11-18T20:43:54.8800352Z Uninstalling packaging-21.3: 2022-11-18T20:43:54.8917907Z Successfully uninstalled packaging-21.3 2022-11-18T20:43:57.5186317Z Successfully installed Pillow-9.3.0 future-0.18.2 packaging-21.3 psutil-5.9.4 pyparsing-3.0.9 torchserve-0.6.1b20221118 wheel-0.38.4 2022-11-18T20:43:59.4706014Z Processing d:\a\serve\serve\model-archiver 2022-11-18T20:43:59.4724379Z Preparing metadata (setup.py): started 2022-11-18T20:44:00.1009309Z Preparing metadata (setup.py): finished with status 'done' 2022-11-18T20:44:00.2789444Z Collecting future 2022-11-18T20:44:00.2903908Z Using cached future-0.18.2-py3-none-any.whl 2022-11-18T20:44:00.4961809Z Collecting enum-compat 2022-11-18T20:44:00.5340240Z Downloading enum_compat-0.0.3-py3-none-any.whl (1.3 kB) 2022-11-18T20:44:00.6216735Z Building wheels for collected packages: torch-model-archiver 2022-11-18T20:44:00.6229858Z Building wheel for torch-model-archiver (setup.py): started 2022-11-18T20:44:01.5439552Z Building wheel for torch-model-archiver (setup.py): finished with status 'done' 2022-11-18T20:44:01.5461354Z Created wheel for torch-model-archiver: filename=torch_model_archiver-0.6.1b20221118-py3-none-any.whl size=14698 sha256=fbdb00e5b209eac1f2b8d393864cf6b5733399008cd55d10fd6490a3cea25fe1 2022-11-18T20:44:01.5462953Z Stored in directory: c:\users\runneradmin\appdata\local\pip\cache\wheels\43\f4\86\f8971a03f8f36fb9f3eb68143bf599d6a26e1696c996f38776 2022-11-18T20:44:01.8756190Z Successfully built torch-model-archiver 2022-11-18T20:44:02.5408311Z Installing collected packages: enum-compat, future, torch-model-archiver 2022-11-18T20:44:02.8606096Z Attempting uninstall: future 2022-11-18T20:44:02.8620754Z Found existing installation: future 0.18.2 2022-11-18T20:44:03.0581655Z Uninstalling future-0.18.2: 2022-11-18T20:44:03.0859231Z Successfully uninstalled future-0.18.2 2022-11-18T20:44:14.0623412Z Successfully installed enum-compat-0.0.3 future-0.18.2 torch-model-archiver-0.6.1b20221118 2022-11-18T20:44:16.4273897Z Processing d:\a\serve\serve\workflow-archiver 2022-11-18T20:44:16.4293352Z Preparing metadata (setup.py): started 2022-11-18T20:44:17.0667961Z Preparing metadata (setup.py): finished with status 'done' 2022-11-18T20:44:17.0714336Z Building wheels for collected packages: torch-workflow-archiver 2022-11-18T20:44:17.0728924Z Building wheel for torch-workflow-archiver (setup.py): started 2022-11-18T20:44:17.9827853Z Building wheel for torch-workflow-archiver (setup.py): finished with status 'done' 2022-11-18T20:44:17.9850851Z Created wheel for torch-workflow-archiver: filename=torch_workflow_archiver-0.2.5b20221118-py3-none-any.whl size=12806 sha256=da27fe0f85c030c25e89fbb3f327b71e8a137a0fe19271f4da272110838e6277 2022-11-18T20:44:17.9852748Z Stored in directory: c:\users\runneradmin\appdata\local\pip\cache\wheels\b4\98\c6\f20cec44b7266f36a9c27e40b1a5f580cc33eff899fd63717f 2022-11-18T20:44:18.6139827Z Successfully built torch-workflow-archiver 2022-11-18T20:44:19.3271302Z Installing collected packages: torch-workflow-archiver 2022-11-18T20:44:21.2679616Z Successfully installed torch-workflow-archiver-0.2.5b20221118 2022-11-18T20:46:42.8891223Z newman 2022-11-18T20:46:42.8891564Z 2022-11-18T20:46:42.8894421Z management_api_collection 2022-11-18T20:46:42.8899479Z 2022-11-18T20:46:42.8899613Z Iteration 1/82 2022-11-18T20:46:42.8957453Z 2022-11-18T20:46:42.9015903Z → management request 2022-11-18T20:46:43.3453776Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:46:43.3794772Z 200 OK ★ 295ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:43.3795733Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:46:43.3801210Z │ { 2022-11-18T20:46:43.3801618Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:46:43.3802061Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:46:43.3803415Z │ o add workers for the model." 2022-11-18T20:46:43.3803751Z │ } 2022-11-18T20:46:43.3803953Z └ 2022-11-18T20:46:43.3804425Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:43.3811253Z 75ms 12ms 9ms 595µs 263ms 7ms 580µs 370ms 2022-11-18T20:46:43.3811471Z 2022-11-18T20:46:43.3811630Z √ Successful request 2022-11-18T20:46:43.3811799Z 2022-11-18T20:46:43.3811910Z Iteration 2/82 2022-11-18T20:46:43.3812065Z 2022-11-18T20:46:43.3812223Z → management request 2022-11-18T20:46:43.6324105Z POST http://localhost:8081/models?url=mnist.mar&model_name=mnist 2022-11-18T20:46:43.6325410Z 200 OK ★ 204ms time ★ 283B↑ 401B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:43.6326232Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 134B 2022-11-18T20:46:43.6328493Z │ { 2022-11-18T20:46:43.6328990Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:46:43.6329651Z │ ith 0 initial workers. Use scale workers API to add wo 2022-11-18T20:46:43.6330170Z │ rkers for the model." 2022-11-18T20:46:43.6330566Z │ } 2022-11-18T20:46:43.6348171Z └ 2022-11-18T20:46:43.6348701Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:43.6349419Z 1ms 868µs (cache) (cache) 199ms 2ms 98µs 205ms 2022-11-18T20:46:43.6349748Z 2022-11-18T20:46:43.6408841Z √ Successful request 2022-11-18T20:46:43.6608826Z 2022-11-18T20:46:43.6609321Z Iteration 3/82 2022-11-18T20:46:43.6764712Z 2022-11-18T20:46:43.6765534Z → management request 2022-11-18T20:46:45.7845601Z POST http://localhost:8081/models?url=densenet161.mar&model_name=densenet161 2022-11-18T20:46:45.7846825Z 200 OK ★ 2s time ★ 295B↑ 407B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:45.7847487Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 140B 2022-11-18T20:46:45.7847952Z │ { 2022-11-18T20:46:45.7848464Z │ "status": "Model \"densenet161\" Version: 1.0 regist 2022-11-18T20:46:45.7848963Z │ ered with 0 initial workers. Use scale workers API to 2022-11-18T20:46:45.7849462Z │ add workers for the model." 2022-11-18T20:46:45.7849867Z │ } 2022-11-18T20:46:45.7865409Z └ 2022-11-18T20:46:45.7865923Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:45.7866611Z 1ms 306µs (cache) (cache) 2s 2ms 72µs 2s 2022-11-18T20:46:45.7866968Z 2022-11-18T20:46:45.8084763Z √ Successful request 2022-11-18T20:46:45.8179768Z 2022-11-18T20:46:45.8180285Z Iteration 4/82 2022-11-18T20:46:45.8338746Z 2022-11-18T20:46:45.8339522Z → management request 2022-11-18T20:46:45.8909246Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/densenet161.mar&model_name=densenet161 2022-11-18T20:46:45.8911098Z 500 Internal Server Error ★ 7ms time ★ 336B↑ 394B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:45.8911971Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 113B 2022-11-18T20:46:45.8913795Z │ { 2022-11-18T20:46:45.8914300Z │ "code": 500, 2022-11-18T20:46:45.8914861Z │ "type": "InternalServerException", 2022-11-18T20:46:45.8915502Z │ "message": "Model file already exists densenet161.ma 2022-11-18T20:46:45.8915956Z │ r" 2022-11-18T20:46:45.8916378Z │ } 2022-11-18T20:46:45.8938520Z └ 2022-11-18T20:46:45.8942935Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:45.8943725Z 1ms 309µs (cache) (cache) 4ms 2ms 83µs 8ms 2022-11-18T20:46:45.8944657Z 2022-11-18T20:46:45.8987493Z √ Successful request 2022-11-18T20:46:45.9118709Z 2022-11-18T20:46:45.9119307Z Iteration 5/82 2022-11-18T20:46:45.9278719Z 2022-11-18T20:46:45.9279735Z → management request 2022-11-18T20:46:46.0092709Z DELETE http://localhost:8081/models/densenet161 2022-11-18T20:46:46.0094062Z 200 OK ★ 32ms time ★ 247B↑ 319B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:46.0094720Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 53B 2022-11-18T20:46:46.0095192Z │ { 2022-11-18T20:46:46.0095666Z │ "status": "Model \"densenet161\" unregistered" 2022-11-18T20:46:46.0096099Z │ } 2022-11-18T20:46:46.0110793Z └ 2022-11-18T20:46:46.0111562Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:46.0112134Z 1ms 1ms 392µs 279µs 27ms 1ms 74µs 32ms 2022-11-18T20:46:46.0112343Z 2022-11-18T20:46:46.0260485Z √ Successful request 2022-11-18T20:46:46.0366697Z 2022-11-18T20:46:46.0367195Z Iteration 6/82 2022-11-18T20:46:46.0527505Z 2022-11-18T20:46:46.0528458Z → management request 2022-11-18T20:46:46.1105699Z POST http://localhost:8081/models [errored] 2022-11-18T20:46:46.1106514Z read ECONNRESET 2022-11-18T20:46:46.1248551Z 2. Successful request 2022-11-18T20:46:46.1305716Z 2022-11-18T20:46:46.1306147Z Iteration 7/82 2022-11-18T20:46:46.1463890Z 2022-11-18T20:46:46.1464584Z → management request 2022-11-18T20:46:46.2128303Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:46:46.2129546Z 200 OK ★ 17ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:46.2130228Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:46:46.2130697Z │ { 2022-11-18T20:46:46.2131162Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:46:46.2131552Z │ } 2022-11-18T20:46:46.2144080Z └ 2022-11-18T20:46:46.2144634Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:46.2145317Z 1ms 883µs 397µs 284µs 13ms 1ms 64µs 18ms 2022-11-18T20:46:46.2145683Z 2022-11-18T20:46:46.2308614Z √ Successful request 2022-11-18T20:46:46.2401563Z 2022-11-18T20:46:46.2401916Z Iteration 8/82 2022-11-18T20:46:46.2555979Z 2022-11-18T20:46:46.2556752Z → management request 2022-11-18T20:46:46.4040345Z POST http://localhost:8081/models?url=mnist.mar&model_name=mnist&handler=invalidHandler 2022-11-18T20:46:46.4041517Z 200 OK ★ 97ms time ★ 306B↑ 401B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:46.4042014Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 134B 2022-11-18T20:46:46.4042408Z │ { 2022-11-18T20:46:46.4042863Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:46:46.4043406Z │ ith 0 initial workers. Use scale workers API to add wo 2022-11-18T20:46:46.4043785Z │ rkers for the model." 2022-11-18T20:46:46.4044130Z │ } 2022-11-18T20:46:46.4059815Z └ 2022-11-18T20:46:46.4060532Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:46.4061165Z 1ms 265µs (cache) (cache) 94ms 1ms 68µs 98ms 2022-11-18T20:46:46.4065050Z 2022-11-18T20:46:46.4166689Z √ Successful request 2022-11-18T20:46:46.4274597Z 2022-11-18T20:46:46.4275039Z Iteration 9/82 2022-11-18T20:46:46.4432005Z 2022-11-18T20:46:46.4432698Z → management request 2022-11-18T20:46:46.4849995Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:46:46.4851153Z 200 OK ★ 8ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:46.4851695Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:46:46.4852074Z │ { 2022-11-18T20:46:46.4852490Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:46:46.4852866Z │ } 2022-11-18T20:46:46.4867738Z └ 2022-11-18T20:46:46.4868443Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:46.4869099Z 1ms 403µs (cache) (cache) 5ms 1ms 54µs 9ms 2022-11-18T20:46:46.4869367Z 2022-11-18T20:46:46.4950361Z √ Successful request 2022-11-18T20:46:46.5056335Z 2022-11-18T20:46:46.5056817Z Iteration 10/82 2022-11-18T20:46:46.5214570Z 2022-11-18T20:46:46.5215277Z → management request 2022-11-18T20:46:46.6684221Z POST http://localhost:8081/models?url=mnist.mar&model_name=mnist&handler=invalidHandler 2022-11-18T20:46:46.6685401Z 200 OK ★ 98ms time ★ 306B↑ 401B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:46.6687037Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 134B 2022-11-18T20:46:46.6687399Z │ { 2022-11-18T20:46:46.6687846Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:46:46.6688345Z │ ith 0 initial workers. Use scale workers API to add wo 2022-11-18T20:46:46.6688765Z │ rkers for the model." 2022-11-18T20:46:46.6689069Z │ } 2022-11-18T20:46:46.6703428Z └ 2022-11-18T20:46:46.6704101Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:46.6704954Z 1ms 333µs (cache) (cache) 94ms 2ms 68µs 99ms 2022-11-18T20:46:46.6705175Z 2022-11-18T20:46:46.6830174Z √ Successful request 2022-11-18T20:46:46.6932046Z 2022-11-18T20:46:46.6932539Z Iteration 11/82 2022-11-18T20:46:46.7097437Z 2022-11-18T20:46:46.7098180Z → management request 2022-11-18T20:46:48.5299835Z PUT http://localhost:8081/models/mnist?min_worker=1&synchronous=true [errored] 2022-11-18T20:46:48.5303940Z read ECONNRESET 2022-11-18T20:46:48.5520066Z 4. Successful request 2022-11-18T20:46:48.5582167Z 2022-11-18T20:46:48.5582572Z Iteration 12/82 2022-11-18T20:46:48.5585383Z 2022-11-18T20:46:48.5585910Z → management request 2022-11-18T20:46:48.6198167Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:46:48.6199269Z 200 OK ★ 12ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:48.6200012Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:46:48.6200480Z │ { 2022-11-18T20:46:48.6200946Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:46:48.6201328Z │ } 2022-11-18T20:46:48.6213217Z └ 2022-11-18T20:46:48.6213880Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:48.6214583Z 1ms 1ms 266µs 274µs 8ms 1ms 70µs 13ms 2022-11-18T20:46:48.6214909Z 2022-11-18T20:46:48.6287017Z √ Successful request 2022-11-18T20:46:48.6367145Z 2022-11-18T20:46:48.6367532Z Iteration 13/82 2022-11-18T20:46:48.6525129Z 2022-11-18T20:46:48.6525940Z → management request 2022-11-18T20:46:48.7094027Z GET http://localhost:8081/models/squeezenet1_1/all 2022-11-18T20:46:48.7095258Z 200 OK ★ 8ms time ★ 250B↑ 538B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:48.7095949Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 271B 2022-11-18T20:46:48.7096471Z │ [ 2022-11-18T20:46:48.7096848Z │ { 2022-11-18T20:46:48.7097251Z │ "modelName": "squeezenet1_1", 2022-11-18T20:46:48.7097721Z │ "modelVersion": "1.0", 2022-11-18T20:46:48.7098215Z │ "modelUrl": "squeezenet1_1.mar", 2022-11-18T20:46:48.7098709Z │ "runtime": "python", 2022-11-18T20:46:48.7099104Z │ "minWorkers": 0, 2022-11-18T20:46:48.7099528Z │ "maxWorkers": 0, 2022-11-18T20:46:48.7099949Z │ "batchSize": 1, 2022-11-18T20:46:48.7100354Z │ "maxBatchDelay": 100, 2022-11-18T20:46:48.7100832Z │ "loadedAtStartup": false, 2022-11-18T20:46:48.7101264Z │ "workers": [] 2022-11-18T20:46:48.7101654Z │ } 2022-11-18T20:46:48.7101973Z │ ] 2022-11-18T20:46:48.7108202Z └ 2022-11-18T20:46:48.7108792Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:48.7109531Z 1ms 668µs (cache) (cache) 5ms 1ms 164µs 9ms 2022-11-18T20:46:48.7109882Z 2022-11-18T20:46:48.7206935Z √ Successful request 2022-11-18T20:46:48.7305588Z 2022-11-18T20:46:48.7306153Z Iteration 14/82 2022-11-18T20:46:48.7464712Z 2022-11-18T20:46:48.7465624Z → management request 2022-11-18T20:46:48.8000777Z GET http://localhost:8081/models/squeezenet1_1/1.0 2022-11-18T20:46:48.8002095Z 200 OK ★ 5ms time ★ 250B↑ 538B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:48.8002818Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 271B 2022-11-18T20:46:48.8003239Z │ [ 2022-11-18T20:46:48.8003613Z │ { 2022-11-18T20:46:48.8004056Z │ "modelName": "squeezenet1_1", 2022-11-18T20:46:48.8004514Z │ "modelVersion": "1.0", 2022-11-18T20:46:48.8004946Z │ "modelUrl": "squeezenet1_1.mar", 2022-11-18T20:46:48.8005406Z │ "runtime": "python", 2022-11-18T20:46:48.8005852Z │ "minWorkers": 0, 2022-11-18T20:46:48.8006235Z │ "maxWorkers": 0, 2022-11-18T20:46:48.8006650Z │ "batchSize": 1, 2022-11-18T20:46:48.8007090Z │ "maxBatchDelay": 100, 2022-11-18T20:46:48.8007728Z │ "loadedAtStartup": false, 2022-11-18T20:46:48.8008113Z │ "workers": [] 2022-11-18T20:46:48.8008500Z │ } 2022-11-18T20:46:48.8008862Z │ ] 2022-11-18T20:46:48.8014001Z └ 2022-11-18T20:46:48.8014727Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:48.8015380Z 1ms 543µs (cache) (cache) 2ms 1ms 69µs 5ms 2022-11-18T20:46:48.8016200Z 2022-11-18T20:46:48.8203316Z √ Successful request 2022-11-18T20:46:48.8243632Z 2022-11-18T20:46:48.8244042Z Iteration 15/82 2022-11-18T20:46:48.8402869Z 2022-11-18T20:46:48.8403805Z → management request 2022-11-18T20:46:48.8935967Z GET http://localhost:8081/models/squeezenet1_1 2022-11-18T20:46:48.8937200Z 200 OK ★ 4ms time ★ 246B↑ 538B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:48.8937895Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 271B 2022-11-18T20:46:48.8938411Z │ [ 2022-11-18T20:46:48.8938738Z │ { 2022-11-18T20:46:48.8939188Z │ "modelName": "squeezenet1_1", 2022-11-18T20:46:48.8939665Z │ "modelVersion": "1.0", 2022-11-18T20:46:48.8940162Z │ "modelUrl": "squeezenet1_1.mar", 2022-11-18T20:46:48.8940614Z │ "runtime": "python", 2022-11-18T20:46:48.8941061Z │ "minWorkers": 0, 2022-11-18T20:46:48.8958450Z │ "maxWorkers": 0, 2022-11-18T20:46:48.8959383Z │ "batchSize": 1, 2022-11-18T20:46:48.8959863Z │ "maxBatchDelay": 100, 2022-11-18T20:46:48.8960338Z │ "loadedAtStartup": false, 2022-11-18T20:46:48.8960783Z │ "workers": [] 2022-11-18T20:46:48.8961128Z │ } 2022-11-18T20:46:48.8961499Z │ ] 2022-11-18T20:46:48.8962540Z └ 2022-11-18T20:46:48.8963090Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:48.8963663Z 1ms 268µs (cache) (cache) 1ms 1ms 55µs 5ms 2022-11-18T20:46:48.8963899Z 2022-11-18T20:46:48.9066380Z √ Successful request 2022-11-18T20:46:48.9180901Z 2022-11-18T20:46:48.9181454Z Iteration 16/82 2022-11-18T20:46:48.9339759Z 2022-11-18T20:46:48.9340616Z → management request 2022-11-18T20:46:48.9928574Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:46:48.9929686Z 200 OK ★ 8ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:48.9930359Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:46:48.9930797Z │ { 2022-11-18T20:46:48.9931302Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:46:48.9931746Z │ } 2022-11-18T20:46:48.9932059Z └ 2022-11-18T20:46:48.9932641Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:48.9933287Z 1ms 502µs (cache) (cache) 5ms 1ms 49µs 9ms 2022-11-18T20:46:48.9933595Z 2022-11-18T20:46:49.0012850Z √ Successful request 2022-11-18T20:46:49.0157384Z 2022-11-18T20:46:49.0157955Z Iteration 17/82 2022-11-18T20:46:49.0278671Z 2022-11-18T20:46:49.0279581Z → management request 2022-11-18T20:46:49.0804937Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&runtime=python4 [errored] 2022-11-18T20:46:49.0805696Z read ECONNRESET 2022-11-18T20:46:49.0948024Z 6. Successful request 2022-11-18T20:46:49.1058345Z 2022-11-18T20:46:49.1058946Z Iteration 18/82 2022-11-18T20:46:49.1218313Z 2022-11-18T20:46:49.1219313Z → management request 2022-11-18T20:46:49.1777714Z GET http://localhost:8081/models?limit=&next_page_token= 2022-11-18T20:46:49.1778913Z 200 OK ★ 7ms time ★ 256B↑ 285B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:49.1779609Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 19B 2022-11-18T20:46:49.1780077Z │ { 2022-11-18T20:46:49.1780423Z │ "models": [] 2022-11-18T20:46:49.1780808Z │ } 2022-11-18T20:46:49.1790604Z └ 2022-11-18T20:46:49.1791155Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:49.1792030Z 1ms 1ms 444µs 273µs 3ms 1ms 71µs 8ms 2022-11-18T20:46:49.1792361Z 2022-11-18T20:46:49.1841240Z √ Successful request 2022-11-18T20:46:49.1883108Z 2022-11-18T20:46:49.1883435Z Iteration 19/82 2022-11-18T20:46:49.1999085Z 2022-11-18T20:46:49.2000350Z → management request 2022-11-18T20:46:49.3480159Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:46:49.3481253Z 200 OK ★ 98ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:49.3481788Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:46:49.3482193Z │ { 2022-11-18T20:46:49.3482637Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:46:49.3483092Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:46:49.3483537Z │ o add workers for the model." 2022-11-18T20:46:49.3483926Z │ } 2022-11-18T20:46:49.3488422Z └ 2022-11-18T20:46:49.3488985Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:49.3489613Z 1ms 261µs (cache) (cache) 94ms 2ms 66µs 99ms 2022-11-18T20:46:49.3489880Z 2022-11-18T20:46:49.3636280Z √ Successful request 2022-11-18T20:46:49.3715422Z 2022-11-18T20:46:49.3716308Z Iteration 20/82 2022-11-18T20:46:49.3872460Z 2022-11-18T20:46:49.3873237Z → management request 2022-11-18T20:46:49.4265927Z PUT http://localhost:8081/models/squeezenet1_1?min_worker=1 2022-11-18T20:46:49.4267515Z 202 Accepted ★ 5ms time ★ 278B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:49.4268154Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:46:49.4268545Z │ { 2022-11-18T20:46:49.4269039Z │ "status": "Processing worker updates..." 2022-11-18T20:46:49.4269472Z │ } 2022-11-18T20:46:49.4269769Z └ 2022-11-18T20:46:49.4270404Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:49.4271119Z 1ms 259µs (cache) (cache) 2ms 1ms 42µs 5ms 2022-11-18T20:46:49.4271419Z 2022-11-18T20:46:49.4413562Z √ Successful request 2022-11-18T20:46:49.4495482Z 2022-11-18T20:46:49.4495914Z Iteration 21/82 2022-11-18T20:46:49.4655093Z 2022-11-18T20:46:49.4655765Z → management request 2022-11-18T20:46:49.5201300Z PUT http://localhost:8081/models/squeezenet1_1?min_worker=1&synchronous=true 2022-11-18T20:46:49.5202443Z 200 OK ★ 5ms time ★ 295B↑ 329B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:49.5202926Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 63B 2022-11-18T20:46:49.5203330Z │ { 2022-11-18T20:46:49.5203779Z │ "status": "Workers scaled to 1 for model: squeezenet 2022-11-18T20:46:49.5204164Z │ 1_1" 2022-11-18T20:46:49.5204436Z │ } 2022-11-18T20:46:49.5204781Z └ 2022-11-18T20:46:49.5205334Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:49.5205943Z 1ms 253µs (cache) (cache) 2ms 1ms 38µs 5ms 2022-11-18T20:46:49.5206160Z 2022-11-18T20:46:49.5338730Z √ Successful request 2022-11-18T20:46:49.5477612Z 2022-11-18T20:46:49.5478524Z Iteration 22/82 2022-11-18T20:46:49.5577252Z 2022-11-18T20:46:49.5578231Z → management request 2022-11-18T20:46:49.6013231Z PUT http://localhost:8081/models/squeezenet1_1/1.0?min_worker=1&synchronous=true 2022-11-18T20:46:49.6014104Z 200 OK ★ 5ms time ★ 299B↑ 343B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:49.6014997Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 77B 2022-11-18T20:46:49.6015412Z │ { 2022-11-18T20:46:49.6015858Z │ "status": "Workers scaled to 1 for model: squeezenet 2022-11-18T20:46:49.6016229Z │ 1_1, version: 1.0" 2022-11-18T20:46:49.6016571Z │ } 2022-11-18T20:46:49.6016883Z └ 2022-11-18T20:46:49.6017962Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:49.6018600Z 1ms 249µs (cache) (cache) 2ms 1ms 45µs 5ms 2022-11-18T20:46:49.6018866Z 2022-11-18T20:46:49.6119935Z √ Successful request 2022-11-18T20:46:49.6201834Z 2022-11-18T20:46:49.6202563Z Iteration 23/82 2022-11-18T20:46:49.6363152Z 2022-11-18T20:46:49.6363907Z → management request 2022-11-18T20:46:49.6880318Z PUT http://localhost:8081/models/squeezenet1_1/0.0?min_worker=1&synchronous=true [errored] 2022-11-18T20:46:49.6881013Z read ECONNRESET 2022-11-18T20:46:49.7135547Z 8. Successful request 2022-11-18T20:46:49.7456846Z 2022-11-18T20:46:49.7457446Z Iteration 24/82 2022-11-18T20:46:49.7616084Z 2022-11-18T20:46:49.7616840Z → management request 2022-11-18T20:46:49.8172951Z PUT http://localhost:8081/models/squeezenet1_1?min_worker=1&number_gpu=1 2022-11-18T20:46:49.8174164Z 202 Accepted ★ 6ms time ★ 291B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:49.8174879Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:46:49.8175340Z │ { 2022-11-18T20:46:49.8175816Z │ "status": "Processing worker updates..." 2022-11-18T20:46:49.8176226Z │ } 2022-11-18T20:46:49.8176566Z └ 2022-11-18T20:46:49.8177177Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:49.8177788Z 1ms 942µs 250µs 238µs 2ms 1ms 44µs 6ms 2022-11-18T20:46:49.8178099Z 2022-11-18T20:46:49.8350446Z √ Successful request 2022-11-18T20:46:49.8374185Z 2022-11-18T20:46:49.8374740Z Iteration 25/82 2022-11-18T20:46:49.8395804Z 2022-11-18T20:46:49.8396498Z → management request 2022-11-18T20:46:49.8967170Z PUT http://localhost:8081/models/squeezenet1_1/1.0/set-default 2022-11-18T20:46:49.8968249Z 200 OK ★ 5ms time ★ 281B↑ 359B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:49.8968937Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 93B 2022-11-18T20:46:49.8969392Z │ { 2022-11-18T20:46:49.8969857Z │ "status": "Default vesion succsesfully updated for m 2022-11-18T20:46:49.8970379Z │ odel \"squeezenet1_1\" to \"1.0\"" 2022-11-18T20:46:49.8970806Z │ } 2022-11-18T20:46:49.8971147Z └ 2022-11-18T20:46:49.8971674Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:49.8972316Z 1ms 259µs (cache) (cache) 2ms 1ms 47µs 5ms 2022-11-18T20:46:49.8972625Z 2022-11-18T20:46:49.9111693Z √ Successful request 2022-11-18T20:46:49.9176352Z 2022-11-18T20:46:49.9176775Z Iteration 26/82 2022-11-18T20:46:49.9337080Z 2022-11-18T20:46:49.9338030Z → management request 2022-11-18T20:46:49.9851900Z PUT http://localhost:8081/models/squeezenet1_1/0.0/set-default [errored] 2022-11-18T20:46:49.9852636Z read ECONNRESET 2022-11-18T20:46:50.0016100Z 10. Successful request 2022-11-18T20:46:50.0115919Z 2022-11-18T20:46:50.0116367Z Iteration 27/82 2022-11-18T20:46:50.0271657Z 2022-11-18T20:46:50.0272456Z → management request 2022-11-18T20:46:50.0829671Z PUT http://localhost:8081/models/squeezenet0_1/1.0/set-default 2022-11-18T20:46:50.0830876Z 404 Not Found ★ 6ms time ★ 281B↑ 370B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:50.0831602Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 101B 2022-11-18T20:46:50.0832087Z │ { 2022-11-18T20:46:50.0832440Z │ "code": 404, 2022-11-18T20:46:50.0832932Z │ "type": "ModelNotFoundException", 2022-11-18T20:46:50.0833455Z │ "message": "Model not found: squeezenet0_1" 2022-11-18T20:46:50.0833882Z │ } 2022-11-18T20:46:50.0834181Z └ 2022-11-18T20:46:50.0834750Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:50.0835399Z 1ms 926µs 373µs 245µs 2ms 1ms 41µs 6ms 2022-11-18T20:46:50.0835702Z 2022-11-18T20:46:50.0962341Z √ Successful request 2022-11-18T20:46:50.1050870Z 2022-11-18T20:46:50.1051397Z Iteration 28/82 2022-11-18T20:46:50.1209625Z 2022-11-18T20:46:50.1210329Z → management request 2022-11-18T20:46:50.2802561Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:46:50.2803703Z 200 OK ★ 111ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:50.2804389Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:46:50.2804853Z │ { 2022-11-18T20:46:50.2805344Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:46:50.2805732Z │ } 2022-11-18T20:46:50.2815742Z └ 2022-11-18T20:46:50.2816327Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:50.2817007Z 1ms 1ms 256µs 239µs 107ms 1ms 56µs 111ms 2022-11-18T20:46:50.2817337Z 2022-11-18T20:46:50.2961554Z √ Successful request 2022-11-18T20:46:50.3081477Z 2022-11-18T20:46:50.3082018Z Iteration 29/82 2022-11-18T20:46:50.3241936Z 2022-11-18T20:46:50.3242728Z → management request 2022-11-18T20:46:50.4713240Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&handler=serve/ts/torch_handler/image_classifier.py:handle 2022-11-18T20:46:50.4714470Z 200 OK ★ 97ms time ★ 357B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:50.4715091Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:46:50.4715561Z │ { 2022-11-18T20:46:50.4716063Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:46:50.4716568Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:46:50.4717086Z │ o add workers for the model." 2022-11-18T20:46:50.4717485Z │ } 2022-11-18T20:46:50.4745280Z └ 2022-11-18T20:46:50.4746082Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:50.4746935Z 1ms 252µs (cache) (cache) 94ms 1ms 42µs 97ms 2022-11-18T20:46:50.4747305Z 2022-11-18T20:46:50.4852583Z √ Successful request 2022-11-18T20:46:50.4958133Z 2022-11-18T20:46:50.4958737Z Iteration 30/82 2022-11-18T20:46:50.5115973Z 2022-11-18T20:46:50.5117534Z → management request 2022-11-18T20:46:50.5680332Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:46:50.5681552Z 200 OK ★ 8ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:50.5682298Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:46:50.5682864Z │ { 2022-11-18T20:46:50.5683362Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:46:50.5683810Z │ } 2022-11-18T20:46:50.5693751Z └ 2022-11-18T20:46:50.5694288Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:50.5694964Z 1ms 261µs (cache) (cache) 5ms 1ms 42µs 8ms 2022-11-18T20:46:50.5695595Z 2022-11-18T20:46:50.5816558Z √ Successful request 2022-11-18T20:46:50.5897204Z 2022-11-18T20:46:50.5897814Z Iteration 31/82 2022-11-18T20:46:50.6062567Z 2022-11-18T20:46:50.6063533Z → management request 2022-11-18T20:46:52.7629739Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&batch_size=3&initial_workers=3&response_timeout=0 [errored] 2022-11-18T20:46:52.7634712Z read ECONNRESET 2022-11-18T20:46:52.7772825Z 12. Successful request 2022-11-18T20:46:52.7864100Z 2022-11-18T20:46:52.7864417Z Iteration 32/82 2022-11-18T20:46:52.8022954Z 2022-11-18T20:46:52.8023738Z → management request 2022-11-18T20:46:52.9485818Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&response_timeout=0 2022-11-18T20:46:52.9487301Z 200 OK ★ 98ms time ★ 318B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:52.9487933Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:46:52.9488399Z │ { 2022-11-18T20:46:52.9488905Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:46:52.9489485Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:46:52.9490007Z │ o add workers for the model." 2022-11-18T20:46:52.9490387Z │ } 2022-11-18T20:46:52.9498423Z └ 2022-11-18T20:46:52.9499075Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:52.9499754Z 1ms 1ms 213µs 378µs 94ms 1ms 40µs 98ms 2022-11-18T20:46:52.9500054Z 2022-11-18T20:46:52.9621495Z √ Successful request 2022-11-18T20:46:52.9740372Z 2022-11-18T20:46:52.9740794Z Iteration 33/82 2022-11-18T20:46:52.9899674Z 2022-11-18T20:46:52.9900472Z → management request 2022-11-18T20:46:53.0475712Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:46:53.0476713Z 200 OK ★ 7ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:53.0477504Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:46:53.0478358Z │ { 2022-11-18T20:46:53.0478832Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:46:53.0479233Z │ } 2022-11-18T20:46:53.0479551Z └ 2022-11-18T20:46:53.0480068Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:53.0480676Z 1ms 266µs (cache) (cache) 5ms 1ms 37µs 8ms 2022-11-18T20:46:53.0480887Z 2022-11-18T20:46:53.0558348Z √ Successful request 2022-11-18T20:46:53.0679552Z 2022-11-18T20:46:53.0680383Z Iteration 34/82 2022-11-18T20:46:53.0837076Z 2022-11-18T20:46:53.0837853Z → management request 2022-11-18T20:46:57.4733179Z POST http://localhost:8081/models?url=resnet-152-batch.mar&model_name=resnet152&batch_size=2 2022-11-18T20:46:57.4734308Z 200 OK ★ 4.3s time ★ 311B↑ 405B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:57.4735075Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 138B 2022-11-18T20:46:57.4735661Z │ { 2022-11-18T20:46:57.4736228Z │ "status": "Model \"resnet152\" Version: 1.0 register 2022-11-18T20:46:57.4736862Z │ ed with 0 initial workers. Use scale workers API to ad 2022-11-18T20:46:57.4737579Z │ d workers for the model." 2022-11-18T20:46:57.4738111Z │ } 2022-11-18T20:46:57.4746368Z └ 2022-11-18T20:46:57.4746989Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:57.4747734Z 1ms 234µs (cache) (cache) 4.3s 1ms 49µs 4.3s 2022-11-18T20:46:57.4748128Z 2022-11-18T20:46:57.4928473Z √ Successful request 2022-11-18T20:46:57.4954625Z 2022-11-18T20:46:57.4954908Z Iteration 35/82 2022-11-18T20:46:57.5045533Z 2022-11-18T20:46:57.5046262Z → management request 2022-11-18T20:46:57.5876636Z DELETE http://localhost:8081/models/resnet152 2022-11-18T20:46:57.5877804Z 200 OK ★ 33ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:46:57.5878730Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:46:57.5879210Z │ { 2022-11-18T20:46:57.5879693Z │ "status": "Model \"resnet152\" unregistered" 2022-11-18T20:46:57.5880076Z │ } 2022-11-18T20:46:57.5880431Z └ 2022-11-18T20:46:57.5881031Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:46:57.5881633Z 1ms 240µs (cache) (cache) 29ms 3ms 40µs 33ms 2022-11-18T20:46:57.5881943Z 2022-11-18T20:46:57.6033605Z √ Successful request 2022-11-18T20:46:57.6137982Z 2022-11-18T20:46:57.6138617Z Iteration 36/82 2022-11-18T20:46:57.6296338Z 2022-11-18T20:46:57.6297190Z → management request 2022-11-18T20:47:18.8112176Z POST http://localhost:8081/models?url=resnet-152-batch.mar&model_name=resnet152&batch_size=dd&initial_workers=1 2022-11-18T20:47:18.8113615Z 200 OK ★ 21.1s time ★ 330B↑ 351B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:18.8114306Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 85B 2022-11-18T20:47:18.8114992Z │ { 2022-11-18T20:47:18.8115606Z │ "status": "Model \"resnet152\" Version: 1.0 register 2022-11-18T20:47:18.8116217Z │ ed with 1 initial workers" 2022-11-18T20:47:18.8116742Z │ } 2022-11-18T20:47:18.8117933Z └ 2022-11-18T20:47:18.8118546Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:18.8119311Z 1ms 253µs (cache) (cache) 21.1s 3ms 90µs 21.1s 2022-11-18T20:47:18.8119687Z 2022-11-18T20:47:18.8335209Z √ Successful request 2022-11-18T20:47:18.8381962Z 2022-11-18T20:47:18.8382431Z Iteration 37/82 2022-11-18T20:47:18.8415632Z 2022-11-18T20:47:18.8416321Z → management request 2022-11-18T20:47:18.9991679Z DELETE http://localhost:8081/models/resnet152 2022-11-18T20:47:18.9992813Z 200 OK ★ 109ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:18.9993501Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:47:18.9994006Z │ { 2022-11-18T20:47:18.9994448Z │ "status": "Model \"resnet152\" unregistered" 2022-11-18T20:47:18.9994892Z │ } 2022-11-18T20:47:19.0019292Z └ 2022-11-18T20:47:19.0020113Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:19.0020843Z 1ms 380µs (cache) (cache) 106ms 1ms 43µs 110ms 2022-11-18T20:47:19.0021164Z 2022-11-18T20:47:19.0095963Z √ Successful request 2022-11-18T20:47:19.0187944Z 2022-11-18T20:47:19.0188323Z Iteration 38/82 2022-11-18T20:47:19.0346353Z 2022-11-18T20:47:19.0346937Z → management request 2022-11-18T20:47:26.7468943Z POST http://localhost:8081/models?url=resnet-152-batch.mar&model_name=resnet152&batch_size=2&initial_workers=1&max_batch_delay=junk 2022-11-18T20:47:26.7470081Z 200 OK ★ 7.6s time ★ 350B↑ 351B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:26.7470829Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 85B 2022-11-18T20:47:26.7471416Z │ { 2022-11-18T20:47:26.7471969Z │ "status": "Model \"resnet152\" Version: 1.0 register 2022-11-18T20:47:26.7472560Z │ ed with 1 initial workers" 2022-11-18T20:47:26.7473060Z │ } 2022-11-18T20:47:26.7484315Z └ 2022-11-18T20:47:26.7485022Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:26.7485796Z 1ms 335µs (cache) (cache) 7.6s 2ms 41µs 7.6s 2022-11-18T20:47:26.7486207Z 2022-11-18T20:47:26.7634875Z √ Successful request 2022-11-18T20:47:26.7744708Z 2022-11-18T20:47:26.7745073Z Iteration 39/82 2022-11-18T20:47:26.7902828Z 2022-11-18T20:47:26.7904012Z → management request 2022-11-18T20:47:26.9437971Z DELETE http://localhost:8081/models/resnet152 2022-11-18T20:47:26.9439036Z 200 OK ★ 105ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:26.9439751Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:47:26.9440507Z │ { 2022-11-18T20:47:26.9440987Z │ "status": "Model \"resnet152\" unregistered" 2022-11-18T20:47:26.9441382Z │ } 2022-11-18T20:47:26.9464036Z └ 2022-11-18T20:47:26.9464762Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:26.9465504Z 1ms 302µs (cache) (cache) 102ms 1ms 44µs 105ms 2022-11-18T20:47:26.9465855Z 2022-11-18T20:47:26.9555410Z √ Successful request 2022-11-18T20:47:26.9668617Z 2022-11-18T20:47:26.9668829Z Iteration 40/82 2022-11-18T20:47:27.6546753Z 2022-11-18T20:47:27.6547267Z → management request 2022-11-18T20:47:27.6547837Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&initial_workers=-1 2022-11-18T20:47:27.6548412Z 200 OK ★ 98ms time ★ 318B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:27.6548873Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:47:27.6549185Z │ { 2022-11-18T20:47:27.6549580Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:47:27.6550030Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:47:27.6550711Z │ o add workers for the model." 2022-11-18T20:47:27.6550989Z │ } 2022-11-18T20:47:27.6551251Z └ 2022-11-18T20:47:27.6551725Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:27.6552224Z 1ms 299µs (cache) (cache) 95ms 2ms 45µs 99ms 2022-11-18T20:47:27.6552436Z 2022-11-18T20:47:27.6552604Z √ Successful request 2022-11-18T20:47:27.6552777Z 2022-11-18T20:47:27.6552887Z Iteration 41/82 2022-11-18T20:47:27.6553043Z 2022-11-18T20:47:27.6553156Z → management request 2022-11-18T20:47:27.6553487Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:47:27.6553949Z 200 OK ★ 7ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:27.6554394Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:47:27.6554670Z │ { 2022-11-18T20:47:27.6555040Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:47:27.6555361Z │ } 2022-11-18T20:47:27.6555556Z └ 2022-11-18T20:47:27.6556028Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:27.6556562Z 2ms 348µs (cache) (cache) 5ms 1ms 42µs 9ms 2022-11-18T20:47:27.6556770Z 2022-11-18T20:47:27.6556891Z √ Successful request 2022-11-18T20:47:27.6557059Z 2022-11-18T20:47:27.6557172Z Iteration 42/82 2022-11-18T20:47:27.6557415Z 2022-11-18T20:47:27.6561301Z → management request 2022-11-18T20:47:28.1473581Z POST http://localhost:8081/models?url=resnet-18.mar&model_name=resnet-18&synchronous=true 2022-11-18T20:47:28.1474598Z 200 OK ★ 834ms time ★ 308B↑ 405B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:28.1475205Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 138B 2022-11-18T20:47:28.1475712Z │ { 2022-11-18T20:47:28.1476231Z │ "status": "Model \"resnet-18\" Version: 1.0 register 2022-11-18T20:47:28.1476757Z │ ed with 0 initial workers. Use scale workers API to ad 2022-11-18T20:47:28.1477477Z │ d workers for the model." 2022-11-18T20:47:28.1477913Z │ } 2022-11-18T20:47:28.1492521Z └ 2022-11-18T20:47:28.1493166Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:28.1493783Z 2ms 316µs (cache) (cache) 831ms 1ms 39µs 835ms 2022-11-18T20:47:28.1494056Z 2022-11-18T20:47:28.1643438Z √ Successful request 2022-11-18T20:47:28.1705491Z 2022-11-18T20:47:28.1705690Z Iteration 43/82 2022-11-18T20:47:28.1862241Z 2022-11-18T20:47:28.1862819Z → management request 2022-11-18T20:47:28.2466369Z DELETE http://localhost:8081/models/resnet-18 2022-11-18T20:47:28.2469135Z 200 OK ★ 12ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:28.2470136Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:47:28.2470502Z │ { 2022-11-18T20:47:28.2470939Z │ "status": "Model \"resnet-18\" unregistered" 2022-11-18T20:47:28.2471336Z │ } 2022-11-18T20:47:28.2482430Z └ 2022-11-18T20:47:28.2484255Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:28.2484962Z 1ms 312µs (cache) (cache) 9ms 1ms 38µs 12ms 2022-11-18T20:47:28.2485237Z 2022-11-18T20:47:28.2696980Z √ Successful request 2022-11-18T20:47:28.2799487Z 2022-11-18T20:47:28.2799961Z Iteration 44/82 2022-11-18T20:47:28.2957354Z 2022-11-18T20:47:28.2958212Z → management request 2022-11-18T20:47:29.1712356Z POST http://localhost:8081/models?url=resnet-18.mar&model_name=resnet-18&synchronous=-1 2022-11-18T20:47:29.1715313Z 200 OK ★ 826ms time ★ 306B↑ 405B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:29.1716332Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 138B 2022-11-18T20:47:29.1716638Z │ { 2022-11-18T20:47:29.1719470Z │ "status": "Model \"resnet-18\" Version: 1.0 register 2022-11-18T20:47:29.1719987Z │ ed with 0 initial workers. Use scale workers API to ad 2022-11-18T20:47:29.1720335Z │ d workers for the model." 2022-11-18T20:47:29.1720638Z │ } 2022-11-18T20:47:29.1720886Z └ 2022-11-18T20:47:29.1721309Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:29.1721860Z 1ms 281µs (cache) (cache) 823ms 1ms 43µs 826ms 2022-11-18T20:47:29.1722071Z 2022-11-18T20:47:29.1935655Z √ Successful request 2022-11-18T20:47:29.2028554Z 2022-11-18T20:47:29.2029006Z Iteration 45/82 2022-11-18T20:47:29.2185168Z 2022-11-18T20:47:29.2185817Z → management request 2022-11-18T20:47:29.2784903Z DELETE http://localhost:8081/models/resnet-18 2022-11-18T20:47:29.2786030Z 200 OK ★ 11ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:29.2786742Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:47:29.2787229Z │ { 2022-11-18T20:47:29.2787719Z │ "status": "Model \"resnet-18\" unregistered" 2022-11-18T20:47:29.2788133Z │ } 2022-11-18T20:47:29.2797552Z └ 2022-11-18T20:47:29.2798167Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:29.2798872Z 1ms 311µs (cache) (cache) 9ms 1ms 37µs 12ms 2022-11-18T20:47:29.2799212Z 2022-11-18T20:47:29.2842331Z √ Successful request 2022-11-18T20:47:29.2964035Z 2022-11-18T20:47:29.2964566Z Iteration 46/82 2022-11-18T20:47:29.3123330Z 2022-11-18T20:47:29.3124299Z → management request 2022-11-18T20:47:30.1840547Z POST http://localhost:8081/models?url=resnet-18.mar&model_name=resnet-18&synchronous=false 2022-11-18T20:47:30.1841593Z 200 OK ★ 823ms time ★ 309B↑ 405B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:30.1842330Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 138B 2022-11-18T20:47:30.1842821Z │ { 2022-11-18T20:47:30.1843278Z │ "status": "Model \"resnet-18\" Version: 1.0 register 2022-11-18T20:47:30.1844046Z │ ed with 0 initial workers. Use scale workers API to ad 2022-11-18T20:47:30.1844607Z │ d workers for the model." 2022-11-18T20:47:30.1845022Z │ } 2022-11-18T20:47:30.1860373Z └ 2022-11-18T20:47:30.1861030Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:30.1861743Z 1ms 304µs (cache) (cache) 820ms 1ms 42µs 824ms 2022-11-18T20:47:30.1862092Z 2022-11-18T20:47:30.1910930Z √ Successful request 2022-11-18T20:47:30.2032551Z 2022-11-18T20:47:30.2032851Z Iteration 47/82 2022-11-18T20:47:30.2190913Z 2022-11-18T20:47:30.2191787Z → management request 2022-11-18T20:47:30.2725439Z GET http://localhost:8081/models?limit=1 2022-11-18T20:47:30.2726932Z 200 OK ★ 4ms time ★ 240B↑ 391B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:30.2727691Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 124B 2022-11-18T20:47:30.2728180Z │ { 2022-11-18T20:47:30.2728675Z │ "nextPageToken": "1", 2022-11-18T20:47:30.2729188Z │ "models": [ 2022-11-18T20:47:30.2729590Z │ { 2022-11-18T20:47:30.2730102Z │ "modelName": "resnet-18", 2022-11-18T20:47:30.2730663Z │ "modelUrl": "resnet-18.mar" 2022-11-18T20:47:30.2731134Z │ } 2022-11-18T20:47:30.2731524Z │ ] 2022-11-18T20:47:30.2731947Z │ } 2022-11-18T20:47:30.2740963Z └ 2022-11-18T20:47:30.2741628Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:30.2742409Z 1ms 309µs (cache) (cache) 1ms 1ms 44µs 5ms 2022-11-18T20:47:30.2742806Z 2022-11-18T20:47:30.2880921Z √ Successful request 2022-11-18T20:47:30.2970777Z 2022-11-18T20:47:30.2971204Z Iteration 48/82 2022-11-18T20:47:30.3128967Z 2022-11-18T20:47:30.3129850Z → management request 2022-11-18T20:47:30.3500466Z GET http://localhost:8081/models?limit=-1 2022-11-18T20:47:30.3501558Z 200 OK ★ 4ms time ★ 241B↑ 367B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:30.3502277Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 100B 2022-11-18T20:47:30.3502774Z │ { 2022-11-18T20:47:30.3503178Z │ "models": [ 2022-11-18T20:47:30.3503535Z │ { 2022-11-18T20:47:30.3503999Z │ "modelName": "resnet-18", 2022-11-18T20:47:30.3504495Z │ "modelUrl": "resnet-18.mar" 2022-11-18T20:47:30.3504880Z │ } 2022-11-18T20:47:30.3505272Z │ ] 2022-11-18T20:47:30.3505654Z │ } 2022-11-18T20:47:30.3520976Z └ 2022-11-18T20:47:30.3521748Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:30.3522444Z 1ms 278µs (cache) (cache) 1ms 1ms 38µs 5ms 2022-11-18T20:47:30.3522767Z 2022-11-18T20:47:30.3718654Z √ Successful request 2022-11-18T20:47:30.3751351Z 2022-11-18T20:47:30.3751676Z Iteration 49/82 2022-11-18T20:47:30.3910508Z 2022-11-18T20:47:30.3911446Z → management request 2022-11-18T20:47:30.4445047Z GET http://localhost:8081/models?limit=1&next_page_token=1 2022-11-18T20:47:30.4446383Z 200 OK ★ 4ms time ★ 258B↑ 285B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:30.4447090Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 19B 2022-11-18T20:47:30.4447564Z │ { 2022-11-18T20:47:30.4448035Z │ "models": [] 2022-11-18T20:47:30.4448383Z │ } 2022-11-18T20:47:30.4486023Z └ 2022-11-18T20:47:30.4486852Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:30.4487684Z 2ms 283µs (cache) (cache) 1ms 1ms 40µs 5ms 2022-11-18T20:47:30.4488096Z 2022-11-18T20:47:30.4603370Z √ Successful request 2022-11-18T20:47:31.1899252Z 2022-11-18T20:47:31.1899685Z Iteration 50/82 2022-11-18T20:47:31.1899903Z 2022-11-18T20:47:31.1900354Z → management request 2022-11-18T20:47:31.1900862Z GET http://localhost:8081/models?limit=1&next_page_token=-1 2022-11-18T20:47:31.1901620Z 200 OK ★ 4ms time ★ 259B↑ 391B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:31.1902096Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 124B 2022-11-18T20:47:31.1902442Z │ { 2022-11-18T20:47:31.1902718Z │ "nextPageToken": "1", 2022-11-18T20:47:31.1903035Z │ "models": [ 2022-11-18T20:47:31.1903311Z │ { 2022-11-18T20:47:31.1903601Z │ "modelName": "resnet-18", 2022-11-18T20:47:31.1903962Z │ "modelUrl": "resnet-18.mar" 2022-11-18T20:47:31.1904272Z │ } 2022-11-18T20:47:31.1904534Z │ ] 2022-11-18T20:47:31.1904795Z │ } 2022-11-18T20:47:31.1905049Z └ 2022-11-18T20:47:31.1905480Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:31.1906015Z 2ms 288µs (cache) (cache) 1ms 1ms 38µs 5ms 2022-11-18T20:47:31.1907072Z 2022-11-18T20:47:31.1907261Z √ Successful request 2022-11-18T20:47:31.1907429Z 2022-11-18T20:47:31.1907540Z Iteration 51/82 2022-11-18T20:47:31.1907654Z 2022-11-18T20:47:31.1907829Z → management request 2022-11-18T20:47:31.1908269Z PUT http://localhost:8081/models/resnet-18?number_gpu=10 2022-11-18T20:47:31.1908768Z 202 Accepted ★ 4ms time ★ 275B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:31.1909176Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:31.1909502Z │ { 2022-11-18T20:47:31.1909863Z │ "status": "Processing worker updates..." 2022-11-18T20:47:31.1910142Z │ } 2022-11-18T20:47:31.1910394Z └ 2022-11-18T20:47:31.1910854Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:31.1911389Z 1ms 306µs (cache) (cache) 2ms 1ms 45µs 5ms 2022-11-18T20:47:31.1911564Z 2022-11-18T20:47:31.1911743Z √ Successful request 2022-11-18T20:47:31.1911910Z 2022-11-18T20:47:31.1912022Z Iteration 52/82 2022-11-18T20:47:31.1912180Z 2022-11-18T20:47:31.1912337Z → management request 2022-11-18T20:47:31.1912732Z PUT http://localhost:8081/models/resnet-18?number_gpu=-1 2022-11-18T20:47:31.1913229Z 202 Accepted ★ 4ms time ★ 275B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:31.1913666Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:31.1913998Z │ { 2022-11-18T20:47:31.1914302Z │ "status": "Processing worker updates..." 2022-11-18T20:47:31.1914630Z │ } 2022-11-18T20:47:31.1914871Z └ 2022-11-18T20:47:31.1915282Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:31.1915807Z 1ms 294µs (cache) (cache) 1ms 1ms 41µs 4ms 2022-11-18T20:47:31.1916080Z 2022-11-18T20:47:31.1916257Z √ Successful request 2022-11-18T20:47:31.1916437Z 2022-11-18T20:47:31.1916554Z Iteration 53/82 2022-11-18T20:47:31.1916667Z 2022-11-18T20:47:31.1916830Z → management request 2022-11-18T20:47:31.1917437Z PUT http://localhost:8081/models/resnet-18?min_worker=1&max_worker=1&synchronous=true 2022-11-18T20:47:31.1917968Z 200 OK ★ 4ms time ★ 304B↑ 325B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:31.1918356Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 59B 2022-11-18T20:47:31.1918684Z │ { 2022-11-18T20:47:31.1919071Z │ "status": "Workers scaled to 1 for model: resnet-18" 2022-11-18T20:47:31.1919360Z │ } 2022-11-18T20:47:31.1919612Z └ 2022-11-18T20:47:31.1920071Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:31.1920597Z 1ms 292µs (cache) (cache) 1ms 1ms 37µs 4ms 2022-11-18T20:47:31.1920758Z 2022-11-18T20:47:31.1920930Z √ Successful request 2022-11-18T20:47:31.1921099Z 2022-11-18T20:47:31.1921284Z Iteration 54/82 2022-11-18T20:47:31.1921444Z 2022-11-18T20:47:31.1921606Z → management request 2022-11-18T20:47:31.1922041Z PUT http://localhost:8081/models/resnet-18?min_worker=1&max_worker=1&synchronous=false 2022-11-18T20:47:31.1922572Z 202 Accepted ★ 4ms time ★ 305B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:31.1923011Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:31.1923345Z │ { 2022-11-18T20:47:31.1923654Z │ "status": "Processing worker updates..." 2022-11-18T20:47:31.1923984Z │ } 2022-11-18T20:47:31.1924223Z └ 2022-11-18T20:47:31.1924627Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:31.1925154Z 1ms 291µs (cache) (cache) 1ms 1ms 39µs 4ms 2022-11-18T20:47:31.1925362Z 2022-11-18T20:47:31.1925529Z √ Successful request 2022-11-18T20:47:31.1925701Z 2022-11-18T20:47:31.1925812Z Iteration 55/82 2022-11-18T20:47:31.1925936Z 2022-11-18T20:47:31.1926092Z → management request 2022-11-18T20:47:31.1926506Z PUT http://localhost:8081/models/resnet-18?timeout=-1 2022-11-18T20:47:31.1926997Z 202 Accepted ★ 4ms time ★ 272B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:31.1927385Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:31.1927707Z │ { 2022-11-18T20:47:31.1928055Z │ "status": "Processing worker updates..." 2022-11-18T20:47:31.1928329Z │ } 2022-11-18T20:47:31.1928575Z └ 2022-11-18T20:47:31.1929032Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:31.1929510Z 1ms 293µs (cache) (cache) 1ms 1ms 40µs 4ms 2022-11-18T20:47:31.1929722Z 2022-11-18T20:47:31.1929886Z √ Successful request 2022-11-18T20:47:31.1930059Z 2022-11-18T20:47:31.1930175Z Iteration 56/82 2022-11-18T20:47:31.1930337Z 2022-11-18T20:47:31.1930493Z → management request 2022-11-18T20:47:31.1930869Z PUT http://localhost:8081/models/resnet-18?timeout=0 2022-11-18T20:47:31.1931352Z 202 Accepted ★ 5ms time ★ 271B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:31.1931799Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:31.1932076Z │ { 2022-11-18T20:47:31.1932441Z │ "status": "Processing worker updates..." 2022-11-18T20:47:31.1932766Z │ } 2022-11-18T20:47:31.1933010Z └ 2022-11-18T20:47:31.1933421Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:31.1933948Z 1ms 288µs (cache) (cache) 2ms 1ms 40µs 5ms 2022-11-18T20:47:31.1934157Z 2022-11-18T20:47:31.1934324Z √ Successful request 2022-11-18T20:47:31.1934497Z 2022-11-18T20:47:31.1934565Z Iteration 57/82 2022-11-18T20:47:31.1934719Z 2022-11-18T20:47:31.1934880Z → management request 2022-11-18T20:47:31.1935312Z POST http://localhost:8081/models?url=&model_name=resnet-18 2022-11-18T20:47:31.1935811Z 404 Not Found ★ 4ms time ★ 278B↑ 348B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:31.1936202Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 80B 2022-11-18T20:47:31.1936525Z │ { 2022-11-18T20:47:31.1936905Z │ "code": 404, 2022-11-18T20:47:31.1937241Z │ "type": "ModelNotFoundException", 2022-11-18T20:47:31.1937612Z │ "message": "empty url" 2022-11-18T20:47:31.1937905Z │ } 2022-11-18T20:47:31.1938104Z └ 2022-11-18T20:47:31.1938557Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:31.1939080Z 1ms 285µs (cache) (cache) 2ms 1ms 43µs 5ms 2022-11-18T20:47:31.1939288Z 2022-11-18T20:47:31.2097345Z √ Successful request 2022-11-18T20:47:31.2192932Z 2022-11-18T20:47:31.2193392Z Iteration 58/82 2022-11-18T20:47:31.2352328Z 2022-11-18T20:47:31.2353208Z → management request 2022-11-18T20:47:31.8197017Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/invalid-resnet-18.mar&model_name=invalid-resnet18 [errored] 2022-11-18T20:47:31.8202027Z read ECONNRESET 2022-11-18T20:47:31.8207952Z 14. Successful request 2022-11-18T20:47:31.8290081Z 2022-11-18T20:47:31.8290643Z Iteration 59/82 2022-11-18T20:47:31.8446340Z 2022-11-18T20:47:31.8447031Z → management request 2022-11-18T20:47:31.9006334Z GET http://localhost:8081/models/invalid_squeezenet1_1 2022-11-18T20:47:31.9007439Z 404 Not Found ★ 6ms time ★ 254B↑ 378B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:31.9007988Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 109B 2022-11-18T20:47:31.9008341Z │ { 2022-11-18T20:47:31.9008693Z │ "code": 404, 2022-11-18T20:47:31.9009125Z │ "type": "ModelNotFoundException", 2022-11-18T20:47:31.9009581Z │ "message": "Model not found: invalid_squeezenet1_1" 2022-11-18T20:47:31.9009973Z │ } 2022-11-18T20:47:31.9010290Z └ 2022-11-18T20:47:31.9010828Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:31.9011403Z 1ms 1ms 438µs 288µs 1ms 1ms 43µs 7ms 2022-11-18T20:47:31.9011661Z 2022-11-18T20:47:31.9115593Z √ Successful request 2022-11-18T20:47:31.9224147Z 2022-11-18T20:47:31.9224826Z Iteration 60/82 2022-11-18T20:47:31.9382865Z 2022-11-18T20:47:31.9383615Z → management request 2022-11-18T20:47:31.9920119Z GET http://localhost:8081/models/squeezenet1_1/0.0 [errored] 2022-11-18T20:47:31.9920815Z read ECONNRESET 2022-11-18T20:47:32.0197049Z 16. Successful request 2022-11-18T20:47:32.0318411Z 2022-11-18T20:47:32.0319056Z Iteration 61/82 2022-11-18T20:47:32.0477202Z 2022-11-18T20:47:32.0478029Z → management request 2022-11-18T20:47:32.1381317Z GET http://localhost:8081/models?next_page_token=12 2022-11-18T20:47:32.1383594Z 200 OK ★ 6ms time ★ 251B↑ 285B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:32.1404006Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 19B 2022-11-18T20:47:32.1430822Z │ { 2022-11-18T20:47:32.1454060Z │ "models": [] 2022-11-18T20:47:32.1455710Z │ } 2022-11-18T20:47:32.1507394Z └ 2022-11-18T20:47:32.1641871Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:32.1644476Z 1ms 959µs 250µs 253µs 2ms 1ms 45µs 6ms 2022-11-18T20:47:32.1725539Z 2022-11-18T20:47:32.2243328Z √ Successful request 2022-11-18T20:47:32.2715030Z 2022-11-18T20:47:32.2716902Z Iteration 62/82 2022-11-18T20:47:32.2730758Z 2022-11-18T20:47:32.2734396Z → management request 2022-11-18T20:47:32.2737375Z PUT http://localhost:8081/models/resnet-18?min_worker=1&synchronous=Nan 2022-11-18T20:47:32.2760426Z 202 Accepted ★ 5ms time ★ 290B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:32.2765549Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:32.2767753Z │ { 2022-11-18T20:47:32.2770000Z │ "status": "Processing worker updates..." 2022-11-18T20:47:32.2785686Z │ } 2022-11-18T20:47:32.2786115Z └ 2022-11-18T20:47:32.2787561Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:32.2788289Z 1ms 304µs (cache) (cache) 2ms 2ms 42µs 6ms 2022-11-18T20:47:32.2788622Z 2022-11-18T20:47:32.2788909Z √ Successful request 2022-11-18T20:47:32.2889564Z 2022-11-18T20:47:32.2889932Z Iteration 63/82 2022-11-18T20:47:32.3047912Z 2022-11-18T20:47:32.3048752Z → management request 2022-11-18T20:47:32.3588258Z PUT http://localhost:8081/models/resnet-18?min_worker=nan&synchronous=nan 2022-11-18T20:47:32.3589640Z 202 Accepted ★ 4ms time ★ 292B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:32.3590752Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:32.3591350Z │ { 2022-11-18T20:47:32.3591887Z │ "status": "Processing worker updates..." 2022-11-18T20:47:32.3592665Z │ } 2022-11-18T20:47:32.3593122Z └ 2022-11-18T20:47:32.3593806Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:32.3594595Z 1ms 255µs (cache) (cache) 1ms 1ms 37µs 5ms 2022-11-18T20:47:32.3594987Z 2022-11-18T20:47:32.3740247Z √ Successful request 2022-11-18T20:47:32.3755954Z 2022-11-18T20:47:32.3756419Z Iteration 64/82 2022-11-18T20:47:32.3826585Z 2022-11-18T20:47:32.3827204Z → management request 2022-11-18T20:47:32.4363896Z PUT http://localhost:8081/models/resnet-18 2022-11-18T20:47:32.4364875Z 202 Accepted ★ 4ms time ★ 261B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:32.4365452Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:32.4365938Z │ { 2022-11-18T20:47:32.4366424Z │ "status": "Processing worker updates..." 2022-11-18T20:47:32.4366880Z │ } 2022-11-18T20:47:32.4367192Z └ 2022-11-18T20:47:32.4367816Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:32.4368480Z 1ms 244µs (cache) (cache) 1ms 1ms 45µs 4ms 2022-11-18T20:47:32.4368815Z 2022-11-18T20:47:32.4499760Z √ Successful request 2022-11-18T20:47:32.4606672Z 2022-11-18T20:47:32.4607249Z Iteration 65/82 2022-11-18T20:47:32.4766025Z 2022-11-18T20:47:32.4766820Z → management request 2022-11-18T20:47:32.5275326Z PUT http://localhost:8081/models/resnet181?min_worker=1 [errored] 2022-11-18T20:47:32.5275994Z read ECONNRESET 2022-11-18T20:47:32.5440995Z 18. Successful request 2022-11-18T20:47:32.5543206Z 2022-11-18T20:47:32.5543589Z Iteration 66/82 2022-11-18T20:47:32.5709760Z 2022-11-18T20:47:32.5710524Z → management request 2022-11-18T20:47:32.6240719Z PUT http://localhost:8081/models/resnet-18?min_worker=2&max_worker=1 [errored] 2022-11-18T20:47:32.6241461Z read ECONNRESET 2022-11-18T20:47:32.6380139Z 20. Successful request 2022-11-18T20:47:32.6481930Z 2022-11-18T20:47:32.6482465Z Iteration 67/82 2022-11-18T20:47:32.6641201Z 2022-11-18T20:47:32.6642000Z → management request 2022-11-18T20:47:32.7176044Z PUT http://localhost:8081/models/resnet-18?min_worker=1 2022-11-18T20:47:32.7218050Z 202 Accepted ★ 5ms time ★ 274B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:32.7218768Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:32.7219262Z │ { 2022-11-18T20:47:32.7219762Z │ "status": "Processing worker updates..." 2022-11-18T20:47:32.7220160Z │ } 2022-11-18T20:47:32.7220525Z └ 2022-11-18T20:47:32.7221124Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:32.7221833Z 1ms 715µs 223µs 254µs 2ms 1ms 34µs 5ms 2022-11-18T20:47:32.7222148Z 2022-11-18T20:47:32.7433795Z √ Successful request 2022-11-18T20:47:32.7576305Z 2022-11-18T20:47:32.7576938Z Iteration 68/82 2022-11-18T20:47:32.7736426Z 2022-11-18T20:47:32.7737239Z → management request 2022-11-18T20:47:32.9006856Z PUT http://localhost:8081/models/resnet-18?min_worker=0 2022-11-18T20:47:32.9008021Z 202 Accepted ★ 79ms time ★ 274B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:32.9008732Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:47:32.9009160Z │ { 2022-11-18T20:47:32.9009645Z │ "status": "Processing worker updates..." 2022-11-18T20:47:32.9010083Z │ } 2022-11-18T20:47:32.9018927Z └ 2022-11-18T20:47:32.9019530Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:32.9020222Z 1ms 240µs (cache) (cache) 76ms 1ms 54µs 79ms 2022-11-18T20:47:32.9020566Z 2022-11-18T20:47:32.9215684Z √ Successful request 2022-11-18T20:47:32.9337917Z 2022-11-18T20:47:32.9338315Z Iteration 69/82 2022-11-18T20:47:32.9496671Z 2022-11-18T20:47:32.9497522Z → management request 2022-11-18T20:47:33.0035271Z PUT http://localhost:8081/models/resnet-18?min_worker=-1 [errored] 2022-11-18T20:47:33.0036017Z read ECONNRESET 2022-11-18T20:47:33.0160106Z 22. Successful request 2022-11-18T20:47:33.0277411Z 2022-11-18T20:47:33.0278023Z Iteration 70/82 2022-11-18T20:47:33.0434802Z 2022-11-18T20:47:33.0435778Z → management request 2022-11-18T20:47:33.0965306Z PUT http://localhost:8081/models/resnet-18?max_worker=-1 [errored] 2022-11-18T20:47:33.0965942Z read ECONNRESET 2022-11-18T20:47:33.1410733Z 24. Successful request 2022-11-18T20:47:33.1413716Z 2022-11-18T20:47:33.1413995Z Iteration 71/82 2022-11-18T20:47:33.1414267Z 2022-11-18T20:47:33.1414762Z → management request 2022-11-18T20:47:33.1911940Z PUT http://localhost:8081/models/invalid_squeezenet1_1/1.0/set-default 2022-11-18T20:47:33.1912967Z 404 Not Found ★ 5ms time ★ 289B↑ 378B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:33.1913568Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 109B 2022-11-18T20:47:33.1914022Z │ { 2022-11-18T20:47:33.1914400Z │ "code": 404, 2022-11-18T20:47:33.1914829Z │ "type": "ModelNotFoundException", 2022-11-18T20:47:33.1915401Z │ "message": "Model not found: invalid_squeezenet1_1" 2022-11-18T20:47:33.1915805Z │ } 2022-11-18T20:47:33.1924892Z └ 2022-11-18T20:47:33.1925403Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:33.1925963Z 1ms 757µs 243µs 273µs 2ms 1ms 43µs 6ms 2022-11-18T20:47:33.1926191Z 2022-11-18T20:47:33.2053245Z √ Successful request 2022-11-18T20:47:33.2151490Z 2022-11-18T20:47:33.2151932Z Iteration 72/82 2022-11-18T20:47:33.2309989Z 2022-11-18T20:47:33.2310826Z → management request 2022-11-18T20:47:33.2968554Z DELETE http://localhost:8081/models/resnet-18 2022-11-18T20:47:33.2969369Z 200 OK ★ 12ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:33.2969868Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:47:33.2970237Z │ { 2022-11-18T20:47:33.2970644Z │ "status": "Model \"resnet-18\" unregistered" 2022-11-18T20:47:33.2971007Z │ } 2022-11-18T20:47:33.2971271Z └ 2022-11-18T20:47:33.2971791Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:33.2972365Z 3ms 726µs 231µs 255µs 9ms 1ms 45µs 15ms 2022-11-18T20:47:33.2972590Z 2022-11-18T20:47:33.3123900Z √ Successful request 2022-11-18T20:47:33.3253633Z 2022-11-18T20:47:33.3256029Z Iteration 73/82 2022-11-18T20:47:33.3404857Z 2022-11-18T20:47:33.3405557Z → management request 2022-11-18T20:47:33.3920136Z DELETE http://localhost:8081/models/squeezenet1_1/0.0 [errored] 2022-11-18T20:47:33.3920672Z read ECONNRESET 2022-11-18T20:47:33.4062829Z 26. Successful request 2022-11-18T20:47:33.4183636Z 2022-11-18T20:47:33.4184142Z Iteration 74/82 2022-11-18T20:47:33.4342867Z 2022-11-18T20:47:33.4343748Z → management request 2022-11-18T20:47:33.5808163Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:47:33.5809859Z 200 OK ★ 98ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:33.5810520Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:47:33.5810949Z │ { 2022-11-18T20:47:33.5811398Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:47:33.5811941Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:47:33.5812436Z │ o add workers for the model." 2022-11-18T20:47:33.5812822Z │ } 2022-11-18T20:47:33.5820326Z └ 2022-11-18T20:47:33.5820876Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:33.5821439Z 1ms 754µs 229µs 253µs 94ms 1ms 42µs 99ms 2022-11-18T20:47:33.5821863Z 2022-11-18T20:47:33.5889403Z √ Successful request 2022-11-18T20:47:33.5905311Z 2022-11-18T20:47:33.5905598Z Iteration 75/82 2022-11-18T20:47:33.6065121Z 2022-11-18T20:47:33.6065583Z → management request 2022-11-18T20:47:33.6625111Z DELETE http://localhost:8081/models/squeezenet1_1/?synchronous=true 2022-11-18T20:47:33.6626524Z 200 OK ★ 7ms time ★ 267B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:33.6627239Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:47:33.6627752Z │ { 2022-11-18T20:47:33.6628298Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:47:33.6628784Z │ } 2022-11-18T20:47:33.6638966Z └ 2022-11-18T20:47:33.6639612Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:33.6640366Z 1ms 271µs (cache) (cache) 5ms 1ms 43µs 8ms 2022-11-18T20:47:33.6640745Z 2022-11-18T20:47:33.6760319Z √ Successful request 2022-11-18T20:47:33.6846699Z 2022-11-18T20:47:33.6847179Z Iteration 76/82 2022-11-18T20:47:33.7004059Z 2022-11-18T20:47:33.7004958Z → management request 2022-11-18T20:47:33.8459422Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:47:33.8460573Z 200 OK ★ 97ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:33.8461226Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:47:33.8461638Z │ { 2022-11-18T20:47:33.8462103Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:47:33.8462634Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:47:33.8463164Z │ o add workers for the model." 2022-11-18T20:47:33.8463542Z │ } 2022-11-18T20:47:33.8473604Z └ 2022-11-18T20:47:33.8474191Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:33.8474844Z 1ms 248µs (cache) (cache) 94ms 1ms 44µs 98ms 2022-11-18T20:47:33.8475193Z 2022-11-18T20:47:33.8603362Z √ Successful request 2022-11-18T20:47:33.8725188Z 2022-11-18T20:47:33.8725671Z Iteration 77/82 2022-11-18T20:47:33.8883461Z 2022-11-18T20:47:33.8884173Z → management request 2022-11-18T20:47:33.9437815Z DELETE http://localhost:8081/models/squeezenet1_1/?synchronous=nan 2022-11-18T20:47:33.9439073Z 200 OK ★ 7ms time ★ 266B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:33.9439720Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:47:33.9440168Z │ { 2022-11-18T20:47:33.9440616Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:47:33.9441033Z │ } 2022-11-18T20:47:33.9449635Z └ 2022-11-18T20:47:33.9450201Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:33.9450852Z 1ms 231µs (cache) (cache) 4ms 1ms 40µs 7ms 2022-11-18T20:47:33.9451179Z 2022-11-18T20:47:33.9580287Z √ Successful request 2022-11-18T20:47:33.9595511Z 2022-11-18T20:47:33.9595957Z Iteration 78/82 2022-11-18T20:47:33.9665898Z 2022-11-18T20:47:33.9666566Z → management request 2022-11-18T20:47:34.1117520Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:47:34.1118820Z 200 OK ★ 97ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:34.1119545Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:47:34.1120056Z │ { 2022-11-18T20:47:34.1120562Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:47:34.1121171Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:47:34.1121761Z │ o add workers for the model." 2022-11-18T20:47:34.1122200Z │ } 2022-11-18T20:47:34.1129573Z └ 2022-11-18T20:47:34.1130133Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:34.1130778Z 1ms 257µs (cache) (cache) 94ms 1ms 39µs 97ms 2022-11-18T20:47:34.1131226Z 2022-11-18T20:47:34.1257789Z √ Successful request 2022-11-18T20:47:34.1380803Z 2022-11-18T20:47:34.1381181Z Iteration 79/82 2022-11-18T20:47:34.1539740Z 2022-11-18T20:47:34.1540564Z → management request 2022-11-18T20:47:34.1939303Z DELETE http://localhost:8081/models/squeezenet1_1/?timeout=true 2022-11-18T20:47:34.1940681Z 200 OK ★ 7ms time ★ 263B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:34.1941407Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:47:34.1941924Z │ { 2022-11-18T20:47:34.1942468Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:47:34.1942907Z │ } 2022-11-18T20:47:34.1951890Z └ 2022-11-18T20:47:34.1952490Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:34.1953160Z 1ms 249µs (cache) (cache) 4ms 1ms 40µs 8ms 2022-11-18T20:47:34.1953514Z 2022-11-18T20:47:34.2057157Z √ Successful request 2022-11-18T20:47:34.2163746Z 2022-11-18T20:47:34.2164269Z Iteration 80/82 2022-11-18T20:47:34.2321946Z 2022-11-18T20:47:34.2322685Z → management request 2022-11-18T20:47:34.3829679Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:47:34.3830792Z 200 OK ★ 100ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:34.3831373Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:47:34.3831780Z │ { 2022-11-18T20:47:34.3832220Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:47:34.3832767Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:47:34.3833221Z │ o add workers for the model." 2022-11-18T20:47:34.3833606Z │ } 2022-11-18T20:47:34.3833921Z └ 2022-11-18T20:47:34.3834442Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:34.3835100Z 2ms 256µs (cache) (cache) 95ms 4ms 41µs 101ms 2022-11-18T20:47:34.3835405Z 2022-11-18T20:47:34.3946858Z √ Successful request 2022-11-18T20:47:34.4038468Z 2022-11-18T20:47:34.4039001Z Iteration 81/82 2022-11-18T20:47:34.4195860Z 2022-11-18T20:47:34.4196750Z → management request 2022-11-18T20:47:34.4747670Z DELETE http://localhost:8081/models/squeezenet1_1/?timeout=true&synchronous=-1 2022-11-18T20:47:34.4748807Z 200 OK ★ 7ms time ★ 278B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:34.4749470Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:47:34.4749917Z │ { 2022-11-18T20:47:34.4750428Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:47:34.4750849Z │ } 2022-11-18T20:47:34.4763858Z └ 2022-11-18T20:47:34.4764589Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:34.4765322Z 1ms 257µs (cache) (cache) 4ms 1ms 37µs 7ms 2022-11-18T20:47:34.4765921Z 2022-11-18T20:47:34.4805842Z √ Successful request 2022-11-18T20:47:34.4975110Z 2022-11-18T20:47:34.4975545Z Iteration 82/82 2022-11-18T20:47:34.5134669Z 2022-11-18T20:47:34.5135845Z → management request 2022-11-18T20:47:34.5652103Z DELETE http://localhost:8081/models/invalid_squeezenet1_1 2022-11-18T20:47:34.5653403Z 404 Not Found ★ 4ms time ★ 257B↑ 378B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:47:34.5654183Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 109B 2022-11-18T20:47:34.5654657Z │ { 2022-11-18T20:47:34.5655067Z │ "code": 404, 2022-11-18T20:47:34.5655598Z │ "type": "ModelNotFoundException", 2022-11-18T20:47:34.5656176Z │ "message": "Model not found: invalid_squeezenet1_1" 2022-11-18T20:47:34.5656628Z │ } 2022-11-18T20:47:34.5673863Z └ 2022-11-18T20:47:34.5674646Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:47:34.5675482Z 1ms 241µs (cache) (cache) 1ms 1ms 35µs 4ms 2022-11-18T20:47:34.5675877Z 2022-11-18T20:47:34.5702650Z √ Successful request 2022-11-18T20:47:38.2100428Z 2022-11-18T20:47:38.2101393Z ┌─────────────────────────┬────────────────────┬────────────────────┐ 2022-11-18T20:47:38.2101957Z │ │ executed │ failed │ 2022-11-18T20:47:38.2102478Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:47:38.2102938Z │ iterations │ 82 │ 0 │ 2022-11-18T20:47:38.2103436Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:47:38.2103897Z │ requests │ 82 │ 13 │ 2022-11-18T20:47:38.2104389Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:47:38.2104826Z │ test-scripts │ 82 │ 0 │ 2022-11-18T20:47:38.2105346Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:47:38.2105825Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:47:38.2106345Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:47:38.2106768Z │ assertions │ 82 │ 13 │ 2022-11-18T20:47:38.2107245Z ├─────────────────────────┴────────────────────┴────────────────────┤ 2022-11-18T20:47:38.2107677Z │ total run duration: 51.7s │ 2022-11-18T20:47:38.2108167Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:47:38.2108604Z │ total data received: 6.23kB (approx) │ 2022-11-18T20:47:38.2109107Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:47:38.2109598Z │ average response time: 563ms [min: 4ms, max: 21.1s, s.d.: 2.6s] │ 2022-11-18T20:47:38.2110398Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:47:38.2110857Z │ average DNS lookup time: 4ms [min: 213µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:47:38.2111475Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:47:38.2111972Z │ average first byte time: 559ms [min: 1ms, max: 21.1s, s.d.: 2.6s] │ 2022-11-18T20:47:38.2112490Z └───────────────────────────────────────────────────────────────────┘ 2022-11-18T20:47:38.2212245Z 2022-11-18T20:47:38.2212523Z # failure detail 2022-11-18T20:47:38.2212875Z 2022-11-18T20:47:38.2213346Z 01. Error 2022-11-18T20:47:38.2213672Z iteration: 6 read ECONNRESET 2022-11-18T20:47:38.2213952Z at request 2022-11-18T20:47:38.2214270Z inside "management request" 2022-11-18T20:47:38.2214611Z 2022-11-18T20:47:38.2214887Z 02. AssertionError Successful request 2022-11-18T20:47:38.2215262Z iteration: 6 expected undefined to be one of [ 400 ] 2022-11-18T20:47:38.2215836Z at assertion:0 in test-script 2022-11-18T20:47:38.2216176Z inside "management request" 2022-11-18T20:47:38.2216444Z 2022-11-18T20:47:38.2216729Z 03. Error 2022-11-18T20:47:38.2217041Z iteration: 11 read ECONNRESET 2022-11-18T20:47:38.2217318Z at request 2022-11-18T20:47:38.2217634Z inside "management request" 2022-11-18T20:47:38.2217941Z 2022-11-18T20:47:38.2218261Z 04. AssertionError Successful request 2022-11-18T20:47:38.2218573Z iteration: 11 expected undefined to be one of [ 500 ] 2022-11-18T20:47:38.2218990Z at assertion:0 in test-script 2022-11-18T20:47:38.2219319Z inside "management request" 2022-11-18T20:47:38.2219582Z 2022-11-18T20:47:38.2219855Z 05. Error 2022-11-18T20:47:38.2220164Z iteration: 17 read ECONNRESET 2022-11-18T20:47:38.2220469Z at request 2022-11-18T20:47:38.2220738Z inside "management request" 2022-11-18T20:47:38.2221044Z 2022-11-18T20:47:38.2221354Z 06. AssertionError Successful request 2022-11-18T20:47:38.2221719Z iteration: 17 expected undefined to be one of [ 400 ] 2022-11-18T20:47:38.2222111Z at assertion:0 in test-script 2022-11-18T20:47:38.2222681Z inside "management request" 2022-11-18T20:47:38.2223117Z 2022-11-18T20:47:38.2223432Z 07. Error 2022-11-18T20:47:38.2223816Z iteration: 23 read ECONNRESET 2022-11-18T20:47:38.2224189Z at request 2022-11-18T20:47:38.2224570Z inside "management request" 2022-11-18T20:47:38.2226562Z 2022-11-18T20:47:38.2226873Z 08. AssertionError Successful request 2022-11-18T20:47:38.2227229Z iteration: 23 expected undefined to be one of [ 404 ] 2022-11-18T20:47:38.2227901Z at assertion:0 in test-script 2022-11-18T20:47:38.2228206Z inside "management request" 2022-11-18T20:47:38.2228528Z 2022-11-18T20:47:38.2228823Z 09. Error 2022-11-18T20:47:38.2229138Z iteration: 26 read ECONNRESET 2022-11-18T20:47:38.2229414Z at request 2022-11-18T20:47:38.2229721Z inside "management request" 2022-11-18T20:47:38.2230027Z 2022-11-18T20:47:38.2230300Z 10. AssertionError Successful request 2022-11-18T20:47:38.2230646Z iteration: 26 expected undefined to be one of [ 404 ] 2022-11-18T20:47:38.2231099Z at assertion:0 in test-script 2022-11-18T20:47:38.2231639Z inside "management request" 2022-11-18T20:47:38.2231917Z 2022-11-18T20:47:38.2232211Z 11. Error 2022-11-18T20:47:38.2232526Z iteration: 31 read ECONNRESET 2022-11-18T20:47:38.2232803Z at request 2022-11-18T20:47:38.2233104Z inside "management request" 2022-11-18T20:47:38.2233387Z 2022-11-18T20:47:38.2233663Z 12. AssertionError Successful request 2022-11-18T20:47:38.2234012Z iteration: 31 expected undefined to be one of [ 500 ] 2022-11-18T20:47:38.2234429Z at assertion:0 in test-script 2022-11-18T20:47:38.2234741Z inside "management request" 2022-11-18T20:47:38.2235010Z 2022-11-18T20:47:38.2235281Z 13. Error 2022-11-18T20:47:38.2235581Z iteration: 58 read ECONNRESET 2022-11-18T20:47:38.2235908Z at request 2022-11-18T20:47:38.2236232Z inside "management request" 2022-11-18T20:47:38.2236540Z 2022-11-18T20:47:38.2236836Z 14. AssertionError Successful request 2022-11-18T20:47:38.2237127Z iteration: 58 expected undefined to be one of [ 400 ] 2022-11-18T20:47:38.2237583Z at assertion:0 in test-script 2022-11-18T20:47:38.2237870Z inside "management request" 2022-11-18T20:47:38.2238111Z 2022-11-18T20:47:38.2238356Z 15. Error 2022-11-18T20:47:38.2238621Z iteration: 60 read ECONNRESET 2022-11-18T20:47:38.2238983Z at request 2022-11-18T20:47:38.2239239Z inside "management request" 2022-11-18T20:47:38.2239501Z 2022-11-18T20:47:38.2239771Z 16. AssertionError Successful request 2022-11-18T20:47:38.2240063Z iteration: 60 expected undefined to be one of [ 404 ] 2022-11-18T20:47:38.2240443Z at assertion:0 in test-script 2022-11-18T20:47:38.2240731Z inside "management request" 2022-11-18T20:47:38.2240988Z 2022-11-18T20:47:38.2241209Z 17. Error 2022-11-18T20:47:38.2241538Z iteration: 65 read ECONNRESET 2022-11-18T20:47:38.2241896Z at request 2022-11-18T20:47:38.2242155Z inside "management request" 2022-11-18T20:47:38.2242418Z 2022-11-18T20:47:38.2242692Z 18. AssertionError Successful request 2022-11-18T20:47:38.2242978Z iteration: 65 expected undefined to be one of [ 404 ] 2022-11-18T20:47:38.2243363Z at assertion:0 in test-script 2022-11-18T20:47:38.2243644Z inside "management request" 2022-11-18T20:47:38.2243905Z 2022-11-18T20:47:38.2244121Z 19. Error 2022-11-18T20:47:38.2244379Z iteration: 66 read ECONNRESET 2022-11-18T20:47:38.2244651Z at request 2022-11-18T20:47:38.2244898Z inside "management request" 2022-11-18T20:47:38.2245166Z 2022-11-18T20:47:38.2245440Z 20. AssertionError Successful request 2022-11-18T20:47:38.2245745Z iteration: 66 expected undefined to be one of [ 400 ] 2022-11-18T20:47:38.2246105Z at assertion:0 in test-script 2022-11-18T20:47:38.2246395Z inside "management request" 2022-11-18T20:47:38.2246660Z 2022-11-18T20:47:38.2246885Z 21. Error 2022-11-18T20:47:38.2247148Z iteration: 69 read ECONNRESET 2022-11-18T20:47:38.2247413Z at request 2022-11-18T20:47:38.2247684Z inside "management request" 2022-11-18T20:47:38.2247925Z 2022-11-18T20:47:38.2248193Z 22. AssertionError Successful request 2022-11-18T20:47:38.2248497Z iteration: 69 expected undefined to be one of [ 500 ] 2022-11-18T20:47:38.2248850Z at assertion:0 in test-script 2022-11-18T20:47:38.2249135Z inside "management request" 2022-11-18T20:47:38.2249393Z 2022-11-18T20:47:38.2249610Z 23. Error 2022-11-18T20:47:38.2249869Z iteration: 70 read ECONNRESET 2022-11-18T20:47:38.2250131Z at request 2022-11-18T20:47:38.2250401Z inside "management request" 2022-11-18T20:47:38.2250740Z 2022-11-18T20:47:38.2251007Z 24. AssertionError Successful request 2022-11-18T20:47:38.2251386Z iteration: 70 expected undefined to be one of [ 400 ] 2022-11-18T20:47:38.2251745Z at assertion:0 in test-script 2022-11-18T20:47:38.2252031Z inside "management request" 2022-11-18T20:47:38.2252291Z 2022-11-18T20:47:38.2252531Z 25. Error 2022-11-18T20:47:38.2252774Z iteration: 73 read ECONNRESET 2022-11-18T20:47:38.2253039Z at request 2022-11-18T20:47:38.2253306Z inside "management request" 2022-11-18T20:47:38.2253541Z 2022-11-18T20:47:38.2253862Z 26. AssertionError Successful request 2022-11-18T20:47:38.2254168Z iteration: 73 expected undefined to be one of [ 404 ] 2022-11-18T20:47:38.2254630Z at assertion:0 in test-script 2022-11-18T20:47:38.2254905Z inside "management request" 2022-11-18T20:47:38.4261263Z TorchServe has stopped. 2022-11-18T20:47:59.9237670Z newman 2022-11-18T20:47:59.9238107Z 2022-11-18T20:47:59.9238421Z inference 2022-11-18T20:47:59.9238696Z 2022-11-18T20:47:59.9238922Z Iteration 1/24 2022-11-18T20:47:59.9256964Z 2022-11-18T20:47:59.9257716Z → Model Zoo - Register Model 2022-11-18T20:48:11.9455501Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/my_text_classifier_v4.mar&model_name=my_text_classifier&initial_workers=1&synchronous=true 2022-11-18T20:48:11.9456492Z 200 OK ★ 11.5s time ★ 388B↑ 360B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:11.9456946Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 94B 2022-11-18T20:48:11.9457230Z │ { 2022-11-18T20:48:11.9457562Z │ "status": "Model \"my_text_classifier\" Version: 1.0 2022-11-18T20:48:11.9457908Z │ registered with 1 initial workers" 2022-11-18T20:48:11.9458172Z │ } 2022-11-18T20:48:11.9458355Z └ 2022-11-18T20:48:11.9458750Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:11.9459231Z 34ms 13ms 10ms 570µs 11.5s 9ms 474µs 11.6s 2022-11-18T20:48:11.9459399Z 2022-11-18T20:48:11.9459554Z √ Successful POST request 2022-11-18T20:48:11.9459705Z 2022-11-18T20:48:11.9459841Z → Model Zoo - Inference Model 2022-11-18T20:48:11.9460161Z POST http://localhost:8080/predictions/my_text_classifier 2022-11-18T20:48:11.9460589Z 200 OK ★ 210ms time ★ 354B↑ 373B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:48:11.9460873Z ┌ ↑ file ★ 77B 2022-11-18T20:48:11.9461084Z │ 2022-11-18T20:48:11.9461279Z └ 2022-11-18T20:48:11.9461574Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 138B 2022-11-18T20:48:11.9461809Z │ { 2022-11-18T20:48:11.9462063Z │ "World": 0.029119664803147316, 2022-11-18T20:48:11.9462369Z │ "Sports": 2.943157232948579e-05, 2022-11-18T20:48:11.9462647Z │ "Business": 0.9074352383613586, 2022-11-18T20:48:11.9462936Z │ "Sci/Tec": 0.06341569125652313 2022-11-18T20:48:11.9463175Z │ } 2022-11-18T20:48:11.9463343Z └ 2022-11-18T20:48:11.9463745Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:11.9464219Z 8ms 1ms 327µs 1ms 204ms 2ms 92µs 218ms 2022-11-18T20:48:11.9464392Z 2022-11-18T20:48:11.9464546Z √ Successful POST request 2022-11-18T20:48:11.9464829Z √ Test expected JSON response 2022-11-18T20:48:12.4604933Z 2022-11-18T20:48:12.4605614Z → Model Zoo - Unregister model 2022-11-18T20:48:12.4606077Z DELETE http://localhost:8081/models/my_text_classifier 2022-11-18T20:48:12.4607209Z 200 OK ★ 128ms time ★ 254B↑ 326B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:12.4607928Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 60B 2022-11-18T20:48:12.4608240Z │ { 2022-11-18T20:48:12.4608636Z │ "status": "Model \"my_text_classifier\" unregistered 2022-11-18T20:48:12.4608973Z │ " 2022-11-18T20:48:12.4609222Z │ } 2022-11-18T20:48:12.4609459Z └ 2022-11-18T20:48:12.4609912Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:12.4610497Z 1ms 681µs (cache) (cache) 125ms 1ms 101µs 129ms 2022-11-18T20:48:12.4610737Z 2022-11-18T20:48:12.4610955Z √ Successful DELETE request 2022-11-18T20:48:12.4611163Z 2022-11-18T20:48:12.4611361Z → Model Zoo - Model Metrics 2022-11-18T20:48:12.4611689Z GET http://localhost:8082/metrics 2022-11-18T20:48:12.4612263Z 200 OK ★ 99ms time ★ 233B↑ 1.08kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:12.4612706Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 791B 2022-11-18T20:48:12.4613147Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:48:12.4613507Z │ erence requests. 2022-11-18T20:48:12.4613892Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:48:12.4614339Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:12.4614856Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:48:12.4615237Z │ l_version="default",} 1.0 2022-11-18T20:48:12.4615679Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:48:12.4625781Z │ ference duration in microseconds 2022-11-18T20:48:12.4626323Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:48:12.4626790Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:12.4627265Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:12.4627710Z │ ",model_version="default",} 114048.9 2022-11-18T20:48:12.4628145Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:48:12.4628526Z │ duration in microseconds 2022-11-18T20:48:12.4628954Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:48:12.4629406Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:12.4629866Z │ -bfe3-7d6b11352b25",model_name="my_text_classifier",mo 2022-11-18T20:48:12.4630339Z │ del_version="default",} 638.7 2022-11-18T20:48:12.4630647Z └ 2022-11-18T20:48:12.4631139Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:12.4631692Z 1ms 996µs 234µs 250µs 94ms 2ms 77µs 100ms 2022-11-18T20:48:12.4631934Z 2022-11-18T20:48:12.4632133Z √ Successful GET request 2022-11-18T20:48:12.4632283Z 2022-11-18T20:48:12.4632372Z Iteration 2/24 2022-11-18T20:48:12.4632505Z 2022-11-18T20:48:12.4632756Z → Model Zoo - Register Model 2022-11-18T20:48:22.4734793Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/my_text_classifier_scripted_v3.mar&model_name=my_text_classifier_scripted&initial_workers=1&synchronous=true 2022-11-18T20:48:22.4736172Z 200 OK ★ 10s time ★ 406B↑ 370B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:22.4736773Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 103B 2022-11-18T20:48:22.4737174Z │ { 2022-11-18T20:48:22.4737605Z │ "status": "Model \"my_text_classifier_scripted\" Ver 2022-11-18T20:48:22.4738091Z │ sion: 1.0 registered with 1 initial workers" 2022-11-18T20:48:22.4738460Z │ } 2022-11-18T20:48:22.4755355Z └ 2022-11-18T20:48:22.4755886Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:22.4756483Z 2ms 463µs (cache) (cache) 10s 4ms 103µs 10s 2022-11-18T20:48:22.4756775Z 2022-11-18T20:48:22.4875169Z √ Successful POST request 2022-11-18T20:48:22.5116048Z 2022-11-18T20:48:22.5117006Z → Model Zoo - Inference Model 2022-11-18T20:48:23.5984488Z POST http://localhost:8080/predictions/my_text_classifier_scripted 2022-11-18T20:48:23.5987354Z 200 OK ★ 1033ms time ★ 363B↑ 372B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:48:23.5989542Z ┌ ↑ file ★ 77B 2022-11-18T20:48:23.5989957Z │ 2022-11-18T20:48:23.5991739Z └ 2022-11-18T20:48:23.5992143Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 137B 2022-11-18T20:48:23.5992446Z │ { 2022-11-18T20:48:23.5992886Z │ "World": 0.04559723287820816, 2022-11-18T20:48:23.5993418Z │ "Sports": 0.0003771767660509795, 2022-11-18T20:48:23.5993782Z │ "Business": 0.08623101562261581, 2022-11-18T20:48:23.5994110Z │ "Sci/Tec": 0.8677946329116821 2022-11-18T20:48:23.5994406Z │ } 2022-11-18T20:48:23.6007560Z └ 2022-11-18T20:48:23.6008070Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:23.6008776Z 4ms 439µs (cache) (cache) 1029ms 2ms 82µs 1037ms 2022-11-18T20:48:23.6009151Z 2022-11-18T20:48:23.6167856Z √ Successful POST request 2022-11-18T20:48:23.6174779Z √ Test expected JSON response 2022-11-18T20:48:23.6407616Z 2022-11-18T20:48:23.6408469Z → Model Zoo - Unregister model 2022-11-18T20:48:23.8086317Z DELETE http://localhost:8081/models/my_text_classifier_scripted 2022-11-18T20:48:24.5847555Z 200 OK ★ 119ms time ★ 263B↑ 335B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:25.0885367Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 69B 2022-11-18T20:48:25.0885688Z │ { 2022-11-18T20:48:25.0886018Z │ "status": "Model \"my_text_classifier_scripted\" unr 2022-11-18T20:48:25.0886323Z │ egistered" 2022-11-18T20:48:25.0886546Z │ } 2022-11-18T20:48:25.0886717Z └ 2022-11-18T20:48:25.0887138Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:25.0887662Z 1ms 348µs (cache) (cache) 115ms 2ms 63µs 119ms 2022-11-18T20:48:25.0887846Z 2022-11-18T20:48:25.0888012Z √ Successful DELETE request 2022-11-18T20:48:25.0888165Z 2022-11-18T20:48:25.0888306Z → Model Zoo - Model Metrics 2022-11-18T20:48:25.0888583Z GET http://localhost:8082/metrics 2022-11-18T20:48:25.0889000Z 200 OK ★ 7ms time ★ 233B↑ 1.53kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:25.0889356Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 1.24kB 2022-11-18T20:48:25.0889732Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:48:25.0890040Z │ erence requests. 2022-11-18T20:48:25.0890354Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:48:25.0890714Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:25.0891104Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:48:25.0891430Z │ l_version="default",} 1.0 2022-11-18T20:48:25.0891780Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:25.0892166Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:48:25.0892503Z │ ted",model_version="default",} 1.0 2022-11-18T20:48:25.0893054Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:48:25.0893419Z │ ference duration in microseconds 2022-11-18T20:48:25.0893776Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:48:25.0894167Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:25.0894533Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:25.0894871Z │ ",model_version="default",} 114048.9 2022-11-18T20:48:25.0895234Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:25.0895595Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:25.0895956Z │ _scripted",model_version="default",} 1027094.1 2022-11-18T20:48:25.0896334Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:48:25.0896735Z │ duration in microseconds 2022-11-18T20:48:25.0897071Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:48:25.0897457Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:25.0897848Z │ -bfe3-7d6b11352b25",model_name="my_text_classifier",mo 2022-11-18T20:48:25.0898162Z │ del_version="default",} 638.7 2022-11-18T20:48:25.0898517Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:25.0898900Z │ -bfe3-7d6b11352b25",model_name="my_text_classifier_scr 2022-11-18T20:48:25.0899231Z │ ipted",model_version="default",} 1122.8 2022-11-18T20:48:25.0899474Z └ 2022-11-18T20:48:25.0899878Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:25.0900341Z 2ms 371µs (cache) (cache) 3ms 2ms 61µs 8ms 2022-11-18T20:48:25.0900522Z 2022-11-18T20:48:25.0900671Z √ Successful GET request 2022-11-18T20:48:25.0900826Z 2022-11-18T20:48:25.0900913Z Iteration 3/24 2022-11-18T20:48:25.0901045Z 2022-11-18T20:48:25.0901196Z → Model Zoo - Register Model 2022-11-18T20:48:26.6032664Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&initial_workers=1&synchronous=true 2022-11-18T20:48:26.6033585Z 200 OK ★ 2.3s time ★ 334B↑ 355B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:26.6034074Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 89B 2022-11-18T20:48:26.6034427Z │ { 2022-11-18T20:48:26.6034808Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:48:26.6035214Z │ stered with 1 initial workers" 2022-11-18T20:48:26.6035528Z │ } 2022-11-18T20:48:26.6035787Z └ 2022-11-18T20:48:26.6036255Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:26.6036813Z 2ms 503µs (cache) (cache) 2.3s 2ms 63µs 2.3s 2022-11-18T20:48:26.6037077Z 2022-11-18T20:48:26.6039511Z √ Successful POST request 2022-11-18T20:48:26.6039773Z 2022-11-18T20:48:26.6040023Z → Model Zoo - Inference Model 2022-11-18T20:48:28.9027500Z POST http://localhost:8080/predictions/squeezenet1_1 2022-11-18T20:48:28.9028781Z 200 OK ★ 2.4s time ★ 111.25kB↑ 409B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:48:28.9030895Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:48:28.9031373Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:48:28.9033305Z └ 2022-11-18T20:48:28.9033821Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 174B 2022-11-18T20:48:28.9034111Z │ { 2022-11-18T20:48:28.9034398Z │ "tabby": 0.27850738167762756, 2022-11-18T20:48:28.9034736Z │ "lynx": 0.25299158692359924, 2022-11-18T20:48:28.9035029Z │ "tiger_cat": 0.24496474862098694, 2022-11-18T20:48:28.9035331Z │ "Egyptian_cat": 0.21722468733787537, 2022-11-18T20:48:28.9035625Z │ "cougar": 0.002217544475570321 2022-11-18T20:48:28.9035846Z │ } 2022-11-18T20:48:28.9036060Z └ 2022-11-18T20:48:28.9036480Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:28.9036955Z 3ms 668µs (cache) (cache) 2.4s 1ms 69µs 2.4s 2022-11-18T20:48:28.9037394Z 2022-11-18T20:48:28.9206704Z √ Successful POST request 2022-11-18T20:48:28.9207453Z √ Test expected JSON response 2022-11-18T20:48:28.9440570Z 2022-11-18T20:48:28.9441422Z → Model Zoo - Unregister model 2022-11-18T20:48:29.0730980Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:48:29.0732009Z 200 OK ★ 80ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:29.0732571Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:48:29.0732928Z │ { 2022-11-18T20:48:29.0733352Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:48:29.0733723Z │ } 2022-11-18T20:48:29.0746683Z └ 2022-11-18T20:48:29.0747178Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:29.0747998Z 1ms 322µs (cache) (cache) 77ms 1ms 248µs 81ms 2022-11-18T20:48:29.0748276Z 2022-11-18T20:48:29.0873598Z √ Successful DELETE request 2022-11-18T20:48:29.1170127Z 2022-11-18T20:48:29.1170929Z → Model Zoo - Model Metrics 2022-11-18T20:48:29.1705007Z GET http://localhost:8082/metrics 2022-11-18T20:48:29.1706221Z 200 OK ★ 5ms time ★ 233B↑ 1.93kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:29.1706741Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 1.64kB 2022-11-18T20:48:29.1707183Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:48:29.1707564Z │ erence requests. 2022-11-18T20:48:29.1707951Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:48:29.1708389Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:29.1708859Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:48:29.1709296Z │ l_version="default",} 1.0 2022-11-18T20:48:29.1709724Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:29.1710155Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:48:29.1710527Z │ sion="default",} 1.0 2022-11-18T20:48:29.1710959Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:29.1711410Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:48:29.1711801Z │ ted",model_version="default",} 1.0 2022-11-18T20:48:29.1712241Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:48:29.1712662Z │ ference duration in microseconds 2022-11-18T20:48:29.1713089Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:48:29.1713536Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:29.1713988Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:29.1714396Z │ ",model_version="default",} 114048.9 2022-11-18T20:48:29.1717330Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:29.1726916Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1",mod 2022-11-18T20:48:29.1729295Z │ el_version="default",} 2472224.3 2022-11-18T20:48:29.1730002Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:29.1731959Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:29.1732426Z │ _scripted",model_version="default",} 1027094.1 2022-11-18T20:48:29.1734478Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:48:29.1735240Z │ duration in microseconds 2022-11-18T20:48:29.1735650Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:48:29.1737683Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:29.1738163Z │ -bfe3-7d6b11352b25",model_name="my_text_classifier",mo 2022-11-18T20:48:29.1738567Z │ del_version="default",} 638.7 2022-11-18T20:48:29.1738974Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:29.1739963Z │ -bfe3-7d6b11352b25",model_name="squeezenet1_1",model_v 2022-11-18T20:48:29.1742435Z │ ersion="default",} 391.8 2022-11-18T20:48:29.1743457Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:29.1745584Z │ -bfe3-7d6b11352b25",model_name="my_text_classifier_scr 2022-11-18T20:48:29.1747582Z │ ipted",model_version="default",} 1122.8 2022-11-18T20:48:29.1749619Z └ 2022-11-18T20:48:29.1750137Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:29.1752075Z 1ms 358µs (cache) (cache) 2ms 1ms 68µs 6ms 2022-11-18T20:48:29.1752342Z 2022-11-18T20:48:29.1830080Z √ Successful GET request 2022-11-18T20:48:29.1954367Z 2022-11-18T20:48:29.1954665Z Iteration 4/24 2022-11-18T20:48:29.2113756Z 2022-11-18T20:48:29.2114558Z → Model Zoo - Register Model 2022-11-18T20:48:31.6834448Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/squeezenet1_1_scripted.mar&model_name=squeezenet1_1_scripted&initial_workers=1&synchronous=true 2022-11-18T20:48:31.6836512Z 200 OK ★ 2.4s time ★ 393B↑ 364B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:31.6837079Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 98B 2022-11-18T20:48:31.6837466Z │ { 2022-11-18T20:48:31.6837915Z │ "status": "Model \"squeezenet1_1_scripted\" Version: 2022-11-18T20:48:31.6838531Z │ 1.0 registered with 1 initial workers" 2022-11-18T20:48:31.6839088Z │ } 2022-11-18T20:48:31.6856917Z └ 2022-11-18T20:48:31.6857540Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:31.6858127Z 1ms 834µs (cache) (cache) 2.4s 2ms 63µs 2.4s 2022-11-18T20:48:31.6858368Z 2022-11-18T20:48:31.7054095Z √ Successful POST request 2022-11-18T20:48:31.7262593Z 2022-11-18T20:48:31.7263587Z → Model Zoo - Inference Model 2022-11-18T20:48:31.9968035Z POST http://localhost:8080/predictions/squeezenet1_1_scripted 2022-11-18T20:48:31.9969108Z 200 OK ★ 221ms time ★ 111.25kB↑ 409B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:48:31.9969586Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:48:31.9970052Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:48:31.9970452Z └ 2022-11-18T20:48:31.9971034Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 174B 2022-11-18T20:48:31.9971437Z │ { 2022-11-18T20:48:31.9971845Z │ "tabby": 0.27850738167762756, 2022-11-18T20:48:31.9972245Z │ "lynx": 0.25299158692359924, 2022-11-18T20:48:31.9972672Z │ "tiger_cat": 0.24496474862098694, 2022-11-18T20:48:31.9973115Z │ "Egyptian_cat": 0.21722468733787537, 2022-11-18T20:48:31.9973549Z │ "cougar": 0.002217544475570321 2022-11-18T20:48:31.9973902Z │ } 2022-11-18T20:48:31.9994447Z └ 2022-11-18T20:48:31.9995218Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:31.9995856Z 2ms 548µs (cache) (cache) 216ms 2ms 49µs 222ms 2022-11-18T20:48:31.9996191Z 2022-11-18T20:48:32.0135922Z √ Successful POST request 2022-11-18T20:48:32.0139943Z √ Test expected JSON response 2022-11-18T20:48:32.0399535Z 2022-11-18T20:48:32.0400572Z → Model Zoo - Unregister model 2022-11-18T20:48:32.1701066Z DELETE http://localhost:8081/models/squeezenet1_1_scripted 2022-11-18T20:48:32.1702069Z 200 OK ★ 81ms time ★ 258B↑ 330B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:32.1702549Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 64B 2022-11-18T20:48:32.1702885Z │ { 2022-11-18T20:48:32.1703256Z │ "status": "Model \"squeezenet1_1_scripted\" unregist 2022-11-18T20:48:32.1703599Z │ ered" 2022-11-18T20:48:32.1703870Z │ } 2022-11-18T20:48:32.1714581Z └ 2022-11-18T20:48:32.1715102Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:32.1715656Z 2ms 473µs (cache) (cache) 78ms 1ms 53µs 82ms 2022-11-18T20:48:32.1715929Z 2022-11-18T20:48:32.1778524Z √ Successful DELETE request 2022-11-18T20:48:32.2045135Z 2022-11-18T20:48:32.2045747Z → Model Zoo - Model Metrics 2022-11-18T20:48:32.2589427Z GET http://localhost:8082/metrics 2022-11-18T20:48:32.2590552Z 200 OK ★ 5ms time ★ 233B↑ 2.36kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:32.2591090Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 2.07kB 2022-11-18T20:48:32.2591607Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:48:32.2592022Z │ erence requests. 2022-11-18T20:48:32.2592459Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:48:32.2592993Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:32.2593520Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:48:32.2593955Z │ l_version="default",} 1.0 2022-11-18T20:48:32.2594429Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:32.2595099Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:48:32.2595530Z │ model_version="default",} 1.0 2022-11-18T20:48:32.2596008Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:32.2596526Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:48:32.2596941Z │ sion="default",} 1.0 2022-11-18T20:48:32.2597380Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:32.2597883Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:48:32.2598340Z │ ted",model_version="default",} 1.0 2022-11-18T20:48:32.2598827Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:48:32.2599271Z │ ference duration in microseconds 2022-11-18T20:48:32.2599748Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:48:32.2600267Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:32.2600771Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:32.2601215Z │ ",model_version="default",} 114048.9 2022-11-18T20:48:32.2601698Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:32.2602213Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1_scri 2022-11-18T20:48:32.2602682Z │ pted",model_version="default",} 211321.7 2022-11-18T20:48:32.2603154Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:32.2603668Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1",mod 2022-11-18T20:48:32.2604113Z │ el_version="default",} 2472224.3 2022-11-18T20:48:32.2604564Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:32.2605064Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:32.2605539Z │ _scripted",model_version="default",} 1027094.1 2022-11-18T20:48:32.2606034Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:48:32.2606452Z │ duration in microseconds 2022-11-18T20:48:32.2606911Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:48:32.2607413Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:32.2607920Z │ -bfe3-7d6b11352b25",model_name="my_text_classifier",mo 2022-11-18T20:48:32.2608346Z │ del_version="default",} 638.7 2022-11-18T20:48:32.2608819Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:32.2609322Z │ -bfe3-7d6b11352b25",model_name="squeezenet1_1_scripted 2022-11-18T20:48:32.2609722Z │ ",model_version="default",} 254.1 2022-11-18T20:48:32.2610274Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:32.2610793Z │ -bfe3-7d6b11352b25",model_name="squeezenet1_1",model_v 2022-11-18T20:48:32.2611236Z │ ersion="default",} 391.8 2022-11-18T20:48:32.2611716Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:32.2612208Z │ -bfe3-7d6b11352b25",model_name="my_text_classifier_scr 2022-11-18T20:48:32.2612632Z │ ipted",model_versio 2022-11-18T20:48:32.2613031Z │ (showing 2.05kB/2.07kB) 2022-11-18T20:48:32.2613360Z └ 2022-11-18T20:48:32.2613892Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:32.2614640Z 1ms 349µs (cache) (cache) 2ms 1ms 68µs 6ms 2022-11-18T20:48:32.2614933Z 2022-11-18T20:48:32.2724644Z √ Successful GET request 2022-11-18T20:48:32.2834679Z 2022-11-18T20:48:32.2835055Z Iteration 5/24 2022-11-18T20:48:32.2991327Z 2022-11-18T20:48:32.2992127Z → Model Zoo - Register Model 2022-11-18T20:48:37.2382466Z POST http://localhost:8081/models?url=densenet161.mar&model_name=densenet161&initial_workers=1&synchronous=true 2022-11-18T20:48:37.2383660Z 200 OK ★ 4.9s time ★ 330B↑ 353B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:37.2384462Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 87B 2022-11-18T20:48:37.2384969Z │ { 2022-11-18T20:48:37.2385497Z │ "status": "Model \"densenet161\" Version: 1.0 regist 2022-11-18T20:48:37.2386240Z │ ered with 1 initial workers" 2022-11-18T20:48:37.2386661Z │ } 2022-11-18T20:48:37.2407955Z └ 2022-11-18T20:48:37.2408705Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:37.2409460Z 1ms 331µs (cache) (cache) 4.9s 1ms 45µs 4.9s 2022-11-18T20:48:37.2409851Z 2022-11-18T20:48:37.2593976Z √ Successful POST request 2022-11-18T20:48:37.2863958Z 2022-11-18T20:48:37.2864903Z → Model Zoo - Inference Model 2022-11-18T20:48:38.3682507Z POST http://localhost:8080/predictions/densenet161 2022-11-18T20:48:38.3683700Z 200 OK ★ 1034ms time ★ 111.24kB↑ 417B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:48:38.3684336Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:48:38.3684844Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:48:38.3685284Z └ 2022-11-18T20:48:38.3685944Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 182B 2022-11-18T20:48:38.3686416Z │ { 2022-11-18T20:48:38.3686882Z │ "tabby": 0.4666188061237335, 2022-11-18T20:48:38.3687369Z │ "tiger_cat": 0.46449077129364014, 2022-11-18T20:48:38.3687867Z │ "Egyptian_cat": 0.06614049524068832, 2022-11-18T20:48:38.3688367Z │ "lynx": 0.0012924447655677795, 2022-11-18T20:48:38.3688841Z │ "plastic_bag": 0.00022909819381311536 2022-11-18T20:48:38.3689280Z │ } 2022-11-18T20:48:38.3697541Z └ 2022-11-18T20:48:38.3698149Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:38.3698824Z 3ms 310µs (cache) (cache) 1030ms 2ms 42µs 1036ms 2022-11-18T20:48:38.3699197Z 2022-11-18T20:48:38.3884389Z √ Successful POST request 2022-11-18T20:48:38.3890078Z √ Test expected JSON response 2022-11-18T20:48:38.4142525Z 2022-11-18T20:48:38.4143325Z → Model Zoo - Unregister model 2022-11-18T20:48:38.5624990Z DELETE http://localhost:8081/models/densenet161 2022-11-18T20:48:38.5626042Z 200 OK ★ 99ms time ★ 247B↑ 319B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:38.5626678Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 53B 2022-11-18T20:48:38.5627094Z │ { 2022-11-18T20:48:38.5627539Z │ "status": "Model \"densenet161\" unregistered" 2022-11-18T20:48:38.5627905Z │ } 2022-11-18T20:48:38.5644867Z └ 2022-11-18T20:48:38.5645551Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:38.5646064Z 1ms 301µs (cache) (cache) 96ms 1ms 49µs 100ms 2022-11-18T20:48:38.5646254Z 2022-11-18T20:48:38.5753916Z √ Successful DELETE request 2022-11-18T20:48:38.6029695Z 2022-11-18T20:48:38.6030515Z → Model Zoo - Model Metrics 2022-11-18T20:48:38.6584311Z GET http://localhost:8082/metrics 2022-11-18T20:48:38.6585388Z 200 OK ★ 6ms time ★ 233B↑ 2.76kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:38.6585985Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 2.47kB 2022-11-18T20:48:38.6586562Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:48:38.6586993Z │ erence requests. 2022-11-18T20:48:38.6587425Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:48:38.6588675Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:38.6589232Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:48:38.6589647Z │ on="default",} 1.0 2022-11-18T20:48:38.6590105Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:38.6590576Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:48:38.6591016Z │ l_version="default",} 1.0 2022-11-18T20:48:38.6591479Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:38.6592598Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:48:38.6592971Z │ model_version="default",} 1.0 2022-11-18T20:48:38.6593357Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:38.6594010Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:48:38.6594304Z │ sion="default",} 1.0 2022-11-18T20:48:38.6594671Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:38.6595073Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:48:38.6595428Z │ ted",model_version="default",} 1.0 2022-11-18T20:48:38.6595788Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:48:38.6596140Z │ ference duration in microseconds 2022-11-18T20:48:38.6596725Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:48:38.6597379Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:38.6597880Z │ 4252-bfe3-7d6b11352b25",model_name="densenet161",model 2022-11-18T20:48:38.6598329Z │ _version="default",} 1026330.8 2022-11-18T20:48:38.6598669Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:38.6599064Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:38.6599428Z │ ",model_version="default",} 114048.9 2022-11-18T20:48:38.6599796Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:38.6600162Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1_scri 2022-11-18T20:48:38.6600528Z │ pted",model_version="default",} 211321.7 2022-11-18T20:48:38.6600903Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:38.6601269Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1",mod 2022-11-18T20:48:38.6601599Z │ el_version="default",} 2472224.3 2022-11-18T20:48:38.6601960Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:38.6602320Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:38.6602681Z │ _scripted",model_version="default",} 1027094.1 2022-11-18T20:48:38.6603063Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:48:38.6603621Z │ duration in microseconds 2022-11-18T20:48:38.6604222Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:48:38.6604637Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:38.6605027Z │ -bfe3-7d6b11352b25",model_name="densenet161",model_ver 2022-11-18T20:48:38.6605334Z │ sion="default",} 374.7 2022-11-18T20:48:38.6605684Z │ ts_queue_latency_microseconds{uuid="3cb3e369-1ec5-4252 2022-11-18T20:48:38.6606071Z │ -bfe3-7d6b11352b25",model_name="my_text_classifier",mo 2022-11-18T20:48:38.6606383Z │ del_version="default",} 638.7 2022-11-18T20:48:38.6606643Z │ ts_qu 2022-11-18T20:48:38.6606943Z │ (showing 2.05kB/2.47kB) 2022-11-18T20:48:38.6607184Z └ 2022-11-18T20:48:38.6607587Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:38.6608076Z 1ms 298µs (cache) (cache) 2ms 1ms 43µs 6ms 2022-11-18T20:48:38.6608261Z 2022-11-18T20:48:38.6714478Z √ Successful GET request 2022-11-18T20:48:38.6808270Z 2022-11-18T20:48:38.6809307Z Iteration 6/24 2022-11-18T20:48:38.6970401Z 2022-11-18T20:48:38.6970969Z → Model Zoo - Register Model 2022-11-18T20:48:58.5629185Z POST http://localhost:8081/models?url=alexnet.mar&model_name=alexnet&initial_workers=1&synchronous=true 2022-11-18T20:48:58.5630726Z 200 OK ★ 19.8s time ★ 322B↑ 349B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:58.5631460Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 83B 2022-11-18T20:48:58.5631944Z │ { 2022-11-18T20:48:58.5632396Z │ "status": "Model \"alexnet\" Version: 1.0 registered 2022-11-18T20:48:58.5632747Z │ with 1 initial workers" 2022-11-18T20:48:58.5632997Z │ } 2022-11-18T20:48:58.5633171Z └ 2022-11-18T20:48:58.5633596Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:58.5634097Z 1ms 306µs (cache) (cache) 19.8s 3ms 77µs 19.8s 2022-11-18T20:48:58.5634278Z 2022-11-18T20:48:58.5795283Z √ Successful POST request 2022-11-18T20:48:58.6035835Z 2022-11-18T20:48:58.6036914Z → Model Zoo - Inference Model 2022-11-18T20:48:58.9009031Z POST http://localhost:8080/predictions/alexnet 2022-11-18T20:48:58.9010219Z 200 OK ★ 246ms time ★ 111.24kB↑ 403B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:48:58.9010848Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:48:58.9011357Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:48:58.9011770Z └ 2022-11-18T20:48:58.9012339Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 168B 2022-11-18T20:48:58.9012800Z │ { 2022-11-18T20:48:58.9013260Z │ "tabby": 0.318473756313324, 2022-11-18T20:48:58.9013725Z │ "tiger_cat": 0.2579393982887268, 2022-11-18T20:48:58.9014228Z │ "Egyptian_cat": 0.2425483912229538, 2022-11-18T20:48:58.9014712Z │ "lynx": 0.1687936782836914, 2022-11-18T20:48:58.9015165Z │ "tiger": 0.006487966049462557 2022-11-18T20:48:58.9015827Z │ } 2022-11-18T20:48:58.9025458Z └ 2022-11-18T20:48:58.9026086Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:58.9026855Z 2ms 338µs (cache) (cache) 243ms 2ms 48µs 248ms 2022-11-18T20:48:58.9027265Z 2022-11-18T20:48:58.9236798Z √ Successful POST request 2022-11-18T20:48:58.9292631Z √ Test expected JSON response 2022-11-18T20:48:58.9490550Z 2022-11-18T20:48:58.9491336Z → Model Zoo - Unregister model 2022-11-18T20:48:59.1064905Z DELETE http://localhost:8081/models/alexnet 2022-11-18T20:48:59.1068451Z 200 OK ★ 108ms time ★ 243B↑ 315B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:59.1069330Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 49B 2022-11-18T20:48:59.1069780Z │ { 2022-11-18T20:48:59.1070214Z │ "status": "Model \"alexnet\" unregistered" 2022-11-18T20:48:59.1070596Z │ } 2022-11-18T20:48:59.1088643Z └ 2022-11-18T20:48:59.1089269Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:59.1089979Z 2ms 327µs (cache) (cache) 105ms 1ms 62µs 109ms 2022-11-18T20:48:59.1090336Z 2022-11-18T20:48:59.1142312Z √ Successful DELETE request 2022-11-18T20:48:59.8366505Z 2022-11-18T20:48:59.8367437Z → Model Zoo - Model Metrics 2022-11-18T20:48:59.8367841Z GET http://localhost:8082/metrics 2022-11-18T20:48:59.8368365Z 200 OK ★ 6ms time ★ 233B↑ 3.14kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:48:59.8368805Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 2.85kB 2022-11-18T20:48:59.8369258Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:48:59.8369610Z │ erence requests. 2022-11-18T20:48:59.8369975Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:48:59.8370393Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:59.8370841Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:48:59.8371182Z │ on="default",} 1.0 2022-11-18T20:48:59.8371548Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:59.8372117Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:48:59.8372491Z │ l_version="default",} 1.0 2022-11-18T20:48:59.8372871Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:59.8373376Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:48:59.8373760Z │ model_version="default",} 1.0 2022-11-18T20:48:59.8374215Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:59.8374631Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:48:59.8374980Z │ sion="default",} 1.0 2022-11-18T20:48:59.8375377Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:59.8375787Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:48:59.8376115Z │ default",} 1.0 2022-11-18T20:48:59.8376500Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:48:59.8376960Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:48:59.8377395Z │ ted",model_version="default",} 1.0 2022-11-18T20:48:59.8377817Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:48:59.8378215Z │ ference duration in microseconds 2022-11-18T20:48:59.8378613Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:48:59.8379065Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:59.8379508Z │ 4252-bfe3-7d6b11352b25",model_name="densenet161",model 2022-11-18T20:48:59.8379859Z │ _version="default",} 1026330.8 2022-11-18T20:48:59.8380263Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:59.8380693Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:59.8381051Z │ ",model_version="default",} 114048.9 2022-11-18T20:48:59.8381461Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:59.8381898Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1_scri 2022-11-18T20:48:59.8382308Z │ pted",model_version="default",} 211321.7 2022-11-18T20:48:59.8382712Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:59.8383211Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1",mod 2022-11-18T20:48:59.8383592Z │ el_version="default",} 2472224.3 2022-11-18T20:48:59.8383980Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:59.8384416Z │ 4252-bfe3-7d6b11352b25",model_name="alexnet",model_ver 2022-11-18T20:48:59.8384769Z │ sion="default",} 238866.0 2022-11-18T20:48:59.8385146Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:48:59.8385637Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:48:59.8386043Z │ _scripted",model_version="default",} 1027094.1 2022-11-18T20:48:59.8386471Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:48:59.8386916Z │ duration in microseconds 2022-11-18T20:48:59.8387256Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:48:59.8387563Z │ ts_queue_latency_ 2022-11-18T20:48:59.8387812Z │ (showing 2.05kB/2.85kB) 2022-11-18T20:48:59.8388033Z └ 2022-11-18T20:48:59.8388538Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:48:59.8389018Z 1ms 304µs (cache) (cache) 3ms 1ms 45µs 6ms 2022-11-18T20:48:59.8389203Z 2022-11-18T20:48:59.8389353Z √ Successful GET request 2022-11-18T20:48:59.8389500Z 2022-11-18T20:48:59.8389589Z Iteration 7/24 2022-11-18T20:48:59.8389723Z 2022-11-18T20:48:59.8389878Z → Model Zoo - Register Model 2022-11-18T20:49:06.7598136Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/alexnet_scripted.mar&model_name=alexnet_scripted&initial_workers=1&synchronous=true 2022-11-18T20:49:06.7599360Z 200 OK ★ 7.4s time ★ 381B↑ 358B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:06.7599983Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 92B 2022-11-18T20:49:06.7600617Z │ { 2022-11-18T20:49:06.7601078Z │ "status": "Model \"alexnet_scripted\" Version: 1.0 r 2022-11-18T20:49:06.7601528Z │ egistered with 1 initial workers" 2022-11-18T20:49:06.7601892Z │ } 2022-11-18T20:49:06.7621021Z └ 2022-11-18T20:49:06.7621666Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:06.7622327Z 2ms 317µs (cache) (cache) 7.4s 1ms 42µs 7.4s 2022-11-18T20:49:06.7622650Z 2022-11-18T20:49:06.7812148Z √ Successful POST request 2022-11-18T20:49:06.8058624Z 2022-11-18T20:49:06.8059241Z → Model Zoo - Inference Model 2022-11-18T20:49:07.4196147Z POST http://localhost:8080/predictions/alexnet_scripted 2022-11-18T20:49:07.4197416Z 200 OK ★ 564ms time ★ 111.25kB↑ 403B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:49:07.4198149Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:49:07.4198700Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:49:07.4199175Z └ 2022-11-18T20:49:07.4199749Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 168B 2022-11-18T20:49:07.4200190Z │ { 2022-11-18T20:49:07.4200654Z │ "tabby": 0.318473756313324, 2022-11-18T20:49:07.4201141Z │ "tiger_cat": 0.2579393982887268, 2022-11-18T20:49:07.4201640Z │ "Egyptian_cat": 0.2425483912229538, 2022-11-18T20:49:07.4202132Z │ "lynx": 0.1687936782836914, 2022-11-18T20:49:07.4202606Z │ "tiger": 0.006487966049462557 2022-11-18T20:49:07.4203242Z │ } 2022-11-18T20:49:07.4216020Z └ 2022-11-18T20:49:07.4216886Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:07.4217655Z 2ms 339µs (cache) (cache) 560ms 2ms 43µs 565ms 2022-11-18T20:49:07.4218039Z 2022-11-18T20:49:07.4364510Z √ Successful POST request 2022-11-18T20:49:07.4368420Z √ Test expected JSON response 2022-11-18T20:49:07.4623716Z 2022-11-18T20:49:07.4624391Z → Model Zoo - Unregister model 2022-11-18T20:49:07.6360229Z DELETE http://localhost:8081/models/alexnet_scripted 2022-11-18T20:49:07.6361321Z 200 OK ★ 125ms time ★ 252B↑ 324B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:07.6361920Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 58B 2022-11-18T20:49:07.6362376Z │ { 2022-11-18T20:49:07.6364745Z │ "status": "Model \"alexnet_scripted\" unregistered" 2022-11-18T20:49:07.6365386Z │ } 2022-11-18T20:49:07.6381025Z └ 2022-11-18T20:49:07.6381900Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:07.6384101Z 1ms 506µs (cache) (cache) 122ms 1ms 44µs 126ms 2022-11-18T20:49:07.6384436Z 2022-11-18T20:49:07.6505019Z √ Successful DELETE request 2022-11-18T20:49:07.6826175Z 2022-11-18T20:49:07.6826785Z → Model Zoo - Model Metrics 2022-11-18T20:49:07.7368785Z GET http://localhost:8082/metrics 2022-11-18T20:49:07.7369794Z 200 OK ★ 6ms time ★ 233B↑ 3.55kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:07.7370447Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 3.26kB 2022-11-18T20:49:07.7370984Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:49:07.7371428Z │ erence requests. 2022-11-18T20:49:07.7372052Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:49:07.7372566Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:07.7373215Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:49:07.7373870Z │ on="default",} 1.0 2022-11-18T20:49:07.7374362Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:07.7374862Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:49:07.7375307Z │ version="default",} 1.0 2022-11-18T20:49:07.7375757Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:07.7376245Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:49:07.7376684Z │ l_version="default",} 1.0 2022-11-18T20:49:07.7377295Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:07.7377773Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:49:07.7378219Z │ model_version="default",} 1.0 2022-11-18T20:49:07.7378794Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:07.7379467Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:49:07.7379884Z │ sion="default",} 1.0 2022-11-18T20:49:07.7380340Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:07.7380822Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:49:07.7381198Z │ default",} 1.0 2022-11-18T20:49:07.7381650Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:07.7382143Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:49:07.7382594Z │ ted",model_version="default",} 1.0 2022-11-18T20:49:07.7383052Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:49:07.7383522Z │ ference duration in microseconds 2022-11-18T20:49:07.7383986Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:49:07.7384746Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:07.7385256Z │ 4252-bfe3-7d6b11352b25",model_name="densenet161",model 2022-11-18T20:49:07.7385787Z │ _version="default",} 1026330.8 2022-11-18T20:49:07.7386157Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:07.7386548Z │ 4252-bfe3-7d6b11352b25",model_name="alexnet_scripted", 2022-11-18T20:49:07.7386882Z │ model_version="default",} 555322.8 2022-11-18T20:49:07.7387223Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:07.7387610Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:49:07.7387945Z │ ",model_version="default",} 114048.9 2022-11-18T20:49:07.7388293Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:07.7388686Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1_scri 2022-11-18T20:49:07.7389041Z │ pted",model_version="default",} 211321.7 2022-11-18T20:49:07.7389394Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:07.7389868Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1",mod 2022-11-18T20:49:07.7390204Z │ el_version="default",} 2472224.3 2022-11-18T20:49:07.7390558Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:07.7391077Z │ 4252-bfe3-7d6b11352b25",model_name="alexnet",model_ver 2022-11-18T20:49:07.7391406Z │ sion="default",} 238866.0 2022-11-18T20:49:07.7391702Z │ ts_inference_latency_ 2022-11-18T20:49:07.7391962Z │ (showing 2.05kB/3.26kB) 2022-11-18T20:49:07.7392189Z └ 2022-11-18T20:49:07.7392608Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:07.7393101Z 1ms 318µs (cache) (cache) 2ms 2ms 51µs 7ms 2022-11-18T20:49:07.7393283Z 2022-11-18T20:49:07.7535752Z √ Successful GET request 2022-11-18T20:49:07.7600131Z 2022-11-18T20:49:07.7600518Z Iteration 8/24 2022-11-18T20:49:07.7757868Z 2022-11-18T20:49:07.7758394Z → Model Zoo - Register Model 2022-11-18T20:49:11.0457524Z POST http://localhost:8081/models?url=resnet-18.mar&model_name=resnet-18&initial_workers=1&synchronous=true 2022-11-18T20:49:11.0458578Z 200 OK ★ 3.2s time ★ 326B↑ 351B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:11.0459396Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 85B 2022-11-18T20:49:11.0459910Z │ { 2022-11-18T20:49:11.0462461Z │ "status": "Model \"resnet-18\" Version: 1.0 register 2022-11-18T20:49:11.0465306Z │ ed with 1 initial workers" 2022-11-18T20:49:11.0465741Z │ } 2022-11-18T20:49:11.0476087Z └ 2022-11-18T20:49:11.0476842Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:11.0477883Z 1ms 326µs (cache) (cache) 3.2s 1ms 42µs 3.2s 2022-11-18T20:49:11.0478324Z 2022-11-18T20:49:11.0589333Z √ Successful POST request 2022-11-18T20:49:11.0856821Z 2022-11-18T20:49:11.0857696Z → Model Zoo - Inference Model 2022-11-18T20:49:11.3222063Z POST http://localhost:8080/predictions/resnet-18 2022-11-18T20:49:11.3223028Z 200 OK ★ 187ms time ★ 111.24kB↑ 408B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:49:11.3223641Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:49:11.3224154Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:49:11.3224570Z └ 2022-11-18T20:49:11.3225091Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 173B 2022-11-18T20:49:11.3225596Z │ { 2022-11-18T20:49:11.3226069Z │ "tabby": 0.40966352820396423, 2022-11-18T20:49:11.3226545Z │ "tiger_cat": 0.3467044532299042, 2022-11-18T20:49:11.3227056Z │ "Egyptian_cat": 0.13002881407737732, 2022-11-18T20:49:11.3227547Z │ "lynx": 0.02391950413584709, 2022-11-18T20:49:11.3228002Z │ "bucket": 0.011532180942595005 2022-11-18T20:49:11.3228459Z │ } 2022-11-18T20:49:11.3237679Z └ 2022-11-18T20:49:11.3238270Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:11.3238973Z 2ms 300µs (cache) (cache) 184ms 1ms 40µs 188ms 2022-11-18T20:49:11.3239364Z 2022-11-18T20:49:11.3425162Z √ Successful POST request 2022-11-18T20:49:11.3429050Z √ Test expected JSON response 2022-11-18T20:49:11.3686429Z 2022-11-18T20:49:11.3687511Z → Model Zoo - Unregister model 2022-11-18T20:49:11.4977178Z DELETE http://localhost:8081/models/resnet-18 2022-11-18T20:49:11.4978006Z 200 OK ★ 81ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:11.4978611Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:49:11.4979004Z │ { 2022-11-18T20:49:11.4979437Z │ "status": "Model \"resnet-18\" unregistered" 2022-11-18T20:49:11.4979816Z │ } 2022-11-18T20:49:11.4999614Z └ 2022-11-18T20:49:11.5000338Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:11.5001048Z 1ms 316µs (cache) (cache) 78ms 1ms 43µs 81ms 2022-11-18T20:49:11.5001559Z 2022-11-18T20:49:11.5140810Z √ Successful DELETE request 2022-11-18T20:49:11.5406693Z 2022-11-18T20:49:11.5407592Z → Model Zoo - Model Metrics 2022-11-18T20:49:11.5831094Z GET http://localhost:8082/metrics 2022-11-18T20:49:11.5832187Z 200 OK ★ 8ms time ★ 233B↑ 3.94kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:11.5832807Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 3.65kB 2022-11-18T20:49:11.5833335Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:49:11.5833765Z │ erence requests. 2022-11-18T20:49:11.5834205Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:49:11.5834704Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:11.5835193Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:49:11.5836019Z │ on="default",} 1.0 2022-11-18T20:49:11.5836507Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:11.5837007Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:49:11.5837639Z │ ="default",} 1.0 2022-11-18T20:49:11.5838115Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:11.5838615Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:49:11.5845004Z │ version="default",} 1.0 2022-11-18T20:49:11.5845752Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:11.5846288Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:49:11.5846735Z │ l_version="default",} 1.0 2022-11-18T20:49:11.5847185Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:11.5847693Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:49:11.5848142Z │ model_version="default",} 1.0 2022-11-18T20:49:11.5848769Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:11.5849135Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:49:11.5849469Z │ sion="default",} 1.0 2022-11-18T20:49:11.5849824Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:11.5850204Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:49:11.5850475Z │ default",} 1.0 2022-11-18T20:49:11.5850816Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:11.5851192Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:49:11.5851516Z │ ted",model_version="default",} 1.0 2022-11-18T20:49:11.5851889Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:49:11.5852234Z │ ference duration in microseconds 2022-11-18T20:49:11.5852581Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:49:11.5852974Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:11.5853373Z │ 4252-bfe3-7d6b11352b25",model_name="densenet161",model 2022-11-18T20:49:11.5853697Z │ _version="default",} 1026330.8 2022-11-18T20:49:11.5854024Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:11.5854410Z │ 4252-bfe3-7d6b11352b25",model_name="resnet-18",model_v 2022-11-18T20:49:11.5854726Z │ ersion="default",} 180375.6 2022-11-18T20:49:11.5855060Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:11.5855495Z │ 4252-bfe3-7d6b11352b25",model_name="alexnet_scripted", 2022-11-18T20:49:11.5855827Z │ model_version="default",} 555322.8 2022-11-18T20:49:11.5856169Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:11.5856552Z │ 4252-bfe3-7d6b11352b25",model_name="my_text_classifier 2022-11-18T20:49:11.5856891Z │ ",model_version="default",} 114048.9 2022-11-18T20:49:11.5857250Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:11.5857608Z │ 4252-bfe3-7d6b11352b25",model_name="squeezenet1_1_scri 2022-11-18T20:49:11.5857970Z │ pted",model_version="default",} 211321.7 2022-11-18T20:49:11.5858297Z │ ts_inference_latency_microseconds{u 2022-11-18T20:49:11.5858574Z │ (showing 2.05kB/3.65kB) 2022-11-18T20:49:11.5858808Z └ 2022-11-18T20:49:11.5859221Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:11.5859691Z 2ms 350µs (cache) (cache) 5ms 1ms 736µs 10ms 2022-11-18T20:49:11.5859876Z 2022-11-18T20:49:11.5917382Z √ Successful GET request 2022-11-18T20:49:11.6013220Z 2022-11-18T20:49:11.6013727Z Iteration 9/24 2022-11-18T20:49:11.6036576Z 2022-11-18T20:49:11.6037115Z → Model Zoo - Register Model 2022-11-18T20:49:14.9710006Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/resnet-18_scripted.mar&model_name=resnet-18_scripted&initial_workers=1&synchronous=true 2022-11-18T20:49:14.9710968Z 200 OK ★ 3.3s time ★ 385B↑ 360B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:14.9711503Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 94B 2022-11-18T20:49:14.9711857Z │ { 2022-11-18T20:49:14.9712279Z │ "status": "Model \"resnet-18_scripted\" Version: 1.0 2022-11-18T20:49:14.9712933Z │ registered with 1 initial workers" 2022-11-18T20:49:14.9713274Z │ } 2022-11-18T20:49:14.9731726Z └ 2022-11-18T20:49:14.9734714Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:14.9735778Z 2ms 444µs (cache) (cache) 3.3s 1ms 44µs 3.3s 2022-11-18T20:49:14.9738774Z 2022-11-18T20:49:14.9900029Z √ Successful POST request 2022-11-18T20:49:15.0208364Z 2022-11-18T20:49:15.0209192Z → Model Zoo - Inference Model 2022-11-18T20:49:15.2854897Z POST http://localhost:8080/predictions/resnet-18_scripted 2022-11-18T20:49:15.2855859Z 200 OK ★ 228ms time ★ 111.25kB↑ 408B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:49:15.2856577Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:49:15.2857026Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:49:15.2857420Z └ 2022-11-18T20:49:15.2857897Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 173B 2022-11-18T20:49:15.2858335Z │ { 2022-11-18T20:49:15.2858777Z │ "tabby": 0.40966352820396423, 2022-11-18T20:49:15.2859222Z │ "tiger_cat": 0.3467044532299042, 2022-11-18T20:49:15.2859660Z │ "Egyptian_cat": 0.13002881407737732, 2022-11-18T20:49:15.2860116Z │ "lynx": 0.02391950413584709, 2022-11-18T20:49:15.2860550Z │ "bucket": 0.011532180942595005 2022-11-18T20:49:15.2860942Z │ } 2022-11-18T20:49:15.2868423Z └ 2022-11-18T20:49:15.2868978Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:15.2869632Z 3ms 358µs (cache) (cache) 225ms 1ms 43µs 230ms 2022-11-18T20:49:15.2869947Z 2022-11-18T20:49:15.3090087Z √ Successful POST request 2022-11-18T20:49:15.3094070Z √ Test expected JSON response 2022-11-18T20:49:15.3358381Z 2022-11-18T20:49:15.3359088Z → Model Zoo - Unregister model 2022-11-18T20:49:15.4518096Z DELETE http://localhost:8081/models/resnet-18_scripted 2022-11-18T20:49:15.4519033Z 200 OK ★ 83ms time ★ 254B↑ 326B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:15.4519509Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 60B 2022-11-18T20:49:15.4519857Z │ { 2022-11-18T20:49:15.4520325Z │ "status": "Model \"resnet-18_scripted\" unregistered 2022-11-18T20:49:15.4520684Z │ " 2022-11-18T20:49:15.4520933Z │ } 2022-11-18T20:49:15.4530882Z └ 2022-11-18T20:49:15.4531436Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:15.4532004Z 1ms 282µs (cache) (cache) 80ms 1ms 43µs 84ms 2022-11-18T20:49:15.4532252Z 2022-11-18T20:49:15.4687879Z √ Successful DELETE request 2022-11-18T20:49:15.4963507Z 2022-11-18T20:49:15.4964164Z → Model Zoo - Model Metrics 2022-11-18T20:49:15.5520333Z GET http://localhost:8082/metrics 2022-11-18T20:49:15.5521363Z 200 OK ★ 5ms time ★ 233B↑ 4.36kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:15.5522004Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 4.07kB 2022-11-18T20:49:15.5522531Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:49:15.5522967Z │ erence requests. 2022-11-18T20:49:15.5523404Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:49:15.5523907Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:15.5524400Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:49:15.5524820Z │ on="default",} 1.0 2022-11-18T20:49:15.5525277Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:15.5525772Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:49:15.5526162Z │ ="default",} 1.0 2022-11-18T20:49:15.5526622Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:15.5527351Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:49:15.5527782Z │ version="default",} 1.0 2022-11-18T20:49:15.5528317Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:15.5528821Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:49:15.5529270Z │ l_version="default",} 1.0 2022-11-18T20:49:15.5529716Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:15.5530212Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:49:15.5530646Z │ l_version="default",} 1.0 2022-11-18T20:49:15.5531110Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:15.5531586Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:49:15.5532039Z │ model_version="default",} 1.0 2022-11-18T20:49:15.5532506Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:15.5532998Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:49:15.5533406Z │ sion="default",} 1.0 2022-11-18T20:49:15.5533874Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:15.5534371Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:49:15.5534775Z │ default",} 1.0 2022-11-18T20:49:15.5535201Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:15.5535695Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:49:15.5536170Z │ ted",model_version="default",} 1.0 2022-11-18T20:49:15.5536666Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:49:15.5537078Z │ ference duration in microseconds 2022-11-18T20:49:15.5537548Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:49:15.5538042Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:15.5538537Z │ 4252-bfe3-7d6b11352b25",model_name="densenet161",model 2022-11-18T20:49:15.5538954Z │ _version="default",} 1026330.8 2022-11-18T20:49:15.5539418Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:15.5539908Z │ 4252-bfe3-7d6b11352b25",model_name="resnet-18",model_v 2022-11-18T20:49:15.5540398Z │ ersion="default",} 180375.6 2022-11-18T20:49:15.5540856Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:15.5541347Z │ 4252-bfe3-7d6b11352b25",model_name="alexnet_scripted", 2022-11-18T20:49:15.5541785Z │ model_version="default",} 555322.8 2022-11-18T20:49:15.5542253Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:15.5542723Z │ 4252-bfe3-7d6b11352b25",model_name="resnet-18_scripted 2022-11-18T20:49:15.5543163Z │ ",model_version="default",} 221513.4 2022-11-18T20:49:15.5543627Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1 2022-11-18T20:49:15.5544026Z │ (showing 2.05kB/4.07kB) 2022-11-18T20:49:15.5544351Z └ 2022-11-18T20:49:15.5544873Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:15.5545477Z 1ms 309µs (cache) (cache) 2ms 1ms 53µs 5ms 2022-11-18T20:49:15.5545763Z 2022-11-18T20:49:15.5655099Z √ Successful GET request 2022-11-18T20:49:15.5754381Z 2022-11-18T20:49:15.5754756Z Iteration 10/24 2022-11-18T20:49:15.5912123Z 2022-11-18T20:49:15.5912976Z → Model Zoo - Register Model 2022-11-18T20:49:41.2042633Z POST http://localhost:8081/models?url=vgg16.mar&model_name=vgg16&initial_workers=1&synchronous=true 2022-11-18T20:49:41.2043905Z 200 OK ★ 25.5s time ★ 318B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:41.2044613Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:49:41.2045146Z │ { 2022-11-18T20:49:41.2045753Z │ "status": "Model \"vgg16\" Version: 1.0 registered w 2022-11-18T20:49:41.2046272Z │ ith 1 initial workers" 2022-11-18T20:49:41.2046750Z │ } 2022-11-18T20:49:41.2047194Z └ 2022-11-18T20:49:41.2048053Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:41.2048806Z 1ms 1ms (cache) (cache) 25.5s 2ms 43µs 25.5s 2022-11-18T20:49:41.2049185Z 2022-11-18T20:49:41.2174788Z √ Successful POST request 2022-11-18T20:49:41.2299705Z 2022-11-18T20:49:41.2300602Z → Model Zoo - Inference Model 2022-11-18T20:49:41.5441748Z POST http://localhost:8080/predictions/vgg16 2022-11-18T20:49:41.5442974Z 200 OK ★ 279ms time ★ 111.24kB↑ 407B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:49:41.5443724Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:49:41.5444254Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:49:41.5444748Z └ 2022-11-18T20:49:41.5445392Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 172B 2022-11-18T20:49:41.5445860Z │ { 2022-11-18T20:49:41.5446396Z │ "tiger_cat": 0.44697266817092896, 2022-11-18T20:49:41.5446941Z │ "tabby": 0.440879762172699, 2022-11-18T20:49:41.5452922Z │ "Egyptian_cat": 0.05904550850391388, 2022-11-18T20:49:41.5453558Z │ "tiger": 0.020596381276845932, 2022-11-18T20:49:41.5453889Z │ "lynx": 0.009934592060744762 2022-11-18T20:49:41.5454184Z │ } 2022-11-18T20:49:41.5458726Z └ 2022-11-18T20:49:41.5461127Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:41.5461732Z 2ms 283µs (cache) (cache) 276ms 2ms 43µs 281ms 2022-11-18T20:49:41.5461940Z 2022-11-18T20:49:41.5613535Z √ Successful POST request 2022-11-18T20:49:41.5619536Z √ Test expected JSON response 2022-11-18T20:49:41.5877204Z 2022-11-18T20:49:41.5877875Z → Model Zoo - Unregister model 2022-11-18T20:49:41.8342630Z DELETE http://localhost:8081/models/vgg16 2022-11-18T20:49:41.8343804Z 200 OK ★ 197ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:41.8344620Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:49:41.8345057Z │ { 2022-11-18T20:49:41.8345590Z │ "status": "Model \"vgg16\" unregistered" 2022-11-18T20:49:41.8350465Z │ } 2022-11-18T20:49:41.8357549Z └ 2022-11-18T20:49:41.8358176Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:41.8358881Z 2ms 294µs (cache) (cache) 194ms 1ms 40µs 198ms 2022-11-18T20:49:41.8359228Z 2022-11-18T20:49:41.8467942Z √ Successful DELETE request 2022-11-18T20:49:41.8547834Z 2022-11-18T20:49:41.8548397Z → Model Zoo - Model Metrics 2022-11-18T20:49:41.8911420Z GET http://localhost:8082/metrics 2022-11-18T20:49:41.8912542Z 200 OK ★ 4ms time ★ 233B↑ 4.74kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:41.8913200Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 4.45kB 2022-11-18T20:49:41.9001164Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:49:41.9001744Z │ erence requests. 2022-11-18T20:49:41.9002101Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:49:41.9002516Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9002923Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:49:41.9003231Z │ on="default",} 1.0 2022-11-18T20:49:41.9004078Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9004877Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:49:41.9005193Z │ ="default",} 1.0 2022-11-18T20:49:41.9005528Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9005919Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:49:41.9006208Z │ fault",} 1.0 2022-11-18T20:49:41.9006528Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9006924Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:49:41.9007243Z │ version="default",} 1.0 2022-11-18T20:49:41.9007596Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9008157Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:49:41.9008491Z │ l_version="default",} 1.0 2022-11-18T20:49:41.9008852Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9009248Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:49:41.9009552Z │ l_version="default",} 1.0 2022-11-18T20:49:41.9009902Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9010289Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:49:41.9010611Z │ model_version="default",} 1.0 2022-11-18T20:49:41.9010970Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9011346Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:49:41.9011629Z │ sion="default",} 1.0 2022-11-18T20:49:41.9011974Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9012355Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:49:41.9012642Z │ default",} 1.0 2022-11-18T20:49:41.9012946Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:41.9013327Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:49:41.9013679Z │ ted",model_version="default",} 1.0 2022-11-18T20:49:41.9014029Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:49:41.9014377Z │ ference duration in microseconds 2022-11-18T20:49:41.9014735Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:49:41.9015123Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:41.9015488Z │ 4252-bfe3-7d6b11352b25",model_name="densenet161",model 2022-11-18T20:49:41.9015809Z │ _version="default",} 1026330.8 2022-11-18T20:49:41.9016155Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:41.9016515Z │ 4252-bfe3-7d6b11352b25",model_name="resnet-18",model_v 2022-11-18T20:49:41.9016831Z │ ersion="default",} 180375.6 2022-11-18T20:49:41.9017195Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:41.9017551Z │ 4252-bfe3-7d6b11352b25",model_name="vgg16",model_versi 2022-11-18T20:49:41.9017844Z │ on="default",} 273056.1 2022-11-18T20:49:41.9018193Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:41.9018531Z │ 4252-bfe3-7d6b11352b25",model_n 2022-11-18T20:49:41.9018787Z │ (showing 2.05kB/4.45kB) 2022-11-18T20:49:41.9019011Z └ 2022-11-18T20:49:41.9019423Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:41.9019894Z 1ms 285µs (cache) (cache) 1ms 1ms 50µs 4ms 2022-11-18T20:49:41.9020079Z 2022-11-18T20:49:41.9065856Z √ Successful GET request 2022-11-18T20:49:41.9167455Z 2022-11-18T20:49:41.9167824Z Iteration 11/24 2022-11-18T20:49:41.9328603Z 2022-11-18T20:49:41.9329297Z → Model Zoo - Register Model 2022-11-18T20:49:56.0480746Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/vgg16_scripted.mar&model_name=vgg16_scripted&initial_workers=1&synchronous=true 2022-11-18T20:49:56.0482171Z 200 OK ★ 14s time ★ 377B↑ 356B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:56.0483044Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 90B 2022-11-18T20:49:56.0483548Z │ { 2022-11-18T20:49:56.0484054Z │ "status": "Model \"vgg16_scripted\" Version: 1.0 reg 2022-11-18T20:49:56.0484573Z │ istered with 1 initial workers" 2022-11-18T20:49:56.0484968Z │ } 2022-11-18T20:49:56.0502210Z └ 2022-11-18T20:49:56.0502907Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:56.0503649Z 2ms 314µs (cache) (cache) 14s 1ms 51µs 14s 2022-11-18T20:49:56.0504198Z 2022-11-18T20:49:56.0701890Z √ Successful POST request 2022-11-18T20:49:56.0922641Z 2022-11-18T20:49:56.0923529Z → Model Zoo - Inference Model 2022-11-18T20:49:56.4977130Z POST http://localhost:8080/predictions/vgg16_scripted 2022-11-18T20:49:56.4978279Z 200 OK ★ 355ms time ★ 111.25kB↑ 407B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:49:56.4978909Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:49:56.4979413Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:49:56.4979913Z └ 2022-11-18T20:49:56.4980431Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 172B 2022-11-18T20:49:56.4980880Z │ { 2022-11-18T20:49:56.4981323Z │ "tiger_cat": 0.44697266817092896, 2022-11-18T20:49:56.4981803Z │ "tabby": 0.440879762172699, 2022-11-18T20:49:56.4982294Z │ "Egyptian_cat": 0.05904550850391388, 2022-11-18T20:49:56.4982753Z │ "tiger": 0.020596381276845932, 2022-11-18T20:49:56.4984421Z │ "lynx": 0.009934592060744762 2022-11-18T20:49:56.4984846Z │ } 2022-11-18T20:49:56.4993926Z └ 2022-11-18T20:49:56.4994606Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:56.4995344Z 2ms 354µs (cache) (cache) 352ms 2ms 46µs 357ms 2022-11-18T20:49:56.4995739Z 2022-11-18T20:49:56.5157799Z √ Successful POST request 2022-11-18T20:49:56.5161499Z √ Test expected JSON response 2022-11-18T20:49:56.5414848Z 2022-11-18T20:49:56.5415756Z → Model Zoo - Unregister model 2022-11-18T20:49:56.7837472Z DELETE http://localhost:8081/models/vgg16_scripted 2022-11-18T20:49:56.7838546Z 200 OK ★ 193ms time ★ 250B↑ 322B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:56.7839152Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 56B 2022-11-18T20:49:56.7839558Z │ { 2022-11-18T20:49:56.7839982Z │ "status": "Model \"vgg16_scripted\" unregistered" 2022-11-18T20:49:56.7840364Z │ } 2022-11-18T20:49:56.7882466Z └ 2022-11-18T20:49:56.7883171Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:56.7883900Z 2ms 312µs (cache) (cache) 189ms 2ms 39µs 194ms 2022-11-18T20:49:56.7884247Z 2022-11-18T20:49:56.8055257Z √ Successful DELETE request 2022-11-18T20:49:56.8288206Z 2022-11-18T20:49:56.8288952Z → Model Zoo - Model Metrics 2022-11-18T20:49:56.8703812Z GET http://localhost:8082/metrics 2022-11-18T20:49:56.8709940Z 200 OK ★ 4ms time ★ 233B↑ 5.14kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:56.8710577Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 4.85kB 2022-11-18T20:49:56.8711117Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:49:56.8711537Z │ erence requests. 2022-11-18T20:49:56.8711943Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:49:56.8712437Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8712935Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:49:56.8713342Z │ on="default",} 1.0 2022-11-18T20:49:56.8713765Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8714245Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:49:56.8714658Z │ ="default",} 1.0 2022-11-18T20:49:56.8715096Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8715561Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:49:56.8716131Z │ fault",} 1.0 2022-11-18T20:49:56.8716584Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8717067Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:49:56.8717467Z │ version="default",} 1.0 2022-11-18T20:49:56.8717918Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8718402Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:49:56.8718834Z │ l_version="default",} 1.0 2022-11-18T20:49:56.8719271Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8719752Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:49:56.8720169Z │ l_version="default",} 1.0 2022-11-18T20:49:56.8720698Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8721186Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:49:56.8721628Z │ model_version="default",} 1.0 2022-11-18T20:49:56.8722098Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8722561Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:49:56.8722968Z │ sion="default",} 1.0 2022-11-18T20:49:56.8723413Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8723887Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:49:56.8724308Z │ default",} 1.0 2022-11-18T20:49:56.8724785Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8725274Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:49:56.8725653Z │ rsion="default",} 1.0 2022-11-18T20:49:56.8726102Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:56.8726591Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:49:56.8727033Z │ ted",model_version="default",} 1.0 2022-11-18T20:49:56.8727483Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:49:56.8727931Z │ ference duration in microseconds 2022-11-18T20:49:56.8728394Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:49:56.8728885Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:56.8729353Z │ 4252-bfe3-7d6b11352b25",model_name="densenet161",model 2022-11-18T20:49:56.8729780Z │ _version="default",} 1026330.8 2022-11-18T20:49:56.8730233Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:56.8730714Z │ 4252-bfe3-7d6b11352b25",model_name="resnet-18",model_v 2022-11-18T20:49:56.8731108Z │ ersion="default",} 180375.6 2022-11-18T20:49:56.8731560Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:56.8731997Z │ 4252-bfe3-7d6b11352b25",model_nam 2022-11-18T20:49:56.8732370Z │ (showing 2.05kB/4.85kB) 2022-11-18T20:49:56.8732691Z └ 2022-11-18T20:49:56.8733208Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:56.8733802Z 1ms 310µs (cache) (cache) 1ms 1ms 46µs 5ms 2022-11-18T20:49:56.8734074Z 2022-11-18T20:49:56.8862947Z √ Successful GET request 2022-11-18T20:49:56.8917671Z 2022-11-18T20:49:56.8918064Z Iteration 12/24 2022-11-18T20:49:56.9072248Z 2022-11-18T20:49:56.9073019Z → Model Zoo - Register Model 2022-11-18T20:49:59.4865230Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/mnist_v2.mar&model_name=mnist&initial_workers=1&synchronous=true 2022-11-18T20:49:59.4867010Z 200 OK ★ 2.5s time ★ 362B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:59.4867906Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:49:59.4874370Z │ { 2022-11-18T20:49:59.4875151Z │ "status": "Model \"mnist\" Version: 2.0 registered w 2022-11-18T20:49:59.4875680Z │ ith 1 initial workers" 2022-11-18T20:49:59.4876092Z │ } 2022-11-18T20:49:59.4876481Z └ 2022-11-18T20:49:59.4877323Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:59.4877997Z 1ms 340µs (cache) (cache) 2.5s 1ms 44µs 2.5s 2022-11-18T20:49:59.4878347Z 2022-11-18T20:49:59.4878700Z √ Successful POST request 2022-11-18T20:49:59.5148521Z 2022-11-18T20:49:59.5149218Z → Model Zoo - Inference Model 2022-11-18T20:49:59.5998529Z POST http://localhost:8080/predictions/mnist 2022-11-18T20:49:59.5999633Z 200 OK ★ 36ms time ★ 537B↑ 234B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:49:59.6000228Z ┌ ↑ file ★ 272B 2022-11-18T20:49:59.6000676Z │ 2022-11-18T20:49:59.6001130Z └ 2022-11-18T20:49:59.6001617Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 1B 2022-11-18T20:49:59.6002105Z │ 0 2022-11-18T20:49:59.6011485Z └ 2022-11-18T20:49:59.6012101Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:59.6012824Z 2ms 283µs (cache) (cache) 32ms 1ms 38µs 37ms 2022-11-18T20:49:59.6013194Z 2022-11-18T20:49:59.6131783Z √ Successful POST request 2022-11-18T20:49:59.6145983Z √ Test expected TEXT response 2022-11-18T20:49:59.6480562Z 2022-11-18T20:49:59.6481371Z → Model Zoo - Unregister model 2022-11-18T20:49:59.7929979Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:49:59.7930985Z 200 OK ★ 105ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:59.7931610Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:49:59.7932019Z │ { 2022-11-18T20:49:59.7932416Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:49:59.7932785Z │ } 2022-11-18T20:49:59.7948490Z └ 2022-11-18T20:49:59.7949084Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:59.7949741Z 1ms 444µs (cache) (cache) 102ms 1ms 45µs 105ms 2022-11-18T20:49:59.7950058Z 2022-11-18T20:49:59.8021519Z √ Successful DELETE request 2022-11-18T20:49:59.8304271Z 2022-11-18T20:49:59.8304930Z → Model Zoo - Model Metrics 2022-11-18T20:49:59.8835108Z GET http://localhost:8082/metrics 2022-11-18T20:49:59.8836104Z 200 OK ★ 5ms time ★ 233B↑ 5.52kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:49:59.8836668Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 5.23kB 2022-11-18T20:49:59.8837239Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:49:59.8837660Z │ erence requests. 2022-11-18T20:49:59.8838096Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:49:59.8838575Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8839074Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:49:59.8839533Z │ version="default",} 1.0 2022-11-18T20:49:59.8839997Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8840472Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:49:59.8840926Z │ l_version="default",} 1.0 2022-11-18T20:49:59.8841391Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8841906Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:49:59.8842307Z │ l_version="default",} 1.0 2022-11-18T20:49:59.8842759Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8843249Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:49:59.8843688Z │ model_version="default",} 1.0 2022-11-18T20:49:59.8844133Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8844614Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:49:59.8850756Z │ sion="default",} 1.0 2022-11-18T20:49:59.8851307Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8851855Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:49:59.8852699Z │ rsion="default",} 1.0 2022-11-18T20:49:59.8853349Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8853770Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:49:59.8854128Z │ ted",model_version="default",} 1.0 2022-11-18T20:49:59.8854479Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8854867Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:49:59.8855166Z │ on="default",} 1.0 2022-11-18T20:49:59.8855490Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8855870Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:49:59.8856163Z │ ="default",} 1.0 2022-11-18T20:49:59.8856482Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8856963Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:49:59.8857250Z │ fault",} 1.0 2022-11-18T20:49:59.8857586Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8857950Z │ fe3-7d6b11352b25",model_name="mnist",model_version="de 2022-11-18T20:49:59.8858235Z │ fault",} 1.0 2022-11-18T20:49:59.8858563Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:49:59.8858915Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:49:59.8859201Z │ default",} 1.0 2022-11-18T20:49:59.8859544Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:49:59.8859880Z │ ference duration in microseconds 2022-11-18T20:49:59.8860249Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:49:59.8860643Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:59.8861028Z │ 4252-bfe3-7d6b11352b25",model_name="alexnet_scripted", 2022-11-18T20:49:59.8861343Z │ model_version="default",} 555322.8 2022-11-18T20:49:59.8861717Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:49:59.8862081Z │ 4252-bfe3-7d6b11352b25",model_name="resnet-1 2022-11-18T20:49:59.8862365Z │ (showing 2.05kB/5.23kB) 2022-11-18T20:49:59.8862607Z └ 2022-11-18T20:49:59.8863029Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:49:59.8863498Z 1ms 359µs (cache) (cache) 1ms 1ms 45µs 5ms 2022-11-18T20:49:59.8863679Z 2022-11-18T20:49:59.8999801Z √ Successful GET request 2022-11-18T20:49:59.9087403Z 2022-11-18T20:49:59.9087829Z Iteration 13/24 2022-11-18T20:49:59.9249059Z 2022-11-18T20:49:59.9249878Z → Model Zoo - Register Model 2022-11-18T20:50:02.8542180Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/mnist_scripted_v2.mar&model_name=mnist_scripted&initial_workers=1&synchronous=true 2022-11-18T20:50:02.8543474Z 200 OK ★ 2.2s time ★ 380B↑ 356B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:50:02.8546461Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 90B 2022-11-18T20:50:02.8546854Z │ { 2022-11-18T20:50:02.8547312Z │ "status": "Model \"mnist_scripted\" Version: 2.0 reg 2022-11-18T20:50:02.8547797Z │ istered with 1 initial workers" 2022-11-18T20:50:02.8548129Z │ } 2022-11-18T20:50:02.8548409Z └ 2022-11-18T20:50:02.8548944Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:02.8549546Z 1ms 311µs (cache) (cache) 2.2s 1ms 56µs 2.2s 2022-11-18T20:50:02.8549821Z 2022-11-18T20:50:02.8550061Z √ Successful POST request 2022-11-18T20:50:02.8550299Z 2022-11-18T20:50:02.8550534Z → Model Zoo - Inference Model 2022-11-18T20:50:02.8550967Z POST http://localhost:8080/predictions/mnist_scripted 2022-11-18T20:50:02.8551519Z 200 OK ★ 99ms time ★ 546B↑ 234B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:50:02.8551927Z ┌ ↑ file ★ 272B 2022-11-18T20:50:02.8552242Z │ 2022-11-18T20:50:02.8552537Z └ 2022-11-18T20:50:02.8552918Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 1B 2022-11-18T20:50:02.8553279Z │ 0 2022-11-18T20:50:02.8553558Z └ 2022-11-18T20:50:02.8554245Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:02.8554846Z 2ms 480µs (cache) (cache) 96ms 1ms 42µs 101ms 2022-11-18T20:50:02.8555115Z 2022-11-18T20:50:02.8555349Z √ Successful POST request 2022-11-18T20:50:02.8555763Z √ Test expected TEXT response 2022-11-18T20:50:02.8555993Z 2022-11-18T20:50:02.8556230Z → Model Zoo - Unregister model 2022-11-18T20:50:02.8556622Z DELETE http://localhost:8081/models/mnist_scripted 2022-11-18T20:50:02.8557163Z 200 OK ★ 72ms time ★ 250B↑ 322B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:50:02.8557663Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 56B 2022-11-18T20:50:02.8558155Z │ { 2022-11-18T20:50:02.8558560Z │ "status": "Model \"mnist_scripted\" unregistered" 2022-11-18T20:50:02.8558936Z │ } 2022-11-18T20:50:02.8559216Z └ 2022-11-18T20:50:02.8559754Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:02.8560350Z 1ms 277µs (cache) (cache) 69ms 1ms 44µs 72ms 2022-11-18T20:50:02.8560620Z 2022-11-18T20:50:02.8560859Z √ Successful DELETE request 2022-11-18T20:50:02.8561093Z 2022-11-18T20:50:02.8561319Z → Model Zoo - Model Metrics 2022-11-18T20:50:02.8561708Z GET http://localhost:8082/metrics 2022-11-18T20:50:02.8562836Z 200 OK ★ 4ms time ★ 233B↑ 5.93kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:50:02.8563472Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 5.64kB 2022-11-18T20:50:02.8564011Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:50:02.8564450Z │ erence requests. 2022-11-18T20:50:02.8564872Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:50:02.8565421Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8565949Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:50:02.8566402Z │ version="default",} 1.0 2022-11-18T20:50:02.8566864Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8567694Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:50:02.8568140Z │ l_version="default",} 1.0 2022-11-18T20:50:02.8568616Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8569100Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:50:02.8569541Z │ l_version="default",} 1.0 2022-11-18T20:50:02.8570019Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8570534Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:50:02.8570965Z │ model_version="default",} 1.0 2022-11-18T20:50:02.8571464Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8571970Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:50:02.8572391Z │ sion="default",} 1.0 2022-11-18T20:50:02.8573624Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8574198Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:50:02.8574626Z │ rsion="default",} 1.0 2022-11-18T20:50:02.8575093Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8575579Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:50:02.8576036Z │ ted",model_version="default",} 1.0 2022-11-18T20:50:02.8576532Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8577035Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:50:02.8577425Z │ rsion="default",} 1.0 2022-11-18T20:50:02.8577891Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8578509Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:50:02.8578923Z │ on="default",} 1.0 2022-11-18T20:50:02.8579362Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8579882Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:50:02.8580292Z │ ="default",} 1.0 2022-11-18T20:50:02.8580742Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8581223Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:50:02.8581619Z │ fault",} 1.0 2022-11-18T20:50:02.8582064Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8582565Z │ fe3-7d6b11352b25",model_name="mnist",model_version="de 2022-11-18T20:50:02.8582934Z │ fault",} 1.0 2022-11-18T20:50:02.8583378Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:02.8583886Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:50:02.8584287Z │ default",} 1.0 2022-11-18T20:50:02.8584745Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:50:02.8585219Z │ ference duration in microseconds 2022-11-18T20:50:02.8585702Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:50:02.8586206Z │ ts_inference_latency_microseconds{uuid="3cb3e369-1ec5- 2022-11-18T20:50:02.8586724Z │ 4252-bfe3-7d6b11352b25",model_name="alexnet_scripted", 2022-11-18T20:50:02.8587123Z │ mod 2022-11-18T20:50:02.8587478Z │ (showing 2.05kB/5.64kB) 2022-11-18T20:50:02.8587785Z └ 2022-11-18T20:50:02.8588331Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:02.8588966Z 1ms 297µs (cache) (cache) 1ms 1ms 55µs 5ms 2022-11-18T20:50:02.8589244Z 2022-11-18T20:50:02.8748222Z √ Successful GET request 2022-11-18T20:50:03.8611419Z 2022-11-18T20:50:03.8611671Z Iteration 14/24 2022-11-18T20:50:03.8611913Z 2022-11-18T20:50:03.8612412Z → Model Zoo - Register Model 2022-11-18T20:50:17.9566872Z POST http://localhost:8081/models?url=fastrcnn.mar&model_name=fastrcnn&initial_workers=1&synchronous=true 2022-11-18T20:50:17.9568279Z 200 OK ★ 15s time ★ 324B↑ 350B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:50:17.9574174Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 84B 2022-11-18T20:50:17.9574685Z │ { 2022-11-18T20:50:17.9575197Z │ "status": "Model \"fastrcnn\" Version: 1.0 registere 2022-11-18T20:50:17.9575720Z │ d with 1 initial workers" 2022-11-18T20:50:17.9576151Z │ } 2022-11-18T20:50:17.9580608Z └ 2022-11-18T20:50:17.9581236Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:17.9581933Z 1ms 291µs (cache) (cache) 15s 1ms 44µs 15s 2022-11-18T20:50:17.9582307Z 2022-11-18T20:50:17.9774713Z √ Successful POST request 2022-11-18T20:50:18.0050625Z 2022-11-18T20:50:18.0051317Z → Model Zoo - Inference Model 2022-11-18T20:50:22.0038303Z POST http://localhost:8080/predictions/fastrcnn 2022-11-18T20:50:22.0040037Z 200 OK ★ 3.3s time ★ 289.21kB↑ 3.01kB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:50:22.0043092Z ┌ ↑ file ★ 288.94kB 2022-11-18T20:50:22.0043666Z │ (showing 2.05kB/288.94kB) 2022-11-18T20:50:22.0044098Z └ 2022-11-18T20:50:22.0044529Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 2.77kB 2022-11-18T20:50:22.0044899Z │ [ 2022-11-18T20:50:22.0045189Z │ { 2022-11-18T20:50:22.0045516Z │ "person": [ 2022-11-18T20:50:22.0045878Z │ 167.39584350585938, 2022-11-18T20:50:22.0046236Z │ 57.20298385620117, 2022-11-18T20:50:22.0046573Z │ 301.3600158691406, 2022-11-18T20:50:22.0046923Z │ 436.7975769042969 2022-11-18T20:50:22.0047247Z │ ], 2022-11-18T20:50:22.0047596Z │ "score": 0.9995203018188477 2022-11-18T20:50:22.0047931Z │ }, 2022-11-18T20:50:22.0048237Z │ { 2022-11-18T20:50:22.0048556Z │ "person": [ 2022-11-18T20:50:22.0049009Z │ 89.54700469970703, 2022-11-18T20:50:22.0049365Z │ 64.8357162475586, 2022-11-18T20:50:22.0049714Z │ 191.42431640625, 2022-11-18T20:50:22.0050043Z │ 446.76068115234375 2022-11-18T20:50:22.0050385Z │ ], 2022-11-18T20:50:22.0050751Z │ "score": 0.9994968175888062 2022-11-18T20:50:22.0051082Z │ }, 2022-11-18T20:50:22.0051366Z │ { 2022-11-18T20:50:22.0051680Z │ "person": [ 2022-11-18T20:50:22.0052040Z │ 362.37823486328125, 2022-11-18T20:50:22.0052369Z │ 161.84130859375, 2022-11-18T20:50:22.0052709Z │ 515.5022583007812, 2022-11-18T20:50:22.0053056Z │ 385.2898254394531 2022-11-18T20:50:22.0053377Z │ ], 2022-11-18T20:50:22.0053715Z │ "score": 0.9977061748504639 2022-11-18T20:50:22.0054047Z │ }, 2022-11-18T20:50:22.0054657Z │ { 2022-11-18T20:50:22.0054978Z │ "handbag": [ 2022-11-18T20:50:22.0055329Z │ 67.3762435913086, 2022-11-18T20:50:22.0055720Z │ 277.5755920410156, 2022-11-18T20:50:22.0056090Z │ 111.67390441894531, 2022-11-18T20:50:22.0056420Z │ 400.2067565917969 2022-11-18T20:50:22.0056737Z │ ], 2022-11-18T20:50:22.0057095Z │ "score": 0.9924989342689514 2022-11-18T20:50:22.0057416Z │ }, 2022-11-18T20:50:22.0058357Z │ { 2022-11-18T20:50:22.0058740Z │ "handbag": [ 2022-11-18T20:50:22.0059099Z │ 228.6824951171875, 2022-11-18T20:50:22.0059415Z │ 146.00694274902344, 2022-11-18T20:50:22.0059751Z │ 303.55120849609375, 2022-11-18T20:50:22.0060074Z │ 231.08848571777344 2022-11-18T20:50:22.0060372Z │ ], 2022-11-18T20:50:22.0060692Z │ "score": 0.9922404289245605 2022-11-18T20:50:22.0061008Z │ }, 2022-11-18T20:50:22.0061293Z │ { 2022-11-18T20:50:22.0061572Z │ "handbag": [ 2022-11-18T20:50:22.0061896Z │ 379.411376953125, 2022-11-18T20:50:22.0062221Z │ 259.9578552246094, 2022-11-18T20:50:22.0062541Z │ 419.0797424316406, 2022-11-18T20:50:22.0062863Z │ 317.9610290527344 2022-11-18T20:50:22.0063157Z │ ], 2022-11-18T20:50:22.0063483Z │ "score": 0.9898614883422852 2022-11-18T20:50:22.0063790Z │ }, 2022-11-18T20:50:22.0064048Z │ { 2022-11-18T20:50:22.0064360Z │ "person": [ 2022-11-18T20:50:22.0064686Z │ 518.4960327148438, 2022-11-18T20:50:22.0064987Z │ 149.73031616210938, 2022-11-18T20:50:22.0065303Z │ 636.6344604492188, 2022-11-18T20:50:22.0065617Z │ 365.4130554199219 2022-11-18T20:50:22.0065915Z │ ], 2022-11-18T20:50:22.0066226Z │ "score": 0.9821100831031799 2022-11-18T20:50:22.0066528Z │ }, 2022-11-18T20:50:22.0066809Z │ { 2022-11-18T20:50:22.0067080Z │ "bench": [ 2022-11-18T20:50:22.0067407Z │ 269.08184814453125, 2022-11-18T20:50:22.0067732Z │ 217.34210205078125, 2022-11-18T20:50:22.0068055Z │ 423.7701416015625, 2022-11-18T20:50:22.0068353Z │ 390.37847900390625 2022-11-18T20:50:22.0068658Z │ ], 2022-11-18T20:50:22.0068987Z │ "score": 0.9573417901992798 2022-11-18T20:50:22.0069290Z │ }, 2022-11-18T20:50:22.0069546Z │ { 2022-11-18T20:50:22.0069844Z │ "person": [ 2022-11-18T20:50:22.0070379Z │ 539.6298217773438, 2022-11-18T20:50:22.0070695Z │ 157.7587127685547, 2022-11-18T20:50:22.0071017Z │ 616.1531982421875, 2022-11-18T20:50:22.0071333Z │ 253.1112823486328 2022-11-18T20:50:22.0071626Z │ ], 2022-11-18T20:50:22.0071967Z │ "score": 0.8995428085327148 2022-11-18T20:50:22.0072276Z │ }, 2022-11-18T20:50:22.0072557Z │ { 2022-11-18T20:50:22.0072847Z │ "person": [ 2022-11-18T20:50:22.0073144Z │ 477.0624084472656, 2022-11-18T20:50:22.0073470Z │ 147.80886840820312, 2022-11-18T20:50:22.0073788Z │ 610.7772827148438, 2022-11-18T20:50:22.0074083Z │ 296.8367004394531 2022-11-18T20:50:22.0074375Z │ ], 2022-11-18T20:50:22.0074706Z │ "score": 0.8751585483551025 2022-11-18T20:50:22.0075121Z │ }, 2022-11-18T20:50:22.0075377Z │ { 2022-11-18T20:50:22.0075668Z │ "bench": [ 2022-11-18T20:50:22.0075985Z │ 286.0877685546875, 2022-11-18T20:50:22.0076305Z │ 216.53292846679688, 2022-11-18T20:50:22.0076611Z │ 550.7698364257812, 2022-11-18T20:50:22.0076932Z │ 383.1821594238281 2022-11-18T20:50:22.0077226Z │ ], 2022-11-18T20:50:22.0077528Z │ "score": 0.8436436057090759 2022-11-18T20:50:22.0077834Z │ }, 2022-11-18T20:50:22.0078110Z │ { 2022-11-18T20:50:22.0078406Z │ "person": [ 2022-11-18T20:50:22.0078702Z │ 627.4741821289062, 2022-11-18T20:50:22.0079023Z │ 177.05836486816406, 2022-11-18T20:50:22.0079326Z │ 640.0, 2022-11-18T20:50:22.0079638Z │ 247.72146606445312 2022-11-18T20:50:22.0079907Z │ ], 2022-11-18T20:50:22.0080235Z │ "score": 0.8257303833961487 2022-11-18T20:50:22.0080541Z │ }, 2022-11-18T20:50:22.0080798Z │ { 2022-11-18T20:50:22.0081101Z │ "bench": [ 2022-11-18T20:50:22.0081425Z │ 88.76312255859375, 2022-11-18T20:50:22.0081739Z │ 226.1650390625, 2022-11-18T20:50:22.0082035Z │ 563.614013671875, 2022-11-18T20:50:22.0082329Z │ 2022-11-18T20:50:22.0082650Z │ (showing 2.05kB/2.77kB) 2022-11-18T20:50:22.0082941Z └ 2022-11-18T20:50:22.0083418Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:22.0083988Z 3ms 304µs (cache) (cache) 3.3s 1ms 63µs 3.3s 2022-11-18T20:50:22.0084233Z 2022-11-18T20:50:22.0084478Z √ Successful POST request 2022-11-18T20:50:22.0084627Z 2022-11-18T20:50:22.0084862Z → Model Zoo - Unregister model 2022-11-18T20:50:22.0085216Z DELETE http://localhost:8081/models/fastrcnn 2022-11-18T20:50:22.0085713Z 200 OK ★ 101ms time ★ 244B↑ 316B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:50:22.0086173Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 50B 2022-11-18T20:50:22.0086527Z │ { 2022-11-18T20:50:22.0087116Z │ "status": "Model \"fastrcnn\" unregistered" 2022-11-18T20:50:22.0087451Z │ } 2022-11-18T20:50:22.0087707Z └ 2022-11-18T20:50:22.0088196Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:22.0088724Z 1ms 320µs (cache) (cache) 98ms 1ms 44µs 102ms 2022-11-18T20:50:22.0088963Z 2022-11-18T20:50:22.0089205Z √ Successful DELETE request 2022-11-18T20:50:22.0089356Z 2022-11-18T20:50:22.0089567Z → Model Zoo - Model Metrics 2022-11-18T20:50:22.0089901Z GET http://localhost:8082/metrics 2022-11-18T20:50:22.0090336Z 200 OK ★ 5ms time ★ 233B↑ 6.32kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:50:22.0090779Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 6.03kB 2022-11-18T20:50:22.0091230Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:50:22.0091602Z │ erence requests. 2022-11-18T20:50:22.0091983Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:50:22.0092436Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0092894Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:50:22.0093376Z │ version="default",} 1.0 2022-11-18T20:50:22.0093790Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0094250Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:50:22.0094640Z │ l_version="default",} 1.0 2022-11-18T20:50:22.0095056Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0095480Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:50:22.0095862Z │ l_version="default",} 1.0 2022-11-18T20:50:22.0096283Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0096739Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:50:22.0097114Z │ model_version="default",} 1.0 2022-11-18T20:50:22.0097624Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0098077Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:50:22.0098446Z │ sion="default",} 1.0 2022-11-18T20:50:22.0098844Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0099293Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:50:22.0099669Z │ rsion="default",} 1.0 2022-11-18T20:50:22.0100083Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0100509Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:50:22.0100924Z │ ted",model_version="default",} 1.0 2022-11-18T20:50:22.0101352Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0101897Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:50:22.0102249Z │ rsion="default",} 1.0 2022-11-18T20:50:22.0102658Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0103113Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:50:22.0103480Z │ on="default",} 1.0 2022-11-18T20:50:22.0103867Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0104312Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:50:22.0104671Z │ ="default",} 1.0 2022-11-18T20:50:22.0105071Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0105493Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:50:22.0105852Z │ fault",} 1.0 2022-11-18T20:50:22.0106251Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0106690Z │ fe3-7d6b11352b25",model_name="mnist",model_version="de 2022-11-18T20:50:22.0107018Z │ fault",} 1.0 2022-11-18T20:50:22.0107428Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0107869Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:50:22.0108229Z │ default",} 1.0 2022-11-18T20:50:22.0108616Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:22.0109050Z │ fe3-7d6b11352b25",model_name="fastrcnn",model_version= 2022-11-18T20:50:22.0109408Z │ "default",} 1.0 2022-11-18T20:50:22.0109827Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:50:22.0110222Z │ ference duration in microseconds 2022-11-18T20:50:22.0110615Z │ # TYPE ts_inference_latency_microsec 2022-11-18T20:50:22.0110978Z │ (showing 2.05kB/6.03kB) 2022-11-18T20:50:22.0111245Z └ 2022-11-18T20:50:22.0111724Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:22.0112277Z 1ms 309µs (cache) (cache) 1ms 2ms 40µs 5ms 2022-11-18T20:50:22.0112526Z 2022-11-18T20:50:22.0246110Z √ Successful GET request 2022-11-18T20:50:23.0083723Z 2022-11-18T20:50:23.0084385Z Iteration 15/24 2022-11-18T20:50:23.0084663Z 2022-11-18T20:50:23.0085217Z → Model Zoo - Register Model 2022-11-18T20:50:37.4347924Z POST http://localhost:8081/models?url=maskrcnn.mar&model_name=maskrcnn&initial_workers=1&synchronous=true 2022-11-18T20:50:37.4412113Z 200 OK ★ 15.3s time ★ 324B↑ 350B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:50:37.4413457Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 84B 2022-11-18T20:50:37.4414278Z │ { 2022-11-18T20:50:37.4415395Z │ "status": "Model \"maskrcnn\" Version: 1.0 registere 2022-11-18T20:50:37.4415904Z │ d with 1 initial workers" 2022-11-18T20:50:37.4416344Z │ } 2022-11-18T20:50:37.4416728Z └ 2022-11-18T20:50:37.4417330Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:37.4418015Z 1ms 304µs (cache) (cache) 15.3s 1ms 53µs 15.3s 2022-11-18T20:50:37.4418369Z 2022-11-18T20:50:37.4421702Z √ Successful POST request 2022-11-18T20:50:37.4635476Z 2022-11-18T20:50:37.4636265Z → Model Zoo - Inference Model 2022-11-18T20:50:41.3853080Z POST http://localhost:8080/predictions/maskrcnn 2022-11-18T20:50:41.3854290Z 200 OK ★ 3.8s time ★ 289.21kB↑ 2.99kB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:50:41.3854909Z ┌ ↑ file ★ 288.94kB 2022-11-18T20:50:41.3855429Z │ (showing 2.05kB/288.94kB) 2022-11-18T20:50:41.3855877Z └ 2022-11-18T20:50:41.3856505Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 2.76kB 2022-11-18T20:50:41.3856961Z │ [ 2022-11-18T20:50:41.3857345Z │ { 2022-11-18T20:50:41.3857778Z │ "person": [ 2022-11-18T20:50:41.3858238Z │ 169.50631713867188, 2022-11-18T20:50:41.3858669Z │ 49.9854850769043, 2022-11-18T20:50:41.3859125Z │ 300.8945617675781, 2022-11-18T20:50:41.3859574Z │ 442.4545593261719 2022-11-18T20:50:41.3859989Z │ ], 2022-11-18T20:50:41.3860430Z │ "score": 0.9995434880256653 2022-11-18T20:50:41.3860867Z │ }, 2022-11-18T20:50:41.3861273Z │ { 2022-11-18T20:50:41.3861671Z │ "person": [ 2022-11-18T20:50:41.3862150Z │ 90.4118423461914, 2022-11-18T20:50:41.3862602Z │ 66.79048919677734, 2022-11-18T20:50:41.3863055Z │ 194.1930389404297, 2022-11-18T20:50:41.3863481Z │ 437.2777099609375 2022-11-18T20:50:41.3863915Z │ ], 2022-11-18T20:50:41.3864378Z │ "score": 0.9993956089019775 2022-11-18T20:50:41.3864789Z │ }, 2022-11-18T20:50:41.3865204Z │ { 2022-11-18T20:50:41.3865628Z │ "person": [ 2022-11-18T20:50:41.3866080Z │ 362.3172912597656, 2022-11-18T20:50:41.3866510Z │ 158.1734619140625, 2022-11-18T20:50:41.3866961Z │ 521.2589721679688, 2022-11-18T20:50:41.3867409Z │ 385.72271728515625 2022-11-18T20:50:41.3867807Z │ ], 2022-11-18T20:50:41.3868291Z │ "score": 0.9952279329299927 2022-11-18T20:50:41.3868738Z │ }, 2022-11-18T20:50:41.3869141Z │ { 2022-11-18T20:50:41.3869556Z │ "handbag": [ 2022-11-18T20:50:41.3869977Z │ 68.57540893554688, 2022-11-18T20:50:41.3870425Z │ 279.3182373046875, 2022-11-18T20:50:41.3870862Z │ 111.15328979492188, 2022-11-18T20:50:41.3871276Z │ 400.9166259765625 2022-11-18T20:50:41.3871691Z │ ], 2022-11-18T20:50:41.3872135Z │ "score": 0.9938817024230957 2022-11-18T20:50:41.3872570Z │ }, 2022-11-18T20:50:41.3872948Z │ { 2022-11-18T20:50:41.3873359Z │ "person": [ 2022-11-18T20:50:41.3873785Z │ 474.015625, 2022-11-18T20:50:41.3874194Z │ 147.3480224609375, 2022-11-18T20:50:41.3874632Z │ 638.1206665039062, 2022-11-18T20:50:41.3875068Z │ 364.6508483886719 2022-11-18T20:50:41.3875477Z │ ], 2022-11-18T20:50:41.3875894Z │ "score": 0.9897462129592896 2022-11-18T20:50:41.3876312Z │ }, 2022-11-18T20:50:41.3876710Z │ { 2022-11-18T20:50:41.3877098Z │ "handbag": [ 2022-11-18T20:50:41.3877550Z │ 225.59584045410156, 2022-11-18T20:50:41.3877993Z │ 142.90003967285156, 2022-11-18T20:50:41.3878432Z │ 302.48638916015625, 2022-11-18T20:50:41.3878859Z │ 230.32838439941406 2022-11-18T20:50:41.3879271Z │ ], 2022-11-18T20:50:41.3879711Z │ "score": 0.9891213774681091 2022-11-18T20:50:41.3880109Z │ }, 2022-11-18T20:50:41.3880506Z │ { 2022-11-18T20:50:41.3881125Z │ "handbag": [ 2022-11-18T20:50:41.3881611Z │ 380.26043701171875, 2022-11-18T20:50:41.3882068Z │ 259.20135498046875, 2022-11-18T20:50:41.3883945Z │ 419.5365295410156, 2022-11-18T20:50:41.3884749Z │ 318.27728271484375 2022-11-18T20:50:41.3887495Z │ ], 2022-11-18T20:50:41.3887963Z │ "score": 0.9688040614128113 2022-11-18T20:50:41.3888391Z │ }, 2022-11-18T20:50:41.3888771Z │ { 2022-11-18T20:50:41.3889191Z │ "bench": [ 2022-11-18T20:50:41.3889641Z │ 273.48565673828125, 2022-11-18T20:50:41.3890091Z │ 217.48858642578125, 2022-11-18T20:50:41.3890511Z │ 441.06439208984375, 2022-11-18T20:50:41.3890950Z │ 396.241455078125 2022-11-18T20:50:41.3891585Z │ ], 2022-11-18T20:50:41.3892512Z │ "score": 0.9617533683776855 2022-11-18T20:50:41.3893163Z │ }, 2022-11-18T20:50:41.3894048Z │ { 2022-11-18T20:50:41.3894479Z │ "person": [ 2022-11-18T20:50:41.3894923Z │ 541.289794921875, 2022-11-18T20:50:41.3895372Z │ 156.66123962402344, 2022-11-18T20:50:41.3895813Z │ 619.938232421875, 2022-11-18T20:50:41.3896237Z │ 249.45277404785156 2022-11-18T20:50:41.3896657Z │ ], 2022-11-18T20:50:41.3897102Z │ "score": 0.8177030682563782 2022-11-18T20:50:41.3897529Z │ }, 2022-11-18T20:50:41.3897905Z │ { 2022-11-18T20:50:41.3898322Z │ "person": [ 2022-11-18T20:50:41.3898767Z │ 362.9620666503906, 2022-11-18T20:50:41.3899190Z │ 163.89923095703125, 2022-11-18T20:50:41.3899638Z │ 500.7699279785156, 2022-11-18T20:50:41.3900092Z │ 293.9123840332031 2022-11-18T20:50:41.3900509Z │ ], 2022-11-18T20:50:41.3900997Z │ "score": 0.8016929626464844 2022-11-18T20:50:41.3901454Z │ }, 2022-11-18T20:50:41.3901835Z │ { 2022-11-18T20:50:41.3902253Z │ "chair": [ 2022-11-18T20:50:41.3902701Z │ 455.20855712890625, 2022-11-18T20:50:41.3903130Z │ 207.54006958007812, 2022-11-18T20:50:41.3903595Z │ 491.0853271484375, 2022-11-18T20:50:41.3904046Z │ 274.64752197265625 2022-11-18T20:50:41.3904464Z │ ], 2022-11-18T20:50:41.3904895Z │ "score": 0.7758282423019409 2022-11-18T20:50:41.3905327Z │ }, 2022-11-18T20:50:41.3905727Z │ { 2022-11-18T20:50:41.3906123Z │ "person": [ 2022-11-18T20:50:41.3906573Z │ 549.1541137695312, 2022-11-18T20:50:41.3907017Z │ 177.42166137695312, 2022-11-18T20:50:41.3907444Z │ 640.0, 2022-11-18T20:50:41.3907861Z │ 364.5395202636719 2022-11-18T20:50:41.3908281Z │ ], 2022-11-18T20:50:41.3908728Z │ "score": 0.7176409959793091 2022-11-18T20:50:41.3909139Z │ }, 2022-11-18T20:50:41.3909544Z │ { 2022-11-18T20:50:41.3909982Z │ "person": [ 2022-11-18T20:50:41.3910425Z │ 626.230712890625, 2022-11-18T20:50:41.3910858Z │ 178.65342712402344, 2022-11-18T20:50:41.3911299Z │ 640.0, 2022-11-18T20:50:41.3911721Z │ 246. 2022-11-18T20:50:41.3912163Z │ (showing 2.05kB/2.76kB) 2022-11-18T20:50:41.3912694Z └ 2022-11-18T20:50:41.3913314Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:41.3914054Z 2ms 865µs (cache) (cache) 3.8s 1ms 60µs 3.8s 2022-11-18T20:50:41.3914430Z 2022-11-18T20:50:41.3923795Z √ Successful POST request 2022-11-18T20:50:41.4149149Z 2022-11-18T20:50:41.4150062Z → Model Zoo - Unregister model 2022-11-18T20:50:41.5712304Z DELETE http://localhost:8081/models/maskrcnn 2022-11-18T20:50:41.5713387Z 200 OK ★ 108ms time ★ 244B↑ 316B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:50:41.5714016Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 50B 2022-11-18T20:50:41.5714443Z │ { 2022-11-18T20:50:41.5714862Z │ "status": "Model \"maskrcnn\" unregistered" 2022-11-18T20:50:41.5715236Z │ } 2022-11-18T20:50:41.5729370Z └ 2022-11-18T20:50:41.5730187Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:41.5730857Z 1ms 304µs (cache) (cache) 105ms 1ms 59µs 109ms 2022-11-18T20:50:41.5731165Z 2022-11-18T20:50:41.5764944Z √ Successful DELETE request 2022-11-18T20:50:41.6013105Z 2022-11-18T20:50:41.6013876Z → Model Zoo - Model Metrics 2022-11-18T20:50:41.6567899Z GET http://localhost:8082/metrics 2022-11-18T20:50:41.6569028Z 200 OK ★ 7ms time ★ 233B↑ 6.71kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:50:41.6569718Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 6.41kB 2022-11-18T20:50:41.6570342Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:50:41.6570834Z │ erence requests. 2022-11-18T20:50:41.6571632Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:50:41.6572205Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6572768Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:50:41.6573282Z │ version="default",} 1.0 2022-11-18T20:50:41.6573822Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6574395Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:50:41.6574952Z │ l_version="default",} 1.0 2022-11-18T20:50:41.6575416Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6575903Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:50:41.6576329Z │ l_version="default",} 1.0 2022-11-18T20:50:41.6576769Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6577262Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:50:41.6577700Z │ model_version="default",} 1.0 2022-11-18T20:50:41.6578177Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6578645Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:50:41.6579077Z │ sion="default",} 1.0 2022-11-18T20:50:41.6579545Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6580015Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:50:41.6580416Z │ "default",} 1.0 2022-11-18T20:50:41.6580858Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6581345Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:50:41.6581737Z │ rsion="default",} 1.0 2022-11-18T20:50:41.6582192Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6582677Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:50:41.6583128Z │ ted",model_version="default",} 1.0 2022-11-18T20:50:41.6583579Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6584069Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:50:41.6584503Z │ rsion="default",} 1.0 2022-11-18T20:50:41.6585068Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6585568Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:50:41.6585882Z │ on="default",} 1.0 2022-11-18T20:50:41.6588813Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6589221Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:50:41.6589523Z │ ="default",} 1.0 2022-11-18T20:50:41.6589839Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6590204Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:50:41.6590492Z │ fault",} 1.0 2022-11-18T20:50:41.6590805Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6591187Z │ fe3-7d6b11352b25",model_name="mnist",model_version="de 2022-11-18T20:50:41.6591591Z │ fault",} 1.0 2022-11-18T20:50:41.6591922Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6592282Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:50:41.6592568Z │ default",} 1.0 2022-11-18T20:50:41.6592914Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:50:41.6593279Z │ fe3-7d6b11352b25",model_name="fastrcnn",model_version= 2022-11-18T20:50:41.6593573Z │ "default",} 1.0 2022-11-18T20:50:41.6593842Z │ (showing 2.05kB/6.41kB) 2022-11-18T20:50:41.6594056Z └ 2022-11-18T20:50:41.6594482Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:50:41.6594977Z 1ms 331µs (cache) (cache) 3ms 2ms 59µs 7ms 2022-11-18T20:50:41.6595162Z 2022-11-18T20:50:41.6684286Z √ Successful GET request 2022-11-18T20:50:41.6792430Z 2022-11-18T20:50:41.6792811Z Iteration 16/24 2022-11-18T20:50:41.6951343Z 2022-11-18T20:50:41.6952018Z → Model Zoo - Register Model 2022-11-18T20:51:01.3825281Z POST http://localhost:8081/models?url=fcn_resnet_101.mar&model_name=fcn_resnet_101&initial_workers=1&synchronous=true 2022-11-18T20:51:01.3831916Z 200 OK ★ 19.6s time ★ 336B↑ 356B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:01.3832744Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 90B 2022-11-18T20:51:01.3833266Z │ { 2022-11-18T20:51:01.3833818Z │ "status": "Model \"fcn_resnet_101\" Version: 1.0 reg 2022-11-18T20:51:01.3834352Z │ istered with 1 initial workers" 2022-11-18T20:51:01.3834774Z │ } 2022-11-18T20:51:01.3852095Z └ 2022-11-18T20:51:01.3852895Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:01.3853662Z 1ms 297µs (cache) (cache) 19.6s 1ms 44µs 19.6s 2022-11-18T20:51:01.3854039Z 2022-11-18T20:51:01.3980618Z √ Successful POST request 2022-11-18T20:51:01.4259572Z 2022-11-18T20:51:01.4260466Z → Model Zoo - Inference Model 2022-11-18T20:51:02.4530194Z POST http://localhost:8080/predictions/fcn_resnet_101 2022-11-18T20:51:02.4533386Z 200 OK ★ 970ms time ★ 289.22kB↑ 2.47MB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:51:02.4534668Z ┌ ↑ file ★ 288.94kB 2022-11-18T20:51:02.4535892Z │ (showing 2.05kB/288.94kB) 2022-11-18T20:51:02.4537347Z └ 2022-11-18T20:51:02.4537819Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 2.47MB 2022-11-18T20:51:02.4538203Z │ [ 2022-11-18T20:51:02.4538521Z │ [ 2022-11-18T20:51:02.4538819Z │ [ 2022-11-18T20:51:02.4539091Z │ 0.0, 2022-11-18T20:51:02.4539371Z │ 0.9993855953216553 2022-11-18T20:51:02.4539722Z │ ], 2022-11-18T20:51:02.4539947Z │ [ 2022-11-18T20:51:02.4540149Z │ 0.0, 2022-11-18T20:51:02.4540395Z │ 0.9993855953216553 2022-11-18T20:51:02.4540630Z │ ], 2022-11-18T20:51:02.4540828Z │ [ 2022-11-18T20:51:02.4541050Z │ 0.0, 2022-11-18T20:51:02.4541331Z │ 0.9993855953216553 2022-11-18T20:51:02.4541542Z │ ], 2022-11-18T20:51:02.4541753Z │ [ 2022-11-18T20:51:02.4541974Z │ 0.0, 2022-11-18T20:51:02.4542200Z │ 0.9993855953216553 2022-11-18T20:51:02.4542435Z │ ], 2022-11-18T20:51:02.4542886Z │ [ 2022-11-18T20:51:02.4543092Z │ 0.0, 2022-11-18T20:51:02.4543337Z │ 0.9993864297866821 2022-11-18T20:51:02.4543568Z │ ], 2022-11-18T20:51:02.4543764Z │ [ 2022-11-18T20:51:02.4543981Z │ 0.0, 2022-11-18T20:51:02.4544233Z │ 0.9993854761123657 2022-11-18T20:51:02.4544453Z │ ], 2022-11-18T20:51:02.4544673Z │ [ 2022-11-18T20:51:02.4544891Z │ 0.0, 2022-11-18T20:51:02.4545116Z │ 0.9993811845779419 2022-11-18T20:51:02.4545345Z │ ], 2022-11-18T20:51:02.4545560Z │ [ 2022-11-18T20:51:02.4545779Z │ 0.0, 2022-11-18T20:51:02.4546008Z │ 0.9993742108345032 2022-11-18T20:51:02.4546235Z │ ], 2022-11-18T20:51:02.4546546Z │ [ 2022-11-18T20:51:02.4546743Z │ 0.0, 2022-11-18T20:51:02.4546993Z │ 0.9993641972541809 2022-11-18T20:51:02.4547225Z │ ], 2022-11-18T20:51:02.4547427Z │ [ 2022-11-18T20:51:02.4547644Z │ 0.0, 2022-11-18T20:51:02.4547895Z │ 0.9993515610694885 2022-11-18T20:51:02.4548112Z │ ], 2022-11-18T20:51:02.4548331Z │ [ 2022-11-18T20:51:02.4548549Z │ 0.0, 2022-11-18T20:51:02.4548783Z │ 0.9993364214897156 2022-11-18T20:51:02.4549013Z │ ], 2022-11-18T20:51:02.4549226Z │ [ 2022-11-18T20:51:02.4549426Z │ 0.0, 2022-11-18T20:51:02.4549673Z │ 0.9993187189102173 2022-11-18T20:51:02.4549907Z │ ], 2022-11-18T20:51:02.4550109Z │ [ 2022-11-18T20:51:02.4550325Z │ 0.0, 2022-11-18T20:51:02.4550575Z │ 0.9992934465408325 2022-11-18T20:51:02.4550788Z │ ], 2022-11-18T20:51:02.4551000Z │ [ 2022-11-18T20:51:02.4551219Z │ 0.0, 2022-11-18T20:51:02.4551445Z │ 0.9992607235908508 2022-11-18T20:51:02.4551684Z │ ], 2022-11-18T20:51:02.4551896Z │ [ 2022-11-18T20:51:02.4552093Z │ 0.0, 2022-11-18T20:51:02.4552340Z │ 0.9992249011993408 2022-11-18T20:51:02.4552577Z │ ], 2022-11-18T20:51:02.4552777Z │ [ 2022-11-18T20:51:02.4552998Z │ 0.0, 2022-11-18T20:51:02.4553247Z │ 0.9991866946220398 2022-11-18T20:51:02.4553457Z │ ], 2022-11-18T20:51:02.4553673Z │ [ 2022-11-18T20:51:02.4553890Z │ 0.0, 2022-11-18T20:51:02.4554114Z │ 0.9991452693939209 2022-11-18T20:51:02.4554344Z │ ], 2022-11-18T20:51:02.4554559Z │ [ 2022-11-18T20:51:02.4554756Z │ 0.0, 2022-11-18T20:51:02.4554999Z │ 0.9991005659103394 2022-11-18T20:51:02.4555228Z │ ], 2022-11-18T20:51:02.4555440Z │ [ 2022-11-18T20:51:02.4555635Z │ 0.0, 2022-11-18T20:51:02.4555876Z │ 0.9990523457527161 2022-11-18T20:51:02.4556107Z │ ], 2022-11-18T20:51:02.4556736Z │ [ 2022-11-18T20:51:02.4556981Z │ 0.0, 2022-11-18T20:51:02.4557231Z │ 0.9990004897117615 2022-11-18T20:51:02.4557445Z │ ], 2022-11-18T20:51:02.4557658Z │ [ 2022-11-18T20:51:02.4557877Z │ 0.0, 2022-11-18T20:51:02.4558112Z │ 0.998968243598938 2022-11-18T20:51:02.4558350Z │ ], 2022-11-18T20:51:02.4558565Z │ [ 2022-11-18T20:51:02.4558769Z │ 0.0, 2022-11-18T20:51:02.4559015Z │ 0.9989573955535889 2022-11-18T20:51:02.4559250Z │ ], 2022-11-18T20:51:02.4559448Z │ [ 2022-11-18T20:51:02.4559674Z │ 0.0, 2022-11-18T20:51:02.4559922Z │ 0.9989456534385681 2022-11-18T20:51:02.4560134Z │ ], 2022-11-18T20:51:02.4560353Z │ [ 2022-11-18T20:51:02.4560571Z │ 0.0, 2022-11-18T20:51:02.4560798Z │ 0.9989331364631653 2022-11-18T20:51:02.4561028Z │ ], 2022-11-18T20:51:02.4561243Z │ [ 2022-11-18T20:51:02.4561447Z │ 0.0, 2022-11-18T20:51:02.4561686Z │ 0.9989200830459595 2022-11-18T20:51:02.4561926Z │ ], 2022-11-18T20:51:02.4562118Z │ [ 2022-11-18T20:51:02.4562335Z │ 0.0, 2022-11-18T20:51:02.4562586Z │ 0.9989060163497925 2022-11-18T20:51:02.4562798Z │ ], 2022-11-18T20:51:02.4563132Z │ [ 2022-11-18T20:51:02.4563364Z │ 0.0, 2022-11-18T20:51:02.4563588Z │ 0.998891294002533 2022-11-18T20:51:02.4563817Z │ ], 2022-11-18T20:51:02.4564038Z │ [ 2022-11-18T20:51:02.4564234Z │ 0.0, 2022-11-18T20:51:02.4564482Z │ 0.9988754391670227 2022-11-18T20:51:02.4564713Z │ ], 2022-11-18T20:51:02.4564908Z │ [ 2022-11-18T20:51:02.4565129Z │ 0.0, 2022-11-18T20:51:02.4565373Z │ 0.9988815188407898 2022-11-18T20:51:02.4565606Z │ ], 2022-11-18T20:51:02.4565799Z │ [ 2022-11-18T20:51:02.4566017Z │ 0.0, 2022-11-18T20:51:02.4566263Z │ 0.9989089965820312 2022-11-18T20:51:02.4566475Z │ ], 2022-11-18T20:51:02.4566694Z │ [ 2022-11-18T20:51:02.4566989Z │ 0.0, 2022-11-18T20:51:02.4567215Z │ 0.9989352822303772 2022-11-18T20:51:02.4567517Z │ ], 2022-11-18T20:51:02.4567734Z │ [ 2022-11-18T20:51:02.4567930Z │ 0.0, 2022-11-18T20:51:02.4568182Z │ 0.9989607334136963 2022-11-18T20:51:02.4568412Z │ ], 2022-11-18T20:51:02.4568607Z │ [ 2022-11-18T20:51:02.4568829Z │ 0.0, 2022-11-18T20:51:02.4569073Z │ 0.9989848732948303 2022-11-18T20:51:02.4569282Z │ ], 2022-11-18T20:51:02.4569495Z │ [ 2022-11-18T20:51:02.4569714Z │ 0.0, 2022-11-18T20:51:02.4569942Z │ 0.9990076422691345 2022-11-18T20:51:02.4570174Z │ ], 2022-11-18T20:51:02.4570391Z │ [ 2022-11-18T20:51:02.4570587Z │ 0.0, 2022-11-18T20:51:02.4570898Z │ 0.9990293979644775 2022-11-18T20:51:02.4571126Z │ ], 2022-11-18T20:51:02.4571318Z │ [ 2022-11-18T20:51:02.4571535Z │ 0.0, 2022-11-18T20:51:02.4571781Z │ 0.999049723148346 2022-11-18T20:51:02.4572003Z │ ], 2022-11-18T20:51:02.4572216Z │ [ 2022-11-18T20:51:02.4572433Z │ 0.0, 2022-11-18T20:51:02.4572660Z │ 0.999051034450531 2022-11-18T20:51:02.4572887Z │ ], 2022-11-18T20:51:02.4573102Z │ [ 2022-11-18T20:51:02.4573309Z │ 0.0, 2022-11-18T20:51:02.4573556Z │ 0.9990313053131104 2022-11-18T20:51:02.4573789Z │ ], 2022-11-18T20:51:02.4573982Z │ [ 2022-11-18T20:51:02.4574201Z │ 0.0, 2022-11-18T20:51:02.4574444Z │ 0.9990096092224121 2022-11-18T20:51:02.4574653Z │ ], 2022-11-18T20:51:02.4574869Z │ [ 2022-11-18T20:51:02.4575090Z │ 0.0, 2022-11-18T20:51:02.4575336Z │ 0.9989851117134094 2022-11-18T20:51:02.4575542Z │ ], 2022-11-18T20:51:02.4575753Z │ [ 2022-11-18T20:51:02.4575974Z │ 0.0, 2022-11-18T20:51:02.4576197Z │ 0.9989587068557739 2022-11-18T20:51:02.4576423Z │ ], 2022-11-18T20:51:02.4576647Z │ [ 2022-11-18T20:51:02.4576856Z │ 0.0, 2022-11-18T20:51:02.4577098Z │ 0.998929798603 2022-11-18T20:51:02.4577363Z │ (showing 2.05kB/2.47MB) 2022-11-18T20:51:02.4577576Z └ 2022-11-18T20:51:02.4578011Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:02.4578525Z 2ms 976µs (cache) (cache) 956ms 12ms 1ms 973ms 2022-11-18T20:51:02.4578715Z 2022-11-18T20:51:03.4376668Z √ Successful POST request 2022-11-18T20:51:03.4376901Z 2022-11-18T20:51:03.4377117Z → Model Zoo - Unregister model 2022-11-18T20:51:03.4377467Z DELETE http://localhost:8081/models/fcn_resnet_101 2022-11-18T20:51:03.4377991Z 200 OK ★ 100ms time ★ 250B↑ 322B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:03.4378462Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 56B 2022-11-18T20:51:03.4378764Z │ { 2022-11-18T20:51:03.4379134Z │ "status": "Model \"fcn_resnet_101\" unregistered" 2022-11-18T20:51:03.4379441Z │ } 2022-11-18T20:51:03.4379678Z └ 2022-11-18T20:51:03.4380154Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:03.4380940Z 2ms 358µs (cache) (cache) 97ms 1ms 67µs 101ms 2022-11-18T20:51:03.4381171Z 2022-11-18T20:51:03.4381379Z √ Successful DELETE request 2022-11-18T20:51:03.4381531Z 2022-11-18T20:51:03.4381697Z → Model Zoo - Model Metrics 2022-11-18T20:51:03.4382005Z GET http://localhost:8082/metrics 2022-11-18T20:51:03.4382471Z 200 OK ★ 5ms time ★ 233B↑ 7.11kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:03.4382886Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 6.82kB 2022-11-18T20:51:03.4383325Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:51:03.4383678Z │ erence requests. 2022-11-18T20:51:03.4384023Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:51:03.4384460Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4385037Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:51:03.4385395Z │ version="default",} 1.0 2022-11-18T20:51:03.4385784Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4386242Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:51:03.4386615Z │ l_version="default",} 1.0 2022-11-18T20:51:03.4386997Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4387433Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:51:03.4387792Z │ l_version="default",} 1.0 2022-11-18T20:51:03.4388173Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4388614Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:51:03.4388990Z │ model_version="default",} 1.0 2022-11-18T20:51:03.4389399Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4389809Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:51:03.4390166Z │ sion="default",} 1.0 2022-11-18T20:51:03.4390563Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4390971Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:51:03.4391310Z │ "default",} 1.0 2022-11-18T20:51:03.4391698Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4392114Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:51:03.4392462Z │ rsion="default",} 1.0 2022-11-18T20:51:03.4392854Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4393262Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:51:03.4393643Z │ ted",model_version="default",} 1.0 2022-11-18T20:51:03.4394056Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4394489Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:51:03.4394816Z │ rsion="default",} 1.0 2022-11-18T20:51:03.4395222Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4395652Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:51:03.4395968Z │ on="default",} 1.0 2022-11-18T20:51:03.4396369Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4396795Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:51:03.4397106Z │ ="default",} 1.0 2022-11-18T20:51:03.4397487Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4397914Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:51:03.4398238Z │ fault",} 1.0 2022-11-18T20:51:03.4398592Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4399014Z │ fe3-7d6b11352b25",model_name="fcn_resnet_101",model_ve 2022-11-18T20:51:03.4399351Z │ rsion="default",} 1.0 2022-11-18T20:51:03.4399725Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4400147Z │ fe3-7d6b11352b25",model_name="mnist",model_version="de 2022-11-18T20:51:03.4400475Z │ fault",} 1.0 2022-11-18T20:51:03.4400829Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:03.4401355Z │ fe3-7d6b11352b25",model_name="alexnet",model_version=" 2022-11-18T20:51:03.4401676Z │ default" 2022-11-18T20:51:03.4401967Z │ (showing 2.05kB/6.82kB) 2022-11-18T20:51:03.4402203Z └ 2022-11-18T20:51:03.4402672Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:03.4403221Z 1ms 308µs (cache) (cache) 3ms 1ms 53µs 6ms 2022-11-18T20:51:03.4403431Z 2022-11-18T20:51:03.4403583Z √ Successful GET request 2022-11-18T20:51:03.4403751Z 2022-11-18T20:51:03.4403850Z Iteration 17/24 2022-11-18T20:51:03.4404001Z 2022-11-18T20:51:03.4404174Z → Model Zoo - Register Model 2022-11-18T20:51:10.1839292Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/fcn_resnet_101_scripted.mar&model_name=fcn_resnet_101_scripted&initial_workers=1&synchronous=true 2022-11-18T20:51:10.1840903Z 200 OK ★ 7.1s time ★ 395B↑ 365B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:10.1841435Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 99B 2022-11-18T20:51:10.1841786Z │ { 2022-11-18T20:51:10.1842191Z │ "status": "Model \"fcn_resnet_101_scripted\" Version 2022-11-18T20:51:10.1842610Z │ : 1.0 registered with 1 initial workers" 2022-11-18T20:51:10.1842940Z │ } 2022-11-18T20:51:10.1851420Z └ 2022-11-18T20:51:10.1851975Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:10.1852552Z 1ms 280µs (cache) (cache) 7.1s 1ms 43µs 7.1s 2022-11-18T20:51:10.1852796Z 2022-11-18T20:51:10.1889264Z √ Successful POST request 2022-11-18T20:51:10.2192589Z 2022-11-18T20:51:10.2193431Z → Model Zoo - Inference Model 2022-11-18T20:51:11.7319836Z POST http://localhost:8080/predictions/fcn_resnet_101_scripted 2022-11-18T20:51:11.7321047Z 200 OK ★ 1452ms time ★ 289.23kB↑ 2.47MB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:51:11.7321458Z ┌ ↑ file ★ 288.94kB 2022-11-18T20:51:11.7321843Z │ (showing 2.05kB/288.94kB) 2022-11-18T20:51:11.7322134Z └ 2022-11-18T20:51:11.7322552Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 2.47MB 2022-11-18T20:51:11.7322885Z │ [ 2022-11-18T20:51:11.7323164Z │ [ 2022-11-18T20:51:11.7323426Z │ [ 2022-11-18T20:51:11.7323713Z │ 0.0, 2022-11-18T20:51:11.7324030Z │ 0.9993855953216553 2022-11-18T20:51:11.7324361Z │ ], 2022-11-18T20:51:11.7324642Z │ [ 2022-11-18T20:51:11.7324931Z │ 0.0, 2022-11-18T20:51:11.7325246Z │ 0.9993855953216553 2022-11-18T20:51:11.7325518Z │ ], 2022-11-18T20:51:11.7325802Z │ [ 2022-11-18T20:51:11.7326089Z │ 0.0, 2022-11-18T20:51:11.7326380Z │ 0.9993855953216553 2022-11-18T20:51:11.7326672Z │ ], 2022-11-18T20:51:11.7326967Z │ [ 2022-11-18T20:51:11.7327249Z │ 0.0, 2022-11-18T20:51:11.7327538Z │ 0.9993855953216553 2022-11-18T20:51:11.7327828Z │ ], 2022-11-18T20:51:11.7328105Z │ [ 2022-11-18T20:51:11.7328368Z │ 0.0, 2022-11-18T20:51:11.7328695Z │ 0.9993864297866821 2022-11-18T20:51:11.7328985Z │ ], 2022-11-18T20:51:11.7329257Z │ [ 2022-11-18T20:51:11.7329519Z │ 0.0, 2022-11-18T20:51:11.7329824Z │ 0.9993854761123657 2022-11-18T20:51:11.7330117Z │ ], 2022-11-18T20:51:11.7330378Z │ [ 2022-11-18T20:51:11.7330660Z │ 0.0, 2022-11-18T20:51:11.7330967Z │ 0.9993811845779419 2022-11-18T20:51:11.7331258Z │ ], 2022-11-18T20:51:11.7331515Z │ [ 2022-11-18T20:51:11.7331799Z │ 0.0, 2022-11-18T20:51:11.7332107Z │ 0.9993742108345032 2022-11-18T20:51:11.7332381Z │ ], 2022-11-18T20:51:11.7332651Z │ [ 2022-11-18T20:51:11.7333262Z │ 0.0, 2022-11-18T20:51:11.7334361Z │ 0.9993641972541809 2022-11-18T20:51:11.7334672Z │ ], 2022-11-18T20:51:11.7334926Z │ [ 2022-11-18T20:51:11.7335205Z │ 0.0, 2022-11-18T20:51:11.7335510Z │ 0.9993515610694885 2022-11-18T20:51:11.7335782Z │ ], 2022-11-18T20:51:11.7336309Z │ [ 2022-11-18T20:51:11.7336604Z │ 0.0, 2022-11-18T20:51:11.7336898Z │ 0.9993364214897156 2022-11-18T20:51:11.7337193Z │ ], 2022-11-18T20:51:11.7337469Z │ [ 2022-11-18T20:51:11.7337746Z │ 0.0, 2022-11-18T20:51:11.7338037Z │ 0.9993187189102173 2022-11-18T20:51:11.7338328Z │ ], 2022-11-18T20:51:11.7338601Z │ [ 2022-11-18T20:51:11.7338864Z │ 0.0, 2022-11-18T20:51:11.7339181Z │ 0.9992934465408325 2022-11-18T20:51:11.7339474Z │ ], 2022-11-18T20:51:11.7340016Z │ [ 2022-11-18T20:51:11.7340555Z │ 0.0, 2022-11-18T20:51:11.7340823Z │ 0.9992607235908508 2022-11-18T20:51:11.7341065Z │ ], 2022-11-18T20:51:11.7341520Z │ [ 2022-11-18T20:51:11.7341725Z │ 0.0, 2022-11-18T20:51:11.7341980Z │ 0.9992249011993408 2022-11-18T20:51:11.7342216Z │ ], 2022-11-18T20:51:11.7342456Z │ [ 2022-11-18T20:51:11.7342675Z │ 0.0, 2022-11-18T20:51:11.7342941Z │ 0.9991866946220398 2022-11-18T20:51:11.7343158Z │ ], 2022-11-18T20:51:11.7343372Z │ [ 2022-11-18T20:51:11.7343595Z │ 0.0, 2022-11-18T20:51:11.7343818Z │ 0.9991452693939209 2022-11-18T20:51:11.7344045Z │ ], 2022-11-18T20:51:11.7344258Z │ [ 2022-11-18T20:51:11.7344455Z │ 0.0, 2022-11-18T20:51:11.7344700Z │ 0.9991005659103394 2022-11-18T20:51:11.7344931Z │ ], 2022-11-18T20:51:11.7345128Z │ [ 2022-11-18T20:51:11.7345345Z │ 0.0, 2022-11-18T20:51:11.7345589Z │ 0.9990523457527161 2022-11-18T20:51:11.7345801Z │ ], 2022-11-18T20:51:11.7346016Z │ [ 2022-11-18T20:51:11.7346232Z │ 0.0, 2022-11-18T20:51:11.7346457Z │ 0.9990004897117615 2022-11-18T20:51:11.7346693Z │ ], 2022-11-18T20:51:11.7346906Z │ [ 2022-11-18T20:51:11.7347103Z │ 0.0, 2022-11-18T20:51:11.7347346Z │ 0.998968243598938 2022-11-18T20:51:11.7347575Z │ ], 2022-11-18T20:51:11.7347777Z │ [ 2022-11-18T20:51:11.7347998Z │ 0.0, 2022-11-18T20:51:11.7348243Z │ 0.9989573955535889 2022-11-18T20:51:11.7348474Z │ ], 2022-11-18T20:51:11.7348663Z │ [ 2022-11-18T20:51:11.7348880Z │ 0.0, 2022-11-18T20:51:11.7349133Z │ 0.9989456534385681 2022-11-18T20:51:11.7349345Z │ ], 2022-11-18T20:51:11.7349561Z │ [ 2022-11-18T20:51:11.7349777Z │ 0.0, 2022-11-18T20:51:11.7350003Z │ 0.9989331364631653 2022-11-18T20:51:11.7350231Z │ ], 2022-11-18T20:51:11.7350443Z │ [ 2022-11-18T20:51:11.7350642Z │ 0.0, 2022-11-18T20:51:11.7350886Z │ 0.9989200830459595 2022-11-18T20:51:11.7351113Z │ ], 2022-11-18T20:51:11.7351308Z │ [ 2022-11-18T20:51:11.7351537Z │ 0.0, 2022-11-18T20:51:11.7351777Z │ 0.9989060163497925 2022-11-18T20:51:11.7351985Z │ ], 2022-11-18T20:51:11.7352198Z │ [ 2022-11-18T20:51:11.7352420Z │ 0.0, 2022-11-18T20:51:11.7352735Z │ 0.998891294002533 2022-11-18T20:51:11.7352971Z │ ], 2022-11-18T20:51:11.7353182Z │ [ 2022-11-18T20:51:11.7353378Z │ 0.0, 2022-11-18T20:51:11.7353624Z │ 0.9988754391670227 2022-11-18T20:51:11.7353845Z │ ], 2022-11-18T20:51:11.7354039Z │ [ 2022-11-18T20:51:11.7354257Z │ 0.0, 2022-11-18T20:51:11.7354501Z │ 0.9988815188407898 2022-11-18T20:51:11.7354709Z │ ], 2022-11-18T20:51:11.7354922Z │ [ 2022-11-18T20:51:11.7355139Z │ 0.0, 2022-11-18T20:51:11.7355366Z │ 0.9989089965820312 2022-11-18T20:51:11.7355594Z │ ], 2022-11-18T20:51:11.7355807Z │ [ 2022-11-18T20:51:11.7356003Z │ 0.0, 2022-11-18T20:51:11.7356248Z │ 0.9989352822303772 2022-11-18T20:51:11.7357308Z │ ], 2022-11-18T20:51:11.7357509Z │ [ 2022-11-18T20:51:11.7357731Z │ 0.0, 2022-11-18T20:51:11.7357974Z │ 0.9989607334136963 2022-11-18T20:51:11.7358191Z │ ], 2022-11-18T20:51:11.7358416Z │ [ 2022-11-18T20:51:11.7358637Z │ 0.0, 2022-11-18T20:51:11.7358865Z │ 0.9989848732948303 2022-11-18T20:51:11.7359090Z │ ], 2022-11-18T20:51:11.7359294Z │ [ 2022-11-18T20:51:11.7359512Z │ 0.0, 2022-11-18T20:51:11.7359734Z │ 0.9990076422691345 2022-11-18T20:51:11.7359951Z │ ], 2022-11-18T20:51:11.7360167Z │ [ 2022-11-18T20:51:11.7360360Z │ 0.0, 2022-11-18T20:51:11.7360601Z │ 0.9990293979644775 2022-11-18T20:51:11.7360820Z │ ], 2022-11-18T20:51:11.7361009Z │ [ 2022-11-18T20:51:11.7361215Z │ 0.0, 2022-11-18T20:51:11.7361441Z │ 0.999049723148346 2022-11-18T20:51:11.7361643Z │ ], 2022-11-18T20:51:11.7361843Z │ [ 2022-11-18T20:51:11.7362061Z │ 0.0, 2022-11-18T20:51:11.7362272Z │ 0.999051034450531 2022-11-18T20:51:11.7362482Z │ ], 2022-11-18T20:51:11.7362679Z │ [ 2022-11-18T20:51:11.7362866Z │ 0.0, 2022-11-18T20:51:11.7363103Z │ 0.9990313053131104 2022-11-18T20:51:11.7363320Z │ ], 2022-11-18T20:51:11.7363506Z │ [ 2022-11-18T20:51:11.7363708Z │ 0.0, 2022-11-18T20:51:11.7363937Z │ 0.9990096092224121 2022-11-18T20:51:11.7364136Z │ ], 2022-11-18T20:51:11.7364332Z │ [ 2022-11-18T20:51:11.7364534Z │ 0.0, 2022-11-18T20:51:11.7364752Z │ 0.9989851117134094 2022-11-18T20:51:11.7364963Z │ ], 2022-11-18T20:51:11.7365161Z │ [ 2022-11-18T20:51:11.7365349Z │ 0.0, 2022-11-18T20:51:11.7365576Z │ 0.9989587068557739 2022-11-18T20:51:11.7365788Z │ ], 2022-11-18T20:51:11.7365971Z │ [ 2022-11-18T20:51:11.7366171Z │ 0.0, 2022-11-18T20:51:11.7366392Z │ 0.998929798603 2022-11-18T20:51:11.7366635Z │ (showing 2.05kB/2.47MB) 2022-11-18T20:51:11.7366848Z └ 2022-11-18T20:51:11.7367275Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:11.7367766Z 2ms 320µs (cache) (cache) 1443ms 8ms 1ms 1455ms 2022-11-18T20:51:11.7367935Z 2022-11-18T20:51:11.9568281Z √ Successful POST request 2022-11-18T20:51:11.9775186Z 2022-11-18T20:51:11.9776060Z → Model Zoo - Unregister model 2022-11-18T20:51:12.1431029Z DELETE http://localhost:8081/models/fcn_resnet_101_scripted 2022-11-18T20:51:12.1432149Z 200 OK ★ 116ms time ★ 259B↑ 331B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:12.1432821Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 65B 2022-11-18T20:51:12.1433309Z │ { 2022-11-18T20:51:12.1433802Z │ "status": "Model \"fcn_resnet_101_scripted\" unregis 2022-11-18T20:51:12.1434233Z │ tered" 2022-11-18T20:51:12.1434584Z │ } 2022-11-18T20:51:12.1451797Z └ 2022-11-18T20:51:12.1452425Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:12.1453085Z 2ms 367µs (cache) (cache) 113ms 1ms 66µs 118ms 2022-11-18T20:51:12.1453686Z 2022-11-18T20:51:12.1598211Z √ Successful DELETE request 2022-11-18T20:51:12.1859482Z 2022-11-18T20:51:12.1860071Z → Model Zoo - Model Metrics 2022-11-18T20:51:12.2378200Z GET http://localhost:8082/metrics 2022-11-18T20:51:12.2383522Z 200 OK ★ 4ms time ★ 233B↑ 7.54kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:12.2383943Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 7.25kB 2022-11-18T20:51:12.2385273Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:51:12.2939303Z │ erence requests. 2022-11-18T20:51:12.2940416Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:51:12.2940954Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2949003Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:51:12.2949622Z │ version="default",} 1.0 2022-11-18T20:51:12.2950011Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2950421Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:51:12.2950782Z │ l_version="default",} 1.0 2022-11-18T20:51:12.2951131Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2951488Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:51:12.2951786Z │ l_version="default",} 1.0 2022-11-18T20:51:12.2952113Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2952465Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:51:12.2952770Z │ model_version="default",} 1.0 2022-11-18T20:51:12.2953110Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2956118Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:51:12.2956481Z │ sion="default",} 1.0 2022-11-18T20:51:12.2956917Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2957347Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:51:12.2957644Z │ "default",} 1.0 2022-11-18T20:51:12.2958022Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2958434Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:51:12.2958744Z │ rsion="default",} 1.0 2022-11-18T20:51:12.2959100Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2959553Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:51:12.2959875Z │ ted",model_version="default",} 1.0 2022-11-18T20:51:12.2960213Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2960575Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:51:12.2960863Z │ rsion="default",} 1.0 2022-11-18T20:51:12.2961181Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2961543Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:51:12.2961844Z │ on="default",} 1.0 2022-11-18T20:51:12.2962177Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2962563Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:51:12.2962848Z │ ="default",} 1.0 2022-11-18T20:51:12.2963170Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2963517Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:51:12.2963784Z │ fault",} 1.0 2022-11-18T20:51:12.2964129Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2964476Z │ fe3-7d6b11352b25",model_name="fcn_resnet_101",model_ve 2022-11-18T20:51:12.2964767Z │ rsion="default",} 1.0 2022-11-18T20:51:12.2965102Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2965649Z │ fe3-7d6b11352b25",model_name="fcn_resnet_101_scripted" 2022-11-18T20:51:12.2965956Z │ ,model_version="default",} 1.0 2022-11-18T20:51:12.2966296Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:12.2966655Z │ fe3-7d6b11352b25",model_name="mnist",model_v 2022-11-18T20:51:12.2967106Z │ (showing 2.05kB/7.25kB) 2022-11-18T20:51:12.2967333Z └ 2022-11-18T20:51:12.2967733Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:12.2968192Z 1ms 276µs (cache) (cache) 1ms 1ms 50µs 4ms 2022-11-18T20:51:12.2968368Z 2022-11-18T20:51:12.2968511Z √ Successful GET request 2022-11-18T20:51:12.2968654Z 2022-11-18T20:51:12.2968738Z Iteration 18/24 2022-11-18T20:51:12.2968865Z 2022-11-18T20:51:12.2969019Z → Model Zoo - Register Model 2022-11-18T20:51:33.0540937Z POST http://localhost:8081/models?url=deeplabv3_resnet_101_eager.mar&model_name=deeplabv3_resnet_101_eager&initial_workers=1&synchronous=true 2022-11-18T20:51:33.0542391Z 200 OK ★ 20.7s time ★ 360B↑ 369B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:33.0543115Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 102B 2022-11-18T20:51:33.0543650Z │ { 2022-11-18T20:51:33.0544196Z │ "status": "Model \"deeplabv3_resnet_101_eager\" Vers 2022-11-18T20:51:33.0544735Z │ ion: 1.0 registered with 1 initial workers" 2022-11-18T20:51:33.0545179Z │ } 2022-11-18T20:51:33.0564751Z └ 2022-11-18T20:51:33.0565335Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:33.0566010Z 1ms 277µs (cache) (cache) 20.7s 3ms 75µs 20.7s 2022-11-18T20:51:33.0566371Z 2022-11-18T20:51:33.0733696Z √ Successful POST request 2022-11-18T20:51:33.0991000Z 2022-11-18T20:51:33.0991892Z → Model Zoo - Inference Model 2022-11-18T20:51:34.1986493Z POST http://localhost:8080/predictions/deeplabv3_resnet_101_eager 2022-11-18T20:51:34.1987742Z 200 OK ★ 1041ms time ★ 289.23kB↑ 2.47MB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:51:34.1988425Z ┌ ↑ file ★ 288.94kB 2022-11-18T20:51:34.1988826Z │ (showing 2.05kB/288.94kB) 2022-11-18T20:51:34.1989163Z └ 2022-11-18T20:51:34.1989592Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 2.47MB 2022-11-18T20:51:34.1989941Z │ [ 2022-11-18T20:51:34.1990242Z │ [ 2022-11-18T20:51:34.1990544Z │ [ 2022-11-18T20:51:34.1990863Z │ 0.0, 2022-11-18T20:51:34.1991194Z │ 0.9988765120506287 2022-11-18T20:51:34.1991516Z │ ], 2022-11-18T20:51:34.1991821Z │ [ 2022-11-18T20:51:34.1992115Z │ 0.0, 2022-11-18T20:51:34.1992454Z │ 0.9988765120506287 2022-11-18T20:51:34.1992802Z │ ], 2022-11-18T20:51:34.1993006Z │ [ 2022-11-18T20:51:34.1993200Z │ 0.0, 2022-11-18T20:51:34.1993495Z │ 0.9988765120506287 2022-11-18T20:51:34.1993708Z │ ], 2022-11-18T20:51:34.1993909Z │ [ 2022-11-18T20:51:34.1994101Z │ 0.0, 2022-11-18T20:51:34.1994346Z │ 0.9988765120506287 2022-11-18T20:51:34.1994556Z │ ], 2022-11-18T20:51:34.1994751Z │ [ 2022-11-18T20:51:34.1994940Z │ 0.0, 2022-11-18T20:51:34.1995166Z │ 0.9988669157028198 2022-11-18T20:51:34.1995381Z │ ], 2022-11-18T20:51:34.1995574Z │ [ 2022-11-18T20:51:34.1995775Z │ 0.0, 2022-11-18T20:51:34.1996000Z │ 0.998843789100647 2022-11-18T20:51:34.1996202Z │ ], 2022-11-18T20:51:34.1996464Z │ [ 2022-11-18T20:51:34.1996665Z │ 0.0, 2022-11-18T20:51:34.1996881Z │ 0.9988172054290771 2022-11-18T20:51:34.1997092Z │ ], 2022-11-18T20:51:34.1997288Z │ [ 2022-11-18T20:51:34.1997478Z │ 0.0, 2022-11-18T20:51:34.1997704Z │ 0.9987861514091492 2022-11-18T20:51:34.1997916Z │ ], 2022-11-18T20:51:34.1998102Z │ [ 2022-11-18T20:51:34.1998302Z │ 0.0, 2022-11-18T20:51:34.1998529Z │ 0.9987504482269287 2022-11-18T20:51:34.1998727Z │ ], 2022-11-18T20:51:34.1998938Z │ [ 2022-11-18T20:51:34.1999139Z │ 0.0, 2022-11-18T20:51:34.1999358Z │ 0.9987117052078247 2022-11-18T20:51:34.1999567Z │ ], 2022-11-18T20:51:34.1999762Z │ [ 2022-11-18T20:51:34.1999954Z │ 0.0, 2022-11-18T20:51:34.2001180Z │ 0.9986679553985596 2022-11-18T20:51:34.2001459Z │ ], 2022-11-18T20:51:34.2001646Z │ [ 2022-11-18T20:51:34.2001850Z │ 0.0, 2022-11-18T20:51:34.2002084Z │ 0.9986202716827393 2022-11-18T20:51:34.2002285Z │ ], 2022-11-18T20:51:34.2002486Z │ [ 2022-11-18T20:51:34.2002687Z │ 0.0, 2022-11-18T20:51:34.2002902Z │ 0.9985180497169495 2022-11-18T20:51:34.2003116Z │ ], 2022-11-18T20:51:34.2003316Z │ [ 2022-11-18T20:51:34.2003502Z │ 0.0, 2022-11-18T20:51:34.2003729Z │ 0.9983478784561157 2022-11-18T20:51:34.2003945Z │ ], 2022-11-18T20:51:34.2004128Z │ [ 2022-11-18T20:51:34.2004328Z │ 0.0, 2022-11-18T20:51:34.2004556Z │ 0.9981531500816345 2022-11-18T20:51:34.2004886Z │ ], 2022-11-18T20:51:34.2005070Z │ [ 2022-11-18T20:51:34.2005272Z │ 0.0, 2022-11-18T20:51:34.2005507Z │ 0.9979324340820312 2022-11-18T20:51:34.2005707Z │ ], 2022-11-18T20:51:34.2005913Z │ [ 2022-11-18T20:51:34.2006117Z │ 0.0, 2022-11-18T20:51:34.2006330Z │ 0.9976813793182373 2022-11-18T20:51:34.2006542Z │ ], 2022-11-18T20:51:34.2006736Z │ [ 2022-11-18T20:51:34.2006930Z │ 0.0, 2022-11-18T20:51:34.2007157Z │ 0.9973964691162109 2022-11-18T20:51:34.2007368Z │ ], 2022-11-18T20:51:34.2007555Z │ [ 2022-11-18T20:51:34.2007800Z │ 0.0, 2022-11-18T20:51:34.2008025Z │ 0.9970728158950806 2022-11-18T20:51:34.2008227Z │ ], 2022-11-18T20:51:34.2008424Z │ [ 2022-11-18T20:51:34.2008630Z │ 0.0, 2022-11-18T20:51:34.2008847Z │ 0.9967058300971985 2022-11-18T20:51:34.2009059Z │ ], 2022-11-18T20:51:34.2009255Z │ [ 2022-11-18T20:51:34.2009453Z │ 0.0, 2022-11-18T20:51:34.2009680Z │ 0.9964045286178589 2022-11-18T20:51:34.2009892Z │ ], 2022-11-18T20:51:34.2010078Z │ [ 2022-11-18T20:51:34.2010279Z │ 0.0, 2022-11-18T20:51:34.2010512Z │ 0.9961955547332764 2022-11-18T20:51:34.2010714Z │ ], 2022-11-18T20:51:34.2010910Z │ [ 2022-11-18T20:51:34.2011112Z │ 0.0, 2022-11-18T20:51:34.2011328Z │ 0.9959736466407776 2022-11-18T20:51:34.2011545Z │ ], 2022-11-18T20:51:34.2011741Z │ [ 2022-11-18T20:51:34.2011929Z │ 0.0, 2022-11-18T20:51:34.2012171Z │ 0.9957383871078491 2022-11-18T20:51:34.2012375Z │ ], 2022-11-18T20:51:34.2012577Z │ [ 2022-11-18T20:51:34.2012785Z │ 0.0, 2022-11-18T20:51:34.2013004Z │ 0.9954885840415955 2022-11-18T20:51:34.2013218Z │ ], 2022-11-18T20:51:34.2013419Z │ [ 2022-11-18T20:51:34.2013608Z │ 0.0, 2022-11-18T20:51:34.2013835Z │ 0.9952237010002136 2022-11-18T20:51:34.2014054Z │ ], 2022-11-18T20:51:34.2014239Z │ [ 2022-11-18T20:51:34.2014440Z │ 0.0, 2022-11-18T20:51:34.2014664Z │ 0.9949430823326111 2022-11-18T20:51:34.2014864Z │ ], 2022-11-18T20:51:34.2015068Z │ [ 2022-11-18T20:51:34.2015269Z │ 0.0, 2022-11-18T20:51:34.2015484Z │ 0.994645357131958 2022-11-18T20:51:34.2015703Z │ ], 2022-11-18T20:51:34.2015901Z │ [ 2022-11-18T20:51:34.2016089Z │ 0.0, 2022-11-18T20:51:34.2016317Z │ 0.9945149421691895 2022-11-18T20:51:34.2016529Z │ ], 2022-11-18T20:51:34.2016725Z │ [ 2022-11-18T20:51:34.2016913Z │ 0.0, 2022-11-18T20:51:34.2017147Z │ 0.9945647716522217 2022-11-18T20:51:34.2017362Z │ ], 2022-11-18T20:51:34.2017548Z │ [ 2022-11-18T20:51:34.2017756Z │ 0.0, 2022-11-18T20:51:34.2017991Z │ 0.9946145415306091 2022-11-18T20:51:34.2018200Z │ ], 2022-11-18T20:51:34.2018397Z │ [ 2022-11-18T20:51:34.2018610Z │ 0.0, 2022-11-18T20:51:34.2018825Z │ 0.9946632385253906 2022-11-18T20:51:34.2019043Z │ ], 2022-11-18T20:51:34.2019241Z │ [ 2022-11-18T20:51:34.2019435Z │ 0.0, 2022-11-18T20:51:34.2019785Z │ 0.9947116374969482 2022-11-18T20:51:34.2020012Z │ ], 2022-11-18T20:51:34.2020196Z │ [ 2022-11-18T20:51:34.2020405Z │ 0.0, 2022-11-18T20:51:34.2020636Z │ 0.9947587847709656 2022-11-18T20:51:34.2020837Z │ ], 2022-11-18T20:51:34.2021037Z │ [ 2022-11-18T20:51:34.2021247Z │ 0.0, 2022-11-18T20:51:34.2021463Z │ 0.9948055148124695 2022-11-18T20:51:34.2021676Z │ ], 2022-11-18T20:51:34.2021872Z │ [ 2022-11-18T20:51:34.2022062Z │ 0.0, 2022-11-18T20:51:34.2022289Z │ 0.9948518872261047 2022-11-18T20:51:34.2022503Z │ ], 2022-11-18T20:51:34.2022688Z │ [ 2022-11-18T20:51:34.2022890Z │ 0.0, 2022-11-18T20:51:34.2023118Z │ 0.9948964715003967 2022-11-18T20:51:34.2023391Z │ ], 2022-11-18T20:51:34.2023590Z │ [ 2022-11-18T20:51:34.2023799Z │ 0.0, 2022-11-18T20:51:34.2024017Z │ 0.9949400424957275 2022-11-18T20:51:34.2024232Z │ ], 2022-11-18T20:51:34.2024433Z │ [ 2022-11-18T20:51:34.2024632Z │ 0.0, 2022-11-18T20:51:34.2024862Z │ 0.9949837923049927 2022-11-18T20:51:34.2025075Z │ ], 2022-11-18T20:51:34.2025261Z │ [ 2022-11-18T20:51:34.2025462Z │ 0.0, 2022-11-18T20:51:34.2025694Z │ 0.9950267672538757 2022-11-18T20:51:34.2025908Z │ ], 2022-11-18T20:51:34.2026092Z │ [ 2022-11-18T20:51:34.2026296Z │ 0.0, 2022-11-18T20:51:34.2026529Z │ 0.9950693845748901 2022-11-18T20:51:34.2026726Z │ ], 2022-11-18T20:51:34.2026924Z │ [ 2022-11-18T20:51:34.2027126Z │ 0.0, 2022-11-18T20:51:34.2027332Z │ 0.9951115250 2022-11-18T20:51:34.2027583Z │ (showing 2.05kB/2.47MB) 2022-11-18T20:51:34.2027799Z └ 2022-11-18T20:51:34.2028210Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:34.2028691Z 3ms 324µs (cache) (cache) 1024ms 15ms 941µs 1045ms 2022-11-18T20:51:34.2028877Z 2022-11-18T20:51:34.4280194Z √ Successful POST request 2022-11-18T20:51:34.4401342Z 2022-11-18T20:51:34.4402149Z → Model Zoo - Unregister model 2022-11-18T20:51:34.5932507Z DELETE http://localhost:8081/models/deeplabv3_resnet_101_eager 2022-11-18T20:51:34.5933555Z 200 OK ★ 103ms time ★ 262B↑ 334B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:34.5934321Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 68B 2022-11-18T20:51:34.5934800Z │ { 2022-11-18T20:51:34.5935204Z │ "status": "Model \"deeplabv3_resnet_101_eager\" unre 2022-11-18T20:51:34.5935596Z │ gistered" 2022-11-18T20:51:34.5935941Z │ } 2022-11-18T20:51:34.5954788Z └ 2022-11-18T20:51:34.5955318Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:34.5955956Z 2ms 409µs (cache) (cache) 100ms 1ms 72µs 105ms 2022-11-18T20:51:34.5956258Z 2022-11-18T20:51:34.6071683Z √ Successful DELETE request 2022-11-18T20:51:34.6343440Z 2022-11-18T20:51:34.6344086Z → Model Zoo - Model Metrics 2022-11-18T20:51:34.6886804Z GET http://localhost:8082/metrics 2022-11-18T20:51:34.6887931Z 200 OK ★ 4ms time ★ 233B↑ 7.99kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:34.6888373Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 7.69kB 2022-11-18T20:51:34.6888755Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:51:34.6889046Z │ erence requests. 2022-11-18T20:51:34.6889356Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:51:34.6889731Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6890099Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:51:34.6890405Z │ version="default",} 1.0 2022-11-18T20:51:34.6890779Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6891154Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:51:34.6891453Z │ l_version="default",} 1.0 2022-11-18T20:51:34.6892082Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6892465Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:51:34.6892760Z │ l_version="default",} 1.0 2022-11-18T20:51:34.6893103Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6893482Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:51:34.6893793Z │ model_version="default",} 1.0 2022-11-18T20:51:34.6894140Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6894505Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:51:34.6894796Z │ sion="default",} 1.0 2022-11-18T20:51:34.6895121Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6895585Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:51:34.6895863Z │ "default",} 1.0 2022-11-18T20:51:34.6896176Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6896553Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:51:34.6896851Z │ rsion="default",} 1.0 2022-11-18T20:51:34.6897171Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6897537Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:51:34.6897867Z │ ted",model_version="default",} 1.0 2022-11-18T20:51:34.6898203Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6898567Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:51:34.6898859Z │ rsion="default",} 1.0 2022-11-18T20:51:34.6899188Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6899551Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:51:34.6899846Z │ on="default",} 1.0 2022-11-18T20:51:34.6900172Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6900524Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:51:34.6900813Z │ ="default",} 1.0 2022-11-18T20:51:34.6901151Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6901513Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:51:34.6901770Z │ fault",} 1.0 2022-11-18T20:51:34.6902088Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6902448Z │ fe3-7d6b11352b25",model_name="fcn_resnet_101",model_ve 2022-11-18T20:51:34.6902722Z │ rsion="default",} 1.0 2022-11-18T20:51:34.6903057Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6903417Z │ fe3-7d6b11352b25",model_name="fcn_resnet_101_scripted" 2022-11-18T20:51:34.6903716Z │ ,model_version="default",} 1.0 2022-11-18T20:51:34.6904070Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:34.6904420Z │ fe3-7d6b11352b25",model_name="mnist",model_v 2022-11-18T20:51:34.6904711Z │ (showing 2.05kB/7.69kB) 2022-11-18T20:51:34.6904914Z └ 2022-11-18T20:51:34.6905330Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:34.6905809Z 1ms 298µs (cache) (cache) 1ms 1ms 57µs 5ms 2022-11-18T20:51:34.6905993Z 2022-11-18T20:51:34.7034560Z √ Successful GET request 2022-11-18T20:51:34.7056525Z 2022-11-18T20:51:34.7056884Z Iteration 19/24 2022-11-18T20:51:34.7125392Z 2022-11-18T20:51:34.7126042Z → Model Zoo - Register Model 2022-11-18T20:51:44.2137236Z POST http://localhost:8081/models?url=https://torchserve.s3.amazonaws.com/mar_files/deeplabv3_resnet_101_scripted.mar&model_name=deeplabv3_resnet_101_scripted&initial_workers=1&synchronous=true 2022-11-18T20:51:44.2138469Z 200 OK ★ 9.4s time ★ 412B↑ 372B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:44.2139140Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 105B 2022-11-18T20:51:44.2139548Z │ { 2022-11-18T20:51:44.2139979Z │ "status": "Model \"deeplabv3_resnet_101_scripted\" V 2022-11-18T20:51:44.2140671Z │ ersion: 1.0 registered with 1 initial workers" 2022-11-18T20:51:44.2141096Z │ } 2022-11-18T20:51:44.2159532Z └ 2022-11-18T20:51:44.2160057Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:44.2160677Z 1ms 274µs (cache) (cache) 9.4s 3ms 79µs 9.4s 2022-11-18T20:51:44.2160979Z 2022-11-18T20:51:44.2323288Z √ Successful POST request 2022-11-18T20:51:44.2583274Z 2022-11-18T20:51:44.2584184Z → Model Zoo - Inference Model 2022-11-18T20:51:45.7047903Z POST http://localhost:8080/predictions/deeplabv3_resnet_101_scripted 2022-11-18T20:51:45.7049569Z 200 OK ★ 1387ms time ★ 289.24kB↑ 2.47MB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:51:45.7050269Z ┌ ↑ file ★ 288.94kB 2022-11-18T20:51:45.7050549Z │ (showing 2.05kB/288.94kB) 2022-11-18T20:51:45.7050784Z └ 2022-11-18T20:51:45.7051084Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 2.47MB 2022-11-18T20:51:45.7051356Z │ [ 2022-11-18T20:51:45.7051556Z │ [ 2022-11-18T20:51:45.7051752Z │ [ 2022-11-18T20:51:45.7051955Z │ 0.0, 2022-11-18T20:51:45.7052196Z │ 0.9988765120506287 2022-11-18T20:51:45.7052438Z │ ], 2022-11-18T20:51:45.7052630Z │ [ 2022-11-18T20:51:45.7052837Z │ 0.0, 2022-11-18T20:51:45.7053070Z │ 0.9988765120506287 2022-11-18T20:51:45.7053273Z │ ], 2022-11-18T20:51:45.7053473Z │ [ 2022-11-18T20:51:45.7053676Z │ 0.0, 2022-11-18T20:51:45.7053899Z │ 0.9988765120506287 2022-11-18T20:51:45.7054295Z │ ], 2022-11-18T20:51:45.7054504Z │ [ 2022-11-18T20:51:45.7054695Z │ 0.0, 2022-11-18T20:51:45.7054933Z │ 0.9988765120506287 2022-11-18T20:51:45.7055172Z │ ], 2022-11-18T20:51:45.7055361Z │ [ 2022-11-18T20:51:45.7055569Z │ 0.0, 2022-11-18T20:51:45.7055803Z │ 0.9988669157028198 2022-11-18T20:51:45.7056009Z │ ], 2022-11-18T20:51:45.7056213Z │ [ 2022-11-18T20:51:45.7056418Z │ 0.0, 2022-11-18T20:51:45.7056633Z │ 0.998843789100647 2022-11-18T20:51:45.7056851Z │ ], 2022-11-18T20:51:45.7057053Z │ [ 2022-11-18T20:51:45.7057241Z │ 0.0, 2022-11-18T20:51:45.7057475Z │ 0.9988172054290771 2022-11-18T20:51:45.7057692Z │ ], 2022-11-18T20:51:45.7057967Z │ [ 2022-11-18T20:51:45.7058177Z │ 0.0, 2022-11-18T20:51:45.7058413Z │ 0.9987861514091492 2022-11-18T20:51:45.7058618Z │ ], 2022-11-18T20:51:45.7058820Z │ [ 2022-11-18T20:51:45.7059509Z │ 0.0, 2022-11-18T20:51:45.7059731Z │ 0.9987504482269287 2022-11-18T20:51:45.7059949Z │ ], 2022-11-18T20:51:45.7060158Z │ [ 2022-11-18T20:51:45.7060361Z │ 0.0, 2022-11-18T20:51:45.7060589Z │ 0.9987117052078247 2022-11-18T20:51:45.7060807Z │ ], 2022-11-18T20:51:45.7061007Z │ [ 2022-11-18T20:51:45.7061196Z │ 0.0, 2022-11-18T20:51:45.7061589Z │ 0.9986679553985596 2022-11-18T20:51:45.7061813Z │ ], 2022-11-18T20:51:45.7062003Z │ [ 2022-11-18T20:51:45.7062221Z │ 0.0, 2022-11-18T20:51:45.7062452Z │ 0.9986202716827393 2022-11-18T20:51:45.7062657Z │ ], 2022-11-18T20:51:45.7062858Z │ [ 2022-11-18T20:51:45.7063062Z │ 0.0, 2022-11-18T20:51:45.7063282Z │ 0.9985180497169495 2022-11-18T20:51:45.7063495Z │ ], 2022-11-18T20:51:45.7063698Z │ [ 2022-11-18T20:51:45.7063886Z │ 0.0, 2022-11-18T20:51:45.7064119Z │ 0.9983478784561157 2022-11-18T20:51:45.7064336Z │ ], 2022-11-18T20:51:45.7064522Z │ [ 2022-11-18T20:51:45.7064727Z │ 0.0, 2022-11-18T20:51:45.7064959Z │ 0.9981531500816345 2022-11-18T20:51:45.7065318Z │ ], 2022-11-18T20:51:45.7065511Z │ [ 2022-11-18T20:51:45.7065724Z │ 0.0, 2022-11-18T20:51:45.7065961Z │ 0.9979324340820312 2022-11-18T20:51:45.7066165Z │ ], 2022-11-18T20:51:45.7066371Z │ [ 2022-11-18T20:51:45.7066584Z │ 0.0, 2022-11-18T20:51:45.7066807Z │ 0.9976813793182373 2022-11-18T20:51:45.7067021Z │ ], 2022-11-18T20:51:45.7067221Z │ [ 2022-11-18T20:51:45.7067411Z │ 0.0, 2022-11-18T20:51:45.7067647Z │ 0.9973964691162109 2022-11-18T20:51:45.7067865Z │ ], 2022-11-18T20:51:45.7068048Z │ [ 2022-11-18T20:51:45.7068250Z │ 0.0, 2022-11-18T20:51:45.7068488Z │ 0.9970728158950806 2022-11-18T20:51:45.7068689Z │ ], 2022-11-18T20:51:45.7068889Z │ [ 2022-11-18T20:51:45.7069095Z │ 0.0, 2022-11-18T20:51:45.7069314Z │ 0.9967058300971985 2022-11-18T20:51:45.7069530Z │ ], 2022-11-18T20:51:45.7069732Z │ [ 2022-11-18T20:51:45.7069921Z │ 0.0, 2022-11-18T20:51:45.7070160Z │ 0.9964045286178589 2022-11-18T20:51:45.7070375Z │ ], 2022-11-18T20:51:45.7070561Z │ [ 2022-11-18T20:51:45.7070768Z │ 0.0, 2022-11-18T20:51:45.7070997Z │ 0.9961955547332764 2022-11-18T20:51:45.7071215Z │ ], 2022-11-18T20:51:45.7071402Z │ [ 2022-11-18T20:51:45.7071605Z │ 0.0, 2022-11-18T20:51:45.7071836Z │ 0.9959736466407776 2022-11-18T20:51:45.7072034Z │ ], 2022-11-18T20:51:45.7072236Z │ [ 2022-11-18T20:51:45.7072443Z │ 0.0, 2022-11-18T20:51:45.7072663Z │ 0.9957383871078491 2022-11-18T20:51:45.7072877Z │ ], 2022-11-18T20:51:45.7073080Z │ [ 2022-11-18T20:51:45.7073270Z │ 0.0, 2022-11-18T20:51:45.7073502Z │ 0.9954885840415955 2022-11-18T20:51:45.7073717Z │ ], 2022-11-18T20:51:45.7073908Z │ [ 2022-11-18T20:51:45.7074115Z │ 0.0, 2022-11-18T20:51:45.7074347Z │ 0.9952237010002136 2022-11-18T20:51:45.7074558Z │ ], 2022-11-18T20:51:45.7074759Z │ [ 2022-11-18T20:51:45.7074968Z │ 0.0, 2022-11-18T20:51:45.7075188Z │ 0.9949430823326111 2022-11-18T20:51:45.7075403Z │ ], 2022-11-18T20:51:45.7075601Z │ [ 2022-11-18T20:51:45.7075804Z │ 0.0, 2022-11-18T20:51:45.7076035Z │ 0.994645357131958 2022-11-18T20:51:45.7076252Z │ ], 2022-11-18T20:51:45.7076439Z │ [ 2022-11-18T20:51:45.7076646Z │ 0.0, 2022-11-18T20:51:45.7076885Z │ 0.9945149421691895 2022-11-18T20:51:45.7077087Z │ ], 2022-11-18T20:51:45.7077292Z │ [ 2022-11-18T20:51:45.7077497Z │ 0.0, 2022-11-18T20:51:45.7077716Z │ 0.9945647716522217 2022-11-18T20:51:45.7077933Z │ ], 2022-11-18T20:51:45.7078137Z │ [ 2022-11-18T20:51:45.7078333Z │ 0.0, 2022-11-18T20:51:45.7078564Z │ 0.9946145415306091 2022-11-18T20:51:45.7078788Z │ ], 2022-11-18T20:51:45.7078978Z │ [ 2022-11-18T20:51:45.7079181Z │ 0.0, 2022-11-18T20:51:45.7079420Z │ 0.9946632385253906 2022-11-18T20:51:45.7079619Z │ ], 2022-11-18T20:51:45.7079812Z │ [ 2022-11-18T20:51:45.7080018Z │ 0.0, 2022-11-18T20:51:45.7080256Z │ 0.9947116374969482 2022-11-18T20:51:45.7080572Z │ ], 2022-11-18T20:51:45.7080788Z │ [ 2022-11-18T20:51:45.7080991Z │ 0.0, 2022-11-18T20:51:45.7081291Z │ 0.9947587847709656 2022-11-18T20:51:45.7081510Z │ ], 2022-11-18T20:51:45.7081716Z │ [ 2022-11-18T20:51:45.7081910Z │ 0.0, 2022-11-18T20:51:45.7082157Z │ 0.9948055148124695 2022-11-18T20:51:45.7082378Z │ ], 2022-11-18T20:51:45.7082569Z │ [ 2022-11-18T20:51:45.7082781Z │ 0.0, 2022-11-18T20:51:45.7083011Z │ 0.9948518872261047 2022-11-18T20:51:45.7083212Z │ ], 2022-11-18T20:51:45.7083419Z │ [ 2022-11-18T20:51:45.7083623Z │ 0.0, 2022-11-18T20:51:45.7083843Z │ 0.9948964715003967 2022-11-18T20:51:45.7084059Z │ ], 2022-11-18T20:51:45.7084352Z │ [ 2022-11-18T20:51:45.7084550Z │ 0.0, 2022-11-18T20:51:45.7084786Z │ 0.9949400424957275 2022-11-18T20:51:45.7085004Z │ ], 2022-11-18T20:51:45.7085191Z │ [ 2022-11-18T20:51:45.7085406Z │ 0.0, 2022-11-18T20:51:45.7085645Z │ 0.9949837923049927 2022-11-18T20:51:45.7085849Z │ ], 2022-11-18T20:51:45.7086051Z │ [ 2022-11-18T20:51:45.7086251Z │ 0.0, 2022-11-18T20:51:45.7086467Z │ 0.9950267672538757 2022-11-18T20:51:45.7086682Z │ ], 2022-11-18T20:51:45.7086880Z │ [ 2022-11-18T20:51:45.7087069Z │ 0.0, 2022-11-18T20:51:45.7087301Z │ 0.9950693845748901 2022-11-18T20:51:45.7087515Z │ ], 2022-11-18T20:51:45.7087759Z │ [ 2022-11-18T20:51:45.7087964Z │ 0.0, 2022-11-18T20:51:45.7088182Z │ 0.9951115250 2022-11-18T20:51:45.7088414Z │ (showing 2.05kB/2.47MB) 2022-11-18T20:51:45.7088630Z └ 2022-11-18T20:51:45.7089039Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:45.7089532Z 4ms 385µs (cache) (cache) 1373ms 12ms 1ms 1392ms 2022-11-18T20:51:45.7089696Z 2022-11-18T20:51:45.9530650Z √ Successful POST request 2022-11-18T20:51:45.9961593Z 2022-11-18T20:51:45.9962378Z → Model Zoo - Unregister model 2022-11-18T20:51:46.1702890Z DELETE http://localhost:8081/models/deeplabv3_resnet_101_scripted 2022-11-18T20:51:46.1704131Z 200 OK ★ 124ms time ★ 265B↑ 337B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:46.1704802Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 71B 2022-11-18T20:51:46.1705256Z │ { 2022-11-18T20:51:46.1705691Z │ "status": "Model \"deeplabv3_resnet_101_scripted\" u 2022-11-18T20:51:46.1706086Z │ nregistered" 2022-11-18T20:51:46.1706422Z │ } 2022-11-18T20:51:46.1728398Z └ 2022-11-18T20:51:46.1728929Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:46.1729662Z 2ms 397µs (cache) (cache) 121ms 1ms 74µs 126ms 2022-11-18T20:51:46.1729970Z 2022-11-18T20:51:46.1956167Z √ Successful DELETE request 2022-11-18T20:51:46.2205533Z 2022-11-18T20:51:46.2206398Z → Model Zoo - Model Metrics 2022-11-18T20:51:46.2577261Z GET http://localhost:8082/metrics 2022-11-18T20:51:46.2578272Z 200 OK ★ 4ms time ★ 233B↑ 8.44kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:51:46.2578874Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 8.15kB 2022-11-18T20:51:46.2579417Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:51:46.2579828Z │ erence requests. 2022-11-18T20:51:46.2580248Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:51:46.2580729Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2581210Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:51:46.2581622Z │ version="default",} 1.0 2022-11-18T20:51:46.2582124Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2582621Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:51:46.2583044Z │ l_version="default",} 1.0 2022-11-18T20:51:46.2583774Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2584284Z │ fe3-7d6b11352b25",model_name="deeplabv3_resnet_101_scr 2022-11-18T20:51:46.2584737Z │ ipted",model_version="default",} 1.0 2022-11-18T20:51:46.2585219Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2585705Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:51:46.2586130Z │ l_version="default",} 1.0 2022-11-18T20:51:46.2586563Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2587057Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:51:46.2587501Z │ model_version="default",} 1.0 2022-11-18T20:51:46.2587963Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2588530Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:51:46.2588939Z │ sion="default",} 1.0 2022-11-18T20:51:46.2589392Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2589884Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:51:46.2590265Z │ "default",} 1.0 2022-11-18T20:51:46.2590707Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2591193Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:51:46.2591603Z │ rsion="default",} 1.0 2022-11-18T20:51:46.2592059Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2592549Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:51:46.2593009Z │ ted",model_version="default",} 1.0 2022-11-18T20:51:46.2593473Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2593966Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:51:46.2594376Z │ rsion="default",} 1.0 2022-11-18T20:51:46.2594796Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2595277Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:51:46.2595684Z │ on="default",} 1.0 2022-11-18T20:51:46.2596125Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2596575Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:51:46.2596971Z │ ="default",} 1.0 2022-11-18T20:51:46.2597402Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2597877Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:51:46.2598241Z │ fault",} 1.0 2022-11-18T20:51:46.2598667Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2599134Z │ fe3-7d6b11352b25",model_name="fcn_resnet_101",model_ve 2022-11-18T20:51:46.2599530Z │ rsion="default",} 1.0 2022-11-18T20:51:46.2599948Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:51:46.2600403Z │ fe3-7d6b11352b25",model_name="fcn_resn 2022-11-18T20:51:46.2600791Z │ (showing 2.05kB/8.15kB) 2022-11-18T20:51:46.2601135Z └ 2022-11-18T20:51:46.2601659Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:51:46.2602255Z 1ms 310µs (cache) (cache) 1ms 1ms 62µs 5ms 2022-11-18T20:51:46.2602535Z 2022-11-18T20:51:46.2642917Z √ Successful GET request 2022-11-18T20:51:46.2672744Z 2022-11-18T20:51:46.2673063Z Iteration 20/24 2022-11-18T20:51:46.2829010Z 2022-11-18T20:51:46.2829669Z → Model Zoo - Register Model 2022-11-18T20:52:06.8956563Z POST http://localhost:8081/models?url=resnet-152-batch.mar&model_name=resnet152&initial_workers=1&synchronous=true 2022-11-18T20:52:06.8957524Z 200 OK ★ 20.5s time ★ 333B↑ 351B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:06.8958251Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 85B 2022-11-18T20:52:06.8958796Z │ { 2022-11-18T20:52:06.8959338Z │ "status": "Model \"resnet152\" Version: 1.0 register 2022-11-18T20:52:06.8959881Z │ ed with 1 initial workers" 2022-11-18T20:52:06.8960299Z │ } 2022-11-18T20:52:06.8977454Z └ 2022-11-18T20:52:06.8978075Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:06.8978793Z 1ms 303µs (cache) (cache) 20.5s 2ms 82µs 20.5s 2022-11-18T20:52:06.8979177Z 2022-11-18T20:52:06.9163477Z √ Successful POST request 2022-11-18T20:52:06.9422545Z 2022-11-18T20:52:06.9423382Z → Model Zoo - Inference Model 2022-11-18T20:52:07.2973398Z POST http://localhost:8080/predictions/resnet152 2022-11-18T20:52:07.2974534Z 200 OK ★ 305ms time ★ 111.24kB↑ 410B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:52:07.2975162Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:52:07.2975690Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:52:07.2976461Z └ 2022-11-18T20:52:07.2976983Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 175B 2022-11-18T20:52:07.2977446Z │ { 2022-11-18T20:52:07.2977894Z │ "tiger_cat": 0.5798621773719788, 2022-11-18T20:52:07.2978421Z │ "tabby": 0.3834409713745117, 2022-11-18T20:52:07.2978931Z │ "Egyptian_cat": 0.034211426973342896, 2022-11-18T20:52:07.2979424Z │ "lynx": 0.000581979809794575, 2022-11-18T20:52:07.2979888Z │ "quilt": 0.00027331957244314253 2022-11-18T20:52:07.2980325Z │ } 2022-11-18T20:52:07.2997693Z └ 2022-11-18T20:52:07.2998357Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:07.2999090Z 2ms 327µs (cache) (cache) 302ms 1ms 46µs 307ms 2022-11-18T20:52:07.2999470Z 2022-11-18T20:52:07.3166372Z √ Successful POST request 2022-11-18T20:52:07.3182277Z √ Test expected JSON response 2022-11-18T20:52:07.3416597Z 2022-11-18T20:52:07.3417277Z → Model Zoo - Unregister model 2022-11-18T20:52:07.4934736Z DELETE http://localhost:8081/models/resnet152 2022-11-18T20:52:07.4935788Z 200 OK ★ 103ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:07.4936467Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:52:07.4936876Z │ { 2022-11-18T20:52:07.4937310Z │ "status": "Model \"resnet152\" unregistered" 2022-11-18T20:52:07.4937695Z │ } 2022-11-18T20:52:07.4949011Z └ 2022-11-18T20:52:07.4949555Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:07.4950165Z 1ms 881µs (cache) (cache) 100ms 1ms 45µs 104ms 2022-11-18T20:52:07.4950483Z 2022-11-18T20:52:07.5098938Z √ Successful DELETE request 2022-11-18T20:52:07.5368988Z 2022-11-18T20:52:07.5369600Z → Model Zoo - Model Metrics 2022-11-18T20:52:07.5913220Z GET http://localhost:8082/metrics 2022-11-18T20:52:07.5914216Z 200 OK ★ 5ms time ★ 233B↑ 8.83kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:07.5914857Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 8.54kB 2022-11-18T20:52:07.5915422Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:52:07.5915830Z │ erence requests. 2022-11-18T20:52:07.5916286Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:52:07.5916787Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5917274Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:52:07.5917705Z │ version="default",} 1.0 2022-11-18T20:52:07.5918172Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5918674Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:52:07.5919098Z │ l_version="default",} 1.0 2022-11-18T20:52:07.5919562Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5920065Z │ fe3-7d6b11352b25",model_name="deeplabv3_resnet_101_scr 2022-11-18T20:52:07.5920529Z │ ipted",model_version="default",} 1.0 2022-11-18T20:52:07.5921008Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5921499Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:52:07.5922170Z │ l_version="default",} 1.0 2022-11-18T20:52:07.5922656Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5923134Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:52:07.5923588Z │ model_version="default",} 1.0 2022-11-18T20:52:07.5924057Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5924545Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:52:07.5924946Z │ sion="default",} 1.0 2022-11-18T20:52:07.5925399Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5925885Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:52:07.5926272Z │ "default",} 1.0 2022-11-18T20:52:07.5926719Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5927316Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:52:07.5927736Z │ rsion="default",} 1.0 2022-11-18T20:52:07.5928182Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5928675Z │ fe3-7d6b11352b25",model_name="resnet152",model_version 2022-11-18T20:52:07.5931339Z │ ="default",} 1.0 2022-11-18T20:52:07.5931879Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5932371Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:52:07.5932827Z │ ted",model_version="default",} 1.0 2022-11-18T20:52:07.5933299Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5933795Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:52:07.5934189Z │ rsion="default",} 1.0 2022-11-18T20:52:07.5934650Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5935136Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:52:07.5935453Z │ on="default",} 1.0 2022-11-18T20:52:07.5935797Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5936193Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:52:07.5936496Z │ ="default",} 1.0 2022-11-18T20:52:07.5939378Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5939863Z │ fe3-7d6b11352b25",model_name="vgg16",model_version="de 2022-11-18T20:52:07.5940161Z │ fault",} 1.0 2022-11-18T20:52:07.5940502Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:07.5940859Z │ fe3-7d6b11352b25",model_name="fcn_resnet_10 2022-11-18T20:52:07.5941157Z │ (showing 2.05kB/8.54kB) 2022-11-18T20:52:07.5941391Z └ 2022-11-18T20:52:07.5941812Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:07.5942283Z 1ms 291µs (cache) (cache) 1ms 2ms 50µs 6ms 2022-11-18T20:52:07.5942486Z 2022-11-18T20:52:07.6043254Z √ Successful GET request 2022-11-18T20:52:07.6156035Z 2022-11-18T20:52:07.6156402Z Iteration 21/24 2022-11-18T20:52:07.6312647Z 2022-11-18T20:52:07.6313265Z → Model Zoo - Register Model 2022-11-18T20:52:15.5306073Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/resnet-152-batch_scripted.mar&model_name=resnet-152-batch_scripted&initial_workers=1&synchronous=true 2022-11-18T20:52:15.5307341Z 200 OK ★ 7.8s time ★ 399B↑ 368B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:15.5308096Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 101B 2022-11-18T20:52:15.5308531Z │ { 2022-11-18T20:52:15.5309001Z │ "status": "Model \"resnet-152-batch_scripted\" Versi 2022-11-18T20:52:15.5309484Z │ on: 1.0 registered with 1 initial workers" 2022-11-18T20:52:15.5309843Z │ } 2022-11-18T20:52:15.5310147Z └ 2022-11-18T20:52:15.5310669Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:15.5311303Z 1ms 296µs (cache) (cache) 7.8s 1ms 49µs 7.8s 2022-11-18T20:52:15.5311588Z 2022-11-18T20:52:15.5479085Z √ Successful POST request 2022-11-18T20:52:15.5757847Z 2022-11-18T20:52:15.5758710Z → Model Zoo - Inference Model 2022-11-18T20:52:16.5472051Z POST http://localhost:8080/predictions/resnet-152-batch_scripted 2022-11-18T20:52:16.5473067Z 200 OK ★ 922ms time ★ 111.26kB↑ 410B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:52:16.5473695Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:52:16.5474222Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:52:16.5474663Z └ 2022-11-18T20:52:16.5475185Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 175B 2022-11-18T20:52:16.5475670Z │ { 2022-11-18T20:52:16.5476142Z │ "tiger_cat": 0.5798621773719788, 2022-11-18T20:52:16.5476635Z │ "tabby": 0.3834409713745117, 2022-11-18T20:52:16.5477147Z │ "Egyptian_cat": 0.034211426973342896, 2022-11-18T20:52:16.5477873Z │ "lynx": 0.000581979809794575, 2022-11-18T20:52:16.5478361Z │ "quilt": 0.00027331957244314253 2022-11-18T20:52:16.5478794Z │ } 2022-11-18T20:52:16.5487206Z └ 2022-11-18T20:52:16.5487831Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:16.5488522Z 2ms 307µs (cache) (cache) 919ms 1ms 41µs 923ms 2022-11-18T20:52:16.5488918Z 2022-11-18T20:52:16.5648698Z √ Successful POST request 2022-11-18T20:52:16.5652631Z √ Test expected JSON response 2022-11-18T20:52:16.5915236Z 2022-11-18T20:52:16.5916028Z → Model Zoo - Unregister model 2022-11-18T20:52:16.7665458Z DELETE http://localhost:8081/models/resnet-152-batch_scripted 2022-11-18T20:52:16.7666352Z 200 OK ★ 126ms time ★ 261B↑ 333B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:16.7666960Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 67B 2022-11-18T20:52:16.7667380Z │ { 2022-11-18T20:52:16.7667874Z │ "status": "Model \"resnet-152-batch_scripted\" unreg 2022-11-18T20:52:16.7668262Z │ istered" 2022-11-18T20:52:16.7668590Z │ } 2022-11-18T20:52:16.7690283Z └ 2022-11-18T20:52:16.7691012Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:16.7691704Z 1ms 288µs (cache) (cache) 123ms 1ms 44µs 126ms 2022-11-18T20:52:16.7692040Z 2022-11-18T20:52:16.7872280Z √ Successful DELETE request 2022-11-18T20:52:16.8150061Z 2022-11-18T20:52:16.8150832Z → Model Zoo - Model Metrics 2022-11-18T20:52:16.8683121Z GET http://localhost:8082/metrics 2022-11-18T20:52:16.8684102Z 200 OK ★ 4ms time ★ 233B↑ 9.27kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:16.8684708Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 8.97kB 2022-11-18T20:52:16.8685293Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:52:16.8685700Z │ erence requests. 2022-11-18T20:52:16.8686162Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:52:16.8686664Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8687154Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:52:16.8687611Z │ version="default",} 1.0 2022-11-18T20:52:16.8688076Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8688577Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:52:16.8689005Z │ l_version="default",} 1.0 2022-11-18T20:52:16.8689472Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8689971Z │ fe3-7d6b11352b25",model_name="deeplabv3_resnet_101_scr 2022-11-18T20:52:16.8690446Z │ ipted",model_version="default",} 1.0 2022-11-18T20:52:16.8690907Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8691394Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:52:16.8691844Z │ l_version="default",} 1.0 2022-11-18T20:52:16.8692318Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8692792Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:52:16.8693234Z │ model_version="default",} 1.0 2022-11-18T20:52:16.8693916Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8694432Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:52:16.8694834Z │ sion="default",} 1.0 2022-11-18T20:52:16.8695308Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8695796Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:52:16.8696180Z │ "default",} 1.0 2022-11-18T20:52:16.8696628Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8697125Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:52:16.8697536Z │ rsion="default",} 1.0 2022-11-18T20:52:16.8697968Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8698554Z │ fe3-7d6b11352b25",model_name="resnet152",model_version 2022-11-18T20:52:16.8698981Z │ ="default",} 1.0 2022-11-18T20:52:16.8699440Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8699923Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:52:16.8700368Z │ ted",model_version="default",} 1.0 2022-11-18T20:52:16.8700737Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8701131Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:52:16.8701436Z │ rsion="default",} 1.0 2022-11-18T20:52:16.8701758Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8702141Z │ fe3-7d6b11352b25",model_name="resnet-152-batch_scripte 2022-11-18T20:52:16.8702473Z │ d",model_version="default",} 1.0 2022-11-18T20:52:16.8702810Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8703191Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:52:16.8703496Z │ on="default",} 1.0 2022-11-18T20:52:16.8703811Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8704183Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:52:16.8704483Z │ ="default",} 1.0 2022-11-18T20:52:16.8704816Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:16.8705105Z │ fe3-7d6b11352b25",model 2022-11-18T20:52:16.8705377Z │ (showing 2.05kB/8.97kB) 2022-11-18T20:52:16.8705594Z └ 2022-11-18T20:52:16.8705996Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:16.8706483Z 1ms 323µs (cache) (cache) 2ms 1ms 51µs 5ms 2022-11-18T20:52:16.8706667Z 2022-11-18T20:52:16.8824261Z √ Successful GET request 2022-11-18T20:52:16.8929693Z 2022-11-18T20:52:16.8930047Z Iteration 22/24 2022-11-18T20:52:16.9088273Z 2022-11-18T20:52:16.9088952Z → Model Zoo - Register Model 2022-11-18T20:52:27.2387294Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/distill_bert_qa_eager.mar&model_name=distill_bert_qa_eager&initial_workers=1&synchronous=true 2022-11-18T20:52:27.2388950Z 200 OK ★ 10.2s time ★ 391B↑ 363B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:27.2389656Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 97B 2022-11-18T20:52:27.2390084Z │ { 2022-11-18T20:52:27.2390533Z │ "status": "Model \"distill_bert_qa_eager\" Version: 2022-11-18T20:52:27.2390990Z │ 1.0 registered with 1 initial workers" 2022-11-18T20:52:27.2391364Z │ } 2022-11-18T20:52:27.2391666Z └ 2022-11-18T20:52:27.2392181Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:27.2392859Z 1ms 330µs (cache) (cache) 10.2s 2ms 41µs 10.2s 2022-11-18T20:52:27.2393162Z 2022-11-18T20:52:27.2530207Z √ Successful POST request 2022-11-18T20:52:27.2804412Z 2022-11-18T20:52:27.2805320Z → Model Zoo - Inference Model 2022-11-18T20:52:27.5394125Z POST http://localhost:8080/predictions/distill_bert_qa_eager 2022-11-18T20:52:27.5395307Z 200 OK ★ 208ms time ★ 360B↑ 247B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:52:27.5395929Z ┌ ↑ file ★ 80B 2022-11-18T20:52:27.5396372Z │ 2022-11-18T20:52:27.5396794Z └ 2022-11-18T20:52:27.5397354Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 13B 2022-11-18T20:52:27.5397845Z │ a nice puppet 2022-11-18T20:52:27.5408851Z └ 2022-11-18T20:52:27.5409461Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:27.5410197Z 3ms 314µs (cache) (cache) 205ms 1ms 44µs 210ms 2022-11-18T20:52:27.5410580Z 2022-11-18T20:52:27.5624694Z √ Successful POST request 2022-11-18T20:52:27.5847808Z 2022-11-18T20:52:27.5848659Z → Model Zoo - Unregister model 2022-11-18T20:52:27.7662830Z DELETE http://localhost:8081/models/distill_bert_qa_eager 2022-11-18T20:52:27.7663872Z 200 OK ★ 134ms time ★ 257B↑ 329B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:27.7664495Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 63B 2022-11-18T20:52:27.7664897Z │ { 2022-11-18T20:52:27.7665353Z │ "status": "Model \"distill_bert_qa_eager\" unregiste 2022-11-18T20:52:27.7665751Z │ red" 2022-11-18T20:52:27.7666082Z │ } 2022-11-18T20:52:27.7691156Z └ 2022-11-18T20:52:27.7691873Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:27.7692608Z 1ms 289µs (cache) (cache) 131ms 1ms 47µs 134ms 2022-11-18T20:52:27.7692947Z 2022-11-18T20:52:27.7796831Z √ Successful DELETE request 2022-11-18T20:52:27.8070138Z 2022-11-18T20:52:27.8070781Z → Model Zoo - Model Metrics 2022-11-18T20:52:27.8737959Z GET http://localhost:8082/metrics 2022-11-18T20:52:27.8739301Z 200 OK ★ 7ms time ★ 233B↑ 9.69kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:27.8744010Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 9.4kB 2022-11-18T20:52:27.8744689Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:52:27.8745135Z │ erence requests. 2022-11-18T20:52:27.8745576Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:52:27.8746084Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8746595Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:52:27.8747032Z │ version="default",} 1.0 2022-11-18T20:52:27.8747476Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8747975Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:52:27.8748417Z │ l_version="default",} 1.0 2022-11-18T20:52:27.8748886Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8749364Z │ fe3-7d6b11352b25",model_name="deeplabv3_resnet_101_scr 2022-11-18T20:52:27.8749831Z │ ipted",model_version="default",} 1.0 2022-11-18T20:52:27.8750327Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8750816Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:52:27.8751228Z │ l_version="default",} 1.0 2022-11-18T20:52:27.8751948Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8752467Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:52:27.8752914Z │ model_version="default",} 1.0 2022-11-18T20:52:27.8753361Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8753850Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:52:27.8754264Z │ sion="default",} 1.0 2022-11-18T20:52:27.8754699Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8755189Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:52:27.8755588Z │ "default",} 1.0 2022-11-18T20:52:27.8756035Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8756612Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:52:27.8757029Z │ rsion="default",} 1.0 2022-11-18T20:52:27.8757490Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8757983Z │ fe3-7d6b11352b25",model_name="resnet152",model_version 2022-11-18T20:52:27.8758396Z │ ="default",} 1.0 2022-11-18T20:52:27.8758869Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8759389Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:52:27.8759827Z │ ted",model_version="default",} 1.0 2022-11-18T20:52:27.8760323Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8760784Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:52:27.8761189Z │ rsion="default",} 1.0 2022-11-18T20:52:27.8761637Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8762125Z │ fe3-7d6b11352b25",model_name="resnet-152-batch_scripte 2022-11-18T20:52:27.8762553Z │ d",model_version="default",} 1.0 2022-11-18T20:52:27.8763019Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8763507Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:52:27.8763910Z │ on="default",} 1.0 2022-11-18T20:52:27.8764334Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8764813Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:52:27.8765204Z │ ="default",} 1.0 2022-11-18T20:52:27.8765640Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:27.8766034Z │ fe3-7d6b11352b25",model 2022-11-18T20:52:27.8766406Z │ (showing 2.05kB/9.4kB) 2022-11-18T20:52:27.8766727Z └ 2022-11-18T20:52:27.8767223Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:27.8767813Z 1ms 304µs (cache) (cache) 4ms 2ms 49µs 8ms 2022-11-18T20:52:27.8768108Z 2022-11-18T20:52:27.8768353Z √ Successful GET request 2022-11-18T20:52:27.8850578Z 2022-11-18T20:52:27.8850903Z Iteration 23/24 2022-11-18T20:52:27.9008723Z 2022-11-18T20:52:27.9009467Z → Model Zoo - Register Model 2022-11-18T20:52:41.2104344Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/bert_token_classification_no_torchscript.mar&model_name=bert_token_classification_no_torchscript&initial_workers=1&synchronous=true 2022-11-18T20:52:42.1793222Z 200 OK ★ 13.2s time ★ 429B↑ 383B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:42.1793834Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 116B 2022-11-18T20:52:42.1794176Z │ { 2022-11-18T20:52:42.1794799Z │ "status": "Model \"bert_token_classification_no_torc 2022-11-18T20:52:42.1795329Z │ hscript\" Version: 1.0 registered with 1 initial worke 2022-11-18T20:52:42.1795750Z │ rs" 2022-11-18T20:52:42.1796067Z │ } 2022-11-18T20:52:42.1797170Z └ 2022-11-18T20:52:42.1800410Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:42.1801082Z 1ms 327µs (cache) (cache) 13.2s 2ms 38µs 13.2s 2022-11-18T20:52:42.1801517Z 2022-11-18T20:52:42.1801788Z √ Successful POST request 2022-11-18T20:52:42.1802022Z 2022-11-18T20:52:42.1802273Z → Model Zoo - Inference Model 2022-11-18T20:52:42.1802747Z POST http://localhost:8080/predictions/bert_token_classification_no_torchscript 2022-11-18T20:52:42.1803413Z 200 OK ★ 255ms time ★ 376B↑ 5.45kB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:52:42.1803822Z ┌ ↑ file ★ 77B 2022-11-18T20:52:42.1804118Z │ 2022-11-18T20:52:42.1804416Z └ 2022-11-18T20:52:42.1804838Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 5.22kB 2022-11-18T20:52:42.1805200Z │ [ 2022-11-18T20:52:42.1805481Z │ [ 2022-11-18T20:52:42.1805795Z │ "[CLS]", 2022-11-18T20:52:42.1806126Z │ "B-LOC" 2022-11-18T20:52:42.1807325Z │ ], 2022-11-18T20:52:42.1807632Z │ [ 2022-11-18T20:52:42.1807961Z │ "bloomberg", 2022-11-18T20:52:42.1808299Z │ "O" 2022-11-18T20:52:42.1808587Z │ ], 2022-11-18T20:52:42.1808890Z │ [ 2022-11-18T20:52:42.1809215Z │ "has", 2022-11-18T20:52:42.1809520Z │ "I-ORG" 2022-11-18T20:52:42.1809831Z │ ], 2022-11-18T20:52:42.1810134Z │ [ 2022-11-18T20:52:42.1810447Z │ "decided", 2022-11-18T20:52:42.1810757Z │ "B-PER" 2022-11-18T20:52:42.1811065Z │ ], 2022-11-18T20:52:42.1811364Z │ [ 2022-11-18T20:52:42.1811653Z │ "to", 2022-11-18T20:52:42.1811968Z │ "O" 2022-11-18T20:52:42.1812272Z │ ], 2022-11-18T20:52:42.1812549Z │ [ 2022-11-18T20:52:42.1812870Z │ "publish", 2022-11-18T20:52:42.1813204Z │ "I-PER" 2022-11-18T20:52:42.1813515Z │ ], 2022-11-18T20:52:42.1813795Z │ [ 2022-11-18T20:52:42.1814101Z │ "a", 2022-11-18T20:52:42.1814422Z │ "I-MISC" 2022-11-18T20:52:42.1814728Z │ ], 2022-11-18T20:52:42.1815024Z │ [ 2022-11-18T20:52:42.1815329Z │ "new", 2022-11-18T20:52:42.1815647Z │ "B-PER" 2022-11-18T20:52:42.1815937Z │ ], 2022-11-18T20:52:42.1816233Z │ [ 2022-11-18T20:52:42.1816559Z │ "report", 2022-11-18T20:52:42.1816862Z │ "O" 2022-11-18T20:52:42.1817166Z │ ], 2022-11-18T20:52:42.1817462Z │ [ 2022-11-18T20:52:42.1817771Z │ "on", 2022-11-18T20:52:42.1818066Z │ "O" 2022-11-18T20:52:42.1818371Z │ ], 2022-11-18T20:52:42.1818668Z │ [ 2022-11-18T20:52:42.1818960Z │ "global", 2022-11-18T20:52:42.1819282Z │ "B-PER" 2022-11-18T20:52:42.1819595Z │ ], 2022-11-18T20:52:42.1819871Z │ [ 2022-11-18T20:52:42.1820191Z │ "economic", 2022-11-18T20:52:42.1820515Z │ "O" 2022-11-18T20:52:42.1820818Z │ ], 2022-11-18T20:52:42.1821100Z │ [ 2022-11-18T20:52:42.1821419Z │ "situation", 2022-11-18T20:52:42.1821756Z │ "B-LOC" 2022-11-18T20:52:42.1822043Z │ ], 2022-11-18T20:52:42.1823138Z │ [ 2022-11-18T20:52:42.1823771Z │ ".", 2022-11-18T20:52:42.1824721Z │ "B-PER" 2022-11-18T20:52:42.1825033Z │ ], 2022-11-18T20:52:42.1825337Z │ [ 2022-11-18T20:52:42.1825650Z │ "[SEP]", 2022-11-18T20:52:42.1825977Z │ "B-PER" 2022-11-18T20:52:42.1826290Z │ ], 2022-11-18T20:52:42.1826590Z │ [ 2022-11-18T20:52:42.1826932Z │ "[PAD]", 2022-11-18T20:52:42.1827237Z │ "I-ORG" 2022-11-18T20:52:42.1827550Z │ ], 2022-11-18T20:52:42.1827845Z │ [ 2022-11-18T20:52:42.1828135Z │ "[PAD]", 2022-11-18T20:52:42.1828460Z │ "B-MISC" 2022-11-18T20:52:42.1828771Z │ ], 2022-11-18T20:52:42.1829051Z │ [ 2022-11-18T20:52:42.1829366Z │ "[PAD]", 2022-11-18T20:52:42.1829686Z │ "B-MISC" 2022-11-18T20:52:42.1829993Z │ ], 2022-11-18T20:52:42.1830274Z │ [ 2022-11-18T20:52:42.1830580Z │ "[PAD]", 2022-11-18T20:52:42.1830897Z │ "B-MISC" 2022-11-18T20:52:42.1831187Z │ ], 2022-11-18T20:52:42.1831501Z │ [ 2022-11-18T20:52:42.1831808Z │ "[PAD]", 2022-11-18T20:52:42.1832126Z │ "B-MISC" 2022-11-18T20:52:42.1832411Z │ ], 2022-11-18T20:52:42.1832710Z │ [ 2022-11-18T20:52:42.1833018Z │ "[PAD]", 2022-11-18T20:52:42.1833556Z │ "B-MISC" 2022-11-18T20:52:42.1833883Z │ ], 2022-11-18T20:52:42.1834183Z │ [ 2022-11-18T20:52:42.1834487Z │ "[PAD]", 2022-11-18T20:52:42.1834787Z │ "B-MISC" 2022-11-18T20:52:42.1835092Z │ ], 2022-11-18T20:52:42.1835389Z │ [ 2022-11-18T20:52:42.1835674Z │ "[PAD]", 2022-11-18T20:52:42.1835994Z │ "B-MISC" 2022-11-18T20:52:42.1836307Z │ ], 2022-11-18T20:52:42.1836584Z │ [ 2022-11-18T20:52:42.1836890Z │ "[PAD]", 2022-11-18T20:52:42.1837204Z │ "B-MISC" 2022-11-18T20:52:42.1837511Z │ ], 2022-11-18T20:52:42.1837791Z │ [ 2022-11-18T20:52:42.1838099Z │ "[PAD]", 2022-11-18T20:52:42.1838416Z │ "B-MISC" 2022-11-18T20:52:42.1838698Z │ ], 2022-11-18T20:52:42.1839108Z │ [ 2022-11-18T20:52:42.1839418Z │ "[PAD]", 2022-11-18T20:52:42.1839732Z │ "B-MISC" 2022-11-18T20:52:42.1840019Z │ ], 2022-11-18T20:52:42.1840315Z │ [ 2022-11-18T20:52:42.1840622Z │ "[PAD]", 2022-11-18T20:52:42.1840928Z │ "B-MISC" 2022-11-18T20:52:42.1841232Z │ ], 2022-11-18T20:52:42.1841532Z │ [ 2022-11-18T20:52:42.1841840Z │ "[PAD]", 2022-11-18T20:52:42.1842134Z │ "B-MISC" 2022-11-18T20:52:42.1842438Z │ ], 2022-11-18T20:52:42.1842740Z │ [ 2022-11-18T20:52:42.1843043Z │ "[PAD]", 2022-11-18T20:52:42.1843356Z │ "B-MISC" 2022-11-18T20:52:42.1843662Z │ ], 2022-11-18T20:52:42.1843941Z │ [ 2022-11-18T20:52:42.1844249Z │ "[PAD]", 2022-11-18T20:52:42.1844564Z │ "B-MISC" 2022-11-18T20:52:42.1844874Z │ ], 2022-11-18T20:52:42.1845156Z │ [ 2022-11-18T20:52:42.1845459Z │ "[PAD]", 2022-11-18T20:52:42.1845772Z │ "B-MISC" 2022-11-18T20:52:42.1846059Z │ ], 2022-11-18T20:52:42.1846363Z │ [ 2022-11-18T20:52:42.1846673Z │ "[PAD]", 2022-11-18T20:52:42.1846984Z │ "B-MISC" 2022-11-18T20:52:42.1847268Z │ ], 2022-11-18T20:52:42.1847567Z │ [ 2022-11-18T20:52:42.1847903Z │ "[PAD]", 2022-11-18T20:52:42.1848210Z │ "B-MISC" 2022-11-18T20:52:42.1848515Z │ ], 2022-11-18T20:52:42.1848811Z │ [ 2022-11-18T20:52:42.1849119Z │ "[PAD]", 2022-11-18T20:52:42.1849419Z │ "B-MISC" 2022-11-18T20:52:42.1849726Z │ ], 2022-11-18T20:52:42.1850020Z │ [ 2022-11-18T20:52:42.1850313Z │ "[PAD]", 2022-11-18T20:52:42.1850629Z │ "B-MISC" 2022-11-18T20:52:42.1850939Z │ ], 2022-11-18T20:52:42.1851216Z │ [ 2022-11-18T20:52:42.1851532Z │ "[PAD]", 2022-11-18T20:52:42.1851845Z │ "B-MISC" 2022-11-18T20:52:42.1852152Z │ ], 2022-11-18T20:52:42.1852429Z │ [ 2022-11-18T20:52:42.1854361Z │ "[PAD]", 2022-11-18T20:52:42.1854694Z │ "B-MISC" 2022-11-18T20:52:42.1854982Z │ ], 2022-11-18T20:52:42.1855300Z │ [ 2022-11-18T20:52:42.1855609Z │ "[PAD]", 2022-11-18T20:52:42.1855924Z │ "B-MISC" 2022-11-18T20:52:42.1856215Z │ ], 2022-11-18T20:52:42.1856515Z │ [ 2022-11-18T20:52:42.1859008Z │ "[PAD]", 2022-11-18T20:52:42.1869189Z │ "B-MISC" 2022-11-18T20:52:42.1869598Z │ ], 2022-11-18T20:52:42.1869924Z │ [ 2022-11-18T20:52:42.1870225Z │ "[PAD]", 2022-11-18T20:52:42.1870561Z │ "B-MISC" 2022-11-18T20:52:42.1870872Z │ ], 2022-11-18T20:52:42.1871175Z │ [ 2022-11-18T20:52:42.1871467Z │ "[PAD]", 2022-11-18T20:52:42.1871790Z │ "B-MISC" 2022-11-18T20:52:42.1872100Z │ ], 2022-11-18T20:52:42.1872381Z │ [ 2022-11-18T20:52:42.1872690Z │ "[PAD]", 2022-11-18T20:52:42.1873013Z │ "B-MISC" 2022-11-18T20:52:42.1873322Z │ ], 2022-11-18T20:52:42.1873609Z │ [ 2022-11-18T20:52:42.1873919Z │ "[PAD]", 2022-11-18T20:52:42.1874237Z │ "B-MISC" 2022-11-18T20:52:42.1874522Z │ ], 2022-11-18T20:52:42.1874834Z │ [ 2022-11-18T20:52:42.1875142Z │ "[PAD]", 2022-11-18T20:52:42.1875461Z │ "B-MISC" 2022-11-18T20:52:42.1875747Z │ ], 2022-11-18T20:52:42.1876049Z │ [ 2022-11-18T20:52:42.1876358Z │ "[PAD]", 2022-11-18T20:52:42.1876866Z │ "B-MISC" 2022-11-18T20:52:42.1877193Z │ ], 2022-11-18T20:52:42.1877494Z │ [ 2022-11-18T20:52:42.1877786Z │ "[PAD]", 2022-11-18T20:52:42.1878105Z │ "B-MISC" 2022-11-18T20:52:42.1878412Z │ ], 2022-11-18T20:52:42.1878711Z │ [ 2022-11-18T20:52:42.1879001Z │ "[PAD]", 2022-11-18T20:52:42.1879317Z │ "B-MISC" 2022-11-18T20:52:42.1879625Z │ ], 2022-11-18T20:52:42.1879907Z │ [ 2022-11-18T20:52:42.1880215Z │ "[PAD]", 2022-11-18T20:52:42.1880532Z │ "B-MISC" 2022-11-18T20:52:42.1880837Z │ ], 2022-11-18T20:52:42.1881114Z │ [ 2022-11-18T20:52:42.1881422Z │ "[PAD]", 2022-11-18T20:52:42.1881740Z │ "B-MISC" 2022-11-18T20:52:42.1882027Z │ ], 2022-11-18T20:52:42.1882472Z │ [ 2022-11-18T20:52:42.1882813Z │ "[PAD]", 2022-11-18T20:52:42.1883134Z │ "B-MISC" 2022-11-18T20:52:42.1883423Z │ ], 2022-11-18T20:52:42.1883725Z │ [ 2022-11-18T20:52:42.1884042Z │ "[PAD]", 2022-11-18T20:52:42.1884346Z │ "B-MISC" 2022-11-18T20:52:42.1884657Z │ ], 2022-11-18T20:52:42.1884955Z │ [ 2022-11-18T20:52:42.1885243Z │ "[PAD]", 2022-11-18T20:52:42.1885562Z │ "B-MISC" 2022-11-18T20:52:42.1885869Z │ ], 2022-11-18T20:52:42.1886168Z │ [ 2022-11-18T20:52:42.1886457Z │ "[PAD]", 2022-11-18T20:52:42.1886773Z │ "B-MISC" 2022-11-18T20:52:42.1887078Z │ ], 2022-11-18T20:52:42.1887356Z │ [ 2022-11-18T20:52:42.1887665Z │ "[PAD]", 2022-11-18T20:52:42.1887978Z │ "B-MISC" 2022-11-18T20:52:42.1888285Z │ ], 2022-11-18T20:52:42.1888566Z │ [ 2022-11-18T20:52:42.1888873Z │ "[PAD]", 2022-11-18T20:52:42.1889193Z │ "B-MISC" 2022-11-18T20:52:42.1889484Z │ ], 2022-11-18T20:52:42.1889798Z │ [ 2022-11-18T20:52:42.1890110Z │ "[PAD]", 2022-11-18T20:52:42.1890427Z │ "B-MISC" 2022-11-18T20:52:42.1890711Z │ ], 2022-11-18T20:52:42.1891009Z │ [ 2022-11-18T20:52:42.1891316Z │ "[PAD]", 2022-11-18T20:52:42.1891621Z │ "B-MISC" 2022-11-18T20:52:42.1891929Z │ ], 2022-11-18T20:52:42.1892229Z │ [ 2022-11-18T20:52:42.1892517Z │ "[PAD]", 2022-11-18T20:52:42.1892836Z │ "B-MISC" 2022-11-18T20:52:42.1893141Z │ ], 2022-11-18T20:52:42.1893438Z │ [ 2022-11-18T20:52:42.1893727Z │ "[PAD]", 2022-11-18T20:52:42.1894040Z │ "B-MISC" 2022-11-18T20:52:42.1894345Z │ ], 2022-11-18T20:52:42.1894622Z │ [ 2022-11-18T20:52:42.1894929Z │ "[PAD]", 2022-11-18T20:52:42.1895298Z │ (showing 2.05kB/5.22kB) 2022-11-18T20:52:42.1895617Z └ 2022-11-18T20:52:42.1896295Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:42.1896931Z 2ms 290µs (cache) (cache) 252ms 1ms 64µs 257ms 2022-11-18T20:52:42.1897211Z 2022-11-18T20:52:42.1897463Z √ Successful POST request 2022-11-18T20:52:42.1897883Z √ Test expected JSON response 2022-11-18T20:52:42.2109305Z 2022-11-18T20:52:42.2110054Z → Model Zoo - Unregister model 2022-11-18T20:52:42.4464984Z DELETE http://localhost:8081/models/bert_token_classification_no_torchscript 2022-11-18T20:52:42.4466155Z 200 OK ★ 187ms time ★ 276B↑ 348B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:42.4466626Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 82B 2022-11-18T20:52:42.4466954Z │ { 2022-11-18T20:52:42.4467344Z │ "status": "Model \"bert_token_classification_no_torc 2022-11-18T20:52:42.4467750Z │ hscript\" unregistered" 2022-11-18T20:52:42.4468056Z │ } 2022-11-18T20:52:42.4479917Z └ 2022-11-18T20:52:42.4480434Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:42.4481024Z 1ms 295µs (cache) (cache) 184ms 2ms 40µs 188ms 2022-11-18T20:52:42.4481259Z 2022-11-18T20:52:42.4552111Z √ Successful DELETE request 2022-11-18T20:52:42.4813955Z 2022-11-18T20:52:42.4815712Z → Model Zoo - Model Metrics 2022-11-18T20:52:42.5345830Z GET http://localhost:8082/metrics 2022-11-18T20:52:42.5346800Z 200 OK ★ 4ms time ★ 233B↑ 10.17kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:42.5347275Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 9.88kB 2022-11-18T20:52:42.5347774Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:52:42.5348150Z │ erence requests. 2022-11-18T20:52:42.5348531Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:52:42.5348972Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5349412Z │ fe3-7d6b11352b25",model_name="bert_token_classificatio 2022-11-18T20:52:42.5349844Z │ n_no_torchscript",model_version="default",} 1.0 2022-11-18T20:52:42.5350540Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5350991Z │ fe3-7d6b11352b25",model_name="alexnet_scripted",model_ 2022-11-18T20:52:42.5351346Z │ version="default",} 1.0 2022-11-18T20:52:42.5351771Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5352213Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:52:42.5352581Z │ l_version="default",} 1.0 2022-11-18T20:52:42.5353001Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5353442Z │ fe3-7d6b11352b25",model_name="deeplabv3_resnet_101_scr 2022-11-18T20:52:42.5353852Z │ ipted",model_version="default",} 1.0 2022-11-18T20:52:42.5354251Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5354686Z │ fe3-7d6b11352b25",model_name="my_text_classifier",mode 2022-11-18T20:52:42.5355059Z │ l_version="default",} 1.0 2022-11-18T20:52:42.5355464Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5355891Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:52:42.5356284Z │ model_version="default",} 1.0 2022-11-18T20:52:42.5356761Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5357205Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:52:42.5357546Z │ sion="default",} 1.0 2022-11-18T20:52:42.5357949Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5358384Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:52:42.5358730Z │ "default",} 1.0 2022-11-18T20:52:42.5359101Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5359537Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:52:42.5359922Z │ rsion="default",} 1.0 2022-11-18T20:52:42.5360302Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5360732Z │ fe3-7d6b11352b25",model_name="resnet152",model_version 2022-11-18T20:52:42.5361086Z │ ="default",} 1.0 2022-11-18T20:52:42.5361479Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5365786Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:52:42.5366486Z │ ted",model_version="default",} 1.0 2022-11-18T20:52:42.5366930Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5367368Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:52:42.5367712Z │ rsion="default",} 1.0 2022-11-18T20:52:42.5368114Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5368556Z │ fe3-7d6b11352b25",model_name="resnet-152-batch_scripte 2022-11-18T20:52:42.5368949Z │ d",model_version="default",} 1.0 2022-11-18T20:52:42.5369350Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:42.5369784Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:52:42.5370143Z │ on="default",} 1.0 2022-11-18T20:52:42.5370628Z │ ts_inference_requests_total{uuid="3cb3e369-1ec 2022-11-18T20:52:42.5370977Z │ (showing 2.05kB/9.88kB) 2022-11-18T20:52:42.5371329Z └ 2022-11-18T20:52:42.5371811Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:42.5372344Z 1ms 313µs (cache) (cache) 1ms 1ms 50µs 5ms 2022-11-18T20:52:42.5372581Z 2022-11-18T20:52:42.5430273Z √ Successful GET request 2022-11-18T20:52:42.5593464Z 2022-11-18T20:52:42.5594092Z Iteration 24/24 2022-11-18T20:52:42.5752672Z 2022-11-18T20:52:42.5753364Z → Model Zoo - Register Model 2022-11-18T20:52:55.6566855Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/bert_seqc_without_torchscript.mar&model_name=bert_seqc_without_torchscript&initial_workers=1&synchronous=true 2022-11-18T20:52:55.6568241Z 200 OK ★ 13s time ★ 407B↑ 372B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:55.6568959Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 105B 2022-11-18T20:52:55.6569375Z │ { 2022-11-18T20:52:55.6569807Z │ "status": "Model \"bert_seqc_without_torchscript\" V 2022-11-18T20:52:55.6570291Z │ ersion: 1.0 registered with 1 initial workers" 2022-11-18T20:52:55.6570689Z │ } 2022-11-18T20:52:55.6578944Z └ 2022-11-18T20:52:55.6579472Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:55.6580119Z 1ms 316µs (cache) (cache) 13s 2ms 52µs 13s 2022-11-18T20:52:55.6580421Z 2022-11-18T20:52:55.6724263Z √ Successful POST request 2022-11-18T20:52:55.6998790Z 2022-11-18T20:52:55.6999654Z → Model Zoo - Inference Model 2022-11-18T20:52:55.9985674Z POST http://localhost:8080/predictions/bert_seqc_without_torchscript 2022-11-18T20:52:55.9986887Z 200 OK ★ 248ms time ★ 365B↑ 246B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:52:55.9987512Z ┌ ↑ file ★ 77B 2022-11-18T20:52:55.9987990Z │ 2022-11-18T20:52:55.9988379Z └ 2022-11-18T20:52:55.9988896Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 12B 2022-11-18T20:52:55.9989419Z │ Not Accepted 2022-11-18T20:52:56.0007511Z └ 2022-11-18T20:52:56.0008231Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:56.0008956Z 3ms 314µs (cache) (cache) 245ms 1ms 41µs 250ms 2022-11-18T20:52:56.0009341Z 2022-11-18T20:52:56.0164865Z √ Successful POST request 2022-11-18T20:52:56.0189926Z √ Test expected TEXT response 2022-11-18T20:52:56.0445158Z 2022-11-18T20:52:56.0446098Z → Model Zoo - Unregister model 2022-11-18T20:52:56.2629478Z DELETE http://localhost:8081/models/bert_seqc_without_torchscript 2022-11-18T20:52:56.2630719Z 200 OK ★ 170ms time ★ 265B↑ 337B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:56.2631448Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 71B 2022-11-18T20:52:56.2631920Z │ { 2022-11-18T20:52:56.2650903Z │ "status": "Model \"bert_seqc_without_torchscript\" u 2022-11-18T20:52:56.2651809Z │ nregistered" 2022-11-18T20:52:56.2652092Z │ } 2022-11-18T20:52:56.2652327Z └ 2022-11-18T20:52:56.2653063Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:56.2653664Z 1ms 295µs (cache) (cache) 167ms 1ms 44µs 171ms 2022-11-18T20:52:56.2653873Z 2022-11-18T20:52:56.2704629Z √ Successful DELETE request 2022-11-18T20:52:56.2982880Z 2022-11-18T20:52:56.2983509Z → Model Zoo - Model Metrics 2022-11-18T20:52:56.3541588Z GET http://localhost:8082/metrics 2022-11-18T20:52:56.3542767Z 200 OK ★ 5ms time ★ 233B↑ 10.63kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:52:56.3543524Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 10.33kB 2022-11-18T20:52:56.3544108Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:52:56.3544530Z │ erence requests. 2022-11-18T20:52:56.3545222Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:52:56.3545701Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3546204Z │ fe3-7d6b11352b25",model_name="bert_token_classificatio 2022-11-18T20:52:56.3546697Z │ n_no_torchscript",model_version="default",} 1.0 2022-11-18T20:52:56.3547192Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3547662Z │ fe3-7d6b11352b25",model_name="resnet-18_scripted",mode 2022-11-18T20:52:56.3548098Z │ l_version="default",} 1.0 2022-11-18T20:52:56.3548554Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3549044Z │ fe3-7d6b11352b25",model_name="deeplabv3_resnet_101_scr 2022-11-18T20:52:56.3549483Z │ ipted",model_version="default",} 1.0 2022-11-18T20:52:56.3549953Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3550443Z │ fe3-7d6b11352b25",model_name="squeezenet1_1_scripted", 2022-11-18T20:52:56.3550878Z │ model_version="default",} 1.0 2022-11-18T20:52:56.3551331Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3551815Z │ fe3-7d6b11352b25",model_name="squeezenet1_1",model_ver 2022-11-18T20:52:56.3552229Z │ sion="default",} 1.0 2022-11-18T20:52:56.3552664Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3553149Z │ fe3-7d6b11352b25",model_name="maskrcnn",model_version= 2022-11-18T20:52:56.3553542Z │ "default",} 1.0 2022-11-18T20:52:56.3553976Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3554442Z │ fe3-7d6b11352b25",model_name="vgg16_scripted",model_ve 2022-11-18T20:52:56.3554853Z │ rsion="default",} 1.0 2022-11-18T20:52:56.3555299Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3555776Z │ fe3-7d6b11352b25",model_name="resnet152",model_version 2022-11-18T20:52:56.3556148Z │ ="default",} 1.0 2022-11-18T20:52:56.3556586Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3557081Z │ fe3-7d6b11352b25",model_name="my_text_classifier_scrip 2022-11-18T20:52:56.3557553Z │ ted",model_version="default",} 1.0 2022-11-18T20:52:56.3558050Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3558538Z │ fe3-7d6b11352b25",model_name="mnist_scripted",model_ve 2022-11-18T20:52:56.3558944Z │ rsion="default",} 1.0 2022-11-18T20:52:56.3559365Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3559843Z │ fe3-7d6b11352b25",model_name="resnet-152-batch_scripte 2022-11-18T20:52:56.3560279Z │ d",model_version="default",} 1.0 2022-11-18T20:52:56.3560734Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3561190Z │ fe3-7d6b11352b25",model_name="densenet161",model_versi 2022-11-18T20:52:56.3561580Z │ on="default",} 1.0 2022-11-18T20:52:56.3562010Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3562473Z │ fe3-7d6b11352b25",model_name="resnet-18",model_version 2022-11-18T20:52:56.3562850Z │ ="default",} 1.0 2022-11-18T20:52:56.3563278Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3563887Z │ fe3-7d6b11352b25",model_name="fcn_resnet_101",model_ve 2022-11-18T20:52:56.3564302Z │ rsion="default",} 1.0 2022-11-18T20:52:56.3564724Z │ ts_inference_requests_total{uuid="3cb3e369-1ec5-4252-b 2022-11-18T20:52:56.3565096Z │ fe3 2022-11-18T20:52:56.3565449Z │ (showing 2.05kB/10.33kB) 2022-11-18T20:52:56.3565755Z └ 2022-11-18T20:52:56.3566263Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:52:56.3566840Z 1ms 304µs (cache) (cache) 1ms 2ms 49µs 5ms 2022-11-18T20:52:56.3567118Z 2022-11-18T20:52:56.3662720Z √ Successful GET request 2022-11-18T20:52:57.4393047Z 2022-11-18T20:52:57.4394185Z ┌─────────────────────────┬────────────────────┬───────────────────┐ 2022-11-18T20:52:57.4395307Z │ │ executed │ failed │ 2022-11-18T20:52:57.4395723Z ├─────────────────────────┼────────────────────┼───────────────────┤ 2022-11-18T20:52:57.4396103Z │ iterations │ 24 │ 0 │ 2022-11-18T20:52:57.4396499Z ├─────────────────────────┼────────────────────┼───────────────────┤ 2022-11-18T20:52:57.4396862Z │ requests │ 96 │ 0 │ 2022-11-18T20:52:57.4397227Z ├─────────────────────────┼────────────────────┼───────────────────┤ 2022-11-18T20:52:57.4397601Z │ test-scripts │ 96 │ 0 │ 2022-11-18T20:52:57.4397992Z ├─────────────────────────┼────────────────────┼───────────────────┤ 2022-11-18T20:52:57.4398366Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:52:57.4398746Z ├─────────────────────────┼────────────────────┼───────────────────┤ 2022-11-18T20:52:57.4399105Z │ assertions │ 113 │ 0 │ 2022-11-18T20:52:57.4399511Z ├─────────────────────────┴────────────────────┴───────────────────┤ 2022-11-18T20:52:57.4399856Z │ total run duration: 4m 56.4s │ 2022-11-18T20:52:57.4400264Z ├──────────────────────────────────────────────────────────────────┤ 2022-11-18T20:52:57.4400643Z │ total data received: 10.04MB (approx) │ 2022-11-18T20:52:57.4401043Z ├──────────────────────────────────────────────────────────────────┤ 2022-11-18T20:52:57.4401406Z │ average response time: 2.9s [min: 4ms, max: 25.5s, s.d.: 5.7s] │ 2022-11-18T20:52:57.4401810Z ├──────────────────────────────────────────────────────────────────┤ 2022-11-18T20:52:57.4402213Z │ average DNS lookup time: 3ms [min: 234µs, max: 10ms, s.d.: 4ms] │ 2022-11-18T20:52:57.4402598Z ├──────────────────────────────────────────────────────────────────┤ 2022-11-18T20:52:57.4402974Z │ average first byte time: 2.9s [min: 1ms, max: 25.5s, s.d.: 5.7s] │ 2022-11-18T20:52:57.4403398Z └──────────────────────────────────────────────────────────────────┘ 2022-11-18T20:52:57.5289068Z TorchServe has stopped. 2022-11-18T20:53:19.5804338Z newman 2022-11-18T20:53:19.5804766Z 2022-11-18T20:53:19.5806562Z inference 2022-11-18T20:53:19.5891269Z 2022-11-18T20:53:19.5891960Z → Model Zoo - Register Model 2022-11-18T20:53:27.0999162Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/densenet161_scripted.mar&model_name=densenet161_scripted&initial_workers=1&synchronous=true 2022-11-18T20:53:27.1001442Z 200 OK ★ 7.4s time ★ 389B↑ 362B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:27.1005540Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 96B 2022-11-18T20:53:27.1007598Z │ { 2022-11-18T20:53:27.1008012Z │ "status": "Model \"densenet161_scripted\" Version: 1 2022-11-18T20:53:27.1008437Z │ .0 registered with 1 initial workers" 2022-11-18T20:53:27.1008754Z │ } 2022-11-18T20:53:27.1088233Z └ 2022-11-18T20:53:27.1089146Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:53:27.1089720Z 35ms 7ms 9ms 581µs 7.3s 9ms 481µs 7.4s 2022-11-18T20:53:27.1089974Z 2022-11-18T20:53:27.1492307Z √ Successful POST request 2022-11-18T20:53:28.1144688Z 2022-11-18T20:53:28.1148356Z → Model Zoo - Inference Model 2022-11-18T20:53:28.9042674Z POST http://localhost:8080/predictions/densenet161_scripted 2022-11-18T20:53:28.9043773Z 200 OK ★ 1657ms time ★ 111.25kB↑ 417B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:53:28.9046016Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:53:28.9047771Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:53:28.9048169Z └ 2022-11-18T20:53:28.9048849Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 182B 2022-11-18T20:53:28.9050036Z │ { 2022-11-18T20:53:28.9050422Z │ "tabby": 0.4666188061237335, 2022-11-18T20:53:28.9050854Z │ "tiger_cat": 0.46449077129364014, 2022-11-18T20:53:28.9051328Z │ "Egyptian_cat": 0.06614049524068832, 2022-11-18T20:53:28.9051790Z │ "lynx": 0.0012924447655677795, 2022-11-18T20:53:28.9052215Z │ "plastic_bag": 0.00022909819381311536 2022-11-18T20:53:28.9052593Z │ } 2022-11-18T20:53:28.9078087Z └ 2022-11-18T20:53:28.9078650Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:53:28.9079346Z 7ms 1ms 356µs 1ms 1648ms 4ms 93µs 1664ms 2022-11-18T20:53:28.9079639Z 2022-11-18T20:53:28.9185654Z √ Successful POST request 2022-11-18T20:53:28.9198609Z √ Test expected JSON response 2022-11-18T20:53:28.9402924Z 2022-11-18T20:53:28.9403777Z → Model Zoo - Unregister model 2022-11-18T20:53:29.1007847Z DELETE http://localhost:8081/models/densenet161_scripted 2022-11-18T20:53:29.1008994Z 200 OK ★ 111ms time ★ 256B↑ 328B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:29.1009696Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 62B 2022-11-18T20:53:29.1011124Z │ { 2022-11-18T20:53:29.1011581Z │ "status": "Model \"densenet161_scripted\" unregister 2022-11-18T20:53:29.1012030Z │ ed" 2022-11-18T20:53:29.1012393Z │ } 2022-11-18T20:53:29.1028185Z └ 2022-11-18T20:53:29.1029012Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:53:29.1029734Z 2ms 802µs (cache) (cache) 106ms 2ms 88µs 112ms 2022-11-18T20:53:29.1030094Z 2022-11-18T20:53:29.1119645Z √ Successful DELETE request 2022-11-18T20:53:29.1220822Z 2022-11-18T20:53:29.1221258Z → Model Zoo - Model Metrics 2022-11-18T20:53:29.1903597Z GET http://localhost:8082/metrics 2022-11-18T20:53:29.1904682Z 200 OK ★ 18ms time ★ 233B↑ 1.09kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:29.1905286Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 798B 2022-11-18T20:53:29.1905817Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:53:29.1906490Z │ erence requests. 2022-11-18T20:53:29.1906909Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:53:29.1907406Z │ ts_inference_requests_total{uuid="f68e86ef-8f56-4c90-b 2022-11-18T20:53:29.1907922Z │ 5a7-28139cd475c5",model_name="densenet161_scripted",mo 2022-11-18T20:53:29.1908368Z │ del_version="default",} 1.0 2022-11-18T20:53:29.1908820Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:53:29.1909278Z │ ference duration in microseconds 2022-11-18T20:53:29.1909752Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:53:29.1910253Z │ ts_inference_latency_microseconds{uuid="f68e86ef-8f56- 2022-11-18T20:53:29.1910737Z │ 4c90-b5a7-28139cd475c5",model_name="densenet161_script 2022-11-18T20:53:29.1911195Z │ ed",model_version="default",} 1622062.1 2022-11-18T20:53:29.1911693Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:53:29.1912124Z │ duration in microseconds 2022-11-18T20:53:29.1912566Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:53:29.1913056Z │ ts_queue_latency_microseconds{uuid="f68e86ef-8f56-4c90 2022-11-18T20:53:29.1913556Z │ -b5a7-28139cd475c5",model_name="densenet161_scripted", 2022-11-18T20:53:29.1914007Z │ model_version="default",} 345.9 2022-11-18T20:53:29.1921948Z └ 2022-11-18T20:53:29.1922477Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:53:29.1923101Z 2ms 1ms 260µs 262µs 13ms 2ms 70µs 19ms 2022-11-18T20:53:29.1923406Z 2022-11-18T20:53:29.2066170Z √ Successful GET request 2022-11-18T20:53:29.4234419Z 2022-11-18T20:53:29.4235347Z ┌─────────────────────────┬───────────────────┬──────────────────┐ 2022-11-18T20:53:29.4245417Z │ │ executed │ failed │ 2022-11-18T20:53:29.4246104Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:53:29.4246600Z │ iterations │ 1 │ 0 │ 2022-11-18T20:53:29.4247131Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:53:29.4247603Z │ requests │ 4 │ 0 │ 2022-11-18T20:53:29.4248087Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:53:29.4248571Z │ test-scripts │ 4 │ 0 │ 2022-11-18T20:53:29.4249068Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:53:29.4249546Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:53:29.4250032Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:53:29.4250491Z │ assertions │ 5 │ 0 │ 2022-11-18T20:53:29.4250981Z ├─────────────────────────┴───────────────────┴──────────────────┤ 2022-11-18T20:53:29.4251452Z │ total run duration: 9.6s │ 2022-11-18T20:53:29.4252195Z ├────────────────────────────────────────────────────────────────┤ 2022-11-18T20:53:29.4252679Z │ total data received: 1.14kB (approx) │ 2022-11-18T20:53:29.4253287Z ├────────────────────────────────────────────────────────────────┤ 2022-11-18T20:53:29.4253785Z │ average response time: 2.2s [min: 18ms, max: 7.4s, s.d.: 3s] │ 2022-11-18T20:53:29.4254267Z ├────────────────────────────────────────────────────────────────┤ 2022-11-18T20:53:29.4254767Z │ average DNS lookup time: 3ms [min: 260µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:53:29.4255282Z ├────────────────────────────────────────────────────────────────┤ 2022-11-18T20:53:29.4255759Z │ average first byte time: 2.2s [min: 13ms, max: 7.3s, s.d.: 3s] │ 2022-11-18T20:53:29.4256239Z └────────────────────────────────────────────────────────────────┘ 2022-11-18T20:53:29.5355970Z TorchServe has stopped. 2022-11-18T20:53:50.9924283Z newman 2022-11-18T20:53:50.9924714Z 2022-11-18T20:53:50.9926817Z https_test_collection 2022-11-18T20:53:51.0042964Z 2022-11-18T20:53:51.0043757Z → HTTPS Inference API Description 2022-11-18T20:53:51.4018724Z OPTIONS https://localhost:8443 2022-11-18T20:53:51.4020176Z 200 OK ★ 255ms time ★ 230B↑ 23.68kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:51.4033394Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 23.41kB 2022-11-18T20:53:51.4033849Z │ { 2022-11-18T20:53:51.4034183Z │ "openapi": "3.0.1", 2022-11-18T20:53:51.4034494Z │ "info": { 2022-11-18T20:53:51.4034829Z │ "title": "TorchServe APIs", 2022-11-18T20:53:51.4035265Z │ "description": "TorchServe is a flexible and easy 2022-11-18T20:53:51.4035697Z │ to use tool for serving deep learning models", 2022-11-18T20:53:51.4036057Z │ "version": "0.6.1\r" 2022-11-18T20:53:51.4036377Z │ }, 2022-11-18T20:53:51.4036671Z │ "paths": { 2022-11-18T20:53:51.4036966Z │ "/": { 2022-11-18T20:53:51.4037249Z │ "options": { 2022-11-18T20:53:51.4037641Z │ "description": "Get openapi description.", 2022-11-18T20:53:51.4038092Z │ "operationId": "apiDescription", 2022-11-18T20:53:51.4038465Z │ "parameters": [], 2022-11-18T20:53:51.4038776Z │ "responses": { 2022-11-18T20:53:51.4039103Z │ "200": { 2022-11-18T20:53:51.4039488Z │ "description": "A openapi 3.0.1 descriptor 2022-11-18T20:53:51.4039801Z │ ", 2022-11-18T20:53:51.4040104Z │ "content": { 2022-11-18T20:53:51.4040456Z │ "application/json": { 2022-11-18T20:53:51.4040807Z │ "schema": { 2022-11-18T20:53:51.4041130Z │ "type": "object", 2022-11-18T20:53:51.4041473Z │ "required": [ 2022-11-18T20:53:51.4041816Z │ "openapi", 2022-11-18T20:53:51.4042120Z │ "info", 2022-11-18T20:53:51.4042441Z │ "paths" 2022-11-18T20:53:51.4042749Z │ ], 2022-11-18T20:53:51.4043086Z │ "properties": { 2022-11-18T20:53:51.4043426Z │ "openapi": { 2022-11-18T20:53:51.4043778Z │ "type": "string" 2022-11-18T20:53:51.4044106Z │ }, 2022-11-18T20:53:51.4044742Z │ "info": { 2022-11-18T20:53:51.4045087Z │ "type": "object" 2022-11-18T20:53:51.4045409Z │ }, 2022-11-18T20:53:51.4045729Z │ "paths": { 2022-11-18T20:53:51.4046050Z │ "type": "object" 2022-11-18T20:53:51.4046367Z │ } 2022-11-18T20:53:51.4046671Z │ } 2022-11-18T20:53:51.4046966Z │ } 2022-11-18T20:53:51.4047238Z │ } 2022-11-18T20:53:51.4047528Z │ } 2022-11-18T20:53:51.4047818Z │ }, 2022-11-18T20:53:51.4048094Z │ "500": { 2022-11-18T20:53:51.4048475Z │ "description": "Internal Server Error", 2022-11-18T20:53:51.4048982Z │ "content": { 2022-11-18T20:53:51.4049337Z │ "application/json": { 2022-11-18T20:53:51.4049659Z │ "schema": { 2022-11-18T20:53:51.4049997Z │ "type": "object", 2022-11-18T20:53:51.4050346Z │ "required": [ 2022-11-18T20:53:51.4050679Z │ "code", 2022-11-18T20:53:51.4050974Z │ "type", 2022-11-18T20:53:51.4051296Z │ "message" 2022-11-18T20:53:51.4051608Z │ ], 2022-11-18T20:53:51.4051922Z │ "properties": { 2022-11-18T20:53:51.4052260Z │ "code": { 2022-11-18T20:53:51.4052610Z │ "type": "integer", 2022-11-18T20:53:51.4052995Z │ "description": "Error code." 2022-11-18T20:53:51.4053318Z │ }, 2022-11-18T20:53:51.4053634Z │ "type": { 2022-11-18T20:53:51.4053975Z │ "type": "string", 2022-11-18T20:53:51.4054370Z │ "description": "Error type." 2022-11-18T20:53:51.4054689Z │ }, 2022-11-18T20:53:51.4055014Z │ "message": { 2022-11-18T20:53:51.4055379Z │ "type": "string", 2022-11-18T20:53:51.4055753Z │ "description": "Error message." 2022-11-18T20:53:51.4056097Z │ } 2022-11-18T20:53:51.4056399Z │ } 2022-11-18T20:53:51.4056698Z │ } 2022-11-18T20:53:51.4056971Z │ } 2022-11-18T20:53:51.4057259Z │ } 2022-11-18T20:53:51.4057543Z │ } 2022-11-18T20:53:51.4057805Z │ } 2022-11-18T20:53:51.4058085Z │ } 2022-11-18T20:53:51.4058362Z │ }, 2022-11-18T20:53:51.4058650Z │ "/ping": { 2022-11-18T20:53:51.4058912Z │ 2022-11-18T20:53:51.4059226Z │ (showing 2.05kB/23.41kB) 2022-11-18T20:53:51.4059507Z └ 2022-11-18T20:53:51.4060095Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:53:51.4060688Z 31ms 17ms 9ms 3ms 169ms 46ms 7ms 494µs 286ms 2022-11-18T20:53:51.4060936Z 2022-11-18T20:53:51.4061135Z √ Status code is 200 2022-11-18T20:53:51.4258093Z 2022-11-18T20:53:51.4258974Z → HTTPS Management API Description 2022-11-18T20:53:51.5634666Z OPTIONS https://localhost:8444 2022-11-18T20:53:51.5635877Z 200 OK ★ 85ms time ★ 230B↑ 58.87kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:51.5636479Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 58.61kB 2022-11-18T20:53:51.5636899Z │ { 2022-11-18T20:53:51.5637260Z │ "openapi": "3.0.1", 2022-11-18T20:53:51.5637601Z │ "info": { 2022-11-18T20:53:51.5637995Z │ "title": "TorchServe APIs", 2022-11-18T20:53:51.5638468Z │ "description": "TorchServe is a flexible and easy 2022-11-18T20:53:51.5638950Z │ to use tool for serving deep learning models", 2022-11-18T20:53:51.5639661Z │ "version": "0.6.1\r" 2022-11-18T20:53:51.5640009Z │ }, 2022-11-18T20:53:51.5640346Z │ "paths": { 2022-11-18T20:53:51.5640664Z │ "/": { 2022-11-18T20:53:51.5641122Z │ "options": { 2022-11-18T20:53:51.5641579Z │ "description": "Get openapi description.", 2022-11-18T20:53:51.5642053Z │ "operationId": "apiDescription", 2022-11-18T20:53:51.5642451Z │ "parameters": [], 2022-11-18T20:53:51.5642834Z │ "responses": { 2022-11-18T20:53:51.5643194Z │ "200": { 2022-11-18T20:53:51.5643611Z │ "description": "A openapi 3.0.1 descriptor 2022-11-18T20:53:51.5643990Z │ ", 2022-11-18T20:53:51.5644333Z │ "content": { 2022-11-18T20:53:51.5644730Z │ "application/json": { 2022-11-18T20:53:51.5645102Z │ "schema": { 2022-11-18T20:53:51.5645486Z │ "type": "object", 2022-11-18T20:53:51.5645885Z │ "required": [ 2022-11-18T20:53:51.5646268Z │ "openapi", 2022-11-18T20:53:51.5646622Z │ "info", 2022-11-18T20:53:51.5646988Z │ "paths" 2022-11-18T20:53:51.5647353Z │ ], 2022-11-18T20:53:51.5647716Z │ "properties": { 2022-11-18T20:53:51.5648111Z │ "openapi": { 2022-11-18T20:53:51.5648524Z │ "type": "string" 2022-11-18T20:53:51.5648906Z │ }, 2022-11-18T20:53:51.5649246Z │ "info": { 2022-11-18T20:53:51.5649629Z │ "type": "object" 2022-11-18T20:53:51.5649984Z │ }, 2022-11-18T20:53:51.5874275Z │ "paths": { 2022-11-18T20:53:51.5876350Z │ "type": "object" 2022-11-18T20:53:51.5877040Z │ } 2022-11-18T20:53:51.5883729Z │ } 2022-11-18T20:53:51.5884222Z │ } 2022-11-18T20:53:51.5884595Z │ } 2022-11-18T20:53:51.5884931Z │ } 2022-11-18T20:53:51.5885236Z │ }, 2022-11-18T20:53:51.5885583Z │ "500": { 2022-11-18T20:53:51.5886040Z │ "description": "Internal Server Error", 2022-11-18T20:53:51.5886465Z │ "content": { 2022-11-18T20:53:51.5886893Z │ "application/json": { 2022-11-18T20:53:51.5887282Z │ "schema": { 2022-11-18T20:53:51.5887663Z │ "type": "object", 2022-11-18T20:53:51.5888044Z │ "required": [ 2022-11-18T20:53:51.5888392Z │ "code", 2022-11-18T20:53:51.5888754Z │ "type", 2022-11-18T20:53:51.5889121Z │ "message" 2022-11-18T20:53:51.5889451Z │ ], 2022-11-18T20:53:51.5889829Z │ "properties": { 2022-11-18T20:53:51.5890213Z │ "code": { 2022-11-18T20:53:51.5890614Z │ "type": "integer", 2022-11-18T20:53:51.5891042Z │ "description": "Error code." 2022-11-18T20:53:51.5891422Z │ }, 2022-11-18T20:53:51.5891781Z │ "type": { 2022-11-18T20:53:51.5892443Z │ "type": "string", 2022-11-18T20:53:51.5892886Z │ "description": "Error type." 2022-11-18T20:53:51.5893326Z │ }, 2022-11-18T20:53:51.5893721Z │ "message": { 2022-11-18T20:53:51.5894117Z │ "type": "string", 2022-11-18T20:53:51.5894556Z │ "description": "Error message." 2022-11-18T20:53:51.5894919Z │ } 2022-11-18T20:53:51.5895263Z │ } 2022-11-18T20:53:51.5895599Z │ } 2022-11-18T20:53:51.5895932Z │ } 2022-11-18T20:53:51.5896240Z │ } 2022-11-18T20:53:51.5896567Z │ } 2022-11-18T20:53:51.5897016Z │ } 2022-11-18T20:53:51.5897315Z │ } 2022-11-18T20:53:51.5897627Z │ }, 2022-11-18T20:53:51.5897964Z │ "/models": { 2022-11-18T20:53:51.5898289Z │ 2022-11-18T20:53:51.5898620Z │ (showing 2.05kB/58.61kB) 2022-11-18T20:53:51.5898954Z └ 2022-11-18T20:53:51.5899510Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:53:51.5900134Z 1ms 1ms 251µs 469µs 56ms 22ms 2ms 127µs 86ms 2022-11-18T20:53:51.5900388Z 2022-11-18T20:53:51.5900722Z √ Status code is 200 2022-11-18T20:53:51.6124526Z 2022-11-18T20:53:51.6125199Z → HTTPS Metrics API Description 2022-11-18T20:53:51.7150130Z OPTIONS https://localhost:8445 2022-11-18T20:53:51.7151330Z 200 OK ★ 65ms time ★ 230B↑ 2.82kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:51.7151982Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 2.55kB 2022-11-18T20:53:51.7152436Z │ { 2022-11-18T20:53:51.7152799Z │ "openapi": "3.0.1", 2022-11-18T20:53:51.7153143Z │ "info": { 2022-11-18T20:53:51.7153534Z │ "title": "TorchServe APIs", 2022-11-18T20:53:51.7154026Z │ "description": "TorchServe is a flexible and easy 2022-11-18T20:53:51.7154491Z │ to use tool for serving deep learning models", 2022-11-18T20:53:51.7154926Z │ "version": "0.6.1\r" 2022-11-18T20:53:51.7155279Z │ }, 2022-11-18T20:53:51.7155616Z │ "paths": { 2022-11-18T20:53:51.7155949Z │ "/metrics": { 2022-11-18T20:53:51.7156297Z │ "get": { 2022-11-18T20:53:51.7156741Z │ "description": "Get TorchServe application met 2022-11-18T20:53:51.7157190Z │ rics in prometheus format.", 2022-11-18T20:53:51.7157590Z │ "operationId": "metrics", 2022-11-18T20:53:51.7157980Z │ "parameters": [ 2022-11-18T20:53:51.7158336Z │ { 2022-11-18T20:53:51.7158670Z │ "in": "query", 2022-11-18T20:53:51.7159042Z │ "name": "name[]", 2022-11-18T20:53:51.7159495Z │ "description": "Names of metrics to filter 2022-11-18T20:53:51.7159898Z │ ", 2022-11-18T20:53:51.7160240Z │ "required": false, 2022-11-18T20:53:51.7160625Z │ "schema": { 2022-11-18T20:53:51.7161005Z │ "type": "string" 2022-11-18T20:53:51.7161339Z │ } 2022-11-18T20:53:51.7161674Z │ } 2022-11-18T20:53:51.7162006Z │ ], 2022-11-18T20:53:51.7162359Z │ "responses": { 2022-11-18T20:53:51.7162699Z │ "200": { 2022-11-18T20:53:51.7163134Z │ "description": "TorchServe application met 2022-11-18T20:53:51.7163543Z │ rics", 2022-11-18T20:53:51.7163911Z │ "content": { 2022-11-18T20:53:51.7164324Z │ "text/plain; version=0.0.4; charset=utf- 2022-11-18T20:53:51.7164695Z │ 8": { 2022-11-18T20:53:51.7165022Z │ "schema": { 2022-11-18T20:53:51.7165411Z │ "type": "object", 2022-11-18T20:53:51.7165809Z │ "required": [ 2022-11-18T20:53:51.7166184Z │ "# HELP", 2022-11-18T20:53:51.7166531Z │ "# TYPE", 2022-11-18T20:53:51.7167134Z │ "metric" 2022-11-18T20:53:51.7167505Z │ ], 2022-11-18T20:53:51.7167883Z │ "properties": { 2022-11-18T20:53:51.7168247Z │ "# HELP": { 2022-11-18T20:53:51.7168640Z │ "type": "string", 2022-11-18T20:53:51.7169079Z │ "description": "Help text for To 2022-11-18T20:53:51.7169450Z │ rchServe metric." 2022-11-18T20:53:51.7169807Z │ }, 2022-11-18T20:53:51.7170168Z │ "# TYPE": { 2022-11-18T20:53:51.7170560Z │ "type": "string", 2022-11-18T20:53:51.7170974Z │ "description": "Type of TorchSer 2022-11-18T20:53:51.7171355Z │ ve metric." 2022-11-18T20:53:51.7171852Z │ }, 2022-11-18T20:53:51.7172224Z │ "metric": { 2022-11-18T20:53:51.7172596Z │ "type": "string", 2022-11-18T20:53:51.7173039Z │ "description": "TorchServe appli 2022-11-18T20:53:51.7173424Z │ cation metric." 2022-11-18T20:53:51.7173751Z │ } 2022-11-18T20:53:51.7174092Z │ } 2022-11-18T20:53:51.7174434Z │ } 2022-11-18T20:53:51.7174768Z │ } 2022-11-18T20:53:51.7175080Z │ } 2022-11-18T20:53:51.7175406Z │ }, 2022-11-18T20:53:51.7175770Z │ "500": { 2022-11-18T20:53:51.7176195Z │ "description": "Internal Server Error", 2022-11-18T20:53:51.7176605Z │ "content": { 2022-11-18T20:53:51.7176974Z │ "application/json": { 2022-11-18T20:53:51.7177357Z │ "schema": { 2022-11-18T20:53:51.7177736Z │ "type": "object", 2022-11-18T20:53:51.7178132Z │ "required": [ 2022-11-18T20:53:51.7178480Z │ "code", 2022-11-18T20:53:51.7178841Z │ "type", 2022-11-18T20:53:51.7179227Z │ "message" 2022-11-18T20:53:51.7179567Z │ ], 2022-11-18T20:53:51.7179941Z │ "properties": { 2022-11-18T20:53:51.7180321Z │ "code": { 2022-11-18T20:53:51.7180662Z │ 2022-11-18T20:53:51.7180999Z │ (showing 2.05kB/2.55kB) 2022-11-18T20:53:51.7181316Z └ 2022-11-18T20:53:51.7181863Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:53:51.7182484Z 1ms 1ms 251µs 455µs 55ms 4ms 2ms 84µs 66ms 2022-11-18T20:53:51.7182763Z 2022-11-18T20:53:51.7333086Z √ Status code is 200 2022-11-18T20:53:51.7533328Z 2022-11-18T20:53:51.7534102Z → HTTPS Register Model - SqueezeNet 2022-11-18T20:53:54.4294483Z POST https://localhost:8444/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&initial_workers=1&synchronous=true 2022-11-18T20:53:54.4295733Z 200 OK ★ 2.6s time ★ 334B↑ 355B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:54.4296337Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 89B 2022-11-18T20:53:54.4296753Z │ { 2022-11-18T20:53:54.4297249Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:53:54.4297712Z │ stered with 1 initial workers" 2022-11-18T20:53:54.4298103Z │ } 2022-11-18T20:53:54.4320885Z └ 2022-11-18T20:53:54.4321562Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:53:54.4322269Z 2ms 704µs (cache) (cache) (cache) 2.6s 2ms 103µs 2.6s 2022-11-18T20:53:54.4322583Z 2022-11-18T20:53:54.4454033Z √ Successful POST request 2022-11-18T20:53:54.4715375Z 2022-11-18T20:53:54.4715872Z → HTTPS Get SqueezeNet Model Description 2022-11-18T20:53:54.5171712Z GET https://localhost:8444/models/squeezenet1_1 2022-11-18T20:53:54.5172687Z 200 OK ★ 10ms time ★ 246B↑ 751B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:54.5173476Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 484B 2022-11-18T20:53:54.5173908Z │ [ 2022-11-18T20:53:54.5174241Z │ { 2022-11-18T20:53:54.5174670Z │ "modelName": "squeezenet1_1", 2022-11-18T20:53:54.5175122Z │ "modelVersion": "1.0", 2022-11-18T20:53:54.5175575Z │ "modelUrl": "squeezenet1_1.mar", 2022-11-18T20:53:54.5176006Z │ "runtime": "python", 2022-11-18T20:53:54.5176425Z │ "minWorkers": 1, 2022-11-18T20:53:54.5176836Z │ "maxWorkers": 1, 2022-11-18T20:53:54.5177241Z │ "batchSize": 1, 2022-11-18T20:53:54.5177645Z │ "maxBatchDelay": 100, 2022-11-18T20:53:54.5178087Z │ "loadedAtStartup": false, 2022-11-18T20:53:54.5178382Z │ "workers": [ 2022-11-18T20:53:54.5179775Z │ { 2022-11-18T20:53:54.5180724Z │ "id": "9000", 2022-11-18T20:53:54.5181225Z │ "startTime": "2022-11-18T20:53:52.191Z", 2022-11-18T20:53:54.5181691Z │ "status": "READY", 2022-11-18T20:53:54.5182119Z │ "memoryUsage": 0, 2022-11-18T20:53:54.5182537Z │ "pid": 436, 2022-11-18T20:53:54.5182944Z │ "gpu": false, 2022-11-18T20:53:54.5183342Z │ "gpuUsage": "N/A" 2022-11-18T20:53:54.5183732Z │ } 2022-11-18T20:53:54.5184092Z │ ] 2022-11-18T20:53:54.5184447Z │ } 2022-11-18T20:53:54.5184779Z │ ] 2022-11-18T20:53:54.5185106Z └ 2022-11-18T20:53:54.5185708Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:53:54.5186393Z 1ms 487µs (cache) (cache) (cache) 6ms 1ms 72µs 10ms 2022-11-18T20:53:54.5186680Z 2022-11-18T20:53:54.5227177Z √ Successful GET request 2022-11-18T20:53:54.5496726Z 2022-11-18T20:53:54.5497616Z → HTTPS Scale up Workers - Synchronous 2022-11-18T20:53:54.6041239Z PUT https://localhost:8444/models/squeezenet1_1?min_worker=1&max_worker=1&synchronous=true 2022-11-18T20:53:54.6042320Z 200 OK ★ 6ms time ★ 308B↑ 329B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:54.6043002Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 63B 2022-11-18T20:53:54.6043504Z │ { 2022-11-18T20:53:54.6044036Z │ "status": "Workers scaled to 1 for model: squeezenet 2022-11-18T20:53:54.6044510Z │ 1_1" 2022-11-18T20:53:54.6044895Z │ } 2022-11-18T20:53:54.6068835Z └ 2022-11-18T20:53:54.6069732Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:53:54.6070545Z 1ms 340µs (cache) (cache) (cache) 3ms 1ms 63µs 6ms 2022-11-18T20:53:54.6070931Z 2022-11-18T20:53:54.6165895Z √ Successful PUT request 2022-11-18T20:53:54.6435715Z 2022-11-18T20:53:54.6436681Z → HTTPS Scale up Workers - Asynchronous 2022-11-18T20:53:54.7003310Z PUT https://localhost:8444/models/squeezenet1_1?min_worker=1&max_worker=1&synchronous=false 2022-11-18T20:53:54.7004688Z 202 Accepted ★ 8ms time ★ 309B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:54.7005524Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:53:54.7005922Z │ { 2022-11-18T20:53:54.7006343Z │ "status": "Processing worker updates..." 2022-11-18T20:53:54.7006722Z │ } 2022-11-18T20:53:54.7022331Z └ 2022-11-18T20:53:54.7022968Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:53:54.7023703Z 1ms 408µs (cache) (cache) (cache) 4ms 1ms 60µs 9ms 2022-11-18T20:53:54.7024027Z 2022-11-18T20:53:54.7074111Z √ Successful PUT request 2022-11-18T20:53:54.7377437Z 2022-11-18T20:53:54.7378377Z → HTTPS - Inference - SqueezeNet 2022-11-18T20:53:54.8865625Z POST https://localhost:8443/predictions/squeezenet1_1 2022-11-18T20:53:54.8866753Z 200 OK ★ 90ms time ★ 111.25kB↑ 409B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:53:54.8867359Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:53:54.8868103Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:53:54.8868550Z └ 2022-11-18T20:53:54.8869067Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 174B 2022-11-18T20:53:54.8869530Z │ { 2022-11-18T20:53:54.8869987Z │ "tabby": 0.27850738167762756, 2022-11-18T20:53:54.8870462Z │ "lynx": 0.25299158692359924, 2022-11-18T20:53:54.8870946Z │ "tiger_cat": 0.24496474862098694, 2022-11-18T20:53:54.8871425Z │ "Egyptian_cat": 0.21722468733787537, 2022-11-18T20:53:54.8871914Z │ "cougar": 0.002217544475570321 2022-11-18T20:53:54.8872338Z │ } 2022-11-18T20:53:54.8886098Z └ 2022-11-18T20:53:54.8886734Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:53:54.8887471Z 8ms 883µs (cache) (cache) (cache) 86ms 2ms 64µs 98ms 2022-11-18T20:53:54.8887990Z 2022-11-18T20:53:54.9004330Z √ Status code is 200 2022-11-18T20:53:54.9254763Z 2022-11-18T20:53:54.9255675Z → HTTPS UnRegister Model SqueezeNet 2022-11-18T20:53:55.0549433Z DELETE https://localhost:8444/models/squeezenet1_1 2022-11-18T20:53:55.0550490Z 200 OK ★ 80ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:53:55.0551111Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:53:55.0551554Z │ { 2022-11-18T20:53:55.0551968Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:53:55.0552349Z │ } 2022-11-18T20:53:55.0568849Z └ 2022-11-18T20:53:55.0569412Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:53:55.0570071Z 2ms 479µs (cache) (cache) (cache) 76ms 2ms 173µs 81ms 2022-11-18T20:53:55.0570421Z 2022-11-18T20:53:55.0645780Z √ Successful DELETE request 2022-11-18T20:53:55.2844316Z 2022-11-18T20:53:55.2845261Z ┌─────────────────────────┬────────────────────┬────────────────────┐ 2022-11-18T20:53:55.2860652Z │ │ executed │ failed │ 2022-11-18T20:53:55.2861312Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:53:55.2861825Z │ iterations │ 1 │ 0 │ 2022-11-18T20:53:55.2862334Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:53:55.2876959Z │ requests │ 9 │ 0 │ 2022-11-18T20:53:55.2877624Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:53:55.2878129Z │ test-scripts │ 9 │ 0 │ 2022-11-18T20:53:55.2878644Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:53:55.2879176Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:53:55.2879673Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:53:55.2880399Z │ assertions │ 9 │ 0 │ 2022-11-18T20:53:55.2880912Z ├─────────────────────────┴────────────────────┴────────────────────┤ 2022-11-18T20:53:55.2881390Z │ total run duration: 4s │ 2022-11-18T20:53:55.2881868Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:53:55.2882348Z │ total data received: 85.48kB (approx) │ 2022-11-18T20:53:55.2882850Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:53:55.2883342Z │ average response time: 358ms [min: 6ms, max: 2.6s, s.d.: 804ms] │ 2022-11-18T20:53:55.2883834Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:53:55.2884317Z │ average DNS lookup time: 3ms [min: 251µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:53:55.2885583Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:53:55.2886091Z │ average first byte time: 319ms [min: 3ms, max: 2.6s, s.d.: 814ms] │ 2022-11-18T20:53:55.2886585Z └───────────────────────────────────────────────────────────────────┘ 2022-11-18T20:53:55.4053000Z TorchServe has stopped. 2022-11-18T20:54:16.8893270Z newman 2022-11-18T20:54:16.8893798Z 2022-11-18T20:54:16.8895756Z management_api_collection 2022-11-18T20:54:16.8900791Z 2022-11-18T20:54:16.8901011Z Iteration 1/82 2022-11-18T20:54:16.8926304Z 2022-11-18T20:54:16.8926965Z → management request 2022-11-18T20:54:17.2926269Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:54:17.2927819Z 200 OK ★ 288ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:17.2942933Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:54:17.2943537Z │ { 2022-11-18T20:54:17.2943997Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:54:17.2944585Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:54:17.2945024Z │ o add workers for the model." 2022-11-18T20:54:17.2945399Z │ } 2022-11-18T20:54:17.2945697Z └ 2022-11-18T20:54:17.2946219Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:17.2946790Z 31ms 11ms 9ms 572µs 257ms 7ms 577µs 319ms 2022-11-18T20:54:17.2947068Z 2022-11-18T20:54:17.2947311Z √ Successful request 2022-11-18T20:54:17.3042908Z 2022-11-18T20:54:17.3043235Z Iteration 2/82 2022-11-18T20:54:17.3200424Z 2022-11-18T20:54:17.3200876Z → management request 2022-11-18T20:54:17.7258828Z POST http://localhost:8081/models?url=mnist.mar&model_name=mnist 2022-11-18T20:54:17.7260063Z 200 OK ★ 372ms time ★ 283B↑ 401B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:17.7260798Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 134B 2022-11-18T20:54:17.7263077Z │ { 2022-11-18T20:54:17.7263521Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:54:17.7264213Z │ ith 0 initial workers. Use scale workers API to add wo 2022-11-18T20:54:17.7264670Z │ rkers for the model." 2022-11-18T20:54:17.7264996Z │ } 2022-11-18T20:54:17.7283599Z └ 2022-11-18T20:54:17.7284230Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:17.7284885Z 2ms 863µs (cache) (cache) 367ms 2ms 96µs 373ms 2022-11-18T20:54:17.7285201Z 2022-11-18T20:54:17.7344997Z √ Successful request 2022-11-18T20:54:17.7421525Z 2022-11-18T20:54:17.7421849Z Iteration 3/82 2022-11-18T20:54:17.7580191Z 2022-11-18T20:54:17.7580751Z → management request 2022-11-18T20:54:26.1573338Z POST http://localhost:8081/models?url=densenet161.mar&model_name=densenet161 2022-11-18T20:54:26.1574757Z 200 OK ★ 8.3s time ★ 295B↑ 407B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:26.1575363Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 140B 2022-11-18T20:54:26.1575778Z │ { 2022-11-18T20:54:26.1576234Z │ "status": "Model \"densenet161\" Version: 1.0 regist 2022-11-18T20:54:26.1576737Z │ ered with 0 initial workers. Use scale workers API to 2022-11-18T20:54:26.1577180Z │ add workers for the model." 2022-11-18T20:54:26.1577519Z │ } 2022-11-18T20:54:26.1593554Z └ 2022-11-18T20:54:26.1594190Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:26.1594948Z 2ms 324µs (cache) (cache) 8.3s 2ms 78µs 8.3s 2022-11-18T20:54:26.1595453Z 2022-11-18T20:54:26.1689898Z √ Successful request 2022-11-18T20:54:26.1791307Z 2022-11-18T20:54:26.1791607Z Iteration 4/82 2022-11-18T20:54:26.1951072Z 2022-11-18T20:54:26.1951779Z → management request 2022-11-18T20:54:26.2522337Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/densenet161.mar&model_name=densenet161 2022-11-18T20:54:26.2523889Z 500 Internal Server Error ★ 7ms time ★ 336B↑ 394B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:26.2524633Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 113B 2022-11-18T20:54:26.2525136Z │ { 2022-11-18T20:54:26.2525527Z │ "code": 500, 2022-11-18T20:54:26.2526046Z │ "type": "InternalServerException", 2022-11-18T20:54:26.2526632Z │ "message": "Model file already exists densenet161.ma 2022-11-18T20:54:26.2527084Z │ r" 2022-11-18T20:54:26.2527442Z │ } 2022-11-18T20:54:26.2539802Z └ 2022-11-18T20:54:26.2540444Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:26.2541207Z 2ms 322µs (cache) (cache) 4ms 2ms 84µs 8ms 2022-11-18T20:54:26.2541556Z 2022-11-18T20:54:26.2601282Z √ Successful request 2022-11-18T20:54:26.2729745Z 2022-11-18T20:54:26.2730143Z Iteration 5/82 2022-11-18T20:54:26.2888380Z 2022-11-18T20:54:26.2889013Z → management request 2022-11-18T20:54:27.3515457Z DELETE http://localhost:8081/models/densenet161 2022-11-18T20:54:27.3516203Z 200 OK ★ 41ms time ★ 247B↑ 319B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:27.3516614Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 53B 2022-11-18T20:54:27.3516891Z │ { 2022-11-18T20:54:27.3517189Z │ "status": "Model \"densenet161\" unregistered" 2022-11-18T20:54:27.3517458Z │ } 2022-11-18T20:54:27.3517646Z └ 2022-11-18T20:54:27.3518045Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:27.3518531Z 1ms 1ms 437µs 306µs 36ms 1ms 84µs 42ms 2022-11-18T20:54:27.3518700Z 2022-11-18T20:54:27.3518844Z √ Successful request 2022-11-18T20:54:27.3518986Z 2022-11-18T20:54:27.3519074Z Iteration 6/82 2022-11-18T20:54:27.3519207Z 2022-11-18T20:54:27.3519340Z → management request 2022-11-18T20:54:27.3519611Z POST http://localhost:8081/models [errored] 2022-11-18T20:54:27.3519866Z read ECONNRESET 2022-11-18T20:54:27.3520069Z 2. Successful request 2022-11-18T20:54:27.3520419Z 2022-11-18T20:54:27.3520510Z Iteration 7/82 2022-11-18T20:54:27.3520638Z 2022-11-18T20:54:27.3520773Z → management request 2022-11-18T20:54:27.3521023Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:54:27.3521424Z 200 OK ★ 17ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:27.3521804Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:54:27.3522077Z │ { 2022-11-18T20:54:27.3522350Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:54:27.3522607Z │ } 2022-11-18T20:54:27.3522797Z └ 2022-11-18T20:54:27.3523179Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:27.3523647Z 1ms 867µs 372µs 286µs 13ms 1ms 64µs 18ms 2022-11-18T20:54:27.3523917Z 2022-11-18T20:54:27.3524061Z √ Successful request 2022-11-18T20:54:27.3524201Z 2022-11-18T20:54:27.3524269Z Iteration 8/82 2022-11-18T20:54:27.3524439Z 2022-11-18T20:54:27.3524582Z → management request 2022-11-18T20:54:27.3524930Z POST http://localhost:8081/models?url=mnist.mar&model_name=mnist&handler=invalidHandler 2022-11-18T20:54:27.3525402Z 200 OK ★ 101ms time ★ 306B↑ 401B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:27.3525761Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 134B 2022-11-18T20:54:27.3526031Z │ { 2022-11-18T20:54:27.3526352Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:54:27.3526717Z │ ith 0 initial workers. Use scale workers API to add wo 2022-11-18T20:54:27.3527025Z │ rkers for the model." 2022-11-18T20:54:27.3527258Z │ } 2022-11-18T20:54:27.3527427Z └ 2022-11-18T20:54:27.3527822Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:27.3528305Z 2ms 288µs (cache) (cache) 98ms 2ms 67µs 102ms 2022-11-18T20:54:27.3528482Z 2022-11-18T20:54:27.3528621Z √ Successful request 2022-11-18T20:54:27.3528755Z 2022-11-18T20:54:27.3528839Z Iteration 9/82 2022-11-18T20:54:27.3528969Z 2022-11-18T20:54:27.3529099Z → management request 2022-11-18T20:54:27.3529367Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:54:27.3529736Z 200 OK ★ 8ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:27.3530104Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:54:27.3530367Z │ { 2022-11-18T20:54:27.3530657Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:54:27.3530893Z │ } 2022-11-18T20:54:27.3531078Z └ 2022-11-18T20:54:27.3531471Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:27.3531917Z 1ms 407µs (cache) (cache) 5ms 1ms 53µs 9ms 2022-11-18T20:54:27.3532110Z 2022-11-18T20:54:27.3532249Z √ Successful request 2022-11-18T20:54:27.3532389Z 2022-11-18T20:54:27.3532474Z Iteration 10/82 2022-11-18T20:54:27.3532614Z 2022-11-18T20:54:27.3532744Z → management request 2022-11-18T20:54:27.3533071Z POST http://localhost:8081/models?url=mnist.mar&model_name=mnist&handler=invalidHandler 2022-11-18T20:54:27.3533527Z 200 OK ★ 98ms time ★ 306B↑ 401B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:27.3533899Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 134B 2022-11-18T20:54:27.3534145Z │ { 2022-11-18T20:54:27.3534461Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:54:27.3534838Z │ ith 0 initial workers. Use scale workers API to add wo 2022-11-18T20:54:27.3535127Z │ rkers for the model." 2022-11-18T20:54:27.3535358Z │ } 2022-11-18T20:54:27.3535546Z └ 2022-11-18T20:54:27.3535922Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:27.3536400Z 1ms 362µs (cache) (cache) 95ms 2ms 68µs 99ms 2022-11-18T20:54:27.3536576Z 2022-11-18T20:54:27.3537529Z √ Successful request 2022-11-18T20:54:27.3537690Z 2022-11-18T20:54:27.3537779Z Iteration 11/82 2022-11-18T20:54:27.3537910Z 2022-11-18T20:54:27.3538040Z → management request 2022-11-18T20:54:28.5161222Z PUT http://localhost:8081/models/mnist?min_worker=1&synchronous=true [errored] 2022-11-18T20:54:28.5166220Z read ECONNRESET 2022-11-18T20:54:28.5471469Z 4. Successful request 2022-11-18T20:54:28.5523083Z 2022-11-18T20:54:28.5523475Z Iteration 12/82 2022-11-18T20:54:28.5556595Z 2022-11-18T20:54:28.5557118Z → management request 2022-11-18T20:54:28.6166985Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:54:28.6168108Z 200 OK ★ 12ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:28.6168609Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:54:28.6169177Z │ { 2022-11-18T20:54:28.6169532Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:54:28.6169860Z │ } 2022-11-18T20:54:28.6181501Z └ 2022-11-18T20:54:28.6182016Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:28.6182568Z 1ms 1ms 270µs 275µs 7ms 1ms 67µs 13ms 2022-11-18T20:54:28.6182803Z 2022-11-18T20:54:28.6307819Z √ Successful request 2022-11-18T20:54:28.6414003Z 2022-11-18T20:54:28.6414365Z Iteration 13/82 2022-11-18T20:54:28.6571393Z 2022-11-18T20:54:28.6572066Z → management request 2022-11-18T20:54:28.7139009Z GET http://localhost:8081/models/squeezenet1_1/all 2022-11-18T20:54:28.7140048Z 200 OK ★ 8ms time ★ 250B↑ 538B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:28.7140526Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 271B 2022-11-18T20:54:28.7140872Z │ [ 2022-11-18T20:54:28.7141180Z │ { 2022-11-18T20:54:28.7141507Z │ "modelName": "squeezenet1_1", 2022-11-18T20:54:28.7141885Z │ "modelVersion": "1.0", 2022-11-18T20:54:28.7142270Z │ "modelUrl": "squeezenet1_1.mar", 2022-11-18T20:54:28.7142656Z │ "runtime": "python", 2022-11-18T20:54:28.7142974Z │ "minWorkers": 0, 2022-11-18T20:54:28.7143311Z │ "maxWorkers": 0, 2022-11-18T20:54:28.7143632Z │ "batchSize": 1, 2022-11-18T20:54:28.7143980Z │ "maxBatchDelay": 100, 2022-11-18T20:54:28.7144325Z │ "loadedAtStartup": false, 2022-11-18T20:54:28.7144667Z │ "workers": [] 2022-11-18T20:54:28.7144956Z │ } 2022-11-18T20:54:28.7145208Z │ ] 2022-11-18T20:54:28.7153237Z └ 2022-11-18T20:54:28.7153757Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:28.7154339Z 1ms 677µs (cache) (cache) 5ms 1ms 164µs 9ms 2022-11-18T20:54:28.7154577Z 2022-11-18T20:54:28.7250788Z √ Successful request 2022-11-18T20:54:28.7350229Z 2022-11-18T20:54:28.7350617Z Iteration 14/82 2022-11-18T20:54:28.7510519Z 2022-11-18T20:54:28.7511286Z → management request 2022-11-18T20:54:28.8043159Z GET http://localhost:8081/models/squeezenet1_1/1.0 2022-11-18T20:54:28.8044171Z 200 OK ★ 5ms time ★ 250B↑ 538B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:28.8044797Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 271B 2022-11-18T20:54:28.8045227Z │ [ 2022-11-18T20:54:28.8045554Z │ { 2022-11-18T20:54:28.8045962Z │ "modelName": "squeezenet1_1", 2022-11-18T20:54:28.8046386Z │ "modelVersion": "1.0", 2022-11-18T20:54:28.8046802Z │ "modelUrl": "squeezenet1_1.mar", 2022-11-18T20:54:28.8047230Z │ "runtime": "python", 2022-11-18T20:54:28.8047621Z │ "minWorkers": 0, 2022-11-18T20:54:28.8048004Z │ "maxWorkers": 0, 2022-11-18T20:54:28.8048362Z │ "batchSize": 1, 2022-11-18T20:54:28.8048761Z │ "maxBatchDelay": 100, 2022-11-18T20:54:28.8049196Z │ "loadedAtStartup": false, 2022-11-18T20:54:28.8049568Z │ "workers": [] 2022-11-18T20:54:28.8049912Z │ } 2022-11-18T20:54:28.8050231Z │ ] 2022-11-18T20:54:28.8057512Z └ 2022-11-18T20:54:28.8058246Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:28.8058903Z 1ms 536µs (cache) (cache) 2ms 1ms 65µs 5ms 2022-11-18T20:54:28.8059222Z 2022-11-18T20:54:28.8259731Z √ Successful request 2022-11-18T20:54:28.8287433Z 2022-11-18T20:54:28.8287712Z Iteration 15/82 2022-11-18T20:54:28.8290236Z 2022-11-18T20:54:28.8290580Z → management request 2022-11-18T20:54:28.8826906Z GET http://localhost:8081/models/squeezenet1_1 2022-11-18T20:54:28.8827905Z 200 OK ★ 5ms time ★ 246B↑ 538B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:28.8828585Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 271B 2022-11-18T20:54:28.8829103Z │ [ 2022-11-18T20:54:28.8829741Z │ { 2022-11-18T20:54:28.8830206Z │ "modelName": "squeezenet1_1", 2022-11-18T20:54:28.8830668Z │ "modelVersion": "1.0", 2022-11-18T20:54:28.8831160Z │ "modelUrl": "squeezenet1_1.mar", 2022-11-18T20:54:28.8831659Z │ "runtime": "python", 2022-11-18T20:54:28.8832081Z │ "minWorkers": 0, 2022-11-18T20:54:28.8832513Z │ "maxWorkers": 0, 2022-11-18T20:54:28.8832938Z │ "batchSize": 1, 2022-11-18T20:54:28.8833392Z │ "maxBatchDelay": 100, 2022-11-18T20:54:28.8833842Z │ "loadedAtStartup": false, 2022-11-18T20:54:28.8851865Z │ "workers": [] 2022-11-18T20:54:28.8852715Z │ } 2022-11-18T20:54:28.8853027Z │ ] 2022-11-18T20:54:28.8853447Z └ 2022-11-18T20:54:28.8854011Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:28.8854623Z 1ms 289µs (cache) (cache) 2ms 1ms 59µs 6ms 2022-11-18T20:54:28.8854913Z 2022-11-18T20:54:28.8955242Z √ Successful request 2022-11-18T20:54:28.9069503Z 2022-11-18T20:54:28.9069868Z Iteration 16/82 2022-11-18T20:54:28.9227393Z 2022-11-18T20:54:28.9227992Z → management request 2022-11-18T20:54:28.9677105Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:54:28.9690617Z 200 OK ★ 11ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:28.9692515Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:54:28.9692983Z │ { 2022-11-18T20:54:28.9693422Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:54:28.9693820Z │ } 2022-11-18T20:54:28.9694237Z └ 2022-11-18T20:54:28.9694775Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:28.9695363Z 1ms 489µs (cache) (cache) 8ms 1ms 53µs 12ms 2022-11-18T20:54:28.9695656Z 2022-11-18T20:54:28.9904809Z √ Successful request 2022-11-18T20:54:29.0006862Z 2022-11-18T20:54:29.0007252Z Iteration 17/82 2022-11-18T20:54:29.0165271Z 2022-11-18T20:54:29.0165871Z → management request 2022-11-18T20:54:29.0685402Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&runtime=python4 [errored] 2022-11-18T20:54:29.0686288Z read ECONNRESET 2022-11-18T20:54:29.0832210Z 6. Successful request 2022-11-18T20:54:29.0873109Z 2022-11-18T20:54:29.0873420Z Iteration 18/82 2022-11-18T20:54:29.0946497Z 2022-11-18T20:54:29.0947144Z → management request 2022-11-18T20:54:29.1504345Z GET http://localhost:8081/models?limit=&next_page_token= 2022-11-18T20:54:29.1505468Z 200 OK ★ 7ms time ★ 256B↑ 285B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:29.1506098Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 19B 2022-11-18T20:54:29.1506524Z │ { 2022-11-18T20:54:29.1506873Z │ "models": [] 2022-11-18T20:54:29.1507186Z │ } 2022-11-18T20:54:29.1517076Z └ 2022-11-18T20:54:29.1517603Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:29.1518482Z 1ms 1ms 444µs 299µs 3ms 1ms 61µs 8ms 2022-11-18T20:54:29.1518794Z 2022-11-18T20:54:29.1616323Z √ Successful request 2022-11-18T20:54:29.1725968Z 2022-11-18T20:54:29.1726312Z Iteration 19/82 2022-11-18T20:54:29.1886162Z 2022-11-18T20:54:29.1886934Z → management request 2022-11-18T20:54:29.3365538Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:54:29.3366681Z 200 OK ★ 98ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:29.3367290Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:54:29.3367722Z │ { 2022-11-18T20:54:29.3368153Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:54:29.3368658Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:54:29.3369111Z │ o add workers for the model." 2022-11-18T20:54:29.3369454Z │ } 2022-11-18T20:54:29.3370811Z └ 2022-11-18T20:54:29.3371303Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:29.3371943Z 1ms 249µs (cache) (cache) 94ms 2ms 59µs 98ms 2022-11-18T20:54:29.3372260Z 2022-11-18T20:54:29.3523980Z √ Successful request 2022-11-18T20:54:29.3603870Z 2022-11-18T20:54:29.3604227Z Iteration 20/82 2022-11-18T20:54:29.3761264Z 2022-11-18T20:54:29.3761807Z → management request 2022-11-18T20:54:29.4296195Z PUT http://localhost:8081/models/squeezenet1_1?min_worker=1 2022-11-18T20:54:29.4306766Z 202 Accepted ★ 5ms time ★ 278B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:29.4316878Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:54:29.4340750Z │ { 2022-11-18T20:54:29.4342443Z │ "status": "Processing worker updates..." 2022-11-18T20:54:29.4388248Z │ } 2022-11-18T20:54:29.4406286Z └ 2022-11-18T20:54:29.4407246Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:29.4407984Z 1ms 242µs (cache) (cache) 2ms 1ms 44µs 5ms 2022-11-18T20:54:29.4408295Z 2022-11-18T20:54:29.4408655Z √ Successful request 2022-11-18T20:54:29.4540050Z 2022-11-18T20:54:29.4540401Z Iteration 21/82 2022-11-18T20:54:29.4725510Z 2022-11-18T20:54:29.4726272Z → management request 2022-11-18T20:54:29.5252403Z PUT http://localhost:8081/models/squeezenet1_1?min_worker=1&synchronous=true 2022-11-18T20:54:29.5254149Z 200 OK ★ 5ms time ★ 295B↑ 329B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:29.5255040Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 63B 2022-11-18T20:54:29.5388257Z │ { 2022-11-18T20:54:29.5389831Z │ "status": "Workers scaled to 1 for model: squeezenet 2022-11-18T20:54:29.5488101Z │ 1_1" 2022-11-18T20:54:29.5489253Z │ } 2022-11-18T20:54:29.5489755Z └ 2022-11-18T20:54:29.5490369Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:29.5491022Z 1ms 256µs (cache) (cache) 2ms 1ms 39µs 5ms 2022-11-18T20:54:29.5491312Z 2022-11-18T20:54:29.5491831Z √ Successful request 2022-11-18T20:54:29.5639904Z 2022-11-18T20:54:29.5640248Z Iteration 22/82 2022-11-18T20:54:29.5802784Z 2022-11-18T20:54:29.5805203Z → management request 2022-11-18T20:54:29.6344979Z PUT http://localhost:8081/models/squeezenet1_1/1.0?min_worker=1&synchronous=true 2022-11-18T20:54:29.6346673Z 200 OK ★ 4ms time ★ 299B↑ 343B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:29.6347591Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 77B 2022-11-18T20:54:29.6354804Z │ { 2022-11-18T20:54:29.6355372Z │ "status": "Workers scaled to 1 for model: squeezenet 2022-11-18T20:54:29.6355812Z │ 1_1, version: 1.0" 2022-11-18T20:54:29.6356137Z │ } 2022-11-18T20:54:29.6356441Z └ 2022-11-18T20:54:29.6356981Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:29.6357887Z 1ms 246µs (cache) (cache) 2ms 1ms 43µs 5ms 2022-11-18T20:54:29.6358177Z 2022-11-18T20:54:29.6358425Z √ Successful request 2022-11-18T20:54:29.6422715Z 2022-11-18T20:54:29.6423042Z Iteration 23/82 2022-11-18T20:54:29.6579197Z 2022-11-18T20:54:29.6579733Z → management request 2022-11-18T20:54:29.7344714Z PUT http://localhost:8081/models/squeezenet1_1/0.0?min_worker=1&synchronous=true [errored] 2022-11-18T20:54:29.7346291Z read ECONNRESET 2022-11-18T20:54:29.7346999Z 8. Successful request 2022-11-18T20:54:29.7515730Z 2022-11-18T20:54:29.7516272Z Iteration 24/82 2022-11-18T20:54:29.7673303Z 2022-11-18T20:54:29.7674006Z → management request 2022-11-18T20:54:29.8253764Z PUT http://localhost:8081/models/squeezenet1_1?min_worker=1&number_gpu=1 2022-11-18T20:54:29.8254902Z 202 Accepted ★ 8ms time ★ 291B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:29.8255520Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:54:29.8255941Z │ { 2022-11-18T20:54:29.8256378Z │ "status": "Processing worker updates..." 2022-11-18T20:54:29.8256744Z │ } 2022-11-18T20:54:29.8257067Z └ 2022-11-18T20:54:29.8257601Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:29.8258194Z 1ms 1ms 1ms 290µs 2ms 1ms 38µs 9ms 2022-11-18T20:54:29.8258480Z 2022-11-18T20:54:29.8412726Z √ Successful request 2022-11-18T20:54:29.8451779Z 2022-11-18T20:54:29.8452159Z Iteration 25/82 2022-11-18T20:54:29.8611213Z 2022-11-18T20:54:29.8611871Z → management request 2022-11-18T20:54:29.9254438Z PUT http://localhost:8081/models/squeezenet1_1/1.0/set-default 2022-11-18T20:54:29.9256004Z 200 OK ★ 4ms time ★ 281B↑ 359B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:29.9256760Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 93B 2022-11-18T20:54:29.9262589Z │ { 2022-11-18T20:54:29.9263119Z │ "status": "Default vesion succsesfully updated for m 2022-11-18T20:54:29.9263548Z │ odel \"squeezenet1_1\" to \"1.0\"" 2022-11-18T20:54:29.9263868Z │ } 2022-11-18T20:54:29.9264148Z └ 2022-11-18T20:54:29.9264641Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:29.9265180Z 1ms 263µs (cache) (cache) 2ms 1ms 53µs 5ms 2022-11-18T20:54:29.9265422Z 2022-11-18T20:54:29.9265625Z √ Successful request 2022-11-18T20:54:29.9391477Z 2022-11-18T20:54:29.9391901Z Iteration 26/82 2022-11-18T20:54:29.9548266Z 2022-11-18T20:54:29.9548866Z → management request 2022-11-18T20:54:30.0064803Z PUT http://localhost:8081/models/squeezenet1_1/0.0/set-default [errored] 2022-11-18T20:54:30.0065310Z read ECONNRESET 2022-11-18T20:54:30.0228799Z 10. Successful request 2022-11-18T20:54:30.0329439Z 2022-11-18T20:54:30.0329773Z Iteration 27/82 2022-11-18T20:54:30.0486152Z 2022-11-18T20:54:30.0486804Z → management request 2022-11-18T20:54:30.1045482Z PUT http://localhost:8081/models/squeezenet0_1/1.0/set-default [errored] 2022-11-18T20:54:30.1046527Z read ECONNRESET 2022-11-18T20:54:30.1185896Z 12. Successful request 2022-11-18T20:54:30.1264917Z 2022-11-18T20:54:30.1265335Z Iteration 28/82 2022-11-18T20:54:30.1424089Z 2022-11-18T20:54:30.1424872Z → management request 2022-11-18T20:54:30.2699555Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:54:30.2700523Z 200 OK ★ 79ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:30.2701004Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:54:30.2701338Z │ { 2022-11-18T20:54:30.2701726Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:54:30.2702060Z │ } 2022-11-18T20:54:30.2712702Z └ 2022-11-18T20:54:30.2713233Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:30.2714011Z 1ms 1ms 268µs 262µs 75ms 1ms 46µs 80ms 2022-11-18T20:54:30.2714249Z 2022-11-18T20:54:30.2866769Z √ Successful request 2022-11-18T20:54:30.2985755Z 2022-11-18T20:54:30.2986151Z Iteration 29/82 2022-11-18T20:54:30.3143995Z 2022-11-18T20:54:30.3144666Z → management request 2022-11-18T20:54:30.4624359Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&handler=serve/ts/torch_handler/image_classifier.py:handle 2022-11-18T20:54:30.4625465Z 200 OK ★ 100ms time ★ 357B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:30.4625958Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:54:30.4626279Z │ { 2022-11-18T20:54:30.4626686Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:54:30.4627160Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:54:30.4627575Z │ o add workers for the model." 2022-11-18T20:54:30.4627890Z │ } 2022-11-18T20:54:30.4639251Z └ 2022-11-18T20:54:30.4639773Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:30.4640361Z 1ms 254µs (cache) (cache) 97ms 1ms 76µs 100ms 2022-11-18T20:54:30.4640608Z 2022-11-18T20:54:30.4764875Z √ Successful request 2022-11-18T20:54:30.4866811Z 2022-11-18T20:54:30.4867250Z Iteration 30/82 2022-11-18T20:54:30.5023933Z 2022-11-18T20:54:30.5024638Z → management request 2022-11-18T20:54:30.5583659Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:54:30.5584694Z 200 OK ★ 7ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:30.5585316Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:54:30.5585750Z │ { 2022-11-18T20:54:30.5586194Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:54:30.5586584Z │ } 2022-11-18T20:54:30.5613410Z └ 2022-11-18T20:54:30.5614079Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:30.5614795Z 1ms 264µs (cache) (cache) 4ms 1ms 38µs 8ms 2022-11-18T20:54:30.5615124Z 2022-11-18T20:54:30.5738276Z √ Successful request 2022-11-18T20:54:30.5768832Z 2022-11-18T20:54:30.5769288Z Iteration 31/82 2022-11-18T20:54:30.5806782Z 2022-11-18T20:54:30.5807418Z → management request 2022-11-18T20:54:33.1127357Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&batch_size=3&initial_workers=3&response_timeout=0 2022-11-18T20:54:33.1143190Z 500 Internal Server Error ★ 2.4s time ★ 349B↑ 413B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:33.1143934Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 132B 2022-11-18T20:54:33.1144363Z │ { 2022-11-18T20:54:33.1144727Z │ "code": 500, 2022-11-18T20:54:33.1145164Z │ "type": "InternalServerException", 2022-11-18T20:54:33.1145673Z │ "message": "Failed to start workers for model squeez 2022-11-18T20:54:33.1146107Z │ enet1_1 version: 1.0" 2022-11-18T20:54:33.1146458Z │ } 2022-11-18T20:54:33.1146765Z └ 2022-11-18T20:54:33.1147471Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:33.1148090Z 2ms 394µs (cache) (cache) 2.4s 3ms 97µs 2.4s 2022-11-18T20:54:33.1148377Z 2022-11-18T20:54:33.1302148Z √ Successful request 2022-11-18T20:54:33.1396186Z 2022-11-18T20:54:33.1396455Z Iteration 32/82 2022-11-18T20:54:33.1555329Z 2022-11-18T20:54:33.1555888Z → management request 2022-11-18T20:54:33.2615933Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&response_timeout=0 2022-11-18T20:54:33.2617127Z 200 OK ★ 99ms time ★ 318B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:33.2617992Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:54:33.2618656Z │ { 2022-11-18T20:54:33.2619110Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:54:33.2619604Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:54:33.2620087Z │ o add workers for the model." 2022-11-18T20:54:33.2620456Z │ } 2022-11-18T20:54:33.2636803Z └ 2022-11-18T20:54:33.2637345Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:33.2637963Z 2ms 1ms 492µs 538µs 94ms 1ms 63µs 101ms 2022-11-18T20:54:33.2638277Z 2022-11-18T20:54:33.2695672Z √ Successful request 2022-11-18T20:54:33.2802403Z 2022-11-18T20:54:33.2802679Z Iteration 33/82 2022-11-18T20:54:33.2962647Z 2022-11-18T20:54:33.2963268Z → management request 2022-11-18T20:54:33.3516948Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:54:33.3517960Z 200 OK ★ 7ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:33.3518597Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:54:33.3519016Z │ { 2022-11-18T20:54:33.3519439Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:54:33.3519831Z │ } 2022-11-18T20:54:33.3536403Z └ 2022-11-18T20:54:33.3536967Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:33.3537616Z 1ms 306µs (cache) (cache) 5ms 1ms 72µs 8ms 2022-11-18T20:54:33.3537943Z 2022-11-18T20:54:33.3622065Z √ Successful request 2022-11-18T20:54:33.3740889Z 2022-11-18T20:54:33.3741186Z Iteration 34/82 2022-11-18T20:54:33.3899121Z 2022-11-18T20:54:33.3899695Z → management request 2022-11-18T20:54:37.7732916Z POST http://localhost:8081/models?url=resnet-152-batch.mar&model_name=resnet152&batch_size=2 2022-11-18T20:54:37.7733826Z 200 OK ★ 4.3s time ★ 311B↑ 405B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:37.7734480Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 138B 2022-11-18T20:54:37.7734900Z │ { 2022-11-18T20:54:37.7735345Z │ "status": "Model \"resnet152\" Version: 1.0 register 2022-11-18T20:54:37.7735848Z │ ed with 0 initial workers. Use scale workers API to ad 2022-11-18T20:54:37.7736297Z │ d workers for the model." 2022-11-18T20:54:37.7736655Z │ } 2022-11-18T20:54:37.7749529Z └ 2022-11-18T20:54:37.7750041Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:37.7750671Z 1ms 240µs (cache) (cache) 4.3s 2ms 50µs 4.3s 2022-11-18T20:54:37.7750986Z 2022-11-18T20:54:37.7930660Z √ Successful request 2022-11-18T20:54:37.8038063Z 2022-11-18T20:54:37.8038287Z Iteration 35/82 2022-11-18T20:54:37.8196038Z 2022-11-18T20:54:37.8196432Z → management request 2022-11-18T20:54:37.9063742Z DELETE http://localhost:8081/models/resnet152 2022-11-18T20:54:37.9064876Z 200 OK ★ 35ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:37.9065506Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:54:37.9065939Z │ { 2022-11-18T20:54:37.9066346Z │ "status": "Model \"resnet152\" unregistered" 2022-11-18T20:54:37.9066967Z │ } 2022-11-18T20:54:37.9067285Z └ 2022-11-18T20:54:37.9067820Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:37.9068401Z 2ms 250µs (cache) (cache) 30ms 3ms 39µs 36ms 2022-11-18T20:54:37.9068698Z 2022-11-18T20:54:37.9194566Z √ Successful request 2022-11-18T20:54:37.9290744Z 2022-11-18T20:54:37.9291282Z Iteration 36/82 2022-11-18T20:54:37.9449624Z 2022-11-18T20:54:37.9450445Z → management request 2022-11-18T20:54:45.6615198Z POST http://localhost:8081/models?url=resnet-152-batch.mar&model_name=resnet152&batch_size=dd&initial_workers=1 2022-11-18T20:54:45.6616141Z 200 OK ★ 7.6s time ★ 330B↑ 351B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:45.6616939Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 85B 2022-11-18T20:54:45.6617363Z │ { 2022-11-18T20:54:45.6617820Z │ "status": "Model \"resnet152\" Version: 1.0 register 2022-11-18T20:54:45.6618340Z │ ed with 1 initial workers" 2022-11-18T20:54:45.6618729Z │ } 2022-11-18T20:54:45.6634936Z └ 2022-11-18T20:54:45.6635556Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:45.6636206Z 2ms 272µs (cache) (cache) 7.6s 1ms 45µs 7.6s 2022-11-18T20:54:45.6636539Z 2022-11-18T20:54:45.6809959Z √ Successful request 2022-11-18T20:54:45.6837931Z 2022-11-18T20:54:45.6838340Z Iteration 37/82 2022-11-18T20:54:45.6998500Z 2022-11-18T20:54:45.6999126Z → management request 2022-11-18T20:54:45.8521088Z DELETE http://localhost:8081/models/resnet152 2022-11-18T20:54:45.8522071Z 200 OK ★ 103ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:45.8522602Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:54:45.8522954Z │ { 2022-11-18T20:54:45.8523312Z │ "status": "Model \"resnet152\" unregistered" 2022-11-18T20:54:45.8523642Z │ } 2022-11-18T20:54:45.8536328Z └ 2022-11-18T20:54:45.8536821Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:45.8537374Z 2ms 255µs (cache) (cache) 100ms 2ms 43µs 105ms 2022-11-18T20:54:45.8537612Z 2022-11-18T20:54:45.8666253Z √ Successful request 2022-11-18T20:54:45.8783124Z 2022-11-18T20:54:45.8783870Z Iteration 38/82 2022-11-18T20:54:45.8937016Z 2022-11-18T20:54:45.8937737Z → management request 2022-11-18T20:54:53.5981319Z POST http://localhost:8081/models?url=resnet-152-batch.mar&model_name=resnet152&batch_size=2&initial_workers=1&max_batch_delay=junk 2022-11-18T20:54:53.5982403Z 200 OK ★ 7.6s time ★ 350B↑ 351B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:53.5983127Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 85B 2022-11-18T20:54:53.5983622Z │ { 2022-11-18T20:54:53.5984161Z │ "status": "Model \"resnet152\" Version: 1.0 register 2022-11-18T20:54:53.5984704Z │ ed with 1 initial workers" 2022-11-18T20:54:53.5985145Z │ } 2022-11-18T20:54:53.5988383Z └ 2022-11-18T20:54:53.5988976Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:53.5989661Z 2ms 280µs (cache) (cache) 7.6s 3ms 87µs 7.6s 2022-11-18T20:54:53.5990038Z 2022-11-18T20:54:53.6062550Z √ Successful request 2022-11-18T20:54:53.6096061Z 2022-11-18T20:54:53.6096449Z Iteration 39/82 2022-11-18T20:54:53.6156212Z 2022-11-18T20:54:53.6156833Z → management request 2022-11-18T20:54:53.7715978Z DELETE http://localhost:8081/models/resnet152 2022-11-18T20:54:53.7717151Z 200 OK ★ 107ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:53.7717855Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:54:53.7718306Z │ { 2022-11-18T20:54:53.7718736Z │ "status": "Model \"resnet152\" unregistered" 2022-11-18T20:54:53.7719105Z │ } 2022-11-18T20:54:53.7737944Z └ 2022-11-18T20:54:53.7738779Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:53.7739465Z 1ms 381µs (cache) (cache) 104ms 1ms 47µs 108ms 2022-11-18T20:54:53.7739787Z 2022-11-18T20:54:53.7814859Z √ Successful request 2022-11-18T20:54:53.7926236Z 2022-11-18T20:54:53.7926597Z Iteration 40/82 2022-11-18T20:54:53.8084818Z 2022-11-18T20:54:53.8085326Z → management request 2022-11-18T20:54:53.9556281Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1&initial_workers=-1 2022-11-18T20:54:53.9557177Z 200 OK ★ 98ms time ★ 318B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:53.9557760Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:54:53.9558385Z │ { 2022-11-18T20:54:53.9558845Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:54:53.9559355Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:54:53.9559807Z │ o add workers for the model." 2022-11-18T20:54:53.9560166Z │ } 2022-11-18T20:54:53.9570894Z └ 2022-11-18T20:54:53.9571434Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:53.9572069Z 1ms 327µs (cache) (cache) 95ms 2ms 40µs 99ms 2022-11-18T20:54:53.9572392Z 2022-11-18T20:54:53.9691532Z √ Successful request 2022-11-18T20:54:53.9802636Z 2022-11-18T20:54:53.9803069Z Iteration 41/82 2022-11-18T20:54:53.9960301Z 2022-11-18T20:54:53.9961069Z → management request 2022-11-18T20:54:54.0516520Z DELETE http://localhost:8081/models/squeezenet1_1 2022-11-18T20:54:54.0517517Z 200 OK ★ 7ms time ★ 249B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:54.0518133Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:54:54.0518570Z │ { 2022-11-18T20:54:54.0519010Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:54:54.0519424Z │ } 2022-11-18T20:54:54.0531017Z └ 2022-11-18T20:54:54.0531577Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:54.0532186Z 2ms 330µs (cache) (cache) 4ms 1ms 35µs 8ms 2022-11-18T20:54:54.0532514Z 2022-11-18T20:54:54.0637237Z √ Successful request 2022-11-18T20:54:54.0742684Z 2022-11-18T20:54:54.0743183Z Iteration 42/82 2022-11-18T20:54:54.0900592Z 2022-11-18T20:54:54.0901456Z → management request 2022-11-18T20:54:57.4256088Z POST http://localhost:8081/models?url=resnet-18.mar&model_name=resnet-18&synchronous=true 2022-11-18T20:54:57.4257055Z 200 OK ★ 3.2s time ★ 308B↑ 405B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:57.4257694Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 138B 2022-11-18T20:54:57.4258128Z │ { 2022-11-18T20:54:57.4258581Z │ "status": "Model \"resnet-18\" Version: 1.0 register 2022-11-18T20:54:57.4259076Z │ ed with 0 initial workers. Use scale workers API to ad 2022-11-18T20:54:57.4259519Z │ d workers for the model." 2022-11-18T20:54:57.4259873Z │ } 2022-11-18T20:54:57.4277388Z └ 2022-11-18T20:54:57.4277983Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:57.4278636Z 1ms 302µs (cache) (cache) 3.2s 1ms 44µs 3.2s 2022-11-18T20:54:57.4278954Z 2022-11-18T20:54:57.4394222Z √ Successful request 2022-11-18T20:54:57.4494275Z 2022-11-18T20:54:57.4494463Z Iteration 43/82 2022-11-18T20:54:57.4647912Z 2022-11-18T20:54:57.4648780Z → management request 2022-11-18T20:54:57.5257720Z DELETE http://localhost:8081/models/resnet-18 2022-11-18T20:54:57.5258587Z 200 OK ★ 13ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:57.5259179Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:54:57.5259597Z │ { 2022-11-18T20:54:57.5260249Z │ "status": "Model \"resnet-18\" unregistered" 2022-11-18T20:54:57.5260630Z │ } 2022-11-18T20:54:57.5271005Z └ 2022-11-18T20:54:57.5271560Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:57.5272219Z 1ms 311µs (cache) (cache) 10ms 1ms 36µs 13ms 2022-11-18T20:54:57.5272538Z 2022-11-18T20:54:57.5520725Z √ Successful request 2022-11-18T20:54:57.5526932Z 2022-11-18T20:54:57.5527416Z Iteration 44/82 2022-11-18T20:54:57.5585870Z 2022-11-18T20:54:57.5586436Z → management request 2022-11-18T20:54:58.8729873Z POST http://localhost:8081/models?url=resnet-18.mar&model_name=resnet-18&synchronous=-1 2022-11-18T20:54:58.8730460Z 200 OK ★ 829ms time ★ 306B↑ 405B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:58.8731103Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 138B 2022-11-18T20:54:58.8731362Z │ { 2022-11-18T20:54:58.8731690Z │ "status": "Model \"resnet-18\" Version: 1.0 register 2022-11-18T20:54:58.8732093Z │ ed with 0 initial workers. Use scale workers API to ad 2022-11-18T20:54:58.8732407Z │ d workers for the model." 2022-11-18T20:54:58.8732657Z │ } 2022-11-18T20:54:58.8732849Z └ 2022-11-18T20:54:58.8733261Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:58.8733735Z 1ms 348µs (cache) (cache) 826ms 1ms 47µs 830ms 2022-11-18T20:54:58.8733919Z 2022-11-18T20:54:58.8734063Z √ Successful request 2022-11-18T20:54:58.8734206Z 2022-11-18T20:54:58.8734293Z Iteration 45/82 2022-11-18T20:54:58.8734428Z 2022-11-18T20:54:58.8734540Z → management request 2022-11-18T20:54:58.8734897Z DELETE http://localhost:8081/models/resnet-18 2022-11-18T20:54:58.8735329Z 200 OK ★ 12ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:58.8735705Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:54:58.8735957Z │ { 2022-11-18T20:54:58.8736268Z │ "status": "Model \"resnet-18\" unregistered" 2022-11-18T20:54:58.8736535Z │ } 2022-11-18T20:54:58.8736705Z └ 2022-11-18T20:54:58.8737106Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:58.8737580Z 1ms 321µs (cache) (cache) 9ms 1ms 39µs 12ms 2022-11-18T20:54:58.8737762Z 2022-11-18T20:54:58.8737883Z √ Successful request 2022-11-18T20:54:58.8738025Z 2022-11-18T20:54:58.8738111Z Iteration 46/82 2022-11-18T20:54:58.8738243Z 2022-11-18T20:54:58.8738374Z → management request 2022-11-18T20:54:59.4538551Z POST http://localhost:8081/models?url=resnet-18.mar&model_name=resnet-18&synchronous=false 2022-11-18T20:54:59.4539483Z 200 OK ★ 830ms time ★ 309B↑ 405B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:59.4540084Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 138B 2022-11-18T20:54:59.4540472Z │ { 2022-11-18T20:54:59.4541132Z │ "status": "Model \"resnet-18\" Version: 1.0 register 2022-11-18T20:54:59.4541645Z │ ed with 0 initial workers. Use scale workers API to ad 2022-11-18T20:54:59.4542096Z │ d workers for the model." 2022-11-18T20:54:59.4542444Z │ } 2022-11-18T20:54:59.4551592Z └ 2022-11-18T20:54:59.4552148Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:59.4552774Z 1ms 302µs (cache) (cache) 826ms 2ms 41µs 830ms 2022-11-18T20:54:59.4553128Z 2022-11-18T20:54:59.4709818Z √ Successful request 2022-11-18T20:54:59.4823968Z 2022-11-18T20:54:59.4824270Z Iteration 47/82 2022-11-18T20:54:59.4981678Z 2022-11-18T20:54:59.4982415Z → management request 2022-11-18T20:54:59.5511525Z GET http://localhost:8081/models?limit=1 2022-11-18T20:54:59.5512966Z 200 OK ★ 4ms time ★ 240B↑ 391B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:59.5513536Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 124B 2022-11-18T20:54:59.5513915Z │ { 2022-11-18T20:54:59.5514300Z │ "nextPageToken": "1", 2022-11-18T20:54:59.5514673Z │ "models": [ 2022-11-18T20:54:59.5514999Z │ { 2022-11-18T20:54:59.5515370Z │ "modelName": "resnet-18", 2022-11-18T20:54:59.5515800Z │ "modelUrl": "resnet-18.mar" 2022-11-18T20:54:59.5516162Z │ } 2022-11-18T20:54:59.5516454Z │ ] 2022-11-18T20:54:59.5516755Z │ } 2022-11-18T20:54:59.5524208Z └ 2022-11-18T20:54:59.5524802Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:59.5525414Z 2ms 325µs (cache) (cache) 1ms 1ms 33µs 5ms 2022-11-18T20:54:59.5525701Z 2022-11-18T20:54:59.5600584Z √ Successful request 2022-11-18T20:54:59.5764698Z 2022-11-18T20:54:59.5765245Z Iteration 48/82 2022-11-18T20:54:59.5922097Z 2022-11-18T20:54:59.5922885Z → management request 2022-11-18T20:54:59.6452403Z GET http://localhost:8081/models?limit=-1 2022-11-18T20:54:59.6453365Z 200 OK ★ 4ms time ★ 241B↑ 367B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:59.6454055Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 100B 2022-11-18T20:54:59.6454510Z │ { 2022-11-18T20:54:59.6454903Z │ "models": [ 2022-11-18T20:54:59.6455294Z │ { 2022-11-18T20:54:59.6455748Z │ "modelName": "resnet-18", 2022-11-18T20:54:59.6456219Z │ "modelUrl": "resnet-18.mar" 2022-11-18T20:54:59.6456635Z │ } 2022-11-18T20:54:59.6457006Z │ ] 2022-11-18T20:54:59.6457347Z │ } 2022-11-18T20:54:59.6475482Z └ 2022-11-18T20:54:59.6476308Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:59.6477105Z 2ms 299µs (cache) (cache) 1ms 1ms 38µs 5ms 2022-11-18T20:54:59.6477506Z 2022-11-18T20:54:59.6581883Z √ Successful request 2022-11-18T20:54:59.6703383Z 2022-11-18T20:54:59.6703690Z Iteration 49/82 2022-11-18T20:54:59.6859479Z 2022-11-18T20:54:59.6860044Z → management request 2022-11-18T20:54:59.7383880Z GET http://localhost:8081/models?limit=1&next_page_token=1 2022-11-18T20:54:59.7385038Z 200 OK ★ 4ms time ★ 258B↑ 285B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:59.7385736Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 19B 2022-11-18T20:54:59.7386219Z │ { 2022-11-18T20:54:59.7386615Z │ "models": [] 2022-11-18T20:54:59.7386982Z │ } 2022-11-18T20:54:59.7522743Z └ 2022-11-18T20:54:59.7529629Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:59.7530574Z 1ms 302µs (cache) (cache) 1ms 1ms 44µs 4ms 2022-11-18T20:54:59.7535521Z 2022-11-18T20:54:59.7538000Z √ Successful request 2022-11-18T20:54:59.7641406Z 2022-11-18T20:54:59.7641753Z Iteration 50/82 2022-11-18T20:54:59.7798251Z 2022-11-18T20:54:59.7798765Z → management request 2022-11-18T20:54:59.8321488Z GET http://localhost:8081/models?limit=1&next_page_token=-1 2022-11-18T20:54:59.8322660Z 200 OK ★ 4ms time ★ 259B↑ 391B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:59.8323362Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 124B 2022-11-18T20:54:59.8323847Z │ { 2022-11-18T20:54:59.8324340Z │ "nextPageToken": "1", 2022-11-18T20:54:59.8324715Z │ "models": [ 2022-11-18T20:54:59.8325059Z │ { 2022-11-18T20:54:59.8325514Z │ "modelName": "resnet-18", 2022-11-18T20:54:59.8325997Z │ "modelUrl": "resnet-18.mar" 2022-11-18T20:54:59.8326419Z │ } 2022-11-18T20:54:59.8334510Z │ ] 2022-11-18T20:54:59.8335098Z │ } 2022-11-18T20:54:59.8357803Z └ 2022-11-18T20:54:59.8358453Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:59.8359340Z 1ms 294µs (cache) (cache) 1ms 1ms 37µs 4ms 2022-11-18T20:54:59.8359660Z 2022-11-18T20:54:59.8491418Z √ Successful request 2022-11-18T20:54:59.8577524Z 2022-11-18T20:54:59.8577844Z Iteration 51/82 2022-11-18T20:54:59.8735112Z 2022-11-18T20:54:59.8735646Z → management request 2022-11-18T20:54:59.9264805Z PUT http://localhost:8081/models/resnet-18?number_gpu=10 2022-11-18T20:54:59.9265664Z 202 Accepted ★ 4ms time ★ 275B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:54:59.9266306Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:54:59.9266724Z │ { 2022-11-18T20:54:59.9267149Z │ "status": "Processing worker updates..." 2022-11-18T20:54:59.9267543Z │ } 2022-11-18T20:54:59.9360079Z └ 2022-11-18T20:54:59.9362385Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:54:59.9411306Z 1ms 290µs (cache) (cache) 1ms 1ms 35µs 4ms 2022-11-18T20:54:59.9411588Z 2022-11-18T20:54:59.9416545Z √ Successful request 2022-11-18T20:54:59.9519424Z 2022-11-18T20:54:59.9519822Z Iteration 52/82 2022-11-18T20:54:59.9520218Z 2022-11-18T20:54:59.9520699Z → management request 2022-11-18T20:55:00.0063449Z PUT http://localhost:8081/models/resnet-18?number_gpu=-1 2022-11-18T20:55:00.0064346Z 202 Accepted ★ 4ms time ★ 275B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:00.0064949Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:00.0065379Z │ { 2022-11-18T20:55:00.0065784Z │ "status": "Processing worker updates..." 2022-11-18T20:55:00.0066170Z │ } 2022-11-18T20:55:00.0066477Z └ 2022-11-18T20:55:00.0067018Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:00.0067603Z 1ms 284µs (cache) (cache) 1ms 1ms 37µs 4ms 2022-11-18T20:55:00.0067931Z 2022-11-18T20:55:00.0340513Z √ Successful request 2022-11-18T20:55:00.0455065Z 2022-11-18T20:55:00.0455601Z Iteration 53/82 2022-11-18T20:55:00.0612475Z 2022-11-18T20:55:00.0613324Z → management request 2022-11-18T20:55:00.1160331Z PUT http://localhost:8081/models/resnet-18?min_worker=1&max_worker=1&synchronous=true 2022-11-18T20:55:00.1161538Z 200 OK ★ 5ms time ★ 304B↑ 325B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:00.1162156Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 59B 2022-11-18T20:55:00.1162576Z │ { 2022-11-18T20:55:00.1163042Z │ "status": "Workers scaled to 1 for model: resnet-18" 2022-11-18T20:55:00.1163436Z │ } 2022-11-18T20:55:00.1163724Z └ 2022-11-18T20:55:00.1164248Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:00.1164853Z 1ms 309µs (cache) (cache) 1ms 2ms 40µs 5ms 2022-11-18T20:55:00.1165149Z 2022-11-18T20:55:00.1347585Z √ Successful request 2022-11-18T20:55:00.1390256Z 2022-11-18T20:55:00.1390714Z Iteration 54/82 2022-11-18T20:55:00.1549763Z 2022-11-18T20:55:00.1550568Z → management request 2022-11-18T20:55:00.2101003Z PUT http://localhost:8081/models/resnet-18?min_worker=1&max_worker=1&synchronous=false 2022-11-18T20:55:00.2102121Z 202 Accepted ★ 4ms time ★ 305B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:00.2102714Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:00.2103133Z │ { 2022-11-18T20:55:00.2103540Z │ "status": "Processing worker updates..." 2022-11-18T20:55:00.2103933Z │ } 2022-11-18T20:55:00.2104240Z └ 2022-11-18T20:55:00.2104742Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:00.2105350Z 2ms 301µs (cache) (cache) 1ms 1ms 48µs 5ms 2022-11-18T20:55:00.2105643Z 2022-11-18T20:55:00.2364755Z √ Successful request 2022-11-18T20:55:00.2485109Z 2022-11-18T20:55:00.2485630Z Iteration 55/82 2022-11-18T20:55:00.2643584Z 2022-11-18T20:55:00.2644416Z → management request 2022-11-18T20:55:00.3029810Z PUT http://localhost:8081/models/resnet-18?timeout=-1 2022-11-18T20:55:00.3030766Z 202 Accepted ★ 4ms time ★ 272B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:00.3031379Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:00.3031800Z │ { 2022-11-18T20:55:00.3032203Z │ "status": "Processing worker updates..." 2022-11-18T20:55:00.3032587Z │ } 2022-11-18T20:55:00.3032887Z └ 2022-11-18T20:55:00.3033398Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:00.3033997Z 2ms 289µs (cache) (cache) 1ms 1ms 35µs 5ms 2022-11-18T20:55:00.3034283Z 2022-11-18T20:55:00.3170301Z √ Successful request 2022-11-18T20:55:00.3264778Z 2022-11-18T20:55:00.3265156Z Iteration 56/82 2022-11-18T20:55:00.3423926Z 2022-11-18T20:55:00.3424723Z → management request 2022-11-18T20:55:00.3961816Z PUT http://localhost:8081/models/resnet-18?timeout=0 2022-11-18T20:55:00.3962736Z 202 Accepted ★ 4ms time ★ 271B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:00.3963373Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:00.3963801Z │ { 2022-11-18T20:55:00.3964224Z │ "status": "Processing worker updates..." 2022-11-18T20:55:00.3964616Z │ } 2022-11-18T20:55:00.3964900Z └ 2022-11-18T20:55:00.3965426Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:00.3966029Z 1ms 279µs (cache) (cache) 2ms 1ms 36µs 5ms 2022-11-18T20:55:00.3966315Z 2022-11-18T20:55:00.4111984Z √ Successful request 2022-11-18T20:55:00.4201979Z 2022-11-18T20:55:00.4202355Z Iteration 57/82 2022-11-18T20:55:00.4360272Z 2022-11-18T20:55:00.4360914Z → management request 2022-11-18T20:55:00.5143513Z POST http://localhost:8081/models?url=&model_name=resnet-18 2022-11-18T20:55:00.5160664Z 404 Not Found ★ 4ms time ★ 278B↑ 348B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:00.5161787Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 80B 2022-11-18T20:55:00.5168523Z │ { 2022-11-18T20:55:00.5169046Z │ "code": 404, 2022-11-18T20:55:00.5169498Z │ "type": "ModelNotFoundException", 2022-11-18T20:55:00.5169946Z │ "message": "empty url" 2022-11-18T20:55:00.5170286Z │ } 2022-11-18T20:55:00.5170595Z └ 2022-11-18T20:55:00.5171125Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:00.5171702Z 1ms 286µs (cache) (cache) 1ms 1ms 43µs 4ms 2022-11-18T20:55:00.5171992Z 2022-11-18T20:55:00.5172245Z √ Successful request 2022-11-18T20:55:00.5302282Z 2022-11-18T20:55:00.5302628Z Iteration 58/82 2022-11-18T20:55:00.5459052Z 2022-11-18T20:55:00.5459662Z → management request 2022-11-18T20:55:01.0125812Z POST http://localhost:8081/models?url=https://torchserve.pytorch.org/mar_files/invalid-resnet-18.mar&model_name=invalid-resnet18 [errored] 2022-11-18T20:55:01.0130058Z read ECONNRESET 2022-11-18T20:55:01.0131015Z 14. Successful request 2022-11-18T20:55:01.0148799Z 2022-11-18T20:55:01.0149142Z Iteration 59/82 2022-11-18T20:55:01.0306500Z 2022-11-18T20:55:01.0307203Z → management request 2022-11-18T20:55:01.0914174Z GET http://localhost:8081/models/invalid_squeezenet1_1 2022-11-18T20:55:01.0915835Z 404 Not Found ★ 5ms time ★ 254B↑ 378B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:01.0916729Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 109B 2022-11-18T20:55:01.0921174Z │ { 2022-11-18T20:55:01.0921632Z │ "code": 404, 2022-11-18T20:55:01.0922075Z │ "type": "ModelNotFoundException", 2022-11-18T20:55:01.0922580Z │ "message": "Model not found: invalid_squeezenet1_1" 2022-11-18T20:55:01.0922994Z │ } 2022-11-18T20:55:01.0923556Z └ 2022-11-18T20:55:01.0924145Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:01.0924747Z 1ms 798µs 453µs 289µs 1ms 1ms 50µs 6ms 2022-11-18T20:55:01.0925045Z 2022-11-18T20:55:01.0925301Z √ Successful request 2022-11-18T20:55:01.1095135Z 2022-11-18T20:55:01.1095565Z Iteration 60/82 2022-11-18T20:55:01.1243374Z 2022-11-18T20:55:01.1244017Z → management request 2022-11-18T20:55:01.2114596Z GET http://localhost:8081/models/squeezenet1_1/0.0 [errored] 2022-11-18T20:55:01.2115895Z read ECONNRESET 2022-11-18T20:55:01.2116595Z 16. Successful request 2022-11-18T20:55:01.2180214Z 2022-11-18T20:55:01.2180653Z Iteration 61/82 2022-11-18T20:55:01.2337929Z 2022-11-18T20:55:01.2338685Z → management request 2022-11-18T20:55:01.2883089Z GET http://localhost:8081/models?next_page_token=12 2022-11-18T20:55:01.2884103Z 200 OK ★ 5ms time ★ 251B↑ 285B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:01.2884612Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 19B 2022-11-18T20:55:01.2884951Z │ { 2022-11-18T20:55:01.2885232Z │ "models": [] 2022-11-18T20:55:01.2885516Z │ } 2022-11-18T20:55:01.2885766Z └ 2022-11-18T20:55:01.2886241Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:01.2886797Z 1ms 869µs 222µs 255µs 1ms 1ms 35µs 5ms 2022-11-18T20:55:01.2887031Z 2022-11-18T20:55:01.3122788Z √ Successful request 2022-11-18T20:55:01.3272631Z 2022-11-18T20:55:01.3273091Z Iteration 62/82 2022-11-18T20:55:01.3431763Z 2022-11-18T20:55:01.3432447Z → management request 2022-11-18T20:55:01.4050223Z PUT http://localhost:8081/models/resnet-18?min_worker=1&synchronous=Nan 2022-11-18T20:55:01.4052441Z 202 Accepted ★ 4ms time ★ 290B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:01.4053240Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:01.4057947Z │ { 2022-11-18T20:55:01.4058423Z │ "status": "Processing worker updates..." 2022-11-18T20:55:01.4058765Z │ } 2022-11-18T20:55:01.4059022Z └ 2022-11-18T20:55:01.4059507Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:01.4060070Z 1ms 273µs (cache) (cache) 1ms 1ms 47µs 5ms 2022-11-18T20:55:01.4060315Z 2022-11-18T20:55:01.4060519Z √ Successful request 2022-11-18T20:55:01.4210385Z 2022-11-18T20:55:01.4210832Z Iteration 63/82 2022-11-18T20:55:01.4368315Z 2022-11-18T20:55:01.4368898Z → management request 2022-11-18T20:55:01.4903760Z PUT http://localhost:8081/models/resnet-18?min_worker=nan&synchronous=nan 2022-11-18T20:55:01.4904598Z 202 Accepted ★ 4ms time ★ 292B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:01.4905085Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:01.4905423Z │ { 2022-11-18T20:55:01.4905811Z │ "status": "Processing worker updates..." 2022-11-18T20:55:01.4906144Z │ } 2022-11-18T20:55:01.4906394Z └ 2022-11-18T20:55:01.4907089Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:01.4907639Z 1ms 232µs (cache) (cache) 1ms 1ms 37µs 4ms 2022-11-18T20:55:01.4907877Z 2022-11-18T20:55:01.5047744Z √ Successful request 2022-11-18T20:55:01.5147850Z 2022-11-18T20:55:01.5151207Z Iteration 64/82 2022-11-18T20:55:01.5305543Z 2022-11-18T20:55:01.5306136Z → management request 2022-11-18T20:55:01.6074317Z PUT http://localhost:8081/models/resnet-18 2022-11-18T20:55:01.6075771Z 202 Accepted ★ 4ms time ★ 261B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:01.6076543Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:01.6083599Z │ { 2022-11-18T20:55:01.6084077Z │ "status": "Processing worker updates..." 2022-11-18T20:55:01.6085354Z │ } 2022-11-18T20:55:01.6085613Z └ 2022-11-18T20:55:01.6086125Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:01.6086681Z 1ms 245µs (cache) (cache) 1ms 1ms 39µs 4ms 2022-11-18T20:55:01.6086935Z 2022-11-18T20:55:01.6087138Z √ Successful request 2022-11-18T20:55:01.6092391Z 2022-11-18T20:55:01.6092740Z Iteration 65/82 2022-11-18T20:55:01.6251989Z 2022-11-18T20:55:01.6252701Z → management request 2022-11-18T20:55:01.6820879Z PUT http://localhost:8081/models/resnet181?min_worker=1 2022-11-18T20:55:01.6821983Z 404 Not Found ★ 4ms time ★ 274B↑ 365B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:01.6822603Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 97B 2022-11-18T20:55:01.6823023Z │ { 2022-11-18T20:55:01.6823386Z │ "code": 404, 2022-11-18T20:55:01.6823797Z │ "type": "ModelNotFoundException", 2022-11-18T20:55:01.6824264Z │ "message": "Model not found: resnet181" 2022-11-18T20:55:01.6824670Z │ } 2022-11-18T20:55:01.6824978Z └ 2022-11-18T20:55:01.6825488Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:01.6826119Z 1ms 244µs (cache) (cache) 1ms 1ms 40µs 4ms 2022-11-18T20:55:01.6826414Z 2022-11-18T20:55:01.6936526Z √ Successful request 2022-11-18T20:55:01.7029688Z 2022-11-18T20:55:01.7030141Z Iteration 66/82 2022-11-18T20:55:01.7188398Z 2022-11-18T20:55:01.7189040Z → management request 2022-11-18T20:55:01.7734117Z PUT http://localhost:8081/models/resnet-18?min_worker=2&max_worker=1 [errored] 2022-11-18T20:55:01.7734776Z read ECONNRESET 2022-11-18T20:55:01.7868213Z 18. Successful request 2022-11-18T20:55:01.7970433Z 2022-11-18T20:55:01.7970880Z Iteration 67/82 2022-11-18T20:55:01.8128813Z 2022-11-18T20:55:01.8129464Z → management request 2022-11-18T20:55:01.8677031Z PUT http://localhost:8081/models/resnet-18?min_worker=1 2022-11-18T20:55:01.8677984Z 202 Accepted ★ 5ms time ★ 274B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:01.8678602Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:01.8679021Z │ { 2022-11-18T20:55:01.8679447Z │ "status": "Processing worker updates..." 2022-11-18T20:55:01.8679835Z │ } 2022-11-18T20:55:01.8680138Z └ 2022-11-18T20:55:01.8680669Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:01.8681243Z 1ms 682µs 210µs 494µs 1ms 1ms 37µs 5ms 2022-11-18T20:55:01.8681530Z 2022-11-18T20:55:01.8798517Z √ Successful request 2022-11-18T20:55:01.8908382Z 2022-11-18T20:55:01.8908793Z Iteration 68/82 2022-11-18T20:55:01.9065748Z 2022-11-18T20:55:01.9066384Z → management request 2022-11-18T20:55:02.0258378Z PUT http://localhost:8081/models/resnet-18?min_worker=0 2022-11-18T20:55:02.0259241Z 202 Accepted ★ 71ms time ★ 274B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:02.0259872Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:02.0260274Z │ { 2022-11-18T20:55:02.0260695Z │ "status": "Processing worker updates..." 2022-11-18T20:55:02.0261254Z │ } 2022-11-18T20:55:02.0270596Z └ 2022-11-18T20:55:02.0271119Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:02.0271751Z 1ms 253µs (cache) (cache) 67ms 2ms 50µs 71ms 2022-11-18T20:55:02.0272071Z 2022-11-18T20:55:02.0389630Z √ Successful request 2022-11-18T20:55:02.0407551Z 2022-11-18T20:55:02.0407797Z Iteration 69/82 2022-11-18T20:55:02.0514960Z 2022-11-18T20:55:02.0515373Z → management request 2022-11-18T20:55:02.1042285Z PUT http://localhost:8081/models/resnet-18?min_worker=-1 [errored] 2022-11-18T20:55:02.1042605Z read ECONNRESET 2022-11-18T20:55:02.1175875Z 20. Successful request 2022-11-18T20:55:02.1296290Z 2022-11-18T20:55:02.1296612Z Iteration 70/82 2022-11-18T20:55:02.1298648Z 2022-11-18T20:55:02.1299103Z → management request 2022-11-18T20:55:02.1835555Z PUT http://localhost:8081/models/resnet-18?max_worker=-1 [errored] 2022-11-18T20:55:02.1836160Z read ECONNRESET 2022-11-18T20:55:02.1957059Z 22. Successful request 2022-11-18T20:55:02.2078337Z 2022-11-18T20:55:02.2078721Z Iteration 71/82 2022-11-18T20:55:02.2236087Z 2022-11-18T20:55:02.2236800Z → management request 2022-11-18T20:55:02.2759943Z PUT http://localhost:8081/models/invalid_squeezenet1_1/1.0/set-default [errored] 2022-11-18T20:55:02.2760597Z read ECONNRESET 2022-11-18T20:55:02.2893124Z 24. Successful request 2022-11-18T20:55:02.3015076Z 2022-11-18T20:55:02.3015371Z Iteration 72/82 2022-11-18T20:55:02.3173270Z 2022-11-18T20:55:02.3173901Z → management request 2022-11-18T20:55:02.3846261Z DELETE http://localhost:8081/models/resnet-18 2022-11-18T20:55:02.3847481Z 200 OK ★ 12ms time ★ 245B↑ 317B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:02.3878967Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 51B 2022-11-18T20:55:02.3879901Z │ { 2022-11-18T20:55:02.3883562Z │ "status": "Model \"resnet-18\" unregistered" 2022-11-18T20:55:02.3883997Z │ } 2022-11-18T20:55:02.3884313Z └ 2022-11-18T20:55:02.3884854Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:02.3885455Z 3ms 629µs 203µs 230µs 9ms 1ms 37µs 14ms 2022-11-18T20:55:02.3885739Z 2022-11-18T20:55:02.3886050Z √ Successful request 2022-11-18T20:55:02.3955667Z 2022-11-18T20:55:02.3955958Z Iteration 73/82 2022-11-18T20:55:02.4109969Z 2022-11-18T20:55:02.4110430Z → management request 2022-11-18T20:55:02.4623833Z DELETE http://localhost:8081/models/squeezenet1_1/0.0 [errored] 2022-11-18T20:55:02.4624571Z read ECONNRESET 2022-11-18T20:55:02.4792870Z 26. Successful request 2022-11-18T20:55:02.4933439Z 2022-11-18T20:55:02.4933878Z Iteration 74/82 2022-11-18T20:55:02.5053678Z 2022-11-18T20:55:02.5054381Z → management request 2022-11-18T20:55:02.6525302Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:55:02.6526657Z 200 OK ★ 99ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:02.6527353Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:55:02.6527830Z │ { 2022-11-18T20:55:02.6528430Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:55:02.6528934Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:55:02.6529389Z │ o add workers for the model." 2022-11-18T20:55:02.6529728Z │ } 2022-11-18T20:55:02.6537144Z └ 2022-11-18T20:55:02.6537692Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:02.6538321Z 1ms 748µs 231µs 250µs 95ms 1ms 42µs 99ms 2022-11-18T20:55:02.6538797Z 2022-11-18T20:55:02.6679091Z √ Successful request 2022-11-18T20:55:02.6771053Z 2022-11-18T20:55:02.6771459Z Iteration 75/82 2022-11-18T20:55:02.6929518Z 2022-11-18T20:55:02.6930229Z → management request 2022-11-18T20:55:02.7480615Z DELETE http://localhost:8081/models/squeezenet1_1/?synchronous=true 2022-11-18T20:55:02.7483073Z 200 OK ★ 7ms time ★ 267B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:02.7483817Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:55:02.7484262Z │ { 2022-11-18T20:55:02.7484704Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:55:02.7485092Z │ } 2022-11-18T20:55:02.7492000Z └ 2022-11-18T20:55:02.7492533Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:02.7493149Z 1ms 234µs (cache) (cache) 4ms 1ms 39µs 7ms 2022-11-18T20:55:02.7493462Z 2022-11-18T20:55:02.7623922Z √ Successful request 2022-11-18T20:55:02.7713212Z 2022-11-18T20:55:02.7713522Z Iteration 76/82 2022-11-18T20:55:02.7869509Z 2022-11-18T20:55:02.7870049Z → management request 2022-11-18T20:55:02.9359982Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:55:02.9361284Z 200 OK ★ 100ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:02.9361928Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:55:02.9362348Z │ { 2022-11-18T20:55:02.9362803Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:55:02.9363304Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:55:02.9363739Z │ o add workers for the model." 2022-11-18T20:55:02.9364099Z │ } 2022-11-18T20:55:02.9371484Z └ 2022-11-18T20:55:02.9372032Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:02.9372649Z 1ms 251µs (cache) (cache) 98ms 1ms 43µs 101ms 2022-11-18T20:55:02.9372986Z 2022-11-18T20:55:02.9414127Z √ Successful request 2022-11-18T20:55:02.9589686Z 2022-11-18T20:55:02.9590131Z Iteration 77/82 2022-11-18T20:55:02.9746807Z 2022-11-18T20:55:02.9747669Z → management request 2022-11-18T20:55:03.0145312Z DELETE http://localhost:8081/models/squeezenet1_1/?synchronous=nan 2022-11-18T20:55:03.0146552Z 200 OK ★ 7ms time ★ 266B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:03.0147254Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:55:03.0147746Z │ { 2022-11-18T20:55:03.0148246Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:55:03.0148672Z │ } 2022-11-18T20:55:03.0157064Z └ 2022-11-18T20:55:03.0157689Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:03.0158422Z 1ms 239µs (cache) (cache) 5ms 1ms 37µs 8ms 2022-11-18T20:55:03.0159045Z 2022-11-18T20:55:03.0288816Z √ Successful request 2022-11-18T20:55:03.0372639Z 2022-11-18T20:55:03.0372902Z Iteration 78/82 2022-11-18T20:55:03.0529190Z 2022-11-18T20:55:03.0530057Z → management request 2022-11-18T20:55:03.2002349Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:55:03.2003805Z 200 OK ★ 99ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:03.2004502Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:55:03.2004979Z │ { 2022-11-18T20:55:03.2010473Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:55:03.2011137Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:55:03.2011665Z │ o add workers for the model." 2022-11-18T20:55:03.2012079Z │ } 2022-11-18T20:55:03.2021975Z └ 2022-11-18T20:55:03.2022471Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:03.2023269Z 1ms 259µs (cache) (cache) 96ms 1ms 43µs 99ms 2022-11-18T20:55:03.2023593Z 2022-11-18T20:55:03.2119178Z √ Successful request 2022-11-18T20:55:03.2243509Z 2022-11-18T20:55:03.2243799Z Iteration 79/82 2022-11-18T20:55:03.2401991Z 2022-11-18T20:55:03.2402525Z → management request 2022-11-18T20:55:03.2954662Z DELETE http://localhost:8081/models/squeezenet1_1/?timeout=true 2022-11-18T20:55:03.2955764Z 200 OK ★ 7ms time ★ 263B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:03.2956379Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:55:03.2956808Z │ { 2022-11-18T20:55:03.2957271Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:55:03.2957647Z │ } 2022-11-18T20:55:03.2966049Z └ 2022-11-18T20:55:03.2966600Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:03.2967233Z 1ms 245µs (cache) (cache) 4ms 1ms 35µs 7ms 2022-11-18T20:55:03.2967584Z 2022-11-18T20:55:03.3036241Z √ Successful request 2022-11-18T20:55:03.3051220Z 2022-11-18T20:55:03.3051494Z Iteration 80/82 2022-11-18T20:55:03.3185357Z 2022-11-18T20:55:03.3185905Z → management request 2022-11-18T20:55:03.4661409Z POST http://localhost:8081/models?url=squeezenet1_1.mar&model_name=squeezenet1_1 2022-11-18T20:55:03.4662492Z 200 OK ★ 99ms time ★ 299B↑ 409B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:03.4663032Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 142B 2022-11-18T20:55:03.4663420Z │ { 2022-11-18T20:55:03.4663855Z │ "status": "Model \"squeezenet1_1\" Version: 1.0 regi 2022-11-18T20:55:03.4664373Z │ stered with 0 initial workers. Use scale workers API t 2022-11-18T20:55:03.4664831Z │ o add workers for the model." 2022-11-18T20:55:03.4665178Z │ } 2022-11-18T20:55:03.4673588Z └ 2022-11-18T20:55:03.4674170Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:03.4674832Z 2ms 255µs (cache) (cache) 96ms 1ms 48µs 100ms 2022-11-18T20:55:03.4675103Z 2022-11-18T20:55:03.4777530Z √ Successful request 2022-11-18T20:55:03.4902639Z 2022-11-18T20:55:03.4903374Z Iteration 81/82 2022-11-18T20:55:03.5063745Z 2022-11-18T20:55:03.5064193Z → management request 2022-11-18T20:55:03.5614864Z DELETE http://localhost:8081/models/squeezenet1_1/?timeout=true&synchronous=-1 2022-11-18T20:55:03.5615772Z 200 OK ★ 6ms time ★ 278B↑ 321B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:03.5616256Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:55:03.5616586Z │ { 2022-11-18T20:55:03.5616967Z │ "status": "Model \"squeezenet1_1\" unregistered" 2022-11-18T20:55:03.5617301Z │ } 2022-11-18T20:55:03.5630319Z └ 2022-11-18T20:55:03.5630916Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:03.5631515Z 1ms 252µs (cache) (cache) 4ms 1ms 36µs 7ms 2022-11-18T20:55:03.5631757Z 2022-11-18T20:55:03.5669935Z √ Successful request 2022-11-18T20:55:03.5844446Z 2022-11-18T20:55:03.5845184Z Iteration 82/82 2022-11-18T20:55:03.6003825Z 2022-11-18T20:55:03.6004488Z → management request 2022-11-18T20:55:03.6517858Z DELETE http://localhost:8081/models/invalid_squeezenet1_1 [errored] 2022-11-18T20:55:03.6518365Z read ECONNRESET 2022-11-18T20:55:03.6663106Z 28. Successful request 2022-11-18T20:55:03.9832076Z 2022-11-18T20:55:03.9832962Z ┌─────────────────────────┬─────────────────────┬────────────────────┐ 2022-11-18T20:55:03.9833476Z │ │ executed │ failed │ 2022-11-18T20:55:03.9833948Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:03.9834389Z │ iterations │ 82 │ 0 │ 2022-11-18T20:55:03.9835129Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:03.9835560Z │ requests │ 82 │ 14 │ 2022-11-18T20:55:03.9836030Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:03.9836471Z │ test-scripts │ 82 │ 0 │ 2022-11-18T20:55:03.9836910Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:03.9837350Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:55:03.9837816Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:03.9838227Z │ assertions │ 82 │ 14 │ 2022-11-18T20:55:03.9838659Z ├─────────────────────────┴─────────────────────┴────────────────────┤ 2022-11-18T20:55:03.9839084Z │ total run duration: 46.8s │ 2022-11-18T20:55:03.9839531Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:55:03.9839980Z │ total data received: 6.14kB (approx) │ 2022-11-18T20:55:03.9840419Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:55:03.9840876Z │ average response time: 592ms [min: 4ms, max: 8.3s, s.d.: 1814ms] │ 2022-11-18T20:55:03.9841345Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:55:03.9841794Z │ average DNS lookup time: 4ms [min: 203µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:55:03.9842245Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:55:03.9842683Z │ average first byte time: 588ms [min: 1ms, max: 8.3s, s.d.: 1814ms] │ 2022-11-18T20:55:03.9843141Z └────────────────────────────────────────────────────────────────────┘ 2022-11-18T20:55:03.9952231Z 2022-11-18T20:55:03.9952556Z # failure detail 2022-11-18T20:55:03.9953089Z 2022-11-18T20:55:03.9953349Z 01. Error 2022-11-18T20:55:03.9953626Z iteration: 6 read ECONNRESET 2022-11-18T20:55:03.9953996Z at request 2022-11-18T20:55:03.9954261Z inside "management request" 2022-11-18T20:55:03.9954811Z 2022-11-18T20:55:03.9955116Z 02. AssertionError Successful request 2022-11-18T20:55:03.9955439Z iteration: 6 expected undefined to be one of [ 400 ] 2022-11-18T20:55:03.9955973Z at assertion:0 in test-script 2022-11-18T20:55:03.9956266Z inside "management request" 2022-11-18T20:55:03.9956537Z 2022-11-18T20:55:03.9956778Z 03. Error 2022-11-18T20:55:03.9957036Z iteration: 11 read ECONNRESET 2022-11-18T20:55:03.9957310Z at request 2022-11-18T20:55:03.9957581Z inside "management request" 2022-11-18T20:55:03.9957828Z 2022-11-18T20:55:03.9958100Z 04. AssertionError Successful request 2022-11-18T20:55:03.9958412Z iteration: 11 expected undefined to be one of [ 500 ] 2022-11-18T20:55:03.9958770Z at assertion:0 in test-script 2022-11-18T20:55:03.9959055Z inside "management request" 2022-11-18T20:55:03.9959313Z 2022-11-18T20:55:03.9959552Z 05. Error 2022-11-18T20:55:03.9959795Z iteration: 17 read ECONNRESET 2022-11-18T20:55:03.9960064Z at request 2022-11-18T20:55:03.9960330Z inside "management request" 2022-11-18T20:55:03.9960573Z 2022-11-18T20:55:03.9960844Z 06. AssertionError Successful request 2022-11-18T20:55:03.9961149Z iteration: 17 expected undefined to be one of [ 400 ] 2022-11-18T20:55:03.9961527Z at assertion:0 in test-script 2022-11-18T20:55:03.9961795Z inside "management request" 2022-11-18T20:55:03.9962055Z 2022-11-18T20:55:03.9962291Z 07. Error 2022-11-18T20:55:03.9962532Z iteration: 23 read ECONNRESET 2022-11-18T20:55:03.9962796Z at request 2022-11-18T20:55:03.9963070Z inside "management request" 2022-11-18T20:55:03.9963328Z 2022-11-18T20:55:03.9963650Z 08. AssertionError Successful request 2022-11-18T20:55:03.9963959Z iteration: 23 expected undefined to be one of [ 404 ] 2022-11-18T20:55:03.9964337Z at assertion:0 in test-script 2022-11-18T20:55:03.9964609Z inside "management request" 2022-11-18T20:55:03.9964869Z 2022-11-18T20:55:03.9965111Z 09. Error 2022-11-18T20:55:03.9965357Z iteration: 26 read ECONNRESET 2022-11-18T20:55:03.9965626Z at request 2022-11-18T20:55:03.9965896Z inside "management request" 2022-11-18T20:55:03.9966225Z 2022-11-18T20:55:03.9966495Z 10. AssertionError Successful request 2022-11-18T20:55:03.9966802Z iteration: 26 expected undefined to be one of [ 404 ] 2022-11-18T20:55:03.9967175Z at assertion:0 in test-script 2022-11-18T20:55:03.9967460Z inside "management request" 2022-11-18T20:55:03.9967698Z 2022-11-18T20:55:03.9967939Z 11. Error 2022-11-18T20:55:03.9968213Z iteration: 27 read ECONNRESET 2022-11-18T20:55:03.9968461Z at request 2022-11-18T20:55:03.9968729Z inside "management request" 2022-11-18T20:55:03.9968990Z 2022-11-18T20:55:03.9969263Z 12. AssertionError Successful request 2022-11-18T20:55:03.9969549Z iteration: 27 expected undefined to be one of [ 404 ] 2022-11-18T20:55:03.9969916Z at assertion:0 in test-script 2022-11-18T20:55:03.9970202Z inside "management request" 2022-11-18T20:55:03.9970439Z 2022-11-18T20:55:03.9970676Z 13. Error 2022-11-18T20:55:03.9970938Z iteration: 58 read ECONNRESET 2022-11-18T20:55:03.9971183Z at request 2022-11-18T20:55:03.9974992Z inside "management request" 2022-11-18T20:55:03.9975473Z 2022-11-18T20:55:03.9975873Z 14. AssertionError Successful request 2022-11-18T20:55:03.9976324Z iteration: 58 expected undefined to be one of [ 400 ] 2022-11-18T20:55:03.9976912Z at assertion:0 in test-script 2022-11-18T20:55:03.9977310Z inside "management request" 2022-11-18T20:55:03.9977696Z 2022-11-18T20:55:03.9978057Z 15. Error 2022-11-18T20:55:03.9978440Z iteration: 60 read ECONNRESET 2022-11-18T20:55:03.9978812Z at request 2022-11-18T20:55:03.9979199Z inside "management request" 2022-11-18T20:55:03.9979574Z 2022-11-18T20:55:03.9979960Z 16. AssertionError Successful request 2022-11-18T20:55:03.9980396Z iteration: 60 expected undefined to be one of [ 404 ] 2022-11-18T20:55:03.9980894Z at assertion:0 in test-script 2022-11-18T20:55:03.9981308Z inside "management request" 2022-11-18T20:55:03.9981842Z 2022-11-18T20:55:03.9982197Z 17. Error 2022-11-18T20:55:03.9982586Z iteration: 66 read ECONNRESET 2022-11-18T20:55:03.9982974Z at request 2022-11-18T20:55:03.9983362Z inside "management request" 2022-11-18T20:55:03.9983726Z 2022-11-18T20:55:03.9984115Z 18. AssertionError Successful request 2022-11-18T20:55:03.9984545Z iteration: 66 expected undefined to be one of [ 400 ] 2022-11-18T20:55:03.9985047Z at assertion:0 in test-script 2022-11-18T20:55:03.9985510Z inside "management request" 2022-11-18T20:55:03.9985891Z 2022-11-18T20:55:03.9986256Z 19. Error 2022-11-18T20:55:03.9986667Z iteration: 69 read ECONNRESET 2022-11-18T20:55:03.9987041Z at request 2022-11-18T20:55:03.9987423Z inside "management request" 2022-11-18T20:55:03.9987792Z 2022-11-18T20:55:03.9988140Z 20. AssertionError Successful request 2022-11-18T20:55:03.9988550Z iteration: 69 expected undefined to be one of [ 500 ] 2022-11-18T20:55:03.9989039Z at assertion:0 in test-script 2022-11-18T20:55:03.9989427Z inside "management request" 2022-11-18T20:55:03.9989772Z 2022-11-18T20:55:03.9990112Z 21. Error 2022-11-18T20:55:03.9990480Z iteration: 70 read ECONNRESET 2022-11-18T20:55:03.9990846Z at request 2022-11-18T20:55:03.9991192Z inside "management request" 2022-11-18T20:55:03.9991550Z 2022-11-18T20:55:03.9991918Z 22. AssertionError Successful request 2022-11-18T20:55:03.9992311Z iteration: 70 expected undefined to be one of [ 400 ] 2022-11-18T20:55:03.9992786Z at assertion:0 in test-script 2022-11-18T20:55:03.9993176Z inside "management request" 2022-11-18T20:55:03.9993534Z 2022-11-18T20:55:03.9993856Z 23. Error 2022-11-18T20:55:03.9994224Z iteration: 71 read ECONNRESET 2022-11-18T20:55:03.9994618Z at request 2022-11-18T20:55:03.9994983Z inside "management request" 2022-11-18T20:55:03.9995323Z 2022-11-18T20:55:03.9995690Z 24. AssertionError Successful request 2022-11-18T20:55:03.9996094Z iteration: 71 expected undefined to be one of [ 404 ] 2022-11-18T20:55:03.9996562Z at assertion:0 in test-script 2022-11-18T20:55:03.9996937Z inside "management request" 2022-11-18T20:55:03.9997294Z 2022-11-18T20:55:03.9997630Z 25. Error 2022-11-18T20:55:03.9997996Z iteration: 73 read ECONNRESET 2022-11-18T20:55:03.9998345Z at request 2022-11-18T20:55:03.9998802Z inside "management request" 2022-11-18T20:55:03.9999176Z 2022-11-18T20:55:03.9999546Z 26. AssertionError Successful request 2022-11-18T20:55:03.9999954Z iteration: 73 expected undefined to be one of [ 404 ] 2022-11-18T20:55:04.0000438Z at assertion:0 in test-script 2022-11-18T20:55:04.0000931Z inside "management request" 2022-11-18T20:55:04.0001296Z 2022-11-18T20:55:04.0001637Z 27. Error 2022-11-18T20:55:04.0001984Z iteration: 82 read ECONNRESET 2022-11-18T20:55:04.0002422Z at request 2022-11-18T20:55:04.0002793Z inside "management request" 2022-11-18T20:55:04.0003158Z 2022-11-18T20:55:04.0003514Z 28. AssertionError Successful request 2022-11-18T20:55:04.0003927Z iteration: 82 expected undefined to be one of [ 404 ] 2022-11-18T20:55:04.0004408Z at assertion:0 in test-script 2022-11-18T20:55:04.0004797Z inside "management request" 2022-11-18T20:55:04.1099813Z TorchServe has stopped. 2022-11-18T20:55:14.1255195Z Current date and time : 2022-11-18 20:42:09 2022-11-18T20:55:14.1256019Z ------------------------------------------------------------------------------------------ 2022-11-18T20:55:14.1256604Z Environment headers 2022-11-18T20:55:14.1257278Z ------------------------------------------------------------------------------------------ 2022-11-18T20:55:14.1257811Z Torchserve branch: upgrade_pytorch_113 2022-11-18T20:55:14.1258136Z 2022-11-18T20:55:14.1258344Z torchserve==0.6.1 2022-11-18T20:55:14.1258822Z torch-model-archiver==0.6.1 2022-11-18T20:55:14.1259128Z 2022-11-18T20:55:14.1259451Z Python version: 3.8 (64-bit runtime) 2022-11-18T20:55:14.1260004Z Python executable: C:\hostedtoolcache\windows\Python\3.8.10\x64\python.exe 2022-11-18T20:55:14.1260379Z 2022-11-18T20:55:14.1260641Z Versions of relevant python libraries: 2022-11-18T20:55:14.1261022Z numpy==1.19.3 2022-11-18T20:55:14.1261286Z 2022-11-18T20:55:14.1261492Z torch==1.13.0 2022-11-18T20:55:14.1261749Z 2022-11-18T20:55:14.1261967Z torchaudio==0.13.0 2022-11-18T20:55:14.1262241Z 2022-11-18T20:55:14.1262455Z torchtext==0.14.0 2022-11-18T20:55:14.1262996Z 2022-11-18T20:55:14.1263226Z torchvision==0.14.0 2022-11-18T20:55:14.1263588Z torch==1.13.0 2022-11-18T20:55:14.1263855Z 2022-11-18T20:55:14.1264070Z torchtext==0.14.0 2022-11-18T20:55:14.1264339Z 2022-11-18T20:55:14.1264559Z torchvision==0.14.0 2022-11-18T20:55:14.1265037Z torchaudio==0.13.0 2022-11-18T20:55:14.1265410Z 2022-11-18T20:55:14.1265549Z 2022-11-18T20:55:14.1265775Z Java Version: 2022-11-18T20:55:14.1266064Z 2022-11-18T20:55:14.1266178Z 2022-11-18T20:55:14.1266459Z OS: Microsoft Windows Server 2022 Datacenter 2022-11-18T20:55:14.1266832Z GCC version: N/A 2022-11-18T20:55:14.1267197Z Clang version: 14.0.6 2022-11-18T20:55:14.1267579Z CMake version: version 3.24.2 2022-11-18T20:55:14.1268253Z ## In directory D:\a\serve\serve | Executing command pip install --force-reinstall . 2022-11-18T20:55:14.1268999Z ## In directory D:\a\serve\serve | Executing command pip install --force-reinstall model-archiver/. 2022-11-18T20:55:14.1269795Z ## In directory D:\a\serve\serve | Executing command pip install --force-reinstall workflow-archiver/. 2022-11-18T20:55:14.1270559Z ## Starting generate_mars, mar_config:D:\a\serve\serve\ts_scripts\..\ts_scripts\mar_config.json, model_store_dir:D:\a\serve\serve\ts_scripts\..\model_store_gen 2022-11-18T20:55:14.1270947Z 2022-11-18T20:55:14.1272649Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name fastrcnn --version 1.0 --model-file examples/object_detector/fast-rcnn/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\fasterrcnn_resnet50_fpn_coco-258fb6c6.pth --handler object_detector --extra-files examples/object_detector/index_to_name.json --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1273681Z 2022-11-18T20:55:14.1273935Z ## fastrcnn.mar is generated. 2022-11-18T20:55:14.1274233Z 2022-11-18T20:55:14.1275476Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name alexnet --version 1.0 --model-file examples/image_classifier/alexnet/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\alexnet-owt-7be5be79.pth --handler image_classifier --extra-files examples/image_classifier/index_to_name.json --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1276515Z 2022-11-18T20:55:14.1276759Z ## alexnet.mar is generated. 2022-11-18T20:55:14.1277058Z 2022-11-18T20:55:14.1278356Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name densenet161 --version 1.0 --model-file examples/image_classifier/densenet_161/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\densenet161-8d451a50.pth --handler image_classifier --extra-files examples/image_classifier/index_to_name.json --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1279270Z 2022-11-18T20:55:14.1279521Z ## densenet161.mar is generated. 2022-11-18T20:55:14.1279818Z 2022-11-18T20:55:14.1287754Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name mnist --version 1.0 --model-file examples/image_classifier/mnist/mnist.py --serialized-file examples/image_classifier/mnist/mnist_cnn.pt --handler examples/image_classifier/mnist/mnist_handler.py --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1288538Z 2022-11-18T20:55:14.1288673Z ## mnist.mar is generated. 2022-11-18T20:55:14.1288846Z 2022-11-18T20:55:14.1289986Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name resnet-152-batch --version 1.0 --model-file examples/image_classifier/resnet_152_batch/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\resnet152-394f9c45.pth --handler image_classifier --extra-files examples/image_classifier/index_to_name.json --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1290793Z 2022-11-18T20:55:14.1290987Z ## resnet-152-batch.mar is generated. 2022-11-18T20:55:14.1291177Z 2022-11-18T20:55:14.1292266Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name resnet-18 --version 1.0 --model-file examples/image_classifier/resnet_18/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\resnet18-f37072fd.pth --handler image_classifier --extra-files examples/image_classifier/index_to_name.json --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1293044Z 2022-11-18T20:55:14.1293231Z ## resnet-18.mar is generated. 2022-11-18T20:55:14.1293405Z 2022-11-18T20:55:14.1294504Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name squeezenet1_1 --version 1.0 --model-file examples/image_classifier/squeezenet/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\squeezenet1_1-b8a52dc0.pth --handler image_classifier --extra-files examples/image_classifier/index_to_name.json --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1295294Z 2022-11-18T20:55:14.1295422Z ## squeezenet1_1.mar is generated. 2022-11-18T20:55:14.1295602Z 2022-11-18T20:55:14.1296897Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name vgg16 --version 1.0 --model-file examples/image_classifier/vgg_16/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\vgg16-397923af.pth --handler examples/image_classifier/vgg_16/vgg_handler.py --extra-files examples/image_classifier/index_to_name.json --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1297683Z 2022-11-18T20:55:14.1297800Z ## vgg16.mar is generated. 2022-11-18T20:55:14.1297963Z 2022-11-18T20:55:14.1299419Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name deeplabv3_resnet_101_eager --version 1.0 --model-file examples/image_segmenter/deeplabv3/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\deeplabv3_resnet101_coco-586e9e4e.pth --handler image_segmenter --extra-files examples/image_segmenter/deeplabv3/deeplabv3.py,examples/image_segmenter/deeplabv3/intermediate_layer_getter.py,examples/image_segmenter/deeplabv3/fcn.py --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1300509Z 2022-11-18T20:55:14.1300675Z ## deeplabv3_resnet_101_eager.mar is generated. 2022-11-18T20:55:14.1300873Z 2022-11-18T20:55:14.1302140Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name fcn_resnet_101 --version 1.0 --model-file examples/image_segmenter/fcn/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\fcn_resnet101_coco-7ecb50ca.pth --handler image_segmenter --extra-files examples/image_segmenter/fcn/fcn.py,examples/image_segmenter/fcn/intermediate_layer_getter.py --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1302987Z 2022-11-18T20:55:14.1303117Z ## fcn_resnet_101.mar is generated. 2022-11-18T20:55:14.1303272Z 2022-11-18T20:55:14.1304393Z ## In directory: D:\a\serve\serve | Executing command: torch-model-archiver --model-name maskrcnn --version 1.0 --model-file examples/object_detector/maskrcnn/model.py --serialized-file D:\a\serve\serve\ts_scripts\..\model_store_gen\maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth --handler object_detector --extra-files examples/object_detector/index_to_name.json --export-path D:\a\serve\serve\ts_scripts\..\model_store_gen --force 2022-11-18T20:55:14.1305200Z 2022-11-18T20:55:14.1305306Z ## maskrcnn.mar is generated. 2022-11-18T20:55:14.1305479Z 2022-11-18T20:55:14.1305603Z ## Starting gen_mar: model_store 2022-11-18T20:55:14.1305777Z 2022-11-18T20:55:14.1305898Z ## Create symlink for mar files 2022-11-18T20:55:14.1306067Z 2022-11-18T20:55:14.1306336Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:55:14.1306879Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:55:14.1307576Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:55:14.1308140Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:55:14.1308698Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:55:14.1309255Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:55:14.1309814Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:55:14.1310568Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:55:14.1311172Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:55:14.1311769Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:55:14.1312384Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:55:14.1312850Z ## Starting TorchServe 2022-11-18T20:55:14.1313145Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:55:14.1313782Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs >> ts_console.log 2022-11-18T20:55:14.1314240Z ## Successfully started TorchServe 2022-11-18T20:55:14.1314520Z ## Stopping TorchServe 2022-11-18T20:55:14.1314981Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:55:25.6036423Z newman 2022-11-18T20:55:25.6036879Z 2022-11-18T20:55:25.6038786Z kf_api_test_collection 2022-11-18T20:55:25.6068099Z 2022-11-18T20:55:25.6068769Z → Model Zoo - Register Model 2022-11-18T20:55:28.1813616Z POST http://localhost:8081/models?url=mnist.mar&model_name=mnist&initial_workers=1&synchronous=true 2022-11-18T20:55:28.1815157Z 200 OK ★ 2.4s time ★ 318B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:28.1819269Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:55:28.1820876Z │ { 2022-11-18T20:55:28.1821373Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:55:28.1821952Z │ ith 1 initial workers" 2022-11-18T20:55:28.1822401Z │ } 2022-11-18T20:55:28.1898660Z └ 2022-11-18T20:55:28.1899319Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:28.7826729Z 30ms 11ms 9ms 545µs 2.4s 8ms 588µs 2.5s 2022-11-18T20:55:28.7826985Z 2022-11-18T20:55:28.7827511Z √ Successful POST request 2022-11-18T20:55:28.7830399Z 2022-11-18T20:55:28.7830766Z → Model Zoo - Inference Model 2022-11-18T20:55:28.7831195Z POST http://localhost:8080/v1/models/mnist:predict 2022-11-18T20:55:28.7831816Z 200 OK ★ 46ms time ★ 688B↑ 266B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:55:28.7832300Z ┌ ↑ file ★ 417B 2022-11-18T20:55:28.7832640Z │ 2022-11-18T20:55:28.7832929Z └ 2022-11-18T20:55:28.7833374Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 32B 2022-11-18T20:55:28.7833747Z │ { 2022-11-18T20:55:28.7834102Z │ "predictions": [ 2022-11-18T20:55:28.7834426Z │ 2 2022-11-18T20:55:28.7834744Z │ ] 2022-11-18T20:55:28.7835053Z │ } 2022-11-18T20:55:28.7835316Z └ 2022-11-18T20:55:28.7835856Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:28.7836520Z 7ms 1ms 339µs 1ms 40ms 2ms 72µs 53ms 2022-11-18T20:55:28.7836786Z 2022-11-18T20:55:28.7837061Z √ Successful POST request 2022-11-18T20:55:28.7837395Z √ Test expected JSON response 2022-11-18T20:55:28.7837570Z 2022-11-18T20:55:28.7837754Z → Model Zoo - Explanations Model 2022-11-18T20:55:28.7838213Z POST http://localhost:8080/v1/models/mnist:explain 2022-11-18T20:55:28.7838777Z 200 OK ★ 75ms time ★ 688B↑ 25.39kB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:55:28.7839169Z ┌ ↑ file ★ 417B 2022-11-18T20:55:28.7839514Z │ 2022-11-18T20:55:28.7839821Z └ 2022-11-18T20:55:28.7840262Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 25.15kB 2022-11-18T20:55:28.7840611Z │ { 2022-11-18T20:55:28.7840963Z │ "explanations": [ 2022-11-18T20:55:28.7841304Z │ [ 2022-11-18T20:55:28.7841623Z │ [ 2022-11-18T20:55:28.7841930Z │ [ 2022-11-18T20:55:28.7842296Z │ 0.004570948717123555, 2022-11-18T20:55:28.7842681Z │ 0.00621696950359869, 2022-11-18T20:55:28.7843038Z │ 0.00819756530257639, 2022-11-18T20:55:28.7843417Z │ 0.009563574554839933, 2022-11-18T20:55:28.7843794Z │ 0.008999274400288457, 2022-11-18T20:55:28.7844171Z │ 0.00967347489884473, 2022-11-18T20:55:28.7844543Z │ 0.00759990546889743, 2022-11-18T20:55:28.7844914Z │ 0.006361381155311607, 2022-11-18T20:55:28.7845290Z │ 0.005768828987233948, 2022-11-18T20:55:28.7845659Z │ 0.004394465972850835, 2022-11-18T20:55:28.7846192Z │ 0.004948217797673561, 2022-11-18T20:55:28.7846583Z │ 0.005273460533175704, 2022-11-18T20:55:28.7846958Z │ 0.005523799504482613, 2022-11-18T20:55:28.7847335Z │ 0.007789355996710426, 2022-11-18T20:55:28.7847683Z │ 0.008759363475167876, 2022-11-18T20:55:28.7848128Z │ 0.004304805525164148, 2022-11-18T20:55:28.7848457Z │ 0.010970578941976502, 2022-11-18T20:55:28.7848762Z │ 0.003248439044307383, 2022-11-18T20:55:28.7849089Z │ 0.005998033264566424, 2022-11-18T20:55:28.7849418Z │ 0.0037543659660004176, 2022-11-18T20:55:28.7849759Z │ 0.002765290966317314, 2022-11-18T20:55:28.7850063Z │ 0.004314086388195293, 2022-11-18T20:55:28.7850495Z │ 0.0014008575673494432, 2022-11-18T20:55:28.7850834Z │ 0.0048418467486106885, 2022-11-18T20:55:28.7851167Z │ 0.0006374841539803208, 2022-11-18T20:55:28.7851486Z │ 0.0018558540185062507, 2022-11-18T20:55:28.7851827Z │ -0.0008280458303867654, 2022-11-18T20:55:28.7852143Z │ -0.0 2022-11-18T20:55:28.7852441Z │ ], 2022-11-18T20:55:28.7852712Z │ [ 2022-11-18T20:55:28.7853032Z │ 0.0016625159530346961, 2022-11-18T20:55:28.7853361Z │ 0.004443791836049446, 2022-11-18T20:55:28.7853671Z │ 0.012387840809404818, 2022-11-18T20:55:28.7853997Z │ 0.009450844566697625, 2022-11-18T20:55:28.7854324Z │ 0.01614382020767612, 2022-11-18T20:55:28.7854656Z │ 0.007797501724012741, 2022-11-18T20:55:28.7854961Z │ 0.013942240452575565, 2022-11-18T20:55:28.7855280Z │ 0.007557430539407937, 2022-11-18T20:55:28.7855618Z │ 0.005479090520470613, 2022-11-18T20:55:28.7855940Z │ 0.009751320252680757, 2022-11-18T20:55:28.7856252Z │ 0.0047644071994580985, 2022-11-18T20:55:28.7856584Z │ 0.007292147627537893, 2022-11-18T20:55:28.7856917Z │ 0.011797999771088712, 2022-11-18T20:55:28.7857248Z │ 0.006462685185030748, 2022-11-18T20:55:28.7857551Z │ 0.003383213242765121, 2022-11-18T20:55:28.7857873Z │ 0.009225058381773676, 2022-11-18T20:55:28.7858200Z │ 0.0016750692665810285, 2022-11-18T20:55:28.7858537Z │ 0.007362305503563809, 2022-11-18T20:55:28.7858839Z │ 0.005270058160700518, 2022-11-18T20:55:28.7859166Z │ 0.0054535979835177665, 2022-11-18T20:55:28.7859497Z │ 0.0043427260383291045, 2022-11-18T20:55:28.7859805Z │ 0.005856133854976941, 2022-11-18T20:55:28.7860127Z │ 0.012168384664165323, 2022-11-18T20:55:28.7860455Z │ 0.009902719801994992, 2022-11-18T20:55:28.7860788Z │ 0.009813113130096628, 2022-11-18T20:55:28.7861097Z │ 0.0034427582223001893, 2022-11-18T20:55:28.7861427Z │ 0.002292428498848947, 2022-11-18T20:55:28.7861737Z │ -0.0 2022-11-18T20:55:28.7862031Z │ ], 2022-11-18T20:55:28.7862300Z │ [ 2022-11-18T20:55:28.7862621Z │ 0.011528255210933541, 2022-11-18T20:55:28.7862952Z │ 0.009146989356439473, 2022-11-18T20:55:28.7863712Z │ 0.022265417890940834, 2022-11-18T20:55:28.7863959Z │ 0.01755821671930447, 2022-11-18T20:55:28.7864220Z │ 0.024770554555923724, 2022-11-18T20:55:28.7864483Z │ 0.02141269337793062, 2022-11-18T20:55:28.7864692Z │ 2022-11-18T20:55:28.7864954Z │ (showing 2.05kB/25.15kB) 2022-11-18T20:55:28.7865173Z └ 2022-11-18T20:55:28.7865567Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:28.7866069Z 3ms 551µs (cache) (cache) 72ms 2ms 117µs 77ms 2022-11-18T20:55:28.7866254Z 2022-11-18T20:55:28.7866406Z √ Successful POST request 2022-11-18T20:55:28.7866863Z √ Test expected JSON response 2022-11-18T20:55:28.7867000Z 2022-11-18T20:55:28.7867169Z → Model Zoo - Unregister model 2022-11-18T20:55:28.7867467Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:55:28.7867882Z 200 OK ★ 99ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:28.7868358Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:28.7868700Z │ { 2022-11-18T20:55:28.7869068Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:55:28.7869397Z │ } 2022-11-18T20:55:28.7869634Z └ 2022-11-18T20:55:28.7870108Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:28.7870655Z 1ms 302µs (cache) (cache) 88ms 8ms 96µs 100ms 2022-11-18T20:55:28.7870997Z 2022-11-18T20:55:28.8065662Z √ Successful DELETE request 2022-11-18T20:55:29.7974573Z 2022-11-18T20:55:29.7975429Z → Model Zoo - Model Metrics 2022-11-18T20:55:29.7979350Z GET http://localhost:8082/metrics 2022-11-18T20:55:29.7980268Z 200 OK ★ 18ms time ★ 233B↑ 1.04kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:29.7980805Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 751B 2022-11-18T20:55:29.7997971Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:55:29.7998439Z │ erence requests. 2022-11-18T20:55:29.7998880Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:55:29.7999747Z │ ts_inference_requests_total{uuid="2339b257-68b8-4232-9 2022-11-18T20:55:29.8000336Z │ 44a-8d85c8972a2e",model_name="mnist",model_version="de 2022-11-18T20:55:29.8000740Z │ fault",} 2.0 2022-11-18T20:55:29.8001181Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:55:29.8001657Z │ ference duration in microseconds 2022-11-18T20:55:29.8002173Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:55:29.8002690Z │ ts_inference_latency_microseconds{uuid="2339b257-68b8- 2022-11-18T20:55:29.8003188Z │ 4232-944a-8d85c8972a2e",model_name="mnist",model_versi 2022-11-18T20:55:29.8003629Z │ on="default",} 90369.6 2022-11-18T20:55:29.8004085Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:55:29.8004512Z │ duration in microseconds 2022-11-18T20:55:29.8004960Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:55:29.8005455Z │ ts_queue_latency_microseconds{uuid="2339b257-68b8-4232 2022-11-18T20:55:29.8005947Z │ -944a-8d85c8972a2e",model_name="mnist",model_version=" 2022-11-18T20:55:29.8006328Z │ default",} 575.8 2022-11-18T20:55:29.8006637Z └ 2022-11-18T20:55:29.8007171Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:55:29.8007784Z 2ms 1ms 287µs 395µs 13ms 1ms 72µs 19ms 2022-11-18T20:55:29.8008059Z 2022-11-18T20:55:29.8008269Z √ Successful GET request 2022-11-18T20:55:29.8008497Z 2022-11-18T20:55:29.8008856Z ┌─────────────────────────┬─────────────────────┬────────────────────┐ 2022-11-18T20:55:29.8009344Z │ │ executed │ failed │ 2022-11-18T20:55:29.8009865Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:29.8010351Z │ iterations │ 1 │ 0 │ 2022-11-18T20:55:29.8010844Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:29.8011314Z │ requests │ 5 │ 0 │ 2022-11-18T20:55:29.8011820Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:29.8012299Z │ test-scripts │ 5 │ 0 │ 2022-11-18T20:55:29.8012794Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:29.8013542Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:55:29.8014068Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:55:29.8014652Z │ assertions │ 7 │ 0 │ 2022-11-18T20:55:29.8015149Z ├─────────────────────────┴─────────────────────┴────────────────────┤ 2022-11-18T20:55:29.8015623Z │ total run duration: 3.3s │ 2022-11-18T20:55:29.8016133Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:55:29.8016622Z │ total data received: 26.06kB (approx) │ 2022-11-18T20:55:29.8017123Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:55:29.8017628Z │ average response time: 544ms [min: 18ms, max: 2.4s, s.d.: 969ms] │ 2022-11-18T20:55:29.8018159Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:55:29.8018788Z │ average DNS lookup time: 3ms [min: 287µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:55:29.8019309Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:55:29.8019825Z │ average first byte time: 533ms [min: 13ms, max: 2.4s, s.d.: 959ms] │ 2022-11-18T20:55:29.8020360Z └────────────────────────────────────────────────────────────────────┘ 2022-11-18T20:55:29.9172140Z TorchServe has stopped. 2022-11-18T20:55:51.4273291Z newman 2022-11-18T20:55:51.4274234Z 2022-11-18T20:55:51.4276028Z kf_https_test_collection 2022-11-18T20:55:51.4303481Z 2022-11-18T20:55:51.4304412Z → HTTPS Inference API Description 2022-11-18T20:55:51.7402005Z OPTIONS https://localhost:8443 2022-11-18T20:55:51.7404154Z 200 OK ★ 216ms time ★ 230B↑ 23.68kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:51.7408816Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 23.41kB 2022-11-18T20:55:51.7411096Z │ { 2022-11-18T20:55:51.7411482Z │ "openapi": "3.0.1", 2022-11-18T20:55:51.7411901Z │ "info": { 2022-11-18T20:55:51.7412302Z │ "title": "TorchServe APIs", 2022-11-18T20:55:51.7412774Z │ "description": "TorchServe is a flexible and easy 2022-11-18T20:55:51.7413441Z │ to use tool for serving deep learning models", 2022-11-18T20:55:51.7413887Z │ "version": "0.6.1\r" 2022-11-18T20:55:51.7414216Z │ }, 2022-11-18T20:55:51.7414552Z │ "paths": { 2022-11-18T20:55:51.7414893Z │ "/": { 2022-11-18T20:55:51.7415219Z │ "options": { 2022-11-18T20:55:51.7415651Z │ "description": "Get openapi description.", 2022-11-18T20:55:51.7416123Z │ "operationId": "apiDescription", 2022-11-18T20:55:51.7416537Z │ "parameters": [], 2022-11-18T20:55:51.7416897Z │ "responses": { 2022-11-18T20:55:51.7417253Z │ "200": { 2022-11-18T20:55:51.7417681Z │ "description": "A openapi 3.0.1 descriptor 2022-11-18T20:55:51.7418040Z │ ", 2022-11-18T20:55:51.7418502Z │ "content": { 2022-11-18T20:55:51.7418897Z │ "application/json": { 2022-11-18T20:55:51.7419291Z │ "schema": { 2022-11-18T20:55:51.7419656Z │ "type": "object", 2022-11-18T20:55:51.7420054Z │ "required": [ 2022-11-18T20:55:51.7420441Z │ "openapi", 2022-11-18T20:55:51.7420813Z │ "info", 2022-11-18T20:55:51.7421161Z │ "paths" 2022-11-18T20:55:51.7421515Z │ ], 2022-11-18T20:55:51.7421894Z │ "properties": { 2022-11-18T20:55:51.7422270Z │ "openapi": { 2022-11-18T20:55:51.7422668Z │ "type": "string" 2022-11-18T20:55:51.7423036Z │ }, 2022-11-18T20:55:51.7423398Z │ "info": { 2022-11-18T20:55:51.7423774Z │ "type": "object" 2022-11-18T20:55:51.7424184Z │ }, 2022-11-18T20:55:51.7424541Z │ "paths": { 2022-11-18T20:55:51.7424924Z │ "type": "object" 2022-11-18T20:55:51.7425277Z │ } 2022-11-18T20:55:51.7425615Z │ } 2022-11-18T20:55:51.7425935Z │ } 2022-11-18T20:55:51.7426265Z │ } 2022-11-18T20:55:51.7426590Z │ } 2022-11-18T20:55:51.7426892Z │ }, 2022-11-18T20:55:51.7427252Z │ "500": { 2022-11-18T20:55:51.7427671Z │ "description": "Internal Server Error", 2022-11-18T20:55:51.7428071Z │ "content": { 2022-11-18T20:55:51.7428437Z │ "application/json": { 2022-11-18T20:55:51.7428821Z │ "schema": { 2022-11-18T20:55:51.7429195Z │ "type": "object", 2022-11-18T20:55:51.7429550Z │ "required": [ 2022-11-18T20:55:51.7429914Z │ "code", 2022-11-18T20:55:51.7430271Z │ "type", 2022-11-18T20:55:51.7430638Z │ "message" 2022-11-18T20:55:51.7430966Z │ ], 2022-11-18T20:55:51.7431337Z │ "properties": { 2022-11-18T20:55:51.7431726Z │ "code": { 2022-11-18T20:55:51.7432109Z │ "type": "integer", 2022-11-18T20:55:51.7432514Z │ "description": "Error code." 2022-11-18T20:55:51.7432890Z │ }, 2022-11-18T20:55:51.7433246Z │ "type": { 2022-11-18T20:55:51.7433608Z │ "type": "string", 2022-11-18T20:55:51.7434025Z │ "description": "Error type." 2022-11-18T20:55:51.7434400Z │ }, 2022-11-18T20:55:51.7434762Z │ "message": { 2022-11-18T20:55:51.7435129Z │ "type": "string", 2022-11-18T20:55:51.7435571Z │ "description": "Error message." 2022-11-18T20:55:51.7435956Z │ } 2022-11-18T20:55:51.7436301Z │ } 2022-11-18T20:55:51.7436616Z │ } 2022-11-18T20:55:51.7436945Z │ } 2022-11-18T20:55:51.7437269Z │ } 2022-11-18T20:55:51.7437680Z │ } 2022-11-18T20:55:51.7438013Z │ } 2022-11-18T20:55:51.7438328Z │ } 2022-11-18T20:55:51.7438618Z │ }, 2022-11-18T20:55:51.7438944Z │ "/ping": { 2022-11-18T20:55:51.7439263Z │ 2022-11-18T20:55:51.7439614Z │ (showing 2.05kB/23.41kB) 2022-11-18T20:55:51.7489259Z └ 2022-11-18T20:55:51.7489866Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:55:51.7490537Z 33ms 9ms 8ms 1ms 148ms 38ms 8ms 494µs 249ms 2022-11-18T20:55:51.7490839Z 2022-11-18T20:55:51.7948780Z √ Status code is 200 2022-11-18T20:55:51.8187869Z 2022-11-18T20:55:51.8188811Z → HTTPS Management API Description 2022-11-18T20:55:51.9572878Z OPTIONS https://localhost:8444 2022-11-18T20:55:51.9574328Z 200 OK ★ 56ms time ★ 230B↑ 58.87kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:51.9583245Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 58.61kB 2022-11-18T20:55:51.9583854Z │ { 2022-11-18T20:55:51.9584203Z │ "openapi": "3.0.1", 2022-11-18T20:55:51.9584565Z │ "info": { 2022-11-18T20:55:51.9585001Z │ "title": "TorchServe APIs", 2022-11-18T20:55:51.9585459Z │ "description": "TorchServe is a flexible and easy 2022-11-18T20:55:51.9585935Z │ to use tool for serving deep learning models", 2022-11-18T20:55:51.9586952Z │ "version": "0.6.1\r" 2022-11-18T20:55:51.9587305Z │ }, 2022-11-18T20:55:51.9587623Z │ "paths": { 2022-11-18T20:55:51.9587962Z │ "/": { 2022-11-18T20:55:51.9588311Z │ "options": { 2022-11-18T20:55:51.9588733Z │ "description": "Get openapi description.", 2022-11-18T20:55:51.9589204Z │ "operationId": "apiDescription", 2022-11-18T20:55:51.9589643Z │ "parameters": [], 2022-11-18T20:55:51.9590022Z │ "responses": { 2022-11-18T20:55:51.9590362Z │ "200": { 2022-11-18T20:55:51.9590805Z │ "description": "A openapi 3.0.1 descriptor 2022-11-18T20:55:51.9591187Z │ ", 2022-11-18T20:55:51.9591534Z │ "content": { 2022-11-18T20:55:51.9591915Z │ "application/json": { 2022-11-18T20:55:51.9592304Z │ "schema": { 2022-11-18T20:55:51.9592688Z │ "type": "object", 2022-11-18T20:55:51.9593055Z │ "required": [ 2022-11-18T20:55:51.9593438Z │ "openapi", 2022-11-18T20:55:51.9593805Z │ "info", 2022-11-18T20:55:51.9594167Z │ "paths" 2022-11-18T20:55:51.9594504Z │ ], 2022-11-18T20:55:51.9594885Z │ "properties": { 2022-11-18T20:55:51.9595276Z │ "openapi": { 2022-11-18T20:55:51.9595706Z │ "type": "string" 2022-11-18T20:55:51.9596101Z │ }, 2022-11-18T20:55:51.9596476Z │ "info": { 2022-11-18T20:55:51.9596860Z │ "type": "object" 2022-11-18T20:55:51.9597231Z │ }, 2022-11-18T20:55:51.9597574Z │ "paths": { 2022-11-18T20:55:51.9597957Z │ "type": "object" 2022-11-18T20:55:51.9598312Z │ } 2022-11-18T20:55:51.9598629Z │ } 2022-11-18T20:55:51.9598963Z │ } 2022-11-18T20:55:51.9599292Z │ } 2022-11-18T20:55:51.9599626Z │ } 2022-11-18T20:55:51.9599932Z │ }, 2022-11-18T20:55:51.9600266Z │ "500": { 2022-11-18T20:55:51.9600683Z │ "description": "Internal Server Error", 2022-11-18T20:55:51.9601069Z │ "content": { 2022-11-18T20:55:51.9601455Z │ "application/json": { 2022-11-18T20:55:51.9601850Z │ "schema": { 2022-11-18T20:55:51.9602230Z │ "type": "object", 2022-11-18T20:55:51.9602589Z │ "required": [ 2022-11-18T20:55:51.9603205Z │ "code", 2022-11-18T20:55:51.9603584Z │ "type", 2022-11-18T20:55:51.9603948Z │ "message" 2022-11-18T20:55:51.9604280Z │ ], 2022-11-18T20:55:51.9604679Z │ "properties": { 2022-11-18T20:55:51.9605056Z │ "code": { 2022-11-18T20:55:51.9605424Z │ "type": "integer", 2022-11-18T20:55:51.9605852Z │ "description": "Error code." 2022-11-18T20:55:51.9606234Z │ }, 2022-11-18T20:55:51.9606590Z │ "type": { 2022-11-18T20:55:51.9606953Z │ "type": "string", 2022-11-18T20:55:51.9607372Z │ "description": "Error type." 2022-11-18T20:55:51.9607885Z │ }, 2022-11-18T20:55:51.9608268Z │ "message": { 2022-11-18T20:55:51.9608657Z │ "type": "string", 2022-11-18T20:55:51.9609107Z │ "description": "Error message." 2022-11-18T20:55:51.9609467Z │ } 2022-11-18T20:55:51.9609803Z │ } 2022-11-18T20:55:51.9610139Z │ } 2022-11-18T20:55:51.9610466Z │ } 2022-11-18T20:55:51.9610774Z │ } 2022-11-18T20:55:51.9611099Z │ } 2022-11-18T20:55:51.9611421Z │ } 2022-11-18T20:55:51.9611721Z │ } 2022-11-18T20:55:51.9612031Z │ }, 2022-11-18T20:55:51.9612361Z │ "/models": { 2022-11-18T20:55:51.9612662Z │ 2022-11-18T20:55:51.9613009Z │ (showing 2.05kB/58.61kB) 2022-11-18T20:55:51.9613331Z └ 2022-11-18T20:55:51.9613882Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:55:51.9614494Z 1ms 1ms 268µs 606µs 30ms 19ms 2ms 104µs 57ms 2022-11-18T20:55:51.9614770Z 2022-11-18T20:55:51.9615003Z √ Status code is 200 2022-11-18T20:55:51.9747775Z 2022-11-18T20:55:51.9748464Z → HTTPS Metrics API Description 2022-11-18T20:55:52.0659416Z OPTIONS https://localhost:8445 2022-11-18T20:55:52.0660483Z 200 OK ★ 39ms time ★ 230B↑ 2.82kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:52.0661060Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 2.55kB 2022-11-18T20:55:52.0661440Z │ { 2022-11-18T20:55:52.0661794Z │ "openapi": "3.0.1", 2022-11-18T20:55:52.0662141Z │ "info": { 2022-11-18T20:55:52.0662524Z │ "title": "TorchServe APIs", 2022-11-18T20:55:52.0662968Z │ "description": "TorchServe is a flexible and easy 2022-11-18T20:55:52.0663443Z │ to use tool for serving deep learning models", 2022-11-18T20:55:52.0663860Z │ "version": "0.6.1\r" 2022-11-18T20:55:52.0664216Z │ }, 2022-11-18T20:55:52.0664550Z │ "paths": { 2022-11-18T20:55:52.0664895Z │ "/metrics": { 2022-11-18T20:55:52.0665232Z │ "get": { 2022-11-18T20:55:52.0665644Z │ "description": "Get TorchServe application met 2022-11-18T20:55:52.0666097Z │ rics in prometheus format.", 2022-11-18T20:55:52.0666505Z │ "operationId": "metrics", 2022-11-18T20:55:52.0666884Z │ "parameters": [ 2022-11-18T20:55:52.0667206Z │ { 2022-11-18T20:55:52.0667578Z │ "in": "query", 2022-11-18T20:55:52.0667944Z │ "name": "name[]", 2022-11-18T20:55:52.0668352Z │ "description": "Names of metrics to filter 2022-11-18T20:55:52.0668718Z │ ", 2022-11-18T20:55:52.0669064Z │ "required": false, 2022-11-18T20:55:52.0669425Z │ "schema": { 2022-11-18T20:55:52.0669773Z │ "type": "string" 2022-11-18T20:55:52.0670112Z │ } 2022-11-18T20:55:52.0670432Z │ } 2022-11-18T20:55:52.0670744Z │ ], 2022-11-18T20:55:52.0671086Z │ "responses": { 2022-11-18T20:55:52.0671432Z │ "200": { 2022-11-18T20:55:52.0671857Z │ "description": "TorchServe application met 2022-11-18T20:55:52.0672483Z │ rics", 2022-11-18T20:55:52.0672859Z │ "content": { 2022-11-18T20:55:52.0673296Z │ "text/plain; version=0.0.4; charset=utf- 2022-11-18T20:55:52.0673661Z │ 8": { 2022-11-18T20:55:52.0673987Z │ "schema": { 2022-11-18T20:55:52.0674368Z │ "type": "object", 2022-11-18T20:55:52.0674748Z │ "required": [ 2022-11-18T20:55:52.0675120Z │ "# HELP", 2022-11-18T20:55:52.0675461Z │ "# TYPE", 2022-11-18T20:55:52.0675818Z │ "metric" 2022-11-18T20:55:52.0676168Z │ ], 2022-11-18T20:55:52.0676540Z │ "properties": { 2022-11-18T20:55:52.0676906Z │ "# HELP": { 2022-11-18T20:55:52.0677412Z │ "type": "string", 2022-11-18T20:55:52.0677842Z │ "description": "Help text for To 2022-11-18T20:55:52.0678212Z │ rchServe metric." 2022-11-18T20:55:52.0678576Z │ }, 2022-11-18T20:55:52.0678935Z │ "# TYPE": { 2022-11-18T20:55:52.0679321Z │ "type": "string", 2022-11-18T20:55:52.0679730Z │ "description": "Type of TorchSer 2022-11-18T20:55:52.0798604Z │ ve metric." 2022-11-18T20:55:52.0800066Z │ }, 2022-11-18T20:55:52.0800833Z │ "metric": { 2022-11-18T20:55:52.0806075Z │ "type": "string", 2022-11-18T20:55:52.0806657Z │ "description": "TorchServe appli 2022-11-18T20:55:52.0807058Z │ cation metric." 2022-11-18T20:55:52.0807393Z │ } 2022-11-18T20:55:52.0807744Z │ } 2022-11-18T20:55:52.0808108Z │ } 2022-11-18T20:55:52.0808441Z │ } 2022-11-18T20:55:52.0808745Z │ } 2022-11-18T20:55:52.0809143Z │ }, 2022-11-18T20:55:52.0809502Z │ "500": { 2022-11-18T20:55:52.0809934Z │ "description": "Internal Server Error", 2022-11-18T20:55:52.0810329Z │ "content": { 2022-11-18T20:55:52.0810722Z │ "application/json": { 2022-11-18T20:55:52.0811109Z │ "schema": { 2022-11-18T20:55:52.0811473Z │ "type": "object", 2022-11-18T20:55:52.0811883Z │ "required": [ 2022-11-18T20:55:52.0812252Z │ "code", 2022-11-18T20:55:52.0812608Z │ "type", 2022-11-18T20:55:52.0812952Z │ "message" 2022-11-18T20:55:52.0813300Z │ ], 2022-11-18T20:55:52.0813672Z │ "properties": { 2022-11-18T20:55:52.0814047Z │ "code": { 2022-11-18T20:55:52.0814371Z │ 2022-11-18T20:55:52.0814722Z │ (showing 2.05kB/2.55kB) 2022-11-18T20:55:52.0815041Z └ 2022-11-18T20:55:52.0815580Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:55:52.0816195Z 1ms 1ms 236µs 449µs 29ms 4ms 2ms 72µs 40ms 2022-11-18T20:55:52.0816474Z 2022-11-18T20:55:52.0816800Z √ Status code is 200 2022-11-18T20:55:52.1002467Z 2022-11-18T20:55:52.1003014Z → HTTPS Register Model - Mnist 2022-11-18T20:55:54.4706047Z POST https://localhost:8444/models?url=mnist.mar&model_name=mnist&initial_workers=1&synchronous=true 2022-11-18T20:55:54.4707233Z 200 OK ★ 2.3s time ★ 318B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:54.4707882Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:55:54.4708348Z │ { 2022-11-18T20:55:54.4708871Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:55:54.4709401Z │ ith 1 initial workers" 2022-11-18T20:55:54.4709821Z │ } 2022-11-18T20:55:54.4727408Z └ 2022-11-18T20:55:54.4728842Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:55:54.4729645Z 2ms 748µs (cache) (cache) (cache) 2.3s 2ms 100µs 2.3s 2022-11-18T20:55:54.4729994Z 2022-11-18T20:55:54.4871178Z √ Successful POST request 2022-11-18T20:55:54.4972440Z 2022-11-18T20:55:54.4973235Z → HTTPS Get Mnist Model Description 2022-11-18T20:55:54.5558762Z GET https://localhost:8444/models/mnist 2022-11-18T20:55:54.5559823Z 200 OK ★ 9ms time ★ 238B↑ 736B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:54.5560469Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 469B 2022-11-18T20:55:54.5560997Z │ [ 2022-11-18T20:55:54.5561396Z │ { 2022-11-18T20:55:54.5561827Z │ "modelName": "mnist", 2022-11-18T20:55:54.5562565Z │ "modelVersion": "1.0", 2022-11-18T20:55:54.5563053Z │ "modelUrl": "mnist.mar", 2022-11-18T20:55:54.5563507Z │ "runtime": "python", 2022-11-18T20:55:54.5563979Z │ "minWorkers": 1, 2022-11-18T20:55:54.5564443Z │ "maxWorkers": 1, 2022-11-18T20:55:54.5564893Z │ "batchSize": 1, 2022-11-18T20:55:54.5565339Z │ "maxBatchDelay": 100, 2022-11-18T20:55:54.5565851Z │ "loadedAtStartup": false, 2022-11-18T20:55:54.5566305Z │ "workers": [ 2022-11-18T20:55:54.5566722Z │ { 2022-11-18T20:55:54.5567127Z │ "id": "9000", 2022-11-18T20:55:54.5567638Z │ "startTime": "2022-11-18T20:55:52.255Z", 2022-11-18T20:55:54.5568120Z │ "status": "READY", 2022-11-18T20:55:54.5568561Z │ "memoryUsage": 0, 2022-11-18T20:55:54.5569008Z │ "pid": 1764, 2022-11-18T20:55:54.5569444Z │ "gpu": false, 2022-11-18T20:55:54.5569892Z │ "gpuUsage": "N/A" 2022-11-18T20:55:54.5570297Z │ } 2022-11-18T20:55:54.5570708Z │ ] 2022-11-18T20:55:54.5571104Z │ } 2022-11-18T20:55:54.5571469Z │ ] 2022-11-18T20:55:54.5580202Z └ 2022-11-18T20:55:54.5580834Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:55:54.5581559Z 1ms 532µs (cache) (cache) (cache) 5ms 1ms 73µs 10ms 2022-11-18T20:55:54.5581907Z 2022-11-18T20:55:54.5643333Z √ Successful GET request 2022-11-18T20:55:54.5755596Z 2022-11-18T20:55:54.5756363Z → HTTPS Scale up Workers - Synchronous for Mnist 2022-11-18T20:55:59.0480301Z PUT https://localhost:8444/models/mnist?min_worker=5&max_worker=5&synchronous=true 2022-11-18T20:55:59.0482308Z 200 OK ★ 4.4s time ★ 300B↑ 321B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:59.0483313Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:55:59.0484384Z │ { 2022-11-18T20:55:59.0484947Z │ "status": "Workers scaled to 5 for model: mnist" 2022-11-18T20:55:59.0485518Z │ } 2022-11-18T20:55:59.0485960Z └ 2022-11-18T20:55:59.0486669Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:55:59.0487453Z 1ms 368µs (cache) (cache) (cache) 4.4s 2ms 64µs 4.4s 2022-11-18T20:55:59.0487878Z 2022-11-18T20:55:59.0504780Z √ Successful PUT request 2022-11-18T20:55:59.0718350Z 2022-11-18T20:55:59.0719404Z → HTTPS Scale up Workers - Asynchronous for Mnist 2022-11-18T20:55:59.1311462Z PUT https://localhost:8444/models/mnist?min_worker=6&max_worker=6&synchronous=false 2022-11-18T20:55:59.1312940Z 202 Accepted ★ 7ms time ★ 301B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:55:59.1322073Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:55:59.1331586Z │ { 2022-11-18T20:55:59.1400792Z │ "status": "Processing worker updates..." 2022-11-18T20:55:59.1402864Z │ } 2022-11-18T20:55:59.1467570Z └ 2022-11-18T20:55:59.1475298Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:55:59.1476813Z 1ms 376µs (cache) (cache) (cache) 4ms 1ms 54µs 8ms 2022-11-18T20:55:59.1477284Z 2022-11-18T20:55:59.1477778Z √ Successful PUT request 2022-11-18T20:55:59.1659906Z 2022-11-18T20:55:59.1660735Z → HTTPS - Inference - Mnist_KF 2022-11-18T20:55:59.2805983Z POST https://localhost:8443/v1/models/mnist:predict 2022-11-18T20:55:59.2811703Z 200 OK ★ 44ms time ★ 688B↑ 266B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:55:59.2812930Z ┌ ↑ file ★ 417B 2022-11-18T20:55:59.2819522Z │ 2022-11-18T20:55:59.2820092Z └ 2022-11-18T20:55:59.2820697Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 32B 2022-11-18T20:55:59.2821206Z │ { 2022-11-18T20:55:59.2821726Z │ "predictions": [ 2022-11-18T20:55:59.2822233Z │ 2 2022-11-18T20:55:59.2822689Z │ ] 2022-11-18T20:55:59.2823405Z │ } 2022-11-18T20:55:59.2823861Z └ 2022-11-18T20:55:59.2824596Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:55:59.2825396Z 7ms 740µs (cache) (cache) (cache) 40ms 2ms 81µs 51ms 2022-11-18T20:55:59.2825830Z 2022-11-18T20:55:59.2826233Z √ Status code is 200 2022-11-18T20:55:59.3104388Z 2022-11-18T20:55:59.3105403Z → HTTPS - Explanations - Mnist_KF 2022-11-18T20:55:59.5550523Z POST https://localhost:8443/v1/models/mnist:explain 2022-11-18T20:55:59.5928852Z 200 OK ★ 101ms time ★ 688B↑ 25.39kB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:55:59.5930116Z ┌ ↑ file ★ 417B 2022-11-18T20:55:59.5962579Z │ 2022-11-18T20:55:59.5963238Z └ 2022-11-18T20:55:59.5963837Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 25.15kB 2022-11-18T20:55:59.5964326Z │ { 2022-11-18T20:55:59.5964798Z │ "explanations": [ 2022-11-18T20:55:59.5965267Z │ [ 2022-11-18T20:55:59.5965708Z │ [ 2022-11-18T20:55:59.5966149Z │ [ 2022-11-18T20:55:59.5966632Z │ 0.004570948717123555, 2022-11-18T20:55:59.5967111Z │ 0.00621696950359869, 2022-11-18T20:55:59.5967612Z │ 0.00819756530257639, 2022-11-18T20:55:59.5968093Z │ 0.009563574554839933, 2022-11-18T20:55:59.5968554Z │ 0.008999274400288457, 2022-11-18T20:55:59.5969040Z │ 0.00967347489884473, 2022-11-18T20:55:59.5969521Z │ 0.00759990546889743, 2022-11-18T20:55:59.5969997Z │ 0.006361381155311607, 2022-11-18T20:55:59.5970454Z │ 0.005768828987233948, 2022-11-18T20:55:59.5970935Z │ 0.004394465972850835, 2022-11-18T20:55:59.5971414Z │ 0.004948217797673561, 2022-11-18T20:55:59.5971889Z │ 0.005273460533175704, 2022-11-18T20:55:59.5972343Z │ 0.005523799504482613, 2022-11-18T20:55:59.5972823Z │ 0.007789355996710426, 2022-11-18T20:55:59.5973343Z │ 0.008759363475167876, 2022-11-18T20:55:59.5973795Z │ 0.004304805525164148, 2022-11-18T20:55:59.5974369Z │ 0.010970578941976502, 2022-11-18T20:55:59.5975184Z │ 0.003248439044307383, 2022-11-18T20:55:59.5975677Z │ 0.005998033264566424, 2022-11-18T20:55:59.5976157Z │ 0.0037543659660004176, 2022-11-18T20:55:59.5976620Z │ 0.002765290966317314, 2022-11-18T20:55:59.5977099Z │ 0.004314086388195293, 2022-11-18T20:55:59.5977580Z │ 0.0014008575673494432, 2022-11-18T20:55:59.5978044Z │ 0.0048418467486106885, 2022-11-18T20:55:59.5978528Z │ 0.0006374841539803208, 2022-11-18T20:55:59.5979012Z │ 0.0018558540185062507, 2022-11-18T20:55:59.5979498Z │ -0.0008280458303867654, 2022-11-18T20:55:59.5979939Z │ -0.0 2022-11-18T20:55:59.5980390Z │ ], 2022-11-18T20:55:59.5980830Z │ [ 2022-11-18T20:55:59.5981405Z │ 0.0016625159530346961, 2022-11-18T20:55:59.5981890Z │ 0.004443791836049446, 2022-11-18T20:55:59.5982374Z │ 0.012387840809404818, 2022-11-18T20:55:59.5982850Z │ 0.009450844566697625, 2022-11-18T20:55:59.5983319Z │ 0.01614382020767612, 2022-11-18T20:55:59.5983807Z │ 0.007797501724012741, 2022-11-18T20:55:59.5984286Z │ 0.013942240452575565, 2022-11-18T20:55:59.5984764Z │ 0.007557430539407937, 2022-11-18T20:55:59.5985220Z │ 0.005479090520470613, 2022-11-18T20:55:59.5985700Z │ 0.009751320252680757, 2022-11-18T20:55:59.5986176Z │ 0.0047644071994580985, 2022-11-18T20:55:59.5986633Z │ 0.007292147627537893, 2022-11-18T20:55:59.5987118Z │ 0.011797999771088712, 2022-11-18T20:55:59.5987600Z │ 0.006462685185030748, 2022-11-18T20:55:59.5988078Z │ 0.003383213242765121, 2022-11-18T20:55:59.5988533Z │ 0.009225058381773676, 2022-11-18T20:55:59.5989029Z │ 0.0016750692665810285, 2022-11-18T20:55:59.5989607Z │ 0.007362305503563809, 2022-11-18T20:55:59.5990153Z │ 0.005270058160700518, 2022-11-18T20:55:59.5990610Z │ 0.0054535979835177665, 2022-11-18T20:55:59.5991076Z │ 0.0043427260383291045, 2022-11-18T20:55:59.5991501Z │ 0.005856133854976941, 2022-11-18T20:55:59.5991947Z │ 0.012168384664165323, 2022-11-18T20:55:59.5992388Z │ 0.009902719801994992, 2022-11-18T20:55:59.5992806Z │ 0.009813113130096628, 2022-11-18T20:55:59.5993254Z │ 0.0034427582223001893, 2022-11-18T20:55:59.5993699Z │ 0.002292428498848947, 2022-11-18T20:55:59.5994116Z │ -0.0 2022-11-18T20:55:59.5994506Z │ ], 2022-11-18T20:55:59.5994912Z │ [ 2022-11-18T20:55:59.5995348Z │ 0.011528255210933541, 2022-11-18T20:55:59.5995793Z │ 0.009146989356439473, 2022-11-18T20:55:59.5996227Z │ 0.022265417890940834, 2022-11-18T20:55:59.5996670Z │ 0.01755821671930447, 2022-11-18T20:55:59.5997141Z │ 0.024770554555923724, 2022-11-18T20:55:59.5997561Z │ 0.02141269337793062, 2022-11-18T20:55:59.5997987Z │ 2022-11-18T20:55:59.5998430Z │ (showing 2.05kB/25.15kB) 2022-11-18T20:55:59.5998833Z └ 2022-11-18T20:55:59.5999449Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:55:59.6000172Z 3ms 512µs (cache) (cache) (cache) 97ms 2ms 193µs 103ms 2022-11-18T20:55:59.6000523Z 2022-11-18T20:55:59.6008269Z √ Status code is 200 2022-11-18T20:55:59.6009124Z 2022-11-18T20:55:59.6009595Z → HTTPS UnRegister Model Mnist 2022-11-18T20:56:00.1048378Z DELETE https://localhost:8444/models/mnist 2022-11-18T20:56:00.1049455Z 200 OK ★ 455ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:00.1050099Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:56:00.1050520Z │ { 2022-11-18T20:56:00.1050910Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:56:00.1051276Z │ } 2022-11-18T20:56:00.1064881Z └ 2022-11-18T20:56:00.1065686Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:00.1066381Z 1ms 431µs (cache) (cache) (cache) 452ms 2ms 95µs 456ms 2022-11-18T20:56:00.1066698Z 2022-11-18T20:56:00.1328813Z √ Successful DELETE request 2022-11-18T20:56:00.3550965Z 2022-11-18T20:56:00.3551908Z ┌─────────────────────────┬─────────────────────┬────────────────────┐ 2022-11-18T20:56:00.3577536Z │ │ executed │ failed │ 2022-11-18T20:56:00.3578770Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:00.3579288Z │ iterations │ 1 │ 0 │ 2022-11-18T20:56:00.3580094Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:00.3580551Z │ requests │ 10 │ 0 │ 2022-11-18T20:56:00.3581075Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:00.3581676Z │ test-scripts │ 10 │ 0 │ 2022-11-18T20:56:00.3582208Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:00.3582668Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:56:00.3583180Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:00.3583646Z │ assertions │ 10 │ 0 │ 2022-11-18T20:56:00.3584143Z ├─────────────────────────┴─────────────────────┴────────────────────┤ 2022-11-18T20:56:00.3584600Z │ total run duration: 8.7s │ 2022-11-18T20:56:00.3585114Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:00.3585630Z │ total data received: 110.45kB (approx) │ 2022-11-18T20:56:00.3586155Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:00.3586628Z │ average response time: 768ms [min: 7ms, max: 4.4s, s.d.: 1390ms] │ 2022-11-18T20:56:00.3587138Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:00.3587635Z │ average DNS lookup time: 3ms [min: 236µs, max: 8ms, s.d.: 4ms] │ 2022-11-18T20:56:00.3588151Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:00.3588615Z │ average first byte time: 740ms [min: 4ms, max: 4.4s, s.d.: 1402ms] │ 2022-11-18T20:56:00.3589121Z └────────────────────────────────────────────────────────────────────┘ 2022-11-18T20:56:00.4777505Z TorchServe has stopped. 2022-11-18T20:56:21.9531212Z newman 2022-11-18T20:56:21.9531666Z 2022-11-18T20:56:21.9534424Z kfv2_api_test_collection 2022-11-18T20:56:21.9640133Z 2022-11-18T20:56:21.9641116Z → Model Zoo - Register Model 2022-11-18T20:56:24.5254589Z POST http://localhost:8081/models?url=mnist.mar&model_name=mnist&initial_workers=1&synchronous=true 2022-11-18T20:56:24.5256214Z 200 OK ★ 2.4s time ★ 318B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:24.5260629Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:56:24.5261924Z │ { 2022-11-18T20:56:24.5262349Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:56:24.5262904Z │ ith 1 initial workers" 2022-11-18T20:56:24.5263346Z │ } 2022-11-18T20:56:24.5338423Z └ 2022-11-18T20:56:24.5339005Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:56:24.5339690Z 31ms 11ms 9ms 556µs 2.4s 8ms 571µs 2.4s 2022-11-18T20:56:24.5340206Z 2022-11-18T20:56:24.5587929Z √ Successful POST request 2022-11-18T20:56:24.5893683Z 2022-11-18T20:56:24.5894361Z → Model Zoo - Inference Model 2022-11-18T20:56:24.6819648Z POST http://localhost:8080/v2/models/mnist/infer 2022-11-18T20:56:24.6820694Z 200 OK ★ 34ms time ★ 7.04kB↑ 473B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:56:24.6821375Z ┌ ↑ file ★ 6.77kB 2022-11-18T20:56:24.6822900Z │ (showing 2.05kB/6.77kB) 2022-11-18T20:56:24.6823352Z └ 2022-11-18T20:56:24.6823858Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 238B 2022-11-18T20:56:24.6824333Z │ { 2022-11-18T20:56:24.6824845Z │ "id": "d3b15cad-50a2-4eaf-80ce-8b0a428bd298", 2022-11-18T20:56:24.6825341Z │ "model_name": "mnist", 2022-11-18T20:56:24.6825779Z │ "model_version": "1.0", 2022-11-18T20:56:24.6826215Z │ "outputs": [ 2022-11-18T20:56:24.6826614Z │ { 2022-11-18T20:56:24.6827021Z │ "name": "predict", 2022-11-18T20:56:24.6827454Z │ "shape": [], 2022-11-18T20:56:24.6827901Z │ "datatype": "INT32", 2022-11-18T20:56:24.6828346Z │ "data": [ 2022-11-18T20:56:24.6828762Z │ 1 2022-11-18T20:56:24.6829157Z │ ] 2022-11-18T20:56:24.6829542Z │ } 2022-11-18T20:56:24.6829912Z │ ] 2022-11-18T20:56:24.6830304Z │ } 2022-11-18T20:56:24.6844884Z └ 2022-11-18T20:56:24.6845453Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:56:24.6846139Z 7ms 1ms 336µs 1ms 28ms 2ms 72µs 41ms 2022-11-18T20:56:24.6846493Z 2022-11-18T20:56:24.6941812Z √ Successful POST request 2022-11-18T20:56:24.7023837Z 1. Test expected JSON response 2022-11-18T20:56:24.7384504Z 2022-11-18T20:56:24.7385412Z → Model Zoo - Explanations Model 2022-11-18T20:56:24.8188658Z POST http://localhost:8080/v2/models/mnist/explain 2022-11-18T20:56:24.8189720Z 200 OK ★ 30ms time ★ 7.04kB↑ 14.06kB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:56:24.8190314Z ┌ ↑ file ★ 6.77kB 2022-11-18T20:56:24.8191411Z │ (showing 2.05kB/6.77kB) 2022-11-18T20:56:24.8191863Z └ 2022-11-18T20:56:24.8192857Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 13.83kB 2022-11-18T20:56:24.8193503Z │ { 2022-11-18T20:56:24.8194070Z │ "id": "d3b15cad-50a2-4eaf-80ce-8b0a428bd298", 2022-11-18T20:56:24.8194613Z │ "model_name": "mnist", 2022-11-18T20:56:24.8195079Z │ "model_version": "1.0", 2022-11-18T20:56:24.8195521Z │ "outputs": [ 2022-11-18T20:56:24.8195920Z │ { 2022-11-18T20:56:24.8196351Z │ "name": "explain", 2022-11-18T20:56:24.8196793Z │ "shape": [ 2022-11-18T20:56:24.8197187Z │ 1, 2022-11-18T20:56:24.8197599Z │ 28, 2022-11-18T20:56:24.8198000Z │ 28 2022-11-18T20:56:24.8198395Z │ ], 2022-11-18T20:56:24.8198817Z │ "datatype": "FP64", 2022-11-18T20:56:24.8199247Z │ "data": [ 2022-11-18T20:56:24.8199656Z │ -0.0, 2022-11-18T20:56:24.8200047Z │ -0.0, 2022-11-18T20:56:24.8200581Z │ -0.0, 2022-11-18T20:56:24.8200980Z │ -0.0, 2022-11-18T20:56:24.8201381Z │ -0.0, 2022-11-18T20:56:24.8201768Z │ -0.0, 2022-11-18T20:56:24.8202165Z │ -0.0, 2022-11-18T20:56:24.8202574Z │ -0.0, 2022-11-18T20:56:24.8202956Z │ -0.0, 2022-11-18T20:56:24.8203357Z │ -0.0, 2022-11-18T20:56:24.8203753Z │ -0.0, 2022-11-18T20:56:24.8204150Z │ -0.0, 2022-11-18T20:56:24.8204534Z │ -0.0, 2022-11-18T20:56:24.8204928Z │ -0.0, 2022-11-18T20:56:24.8205330Z │ -0.0, 2022-11-18T20:56:24.8205711Z │ -0.0, 2022-11-18T20:56:24.8206113Z │ 0.0, 2022-11-18T20:56:24.8206514Z │ -0.0, 2022-11-18T20:56:24.8206967Z │ -0.0, 2022-11-18T20:56:24.8207349Z │ 0.0, 2022-11-18T20:56:24.8207744Z │ -0.0, 2022-11-18T20:56:24.8208132Z │ 0.0, 2022-11-18T20:56:24.8208508Z │ -0.0, 2022-11-18T20:56:24.8208904Z │ -0.0, 2022-11-18T20:56:24.8209299Z │ -0.0, 2022-11-18T20:56:24.8209670Z │ -0.0, 2022-11-18T20:56:24.8210061Z │ -0.0, 2022-11-18T20:56:24.8210448Z │ 0.0, 2022-11-18T20:56:24.8210836Z │ -0.0, 2022-11-18T20:56:24.8211221Z │ -0.0, 2022-11-18T20:56:24.8211612Z │ -0.0, 2022-11-18T20:56:24.8212000Z │ -0.0, 2022-11-18T20:56:24.8212373Z │ -0.0, 2022-11-18T20:56:24.8212766Z │ -0.0, 2022-11-18T20:56:24.8213170Z │ -0.0, 2022-11-18T20:56:24.8213396Z │ 0.0, 2022-11-18T20:56:24.8213610Z │ -0.0, 2022-11-18T20:56:24.8213804Z │ 0.0, 2022-11-18T20:56:24.8217991Z │ -0.0, 2022-11-18T20:56:24.8218319Z │ -0.0, 2022-11-18T20:56:24.8218543Z │ -0.0, 2022-11-18T20:56:24.8218766Z │ 0.0, 2022-11-18T20:56:24.8218971Z │ 0.0, 2022-11-18T20:56:24.8219177Z │ 0.0, 2022-11-18T20:56:24.8219391Z │ 0.0, 2022-11-18T20:56:24.8219617Z │ 0.0, 2022-11-18T20:56:24.8219828Z │ 0.0, 2022-11-18T20:56:24.8220035Z │ 0.0, 2022-11-18T20:56:24.8220229Z │ 0.0, 2022-11-18T20:56:24.8220436Z │ 0.0, 2022-11-18T20:56:24.8220659Z │ -0.0, 2022-11-18T20:56:24.8220860Z │ -0.0, 2022-11-18T20:56:24.8221071Z │ 0.0, 2022-11-18T20:56:24.8221281Z │ 0.0, 2022-11-18T20:56:24.8221477Z │ -0.0, 2022-11-18T20:56:24.8221685Z │ 0.0, 2022-11-18T20:56:24.8221895Z │ -0.0, 2022-11-18T20:56:24.8222092Z │ -0.0, 2022-11-18T20:56:24.8222299Z │ -0.0, 2022-11-18T20:56:24.8222512Z │ -0.0, 2022-11-18T20:56:24.8222706Z │ -0.0, 2022-11-18T20:56:24.8222910Z │ 0.0, 2022-11-18T20:56:24.8223116Z │ 0.0, 2022-11-18T20:56:24.8223310Z │ 0.0, 2022-11-18T20:56:24.8223516Z │ 0.0, 2022-11-18T20:56:24.8223723Z │ 0.0, 2022-11-18T20:56:24.8223916Z │ 0.0, 2022-11-18T20:56:24.8224131Z │ 0.0, 2022-11-18T20:56:24.8224341Z │ 0.0, 2022-11-18T20:56:24.8224534Z │ 0.0, 2022-11-18T20:56:24.8224746Z │ -0.0, 2022-11-18T20:56:24.8224959Z │ -0.0, 2022-11-18T20:56:24.8225357Z │ -0.0, 2022-11-18T20:56:24.8225577Z │ 0.0, 2022-11-18T20:56:24.8225788Z │ -0.0, 2022-11-18T20:56:24.8225996Z │ 0.0, 2022-11-18T20:56:24.8226189Z │ 0.0, 2022-11-18T20:56:24.8226398Z │ 0.0, 2022-11-18T20:56:24.8226615Z │ -0.0, 2022-11-18T20:56:24.8226811Z │ -0.0, 2022-11-18T20:56:24.8227023Z │ -0.0, 2022-11-18T20:56:24.8227228Z │ 0.0, 2022-11-18T20:56:24.8227421Z │ -0.0, 2022-11-18T20:56:24.8227629Z │ 0.0, 2022-11-18T20:56:24.8227836Z │ -0.0, 2022-11-18T20:56:24.8228034Z │ -0.0, 2022-11-18T20:56:24.8228243Z │ -0.0, 2022-11-18T20:56:24.8228452Z │ -0.0, 2022-11-18T20:56:24.8228735Z │ 0.0, 2022-11-18T20:56:24.8228943Z │ 0.0, 2022-11-18T20:56:24.8229153Z │ 0.0, 2022-11-18T20:56:24.8229348Z │ 0.0, 2022-11-18T20:56:24.8229555Z │ 0.0, 2022-11-18T20:56:24.8229777Z │ 0.0, 2022-11-18T20:56:24.8229970Z │ 0.0, 2022-11-18T20:56:24.8230179Z │ -0.0, 2022-11-18T20:56:24.8230393Z │ -0.0, 2022-11-18T20:56:24.8230590Z │ 0.0, 2022-11-18T20:56:24.8230794Z │ 0.0, 2022-11-18T20:56:24.8231001Z │ -0.0, 2022-11-18T20:56:24.8231196Z │ -0.0, 2022-11-18T20:56:24.8231411Z │ -0.0, 2022-11-18T20:56:24.8231612Z │ -0.0, 2022-11-18T20:56:24.8231823Z │ -0.0, 2022-11-18T20:56:24.8232041Z │ 0.0, 2022-11-18T20:56:24.8232241Z │ 0.0, 2022-11-18T20:56:24.8232452Z │ -0.0, 2022-11-18T20:56:24.8232662Z │ -0.0, 2022-11-18T20:56:24.8232857Z │ -0.0, 2022-11-18T20:56:24.8233062Z │ 0.0, 2022-11-18T20:56:24.8233279Z │ 0.0, 2022-11-18T20:56:24.8233471Z │ 0.0, 2022-11-18T20:56:24.8233682Z │ -0.0, 2022-11-18T20:56:24.8233891Z │ -0.0, 2022-11-18T20:56:24.8234089Z │ -0.0, 2022-11-18T20:56:24.8234305Z │ -0.0, 2022-11-18T20:56:24.8234514Z │ 0.0, 2022-11-18T20:56:24.8234707Z │ 0.0, 2022-11-18T20:56:24.8234910Z │ 0.0, 2022-11-18T20:56:24.8235121Z │ 0.0, 2022-11-18T20:56:24.8235316Z │ 0.0, 2022-11-18T20:56:24.8235528Z │ 0.0, 2022-11-18T20:56:24.8235741Z │ -0.0, 2022-11-18T20:56:24.8235999Z │ -0.004054752791506105, 2022-11-18T20:56:24.8236267Z │ -0.00022612876928139407, 2022-11-18T20:56:24.8236533Z │ -0.0001273412993359035, 2022-11-18T20:56:24.8236795Z │ 0.005648369033259622, 2022-11-18T20:56:24.8237006Z │ 2022-11-18T20:56:24.8237259Z │ (showing 2.05kB/13.83kB) 2022-11-18T20:56:24.8237476Z └ 2022-11-18T20:56:24.8237889Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:56:24.8238366Z 2ms 490µs (cache) (cache) 26ms 2ms 96µs 32ms 2022-11-18T20:56:24.8238555Z 2022-11-18T20:56:24.8479170Z √ Successful POST request 2022-11-18T20:56:24.8510345Z √ Test expected JSON response 2022-11-18T20:56:24.8793197Z 2022-11-18T20:56:24.8793958Z → Model Zoo - Unregister model 2022-11-18T20:56:25.0157896Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:56:25.0158909Z 200 OK ★ 87ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:25.0159951Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:56:25.0160575Z │ { 2022-11-18T20:56:25.0160951Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:56:25.0161328Z │ } 2022-11-18T20:56:25.0232433Z └ 2022-11-18T20:56:25.0233163Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:56:25.0233870Z 1ms 293µs (cache) (cache) 83ms 2ms 90µs 88ms 2022-11-18T20:56:25.0234177Z 2022-11-18T20:56:25.0416541Z √ Successful DELETE request 2022-11-18T20:56:25.0655162Z 2022-11-18T20:56:25.0656017Z → Model Zoo - Model Metrics 2022-11-18T20:56:25.1416750Z GET http://localhost:8082/metrics 2022-11-18T20:56:25.1417801Z 200 OK ★ 38ms time ★ 233B↑ 1.04kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:25.1418377Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 751B 2022-11-18T20:56:25.1418896Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:56:25.1419290Z │ erence requests. 2022-11-18T20:56:25.1419715Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:56:25.1420192Z │ ts_inference_requests_total{uuid="31bdcd13-b143-499b-b 2022-11-18T20:56:25.1420674Z │ 131-1059b41f9d1e",model_name="mnist",model_version="de 2022-11-18T20:56:25.1421043Z │ fault",} 2.0 2022-11-18T20:56:25.1421745Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:56:25.1422203Z │ ference duration in microseconds 2022-11-18T20:56:25.1422647Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:56:25.1423153Z │ ts_inference_latency_microseconds{uuid="31bdcd13-b143- 2022-11-18T20:56:25.1423634Z │ 499b-b131-1059b41f9d1e",model_name="mnist",model_versi 2022-11-18T20:56:25.1424032Z │ on="default",} 33582.7 2022-11-18T20:56:25.1424455Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:56:25.1424881Z │ duration in microseconds 2022-11-18T20:56:25.1425305Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:56:25.1425780Z │ ts_queue_latency_microseconds{uuid="31bdcd13-b143-499b 2022-11-18T20:56:25.1426241Z │ -b131-1059b41f9d1e",model_name="mnist",model_version=" 2022-11-18T20:56:25.1426626Z │ default",} 258.5 2022-11-18T20:56:25.1429808Z └ 2022-11-18T20:56:25.1430723Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:56:25.1431340Z 2ms 1ms 269µs 270µs 31ms 4ms 76µs 39ms 2022-11-18T20:56:25.1431635Z 2022-11-18T20:56:25.1665733Z √ Successful GET request 2022-11-18T20:56:25.3901967Z 2022-11-18T20:56:25.3902980Z ┌─────────────────────────┬─────────────────────┬────────────────────┐ 2022-11-18T20:56:25.3903620Z │ │ executed │ failed │ 2022-11-18T20:56:25.3904158Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:25.3904632Z │ iterations │ 1 │ 0 │ 2022-11-18T20:56:25.3905115Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:25.3905566Z │ requests │ 5 │ 0 │ 2022-11-18T20:56:25.3906055Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:25.3918489Z │ test-scripts │ 5 │ 0 │ 2022-11-18T20:56:25.3919657Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:25.3920450Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:56:25.3921013Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:25.3921479Z │ assertions │ 7 │ 1 │ 2022-11-18T20:56:25.3921954Z ├─────────────────────────┴─────────────────────┴────────────────────┤ 2022-11-18T20:56:25.3922415Z │ total run duration: 3.2s │ 2022-11-18T20:56:25.3922906Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:25.3923375Z │ total data received: 14.94kB (approx) │ 2022-11-18T20:56:25.3923860Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:25.3924336Z │ average response time: 531ms [min: 30ms, max: 2.4s, s.d.: 968ms] │ 2022-11-18T20:56:25.3924963Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:25.3925467Z │ average DNS lookup time: 3ms [min: 269µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:56:25.3925963Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:25.3926432Z │ average first byte time: 521ms [min: 26ms, max: 2.4s, s.d.: 958ms] │ 2022-11-18T20:56:25.3926926Z └────────────────────────────────────────────────────────────────────┘ 2022-11-18T20:56:25.3927277Z 2022-11-18T20:56:25.3927503Z # failure detail 2022-11-18T20:56:25.3927843Z 2022-11-18T20:56:25.3928224Z 1. AssertionError Test expected JSON response 2022-11-18T20:56:25.3928626Z expected false to be true 2022-11-18T20:56:25.3929086Z at assertion:1 in test-script 2022-11-18T20:56:25.3929534Z inside "Model Zoo - Inference Model" 2022-11-18T20:56:25.5073688Z TorchServe has stopped. 2022-11-18T20:56:46.9865749Z newman 2022-11-18T20:56:46.9866168Z 2022-11-18T20:56:46.9869008Z kfv2_https_test_collection 2022-11-18T20:56:47.0043871Z 2022-11-18T20:56:47.0044521Z → HTTPS Inference API Description 2022-11-18T20:56:47.3832098Z OPTIONS https://localhost:8443 2022-11-18T20:56:47.3836944Z 200 OK ★ 228ms time ★ 230B↑ 23.68kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:47.3854802Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 23.41kB 2022-11-18T20:56:47.3855379Z │ { 2022-11-18T20:56:47.3855742Z │ "openapi": "3.0.1", 2022-11-18T20:56:47.3856088Z │ "info": { 2022-11-18T20:56:47.3856984Z │ "title": "TorchServe APIs", 2022-11-18T20:56:47.3857522Z │ "description": "TorchServe is a flexible and easy 2022-11-18T20:56:47.3857986Z │ to use tool for serving deep learning models", 2022-11-18T20:56:47.3858427Z │ "version": "0.6.1\r" 2022-11-18T20:56:47.3858762Z │ }, 2022-11-18T20:56:47.3859069Z │ "paths": { 2022-11-18T20:56:47.3859386Z │ "/": { 2022-11-18T20:56:47.3859715Z │ "options": { 2022-11-18T20:56:47.3860393Z │ "description": "Get openapi description.", 2022-11-18T20:56:47.3862275Z │ "operationId": "apiDescription", 2022-11-18T20:56:47.3862681Z │ "parameters": [], 2022-11-18T20:56:47.3863060Z │ "responses": { 2022-11-18T20:56:47.3863391Z │ "200": { 2022-11-18T20:56:47.3863801Z │ "description": "A openapi 3.0.1 descriptor 2022-11-18T20:56:47.3864160Z │ ", 2022-11-18T20:56:47.3864476Z │ "content": { 2022-11-18T20:56:47.3865447Z │ "application/json": { 2022-11-18T20:56:47.3865827Z │ "schema": { 2022-11-18T20:56:47.3866200Z │ "type": "object", 2022-11-18T20:56:47.3866558Z │ "required": [ 2022-11-18T20:56:47.3867084Z │ "openapi", 2022-11-18T20:56:47.3867439Z │ "info", 2022-11-18T20:56:47.3867790Z │ "paths" 2022-11-18T20:56:47.3868116Z │ ], 2022-11-18T20:56:47.3868511Z │ "properties": { 2022-11-18T20:56:47.3869007Z │ "openapi": { 2022-11-18T20:56:47.3869407Z │ "type": "string" 2022-11-18T20:56:47.3869751Z │ }, 2022-11-18T20:56:47.3870088Z │ "info": { 2022-11-18T20:56:47.3870434Z │ "type": "object" 2022-11-18T20:56:47.3870772Z │ }, 2022-11-18T20:56:47.3871110Z │ "paths": { 2022-11-18T20:56:47.3871461Z │ "type": "object" 2022-11-18T20:56:47.3871797Z │ } 2022-11-18T20:56:47.3872116Z │ } 2022-11-18T20:56:47.3872433Z │ } 2022-11-18T20:56:47.3872735Z │ } 2022-11-18T20:56:47.3873053Z │ } 2022-11-18T20:56:47.3873859Z │ }, 2022-11-18T20:56:47.3874214Z │ "500": { 2022-11-18T20:56:47.3874620Z │ "description": "Internal Server Error", 2022-11-18T20:56:47.3875019Z │ "content": { 2022-11-18T20:56:47.3875391Z │ "application/json": { 2022-11-18T20:56:47.3875742Z │ "schema": { 2022-11-18T20:56:47.3876103Z │ "type": "object", 2022-11-18T20:56:47.3876462Z │ "required": [ 2022-11-18T20:56:47.3876808Z │ "code", 2022-11-18T20:56:47.3877132Z │ "type", 2022-11-18T20:56:47.3877728Z │ "message" 2022-11-18T20:56:47.3878062Z │ ], 2022-11-18T20:56:47.3878404Z │ "properties": { 2022-11-18T20:56:47.3878762Z │ "code": { 2022-11-18T20:56:47.3879375Z │ "type": "integer", 2022-11-18T20:56:47.3879828Z │ "description": "Error code." 2022-11-18T20:56:47.3880179Z │ }, 2022-11-18T20:56:47.3880520Z │ "type": { 2022-11-18T20:56:47.3880930Z │ "type": "string", 2022-11-18T20:56:47.3881341Z │ "description": "Error type." 2022-11-18T20:56:47.3881706Z │ }, 2022-11-18T20:56:47.3882057Z │ "message": { 2022-11-18T20:56:47.3882433Z │ "type": "string", 2022-11-18T20:56:47.3882845Z │ "description": "Error message." 2022-11-18T20:56:47.3883213Z │ } 2022-11-18T20:56:47.3883533Z │ } 2022-11-18T20:56:47.3883850Z │ } 2022-11-18T20:56:47.3884149Z │ } 2022-11-18T20:56:47.3884452Z │ } 2022-11-18T20:56:47.3884755Z │ } 2022-11-18T20:56:47.3885043Z │ } 2022-11-18T20:56:47.3885353Z │ } 2022-11-18T20:56:47.3885650Z │ }, 2022-11-18T20:56:47.3886024Z │ "/ping": { 2022-11-18T20:56:47.3886325Z │ 2022-11-18T20:56:47.3886661Z │ (showing 2.05kB/23.41kB) 2022-11-18T20:56:47.3886961Z └ 2022-11-18T20:56:47.3887602Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:47.3888219Z 42ms 13ms 9ms 1ms 154ms 39ms 7ms 489µs 270ms 2022-11-18T20:56:47.3888492Z 2022-11-18T20:56:47.3888712Z √ Status code is 200 2022-11-18T20:56:47.4100614Z 2022-11-18T20:56:47.4101318Z → HTTPS Management API Description 2022-11-18T20:56:47.5064498Z OPTIONS https://localhost:8444 2022-11-18T20:56:47.5065653Z 200 OK ★ 58ms time ★ 230B↑ 58.87kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:47.5066255Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 58.61kB 2022-11-18T20:56:47.5066660Z │ { 2022-11-18T20:56:47.5066992Z │ "openapi": "3.0.1", 2022-11-18T20:56:47.5070775Z │ "info": { 2022-11-18T20:56:47.5071167Z │ "title": "TorchServe APIs", 2022-11-18T20:56:47.5071619Z │ "description": "TorchServe is a flexible and easy 2022-11-18T20:56:47.5072110Z │ to use tool for serving deep learning models", 2022-11-18T20:56:47.5073624Z │ "version": "0.6.1\r" 2022-11-18T20:56:47.5073968Z │ }, 2022-11-18T20:56:47.5074279Z │ "paths": { 2022-11-18T20:56:47.5074601Z │ "/": { 2022-11-18T20:56:47.5074937Z │ "options": { 2022-11-18T20:56:47.5075358Z │ "description": "Get openapi description.", 2022-11-18T20:56:47.5075801Z │ "operationId": "apiDescription", 2022-11-18T20:56:47.5076201Z │ "parameters": [], 2022-11-18T20:56:47.5076561Z │ "responses": { 2022-11-18T20:56:47.5076890Z │ "200": { 2022-11-18T20:56:47.5077304Z │ "description": "A openapi 3.0.1 descriptor 2022-11-18T20:56:47.5078091Z │ ", 2022-11-18T20:56:47.5078442Z │ "content": { 2022-11-18T20:56:47.5078813Z │ "application/json": { 2022-11-18T20:56:47.5079186Z │ "schema": { 2022-11-18T20:56:47.5079557Z │ "type": "object", 2022-11-18T20:56:47.5079927Z │ "required": [ 2022-11-18T20:56:47.5081293Z │ "openapi", 2022-11-18T20:56:47.5081654Z │ "info", 2022-11-18T20:56:47.5082008Z │ "paths" 2022-11-18T20:56:47.5082336Z │ ], 2022-11-18T20:56:47.5082717Z │ "properties": { 2022-11-18T20:56:47.5083539Z │ "openapi": { 2022-11-18T20:56:47.5083962Z │ "type": "string" 2022-11-18T20:56:47.5084386Z │ }, 2022-11-18T20:56:47.5084745Z │ "info": { 2022-11-18T20:56:47.5085101Z │ "type": "object" 2022-11-18T20:56:47.5085440Z │ }, 2022-11-18T20:56:47.5085797Z │ "paths": { 2022-11-18T20:56:47.5086147Z │ "type": "object" 2022-11-18T20:56:47.5086731Z │ } 2022-11-18T20:56:47.5087060Z │ } 2022-11-18T20:56:47.5087777Z │ } 2022-11-18T20:56:47.5088088Z │ } 2022-11-18T20:56:47.5088398Z │ } 2022-11-18T20:56:47.5088706Z │ }, 2022-11-18T20:56:47.5089030Z │ "500": { 2022-11-18T20:56:47.5089426Z │ "description": "Internal Server Error", 2022-11-18T20:56:47.5089817Z │ "content": { 2022-11-18T20:56:47.5090184Z │ "application/json": { 2022-11-18T20:56:47.5090535Z │ "schema": { 2022-11-18T20:56:47.5090898Z │ "type": "object", 2022-11-18T20:56:47.5091259Z │ "required": [ 2022-11-18T20:56:47.5091603Z │ "code", 2022-11-18T20:56:47.5091923Z │ "type", 2022-11-18T20:56:47.5092278Z │ "message" 2022-11-18T20:56:47.5092609Z │ ], 2022-11-18T20:56:47.5093188Z │ "properties": { 2022-11-18T20:56:47.5093550Z │ "code": { 2022-11-18T20:56:47.5094087Z │ "type": "integer", 2022-11-18T20:56:47.5094509Z │ "description": "Error code." 2022-11-18T20:56:47.5094855Z │ }, 2022-11-18T20:56:47.5095192Z │ "type": { 2022-11-18T20:56:47.5095793Z │ "type": "string", 2022-11-18T20:56:47.5096203Z │ "description": "Error type." 2022-11-18T20:56:47.5096589Z │ }, 2022-11-18T20:56:47.5096956Z │ "message": { 2022-11-18T20:56:47.5097328Z │ "type": "string", 2022-11-18T20:56:47.5097729Z │ "description": "Error message." 2022-11-18T20:56:47.5098089Z │ } 2022-11-18T20:56:47.5098518Z │ } 2022-11-18T20:56:47.5098836Z │ } 2022-11-18T20:56:47.5099137Z │ } 2022-11-18T20:56:47.5099447Z │ } 2022-11-18T20:56:47.5099752Z │ } 2022-11-18T20:56:47.5100057Z │ } 2022-11-18T20:56:47.5100361Z │ } 2022-11-18T20:56:47.5100656Z │ }, 2022-11-18T20:56:47.5100955Z │ "/models": { 2022-11-18T20:56:47.5101259Z │ 2022-11-18T20:56:47.5101597Z │ (showing 2.05kB/58.61kB) 2022-11-18T20:56:47.5102464Z └ 2022-11-18T20:56:47.5103017Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:47.5103621Z 1ms 1ms 248µs 459µs 29ms 22ms 2ms 149µs 59ms 2022-11-18T20:56:47.5103888Z 2022-11-18T20:56:47.5414740Z √ Status code is 200 2022-11-18T20:56:47.5660279Z 2022-11-18T20:56:47.5661064Z → HTTPS Metrics API Description 2022-11-18T20:56:47.6579621Z OPTIONS https://localhost:8445 2022-11-18T20:56:47.6586460Z 200 OK ★ 40ms time ★ 230B↑ 2.82kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:47.6587117Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 2.55kB 2022-11-18T20:56:47.6587541Z │ { 2022-11-18T20:56:47.6587897Z │ "openapi": "3.0.1", 2022-11-18T20:56:47.6588226Z │ "info": { 2022-11-18T20:56:47.6588605Z │ "title": "TorchServe APIs", 2022-11-18T20:56:47.6589554Z │ "description": "TorchServe is a flexible and easy 2022-11-18T20:56:47.6710850Z │ to use tool for serving deep learning models", 2022-11-18T20:56:47.6713010Z │ "version": "0.6.1\r" 2022-11-18T20:56:47.6713717Z │ }, 2022-11-18T20:56:47.6720003Z │ "paths": { 2022-11-18T20:56:47.6724538Z │ "/metrics": { 2022-11-18T20:56:47.6725322Z │ "get": { 2022-11-18T20:56:47.6725798Z │ "description": "Get TorchServe application met 2022-11-18T20:56:47.6726242Z │ rics in prometheus format.", 2022-11-18T20:56:47.6726675Z │ "operationId": "metrics", 2022-11-18T20:56:47.6727057Z │ "parameters": [ 2022-11-18T20:56:47.6727394Z │ { 2022-11-18T20:56:47.6727737Z │ "in": "query", 2022-11-18T20:56:47.6728103Z │ "name": "name[]", 2022-11-18T20:56:47.6728526Z │ "description": "Names of metrics to filter 2022-11-18T20:56:47.6729219Z │ ", 2022-11-18T20:56:47.6729562Z │ "required": false, 2022-11-18T20:56:47.6729920Z │ "schema": { 2022-11-18T20:56:47.6730280Z │ "type": "string" 2022-11-18T20:56:47.6730616Z │ } 2022-11-18T20:56:47.6730918Z │ } 2022-11-18T20:56:47.6731231Z │ ], 2022-11-18T20:56:47.6731568Z │ "responses": { 2022-11-18T20:56:47.6731910Z │ "200": { 2022-11-18T20:56:47.6732315Z │ "description": "TorchServe application met 2022-11-18T20:56:47.6734053Z │ rics", 2022-11-18T20:56:47.6734438Z │ "content": { 2022-11-18T20:56:47.6734912Z │ "text/plain; version=0.0.4; charset=utf- 2022-11-18T20:56:47.6735271Z │ 8": { 2022-11-18T20:56:47.6736039Z │ "schema": { 2022-11-18T20:56:47.6736645Z │ "type": "object", 2022-11-18T20:56:47.6737026Z │ "required": [ 2022-11-18T20:56:47.6737385Z │ "# HELP", 2022-11-18T20:56:47.6737721Z │ "# TYPE", 2022-11-18T20:56:47.6738062Z │ "metric" 2022-11-18T20:56:47.6738408Z │ ], 2022-11-18T20:56:47.6738767Z │ "properties": { 2022-11-18T20:56:47.6739119Z │ "# HELP": { 2022-11-18T20:56:47.6739492Z │ "type": "string", 2022-11-18T20:56:47.6741143Z │ "description": "Help text for To 2022-11-18T20:56:47.6741564Z │ rchServe metric." 2022-11-18T20:56:47.6741916Z │ }, 2022-11-18T20:56:47.6742445Z │ "# TYPE": { 2022-11-18T20:56:47.6742816Z │ "type": "string", 2022-11-18T20:56:47.6743216Z │ "description": "Type of TorchSer 2022-11-18T20:56:47.6743588Z │ ve metric." 2022-11-18T20:56:47.6743926Z │ }, 2022-11-18T20:56:47.6744267Z │ "metric": { 2022-11-18T20:56:47.6744625Z │ "type": "string", 2022-11-18T20:56:47.6745042Z │ "description": "TorchServe appli 2022-11-18T20:56:47.6750829Z │ cation metric." 2022-11-18T20:56:47.6755909Z │ } 2022-11-18T20:56:47.6756297Z │ } 2022-11-18T20:56:47.6756639Z │ } 2022-11-18T20:56:47.6756957Z │ } 2022-11-18T20:56:47.6757262Z │ } 2022-11-18T20:56:47.6757574Z │ }, 2022-11-18T20:56:47.6757959Z │ "500": { 2022-11-18T20:56:47.6758394Z │ "description": "Internal Server Error", 2022-11-18T20:56:47.6758800Z │ "content": { 2022-11-18T20:56:47.6759795Z │ "application/json": { 2022-11-18T20:56:47.6760181Z │ "schema": { 2022-11-18T20:56:47.6760554Z │ "type": "object", 2022-11-18T20:56:47.6760925Z │ "required": [ 2022-11-18T20:56:47.6761280Z │ "code", 2022-11-18T20:56:47.6761632Z │ "type", 2022-11-18T20:56:47.6761964Z │ "message" 2022-11-18T20:56:47.6762299Z │ ], 2022-11-18T20:56:47.6762656Z │ "properties": { 2022-11-18T20:56:47.6763024Z │ "code": { 2022-11-18T20:56:47.6763331Z │ 2022-11-18T20:56:47.6763673Z │ (showing 2.05kB/2.55kB) 2022-11-18T20:56:47.6763975Z └ 2022-11-18T20:56:47.6764891Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:47.6765497Z 1ms 1ms 249µs 437µs 29ms 4ms 2ms 80µs 41ms 2022-11-18T20:56:47.6765776Z 2022-11-18T20:56:47.6766093Z √ Status code is 200 2022-11-18T20:56:47.6912261Z 2022-11-18T20:56:47.6912794Z → HTTPS Register Model - Mnist 2022-11-18T20:56:50.0900910Z POST https://localhost:8444/models?url=mnist.mar&model_name=mnist&initial_workers=1&synchronous=true 2022-11-18T20:56:50.0902065Z 200 OK ★ 2.3s time ★ 318B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:50.0902766Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:56:50.0903245Z │ { 2022-11-18T20:56:50.0903762Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:56:50.0904244Z │ ith 1 initial workers" 2022-11-18T20:56:50.0904659Z │ } 2022-11-18T20:56:50.0922193Z └ 2022-11-18T20:56:50.0922826Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:50.0923551Z 2ms 746µs (cache) (cache) (cache) 2.3s 2ms 93µs 2.3s 2022-11-18T20:56:50.0923955Z 2022-11-18T20:56:50.1052760Z √ Successful POST request 2022-11-18T20:56:50.1314374Z 2022-11-18T20:56:50.1315438Z → HTTPS Get Mnist Model Description 2022-11-18T20:56:50.1771321Z GET https://localhost:8444/models/mnist 2022-11-18T20:56:50.1772406Z 200 OK ★ 8ms time ★ 238B↑ 736B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:50.1773086Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 469B 2022-11-18T20:56:50.1773612Z │ [ 2022-11-18T20:56:50.1774001Z │ { 2022-11-18T20:56:50.1774453Z │ "modelName": "mnist", 2022-11-18T20:56:50.1774932Z │ "modelVersion": "1.0", 2022-11-18T20:56:50.1775391Z │ "modelUrl": "mnist.mar", 2022-11-18T20:56:50.1775869Z │ "runtime": "python", 2022-11-18T20:56:50.1776325Z │ "minWorkers": 1, 2022-11-18T20:56:50.1776774Z │ "maxWorkers": 1, 2022-11-18T20:56:50.1777197Z │ "batchSize": 1, 2022-11-18T20:56:50.1778672Z │ "maxBatchDelay": 100, 2022-11-18T20:56:50.1779164Z │ "loadedAtStartup": false, 2022-11-18T20:56:50.1779620Z │ "workers": [ 2022-11-18T20:56:50.1780016Z │ { 2022-11-18T20:56:50.1780465Z │ "id": "9000", 2022-11-18T20:56:50.1780975Z │ "startTime": "2022-11-18T20:56:47.877Z", 2022-11-18T20:56:50.1781441Z │ "status": "READY", 2022-11-18T20:56:50.1781906Z │ "memoryUsage": 0, 2022-11-18T20:56:50.1782363Z │ "pid": 6360, 2022-11-18T20:56:50.1782799Z │ "gpu": false, 2022-11-18T20:56:50.1783235Z │ "gpuUsage": "N/A" 2022-11-18T20:56:50.1783656Z │ } 2022-11-18T20:56:50.1784050Z │ ] 2022-11-18T20:56:50.1784428Z │ } 2022-11-18T20:56:50.1784820Z │ ] 2022-11-18T20:56:50.1792824Z └ 2022-11-18T20:56:50.1793422Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:50.1794181Z 1ms 498µs (cache) (cache) (cache) 5ms 1ms 67µs 9ms 2022-11-18T20:56:50.1794571Z 2022-11-18T20:56:50.1982917Z √ Successful GET request 2022-11-18T20:56:50.2251641Z 2022-11-18T20:56:50.2252412Z → HTTPS Scale up Workers - Synchronous for Mnist 2022-11-18T20:56:54.9638508Z PUT https://localhost:8444/models/mnist?min_worker=5&max_worker=5&synchronous=true 2022-11-18T20:56:54.9639662Z 200 OK ★ 4.6s time ★ 300B↑ 321B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:54.9640343Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 55B 2022-11-18T20:56:54.9640848Z │ { 2022-11-18T20:56:54.9641351Z │ "status": "Workers scaled to 5 for model: mnist" 2022-11-18T20:56:54.9641801Z │ } 2022-11-18T20:56:54.9683752Z └ 2022-11-18T20:56:54.9685228Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:54.9686299Z 1ms 355µs (cache) (cache) (cache) 4.6s 2ms 71µs 4.6s 2022-11-18T20:56:54.9687235Z 2022-11-18T20:56:54.9717929Z √ Successful PUT request 2022-11-18T20:56:54.9939502Z 2022-11-18T20:56:54.9940246Z → HTTPS Scale up Workers - Asynchronous for Mnist 2022-11-18T20:56:55.0525841Z PUT https://localhost:8444/models/mnist?min_worker=6&max_worker=6&synchronous=false 2022-11-18T20:56:55.0527251Z 202 Accepted ★ 6ms time ★ 301B↑ 319B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:55.0536098Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:56:55.0546338Z │ { 2022-11-18T20:56:55.0662669Z │ "status": "Processing worker updates..." 2022-11-18T20:56:55.0665135Z │ } 2022-11-18T20:56:55.0691991Z └ 2022-11-18T20:56:55.0698924Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:55.0699914Z 1ms 395µs (cache) (cache) (cache) 3ms 1ms 57µs 7ms 2022-11-18T20:56:55.0700366Z 2022-11-18T20:56:55.0700835Z √ Successful PUT request 2022-11-18T20:56:55.0884293Z 2022-11-18T20:56:55.0885229Z → HTTPS - Inference - Mnist_KF 2022-11-18T20:56:55.1736719Z POST https://localhost:8443/v2/models/mnist/infer 2022-11-18T20:56:55.1738103Z 200 OK ★ 40ms time ★ 7.04kB↑ 473B↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:56:55.1738713Z ┌ ↑ file ★ 6.77kB 2022-11-18T20:56:55.1739196Z │ (showing 2.05kB/6.77kB) 2022-11-18T20:56:55.1739623Z └ 2022-11-18T20:56:55.1740112Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 238B 2022-11-18T20:56:55.1740557Z │ { 2022-11-18T20:56:55.1741074Z │ "id": "d3b15cad-50a2-4eaf-80ce-8b0a428bd298", 2022-11-18T20:56:55.1741568Z │ "model_name": "mnist", 2022-11-18T20:56:55.1742030Z │ "model_version": "1.0", 2022-11-18T20:56:55.1742468Z │ "outputs": [ 2022-11-18T20:56:55.1742878Z │ { 2022-11-18T20:56:55.1743300Z │ "name": "predict", 2022-11-18T20:56:55.1743739Z │ "shape": [], 2022-11-18T20:56:55.1744210Z │ "datatype": "INT32", 2022-11-18T20:56:55.1744761Z │ "data": [ 2022-11-18T20:56:55.1745170Z │ 1 2022-11-18T20:56:55.1745573Z │ ] 2022-11-18T20:56:55.1745969Z │ } 2022-11-18T20:56:55.1746346Z │ ] 2022-11-18T20:56:55.1746742Z │ } 2022-11-18T20:56:55.1747109Z └ 2022-11-18T20:56:55.1747722Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:55.1748429Z 7ms 736µs (cache) (cache) (cache) 36ms 2ms 70µs 47ms 2022-11-18T20:56:55.1748779Z 2022-11-18T20:56:55.1912483Z √ Status code is 200 2022-11-18T20:56:55.2131619Z 2022-11-18T20:56:55.2132582Z → HTTPS - Explanations - Mnist_KF 2022-11-18T20:56:55.3139085Z POST https://localhost:8443/v2/models/mnist/explain 2022-11-18T20:56:55.3140371Z 200 OK ★ 61ms time ★ 7.04kB↑ 14.06kB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:56:55.3140948Z ┌ ↑ file ★ 6.77kB 2022-11-18T20:56:55.3141464Z │ (showing 2.05kB/6.77kB) 2022-11-18T20:56:55.3141919Z └ 2022-11-18T20:56:55.3142435Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 13.83kB 2022-11-18T20:56:55.3142900Z │ { 2022-11-18T20:56:55.3143449Z │ "id": "d3b15cad-50a2-4eaf-80ce-8b0a428bd298", 2022-11-18T20:56:55.3143979Z │ "model_name": "mnist", 2022-11-18T20:56:55.3144451Z │ "model_version": "1.0", 2022-11-18T20:56:55.3144911Z │ "outputs": [ 2022-11-18T20:56:55.3145341Z │ { 2022-11-18T20:56:55.3145784Z │ "name": "explain", 2022-11-18T20:56:55.3146243Z │ "shape": [ 2022-11-18T20:56:55.3146679Z │ 1, 2022-11-18T20:56:55.3147107Z │ 28, 2022-11-18T20:56:55.3147522Z │ 28 2022-11-18T20:56:55.3147944Z │ ], 2022-11-18T20:56:55.3148410Z │ "datatype": "FP64", 2022-11-18T20:56:55.3148851Z │ "data": [ 2022-11-18T20:56:55.3149291Z │ -0.0, 2022-11-18T20:56:55.3149724Z │ -0.0, 2022-11-18T20:56:55.3150154Z │ -0.0, 2022-11-18T20:56:55.3150582Z │ -0.0, 2022-11-18T20:56:55.3151010Z │ -0.0, 2022-11-18T20:56:55.3151434Z │ -0.0, 2022-11-18T20:56:55.3151842Z │ -0.0, 2022-11-18T20:56:55.3152268Z │ -0.0, 2022-11-18T20:56:55.3152910Z │ -0.0, 2022-11-18T20:56:55.3153350Z │ -0.0, 2022-11-18T20:56:55.3153763Z │ -0.0, 2022-11-18T20:56:55.3154187Z │ -0.0, 2022-11-18T20:56:55.3154664Z │ -0.0, 2022-11-18T20:56:55.3155103Z │ -0.0, 2022-11-18T20:56:55.3155527Z │ -0.0, 2022-11-18T20:56:55.3155937Z │ -0.0, 2022-11-18T20:56:55.3156359Z │ 0.0, 2022-11-18T20:56:55.3156786Z │ -0.0, 2022-11-18T20:56:55.3157188Z │ -0.0, 2022-11-18T20:56:55.3157618Z │ 0.0, 2022-11-18T20:56:55.3158039Z │ -0.0, 2022-11-18T20:56:55.3158480Z │ 0.0, 2022-11-18T20:56:55.3158905Z │ -0.0, 2022-11-18T20:56:55.3159327Z │ -0.0, 2022-11-18T20:56:55.3159872Z │ -0.0, 2022-11-18T20:56:55.3160284Z │ -0.0, 2022-11-18T20:56:55.3160710Z │ -0.0, 2022-11-18T20:56:55.3161130Z │ 0.0, 2022-11-18T20:56:55.3161553Z │ -0.0, 2022-11-18T20:56:55.3161972Z │ -0.0, 2022-11-18T20:56:55.3162395Z │ -0.0, 2022-11-18T20:56:55.3162818Z │ -0.0, 2022-11-18T20:56:55.3163227Z │ -0.0, 2022-11-18T20:56:55.3163649Z │ -0.0, 2022-11-18T20:56:55.3164074Z │ -0.0, 2022-11-18T20:56:55.3164493Z │ 0.0, 2022-11-18T20:56:55.3164906Z │ -0.0, 2022-11-18T20:56:55.3165330Z │ 0.0, 2022-11-18T20:56:55.3165752Z │ -0.0, 2022-11-18T20:56:55.3166156Z │ -0.0, 2022-11-18T20:56:55.3166582Z │ -0.0, 2022-11-18T20:56:55.3167006Z │ 0.0, 2022-11-18T20:56:55.3167426Z │ 0.0, 2022-11-18T20:56:55.3167864Z │ 0.0, 2022-11-18T20:56:55.3168282Z │ 0.0, 2022-11-18T20:56:55.3168706Z │ 0.0, 2022-11-18T20:56:55.3169133Z │ 0.0, 2022-11-18T20:56:55.3169554Z │ 0.0, 2022-11-18T20:56:55.3169965Z │ 0.0, 2022-11-18T20:56:55.3170385Z │ 0.0, 2022-11-18T20:56:55.3170808Z │ -0.0, 2022-11-18T20:56:55.3171226Z │ -0.0, 2022-11-18T20:56:55.3171652Z │ 0.0, 2022-11-18T20:56:55.3172074Z │ 0.0, 2022-11-18T20:56:55.3172498Z │ -0.0, 2022-11-18T20:56:55.3172910Z │ 0.0, 2022-11-18T20:56:55.3173335Z │ -0.0, 2022-11-18T20:56:55.3173761Z │ -0.0, 2022-11-18T20:56:55.3174172Z │ -0.0, 2022-11-18T20:56:55.3174596Z │ -0.0, 2022-11-18T20:56:55.3175019Z │ -0.0, 2022-11-18T20:56:55.3175440Z │ 0.0, 2022-11-18T20:56:55.3175848Z │ 0.0, 2022-11-18T20:56:55.3176268Z │ 0.0, 2022-11-18T20:56:55.3176687Z │ 0.0, 2022-11-18T20:56:55.3177093Z │ 0.0, 2022-11-18T20:56:55.3177516Z │ 0.0, 2022-11-18T20:56:55.3177995Z │ 0.0, 2022-11-18T20:56:55.3178422Z │ 0.0, 2022-11-18T20:56:55.3178832Z │ 0.0, 2022-11-18T20:56:55.3179252Z │ -0.0, 2022-11-18T20:56:55.3179677Z │ -0.0, 2022-11-18T20:56:55.3180097Z │ -0.0, 2022-11-18T20:56:55.3180522Z │ 0.0, 2022-11-18T20:56:55.3180978Z │ -0.0, 2022-11-18T20:56:55.3181385Z │ 0.0, 2022-11-18T20:56:55.3181811Z │ 0.0, 2022-11-18T20:56:55.3182231Z │ 0.0, 2022-11-18T20:56:55.3182654Z │ -0.0, 2022-11-18T20:56:55.3183068Z │ -0.0, 2022-11-18T20:56:55.3183493Z │ -0.0, 2022-11-18T20:56:55.3183926Z │ 0.0, 2022-11-18T20:56:55.3184333Z │ -0.0, 2022-11-18T20:56:55.3184761Z │ 0.0, 2022-11-18T20:56:55.3185189Z │ -0.0, 2022-11-18T20:56:55.3185632Z │ -0.0, 2022-11-18T20:56:55.3186044Z │ -0.0, 2022-11-18T20:56:55.3186465Z │ -0.0, 2022-11-18T20:56:55.3186891Z │ 0.0, 2022-11-18T20:56:55.3187309Z │ 0.0, 2022-11-18T20:56:55.3187734Z │ 0.0, 2022-11-18T20:56:55.3188159Z │ 0.0, 2022-11-18T20:56:55.3188578Z │ 0.0, 2022-11-18T20:56:55.3188986Z │ 0.0, 2022-11-18T20:56:55.3190295Z │ 0.0, 2022-11-18T20:56:55.3190789Z │ -0.0, 2022-11-18T20:56:55.3191204Z │ -0.0, 2022-11-18T20:56:55.3191629Z │ 0.0, 2022-11-18T20:56:55.3192051Z │ 0.0, 2022-11-18T20:56:55.3192479Z │ -0.0, 2022-11-18T20:56:55.3192897Z │ -0.0, 2022-11-18T20:56:55.3193322Z │ -0.0, 2022-11-18T20:56:55.3193750Z │ -0.0, 2022-11-18T20:56:55.3194159Z │ -0.0, 2022-11-18T20:56:55.3194598Z │ 0.0, 2022-11-18T20:56:55.3195064Z │ 0.0, 2022-11-18T20:56:55.3195497Z │ -0.0, 2022-11-18T20:56:55.3195922Z │ -0.0, 2022-11-18T20:56:55.3196336Z │ -0.0, 2022-11-18T20:56:55.3196762Z │ 0.0, 2022-11-18T20:56:55.3197303Z │ 0.0, 2022-11-18T20:56:55.3197723Z │ 0.0, 2022-11-18T20:56:55.3198138Z │ -0.0, 2022-11-18T20:56:55.3198563Z │ -0.0, 2022-11-18T20:56:55.3198990Z │ -0.0, 2022-11-18T20:56:55.3199396Z │ -0.0, 2022-11-18T20:56:55.3199834Z │ 0.0, 2022-11-18T20:56:55.3200256Z │ 0.0, 2022-11-18T20:56:55.3200681Z │ 0.0, 2022-11-18T20:56:55.3201088Z │ 0.0, 2022-11-18T20:56:55.3201509Z │ 0.0, 2022-11-18T20:56:55.3201929Z │ 0.0, 2022-11-18T20:56:55.3202335Z │ -0.0, 2022-11-18T20:56:55.3202811Z │ -0.004054752791506105, 2022-11-18T20:56:55.3203296Z │ -0.00022612876928139407, 2022-11-18T20:56:55.3203773Z │ -0.0001273412993359035, 2022-11-18T20:56:55.3204229Z │ 0.005648369033259622, 2022-11-18T20:56:55.3204665Z │ 2022-11-18T20:56:55.3205136Z │ (showing 2.05kB/13.83kB) 2022-11-18T20:56:55.3205542Z └ 2022-11-18T20:56:55.3206199Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:55.3206946Z 3ms 365µs (cache) (cache) (cache) 58ms 2ms 169µs 64ms 2022-11-18T20:56:55.3207332Z 2022-11-18T20:56:55.3453222Z √ Status code is 200 2022-11-18T20:56:55.3532990Z 2022-11-18T20:56:55.3533669Z → HTTPS UnRegister Model Mnist 2022-11-18T20:56:55.8275549Z DELETE https://localhost:8444/models/mnist 2022-11-18T20:56:55.8276568Z 200 OK ★ 435ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:56:55.8277158Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:56:55.8277555Z │ { 2022-11-18T20:56:55.8277980Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:56:55.8278320Z │ } 2022-11-18T20:56:55.8292360Z └ 2022-11-18T20:56:55.8292904Z prepare wait dns-lookup tcp-handshake ssl-handshake transfer-start download process total 2022-11-18T20:56:55.8293602Z 1ms 406µs (cache) (cache) (cache) 432ms 1ms 86µs 436ms 2022-11-18T20:56:55.8293900Z 2022-11-18T20:56:55.8424727Z √ Successful DELETE request 2022-11-18T20:56:56.0545768Z 2022-11-18T20:56:56.0546731Z ┌─────────────────────────┬─────────────────────┬────────────────────┐ 2022-11-18T20:56:56.0580685Z │ │ executed │ failed │ 2022-11-18T20:56:56.0582014Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:56.0582500Z │ iterations │ 1 │ 0 │ 2022-11-18T20:56:56.0583018Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:56.0583488Z │ requests │ 10 │ 0 │ 2022-11-18T20:56:56.0583981Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:56.0584496Z │ test-scripts │ 10 │ 0 │ 2022-11-18T20:56:56.0585033Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:56.0585494Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:56:56.0586233Z ├─────────────────────────┼─────────────────────┼────────────────────┤ 2022-11-18T20:56:56.0586685Z │ assertions │ 10 │ 0 │ 2022-11-18T20:56:56.0587171Z ├─────────────────────────┴─────────────────────┴────────────────────┤ 2022-11-18T20:56:56.0587637Z │ total run duration: 8.8s │ 2022-11-18T20:56:56.0588123Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:56.0588590Z │ total data received: 99.33kB (approx) │ 2022-11-18T20:56:56.0589084Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:56.0589559Z │ average response time: 791ms [min: 6ms, max: 4.6s, s.d.: 1466ms] │ 2022-11-18T20:56:56.0590162Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:56.0590646Z │ average DNS lookup time: 3ms [min: 248µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:56:56.0591156Z ├────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:56:56.0591626Z │ average first byte time: 763ms [min: 3ms, max: 4.6s, s.d.: 1477ms] │ 2022-11-18T20:56:56.0592122Z └────────────────────────────────────────────────────────────────────┘ 2022-11-18T20:56:56.1727778Z TorchServe has stopped. 2022-11-18T20:57:17.6955463Z newman 2022-11-18T20:57:17.6955858Z 2022-11-18T20:57:17.6957811Z explanation_api_test_collection 2022-11-18T20:57:17.6994336Z 2022-11-18T20:57:17.6995108Z → Model Zoo - Register Model 2022-11-18T20:57:20.2858330Z POST http://localhost:8081/models?url=mnist.mar&model_name=mnist&initial_workers=1&synchronous=true 2022-11-18T20:57:20.2860096Z 200 OK ★ 2.4s time ★ 318B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:20.2864333Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:57:20.2865942Z │ { 2022-11-18T20:57:20.2866461Z │ "status": "Model \"mnist\" Version: 1.0 registered w 2022-11-18T20:57:20.2867037Z │ ith 1 initial workers" 2022-11-18T20:57:20.2867706Z │ } 2022-11-18T20:57:20.2942978Z └ 2022-11-18T20:57:20.2943564Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:20.2944251Z 33ms 7ms 9ms 525µs 2.4s 8ms 582µs 2.5s 2022-11-18T20:57:20.2944623Z 2022-11-18T20:57:20.3203075Z √ Successful POST request 2022-11-18T20:57:20.3566065Z 2022-11-18T20:57:20.3566994Z → Model Zoo - Explanations Model 2022-11-18T20:57:20.4598196Z POST http://localhost:8080/explanations/mnist 2022-11-18T20:57:20.4599309Z 200 OK ★ 59ms time ★ 538B↑ 23.18kB↓ size ★ 8↑ 6↓ headers ★ 0 cookies 2022-11-18T20:57:20.4599907Z ┌ ↑ file ★ 272B 2022-11-18T20:57:20.4601139Z │ 2022-11-18T20:57:20.4601791Z └ 2022-11-18T20:57:20.4602841Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 22.95kB 2022-11-18T20:57:20.4603379Z │ [ 2022-11-18T20:57:20.4603794Z │ [ 2022-11-18T20:57:20.4604204Z │ [ 2022-11-18T20:57:20.4604687Z │ -0.0003991945379418765, 2022-11-18T20:57:20.4605136Z │ -0.00019002195787347256, 2022-11-18T20:57:20.4605598Z │ -0.0008597192658981042, 2022-11-18T20:57:20.4606041Z │ -0.00032936724458633936, 2022-11-18T20:57:20.4606475Z │ -0.0009114925551104761, 2022-11-18T20:57:20.4606915Z │ -0.00017816116824196927, 2022-11-18T20:57:20.4607362Z │ -0.0005801030055923073, 2022-11-18T20:57:20.4607806Z │ -5.752922558253175e-05, 2022-11-18T20:57:20.4608291Z │ -0.0001303639565907849, 2022-11-18T20:57:20.4608725Z │ -3.622335241487745e-05, 2022-11-18T20:57:20.4609156Z │ 1.2628187395369476e-05, 2022-11-18T20:57:20.4609570Z │ -3.151632704588537e-05, 2022-11-18T20:57:20.4610017Z │ -7.058355451639471e-05, 2022-11-18T20:57:20.4610445Z │ -6.872769706645689e-05, 2022-11-18T20:57:20.4610876Z │ -7.877426820749838e-05, 2022-11-18T20:57:20.4611300Z │ -0.00013302446908697283, 2022-11-18T20:57:20.4611751Z │ -3.925595776674495e-05, 2022-11-18T20:57:20.4612182Z │ -0.0002292955807398322, 2022-11-18T20:57:20.4612617Z │ 1.7131077681401936e-05, 2022-11-18T20:57:20.4613035Z │ -0.0003846353431148537, 2022-11-18T20:57:20.4613463Z │ -0.0005095515504758379, 2022-11-18T20:57:20.4613889Z │ -0.0003988011163787526, 2022-11-18T20:57:20.4614302Z │ -0.0008459619540549222, 2022-11-18T20:57:20.4614731Z │ -0.0005034791464513466, 2022-11-18T20:57:20.4615188Z │ -0.0006029827227030535, 2022-11-18T20:57:20.4615619Z │ -0.00017686120640438162, 2022-11-18T20:57:20.4616041Z │ -0.00015337619414885925, 2022-11-18T20:57:20.4616443Z │ -0.0 2022-11-18T20:57:20.4616838Z │ ], 2022-11-18T20:57:20.4617217Z │ [ 2022-11-18T20:57:20.4617637Z │ -0.0004593671523672321, 2022-11-18T20:57:20.4618071Z │ -0.000697201997943393, 2022-11-18T20:57:20.4618509Z │ -0.0006564255303668798, 2022-11-18T20:57:20.4618936Z │ -0.0006832954233361048, 2022-11-18T20:57:20.4619366Z │ -0.0004380070205926574, 2022-11-18T20:57:20.4619792Z │ -0.0006188232836199017, 2022-11-18T20:57:20.4620294Z │ 8.71765494539121e-06, 2022-11-18T20:57:20.4620718Z │ -0.0003930583143097941, 2022-11-18T20:57:20.4621155Z │ -0.00010256503150031404, 2022-11-18T20:57:20.4621627Z │ -0.00018236528318850878, 2022-11-18T20:57:20.4622070Z │ -0.0005491941239779937, 2022-11-18T20:57:20.4622501Z │ 4.519852496218651e-05, 2022-11-18T20:57:20.4622936Z │ -0.00043977567866099583, 2022-11-18T20:57:20.4623371Z │ -0.0003280011585836224, 2022-11-18T20:57:20.4623793Z │ -0.00038462530266255375, 2022-11-18T20:57:20.4624229Z │ -0.0007280317118263079, 2022-11-18T20:57:20.4624660Z │ -0.0003807701305064843, 2022-11-18T20:57:20.4625074Z │ -0.0006797485179225832, 2022-11-18T20:57:20.4625507Z │ -0.000941058793677313, 2022-11-18T20:57:20.4626102Z │ -0.0005321377388563904, 2022-11-18T20:57:20.4626541Z │ -0.0010959785428022665, 2022-11-18T20:57:20.4626956Z │ -0.0009239371270925911, 2022-11-18T20:57:20.4627387Z │ -0.0007882727790940376, 2022-11-18T20:57:20.4627818Z │ -0.0007635523394724341, 2022-11-18T20:57:20.4628234Z │ -0.0003335172527655803, 2022-11-18T20:57:20.4628663Z │ 5.10122769435689e-05, 2022-11-18T20:57:20.4629097Z │ 7.699817726334496e-05, 2022-11-18T20:57:20.4629509Z │ -0.0 2022-11-18T20:57:20.4629885Z │ ], 2022-11-18T20:57:20.4630787Z │ [ 2022-11-18T20:57:20.4631133Z │ -0.001316505823457328, 2022-11-18T20:57:20.4631400Z │ -0.0009519237452816437, 2022-11-18T20:57:20.4631662Z │ -0.0021000623776788146, 2022-11-18T20:57:20.4632152Z │ -0.0007986434054598246, 2022-11-18T20:57:20.4632409Z │ -0.0015434155244661897, 2022-11-18T20:57:20.4632652Z │ -0.0008677096095781197, 2022-11-18T20:57:20.4632911Z │ -0.0006676293963547219, 2022-11-18T20:57:20.4633164Z │ -0.0010158373777369892, 2022-11-18T20:57:20.4633398Z │ -0.0006892678666761079, 2022-11-18T20:57:20.4633643Z │ -0.0010906412195348645, 2022-11-18T20:57:20.4633899Z │ -0.000918481924629697, 2022-11-18T20:57:20.4634134Z │ -0.001018668612639547, 2022-11-18T20:57:20.4634358Z │ -0. 2022-11-18T20:57:20.4634622Z │ (showing 2.05kB/22.95kB) 2022-11-18T20:57:20.4634840Z └ 2022-11-18T20:57:20.4635256Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:20.4635740Z 7ms 1ms 348µs 1ms 52ms 2ms 96µs 66ms 2022-11-18T20:57:20.4635911Z 2022-11-18T20:57:20.4979558Z √ Successful POST request 2022-11-18T20:57:20.5020075Z √ Test expected JSON response 2022-11-18T20:57:20.5311518Z 2022-11-18T20:57:20.5312291Z → Model Zoo - Unregister model 2022-11-18T20:57:20.6639843Z DELETE http://localhost:8081/models/mnist 2022-11-18T20:57:20.6640865Z 200 OK ★ 83ms time ★ 241B↑ 313B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:20.6641667Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 47B 2022-11-18T20:57:20.6642108Z │ { 2022-11-18T20:57:20.6642522Z │ "status": "Model \"mnist\" unregistered" 2022-11-18T20:57:20.6642889Z │ } 2022-11-18T20:57:20.6658006Z └ 2022-11-18T20:57:20.6658616Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:20.6659273Z 2ms 558µs (cache) (cache) 80ms 2ms 95µs 84ms 2022-11-18T20:57:20.6659580Z 2022-11-18T20:57:20.6892717Z √ Successful DELETE request 2022-11-18T20:57:20.7123976Z 2022-11-18T20:57:20.7124973Z → Model Zoo - Model Metrics 2022-11-18T20:57:20.8074430Z GET http://localhost:8082/metrics 2022-11-18T20:57:20.8075521Z 200 OK ★ 43ms time ★ 233B↑ 1.04kB↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:20.8076135Z ┌ ↓ text/plain ★ text ★ plain ★ utf8 ★ 751B 2022-11-18T20:57:20.8076666Z │ # HELP ts_inference_requests_total Total number of inf 2022-11-18T20:57:20.8077064Z │ erence requests. 2022-11-18T20:57:20.8077409Z │ # TYPE ts_inference_requests_total counter 2022-11-18T20:57:20.8077788Z │ ts_inference_requests_total{uuid="40fe2428-9b69-4bd8-a 2022-11-18T20:57:20.8078192Z │ 750-74a38b645715",model_name="mnist",model_version="de 2022-11-18T20:57:20.8078459Z │ fault",} 1.0 2022-11-18T20:57:20.8078791Z │ # HELP ts_inference_latency_microseconds Cumulative in 2022-11-18T20:57:20.8079133Z │ ference duration in microseconds 2022-11-18T20:57:20.8079468Z │ # TYPE ts_inference_latency_microseconds counter 2022-11-18T20:57:20.8079849Z │ ts_inference_latency_microseconds{uuid="40fe2428-9b69- 2022-11-18T20:57:20.8080243Z │ 4bd8-a750-74a38b645715",model_name="mnist",model_versi 2022-11-18T20:57:20.8080530Z │ on="default",} 34212.8 2022-11-18T20:57:20.8081094Z │ # HELP ts_queue_latency_microseconds Cumulative queue 2022-11-18T20:57:20.8081435Z │ duration in microseconds 2022-11-18T20:57:20.8081761Z │ # TYPE ts_queue_latency_microseconds counter 2022-11-18T20:57:20.8082117Z │ ts_queue_latency_microseconds{uuid="40fe2428-9b69-4bd8 2022-11-18T20:57:20.8082488Z │ -a750-74a38b645715",model_name="mnist",model_version=" 2022-11-18T20:57:20.8082772Z │ default",} 256.3 2022-11-18T20:57:20.8082966Z └ 2022-11-18T20:57:20.8083368Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:20.8083838Z 2ms 965µs 241µs 261µs 38ms 2ms 68µs 44ms 2022-11-18T20:57:20.8084008Z 2022-11-18T20:57:20.8286467Z √ Successful GET request 2022-11-18T20:57:21.0145949Z 2022-11-18T20:57:21.0146899Z ┌─────────────────────────┬─────────────────────┬─────────────────────┐ 2022-11-18T20:57:21.0158008Z │ │ executed │ failed │ 2022-11-18T20:57:21.0159262Z ├─────────────────────────┼─────────────────────┼─────────────────────┤ 2022-11-18T20:57:21.0159787Z │ iterations │ 1 │ 0 │ 2022-11-18T20:57:21.0160288Z ├─────────────────────────┼─────────────────────┼─────────────────────┤ 2022-11-18T20:57:21.0160752Z │ requests │ 4 │ 0 │ 2022-11-18T20:57:21.0161244Z ├─────────────────────────┼─────────────────────┼─────────────────────┤ 2022-11-18T20:57:21.0161717Z │ test-scripts │ 4 │ 0 │ 2022-11-18T20:57:21.0162196Z ├─────────────────────────┼─────────────────────┼─────────────────────┤ 2022-11-18T20:57:21.0162667Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:57:21.0163192Z ├─────────────────────────┼─────────────────────┼─────────────────────┤ 2022-11-18T20:57:21.0163647Z │ assertions │ 5 │ 0 │ 2022-11-18T20:57:21.0164133Z ├─────────────────────────┴─────────────────────┴─────────────────────┤ 2022-11-18T20:57:21.0164598Z │ total run duration: 3.1s │ 2022-11-18T20:57:21.0165085Z ├─────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:57:21.0165556Z │ total data received: 23.82kB (approx) │ 2022-11-18T20:57:21.0166040Z ├─────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:57:21.0166522Z │ average response time: 669ms [min: 43ms, max: 2.4s, s.d.: 1052ms] │ 2022-11-18T20:57:21.0167025Z ├─────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:57:21.0167513Z │ average DNS lookup time: 3ms [min: 241µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:57:21.0168025Z ├─────────────────────────────────────────────────────────────────────┤ 2022-11-18T20:57:21.0168500Z │ average first byte time: 658ms [min: 38ms, max: 2.4s, s.d.: 1042ms] │ 2022-11-18T20:57:21.0169287Z └─────────────────────────────────────────────────────────────────────┘ 2022-11-18T20:57:21.1354120Z TorchServe has stopped. 2022-11-18T20:57:31.1624959Z ## Successfully stopped TorchServe 2022-11-18T20:57:31.1643146Z ## Starting gen_mar: model_store 2022-11-18T20:57:31.1707409Z 2022-11-18T20:57:31.1721841Z ## Create symlink for mar files 2022-11-18T20:57:31.1725206Z 2022-11-18T20:57:31.1728361Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:57:31.1729191Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:57:31.1730157Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:57:31.1731032Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:57:31.1731629Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:57:31.1732240Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:57:31.1732850Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:57:31.1734575Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:57:31.1735416Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:57:31.1736055Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:57:31.1736705Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:57:31.1737167Z ## Starting TorchServe 2022-11-18T20:57:31.1737548Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:57:31.1738352Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs >> ts_console.log 2022-11-18T20:57:31.1738864Z ## Successfully started TorchServe 2022-11-18T20:57:31.1739214Z ## Stopping TorchServe 2022-11-18T20:57:31.1739752Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:57:31.1740180Z ## Successfully stopped TorchServe 2022-11-18T20:57:31.1740541Z ## Starting gen_mar: model_store 2022-11-18T20:57:31.1740795Z 2022-11-18T20:57:31.1740999Z ## Create symlink for mar files 2022-11-18T20:57:31.1741245Z 2022-11-18T20:57:31.1741577Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:57:31.1742174Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:57:31.1743025Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:57:31.1743622Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:57:31.1744214Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:57:31.1744847Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:57:31.1745591Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:57:31.1746358Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:57:31.1747087Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:57:31.1747729Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:57:31.1748378Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:57:31.1748801Z ## Starting TorchServe 2022-11-18T20:57:31.1749168Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:57:31.1749904Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs --ts-config=config.properties >> ts_console.log 2022-11-18T20:57:31.1750670Z ## Successfully started TorchServe 2022-11-18T20:57:31.1750994Z ## Stopping TorchServe 2022-11-18T20:57:31.1751525Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:57:31.1751893Z ## Successfully stopped TorchServe 2022-11-18T20:57:31.1752163Z ## Starting gen_mar: model_store 2022-11-18T20:57:31.1752319Z 2022-11-18T20:57:31.1752408Z ## Create symlink for mar files 2022-11-18T20:57:31.1752557Z 2022-11-18T20:57:31.1752789Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:57:31.1753278Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:57:31.1753883Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:57:31.1754339Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:57:31.1754822Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:57:31.1755324Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:57:31.1755818Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:57:31.1756424Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:57:31.1756948Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:57:31.1757484Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:57:31.1758134Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:57:31.1758475Z ## Starting TorchServe 2022-11-18T20:57:31.1758749Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:57:31.1759427Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs --ts-config=resources\config.properties >> ts_console.log 2022-11-18T20:57:31.1759992Z ## Successfully started TorchServe 2022-11-18T20:57:31.1760226Z ## Stopping TorchServe 2022-11-18T20:57:31.1760650Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:57:31.1760992Z ## Successfully stopped TorchServe 2022-11-18T20:57:31.1761231Z ## Starting gen_mar: model_store 2022-11-18T20:57:31.1761384Z 2022-11-18T20:57:31.1761487Z ## Create symlink for mar files 2022-11-18T20:57:31.1761634Z 2022-11-18T20:57:31.1761864Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:57:31.1762355Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:57:31.1762997Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:57:31.1763474Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:57:31.1763958Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:57:31.1764456Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:57:31.1764922Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:57:31.1765546Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:57:31.1766075Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:57:31.1766617Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:57:31.1767130Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:57:31.1767478Z ## Starting TorchServe 2022-11-18T20:57:31.1767745Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:57:31.1768366Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs --ts-config=config.properties >> ts_console.log 2022-11-18T20:57:31.1768793Z ## Successfully started TorchServe 2022-11-18T20:57:31.1769041Z ## Stopping TorchServe 2022-11-18T20:57:31.1769455Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:57:31.1769777Z ## Successfully stopped TorchServe 2022-11-18T20:57:31.1770036Z ## Starting gen_mar: model_store 2022-11-18T20:57:31.1770189Z 2022-11-18T20:57:31.1770297Z ## Create symlink for mar files 2022-11-18T20:57:31.1770448Z 2022-11-18T20:57:31.1770678Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:57:31.1771148Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:57:31.1771744Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:57:31.1772223Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:57:31.1772709Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:57:31.1773191Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:57:31.1773680Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:57:31.1774307Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:57:42.6114745Z newman 2022-11-18T20:57:42.6115556Z 2022-11-18T20:57:42.6116692Z management_api_collection 2022-11-18T20:57:42.6122122Z 2022-11-18T20:57:42.6122405Z Iteration 1/11 2022-11-18T20:57:42.6147171Z 2022-11-18T20:57:42.6147927Z → workflow management request 2022-11-18T20:57:49.8591892Z POST http://localhost:8081/workflows?url=https://torchserve.s3.amazonaws.com/war_files/densenet_wf.war 2022-11-18T20:57:49.8593754Z 200 OK ★ 7.1s time ★ 321B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:49.8597943Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:57:49.8599538Z │ { 2022-11-18T20:57:49.8599991Z │ "status": "Workflow densenet has been registered and 2022-11-18T20:57:49.8600626Z │ scaled successfully." 2022-11-18T20:57:49.8601295Z │ } 2022-11-18T20:57:49.8676853Z └ 2022-11-18T20:57:49.8677512Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:49.8678219Z 33ms 7ms 9ms 567µs 7.1s 8ms 565µs 7.1s 2022-11-18T20:57:49.8678595Z 2022-11-18T20:57:49.8918977Z √ Successful request 2022-11-18T20:57:49.9104861Z 2022-11-18T20:57:49.9105418Z Iteration 2/11 2022-11-18T20:57:49.9262351Z 2022-11-18T20:57:49.9263164Z → workflow management request 2022-11-18T20:57:49.9848300Z GET http://localhost:8081/workflows 2022-11-18T20:57:49.9849848Z 200 OK ★ 10ms time ★ 235B↑ 423B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:49.9850419Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 156B 2022-11-18T20:57:49.9851756Z │ { 2022-11-18T20:57:49.9852089Z │ "workflows": [ 2022-11-18T20:57:49.9852544Z │ { 2022-11-18T20:57:49.9853036Z │ "workflowName": "densenet", 2022-11-18T20:57:49.9853630Z │ "workflowUrl": "https://torchserve.s3.amazonaws. 2022-11-18T20:57:49.9854214Z │ com/war_files/densenet_wf.war" 2022-11-18T20:57:49.9854644Z │ } 2022-11-18T20:57:49.9855057Z │ ] 2022-11-18T20:57:49.9855461Z │ } 2022-11-18T20:57:49.9872709Z └ 2022-11-18T20:57:49.9873299Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:49.9873995Z 1ms 878µs (cache) (cache) 6ms 2ms 93µs 10ms 2022-11-18T20:57:49.9874370Z 2022-11-18T20:57:50.0094969Z √ Successful request 2022-11-18T20:57:50.0197545Z 2022-11-18T20:57:50.0197904Z Iteration 3/11 2022-11-18T20:57:50.0354688Z 2022-11-18T20:57:50.0355341Z → workflow management request 2022-11-18T20:57:50.0914480Z GET http://localhost:8081/workflows/densenet 2022-11-18T20:57:50.0915746Z 200 OK ★ 7ms time ★ 244B↑ 559B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:50.0916460Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 292B 2022-11-18T20:57:50.0916986Z │ [ 2022-11-18T20:57:50.0917344Z │ { 2022-11-18T20:57:50.0917802Z │ "workflowName": "densenet", 2022-11-18T20:57:50.0918387Z │ "workflowUrl": "https://torchserve.s3.amazonaws.co 2022-11-18T20:57:50.0918950Z │ m/war_files/densenet_wf.war", 2022-11-18T20:57:50.0919387Z │ "minWorkers": 1, 2022-11-18T20:57:50.0919821Z │ "maxWorkers": 1, 2022-11-18T20:57:50.0920248Z │ "batchSize": 1, 2022-11-18T20:57:50.0920677Z │ "maxBatchDelay": 50, 2022-11-18T20:57:50.0921225Z │ "workflowDag": "{pre_processing=[densenet], densen 2022-11-18T20:57:50.0921725Z │ et=[post_processing]}" 2022-11-18T20:57:50.0922131Z │ } 2022-11-18T20:57:50.0922477Z │ ] 2022-11-18T20:57:50.0934339Z └ 2022-11-18T20:57:50.0934922Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:50.0935599Z 1ms 517µs (cache) (cache) 3ms 1ms 80µs 7ms 2022-11-18T20:57:50.0935936Z 2022-11-18T20:57:50.1030823Z √ Successful request 2022-11-18T20:57:50.1140305Z 2022-11-18T20:57:50.1140672Z Iteration 4/11 2022-11-18T20:57:50.1300520Z 2022-11-18T20:57:50.1301347Z → workflow management request 2022-11-18T20:57:50.1877784Z POST http://localhost:8081/workflows?url=https://torchserve.s3.amazonaws.com/war_files/densenet_wf.war [errored] 2022-11-18T20:57:50.1878618Z read ECONNRESET 2022-11-18T20:57:50.2023951Z 2. Successful request 2022-11-18T20:57:50.2078866Z 2022-11-18T20:57:50.2079145Z Iteration 5/11 2022-11-18T20:57:50.2236472Z 2022-11-18T20:57:50.2237238Z → workflow management request 2022-11-18T20:57:50.3465139Z DELETE http://localhost:8081/workflows/densenet 2022-11-18T20:57:50.3628694Z 200 OK ★ 52ms time ★ 247B↑ 319B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:50.3652610Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 53B 2022-11-18T20:57:50.3677875Z │ { 2022-11-18T20:57:50.3678455Z │ "status": "Workflow \"densenet\" unregistered" 2022-11-18T20:57:50.3679080Z │ } 2022-11-18T20:57:50.3679374Z └ 2022-11-18T20:57:50.3679920Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:50.3680533Z 1ms 1ms 433µs 336µs 47ms 2ms 78µs 53ms 2022-11-18T20:57:50.3680819Z 2022-11-18T20:57:50.3681071Z √ Successful request 2022-11-18T20:57:50.3806441Z 2022-11-18T20:57:50.3806750Z Iteration 6/11 2022-11-18T20:57:50.3938986Z 2022-11-18T20:57:50.3939529Z → workflow management request 2022-11-18T20:57:50.5118597Z POST http://localhost:8081/workflows?url=https://torchserve.s3.amazonaws.com/war_files/does_not_exist.war 2022-11-18T20:57:50.5119839Z 400 Bad Request ★ 69ms time ★ 324B↑ 441B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:50.5120441Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 170B 2022-11-18T20:57:50.5120865Z │ { 2022-11-18T20:57:50.5121232Z │ "code": 400, 2022-11-18T20:57:50.5121710Z │ "type": "DownloadArchiveException", 2022-11-18T20:57:50.5122193Z │ "message": "Failed to download archive from: https:/ 2022-11-18T20:57:50.5122708Z │ /torchserve.s3.amazonaws.com/war_files/does_not_exist. 2022-11-18T20:57:50.5123120Z │ war" 2022-11-18T20:57:50.5123445Z │ } 2022-11-18T20:57:50.5139232Z └ 2022-11-18T20:57:50.5139775Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:50.5140404Z 1ms 273µs (cache) (cache) 64ms 2ms 91µs 69ms 2022-11-18T20:57:50.5140727Z 2022-11-18T20:57:50.5193696Z √ Successful request 2022-11-18T20:57:50.5257078Z 2022-11-18T20:57:50.5257450Z Iteration 7/11 2022-11-18T20:57:50.5346402Z 2022-11-18T20:57:50.5347050Z → workflow management request 2022-11-18T20:57:50.5900399Z GET http://localhost:8081/workflows/does_not_exist [errored] 2022-11-18T20:57:50.5901037Z read ECONNRESET 2022-11-18T20:57:50.6021438Z 4. Successful request 2022-11-18T20:57:50.6126225Z 2022-11-18T20:57:50.6126724Z Iteration 8/11 2022-11-18T20:57:50.6283601Z 2022-11-18T20:57:50.6284505Z → workflow management request 2022-11-18T20:57:50.6849995Z DELETE http://localhost:8081/workflows/does_not_exist 2022-11-18T20:57:50.6851306Z 404 Not Found ★ 7ms time ★ 253B↑ 377B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:50.6852037Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 108B 2022-11-18T20:57:50.6852496Z │ { 2022-11-18T20:57:50.6852897Z │ "code": 404, 2022-11-18T20:57:50.6853406Z │ "type": "WorkflowNotFoundException", 2022-11-18T20:57:50.6853949Z │ "message": "Workflow not found: does_not_exist" 2022-11-18T20:57:50.6854389Z │ } 2022-11-18T20:57:50.6865669Z └ 2022-11-18T20:57:50.6866287Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:50.6867021Z 1ms 956µs 240µs 256µs 3ms 1ms 61µs 8ms 2022-11-18T20:57:50.6867397Z 2022-11-18T20:57:50.6953909Z √ Successful request 2022-11-18T20:57:50.7065223Z 2022-11-18T20:57:50.7065617Z Iteration 9/11 2022-11-18T20:57:50.7221510Z 2022-11-18T20:57:50.7222184Z → workflow management request 2022-11-18T20:57:50.7790283Z POST http://localhost:8081/workflows?url=malformed_url,? 2022-11-18T20:57:50.7791400Z 404 Not Found ★ 7ms time ★ 275B↑ 396B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:50.7792042Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 127B 2022-11-18T20:57:50.7792445Z │ { 2022-11-18T20:57:50.7792794Z │ "code": 404, 2022-11-18T20:57:50.7793236Z │ "type": "WorkflowNotFoundException", 2022-11-18T20:57:50.7793719Z │ "message": "Workflow not found in workflow store: ma 2022-11-18T20:57:50.7794130Z │ lformed_url,?" 2022-11-18T20:57:50.7794468Z │ } 2022-11-18T20:57:50.7806427Z └ 2022-11-18T20:57:50.7806968Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:50.7807758Z 2ms 1ms 348µs 260µs 2ms 2ms 61µs 8ms 2022-11-18T20:57:50.7808067Z 2022-11-18T20:57:50.7902378Z √ Successful request 2022-11-18T20:57:50.8000462Z 2022-11-18T20:57:50.8000780Z Iteration 10/11 2022-11-18T20:57:50.8158784Z 2022-11-18T20:57:50.8159519Z → workflow management request 2022-11-18T20:57:54.4200153Z POST http://localhost:8081/workflows?url=https://torchserve.s3.amazonaws.com/war_files/custom_python_dep.war 2022-11-18T20:57:54.4203818Z 500 Internal Server Error ★ 1633ms time ★ 327B↑ 531B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:54.4204538Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 250B 2022-11-18T20:57:54.4204943Z │ { 2022-11-18T20:57:54.4205258Z │ "code": 500, 2022-11-18T20:57:54.4205672Z │ "type": "WorkflowException", 2022-11-18T20:57:54.4206166Z │ "message": "Workflow custom_python_dep has failed to 2022-11-18T20:57:54.4206688Z │ register. Failures: [Workflow Node custom_python_dep_ 2022-11-18T20:57:54.4207217Z │ _custom_python_dep failed to register. Details: Model 2022-11-18T20:57:54.4207703Z │ not found at: custom_python_dep.mar]" 2022-11-18T20:57:54.4208066Z │ } 2022-11-18T20:57:54.4208332Z └ 2022-11-18T20:57:54.4208886Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:54.4209517Z 1ms 1ms 427µs 271µs 1628ms 1ms 76µs 1634ms 2022-11-18T20:57:54.4209783Z 2022-11-18T20:57:54.4210015Z √ Successful request 2022-11-18T20:57:54.4210244Z 2022-11-18T20:57:54.4210404Z Iteration 11/11 2022-11-18T20:57:54.4210618Z 2022-11-18T20:57:54.4210840Z → workflow management request 2022-11-18T20:57:54.4211536Z POST http://localhost:8081/workflows?url=https://torchserve.s3.amazonaws.com/war_files/loading-memory-error.war 2022-11-18T20:57:54.4212262Z 500 Internal Server Error ★ 1488ms time ★ 330B↑ 543B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:57:54.4212812Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 262B 2022-11-18T20:57:54.4213170Z │ { 2022-11-18T20:57:54.4213504Z │ "code": 500, 2022-11-18T20:57:54.4213898Z │ "type": "WorkflowException", 2022-11-18T20:57:54.4214409Z │ "message": "Workflow loading-memory-error has failed 2022-11-18T20:57:54.4214925Z │ to register. Failures: [Workflow Node loading-memory- 2022-11-18T20:57:54.4215455Z │ error__loading-memory-error failed to register. Detail 2022-11-18T20:57:54.4215987Z │ s: Model not found at: loading-memory-error.mar]" 2022-11-18T20:57:54.4216387Z │ } 2022-11-18T20:57:54.4216653Z └ 2022-11-18T20:57:54.4217200Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:57:54.4217804Z 1ms 965µs 411µs 277µs 1483ms 2ms 62µs 1488ms 2022-11-18T20:57:54.4218068Z 2022-11-18T20:57:54.4218293Z √ Successful request 2022-11-18T20:57:54.4218500Z 2022-11-18T20:57:54.4218872Z ┌─────────────────────────┬────────────────────┬────────────────────┐ 2022-11-18T20:57:54.4219358Z │ │ executed │ failed │ 2022-11-18T20:57:54.4220082Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:57:54.4220579Z │ iterations │ 11 │ 0 │ 2022-11-18T20:57:54.4221098Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:57:54.4221580Z │ requests │ 11 │ 2 │ 2022-11-18T20:57:54.4222096Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:57:54.4222586Z │ test-scripts │ 11 │ 0 │ 2022-11-18T20:57:54.4223085Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:57:54.4223580Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:57:54.4224115Z ├─────────────────────────┼────────────────────┼────────────────────┤ 2022-11-18T20:57:54.4224705Z │ assertions │ 11 │ 2 │ 2022-11-18T20:57:54.4225194Z ├─────────────────────────┴────────────────────┴────────────────────┤ 2022-11-18T20:57:54.4225696Z │ total run duration: 11.5s │ 2022-11-18T20:57:54.4226216Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:57:54.4226708Z │ total data received: 1.5kB (approx) │ 2022-11-18T20:57:54.4227211Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:57:54.4227797Z │ average response time: 1310ms [min: 7ms, max: 7.1s, s.d.: 2.4s] │ 2022-11-18T20:57:54.4228376Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:57:54.4228810Z │ average DNS lookup time: 4ms [min: 240µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:57:54.4229280Z ├───────────────────────────────────────────────────────────────────┤ 2022-11-18T20:57:54.4229735Z │ average first byte time: 1303ms [min: 2ms, max: 7.1s, s.d.: 2.4s] │ 2022-11-18T20:57:54.4230181Z └───────────────────────────────────────────────────────────────────┘ 2022-11-18T20:57:54.4237427Z 2022-11-18T20:57:54.4237795Z # failure detail 2022-11-18T20:57:54.4238133Z 2022-11-18T20:57:54.4238444Z 1. Error 2022-11-18T20:57:54.4238773Z iteration: 4 read ECONNRESET 2022-11-18T20:57:54.4239084Z at request 2022-11-18T20:57:54.4239432Z inside "workflow management request" 2022-11-18T20:57:54.4239760Z 2022-11-18T20:57:54.4240088Z 2. AssertionError Successful request 2022-11-18T20:57:54.4240443Z iteration: 4 expected undefined to be one of [ 500 ] 2022-11-18T20:57:54.4241034Z at assertion:0 in test-script 2022-11-18T20:57:54.4241402Z inside "workflow management request" 2022-11-18T20:57:54.4241714Z 2022-11-18T20:57:54.4242015Z 3. Error 2022-11-18T20:57:54.4242349Z iteration: 7 read ECONNRESET 2022-11-18T20:57:54.4242761Z at request 2022-11-18T20:57:54.4243141Z inside "workflow management request" 2022-11-18T20:57:54.4243474Z 2022-11-18T20:57:54.4243810Z 4. AssertionError Successful request 2022-11-18T20:57:54.4244185Z iteration: 7 expected undefined to be one of [ 404 ] 2022-11-18T20:57:54.4244600Z at assertion:0 in test-script 2022-11-18T20:57:54.4244960Z inside "workflow management request" 2022-11-18T20:57:54.5456946Z TorchServe has stopped. 2022-11-18T20:58:16.0201212Z newman 2022-11-18T20:58:16.0201669Z 2022-11-18T20:58:16.0203905Z workflow_inference 2022-11-18T20:58:16.0209200Z 2022-11-18T20:58:16.0209400Z Iteration 1/3 2022-11-18T20:58:16.0234652Z 2022-11-18T20:58:16.0235287Z → Workflow - Register Workflow 2022-11-18T20:58:22.9591923Z POST http://localhost:8081/workflows?url=https://torchserve.s3.amazonaws.com/war_files/densenet_wf.war 2022-11-18T20:58:22.9593867Z 200 OK ★ 6.8s time ★ 321B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:58:22.9598111Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:58:22.9599755Z │ { 2022-11-18T20:58:22.9600293Z │ "status": "Workflow densenet has been registered and 2022-11-18T20:58:22.9600854Z │ scaled successfully." 2022-11-18T20:58:22.9601264Z │ } 2022-11-18T20:58:22.9677054Z └ 2022-11-18T20:58:22.9677680Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:58:22.9678433Z 35ms 7ms 9ms 579µs 6.8s 8ms 594µs 6.8s 2022-11-18T20:58:22.9678810Z 2022-11-18T20:58:22.9927846Z √ Successful POST request 2022-11-18T20:58:23.0301999Z 2022-11-18T20:58:23.0302708Z → Workflow - Inference 2022-11-18T20:58:23.4600178Z POST http://localhost:8080/wfpredict/densenet 2022-11-18T20:58:23.4601406Z 200 OK ★ 374ms time ★ 111.24kB↑ 449B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:58:23.4604822Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:58:23.4605647Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:58:23.4606108Z └ 2022-11-18T20:58:23.4606637Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 182B 2022-11-18T20:58:23.4607108Z │ { 2022-11-18T20:58:23.4607553Z │ "tabby": 0.4666188061237335, 2022-11-18T20:58:23.4608051Z │ "tiger_cat": 0.46449077129364014, 2022-11-18T20:58:23.4608591Z │ "Egyptian_cat": 0.06614049524068832, 2022-11-18T20:58:23.4609092Z │ "lynx": 0.0012924447655677795, 2022-11-18T20:58:23.4609579Z │ "plastic_bag": 0.00022909819381311536 2022-11-18T20:58:23.4610025Z │ } 2022-11-18T20:58:23.4626109Z └ 2022-11-18T20:58:23.4626752Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:58:23.4627460Z 7ms 1ms 334µs 551µs 367ms 2ms 78µs 380ms 2022-11-18T20:58:23.4627815Z 2022-11-18T20:58:23.4788458Z √ Successful request 2022-11-18T20:58:23.4800008Z √ Test expected JSON response 2022-11-18T20:58:23.5030436Z 2022-11-18T20:58:23.5031201Z → Workflow - Unregister 2022-11-18T20:58:23.8961407Z DELETE http://localhost:8081/workflows/densenet 2022-11-18T20:58:23.8962416Z 200 OK ★ 344ms time ★ 247B↑ 319B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:58:23.8963052Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 53B 2022-11-18T20:58:23.8964208Z │ { 2022-11-18T20:58:23.8964650Z │ "status": "Workflow \"densenet\" unregistered" 2022-11-18T20:58:23.8965017Z │ } 2022-11-18T20:58:23.8978955Z └ 2022-11-18T20:58:23.8979542Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:58:23.8980189Z 1ms 660µs (cache) (cache) 340ms 1ms 94µs 344ms 2022-11-18T20:58:23.8980506Z 2022-11-18T20:58:23.9090743Z √ Successful DELETE request 2022-11-18T20:58:23.9192459Z 2022-11-18T20:58:23.9192733Z Iteration 2/3 2022-11-18T20:58:23.9352082Z 2022-11-18T20:58:23.9352732Z → Workflow - Register Workflow 2022-11-18T20:58:29.6215349Z POST http://localhost:8081/workflows?url=https://torchserve.s3.amazonaws.com/war_files/timeout-retry-test.war 2022-11-18T20:58:29.6216400Z 200 OK ★ 5.6s time ★ 328B↑ 347B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:58:29.6217108Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 81B 2022-11-18T20:58:29.6217664Z │ { 2022-11-18T20:58:29.6218199Z │ "status": "Workflow densenet has been registered and 2022-11-18T20:58:29.6218713Z │ scaled successfully." 2022-11-18T20:58:29.6219157Z │ } 2022-11-18T20:58:29.6233539Z └ 2022-11-18T20:58:29.6234149Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:58:29.6234864Z 1ms 297µs (cache) (cache) 5.6s 2ms 93µs 5.6s 2022-11-18T20:58:29.6235245Z 2022-11-18T20:58:29.6326629Z √ Successful POST request 2022-11-18T20:58:29.6589892Z 2022-11-18T20:58:29.6590672Z → Workflow - Inference 2022-11-18T20:58:41.7759371Z POST http://localhost:8080/wfpredict/densenet 2022-11-18T20:58:41.7760566Z 500 Internal Server Error ★ 12s time ★ 111.24kB↑ 383B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:58:41.7761123Z ┌ ↑ file ★ 110.97kB 2022-11-18T20:58:41.7761545Z │ (showing 2.05kB/110.97kB) 2022-11-18T20:58:41.7761925Z └ 2022-11-18T20:58:41.7762414Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 102B 2022-11-18T20:58:41.7762830Z │ { 2022-11-18T20:58:41.7763153Z │ "code": 500, 2022-11-18T20:58:41.7763585Z │ "type": "InternalServerException", 2022-11-18T20:58:41.7764089Z │ "message": "Error executing pre_processing" 2022-11-18T20:58:41.7764452Z │ } 2022-11-18T20:58:41.7789223Z └ 2022-11-18T20:58:41.7789843Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:58:41.7790522Z 2ms 306µs (cache) (cache) 12s 4ms 121µs 12s 2022-11-18T20:58:41.7790845Z 2022-11-18T20:58:41.7882917Z √ Successful request 2022-11-18T20:58:41.8113271Z 2022-11-18T20:58:41.8114047Z → Workflow - Unregister 2022-11-18T20:58:41.9802889Z DELETE http://localhost:8081/workflows/densenet 2022-11-18T20:58:41.9803906Z 200 OK ★ 119ms time ★ 247B↑ 319B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:58:41.9804521Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 53B 2022-11-18T20:58:41.9804983Z │ { 2022-11-18T20:58:41.9805416Z │ "status": "Workflow \"densenet\" unregistered" 2022-11-18T20:58:41.9805844Z │ } 2022-11-18T20:58:41.9820908Z └ 2022-11-18T20:58:41.9821643Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:58:41.9822270Z 2ms 428µs (cache) (cache) 115ms 2ms 64µs 121ms 2022-11-18T20:58:41.9822594Z 2022-11-18T20:58:41.9952708Z √ Successful DELETE request 2022-11-18T20:58:42.0051825Z 2022-11-18T20:58:42.0052193Z Iteration 3/3 2022-11-18T20:58:42.0209571Z 2022-11-18T20:58:42.0210219Z → Workflow - Register Workflow 2022-11-18T20:58:48.1278110Z POST http://localhost:8081/workflows?url=https://torchserve.s3.amazonaws.com/war_files/dog_breed_wf.war 2022-11-18T20:58:48.1279531Z 200 OK ★ 6s time ★ 322B↑ 351B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:58:48.1280240Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 85B 2022-11-18T20:58:48.1280945Z │ { 2022-11-18T20:58:48.1281502Z │ "status": "Workflow dog_breed_wf has been registered 2022-11-18T20:58:48.1282021Z │ and scaled successfully." 2022-11-18T20:58:48.1282480Z │ } 2022-11-18T20:58:48.1314890Z └ 2022-11-18T20:58:48.1315725Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:58:48.1316500Z 2ms 354µs (cache) (cache) 6s 2ms 61µs 6s 2022-11-18T20:58:48.1316891Z 2022-11-18T20:58:48.1494267Z √ Successful POST request 2022-11-18T20:58:48.1748271Z 2022-11-18T20:58:48.1749022Z → Workflow - Inference 2022-11-18T20:58:48.6611740Z POST http://localhost:8080/wfpredict/dog_breed_wf 2022-11-18T20:58:48.6612827Z 200 OK ★ 449ms time ★ 77.75kB↑ 473B↓ size ★ 8↑ 7↓ headers ★ 0 cookies 2022-11-18T20:58:48.6613453Z ┌ ↑ file ★ 77.47kB 2022-11-18T20:58:48.6613990Z │ (showing 2.05kB/77.47kB) 2022-11-18T20:58:48.6614733Z └ 2022-11-18T20:58:48.6615423Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 206B 2022-11-18T20:58:48.6615915Z │ { 2022-11-18T20:58:48.6616427Z │ "Cardigan_welsh_corgi": 0.6281963586807251, 2022-11-18T20:58:48.6616962Z │ "Briard": 0.27462807297706604, 2022-11-18T20:58:48.6617488Z │ "Icelandic_sheepdog": 0.07453605532646179, 2022-11-18T20:58:48.6618011Z │ "Tibetan_mastiff": 0.01554576214402914, 2022-11-18T20:58:48.6618523Z │ "Chow_chow": 0.0020620424766093493 2022-11-18T20:58:48.6618970Z │ } 2022-11-18T20:58:48.6632210Z └ 2022-11-18T20:58:48.6632808Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:58:48.6633473Z 4ms 983µs 460µs 270µs 444ms 2ms 78µs 452ms 2022-11-18T20:58:48.6633851Z 2022-11-18T20:58:48.6709033Z √ Successful request 2022-11-18T20:58:48.6717224Z √ Test expected JSON response 2022-11-18T20:58:48.6931857Z 2022-11-18T20:58:48.6932659Z → Workflow - Unregister 2022-11-18T20:58:48.9014783Z DELETE http://localhost:8081/workflows/dog_breed_wf 2022-11-18T20:58:48.9015867Z 200 OK ★ 159ms time ★ 251B↑ 323B↓ size ★ 7↑ 7↓ headers ★ 0 cookies 2022-11-18T20:58:48.9016507Z ┌ ↓ application/json ★ text ★ json ★ utf8 ★ 57B 2022-11-18T20:58:48.9016936Z │ { 2022-11-18T20:58:48.9017383Z │ "status": "Workflow \"dog_breed_wf\" unregistered" 2022-11-18T20:58:48.9017783Z │ } 2022-11-18T20:58:48.9033196Z └ 2022-11-18T20:58:48.9033738Z prepare wait dns-lookup tcp-handshake transfer-start download process total 2022-11-18T20:58:48.9034369Z 2ms 451µs (cache) (cache) 155ms 2ms 65µs 160ms 2022-11-18T20:58:48.9034690Z 2022-11-18T20:58:48.9129417Z √ Successful DELETE request 2022-11-18T20:58:49.1287023Z 2022-11-18T20:58:49.1287911Z ┌─────────────────────────┬───────────────────┬──────────────────┐ 2022-11-18T20:58:49.1297110Z │ │ executed │ failed │ 2022-11-18T20:58:49.1298143Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:58:49.1298624Z │ iterations │ 3 │ 0 │ 2022-11-18T20:58:49.1299400Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:58:49.1299911Z │ requests │ 9 │ 0 │ 2022-11-18T20:58:49.1300412Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:58:49.1300879Z │ test-scripts │ 9 │ 0 │ 2022-11-18T20:58:49.1301377Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:58:49.1301859Z │ prerequest-scripts │ 0 │ 0 │ 2022-11-18T20:58:49.1302423Z ├─────────────────────────┼───────────────────┼──────────────────┤ 2022-11-18T20:58:49.1302870Z │ assertions │ 11 │ 0 │ 2022-11-18T20:58:49.1303366Z ├─────────────────────────┴───────────────────┴──────────────────┤ 2022-11-18T20:58:49.1303960Z │ total run duration: 32.9s │ 2022-11-18T20:58:49.1304455Z ├────────────────────────────────────────────────────────────────┤ 2022-11-18T20:58:49.1304928Z │ total data received: 900B (approx) │ 2022-11-18T20:58:49.1305437Z ├────────────────────────────────────────────────────────────────┤ 2022-11-18T20:58:49.1305937Z │ average response time: 3.5s [min: 119ms, max: 12s, s.d.: 4s] │ 2022-11-18T20:58:49.1306449Z ├────────────────────────────────────────────────────────────────┤ 2022-11-18T20:58:49.1306924Z │ average DNS lookup time: 4ms [min: 334µs, max: 9ms, s.d.: 4ms] │ 2022-11-18T20:58:49.1307438Z ├────────────────────────────────────────────────────────────────┤ 2022-11-18T20:58:49.1307927Z │ average first byte time: 3.5s [min: 115ms, max: 12s, s.d.: 4s] │ 2022-11-18T20:58:49.1308435Z └────────────────────────────────────────────────────────────────┘ 2022-11-18T20:58:49.2416659Z TorchServe has stopped. 2022-11-18T20:58:59.2622004Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:58:59.2623375Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:58:59.2624045Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:58:59.2626850Z ## Starting TorchServe 2022-11-18T20:58:59.2627161Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:58:59.2628052Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs --ts-config=config.properties >> ts_console.log 2022-11-18T20:58:59.2628582Z ## Successfully started TorchServe 2022-11-18T20:58:59.2628876Z ## Stopping TorchServe 2022-11-18T20:58:59.2629342Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:58:59.2629913Z ## Successfully stopped TorchServe 2022-11-18T20:58:59.2630216Z ## Starting gen_mar: model_store 2022-11-18T20:58:59.2630391Z 2022-11-18T20:58:59.2630483Z ## Create symlink for mar files 2022-11-18T20:58:59.2630651Z 2022-11-18T20:58:59.2630908Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:58:59.2631442Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:58:59.2632098Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:58:59.2632578Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:58:59.2633115Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:58:59.2633659Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:58:59.2634191Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:58:59.2634823Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:58:59.2635383Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:58:59.2635964Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:58:59.2636532Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:58:59.2636884Z ## Starting TorchServe 2022-11-18T20:58:59.2637194Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:58:59.2645456Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs --ts-config=resources\config_kf.properties >> ts_console.log 2022-11-18T20:58:59.2646099Z ## Successfully started TorchServe 2022-11-18T20:58:59.2646392Z ## Stopping TorchServe 2022-11-18T20:58:59.2646950Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:58:59.2647392Z ## Successfully stopped TorchServe 2022-11-18T20:58:59.2647695Z ## Starting gen_mar: model_store 2022-11-18T20:58:59.2647892Z 2022-11-18T20:58:59.2648032Z ## Create symlink for mar files 2022-11-18T20:58:59.2648230Z 2022-11-18T20:58:59.2648526Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:58:59.2649228Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:58:59.2649943Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:58:59.2650552Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:58:59.2651346Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:58:59.2651972Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:58:59.2652640Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:58:59.2653429Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:58:59.2654089Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:58:59.2654758Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:58:59.2655465Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:58:59.2655915Z ## Starting TorchServe 2022-11-18T20:58:59.2656271Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:58:59.2657106Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs --ts-config=config.properties >> ts_console.log 2022-11-18T20:58:59.2657760Z ## Successfully started TorchServe 2022-11-18T20:58:59.2658051Z ## Stopping TorchServe 2022-11-18T20:58:59.2658593Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:58:59.2659029Z ## Successfully stopped TorchServe 2022-11-18T20:58:59.2659327Z ## Starting gen_mar: model_store 2022-11-18T20:58:59.2659521Z 2022-11-18T20:58:59.2659666Z ## Create symlink for mar files 2022-11-18T20:58:59.2659866Z 2022-11-18T20:58:59.2660158Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:58:59.2660766Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:58:59.2661458Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:58:59.2662157Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:58:59.2662765Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:58:59.2663384Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:58:59.2663952Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:58:59.2664732Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:58:59.2665390Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:58:59.2666056Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:58:59.2666672Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:58:59.2667222Z ## Starting TorchServe 2022-11-18T20:58:59.2667575Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:58:59.2668326Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs --ts-config=resources\config_kfv2.properties >> ts_console.log 2022-11-18T20:58:59.2668905Z ## Successfully started TorchServe 2022-11-18T20:58:59.2669230Z ## Stopping TorchServe 2022-11-18T20:58:59.2669754Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:58:59.2670317Z ## Successfully stopped TorchServe 2022-11-18T20:58:59.2670674Z ## Starting gen_mar: model_store 2022-11-18T20:58:59.2670867Z 2022-11-18T20:58:59.2671009Z ## Create symlink for mar files 2022-11-18T20:58:59.2671202Z 2022-11-18T20:58:59.2671451Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:58:59.2672058Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:58:59.2672805Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:58:59.2673400Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:58:59.2674028Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:58:59.2674757Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:58:59.2675366Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:58:59.2676239Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:58:59.2676855Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:58:59.2677517Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:58:59.2678171Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:58:59.2678619Z ## Starting TorchServe 2022-11-18T20:58:59.2678932Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:58:59.2679626Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --ncs >> ts_console.log 2022-11-18T20:58:59.2680105Z ## Successfully started TorchServe 2022-11-18T20:58:59.2680364Z ## Stopping TorchServe 2022-11-18T20:58:59.2680924Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:58:59.2681312Z ## Successfully stopped TorchServe 2022-11-18T20:58:59.2681603Z ## Starting gen_mar: model_store 2022-11-18T20:58:59.2681754Z 2022-11-18T20:58:59.2681874Z ## Create symlink for mar files 2022-11-18T20:58:59.2682058Z 2022-11-18T20:58:59.2682323Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:58:59.2682896Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:58:59.2683640Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:58:59.2684286Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:58:59.2684875Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:58:59.2685495Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:58:59.2686098Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:58:59.2686827Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:58:59.2687477Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:58:59.2688141Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:58:59.2688865Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:58:59.2689369Z ## Starting TorchServe 2022-11-18T20:58:59.2689729Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:58:59.2690492Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --workflow-store=model_store --ncs >> ts_console.log 2022-11-18T20:58:59.2691001Z ## Successfully started TorchServe 2022-11-18T20:58:59.2691293Z ## Stopping TorchServe 2022-11-18T20:58:59.2691782Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:58:59.2692200Z ## Successfully stopped TorchServe 2022-11-18T20:58:59.2692557Z ## Starting gen_mar: model_store 2022-11-18T20:58:59.2692740Z 2022-11-18T20:58:59.2692880Z ## Create symlink for mar files 2022-11-18T20:58:59.2693071Z 2022-11-18T20:58:59.2693367Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/mnist.mar, model_store/mnist.mar successfully. 2022-11-18T20:58:59.2694009Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/maskrcnn.mar, model_store/maskrcnn.mar successfully. 2022-11-18T20:58:59.2694732Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-18.mar, model_store/resnet-18.mar successfully. 2022-11-18T20:58:59.2695316Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/vgg16.mar, model_store/vgg16.mar successfully. 2022-11-18T20:58:59.2695916Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/squeezenet1_1.mar, model_store/squeezenet1_1.mar successfully. 2022-11-18T20:58:59.2696494Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fcn_resnet_101.mar, model_store/fcn_resnet_101.mar successfully. 2022-11-18T20:58:59.2697204Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/alexnet.mar, model_store/alexnet.mar successfully. 2022-11-18T20:58:59.2697986Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/resnet-152-batch.mar, model_store/resnet-152-batch.mar successfully. 2022-11-18T20:58:59.2698636Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/densenet161.mar, model_store/densenet161.mar successfully. 2022-11-18T20:58:59.2699260Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/deeplabv3_resnet_101_eager.mar, model_store/deeplabv3_resnet_101_eager.mar successfully. 2022-11-18T20:58:59.2699917Z ## Symlink D:\a\serve\serve\ts_scripts\..\model_store_gen/fastrcnn.mar, model_store/fastrcnn.mar successfully. 2022-11-18T20:58:59.2700368Z ## Starting TorchServe 2022-11-18T20:58:59.2700822Z ## Console logs redirected to file: ts_console.log 2022-11-18T20:58:59.2701533Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --start --model-store=model_store --workflow-store=model_store --ncs >> ts_console.log 2022-11-18T20:58:59.2702037Z ## Successfully started TorchServe 2022-11-18T20:58:59.2702317Z ## Stopping TorchServe 2022-11-18T20:58:59.2702783Z ## In directory: D:\a\serve\serve\test | Executing command: torchserve.exe --stop 2022-11-18T20:58:59.2703167Z ## Successfully stopped TorchServe 2022-11-18T20:58:59.3036347Z ## Newman API Tests Failed ! 2022-11-18T20:59:01.2255398Z ##[error]Process completed with exit code 1. 2022-11-18T20:59:01.2732025Z Post job cleanup. 2022-11-18T20:59:04.5809209Z [command]"C:\Program Files\Git\bin\git.exe" version 2022-11-18T20:59:04.6365431Z git version 2.38.1.windows.1 2022-11-18T20:59:04.6475773Z Temporarily overriding HOME='D:\a\_temp\517509d3-b0b2-4153-ac7a-11d42f4e2e6f' before making global git config changes 2022-11-18T20:59:04.6476453Z Adding repository directory to the temporary git global config as a safe directory 2022-11-18T20:59:04.6486241Z [command]"C:\Program Files\Git\bin\git.exe" config --global --add safe.directory D:\a\serve\serve 2022-11-18T20:59:04.6769271Z [command]"C:\Program Files\Git\bin\git.exe" config --local --name-only --get-regexp core\.sshCommand 2022-11-18T20:59:04.6997431Z [command]"C:\Program Files\Git\bin\git.exe" submodule foreach --recursive "git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :" 2022-11-18T20:59:05.2378614Z [command]"C:\Program Files\Git\bin\git.exe" config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader 2022-11-18T20:59:05.2578944Z http.https://github.com/.extraheader 2022-11-18T20:59:05.2613733Z [command]"C:\Program Files\Git\bin\git.exe" config --local --unset-all http.https://github.com/.extraheader 2022-11-18T20:59:05.2858859Z [command]"C:\Program Files\Git\bin\git.exe" submodule foreach --recursive "git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :" 2022-11-18T20:59:05.7768954Z Post job cleanup. 2022-11-18T20:59:06.0298940Z Cleaning up orphan processes 2022-11-18T20:59:06.1291416Z Terminate orphan process: pid (4380) (java)