Can we optimize download speed in mainland China #7

MichaelSimons · 2019-08-15T14:26:20Z

Copied from dotnet/dotnet-docker#1269 by ZhangGaoxing

Steps to reproduce the issue

Every time I pull a image from mcr.microsoft.com, the speed is very slow(about 200k/s). It's painful
☹. Other products download speed of .NET are normal, such as direct from .NET homepage download installation package.

The text was updated successfully, but these errors were encountered:

rotata · 2019-08-15T14:40:49Z

We will look at this right away and update you shortly. Could you provide us some logs around the usage.

expcat · 2020-02-17T19:43:40Z

This is my docker pull information.

PS C:\> docker pull mcr.microsoft.com/dotnet/core/sdk:3.1
3.1: Pulling from dotnet/core/sdk
dc65f448a2e2: Pulling fs layer
346ffb2b67d7: Downloading [===>                                               ]  489.6kB/7.812MB
dea4ecac934f: Downloading [=>                                                 ]  342.2kB/9.996MB
8ac92ddf84b3: Waiting
d6bef01952b9: Waiting
9311becdb99f: Waiting
cdbf4d78a668: Waiting
error pulling image configuration: Get https://mcrea0.cdn.mscr.io/b29889755b1f4e46b6b44eb3182530c5-0joxcrfl25//docker/registry/v2/blobs/sha256/0a/0a4c7c13f9d63ac865ae05b9253eda176556b748ab87cfbc59ca45b5a8beff9b/data?P1=1581968369&P2=1&P3=1&P4=FSwgCGN3ter9PmlpK9LN27dMwIN3WYFXV4X0oKhBN20%3D&se=2020-02-17T19%3A39%3A29Z&sig=W3x5AjBpCbXHXP1Y0T1NlOg2OeI63Dc%2FS6dS7LhOWD8%3D&sp=r&sr=b&sv=2016-05-31&regid=b29889755b1f4e46b6b44eb3182530c5: net/http: TLS handshake timeout
PS C:\> ping mcrea0.cdn.mscr.io

正在 Ping a-0016.a-msedge.net [204.79.197.219] 具有 32 字节的数据:
请求超时。   <-- timeout
请求超时。   <-- timeout
来自 204.79.197.219 的回复: 字节=32 时间=51ms TTL=117
来自 204.79.197.219 的回复: 字节=32 时间=51ms TTL=117

204.79.197.219 的 Ping 统计信息:
    数据包: 已发送 = 4，已接收 = 2，丢失 = 2 (50% 丢失)，  <--50% lost
往返行程的估计时间(以毫秒为单位):
    最短 = 51ms，最长 = 51ms，平均 = 51ms

docker logs

[03:19:20.753][ApiProxy          ][Info   ] time="2020-02-18T03:19:20+08:00" msg="proxy >> HEAD /_ping\n"
[03:19:20.754][ApiProxy          ][Info   ] time="2020-02-18T03:19:20+08:00" msg="proxy << HEAD /_ping (1.0004ms)\n"
[03:19:22.647][ApiProxy          ][Info   ] time="2020-02-18T03:19:22+08:00" msg="proxy >> POST /v1.40/images/create?fromImage=mcr.microsoft.com%2Fdotnet%2Fcore%2Fsdk&tag=3.1\n"
[03:19:22.648][VpnKit            ][Info   ] vpnkit.exe: Connected Ethernet interface f6:16:36:bc:f9:c6
[03:19:22.650][APIRequestLogger  ][Info   ] [1f5dc5cd] <Server start> POST http://unix/usage
[03:19:22.650][APIRequestLogger  ][Info   ] [1f5dc5cd] <Server end> POST http://unix/usage -> 200 OK (took 0ms)
[03:19:22.648][VpnKit            ][Info   ] vpnkit.exe: UDP interface connected on 192.168.65.1
[03:19:22.906][VpnKit            ][Info   ] vpnkit.exe: Connected Ethernet interface f6:16:36:bc:f9:c6
[03:19:22.906][VpnKit            ][Info   ] vpnkit.exe: UDP interface connected on 13.75.34.159
[03:19:26.995][VpnKit            ][Info   ] vpnkit.exe: Connected Ethernet interface f6:16:36:bc:f9:c6
[03:19:26.995][VpnKit            ][Info   ] vpnkit.exe: UDP interface connected on 204.79.197.219
[03:19:39.624][ApiProxy          ][Info   ] time="2020-02-18T03:19:39+08:00" msg="proxy << POST /v1.40/images/create?fromImage=mcr.microsoft.com%2Fdotnet%2Fcore%2Fsdk&tag=3.1 (16.9766979s)\n"
[03:19:39.625][ApiProxy          ][Info   ] time="2020-02-18T03:19:39+08:00" msg="Cancel connection..."

kkbruce · 2020-02-20T01:31:39Z

China Internet is special, especially GFW. It's not easy.

LiangZugeng · 2020-02-24T16:27:29Z

I think there are two ways to solve the problem:

Microsoft adds MCR CDN nodes in China or in Hongkong where the access speed is acceptable from China mainland.
For each every docker image hosted in MCR, Microsoft publishes a public image, which simply uses its MCR version as the image in Dockerfile but nothing else, on Docker hub, so docker registry mirrors in China (like docker CN or Aliyun docker mirrors) can pick those images up and sync to servers in China mainland.

The first solution is preferred but I'm not sure if CDN nodes can be added considering Chinese government requires a local registered domain name to leverage CDN nodes in China (apparently mcr.microsoft.com is not a local registered domain in China), but Microsoft may come up with other means as workarounds.

The second way requires additional Dockerfile files to be produced by Microsoft but I guess it can totally be automated, actually ElasticSearch uses this technique to generate public images on Docker Hub, see their Dockerfile (https://github.com/docker-library/elasticsearch/blob/master/7/Dockerfile).

wjsgzcn · 2020-02-27T06:45:48Z

why don't microsoft place a copy on docker hub? then we can use mirrors in china to get the images.

bariscaglar · 2020-02-27T18:42:22Z

The whole point of Microsoft Container Registry is to be a trustable endpoint for official Microsoft container images. Mirroring them elsewhere defeats this core purpose. That said, we understand the pain of low download speeds and are looking into ways to make the experience better.

shikelong · 2020-03-18T06:18:06Z

+1.

taoyouh · 2020-04-02T05:27:07Z

mcr.microsoft.com is very very slow. However, nuget.org is much faster.

taoyouh · 2020-04-02T05:32:44Z

Microsoft Store installation, Windows Update, NuGet package restoring and Visual Studio Installation are all much faster than pulling from mcr.microsoft.com in my network environment. This issue has a critical impact on the usability of Microsoft Container Registry.

PMExtra · 2020-04-07T07:57:07Z

Since last week, Azure disable mcr.azk8s.cn from Public Internet. We can't find any alternative.
Could you make a consultation with Azure China?

jackytse · 2020-04-09T14:27:28Z

I can't pull image from mcr.azk8s.cn. any other mirrors?

dorthl · 2020-04-14T09:14:08Z

+1

stratosblue · 2020-05-26T16:49:01Z

Is anybody there? Help the kids!!!

toddysm · 2020-06-05T23:10:36Z

To update this thread - we are evaluating options to have a MCR replica in China region. It will take a while but we will try to keep you updated.

newbe36524 · 2020-06-08T13:28:37Z

Hi, @LiangZugeng , as you options 2 , I setup a github repository to re-publish that images by github action, you can check it out https://github.com/newbe36524/Newbe.McrMirror

newbe36524 · 2020-06-17T00:29:24Z

Hi issue visitors, I just make a dotnet global tool to help you pull netcore related images from server hosted in China mainland, you check my post to use it . have a nice travel. docker-mcr

darkguy2008 · 2020-07-29T19:43:44Z

Can this be used in Costa Rica? I'm experiencing the same issue here... already went through two FTTH ISPs 100/100, download speeds are just ridiculous

newbe36524 · 2020-07-30T00:50:41Z

@darkguy2008 do you mean my project using in Costa Rica?
Maybe you could take a try on it.
If it is still slow, you can fork my project to re-publish the image to your server in Costa Rica.
My docker-mcr can download from specific server by pass some parameters. You can take look at with --help.

maikebing · 2020-08-14T16:27:52Z

Three hours also failed to complete. It's torture.

maikebing · 2020-08-14T16:28:09Z

docker pull mcr.microsoft.com/dotnet/core/sdk:3.1-buster && docker pull mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim

PMExtra · 2020-08-14T16:34:13Z

@newbe36524 Did you sync any other mcr images such as mssql?

crclz · 2020-08-30T16:32:18Z

@maikebing @PMExtra I hope this guide helps.

Guide to pull safe dotnet MCR images in mainland
教程：在中国大陆获取安全的MCR镜像: https://github.com/crclz/McrTransfer

newbe36524 · 2020-08-31T01:12:26Z

docker pull mcr.microsoft.com/dotnet/core/sdk:3.1-buster && docker pull mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim

maby you can take look on my solution https://github.com/newbe36524/Newbe.McrMirror

newbe36524 · 2020-08-31T01:13:02Z

@newbe36524 Did you sync any other mcr images such as mssql?

It is done . please check the the README for all tags https://github.com/newbe36524/Newbe.McrMirror

bariscaglar · 2020-10-14T18:32:40Z

We have started working on bringing an official MCR mirror for mainland China. It should be ready by the end of the year.

fsdrw08 · 2020-10-23T16:36:31Z

thank you very much, hope the MCR mirror to mainland China ASAP

xky0007 · 2020-12-11T08:21:39Z

Almost the end of the year...

xmapst · 2021-01-28T08:59:33Z

It's the beginning of the year...

bariscaglar · 2021-02-05T00:44:13Z

Sorry to keep you all waiting, but I'm happy to say that we have set up CDN endpoints in mainland China. Please let me know how this is working out.

Dolphinsimon · 2021-02-05T02:36:13Z

I have tested the sever location. Looks it still routes me to the HK data center.

Dolphinsimon · 2021-02-05T02:37:16Z

nslookup mcr.microsoft.com

Non-authoritative answer:
Name:    rpm0125ea.eastasia.cloudapp.azure.com
Address:  13.75.34.157
Aliases:  mcr.microsoft.com
          global.fe.mscr.io
          mcr-global.trafficmanager.net

nslookup download.microsoft.com

Non-authoritative answer:
Name:    zlxxipv6.v.lxcvc.com
Addresses:  2408:873c:b010:2001:1000:0:1:18
          2408:873c:b010:2001:1000:0:1:2a
          2408:873c:b010:2001:1000:0:1:23
          2408:873c:b010:2001:1000:0:1:15
          2408:873c:b010:2001:1000:0:1:14
          2408:873c:b010:2001:1000:0:1:26
          218.98.50.49
Aliases:  download.microsoft.com
          dlc-shim.trafficmanager.net
          download.microsoft.com.ccgslb.com.cn
          download.microsoft.com.lxcvc.com

The download site cdn works but the mcr site not.

crclz · 2021-02-05T02:44:42Z

@bariscaglar
It works fine, both on my home computer and on my aliyun cloud server. docker pull speed is fast.

qweeah · 2021-02-05T06:59:11Z

nslookup mcr.microsoft.com

Non-authoritative answer:
Name:    rpm0125ea.eastasia.cloudapp.azure.com
Address:  13.75.34.157
Aliases:  mcr.microsoft.com
          global.fe.mscr.io
          mcr-global.trafficmanager.net

nslookup download.microsoft.com

Non-authoritative answer:
Name:    zlxxipv6.v.lxcvc.com
Addresses:  2408:873c:b010:2001:1000:0:1:18
          2408:873c:b010:2001:1000:0:1:2a
          2408:873c:b010:2001:1000:0:1:23
          2408:873c:b010:2001:1000:0:1:15
          2408:873c:b010:2001:1000:0:1:14
          2408:873c:b010:2001:1000:0:1:26
          218.98.50.49
Aliases:  download.microsoft.com
          dlc-shim.trafficmanager.net
          download.microsoft.com.ccgslb.com.cn
          download.microsoft.com.lxcvc.com

The download site cdn works but the mcr site not.

mcr.microsoft.com is the domain for Registry Runtime. You will be automatically redirected to our Registry Data Endpoint based on you geo-location. If you are in mainland China, probably eastasia.data.mcr.microsoft.com or southeastasia.data.mcr.microsoft.com.

Dolphinsimon · 2021-02-05T07:03:44Z

nslookup mcr.microsoft.com
Non-authoritative answer:

Name: rpm0125ea.eastasia.cloudapp.azure.com

Address: 13.75.34.157

Aliases: mcr.microsoft.com
      global.fe.mscr.io
      mcr-global.trafficmanager.net
nslookup download.microsoft.com
Non-authoritative answer:

Name: zlxxipv6.v.lxcvc.com

Addresses: 2408:873c:b010:2001:1000:0:1:18
      2408:873c:b010:2001:1000:0:1:2a
      2408:873c:b010:2001:1000:0:1:23
      2408:873c:b010:2001:1000:0:1:15
      2408:873c:b010:2001:1000:0:1:14
      2408:873c:b010:2001:1000:0:1:26
      218.98.50.49
Aliases: download.microsoft.com
      dlc-shim.trafficmanager.net
      download.microsoft.com.ccgslb.com.cn
      download.microsoft.com.lxcvc.com
The download site cdn works but the mcr site not.

mcr.microsoft.com is the domain for Registry Runtime. You will be automatically redirected to our Registry Data Endpoint based on you geo-location. If you are in mainland China, probably eastasia.data.mcr.microsoft.com or southeastasia.data.mcr.microsoft.com.

So, there is no cdn endpoints in China Mainland. The nearest data center is located in HK.

qweeah · 2021-02-05T07:05:17Z

nslookup mcr.microsoft.com

Non-authoritative answer:

Name: rpm0125ea.eastasia.cloudapp.azure.com

Address: 13.75.34.157

Aliases: mcr.microsoft.com
      global.fe.mscr.io
      mcr-global.trafficmanager.net
nslookup download.microsoft.com

Non-authoritative answer:

Name: zlxxipv6.v.lxcvc.com

Addresses: 2408:873c:b010:2001:1000:0:1:18
      2408:873c:b010:2001:1000:0:1:2a
      2408:873c:b010:2001:1000:0:1:23
      2408:873c:b010:2001:1000:0:1:15
      2408:873c:b010:2001:1000:0:1:14
      2408:873c:b010:2001:1000:0:1:26
      218.98.50.49
Aliases: download.microsoft.com
      dlc-shim.trafficmanager.net
      download.microsoft.com.ccgslb.com.cn
      download.microsoft.com.lxcvc.com
The download site cdn works but the mcr site not.

mcr.microsoft.com is the domain for Registry Runtime. You will be automatically redirected to our Registry Data Endpoint based on you geo-location. If you are in mainland China, probably eastasia.data.mcr.microsoft.com or southeastasia.data.mcr.microsoft.com.
So, there is no cdn endpoints in China Mainland. The nearest data center is located in HK.

Try nslookup the Registry Data Endpoint domains and you will get answer :)

Dolphinsimon · 2021-02-05T07:14:59Z

nslookup mcr.microsoft.com

Non-authoritative answer:

Name: rpm0125ea.eastasia.cloudapp.azure.com

Address: 13.75.34.157

Aliases: mcr.microsoft.com
      global.fe.mscr.io
      mcr-global.trafficmanager.net
nslookup download.microsoft.com

Non-authoritative answer:

Name: zlxxipv6.v.lxcvc.com

Addresses: 2408:873c:b010:2001:1000:0:1:18
      2408:873c:b010:2001:1000:0:1:2a
      2408:873c:b010:2001:1000:0:1:23
      2408:873c:b010:2001:1000:0:1:15
      2408:873c:b010:2001:1000:0:1:14
      2408:873c:b010:2001:1000:0:1:26
      218.98.50.49
Aliases: download.microsoft.com
      dlc-shim.trafficmanager.net
      download.microsoft.com.ccgslb.com.cn
      download.microsoft.com.lxcvc.com
The download site cdn works but the mcr site not.

mcr.microsoft.com is the domain for Registry Runtime. You will be automatically redirected to our Registry Data Endpoint based on you geo-location. If you are in mainland China, probably eastasia.data.mcr.microsoft.com or southeastasia.data.mcr.microsoft.com.
So, there is no cdn endpoints in China Mainland. The nearest data center is located in HK.
Try nslookup the Registry Data Endpoint domains and you will get answer :)

The eastasia.data.mcr.microsoft.com works, I've got a cdn node in Azure China.
Thanks for your explanation.

MichaelSimons mentioned this issue Aug 15, 2019

Can we optimize download speed in mainland China dotnet/dotnet-docker#1269

Closed

bariscaglar assigned qweeah Oct 30, 2020

bariscaglar added the enhancement New feature or request label Dec 2, 2020

bariscaglar closed this as completed Feb 5, 2021

Can we optimize download speed in mainland China #7

Can we optimize download speed in mainland China #7

Comments

MichaelSimons commented Aug 15, 2019

Steps to reproduce the issue

rotata commented Aug 15, 2019

expcat commented Feb 17, 2020

kkbruce commented Feb 20, 2020

LiangZugeng commented Feb 24, 2020

wjsgzcn commented Feb 27, 2020

bariscaglar commented Feb 27, 2020

shikelong commented Mar 18, 2020

taoyouh commented Apr 2, 2020

taoyouh commented Apr 2, 2020

PMExtra commented Apr 7, 2020 • edited Loading

jackytse commented Apr 9, 2020

dorthl commented Apr 14, 2020

stratosblue commented May 26, 2020

toddysm commented Jun 5, 2020

newbe36524 commented Jun 8, 2020 • edited Loading

newbe36524 commented Jun 17, 2020

darkguy2008 commented Jul 29, 2020

newbe36524 commented Jul 30, 2020

maikebing commented Aug 14, 2020

maikebing commented Aug 14, 2020

PMExtra commented Aug 14, 2020

crclz commented Aug 30, 2020

newbe36524 commented Aug 31, 2020

newbe36524 commented Aug 31, 2020 • edited Loading

bariscaglar commented Oct 14, 2020

fsdrw08 commented Oct 23, 2020

xky0007 commented Dec 11, 2020

xmapst commented Jan 28, 2021

bariscaglar commented Feb 5, 2021

Dolphinsimon commented Feb 5, 2021

Dolphinsimon commented Feb 5, 2021 • edited Loading

crclz commented Feb 5, 2021

qweeah commented Feb 5, 2021 • edited Loading

Dolphinsimon commented Feb 5, 2021

qweeah commented Feb 5, 2021

Dolphinsimon commented Feb 5, 2021

PMExtra commented Apr 7, 2020 •

edited

Loading

newbe36524 commented Jun 8, 2020 •

edited

Loading

newbe36524 commented Aug 31, 2020 •

edited

Loading

Dolphinsimon commented Feb 5, 2021 •

edited

Loading

qweeah commented Feb 5, 2021 •

edited

Loading