#54 Preliminary Windows Image Support #1894

jeremyje · 2020-11-11T01:40:11Z

What type of PR is this?
/kind feature

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespace from that line:

/kind breaking
/kind bug
/kind cleanup
/kind documentation
/kind feature
/kind hotfix

What this PR does / Why we need it:
This adds some basic Windows container support to Agones.

This PR is focused on getting the Windows support in Agones started rather than be a complete PR.
There's a few changes that Agones needs to properly support Windows:

make WITH_WINDOWS=1 will add Windows container artifacts for sdk-server. This option is disabled by default a follow up PR will be sent to turn it on.
Like the sdk-server any binaries that can run in Windows should be written to the local file system rather than in the context of a base docker container. COPY --from does not work between Linux and Windows container images.
The changes here will work on Linux and should work on Google Cloud Build since they run 19.03+ (required for buildx) on the backend.
There's a bug in buildx where the os.platform cannot easily be set, see Added support for setting OS version in docker manifest annotate. docker/cli#2578 (fixed in docker 20+). The tagged bug goes into detail but since you're only able to target 1 version of Windows I strongly suggest Windows Server 2019 (WINDOWS_LTSC).
WINDOWS_VERSION was introduced to provide a set of Windows versions to target. It is set to 1809 which is aliased to ltsc2019 (not all Windows images support the ltsc2019 tagging).
The Makefile in general will need to be updated to have image tagging for Linux container images to have the suffix -linux_amd64. This will allow you to have multi-arch and multi-platform images going forward. This means that you can use the old references for Agones (no platform/OS modifier in the tag) and it'll pull the correct image that aligns with the host.
Windows Server container images have a strong host compatibility requirement. Windows Server 2019 (LTSC) images will NOT work on Windows Server 1909. Effectively you need to treat them as separate platforms.
You'll need to avoid running powershell or cmd commands in your Dockerfile.windows. If you do this you WILL need a Windows host. The alternative is to run all the modifications in a Linux container export it to local file system (as is) and use the COPY command in your Dockerfile.windows.

Which issue(s) this PR fixes:

First set of changes for #54
Fixes #110

Special notes for your reviewer:

The Dockerfile.windows naming convention is following how Kubernetes is typically introducing Windows support. There's no strong guidance there.
The make targets are not added to the official build.
I did a very simple build pass to see if things compile.
The PR is very verbose and there's a lot of duplication. This is intentional, there are deeper changes to the Agones build system that are needed to properly fix them.

agones-bot · 2020-11-11T02:02:07Z

Build Succeeded 👏

Build Id: 6044f7ef-eae4-4d1a-a174-4c135db5f9e6

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.10.0-37af959
image: gcr.io/agones-images/agones-sdk:1.10.0-37af959
image: gcr.io/agones-images/agones-ping:1.10.0-37af959
Linux C++ SDK (build): agonessdk-1.10.0-37af959-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.10.0-37af959.zip

A preview of the website (the last 30 builds are retained):

https://37af959-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/1894/head:pr_1894 && git checkout pr_1894
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.10.0-37af959

jeremyje · 2020-11-11T19:31:52Z

cc: @WeetA34 @markmandel

agones-bot · 2020-11-11T19:49:46Z

Build Succeeded 👏

Build Id: b5e52a0a-c86e-478d-a246-bfeb81c0ff43

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.10.0-8d42fa4
image: gcr.io/agones-images/agones-sdk:1.10.0-8d42fa4
image: gcr.io/agones-images/agones-ping:1.10.0-8d42fa4
Linux C++ SDK (build): agonessdk-1.10.0-8d42fa4-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.10.0-8d42fa4.zip

A preview of the website (the last 30 builds are retained):

https://8d42fa4-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/1894/head:pr_1894 && git checkout pr_1894
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.10.0-8d42fa4

markmandel · 2020-11-16T23:36:27Z

@WeetA34 are you able to take this for a spin? You're probably placed as the best person to see how well it works.

WeetA34 · 2020-11-17T09:03:11Z

Hi @markmandel
agones-sdk 1.10.0-8d42fa4 windows image is not found

  Warning  Failed     11s               kubelet, gke-agones-win-windows-2e403fc4-ch0p  Failed to pull image "gcr.io/agones-images/agones-sdk:1.10.0-8d42fa4": rpc error: code = Unknown desc = image operating system "linux" cannot be used on this platform

markmandel · 2020-11-17T16:38:16Z

Hi @markmandel
agones-sdk 1.10.0-8d42fa4 windows image is not found

  Warning  Failed     11s               kubelet, gke-agones-win-windows-2e403fc4-ch0p  Failed to pull image "gcr.io/agones-images/agones-sdk:1.10.0-8d42fa4": rpc error: code = Unknown desc = image operating system "linux" cannot be used on this platform

Yes, you would still need to apply it to your own registry, you don't have access to the agones-images project.

You can see details in our development guide: https://github.com/googleforgames/agones/blob/master/build/README.md#running-a-test-google-kubernetes-engine-cluster

WeetA34 · 2020-11-17T18:11:59Z

Hi
I've already built agones-sdk 1.9.0, 1.10.0 windows images on my side.
My problem was how to use them as the image path is automaticaly of the sidecar container is automaticaly defined.
I didn't check recently if i can tell where to pull the agones-sdk from.

I thought 1.10.0-8d42fa4 images have been built and were available on the agones-images registry like the stable images.

markmandel · 2020-11-17T18:18:00Z

i thought images have been built and were available on the agones-images registry like the stable images

Given this is currently just a PR, I'm not sure how that would have happened 😄

If you are able to read the description of the PR, and see if it works for you to build the images as expected and you are able to use them -- would be very useful to get your feedback.

jeremyje · 2020-11-17T18:20:25Z

The PR will not build the Windows images by default you have to specifically run the new make file commands.
make push-agones-sdk-image-multi

and run the make push-multi-arch-image in the simple-tcp. You'll need to override the registry as well.

WeetA34 · 2020-11-17T18:30:37Z

Given this is currently just a PR, I'm not sure how that would have happened 😄
yep of course 🤦‍♂️ artifacts 🤣

If you are able to read the description of the PR, and see if it works for you to build the images as expected and you are able to use them -- would be very useful to get your feedback.
Ok i'll try it and use agones.image.registry helm overrides to test

WeetA34 · 2020-11-17T18:31:24Z

The PR will not build the Windows images by default you have to specifically run the new make file commands.
make push-agones-sdk-image-multi

and run the make push-multi-arch-image in the simple-tcp. You'll need to override the registry as well.

Ok will try this tomorrow

WeetA34 · 2020-11-17T18:31:46Z

Thx to both of you

jeremyje · 2020-11-17T18:34:55Z

You'll multiple images for the same thing. -windows_amd64-1809 suffix for the direct image for Windows 2019. And -multi that will work on Linux or Windows. Try the -windows_amd64-1809 first since that's avoiding a hop. Then try -multi to confirm that the image build will work crossplatform.

@markmandel I don't have really any more cycles to work on this. Can someone take this PR over?

markmandel · 2020-11-17T18:47:59Z

@markmandel I don't have really any more cycles to work on this. Can someone take this PR over?

I honestly don't know anyone who has the knowledge you do here 😬

jeremyje · 2020-11-19T23:51:47Z

@markmandel The PR in its current form won't break Agones. I can add some instructions on how to build the binaries and containers. Though it's been over a week and no one has reviewed it so it's not clear if this is a priority for Agones.

@markmandel @WeetA34 if you have a list of artifacts you want to be built on Windows I might be able to slip that in (untested).

WeetA34 · 2020-11-20T09:17:52Z

Hello,
sorry for the delay.
I've tried to compile on my WSL.
For now, i'm investigating the following error during make push-agones-sdk-image-multi on windows part:
auto-push is currently not implemented for docker driver

jeremyje · 2020-11-20T17:00:04Z

I don't know about WSL2 but it should work in a Linux VM. I cannot test this because my WSL2 environment is tainted (I run make on the host).
Make sure that your Docker installation is current otherwise try linux.

agones-bot · 2020-11-20T17:18:04Z

Build Succeeded 👏

Build Id: 37eb489e-0c04-46e5-aad1-de72b847272d

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.11.0-d5f29f7
image: gcr.io/agones-images/agones-sdk:1.11.0-d5f29f7
image: gcr.io/agones-images/agones-ping:1.11.0-d5f29f7
Linux C++ SDK (build): agonessdk-1.11.0-d5f29f7-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.11.0-d5f29f7.zip

A preview of the website (the last 30 builds are retained):

https://d5f29f7-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/1894/head:pr_1894 && git checkout pr_1894
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.11.0-d5f29f7

agones-bot · 2020-11-25T19:35:55Z

Build Succeeded 👏

Build Id: 2e192be7-e21c-49e7-8715-d5ee15a40ab4

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.11.0-43ce7ee
image: gcr.io/agones-images/agones-sdk:1.11.0-43ce7ee
image: gcr.io/agones-images/agones-ping:1.11.0-43ce7ee
Linux C++ SDK (build): agonessdk-1.11.0-43ce7ee-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.11.0-43ce7ee.zip

A preview of the website (the last 30 builds are retained):

https://43ce7ee-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/1894/head:pr_1894 && git checkout pr_1894
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.11.0-43ce7ee

agones-bot · 2020-11-30T18:44:43Z

Build Failed 😱

Build Id: bc85ea9c-ec0b-4ded-87cb-93b485d06b1f

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2020-11-30T19:03:11Z

Build Failed 😱

Build Id: 1f9871bf-33c3-48a9-9a88-d3acd616e5d6

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

markmandel · 2020-11-30T19:06:35Z

Expected failure, or flakiness?

Already have image: make-docker
mkdir -p ~/.kube/
mkdir -p /workspace/build//.gocache
mkdir -p /workspace/build//.config/gcloud
mkdir -p ~/.config/helm
mkdir -p ~/.cache/helm
/usr/bin/make ensure-image IMAGE_TAG=agones-build:f5378ddabe BUILD_TARGET=build-build-image
make[1]: Entering directory '/workspace/build'
make[1]: Leaving directory '/workspace/build'
docker push gcr.io/agones-images/agones-controller:1.11.0-59ebcf4
docker push gcr.io/agones-images/agones-sdk:1.11.0-59ebcf4-linux_amd64
docker push gcr.io/agones-images/agones-ping:1.11.0-59ebcf4
docker push gcr.io/agones-images/agones-allocator:1.11.0-59ebcf4
The push refers to repository [gcr.io/agones-images/agones-allocator]
The push refers to repository [gcr.io/agones-images/agones-controller]
6fe5e06d42a9: Preparing
4d36935cfadc: Preparing
ef167023d537: Preparing
The push refers to repository [gcr.io/agones-images/agones-sdk]
3e207b409db3: Preparing
The push refers to repository [gcr.io/agones-images/agones-ping]
5314e2ad398b: Preparing
c1f25e205343: Preparing
ef167023d537: Preparing
3e207b409db3: Preparing
3e207b409db3: Waiting
ef167023d537: Waiting
5314e2ad398b: Preparing
9a3987255414: Preparing
ef167023d537: Preparing
3e207b409db3: Preparing
3e207b409db3: Waiting
ef167023d537: Waiting
5314e2ad398b: Waiting
9a3987255414: Waiting
1253f678b740: Preparing
be3f5eef7a2c: Preparing
ef167023d537: Preparing
3e207b409db3: Preparing
1253f678b740: Waiting
be3f5eef7a2c: Waiting
3e207b409db3: Waiting
3e207b409db3: Layer already exists
6fe5e06d42a9: Pushed
ef167023d537: Pushed
3e207b409db3: Layer already exists
5314e2ad398b: Pushed
5314e2ad398b: Pushed
ef167023d537: Layer already exists
3e207b409db3: Layer already exists
ef167023d537: Pushed
1253f678b740: Pushed
ef167023d537: Layer already exists
3e207b409db3: Layer already exists
4d36935cfadc: Pushed
9a3987255414: Pushed
c1f25e205343: Pushed
1.11.0-59ebcf4: digest: sha256:b5acc16242b9405ceaaf6e69039fab1682c3b4575637dc34f4fddf088083d197 size: 1160
1.11.0-59ebcf4: digest: sha256:1e86462b3c6383c01590e3365c3eb91e730753517eefc7ea1add0207f4a74c56 size: 1159
1.11.0-59ebcf4: digest: sha256:cf03c406645e57639c1b1f03eb914e148a88c6f2da325dd785baed47bcbad081 size: 1160
be3f5eef7a2c: Pushed
unknown blob
Makefile:471: recipe for target 'push-agones-sdk-linux-image' failed
make: *** [push-agones-sdk-linux-image] Error 1

agones-bot · 2020-11-30T20:40:09Z

Build Succeeded 👏

Build Id: 958d2f7f-af51-442d-a91d-cd3202c9645e

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.11.0-ff3a2e7
image: gcr.io/agones-images/agones-sdk:1.11.0-ff3a2e7
image: gcr.io/agones-images/agones-sdk:1.11.0-ff3a2e7-linux_amd64
image: gcr.io/agones-images/agones-ping:1.11.0-ff3a2e7
Linux C++ SDK (build): agonessdk-1.11.0-ff3a2e7-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.11.0-ff3a2e7.zip

A preview of the website (the last 30 builds are retained):

https://ff3a2e7-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/1894/head:pr_1894 && git checkout pr_1894
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.11.0-ff3a2e7

agones-bot · 2020-11-30T21:19:06Z

Build Succeeded 👏

Build Id: 2b958572-1a03-4cfd-9b44-75e153c75e9c

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.11.0-2b275de
image: gcr.io/agones-images/agones-sdk:1.11.0-2b275de
image: gcr.io/agones-images/agones-ping:1.11.0-2b275de
Linux C++ SDK (build): agonessdk-1.11.0-2b275de-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.11.0-2b275de.zip

A preview of the website (the last 30 builds are retained):

https://2b275de-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/1894/head:pr_1894 && git checkout pr_1894
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.11.0-2b275de

markmandel · 2020-12-01T20:09:32Z

Did a quick smoke test with WITH_WINDOWS enabled to ensure everything still works on Linux. Was able to build the image, and push it up, and everything worked as expected.

Nice first step! Approving!

google-oss-robot · 2020-12-01T20:10:07Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jeremyje, markmandel

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [markmandel]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

google-oss-robot · 2020-12-01T20:10:13Z

New changes are detected. LGTM label has been removed.

agones-bot · 2020-12-01T20:59:43Z

Build Succeeded 👏

Build Id: 4fe30f3a-db1f-42a5-8a17-e87ad3ea4ef3

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.11.0-47e7354
image: gcr.io/agones-images/agones-sdk:1.11.0-47e7354
image: gcr.io/agones-images/agones-ping:1.11.0-47e7354
Linux C++ SDK (build): agonessdk-1.11.0-47e7354-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.11.0-47e7354.zip

A preview of the website (the last 30 builds are retained):

https://47e7354-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/1894/head:pr_1894 && git checkout pr_1894
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.11.0-47e7354

WeetA34 · 2020-12-01T23:06:12Z

Hi @jeremyje @markmandel,
FYI, i successfully built and pushed images with Docker Desktop 2.5.0.1 with WSL2 Integration.

Regarding agones-sdk windows image, you can get a smaller image by using @EricFortin 's docker file
#54 (comment)

Thank you to both of you

WeetA34 · 2020-12-02T08:57:55Z

On a second workstation which is behind a proxy, i had to add ARG in build-image/Dockerfile.
Then, i encountered the following error:

------
 > [1/3] FROM mcr.microsoft.com/windows/servercore:ltsc2019@sha256:0da0b0e6cf9cf435c6470f23a97a0b9b80495f3712529abc748d80c5fb59209c:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to build LLB: failed to register layer: Error processing tar file(exit status 1): link /Files/Program Files/common files/Microsoft Shared/Ink/en-US/micaut.dll.mui /Files/Program Files (x86)/common files/Microsoft Shared/ink/en-US/micaut.dll.mui: no such file or directory
Makefile:412: recipe for target 'build-agones-sdk-image-windows-ltsc2019' failed
make: *** [build-agones-sdk-image-windows-ltsc2019] Error 1

edit: I retried this morning and it worked

jeremyje · 2020-12-02T16:20:49Z

I've seen this a few times you can see others on the internet as well reasons why this happens off the top of my head.

The buildx context is pointing to default and is not backed by the qemu instance that's created.
I find that buildx works better on a Linux host. If you're on windows run it straight with docker build though the Makefile is not configured for that since the expectation is that it's always running in a Linux host. (It probably will run in WSL2 but this might be an awkward artifact).
Running concurrent builds. If that's the case each artifact can have a separate context though you'll want to delete them after a build completes.

The way I develop is running a Hyper-V VM that runs Linux and build there. On the Windows host I run in Windows Container mode which allows me to build and run those containers. There's a Xonotic example that is not linked up that should build and run exclusively in a Windows Container environment. The qemu backed buildx context will not build Xonotic because it has powershell in it.

WeetA34 · 2020-12-02T19:03:41Z

At the end, it worked fine on both workstations running Docker Desktop 2.5.0.1 with WSL2 integration.

google-cla bot added the cla: yes label Nov 11, 2020

google-oss-robot requested review from pooneh-m and roberthbailey November 11, 2020 01:40

google-oss-robot added the size/M label Nov 11, 2020

google-oss-robot added size/L and removed size/M labels Nov 11, 2020

jeremyje changed the title ~~#54 Preliminary Windows support for Agones~~ #54 Preliminary Windows Image Support Nov 11, 2020

jeremyje mentioned this pull request Nov 27, 2020

Running windows game server #54

Open

google-oss-robot added size/XL and removed size/L labels Nov 30, 2020

Preliminary Windows Docker Image support for Agones

2b275de

markmandel approved these changes Dec 1, 2020

View reviewed changes

google-oss-robot assigned markmandel Dec 1, 2020

google-oss-robot added the lgtm label Dec 1, 2020

google-oss-robot added the approved label Dec 1, 2020

Merge branch 'master' into winagones

80e0952

google-oss-robot removed the lgtm label Dec 1, 2020

markmandel mentioned this pull request Dec 1, 2020

Deprecate and remove the udp-server and tcp-server images #1890

Closed

Merge branch 'master' into winagones

47e7354

markmandel merged commit 2e6c7bd into googleforgames:master Dec 1, 2020

markmandel added area/build-tools Development tooling. I.e. pretty much everything in the `build` directory. kind/feature New features for Agones labels Dec 1, 2020

jeremyje deleted the winagones branch December 2, 2020 16:10

markmandel added this to the 1.11.0 milestone Dec 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

#54 Preliminary Windows Image Support #1894

#54 Preliminary Windows Image Support #1894

jeremyje commented Nov 11, 2020 •

edited

Loading

agones-bot commented Nov 11, 2020

jeremyje commented Nov 11, 2020

agones-bot commented Nov 11, 2020

markmandel commented Nov 16, 2020

WeetA34 commented Nov 17, 2020

markmandel commented Nov 17, 2020

WeetA34 commented Nov 17, 2020 •

edited

Loading

markmandel commented Nov 17, 2020

jeremyje commented Nov 17, 2020 •

edited

Loading

WeetA34 commented Nov 17, 2020

WeetA34 commented Nov 17, 2020

WeetA34 commented Nov 17, 2020

jeremyje commented Nov 17, 2020

markmandel commented Nov 17, 2020

jeremyje commented Nov 19, 2020

WeetA34 commented Nov 20, 2020

jeremyje commented Nov 20, 2020

agones-bot commented Nov 20, 2020

agones-bot commented Nov 25, 2020

agones-bot commented Nov 30, 2020

agones-bot commented Nov 30, 2020

markmandel commented Nov 30, 2020

agones-bot commented Nov 30, 2020

agones-bot commented Nov 30, 2020

markmandel commented Dec 1, 2020

google-oss-robot commented Dec 1, 2020

google-oss-robot commented Dec 1, 2020

agones-bot commented Dec 1, 2020

WeetA34 commented Dec 1, 2020 •

edited

Loading

WeetA34 commented Dec 2, 2020 •

edited

Loading

jeremyje commented Dec 2, 2020

WeetA34 commented Dec 2, 2020

#54 Preliminary Windows Image Support #1894

#54 Preliminary Windows Image Support #1894

Conversation

jeremyje commented Nov 11, 2020 • edited Loading

agones-bot commented Nov 11, 2020

jeremyje commented Nov 11, 2020

agones-bot commented Nov 11, 2020

markmandel commented Nov 16, 2020

WeetA34 commented Nov 17, 2020

markmandel commented Nov 17, 2020

WeetA34 commented Nov 17, 2020 • edited Loading

markmandel commented Nov 17, 2020

jeremyje commented Nov 17, 2020 • edited Loading

WeetA34 commented Nov 17, 2020

WeetA34 commented Nov 17, 2020

WeetA34 commented Nov 17, 2020

jeremyje commented Nov 17, 2020

markmandel commented Nov 17, 2020

jeremyje commented Nov 19, 2020

WeetA34 commented Nov 20, 2020

jeremyje commented Nov 20, 2020

agones-bot commented Nov 20, 2020

agones-bot commented Nov 25, 2020

agones-bot commented Nov 30, 2020

agones-bot commented Nov 30, 2020

markmandel commented Nov 30, 2020

agones-bot commented Nov 30, 2020

agones-bot commented Nov 30, 2020

markmandel commented Dec 1, 2020

google-oss-robot commented Dec 1, 2020

google-oss-robot commented Dec 1, 2020

agones-bot commented Dec 1, 2020

WeetA34 commented Dec 1, 2020 • edited Loading

WeetA34 commented Dec 2, 2020 • edited Loading

jeremyje commented Dec 2, 2020

WeetA34 commented Dec 2, 2020

jeremyje commented Nov 11, 2020 •

edited

Loading

WeetA34 commented Nov 17, 2020 •

edited

Loading

jeremyje commented Nov 17, 2020 •

edited

Loading

WeetA34 commented Dec 1, 2020 •

edited

Loading

WeetA34 commented Dec 2, 2020 •

edited

Loading