How to deploy Navi Agent on my current OS and test a custom task

[Calvvnono]

Hello,

I would like to directly deploy the Navi Agent on my current operating system (without the need for virtual machines or cloud setups) and test a specific custom task. My goal is to observe the agent's performance in real-time on my local environment.

Could you please clarify if this is feasible, and if so, what steps should I follow to achieve this?

Any guidance on this process would be greatly appreciated.

Thank you!

[francedot]

This feature is on our roadmap, but unfortunately, I can’t provide a specific timeline. It will require some modifications to the current Docker setup. We welcome contributions, so if you feel up to the task, here's some guidance to help you get started:

1. In the current architecture, both the client and server processes are hosted in the same Docker container. Specifically, the VM runs within the Docker container and executes the server (VM Controller). You can refer to this diagram for the components involved. To achieve this, you will need to decouple the client and server, and run the server directly on your Windows host.
2. To run the benchmark tasks, you'll first need to set up your environment with the necessary tools and programs. The setup.ps1 PowerShell script provides what’s required to do so. You might want to generalize the script so it works both on your host and within the current Docker setup.
3. Next, you will need to run the Python server (VM Controller) component directly on your Windows host. To do this, create a new Conda environment on your Windows system and run pip install -r requirements from this requirements file.
4. The client process will need to point to the running py server on your localhost. You can find the references that need to be updated in the client here.
5. The user folder, where programs are installed and recordings are stored, will need to be abstracted to support both Docker and the host setup. The relevant references to update are here.

[Calvvnono]

Much appreciation for your guidance!

Another issue that one of my colleagues came across is that during the process of buliding WAA golden image, the terminal was stuck at displaying starting arena... and the 'localhost:8006' site also showing nothing. He didn't use the official Microsoft iso as README guides because it was too slow to download. Could this be the reason, or are there other possible causes?

[francedot]

Yes, I have seen this error before when the Windows 11 Enterprise Evaluation ISO (90-day trial) is not used. This is the only edition of Windows 11 that we support.

[Calvvnono]

Thank you again for your response, which helped us investigate the issue!
Actually after submitting the issue, we redownloaded the specified image and went through the deployment process from the beginning, but we got stuck at the same point again.
This is quite frustrating, as we barely have way to pinpoint where the issue lies. We’ve checked something we could think of, including KVM, Customizing resource, etc. Could you provide any other potential sources of the problem or share any valuable insights?

[francedot]

Please follow the same troubleshooting steps as outlined in this issue.

Let me know if you have any questions.

[Calvvnono]

You can troubleshoot any errors occurring during the preparation phase of the golden image (./run-local --prepare-image true), by looking at the logs under [src/win-arena-container/vm/setup/ps_script_log.txt] >

Yes, we tried to pinpoint the issue through logs, but the logs just didn't exist.

It seems that the preparation of the golden image just never really started, which is exactly what confuses us.

[francedot]

I have seen this happen only when the wrong ISO edition is provided as setup.iso. In this case, the automated preparation step (and corresponding logs) will never start through the unattended file here. Please make sure to pick the correct ISO edition here and start fresh by deleting the src/win-arena-container/vm/storage content before running the preparation step again.

[MovLab2]

I am having the same problem as Calvvnono . I tried the image you had suggested also. Wsl and docker desktop with conda. windowsarena/winarena:latest -c './entry.sh --prepare-image true --start-client true --agent navi --model gpt-4.o --som-origin oss --a11y-backend uia --gpu-enabled true'
Starting WinArena...
It just sits there... Any idea? windows 10.

2024-10-12 00:11:48 Starting WinArena... nothing else happening . this from log on docker desktop.

[Calvvnono]

I am having the same problem as Calvvnono . I tried the image you had suggested also. Wsl and docker desktop with conda. windowsarena/winarena:latest -c './entry.sh --prepare-image true --start-client true --agent navi --model gpt-4.o --som-origin oss --a11y-backend uia --gpu-enabled true' Starting WinArena... It just sits there... Any idea? windows 10.

2024-10-12 00:11:48 Starting WinArena... nothing else happening . this from log on docker desktop.

Try manually start it without gpu-enabled parameter, it worked out for me!
The issue seems to be down to there is no support for gpu-enabled parameter in entry.sh @francedot

[MovLab2]

I still can't start the VM no matter what I do. I tested the Interactive mode and was able to start the container and do the next trouble shooting step. It cant find the image and the location was never mounted, but the other 3 locations were. See code output.. Any Idea what next to try? yes a small system I use for testing .
(winarena) D:\WindowsAgentArena\scripts>bash ./run-local.sh --interactive true --ram-size 4G --cpu-cores 4 --gpu-enabled true --use-kvm false
Using configuration file: /mnt/d/WindowsAgentArena/config.json
Using mode: azure
Using VM Setup Image path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/image
Using VM storage mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage
Using server mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/setup
Using client mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/client
/dev/kvm not found. Setting use_kvm to false.
Invoking Docker Container with the command:
docker run -it --rm -p 8006:8006 -p 3390:3389 --name winarena --platform linux/amd64 -e KVM=N -e RAM_SIZE=4G -e CPU_CORES=4 -v /mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage/.:/storage -v /mnt/d/WindowsAgentArena/src/win-arena-container/vm/setup/.:/shared -v /mnt/d/WindowsAgentArena/src/win-arena-container/client/.:/client --cap-add NET_ADMIN --stop-timeout 120 --entrypoint /bin/bash --gpus all -e OPENAI_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx windowsarena/winarena:latest
root@3e72b869621e:/# ./start_vm.sh
❯ Starting Windows for Docker v0.0...
❯ For support visit https://github.com/dockur/windows
❯ CPU: AMD Ryzen 9 5900X 12 Core Processor | RAM: 6/8 GB | DISK: 609 GB (v9fs) | HOST: 5.15.153.1-microsoft-standard-WSL2...

Error: ISO file not found or is empty.
root@3e72b869621e:/#

Without GPU Enabled same thing \ windowsarena/winarena:latest -c './entry.sh --prepare-image true --start-client true --agent navi --model gpt-4-vision-preview --som-origin oss --a11y-backend uia --gpu-enabled false'
Starting WinArena...

[francedot]

I still can't start the VM no matter what I do. I tested the Interactive mode and was able to start the container and do the next trouble shooting step. It cant find the image and the location was never mounted, but the other 3 locations were. See code output.. Any Idea what next to try? yes a small system I use for testing . (winarena) D:\WindowsAgentArena\scripts>bash ./run-local.sh --interactive true --ram-size 4G --cpu-cores 4 --gpu-enabled true --use-kvm false Using configuration file: /mnt/d/WindowsAgentArena/config.json Using mode: azure Using VM Setup Image path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/image Using VM storage mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage Using server mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/setup Using client mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/client /dev/kvm not found. Setting use_kvm to false. Invoking Docker Container with the command: docker run -it --rm -p 8006:8006 -p 3390:3389 --name winarena --platform linux/amd64 -e KVM=N -e RAM_SIZE=4G -e CPU_CORES=4 -v /mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage/.:/storage -v /mnt/d/WindowsAgentArena/src/win-arena-container/vm/setup/.:/shared -v /mnt/d/WindowsAgentArena/src/win-arena-container/client/.:/client --cap-add NET_ADMIN --stop-timeout 120 --entrypoint /bin/bash --gpus all -e OPENAI_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx windowsarena/winarena:latest root@3e72b869621e:/# ./start_vm.sh ❯ Starting Windows for Docker v0.0... ❯ For support visit https://github.com/dockur/windows ❯ CPU: AMD Ryzen 9 5900X 12 Core Processor | RAM: 6/8 GB | DISK: 609 GB (v9fs) | HOST: 5.15.153.1-microsoft-standard-WSL2...

Error: ISO file not found or is empty. root@3e72b869621e:/#

Without GPU Enabled same thing \ windowsarena/winarena:latest -c './entry.sh --prepare-image true --start-client true --agent navi --model gpt-4-vision-preview --som-origin oss --a11y-backend uia --gpu-enabled false' Starting WinArena...

Note that for the golden image preparation step, the only params that can be specified (the rest will ignored) are prepare-image and use-kvm (the latter will default to false if your host does not support KVM accelleration). In your case, it seems that the setup.iso (make sure to download from here) is not mounted as volume file (using the docker --mount parameter). I would recommend first making sure that the setup.iso file exists in src/win-arena-container/vm/image before running run-local.sh. If still facing issues, directly execute the logged docker run -it command including --mount type=bind,source=/mnt/d/WindowsAgentArena/src/win-arena-container/vm/image/setup.iso,target=/custom.iso (the related code is here).

Please let us know if this helps.

[MovLab2]

I still can't start the VM no matter what I do. I tested the Interactive mode and was able to start the container and do the next trouble shooting step. It cant find the image and the location was never mounted, but the other 3 locations were. See code output.. Any Idea what next to try? yes a small system I use for testing . (winarena) D:\WindowsAgentArena\scripts>bash ./run-local.sh --interactive true --ram-size 4G --cpu-cores 4 --gpu-enabled true --use-kvm false Using configuration file: /mnt/d/WindowsAgentArena/config.json Using mode: azure Using VM Setup Image path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/image Using VM storage mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage Using server mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/setup Using client mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/client /dev/kvm not found. Setting use_kvm to false. Invoking Docker Container with the command: docker run -it --rm -p 8006:8006 -p 3390:3389 --name winarena --platform linux/amd64 -e KVM=N -e RAM_SIZE=4G -e CPU_CORES=4 -v /mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage/.:/storage -v /mnt/d/WindowsAgentArena/src/win-arena-container/vm/setup/.:/shared -v /mnt/d/WindowsAgentArena/src/win-arena-container/client/.:/client --cap-add NET_ADMIN --stop-timeout 120 --entrypoint /bin/bash --gpus all -e OPENAI_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx windowsarena/winarena:latest root@3e72b869621e:/# ./start_vm.sh ❯ Starting Windows for Docker v0.0... ❯ For support visit https://github.com/dockur/windows ❯ CPU: AMD Ryzen 9 5900X 12 Core Processor | RAM: 6/8 GB | DISK: 609 GB (v9fs) | HOST: 5.15.153.1-microsoft-standard-WSL2...
Error: ISO file not found or is empty. root@3e72b869621e:/#
Without GPU Enabled same thing \ windowsarena/winarena:latest -c './entry.sh --prepare-image true --start-client true --agent navi --model gpt-4-vision-preview --som-origin oss --a11y-backend uia --gpu-enabled false' Starting WinArena...

Note that for the golden image preparation step, the only params that can be specified (the rest will ignored) are prepare-image and use-kvm (the latter will default to false if your host does not support KVM accelleration). In your case, it seems that the setup.iso (make sure to download from here) is not mounted as volume file (using the docker --mount parameter). I would recommend first making sure that the setup.iso file exists in src/win-arena-container/vm/image before running run-local.sh. If still facing issues, directly execute the logged docker run -it command including --mount type=bind,source=/mnt/d/WindowsAgentArena/src/win-arena-container/vm/image/setup.iso,target=/custom.iso (the related code is here).

Please let us know if this helps.

Hi I updated my windows to 11. I used this :
sudo docker run -it
--mount type=bind,source=/mnt/d/WindowsAgentArena/src/win-arena-container/vm/image/setup.iso,target=/custom.iso
--mount type=bind,source=/mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage,target=/storage
--env RAM_SIZE=4G
--env VERSION=win11
--cap-add NET_ADMIN
--cap-add SYS_ADMIN
--device /dev/kvm
-p 8006:8006
sha256:51883b624551b260425fd422de8922000f879279ac1fcdbb2f7a765c6bb295d2 ./start_vm.sh
and i got

It just can't get past this looking for drivers. this was from the windows site. When i started from scratch i had got the same Starting WinArena... problem so i did what you suggested.
I did do a test and made a simple windows 11 container and that was fine running in it. Thanks for the help. 👍

[MovLab2]

I still can't start the VM no matter what I do. I tested the Interactive mode and was able to start the container and do the next trouble shooting step. It cant find the image and the location was never mounted, but the other 3 locations were. See code output.. Any Idea what next to try? yes a small system I use for testing . (winarena) D:\WindowsAgentArena\scripts>bash ./run-local.sh --interactive true --ram-size 4G --cpu-cores 4 --gpu-enabled true --use-kvm false Using configuration file: /mnt/d/WindowsAgentArena/config.json Using mode: azure Using VM Setup Image path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/image Using VM storage mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage Using server mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/vm/setup Using client mount path: /mnt/d/WindowsAgentArena/src/win-arena-container/client /dev/kvm not found. Setting use_kvm to false. Invoking Docker Container with the command: docker run -it --rm -p 8006:8006 -p 3390:3389 --name winarena --platform linux/amd64 -e KVM=N -e RAM_SIZE=4G -e CPU_CORES=4 -v /mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage/.:/storage -v /mnt/d/WindowsAgentArena/src/win-arena-container/vm/setup/.:/shared -v /mnt/d/WindowsAgentArena/src/win-arena-container/client/.:/client --cap-add NET_ADMIN --stop-timeout 120 --entrypoint /bin/bash --gpus all -e OPENAI_API_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx windowsarena/winarena:latest root@3e72b869621e:/# ./start_vm.sh ❯ Starting Windows for Docker v0.0... ❯ For support visit https://github.com/dockur/windows ❯ CPU: AMD Ryzen 9 5900X 12 Core Processor | RAM: 6/8 GB | DISK: 609 GB (v9fs) | HOST: 5.15.153.1-microsoft-standard-WSL2...
Error: ISO file not found or is empty. root@3e72b869621e:/#
Without GPU Enabled same thing \ windowsarena/winarena:latest -c './entry.sh --prepare-image true --start-client true --agent navi --model gpt-4-vision-preview --som-origin oss --a11y-backend uia --gpu-enabled false' Starting WinArena...

Note that for the golden image preparation step, the only params that can be specified (the rest will ignored) are prepare-image and use-kvm (the latter will default to false if your host does not support KVM accelleration). In your case, it seems that the setup.iso (make sure to download from here) is not mounted as volume file (using the docker --mount parameter). I would recommend first making sure that the setup.iso file exists in src/win-arena-container/vm/image before running run-local.sh. If still facing issues, directly execute the logged docker run -it command including --mount type=bind,source=/mnt/d/WindowsAgentArena/src/win-arena-container/vm/image/setup.iso,target=/custom.iso (the related code is here).

Please let us know if this helps.

Hi, I used the image you provided in this GitHub page and it worked. The image on Microsoft's website did not work, and i just did both just now. Your image is bigger in size too..
The code i used to start it was:

PS C:\Users\movla> wsl -d Ubuntu
movlab@DESKTOP-GAME:/mnt/c/Users/movla$ sudo docker run -it
--mount type=bind,source=/mnt/d/WindowsAgentArena/src/win-arena-container/vm/image/setup.iso,target=/custom.iso
--mount type=bind,source=/mnt/d/WindowsAgentArena/src/win-arena-container/vm/storage,target=/storage
--mount type=bind,source=/mnt/d/WindowsAgentArena/src/win-arena-container/vm/setup,target=/setup
--mount type=bind,source=/mnt/d/WindowsAgentArena/src/win-arena-container/client,target=/client
--env RAM_SIZE=4G
--env VERSION=win11
--cap-add NET_ADMIN
--cap-add SYS_ADMIN
--device /dev/kvm
-p 8006:8006
sha256:51883b624551b260425fd422de8922000f879279ac1fcdbb2f7a765c6bb295d2 ./start_vm.sh
[sudo] password for movlab:
❯ Starting Windows for Docker v0.0...
❯ For support visit https://github.com/dockur/windows
❯ CPU: AMD Ryzen 9 5900X 12 Core Processor | RAM: 6/8 GB | DISK: 497 GB (v9fs) | HOST: 5.15.153.1-microsoft-standard-WSL2...

❯ Extracting local ISO image...
❯ Detecting version from ISO image...
❯ Detected: Windows 11 Enterprise (Evaluation)
❯ Adding drivers to image...
❯ Adding OEM folder to image...
❯ Adding win11x64-enterprise-eval.xml for automatic installation...
❯ Building Windows 11 image...
❯ Creating a 30G growable disk image in raw format...
Cannot find device "dockerbridge"
❯ Booting Windows using QEMU v8.2.4...
BdsDxe: failed to load Boot0002 "UEFI QEMU QEMU HARDDISK " from PciRoot(0x0)/Pci(0xA,0x0)/Scsi(0x0,0x0): Not Found
BdsDxe: loading Boot0001 "UEFI QEMU QEMU CD-ROM " from PciRoot(0x0)/Pci(0x5,0x0)/Scsi(0x0,0x0)
BdsDxe: starting Boot0001 "UEFI QEMU QEMU CD-ROM " from PciRoot(0x0)/Pci(0x5,0x0)/Scsi(0x0,0x0)
❯ Windows has started successfully. You can connect using either a web browser or RDP, both accessible at localhost on the specified ports...
BdsDxe: loading Boot0004 "Windows Boot Manager" from HD(1,GPT,E5F1B558-748B-47BC-B6FB-1BE4CCDD92E9,0x800,0x40000)/\EFI\Microsoft\Boot\bootmgfw.efi
BdsDxe: starting Boot0004 "Windows Boot Manager" from HD(1,GPT,E5F1B558-748B-47BC-B6FB-1BE4CCDD92E9,0x800,0x40000)/\EFI\Microsoft\Boot\bootmgfw.efi
BdsDxe: loading Boot0004 "Windows Boot Manager" from HD(1,GPT,E5F1B558-748B-47BC-B6FB-1BE4CCDD92E9,0x800,0x40000)/\EFI\Microsoft\Boot\bootmgfw.efi
BdsDxe: starting Boot0004 "Windows Boot Manager" from HD(1,GPT,E5F1B558-748B-47BC-B6FB-1BE4CCDD92E9,0x800,0x40000)/\EFI\Microsoft\Boot\bootmgfw.efi
BdsDxe: loading Boot0004 "Windows Boot Manager" from HD(1,GPT,E5F1B558-748B-47BC-B6FB-1BE4CCDD92E9,0x800,0x40000)/\EFI\Microsoft\Boot\bootmgfw.efi
BdsDxe: starting Boot0004 "Windows Boot Manager" from HD(1,GPT,E5F1B558-748B-47BC-B6FB-1BE4CCDD92E9,0x800,0x40000)/\EFI\Microsoft\Boot\bootmgfw.efi

As you can see it built successful. When i tried the initial start again using " ./run-local.sh --prepare-image " it just hangs as it did from the beginning. Note, I am using windows 11 home edition. It does not support Hyper V. Maybe it's not starting the VM part in your code? I'm not sure, but I wanted to show you this for evaluation.

[francedot]

Judging from your latest screenshot, the VM process has started successfully. The Docker container will shut down automatically once all the automated steps (such as installing Python packages, tools and programs) are completed, as shown in this video walkthrough. This process can take 20-30 minutes, depending on your hardware and internet speed. If it seems to be stuck, please follow the troubleshooting steps outlined in this issue and post the logs here.