Document and recommend VMs for local setup and manual testing and debugging. Especially crucial for debugging.

[cfarm]

• Schedule a meeting where we all braindump our knowledge into this thread.
• Edit and write up the docs and make it easy to find in a central place

[anselmbradford]

I have this bookmarked and use it quite often https://github.com/xdissent/ievms. Only reason to re-run it is some Windows VMs require activation after 30 days.

[cfarm]

Similar documentation for testing manually in Sauce Labs is also a need

[cfarm]

This is what I used for IE VMs https://developer.microsoft.com/en-us/microsoft-edge/

[jimmynotjim]

Tip for IE VMs, after installing the VM and running the windows updates, create a snapshot in Virtual Box so you can rollback.

[anselmbradford]

Sauce instructions (note, says Sauce gulp command is not working, but that has been fixed):
https://github.com/cfpb/cfgov-refresh/blob/master/docs/testing.md#sauce-connect---send-tests-to-the-cloud

[anselmbradford]

Updated Sauce testing settings in cfpb/consumerfinance.gov#3461

[cfarm]

@marcesher posted this new Sauce Labs service in chat - private mobile device lab testing, seems especially useful for manual debugging https://saucelabs.com/enterprise/private-device-cloud

[Scotchester]

Instructions for setting up Microsoft-provided Windows VMs for local IE/Edge testing

Microsoft officially makes VMs available for testing IE and Edge at https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/. They can be downloaded one at a time from that site and set up individually, but there is a GitHub repository that provides a script for mass installation: https://github.com/xdissent/ievms.

Instructions follow for individual installations, because my original run of IEVMS_VERSIONS="10 EDGE" failed due to the 10 VM needing the REUSE_WIN7="no" option on my machine.

What worked for me for Edge

1. Log off of all VPNs for maximum bandwidth
2. brew install unar (see "Notes and further explanations")
3. curl -s https://raw.githubusercontent.com/xdissent/ievms/master/ievms.sh | env IEVMS_VERSIONS="EDGE" bash
4. When it finishes, open VirtualBox and start the newly-created "MSEdge - Win10" VM
5. Visit http://10.0.2.2:8000/ to access a server running on localhost:8000

What worked for me for IE 10

1. Log off of all VPNs for maximum bandwidth
2. brew install unar (if you didn't already)
3. curl -s https://raw.githubusercontent.com/xdissent/ievms/master/ievms.sh | env IEVMS_VERSIONS="10" REUSE_WIN7="no" bash
4. When "Waiting for IE10 - Win8 to shutdown..." starts looping in your terminal:
   1. Open VirtualBox
   2. Right-click the newly created "IE 10 - Win8" VM and choose "Show"
   3. Click the "Desktop" tile to dismiss the Start Menu.
   4. Dismiss the dialog prompting you to install something-or-other that's Oracle-related
   5. You should see a script that was paused in a Command Prompt window now continue to run
   6. After it completes and a 30-second timer counts down, the machine will shut down and the ievms script in your terminal will complete
5. Open VirtualBox and start the "IE10 - Win8" VM
6. Visit http://10.0.2.2:8000/ to access a server running on localhost:8000

Notes and further explanations

• When attempting to brew install unar, I got an error saying

  Error: Xcode alone is not sufficient on Sierra.
  Install the Command Line Tools:
    xcode-select --install

  I ran the exact command given. It prompted me to accept a license agreement and did its thing. After that, I was able to run brew install unar successfully. If you have issues with this, you may need to contact Mac Engineering for assistance fixing your Xcode.

• I already had VMs for IE 8, 9, and 11, so I didn't test the full-blown, from-scratch run. I will try the full run this evening and post an update with my results.

• It's worth noting that other browsers can also be installed and tested in these VMs, if that's something you want to do.

[Scotchester]

How the full run went last night

curl -s https://raw.githubusercontent.com/xdissent/ievms/master/ievms.sh | env IEVMS_VERSIONS="8 9 10 11 EDGE" bash

• IE 8 - WinXP: Fully automated, no hitches

• IE 9 - Win7: Got stuck on a restart prompt. Selected Restart Later, then Install on Oracle Corporation install dialog (twice).

  

  

• IE 10 - Win7: Same as 9, except after installing IE10 (on VM originally based on IE9 version), it got stuck waiting for a shutdown again. Had to manually go in and shut it down for the script to continue.

• IE 11 - Win7: Same as 10.

• MSEdge - Win10: Fully automated, no hitches

[Scotchester]

I noticed on Friday that when I tried to run IE 7, it had not automatically run the updater to update IE9 to IE11, as I assumed it had. Need to investigate whether that's a bug or something I should've known to do.