I finally figured out how to write German Umlaute on an US keyboard, after studying the Windows 10 documentation for a while.
tl;dr Switch your keyboard from ENG (US) or (UK) to ENG (International). Then type quotationmarks + the letter you want to change into an Umlaut.
"+a = ä
"+A = Ä
"+o = ö
"+O = Ö
"+u = ü
"+U = Ü
ALT+s = ß
Go to Settings and choose Time&Language
Add a language
Choose English
English (United States)
Go back to the screen from step 2 and click on options
Add a keyboard
Choose United States International. You can then go back, make this language your default and delete your previous language pack.