Pretty simple. You just use it for whatever letter you want to umlaut. A, E, I, O, U or Y (uppercase or lowercase).
Like this:
Pre{{umlaut|e}}mptive co{{umlaut|o}}peration
Preëmptive coöperation
