Edit: This is apparently required when the proxy is not set properly. When using the app build environment make sure to unselect the option to use the proxy for forwarding if you have no proxy.
I was able to get this working. Here were the steps I took (may or may not be required).
Under the SSH settings I put a path to a config file so that I could specify the port.
In that file I had the following content
I also changed the Local Server download so that the vscode server was downloaded locally and then pushed to the virtual core.
I then closed and reopend VSCode and after pressing F1 I selected/typed the SSH-remote connection and selected the connection configured in the ssh.config file
From there select linux and input the password.