Working with a WaaS, or Workstation As A Service, can be either thrilling or nightmarish, depending on the reliability of the link between the thin client and workstation. The more mobile the workplace, the more important the tools' ability to handle disconnects.
There's no magic bullet for handling network issues. But, over time, developers have figured out solutions which take the edge off. Some of the best are offline support, persistent sessions and predictive typing.
Standard ssh uses none of these techniques. It offers the bare minimum when it comes to operating remote shells. Best it can do is regularly send a keep-alive ping to the server, so it doesn't auto close the connection when the user is idle.
There's a variety of alternative tools that can ssh to a server and also deal with unreliable connections. One is mosh. It's a well know ssh replacement, which can persist shell sessions and predict how the terminal will change during hight latency moments. With mosh, it's possible to start a long-running program on the server using a laptop, close the laptop, put it an a bag, travel for five hours, then open the laptop and still see the program running in the foreground. It's amazingly simple to switch to mosh and it doesn't require any configuration to automatically resume sessions when internet comes back up. It does it transparently.
It's an opinionated tool, though. Mosh does a few things differently than other shells. Two worth noting are the inability to scroll up the terminal window on macOS (irritating) and no agent forwarding. Scroll in mosh is remapped to browsing the command history. Agent forwarding is simply not there, because mosh tries to focus on reliability and security. Agent forwarding is treated as a unnecessary complication to the mosh codebase.
Eternal Terminal (ET) solves scrolling on macOS but loses predictive typing. Overall, it's more comfortable to use than mosh on a daily basis.
Mosh and ET allow opening multiple ssh sessions, and thus terminal windows, at once. It's also possible to have a server program manage them. This is where tmux comes in. It emulates windows in a single terminal window. windowception! However, it may be overkill for some developers, as managing tmux windows requires learning new shortcuts. And if tmux dies, it may take all the shell sessions with it.
There's also a bunch of terminals for Android and iOS. While using a mobile terminal is not the most comfortable, it may be helpful to setup in case of emergencies.
Easing The Switch To Remote Shell
Using mosh and ET is as simple as writing mosh <ssh config> or et <ssh config> in the terminal. Of course, the devil lies in <ssh config>. At a minimum, it's going to contain -i <path to private key>, host and username. To avoid writing all of these, it's possible to save them inside ~/.ssh/config. Both, mosh and ET, rely on the stanard ssh config file for resolving servers.
Writing:
Host dev
HostName 123.456.789.101
User root
IdentityFile "/Users/me/.ssh/myprivatekey"
ServerAliveInterval 60
ForwardAgent yes # Improves security and convienience of using a remote by forwarding private SSH keys to the remote machine. With this setting, the same keys are available for use inside the remote as on the local machine. No need to copy keys to the remote.
allows connecting to 123.456.789.101 as root by running mosh/et dev, a much shorter alternative to providing all the settings every time. Note: ServerAliveInternal and ForwardAgent are only really used when doing ssh dev. mosh and ET don't benefit from these two settings.
Suplement On Agent Forwarding
For having a truly independent remote server, agent forwarding isn't very convenient anyway. A more convenient, albeit less secure option, is to create new keys for the remote for services that it's supposed to access and save those in the remote. That adds convenience and still some level of security (as long as the new keys are only used by the remote and nowhere else). In case of a security breach, blocking access from the remote to services is straightforward and comes down to blocking keys used exclusively by the remote.
Troubleshooting Tips
The versions of mosh and ET used locally and remotely should be the same. An example of a issue resulting from breaking this rule is not being able to fully stop et after closing the ssh session.