With Tailscale SSH you can do exactly that. Tailscale gives each server and user device a node key to connect to its VPN, and uses the same node key for SSH authorization and encryption. Basically, you’re SSHing the same way you manage app access.
What’s the benefit? Built-in key rotation, permissions as code, connectivity between any two devices, reduced latency… (there’s more). You can also ask users to re-authenticate for that extra bit of security. Sounds expensive? Nope: Tailscale is completely free for personal use on up to 20 devices. Learn more, read docs, or get started