Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Machine ID: Switch to using new tbot ssh-proxy-command by default. #41482

Merged
merged 22 commits into from
May 15, 2024

Conversation

strideynet
Copy link
Contributor

@strideynet strideynet commented May 13, 2024

Closes #28910

Switches the generated ssh_config to use the new ssh-proxy-command. Plans the deprecation of the old ssh proxy command to occur in v17. The new behavior is significantly more performant and less resource intense, and reduces the dependency on tsh.

The old behavior can be temporarily selected by setting TBOT_SSH_CONFIG_PROXY_COMMAND_MODE=legacy - this will be removed in v17.

In addition, I've made some minor changes to the new command to avoid it depending so explicitly on the complete BotConfig.

I will backport this change to v15, but flip the behaviour so that this mode is opted-in to rather than the default. This will allow select users to switch to the new behavior without waiting for v16.

Associated with #41463

changelog: Switches the default SSH proxying mode in Machine ID to the newer, more performant, version. If you use Machine ID and OpenSSH, you may need to adjust your configuration, see https://goteleport.com/docs/machine-id/reference/v16-upgrade-guide/ for more information.

@@ -62,17 +63,27 @@ type ProxySSHConfig struct {
// ProxySSH creates a local ssh proxy, dialing a node and transferring data through
// stdin and stdout, to be used as an OpenSSH/PuTTY proxy command.
func ProxySSH(ctx context.Context, proxyConfig ProxySSHConfig) error {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rosstimothy - I've made some minor changes here, perhaps worth your attention...

@@ -21,6 +21,6 @@ Host *.tele.aperture.labs tele.blackmesa.gov
# Flags for all tele.aperture.labs hosts except the proxy
Host *.tele.aperture.labs !tele.blackmesa.gov
Port 3022
ProxyCommand "/path/to/tbot" proxy --destination-dir=/test/dir --proxy-server=tele.blackmesa.gov:443 ssh --cluster=tele.aperture.labs %r@%h:%p
ProxyCommand "/path/to/tbot" ssh-proxy-command --destination-dir=/test/dir --proxy-server=tele.blackmesa.gov:443 --cluster=tele.aperture.labs --tls-routing=true --connection-upgrade=false --resume=true --user=%r --host=%h --port=%p
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rosstimothy / @espadolini - double check that tI'm not missing anything here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks about right to me. Whats the story for proxy templates? Does that require humans to manually tweak the config?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah right now the answer is "Manually modify the file" - I wanted to keep this PR a manageable size. What I'll do in another PR is allow them to be configured directly within the tbot.yaml and then they will be written to proxytemplates.yaml in the output destination.

@strideynet
Copy link
Contributor Author

I'm going to manually test this against:

  • TLS Routing OFF
  • TLS Routing ON / No ALPN Upgrade
  • TLS Routing ON / ALPN Upgrade

lib/tbot/config/bot.go Show resolved Hide resolved
lib/tbot/config/template_ssh_client.go Show resolved Hide resolved
@@ -21,6 +21,6 @@ Host *.tele.aperture.labs tele.blackmesa.gov
# Flags for all tele.aperture.labs hosts except the proxy
Host *.tele.aperture.labs !tele.blackmesa.gov
Port 3022
ProxyCommand "/path/to/tbot" proxy --destination-dir=/test/dir --proxy-server=tele.blackmesa.gov:443 ssh --cluster=tele.aperture.labs %r@%h:%p
ProxyCommand "/path/to/tbot" ssh-proxy-command --destination-dir=/test/dir --proxy-server=tele.blackmesa.gov:443 --cluster=tele.aperture.labs --tls-routing=true --connection-upgrade=false --resume=true --user=%r --host=%h --port=%p
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks about right to me. Whats the story for proxy templates? Does that require humans to manually tweak the config?

strideynet and others added 2 commits May 14, 2024 16:16
Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
lib/config/openssh/openssh.go Outdated Show resolved Hide resolved
lib/tbot/ssh_proxy.go Show resolved Hide resolved
lib/tbot/tbot.go Outdated Show resolved Hide resolved
lib/tbot/tbot.go Outdated Show resolved Hide resolved
tool/tbot/proxy.go Outdated Show resolved Hide resolved
strideynet and others added 6 commits May 15, 2024 08:49
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
…avitational/teleport into strideynet/switch-to-new-ssh-proxying
lib/config/openssh/openssh.go Outdated Show resolved Hide resolved
lib/config/openssh/openssh.go Outdated Show resolved Hide resolved
@strideynet
Copy link
Contributor Author

Test against my local cluster (with a TLS terminating L7LB and TLS routing)

➜  teleport git:(strideynet/switch-to-new-ssh-proxying) ✗ ssh -F ./tbot-user/ssh_config -vvv noah@macbook.leaf.tele.ottr.sh hostname
OpenSSH_9.6p1, LibreSSL 3.3.6
debug1: Reading configuration data ./tbot-user/ssh_config
debug1: ./tbot-user/ssh_config line 4: Applying options for *.leaf.tele.ottr.sh
debug1: ./tbot-user/ssh_config line 11: Applying options for *.leaf.tele.ottr.sh
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: channel_clear_timeouts: clearing
debug1: Executing proxy command: exec '/Users/noah/code/gravitational/teleport/build/tbot' ssh-proxy-command --destination-dir='/Users/noah/code/gravitational/teleport/tbot-user' --proxy-server='leaf.tele.ottr.sh:443' --cluster='leaf.tele.ottr.sh' --tls-routing --connection-upgrade --resume --user=noah --host=macbook.leaf.tele.ottr.sh --port=3022
debug1: identity file /Users/noah/code/gravitational/teleport/tbot-user/key type 0
debug1: certificate file /Users/noah/code/gravitational/teleport/tbot-user/key-cert.pub type 4
debug1: Local version string SSH-2.0-OpenSSH_9.6
debug1: Remote protocol version 2.0, remote software version Teleport
debug1: compat_banner: no match: Teleport
debug2: fd 5 setting O_NONBLOCK
debug2: fd 4 setting O_NONBLOCK
debug1: Authenticating to macbook.leaf.tele.ottr.sh:3022 as 'noah'
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,kex-strict-c-v00@openssh.com
debug2: host key algorithms: rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha256,kex-strict-s-v00@openssh.com
debug2: host key algorithms: rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com
debug2: ciphers ctos: aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr
debug2: ciphers stoc: aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-512
debug2: compression ctos: none
debug2: compression stoc: none
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug3: kex_choose_conf: will use strict KEX ordering
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512-cert-v01@openssh.com
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host certificate: ssh-rsa-cert-v01@openssh.com SHA256:JHEG1ZZMVpkBJxFERvij/jXoXtQ3cZrjv3YJQhbRf3A, serial 0 ID "" CA ssh-rsa SHA256:PiLJ10QUbrkyTs428O8KHuXXlaQqLigy8CLrxD9UH6A valid after 2024-01-15T10:54:05
debug2: Server host certificate hostname: b2210ff1-9b5b-4ef1-b542-41e7472f0deb.leaf.tele.ottr.sh
debug2: Server host certificate hostname: b2210ff1-9b5b-4ef1-b542-41e7472f0deb
debug2: Server host certificate hostname: macbook.leaf.tele.ottr.sh
debug2: Server host certificate hostname: macbook
debug2: Server host certificate hostname: localhost
debug2: Server host certificate hostname: 127.0.0.1
debug2: Server host certificate hostname: ::1
debug3: put_host_port: [macbook.leaf.tele.ottr.sh]:3022
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: checking without port identifier
debug3: record_hostkey: found ca key type RSA in file /Users/noah/code/gravitational/teleport/tbot-user/known_hosts:1
debug3: load_hostkeys_file: loaded 1 keys from macbook.leaf.tele.ottr.sh
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'macbook.leaf.tele.ottr.sh' is known and matches the RSA-CERT host certificate.
debug1: Found CA key in /Users/noah/code/gravitational/teleport/tbot-user/known_hosts:1
debug1: found matching key w/out port
debug3: send packet: type 21
debug1: ssh_packet_send2_wrapped: resetting send seqnr 3
debug2: ssh_set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: ssh_packet_read_poll2: resetting read seqnr 3
debug1: SSH2_MSG_NEWKEYS received
debug2: ssh_set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug3: kex_input_ext_info: extension server-sig-algs
debug1: kex_ext_info_client_parse: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-256,rsa-sha2-512,ssh-rsa,ssh-dss>
debug3: kex_input_ext_info: extension ping@openssh.com
debug1: kex_ext_info_check_ver: ping@openssh.com=<0>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug3: ssh_get_authentication_socket_path: path '/private/tmp/com.apple.launchd.TfRJzbgZMg/Listeners'
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: agent returned 3 keys
debug1: Will attempt key: /Users/noah/code/gravitational/teleport/tbot-user/key-cert.pub RSA-CERT SHA256:WWBmCgLC2GIUD6lb9cFO185B1wkx2Z0I00TvZ2t4wnU explicit
debug1: Will attempt key: noah@noahstride.co.uk ED25519 SHA256:p8EWTJtUix2aJCx6tiMGPgFFhrLGZ2UUErrlo/QHJZI agent
debug1: Will attempt key: teleport:tbot-proving-ground.teleport.sh:tbot-proving-ground.teleport.sh:noah.stride+proving-ground@goteleport.com RSA-CERT SHA256:tZVSb3UuWeavho9PbdKMihXSMF+PlZkD/lYuEz84B7s agent
debug1: Will attempt key: teleport:tbot-proving-ground.teleport.sh:tbot-proving-ground.teleport.sh:noah.stride+proving-ground@goteleport.com RSA SHA256:tZVSb3UuWeavho9PbdKMihXSMF+PlZkD/lYuEz84B7s agent
debug1: Will attempt key: /Users/noah/code/gravitational/teleport/tbot-user/key RSA SHA256:WWBmCgLC2GIUD6lb9cFO185B1wkx2Z0I00TvZ2t4wnU explicit
debug2: pubkey_prepare: done
debug1: Offering public key: /Users/noah/code/gravitational/teleport/tbot-user/key-cert.pub RSA-CERT SHA256:WWBmCgLC2GIUD6lb9cFO185B1wkx2Z0I00TvZ2t4wnU explicit
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: /Users/noah/code/gravitational/teleport/tbot-user/key-cert.pub RSA-CERT SHA256:WWBmCgLC2GIUD6lb9cFO185B1wkx2Z0I00TvZ2t4wnU explicit
debug3: sign_and_send_pubkey: using publickey with RSA-CERT SHA256:WWBmCgLC2GIUD6lb9cFO185B1wkx2Z0I00TvZ2t4wnU
debug2: sign_and_send_pubkey: using private key "/Users/noah/code/gravitational/teleport/tbot-user/key" for certificate
debug3: sign_and_send_pubkey: signing using rsa-sha2-512-cert-v01@openssh.com SHA256:WWBmCgLC2GIUD6lb9cFO185B1wkx2Z0I00TvZ2t4wnU
debug3: send packet: type 50
debug3: receive packet: type 52
Authenticated to macbook.leaf.tele.ottr.sh (via proxy) using "publickey".
debug1: channel 0: new session [client-session] (inactive timeout: 0)
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Entering interactive session.
debug1: pledge: proc
debug3: client_repledge: enter
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: client_session2_setup: id 0
debug1: Sending command: hostname
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug3: client_repledge: enter
debug1: pledge: fork
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 2097152 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
Noahs-MBP.net.stellar.haus
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug3: receive packet: type 97
debug2: channel 0: rcvd close
debug2: channel 0: output open -> drain
debug2: chan_shutdown_read: channel 0: (i0 o1 sock -1 wfd 3 efd 7 [write])
debug2: channel 0: input open -> closed
debug3: channel 0: will not send data after close
debug2: channel 0: obuf empty
debug2: chan_shutdown_write: channel 0: (i3 o1 sock -1 wfd 6 efd 7 [write])
debug2: channel 0: output drain -> closed
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug3: send packet: type 97
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 [session] r0 i3/0 o3/0 e[write]/0 fd -1/-1/7 sock -1 cc -1 io 0x00/0x00)

debug3: send packet: type 1
Transferred: sent 4720, received 4216 bytes, in 2.3 seconds
Bytes per second: sent 2051.4, received 1832.3
debug1: Exit status 0
➜  teleport git:(strideynet/switch-to-new-ssh-proxying) ✗ time ssh -F ./tbot-user/ssh_config noah@macbook.leaf.tele.ottr.sh hostname
Noahs-MBP.net.stellar.haus
ssh -F ./tbot-user/ssh_config noah@macbook.leaf.tele.ottr.sh hostname  0.01s user 0.00s system 1% cpu 0.611 total

And using the TBOT_SSH_CONFIG_PROXY_COMMAND_MODE=legacy env:

➜  teleport git:(strideynet/switch-to-new-ssh-proxying) ✗ time ssh -F ./tbot-user/ssh_config noah@macbook.leaf.tele.ottr.sh hostname
2024-05-15T13:51:38+01:00 WARN [TBOT]      `tbot proxy ssh` is deprecated and will stop working in v17. See https://goteleport.com/docs/machine-id/reference/v16-upgrade-guide/ tbot/proxy.go:79
Noahs-MBP.net.stellar.haus
ssh -F ./tbot-user/ssh_config noah@macbook.leaf.tele.ottr.sh hostname  0.01s user 0.00s system 0% cpu 5.582 total

Comment on lines +41 to +46
s = `'` + strings.ReplaceAll(s, `'`, `'"'"'`) + `'`
// escape any percent signs which could trigger the percent expansion
// for ProxyCommand.
s = strings.ReplaceAll(s, `%`, `%%`)
// escape any newlines which could impact the parsing of ssh_config
s = strings.ReplaceAll(s, "\n", `'"\n"'`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we use safetext here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We couldn't find a sensible way to then go back and replace literal newlines with their escaped form with a shsprintf.DefaultWhatever - which we need because we're trying to build a string that, when fed through openssh's config parsing, percent replacing and execution through a shell, results in the process we want being executed with the arguments that we want.

@strideynet
Copy link
Contributor Author

Tested against a Cloud tenant (TLS routing on, but no need for connection upgrade)

➜  teleport git:(strideynet/switch-to-new-ssh-proxying) ✗ time ssh -F ./tbot-user/ssh_config -vvv root@noah-test.noah.teleport.sh hostname
OpenSSH_9.6p1, LibreSSL 3.3.6
debug1: Reading configuration data ./tbot-user/ssh_config
debug1: ./tbot-user/ssh_config line 4: Applying options for *.noah.teleport.sh
debug1: ./tbot-user/ssh_config line 11: Applying options for *.noah.teleport.sh
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: channel_clear_timeouts: clearing
debug1: Executing proxy command: exec '/Users/noah/code/gravitational/teleport/build/tbot' ssh-proxy-command --destination-dir='/Users/noah/code/gravitational/teleport/tbot-user' --proxy-server='noah.teleport.sh:443' --cluster='noah.teleport.sh' --tls-routing --no-connection-upgrade --resume --user=root --host=noah-test.noah.teleport.sh --port=3022
debug1: identity file /Users/noah/code/gravitational/teleport/tbot-user/key type 0
debug1: certificate file /Users/noah/code/gravitational/teleport/tbot-user/key-cert.pub type 4
debug1: Local version string SSH-2.0-OpenSSH_9.6
debug1: Remote protocol version 2.0, remote software version Teleport
debug1: compat_banner: no match: Teleport
debug2: fd 5 setting O_NONBLOCK
debug2: fd 4 setting O_NONBLOCK
debug1: Authenticating to noah-test.noah.teleport.sh:3022 as 'root'
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c,kex-strict-c-v00@openssh.com
debug2: host key algorithms: rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha256
debug2: host key algorithms: rsa-sha2-256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com
debug2: ciphers ctos: aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr
debug2: ciphers stoc: aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha2-256,hmac-sha2-512
debug2: compression ctos: none
debug2: compression stoc: none
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512-cert-v01@openssh.com
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host certificate: ssh-rsa-cert-v01@openssh.com SHA256:U1YfVGYohNFxl/xUQ/fXXNpDkqFTillbNeNUbhAbFTg, serial 0 ID "" CA ssh-rsa SHA256:6pvfRdCRQ/2MxPWLAjBRpfloJGaHDORBsI/FMnUv5RA valid after 2022-09-29T12:16:14
debug2: Server host certificate hostname: 641d43ef-257c-4e21-b875-07d2de6a3b14.noah.teleport.sh
debug2: Server host certificate hostname: 641d43ef-257c-4e21-b875-07d2de6a3b14
debug2: Server host certificate hostname: noah-test.noah.teleport.sh
debug2: Server host certificate hostname: noah-test
debug2: Server host certificate hostname: localhost
debug2: Server host certificate hostname: 127.0.0.1
debug2: Server host certificate hostname: ::1
debug3: put_host_port: [noah-test.noah.teleport.sh]:3022
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: checking without port identifier
debug3: record_hostkey: found ca key type RSA in file /Users/noah/code/gravitational/teleport/tbot-user/known_hosts:1
debug3: load_hostkeys_file: loaded 1 keys from noah-test.noah.teleport.sh
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'noah-test.noah.teleport.sh' is known and matches the RSA-CERT host certificate.
debug1: Found CA key in /Users/noah/code/gravitational/teleport/tbot-user/known_hosts:1
debug1: found matching key w/out port
debug3: send packet: type 21
debug2: ssh_set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: ssh_set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug3: kex_input_ext_info: extension server-sig-algs
debug1: kex_ext_info_client_parse: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-256,rsa-sha2-512,ssh-rsa,ssh-dss>
debug3: kex_input_ext_info: extension ping@openssh.com
debug1: kex_ext_info_check_ver: ping@openssh.com=<0>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug3: ssh_get_authentication_socket_path: path '/private/tmp/com.apple.launchd.TfRJzbgZMg/Listeners'
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: agent returned 5 keys
debug1: Will attempt key: /Users/noah/code/gravitational/teleport/tbot-user/key-cert.pub RSA-CERT SHA256:KFcGwNJMMSOsmyC/BmcUhBT2923lcSZxSOOBFYq+MEM explicit
debug1: Will attempt key: noah@noahstride.co.uk ED25519 SHA256:p8EWTJtUix2aJCx6tiMGPgFFhrLGZ2UUErrlo/QHJZI agent
debug1: Will attempt key: teleport:tbot-proving-ground.teleport.sh:tbot-proving-ground.teleport.sh:noah.stride+proving-ground@goteleport.com RSA-CERT SHA256:tZVSb3UuWeavho9PbdKMihXSMF+PlZkD/lYuEz84B7s agent
debug1: Will attempt key: teleport:tbot-proving-ground.teleport.sh:tbot-proving-ground.teleport.sh:noah.stride+proving-ground@goteleport.com RSA SHA256:tZVSb3UuWeavho9PbdKMihXSMF+PlZkD/lYuEz84B7s agent
debug1: Will attempt key: teleport:noah.teleport.sh:noah.teleport.sh:noahstride RSA-CERT SHA256:/mdUTvYPKdg36VLIn+3bynS2xfevllH7Xp0+P3XP+BM agent
debug1: Will attempt key: teleport:noah.teleport.sh:noah.teleport.sh:noahstride RSA SHA256:/mdUTvYPKdg36VLIn+3bynS2xfevllH7Xp0+P3XP+BM agent
debug1: Will attempt key: /Users/noah/code/gravitational/teleport/tbot-user/key RSA SHA256:KFcGwNJMMSOsmyC/BmcUhBT2923lcSZxSOOBFYq+MEM explicit
debug2: pubkey_prepare: done
debug1: Offering public key: /Users/noah/code/gravitational/teleport/tbot-user/key-cert.pub RSA-CERT SHA256:KFcGwNJMMSOsmyC/BmcUhBT2923lcSZxSOOBFYq+MEM explicit
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: /Users/noah/code/gravitational/teleport/tbot-user/key-cert.pub RSA-CERT SHA256:KFcGwNJMMSOsmyC/BmcUhBT2923lcSZxSOOBFYq+MEM explicit
debug3: sign_and_send_pubkey: using publickey with RSA-CERT SHA256:KFcGwNJMMSOsmyC/BmcUhBT2923lcSZxSOOBFYq+MEM
debug2: sign_and_send_pubkey: using private key "/Users/noah/code/gravitational/teleport/tbot-user/key" for certificate
debug3: sign_and_send_pubkey: signing using rsa-sha2-512-cert-v01@openssh.com SHA256:KFcGwNJMMSOsmyC/BmcUhBT2923lcSZxSOOBFYq+MEM
debug3: send packet: type 50
debug3: receive packet: type 52
Authenticated to noah-test.noah.teleport.sh (via proxy) using "publickey".
debug1: channel 0: new session [client-session] (inactive timeout: 0)
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Entering interactive session.
debug1: pledge: proc
debug3: client_repledge: enter
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: client_session2_setup: id 0
debug1: Sending command: hostname
debug2: channel 0: request exec confirm 1
debug3: send packet: type 98
debug3: client_repledge: enter
debug1: pledge: fork
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 2097152 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: exec request accepted on channel 0
noah-test
debug3: receive packet: type 98
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug3: receive packet: type 97
debug2: channel 0: rcvd close
debug2: channel 0: output open -> drain
debug2: chan_shutdown_read: channel 0: (i0 o1 sock -1 wfd 3 efd 7 [write])
debug2: channel 0: input open -> closed
debug3: channel 0: will not send data after close
debug2: channel 0: obuf empty
debug2: chan_shutdown_write: channel 0: (i3 o1 sock -1 wfd 6 efd 7 [write])
debug2: channel 0: output drain -> closed
debug2: channel 0: almost dead
debug2: channel 0: gc: notify user
debug2: channel 0: gc: user detached
debug2: channel 0: send close
debug3: send packet: type 97
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: client-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 client-session (t4 [session] r0 i3/0 o3/0 e[write]/0 fd -1/-1/7 sock -1 cc -1 io 0x00/0x00)

debug3: send packet: type 1
Transferred: sent 4752, received 4176 bytes, in 0.4 seconds
Bytes per second: sent 13196.2, received 11596.6
debug1: Exit status 0
ssh -F ./tbot-user/ssh_config -vvv root@noah-test.noah.teleport.sh hostname  0.01s user 0.01s system 2% cpu 0.792 total

Copy link
Contributor

@timothyb89 timothyb89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working for me now!

@strideynet strideynet enabled auto-merge May 15, 2024 17:14
@strideynet strideynet added this pull request to the merge queue May 15, 2024
Merged via the queue into master with commit bd4dcab May 15, 2024
39 checks passed
@strideynet strideynet deleted the strideynet/switch-to-new-ssh-proxying branch May 15, 2024 17:48
@public-teleport-github-review-bot

@strideynet See the table below for backport results.

Branch Result
branch/v15 Failed

strideynet added a commit that referenced this pull request May 17, 2024
…41482)

* Add warning to previous command

* Hash out basics of new tbot proxying command use

* Avoid passing full botConfig into `tbot ssh-proxy-command`

* wip

* simplify impl

* Set up test suite for ssh config generation

* Add support for checking if ALPNConnUpgrade is required

* Cache the alpn upgrade result

* Americanize Spellings

* Update lib/tbot/config/template_ssh_client.go

Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>

* Add CHANGELOG.md entry for breaking change

* Update tool/tbot/proxy.go

Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>

* Update lib/tbot/tbot.go

Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>

* Switch to `--no` variants rather than `={bool}`

* Use SingleFlight on alpn proxy upgrade cache

* Improve singleflight

* Add rudimentary shell quoting

* Uniformly pad the templating

* Rename shellQuote -> proxyCommandQuote

* Fix imports

---------

Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
github-merge-queue bot pushed a commit that referenced this pull request May 17, 2024
…41482) (#41694)

* Add warning to previous command

* Hash out basics of new tbot proxying command use

* Avoid passing full botConfig into `tbot ssh-proxy-command`

* wip

* simplify impl

* Set up test suite for ssh config generation

* Add support for checking if ALPNConnUpgrade is required

* Cache the alpn upgrade result

* Americanize Spellings

* Update lib/tbot/config/template_ssh_client.go



* Add CHANGELOG.md entry for breaking change

* Update tool/tbot/proxy.go



* Update lib/tbot/tbot.go



* Switch to `--no` variants rather than `={bool}`

* Use SingleFlight on alpn proxy upgrade cache

* Improve singleflight

* Add rudimentary shell quoting

* Uniformly pad the templating

* Rename shellQuote -> proxyCommandQuote

* Fix imports

---------

Co-authored-by: rosstimothy <39066650+rosstimothy@users.noreply.github.com>
Co-authored-by: Edoardo Spadolini <edoardo.spadolini@goteleport.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

proxy ssh subcommand via tbot and tsh is too heavy
4 participants