Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

repeated errors on ssh attempts #45256

Closed
stevenGravy opened this issue Aug 8, 2024 · 6 comments · Fixed by #47351
Closed

repeated errors on ssh attempts #45256

stevenGravy opened this issue Aug 8, 2024 · 6 comments · Fixed by #47351
Labels
bug error-msg Improving customer facing error messages. ux

Comments

@stevenGravy
Copy link
Contributor

Expected behavior:

The error messages wouldn't repeat.

Current behavior:

In the CLI or web ui you get the same error messages repeated when a SSH connection isn't available.

disconnected
Teleport proxy failed to connect to "node" agent "@local-node" over reverse tunnel:

  no tunnel connection found: no node reverse tunnel for 44444-b3ce-4efb-444-a77130b076f5.example.teleport.sh found, error dialing peer proxies [44444-a1b1-449b-4444-ca54d4552c6e 7ed20621-d038-4c90-8948-28fddbfaf06b]: error receiving dial response: Teleport proxy failed to connect to "node" agent "@local-node" over reverse tunnel:

  no tunnel connection found: no node reverse tunnel for 4444-4444-4efb-a6ad-a77130b076f5.example.teleport.sh found

This usually means that the agent is offline or has disconnected. Check the
agent logs and, if the issue persists, try restarting it or re-registering it
with the cluster., error receiving dial response: Teleport proxy failed to connect to "node" agent "@local-node" over reverse tunnel:

  no tunnel connection found: no node reverse tunnel for 4444-b3ce-4efb-444-a77130b076f5.example.teleport.sh found

This usually means that the agent is offline or has disconnected. Check the
agent logs and, if the issue persists, try restarting it or re-registering it
with the cluster.

This usually means that the agent is offline or has disconnected. Check the
agent logs and, if the issue persists, try restarting it or re-registering it
with the cluster.

Bug details:

  • Teleport version: 16.1.1
  • Recreation steps
    1. register a SSH server in cloud
    2. shutdown the ssh service and attempt to connect.
@stevenGravy stevenGravy added bug ux error-msg Improving customer facing error messages. labels Aug 8, 2024
@rosstimothy
Copy link
Contributor

Related to #31517

@zmb3
Copy link
Collaborator

zmb3 commented Oct 7, 2024

@rosstimothy is this fixed?

@rosstimothy
Copy link
Contributor

I closed #31517 because I was never able to reproduce. @stevenGravy if you have a reliable way to reproduce this can you please share it so we can investigate?

@stevenGravy
Copy link
Contributor Author

stevenGravy commented Oct 7, 2024 •
edited
Loading

I closed #31517 because I was never able to reproduce. @stevenGravy if you have a reliable way to reproduce this can you please share it so we can investigate?

Yes, it's happening with a cloud instance that's on 16.4.2 with a Teleport SSH service on 16.4.2. Same behavior. I started the SSH service connected to the cloud. Stopped it. Attempted to connect and you get the double errors.

@rosstimothy
Copy link
Contributor

The source of the duplicate message seems to stem from Proxy Peering. I've been able to repro and should be able to clean up some of the duplicate information being presented.

@rosstimothy
Copy link
Contributor

As suspected, we are combining errors from attempting to connect to the host directly via a reverse tunnel, and via a peers reverse tunnel: https://github.com/gravitational/teleport/blob/master/lib/reversetunnel/localsite.go#L613.

rosstimothy added a commit that referenced this issue Oct 8, 2024
@rosstimothy
Improve error messaging when a connections to nodes fail
9e251c8 
Prevents duplicate information about missing tunnels preventing a
connection when attempting to connect locally and via a proxy peer.
The error from the remote dial is now omitted from the message
provided to end users. The information about how the connection
was attempted was instead updated to reflect that tunnels were
attempted via both the local proxy and a peer and they both
failed. Additionally, if a hostname was provided in the dial
request, it will be included in the error message instead of
the internal @local-node identifier for tunnels or the public
address for direct dial nodes.

Closes #45256.
@rosstimothy rosstimothy mentioned this issue Oct 8, 2024
Merged
github-merge-queue bot pushed a commit that referenced this issue Oct 8, 2024
@rosstimothy
Improve error messaging when a connections to nodes fail (#47351)
b2de7f2 
Prevents duplicate information about missing tunnels preventing a
connection when attempting to connect locally and via a proxy peer.
The error from the remote dial is now omitted from the message
provided to end users. The information about how the connection
was attempted was instead updated to reflect that tunnels were
attempted via both the local proxy and a peer and they both
failed. Additionally, if a hostname was provided in the dial
request, it will be included in the error message instead of
the internal @local-node identifier for tunnels or the public
address for direct dial nodes.

Closes #45256.
github-actions bot pushed a commit that referenced this issue Oct 8, 2024
@rosstimothy
Improve error messaging when a connections to nodes fail
2afdf76 
Prevents duplicate information about missing tunnels preventing a
connection when attempting to connect locally and via a proxy peer.
The error from the remote dial is now omitted from the message
provided to end users. The information about how the connection
was attempted was instead updated to reflect that tunnels were
attempted via both the local proxy and a peer and they both
failed. Additionally, if a hostname was provided in the dial
request, it will be included in the error message instead of
the internal @local-node identifier for tunnels or the public
address for direct dial nodes.

Closes #45256.
github-actions bot pushed a commit that referenced this issue Oct 8, 2024
@rosstimothy
Improve error messaging when a connections to nodes fail
cab4eac 
Prevents duplicate information about missing tunnels preventing a
connection when attempting to connect locally and via a proxy peer.
The error from the remote dial is now omitted from the message
provided to end users. The information about how the connection
was attempted was instead updated to reflect that tunnels were
attempted via both the local proxy and a peer and they both
failed. Additionally, if a hostname was provided in the dial
request, it will be included in the error message instead of
the internal @local-node identifier for tunnels or the public
address for direct dial nodes.

Closes #45256.
github-actions bot pushed a commit that referenced this issue Oct 8, 2024
@rosstimothy
Improve error messaging when a connections to nodes fail
66c8bab 
Prevents duplicate information about missing tunnels preventing a
connection when attempting to connect locally and via a proxy peer.
The error from the remote dial is now omitted from the message
provided to end users. The information about how the connection
was attempted was instead updated to reflect that tunnels were
attempted via both the local proxy and a peer and they both
failed. Additionally, if a hostname was provided in the dial
request, it will be included in the error message instead of
the internal @local-node identifier for tunnels or the public
address for direct dial nodes.

Closes #45256.
github-merge-queue bot pushed a commit that referenced this issue Oct 9, 2024
@rosstimothy
Improve error messaging when a connections to nodes fail (#47363)
3db8e57 
Prevents duplicate information about missing tunnels preventing a
connection when attempting to connect locally and via a proxy peer.
The error from the remote dial is now omitted from the message
provided to end users. The information about how the connection
was attempted was instead updated to reflect that tunnels were
attempted via both the local proxy and a peer and they both
failed. Additionally, if a hostname was provided in the dial
request, it will be included in the error message instead of
the internal @local-node identifier for tunnels or the public
address for direct dial nodes.

Closes #45256.
github-merge-queue bot pushed a commit that referenced this issue Oct 9, 2024
@rosstimothy
Improve error messaging when a connections to nodes fail (#47361)
3ddcff1 
Prevents duplicate information about missing tunnels preventing a
connection when attempting to connect locally and via a proxy peer.
The error from the remote dial is now omitted from the message
provided to end users. The information about how the connection
was attempted was instead updated to reflect that tunnels were
attempted via both the local proxy and a peer and they both
failed. Additionally, if a hostname was provided in the dial
request, it will be included in the error message instead of
the internal @local-node identifier for tunnels or the public
address for direct dial nodes.

Closes #45256.
github-merge-queue bot pushed a commit that referenced this issue Oct 9, 2024
@rosstimothy
Improve error messaging when a connections to nodes fail (#47362)
590988f 
Prevents duplicate information about missing tunnels preventing a
connection when attempting to connect locally and via a proxy peer.
The error from the remote dial is now omitted from the message
provided to end users. The information about how the connection
was attempted was instead updated to reflect that tunnels were
attempted via both the local proxy and a peer and they both
failed. Additionally, if a hostname was provided in the dial
request, it will be included in the error message instead of
the internal @local-node identifier for tunnels or the public
address for direct dial nodes.

Closes #45256.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug error-msg Improving customer facing error messages. ux
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Improve error messaging when connections to nodes fail gravitational/teleport
3 participants
@zmb3 @rosstimothy @stevenGravy