Code access troubleshooting

# Are your SSH keys correctly set up?

• Have you setup SSH keys and uploaded the public part to fortrabbit?
• Have you been able to access code before?
• Is the same key you are using now here working elsewhere?

See our SSH key setup article if you are unsure about your SSH keys.

# Use verbose mode to see what's going on

Add the -v (verbose) flag to your login command. The output from this command is useful for troubleshooting issues.

$ ssh {{app-env-id}}@ssh.{{region}}.frbit.app -v

# OpenSSH_8.4p1, OpenSSL 1.1.1i  8 Dec 2022
# debug1: Reading configuration data /home/user/.ssh/config
# debug1: /home/user/.ssh/config line 6: Applying options for *
# debug1: Reading configuration data /etc/ssh/ssh_config
# debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
# … more lines here

# Consider rate limiting

We employ IP-based connection restrictions to block brute-force SSH login attempts and abusive SSH service over-usage. This may apply to you when trying to

• repetitively connect with wrong SSH credentials
• connect many times by SSH in a short period of time
• open too many connections at the same time (max: 15 concurrent connections)

This can even be something passive happening in the background. Think of a MySQL client that is opening many tunnel connections, or an SFTP plugin for your IDE.

The restrictions are short-term and are automatically removed within a few minutes after the suspicious usage stops! E.g., if the IDE is stuck in a loop trying to open 100 connections every minute again and again, the restrictions for the IP will be removed only after the misbehaving IDE is terminated/restarted.

We don't put customers' IPs on the static allowlist.

# How to identify possible rate limiting

It's likely not rate limiting when you have never been able to connect by SSH. It's more likely rate limiting, when the same type of SSH connection worked before.

One way to identify IP-based restrictions is to try with a different public IP from a different network (for instance, by tethering through your phone) or eventually by restarting your router. The other way is to investigate the problem over the CLI ssh client.

# Case 1: Error in SSH protocol version identification

You might get a relevant error message through the SSH protocol identification:

$ ssh {{app-env-id}}@ssh.{{region}}.frbit.app
Bad remote protocol version identification: 'SSH-Restricted_TooManyConcurrentConnections'
banner exchange: Connection to X.X.X.X port 22: invalid format

In such a case, you can also enable verbose logging (-v) to see some more details on banner lines:

$ ssh {{app-env-id}}@ssh.{{region}}.frbit.app -v
...
debug1: Local version string SSH-2.0-OpenSSH_10.0
debug1: kex_exchange_identification: banner line 0:
debug1: kex_exchange_identification: banner line 1: ******************************************************
debug1: kex_exchange_identification: banner line 2:        SSH connection rejected due to overusage!
debug1: kex_exchange_identification: banner line 3:
debug1: kex_exchange_identification: banner line 4:   Reason:    TooManyConcurrentConnections
debug1: kex_exchange_identification: banner line 5:   Active:    16 concurrent connections (limit: 15)
debug1: kex_exchange_identification: banner line 6:   Rate:      10 connections per minute (limit: 100)
debug1: kex_exchange_identification: banner line 7:   Client IP: Y.Y.Y.Y
debug1: kex_exchange_identification: banner line 8: ******************************************************
debug1: kex_exchange_identification: banner line 9:
Bad remote protocol version identification: 'SSH-Restricted_TooManyConcurrentConnections'
banner exchange: Connection to X.X.X.X port 22: invalid format

The restrictions in this case are automatically removed once the number of concurrent connections and/or connection rate are below the limits.

# Case 2: Connection closed

If there is no such protocol error and instead the server immediately closes the connection, your IP might be restricted after too many failed SSH authentications.

$ ssh {{app-env-id}}@ssh.{{region}}.frbit.app
Connection closed by X.X.X.X port 22

Wait a few minutes without trying to reach our service. After a few minutes of inactivity, the restrictions are removed.

# Case 3: Permission denied

If you get generic permission denied error, your IP is not restricted (yet), but you are trying with the wrong SSH credentials:

$ ssh {{app-env-id}}@ssh.{{region}}.frbit.app
Permission denied (publickey).

# Check your local SSH configuration

Open the files ~/.ssh/config and ~/.ssh/known_hosts and see if there is anything unusual. Delete all entries related to fortrabbit services to reset your local configuration.

# Access error for recently created environment

If an environment was recently created, please wait about 5 minutes for it to become ready. Trying to connect before it is ready produces an error. If more than 10 minutes have passed and you still get a similar error (despite successful authentication), then please contact us.

# Access error with recently created SSH key

After importing a key in the dashboard, please allow up to 5 minutes for it to be activated.

# You are asked for a password

If you face a password prompt it can mean a few things:

• Your SSH key is protected by a local passphrase
• The keys in standard location ~/.ssh/ have been rejected
• Other issues on your side, check the verbose output, reset config

# If it takes forever

If you can't establish a connection at all, or if it takes "forever" to connect and nothing actually happens a few things may be the cause.

• a firewall on your end blocking outgoing connections (not very likely)
• your IP has been quarantined by our service due to too many failed attempts (possible)

Try to figure out if you can connect to external services on port 22 from your network. Try disabling your (corporate) VPN connection, if any.

# Connecting from within a container

When your local development environment is containerized with Docker, DDEV or alike and you want to deploy from within the container, make sure to have the SSH keys installed there as well.

# If it worked before and suddenly stops working

If you have deployed using SSH keys before and now it doesn't work any more:

1. Check if you have changed something, compare your local keys with the remote one
2. See if any change in collaboration happened, do you have access on the app still?

# Host authenticity warning

The first time you are connecting to fortrabbit service via SSH, there will be a warning about connection to a new SSH server. Just type yes, which will record the fingerprint of the fortrabbit deploy service into the ~/.ssh/known_hosts file. Say yes here:

The authenticity of host '…' can't be established
RSA key fingerprint is …
Are you sure you want to continue connecting (yes/no)?

If this fingerprint changes in the future, then you will se an error about that and will have to take further action depending on the situation.