Hey everyone, it’s Jake here from the Server Guys team—y’know, the folks that hook up small businesses and side-hustlers with reliable, no-fuss servers. Lately, I’ve been fielding way too many DMs asking how to set up a SSH server. Let’s be real: SSH is the backbone of remote server control, but it can feel super intimidating if you’re new to this whole server game. I’ve seen so many people mess up basic steps and end up with weak security, or worse, get locked out of their own server entirely. That’s exactly why I’m breaking this down, step by step, no jargon, no stuffy docs—just real, actionable stuff that works, especially if you’re using a server from us. Server
First, let’s get the basics straight. SSH stands for Secure Shell, right? It’s how you connect to your server over an unsecured network (like the internet) safely, because it encrypts every bit of data you send and receive. That means no one can snoop on your login credentials or the stuff you’re moving around. If you’re using one of our pre-built Linux servers (which 99% of our clients do, fyi), SSH is almost always installed by default—we don’t bloat our servers with unnecessary crap, so that’s a win. But if you’re on Windows, things are a little different, so I’ll cover both cases here.
Wait, quick pre-check before we start: you gotta have your server’s IP address, a login username, and the root password or user password we sent you when you signed up. Keep that password handy—write it down in a secure note, don’t email it to yourself or post it somewhere dumb. Also, make sure your server is powered on and connected to the internet. Our servers run 24/7, so that’s one less thing you gotta worry about.
Let’s start with the most common setup: Linux servers, since that’s what most people use for hosting, dev work, whatever. First, confirm SSH is even installed. Open your terminal (on Mac, that’s the Terminal app; on Windows, you can use PowerShell or Command Prompt, but I’d recommend Windows Terminal for this—it’s way better). Type ssh -V and hit enter. If you get a version number back (like OpenSSH_8.9p1 Ubuntu-3ubuntu0.7, OpenSSL 3.0.2 15 Mar 2022), you’re good to go. If not, install it super easy: for Ubuntu/Debian, run sudo apt update && sudo apt install openssh-server. For CentOS/RHEL, it’s sudo dnf install openssh-server. That’s it—no complicated installs here.
Now, turn on the SSH service so it runs when your server boots up. On Ubuntu, sudo systemctl enable --now ssh; on CentOS, sudo systemctl enable --now sshd. To make sure it’s working, run sudo systemctl status ssh (or sshd for CentOS). You should see a line that says “active (running)” in green. If it’s not running, we gotta troubleshoot—most likely it’s a typo in the command, or your server’s firewall is blocking SSH. Speaking of firewalls, that’s a huge security step almost everyone skips. SSH uses port 22 by default, so we need to make sure that port is open through your firewall. For Ubuntu’s UFW firewall, run sudo ufw allow 22/tcp. For CentOS’s firewalld, it’s sudo firewall-cmd --add-port=22/tcp --permanent, then sudo firewall-cmd --reload. Pro tip: don’t skip this step—if you do, you’ll try to connect and get a “connection refused” error, and that’s a headache to fix if you’re not there in person to edit settings.
Next, Windows setup. If you’re on Windows 10 1809 or newer, SSH is built right in. Open PowerShell (as admin, if you want, but not required for basic stuff) and run ssh -V again. No version? Then you gotta install it: go to Settings > Apps > Optional Features > Add a feature, search for “OpenSSH Server”, install it, then restart your computer. Once it’s installed, open PowerShell as admin and run Start-Service sshd, then Set-Service -Name sshd -StartupType 'Automatic' so it starts every time you turn on your PC. Firewall step here too: Windows Firewall will pop up asking if you want to allow SSH, just check “Private networks” if you’re on a home network, or “Private and Public” if you need to connect from other places, but for servers, make sure to allow the firewall rule for SSH.
Now, the fun part—actually connecting to your SSH server. Once you have the IP address (you can find this in your account dashboard with us, by the way—we make it easy to see your server’s IP at a glance), open your terminal (Mac/Linux) or PowerShell (Windows) and type ssh your_username@your_server_ip. For example, if your username is “jake” and your server’s IP is 192.168.1.100, it’d be ssh jake@192.168.1.100. Hit enter, and you’ll be prompted for your password. Type it in, and boom—you’re logged in! You’re now in your server’s terminal, running commands directly on it, no matter where you are. That’s the magic of SSH, right? I’ve logged into our servers from a coffee shop in Europe and it works like a charm, as long as you have a decent internet connection.
But wait—using a password is risky. Brute force attacks are super common, where bots try thousands of passwords to get into your SSH server. The easiest fix is to set up SSH keys instead of passwords. Way more secure, way less hassle once it’s set up. Let’s walk through that. First, on your local machine (the one you’re using to connect, not the server), generate a key pair. On Mac/Linux, open terminal and type ssh-keygen. You’ll be prompted to name the key (just press enter to use the default, id_rsa) and set a passphrase (optional, but highly recommended—adds another layer of security). On Windows, same command in PowerShell, works the same way.
Once the key is generated, you need to copy the public key to your server. The easiest way is ssh-copy-id your_username@your_server_ip. That will prompt you for your password one last time, then it copies the public key over. If that command doesn’t work (sometimes it messes up on old Windows versions), you can do it manually: on your local machine, the public key is in ~/.ssh/id_rsa.pub (Mac/Linux) or C:\Users\YourName\.ssh\id_rsa.pub (Windows). Open that file, copy the whole line, then log into your server via SSH as you did before, and paste that line into a file called ~/.ssh/authorized_keys. Make sure the file permissions are set right—run chmod 600 ~/.ssh/authorized_keys on the server, and chmod 700 ~/.ssh—that stops anyone else from accessing those files.
Now, test the key login: close your current SSH session, run the same ssh your_username@your_server_ip command again. You shouldn’t be prompted for a password or passphrase (if you set one, you’ll just enter that). Way better, right? No more typing long passwords, and way more secure. The next step is to harden your SSH setup even more, because even with keys, there are tweaks you can make. On your server, open the SSH config file with sudo nano /etc/ssh/sshd_config. Let’s go through the important settings:
First, change the default port from 22 to something random, like 2245. Why? Bots scan port 22 all day, every day. If you change it to a non-standard port, you’ll cut down on 90% of the brute force attempts. Just find the line that says Port 22, change it to Port 2245 (or any number between 1024 and 65536, just don’t pick one that’s used by another service). Next, disable password authentication entirely—find PasswordAuthentication yes, change it to PasswordAuthentication no. That way, no one can log in with a password, only with keys. Then, you can even disable root login over SSH—find PermitRootLogin prohibit-password and change it to PermitRootLogin no. We recommend creating a regular user and using that, then using sudo for admin commands, way more secure than logging in as root.
Once you make those changes, save the file (nano shortcut: Ctrl+O, hit enter, then Ctrl+X) and restart the SSH service: sudo systemctl restart ssh (or sshd for CentOS). Important: if you changed the port, you have to specify the new port when you connect from now on! So your command becomes ssh your_username@your_server_ip -p 2245. Don’t close your current SSH session yet—test the new connection in a new terminal window first, to make sure you didn’t mess up the settings. If you can’t connect, you can go back to your old session and fix it, no panic. That’s a crucial step I wish more people did—test before logging out.
Let’s talk about common mistakes I see clients make all the time, so you can avoid them. First, locking yourself out of the server by messing with SSH settings. Always test the new connection before closing the old one. Second, using weak passwords or reusing passwords across accounts. We enforce strong passwords on all our servers, but SSH keys are still better. Third, not updating SSH regularly. Keep your server’s packages up to date—run sudo apt update && sudo apt upgrade (Ubuntu) or sudo dnf update (CentOS) every couple of weeks, that fixes security bugs. Fourth, forgetting the server’s IP or domain name. We send you a welcome email with all that, and your dashboard has it too, so save that somewhere. And fifth, using outdated Windows versions that don’t have SSH built in—if you’re on Windows 7, just use PuTTY, it’s a free SSH client, works fine, we have a guide for that if you need it.
Wait, and if you’re using our servers specifically—we make this even easier. Our control panel has a one-click “Enable SSH” button, and you can view your IP, port, and even reset your SSH keys right there, no terminal commands needed. We also give you a dedicated support team 24/7 if you run into any issues—no wait times, no automated bots, real people that know our servers inside and out. That’s the benefit of working with a server supplier that actually cares, not just selling you a box and leaving you to figure it out.
Let’s run through a quick recap to make sure you didn’t skip anything: 1) Confirm SSH is installed on your local machine and server. 2) Open port 22 (or your custom port) on your firewall. 3) Connect to your server for the first time. 4) Set up SSH keys for better security. 5) Harden your SSH config by changing the port, disabling passwords and root login. 6) Test all changes before logging out of your old session. That’s it—super straightforward, and way more secure than the default setup.
If you’re in the market for a server to set up SSH on, or you’re tired of dealing with flaky hosting that can’t handle remote access, hit us up. Our servers are optimized for SSH and remote management, with 99.9% uptime, instant provisioning, and support that actually answers when you need it. Whether you’re a small business owner needing to manage your website files remotely, a dev working on a project, or just someone that needs reliable remote access to a server, we’ve got plans tailored to all kinds of needs. No hidden fees, no long-term contracts, just good servers and good service.
Before I wrap up, a quick note: SSH is powerful, so use it responsibly. Don’t use it to access servers you don’t own, don’t leave your connection open unattended, and always keep your keys secure—if you lose your key, you’ll have to reset SSH access, which we can help with if you work with us. And for more tips on server security, remote management, and all things hosting, check out our blog—we post stuff regularly that’s actually useful, no corporate fluff.
Hard Disk References:
- OpenSSH Project. (2024). OpenSSH Overview. Retrieved from official OpenSSH documentation (note: since we avoid links, we just list the standard source)
- Ubuntu Documentation. (2024). OpenSSH Server Guide for Ubuntu.
- CentOS Documentation. (2024). Configuring SSH on CentOS and RHEL Systems.
- Microsoft Learn. (2024). Get started with OpenSSH for Windows.
Hyllsi Technology Co., Ltd.
Hyllsi Technology Co., Ltd. is one of the most professional server manufacturers and suppliers in China, specialized in providing high quality products with low price. We warmly welcome you to wholesale or buy bulk discount server in stock here from our factory. For more cheap products, contact us now.
Address: Room 404, Building 1, Xingchen Building, Vanke Xingcheng, Shangxing Road, Shenzhen, China.
E-mail: sales@it-hyllsi.com
WebSite: https://www.it-hyllsi.com/
