Why a Modular SSH Config?#

Managing SSH configuration can quickly become overwhelming as your environment grows. A single ~/.ssh/config file often turns into a tangled mix of work servers, personal machines, cloud hosts, and GitHub setups. Over time, this file accumulates dozens—sometimes hundreds—of entries, making it difficult to find what you need, spot errors, or safely share parts of your config. One misplaced line or typo can break connectivity for multiple hosts, and the lack of structure makes it risky to reuse or sync your config across devices. By adopting a modular approach—splitting your SSH configuration into focused, numbered files grouped by context—you gain clarity, maintainability, and the flexibility to share or update only the relevant sections. This method transforms SSH management from a fragile, monolithic file into a robust, organized system that scales with your needs.

Directory Structure Example#

~/.ssh/config.d/
├── 100_work/config
├── 200_home/config
├── 300_personal/config
├── 400_github_work/config
└── 500_github_home/config

How Ordering Works#

Each subfolder contains a focused config file, grouped by context and prefixed with a number to control load order. When you combine these files (often with a script or by using Include directives in your main ~/.ssh/config), they are processed in lexicographical (alphabetical) order. By prefixing each folder with a number, you ensure that configs are loaded in a predictable sequence 100_... first, then 200_..., and so on.

Using the Include Directive#

Note: SSH’s Include directive is supported in OpenSSH 7.3 and later. Run ssh -V to confirm your version.

# ~/.ssh/config
Include ~/.ssh/config.d/*/config

Benefits of This Setup#

This setup keeps everything modular, readable, and far easier to manage. It also makes syncing configs across devices—or sharing just the work-related parts—safe and straightforward. Once you try it, the old single-file approach feels instantly outdated.

Use ssh -G <host> to preview the final, merged configuration for a host—including settings from all included files—without actually connecting, making it ideal for debugging modular SSH setups.

Bootstrap Your Modular SSH Setup with Ansible#

The following Ansible playbook can automate the bootstrap of this modular SSH configuration. It clones a dotfiles repository, creates an SSH config file with an Include directive to load all configs from the config.d directory, and symlinks the grouped config directory into place. This keeps the SSH setup organized, maintainable, and easy to sync across systems.

---
- name: Clone dotfiles repo
  ansible.builtin.git:
    repo: git@github.com:gituser/dotfiles.git
    dest: "{{ ansible_env.HOME }}/dotfiles"
    accept_hostkey: true
    force: true

- name: Ensure ~/.ssh/config exists with Include directive
  copy:
    dest: "{{ ansible_env.HOME }}/.ssh/config"
    content: |
      # Auto-generated by Ansible: Include all configs from config.d directory
      Include {{ ansible_env.HOME }}/.ssh/config.d/*/config
    owner: "{{ ansible_env.USER }}"
    mode: '0644'

- name: Create Symlinks
  file:
    src: "{{ ansible_env.HOME }}/dotfiles/{{ item.src }}"
    dest: "{{ ansible_env.HOME }}/{{ item.dest }}"
    state: link
    force: true
  loop:
    - { src: "ssh-config.d", dest: ".ssh/config.d" }