Fixing Default Route Override on Ubuntu (WMS Server on AWS)

In our AWS environment, the WMS server (Warehouse Management System) operates strictly within the internal network.
All external traffic is routed through a single NAT gateway — TPM1 (172.17.10.10).

However, after some time in operation, the WMS server’s default route keeps getting replaced with 172.17.10.1, breaking external connectivity and disrupting communication with other systems.

🔍 Problem Symptoms

Running route -n or ip route reveals the following:

root@wms:~# route -n
Destination     Gateway         Genmask         Flags Metric Ref Use Iface
0.0.0.0         172.17.10.10    0.0.0.0         UG    0      0   0  ens5
0.0.0.0         172.17.10.1     0.0.0.0         UG    100    0   0  ens5
...

The manually added gateway (172.17.10.10) occasionally disappears, especially after reboots or DHCP renewals, leaving only the default gateway from the DHCP server (172.17.10.1).

🧠 Root Cause

By checking the system journal (journalctl -u systemd-networkd -b), we can see the cause clearly:

ens5: DHCPv4 address 172.17.10.20/24, gateway 172.17.10.1 acquired from 172.17.10.1

👉 The culprit is systemd-networkd, which automatically installs the gateway provided by the DHCP server.
Every time the DHCP lease is renewed, the system overwrites the manually configured route.

🧩 Solution Strategy

We want to achieve the following:

Keep using DHCP to automatically obtain the IP address.
Ignore the default route offered by the DHCP server.
Manually define the correct gateway (172.17.10.10 for NAT via TPM1).

Ubuntu 20.04 and newer versions use Netplan + systemd-networkd, so the most robust solution is to configure these behaviors directly in the Netplan YAML file.

⚙️ Original Netplan Configuration

Before modification (/etc/netplan/50-cloud-init.yaml):

network:
  version: 2
  ethernets:
    ens5:
      match:
        macaddress: "06:43:c1:61:32:c4"
      dhcp4: true
      dhcp6: false
      set-name: "ens5"

This setup allows DHCP to assign both the IP address and the gateway route — which is the root cause of the problem.

✅ Fixed Configuration

We can keep DHCP for IP assignment while explicitly disabling DHCP routes and defining our own static default routes.

# /etc/netplan/50-cloud-init.yaml
network:
  version: 2
  renderer: networkd
  ethernets:
    ens5:
      match:
        macaddress: "06:43:c1:61:32:c4"
      set-name: "ens5"
      dhcp4: true
      dhcp6: false
      dhcp4-overrides:
        use-routes: false          # Ignore routes provided by DHCP
      routes:
        - to: 0.0.0.0/0
          via: 172.17.10.10        # Primary gateway (TPM1 NAT)
          metric: 10
        - to: 0.0.0.0/0
          via: 172.17.10.1         # Secondary/backup route
          metric: 100

🧪 Applying the Configuration Safely

When working over SSH (especially on AWS), use netplan try for safety.
This command provides a 120-second rollback timer — if something goes wrong, the system automatically reverts.

sudo cp /etc/netplan/50-cloud-init.yaml /etc/netplan/50-cloud-init.yaml.bak.$(date +%F-%H%M%S)
sudo netplan generate
sudo netplan try

If the connection remains stable, press Enter to confirm and make the change permanent.

Verify the result:

ip route show default

Expected output:

default via 172.17.10.10 dev ens5 proto static metric 10
default via 172.17.10.1  dev ens5 proto static metric 100

📘 Explanation of Key Parameters

Parameter	Description
`dhcp4-overrides.use-routes: false`	Instructs `systemd-networkd` to ignore DHCP-provided routes.
`routes:`	Defines custom static routes (multiple routes can coexist with different metrics).
`renderer: networkd`	Ensures that `systemd-networkd` handles network configuration (default on Ubuntu EC2).

🧱 Outcome

After applying this configuration, the server continues to receive its IP address via DHCP, but the default route is now permanently fixed as:

Primary: 172.17.10.10 (NAT via TPM1)
Backup : 172.17.10.1  (DHCP default gateway)

Even after DHCP renewals, interface restarts, or reboots, the route remains consistent.
✅ Problem solved permanently.

🧰 Optional Safety Script

If your environment still has unpredictable DHCP behavior, you can run a small “watchdog” script to periodically verify and restore the correct route.
However, with the above Netplan configuration, such scripts are typically no longer needed.

🧾 Environment Summary

Item	Value
Host Type	AWS EC2 (internal service only)
OS	Ubuntu 24.04 LTS
Network Service	systemd-networkd + Netplan
Problem	DHCP overwrote the default route
Solution	`use-routes: false` + manually defined static routes