Needing a VPN to connect to my MC Server

Hi, im running a homestead modpack on my server and when i try to connect to the domain it says theres no connection to the server at all but when i connect to a VPN on my side i can connect to the server with sub 30ms ping.

Now i only noticed this as i was connecting to the server with my local IP so i have no issues originally. But when giving my domain to my friend in germany to connect to they was getting 350ms ping or not being able to connect at all. I gave them access to my nordvpn and they had no problem connecting with a german VPN as they now had 50ms ping.

I actually suspected there was an issue connecting via the domain as i tired to connect using the domain and had 150 plus ms ping at first then it got worse and worse.

The second you connect with a VPN, the connection is fine when joining the server with the domain.

Any ideas would be geat thank you