Hi, welcome to the forum.
Would it be possible for you to supply the ip config for network 1 and network 2 and the ipconfig of each of the nics in the server. I can picture the physical setup and think I know whats going on but without ip details for the 2 networks it's difficult to truly picture whats going on. May I also ask why 2 modems?