If they're syncing from the same source there shouldn't be an issue. If this is in a domain the source should be your main DC I would verify they're both syncing to the same source
Turning off ntp will eventually cause the systems to drift too far apart and you'll have problems again.