Well, I wouldn't say it only took two years, but it surely wasn't five years. Of all the work they actually used coming from the end of XP to the start of Vista's RTM, it was probably a little over three years worth of work.
Even though Longhorn was dropped in favor of a code refresh, many things (Like the sidebar) made their way into Vista.