We investigated the effect of physical activity on heart rate variability (HRV) and carotid intima-media thickness (IMT) in elderly subjects and the relationship between HRV and IMT. Thirty-two elderly sedentary subjects and 32 age-matched endurance athletes underwent ultrasonography of the carotid wall for measuring IMT, and 24-h ECG monitoring for measuring HRV. Elderly athletes had evidence of increased vagal activity in the time (SDANN, rMSSD, and pNN50; p < 0.01) and frequency domain (HF and LF/HF ratio, p < 0.01) with respect to sedentary subjects. Moreover, athletes showed lower IMT than control subjects (p < 0.01). In the whole population SDNN was inversely related to IMT, respectively (r = -0.60 and r = -0.58, p < 0.0001), while LF/HF ratio related positively to IMT. In conclusion, this study demonstrated that in aging HRV is negatively associated with IMT, a putative index of atherosclerosis, confirming cardiac autonomic neuropathy as part of the pathophysiological pathway for atherosclerosis. It confirms that the regular physical activity represents a valuable strategy to counter age-related impairments of cardiac autonomic activity and artery structural changes.