Occasionally, the change in a man's fertility can be traced to a chronic illness such as hypertension or diabetes, explains Esther Eisenberg, M.D., director of the Reproductive Endocrinology and Infertility Center at Vanderbilt University Medical Center, in Nashville.