Other factors that can negatively affect your ability to handle cold temperatures can include inadequate nutrition, dehydration, alcohol consumption, certain medications and health conditions such as diabetes and heart disease, which can significantly decrease a person's ability to exercise outdoors in the cold.