But the black population of San Francisco today is less than half of what it was back in 1970, even though the city's total population has grown.Severe restrictions on building housing in San Francisco have driven rents and home prices so high that blacks and other people with low or moderate incomes have been driven out of the city.