Building homes on grey belt land is “unlikely to make any significant difference” to the government’s ambitious target of 1.5 million homes.