Speaking at a fringe meeting at Labour conference, held in partnership between the National Federation of Builders and the Federation of Master Builders, Dromey emphasised the importance of house building in bringing the country back into economic growth.
On government schemes, Dromey said he felt First Buy was "reasonably successful", welcoming the emphasis on using public land.
However, he noted that the New Buy scheme is "overhyped" whilst the New Homes Bonus will have less impact than predicted.
He spoke of the importance of acting now in order to get the economy moving.
Drawing on Ed Balls' announcement of Labour's plans to build 100,000 houses with the proceeds of the 4G spectrum auction, Dromey said it is a "carefully costed package".
The shadow minister also reiterated Labour's calls for a cut on VAT to home improvements, something echoed by another speaker, Brian Berry, chief executive of the Federation of Master Builders.
Bringing a constituency angle to the discussion, MP for Lewisham and member of the communities and local government committee Heidi Alexander spoke about the difficulties people face in accessing the housing ladder, as well as the large waiting lists across the country for social housing.
In order to face demand, Alexander said 240,000 houses a year would need to be built each year but the latest figures suggest not quite half of this figure has been achieved.
Alexander also pressed on the fact that construction productivity levels in 2016 are still destined to be lower than they were in 2007.
Julia Evans, chief executive of the National Federation of Builders, said that her members are trying to build but finding it "incredibly difficult".
Describing the current planning arrangements as "complete chaos", Evans said that the planning mechanism is very expensive and difficult to complete.
Reiterating Evans' concern, chief executive of the Federation of Master Builders, Brian Berry called for the planning process to be streamlined.
He also noted the importance of tackling the existing building stock with retro-fitting, to help support the sector.