Now, they are coming forward, blowing the whistle on the practices they say Lincoln Military Housing used so long.
"When you go in, open up a wall and see that there is mold there, and it just sits there, and sits there, and sits there."
Brad Perkins came across a lot of military families while working as a contractor in Lincoln's housing complexes - but one particular case in Yorktown still haunts him.
"She had a pretty bad situation, had lots of mold, provided daycare and was pregnant, when we were doing this work, she was eight months pregnant," Perkins said.
According to Perkins, about half a dozen units in Yorktown he worked on had mold problems. But he says the real issue came in the units his teams fixed at Sandpiper Crescent in Virginia Beach.
"All had mold and moisture problems...every one of them."
"When we tear this drywall down, tear this siding off, they can see, aren't dumb, they know what's going on."
Perkins says most of his work dealt with roof leaks. He says it looked to him like many had been left unattended for years - and when Lincoln finally did address them:
"Didn't want to spend $5,000 to put a new roof on, they wanted to spend $200 to try and do a jack leg repair job, and it just didn't work."
Former Lincoln employee Deanette Smith says that was standard company protocol during the six months she worked there, back in 2005.
"When the maintenance guy goes out to the unit, does the least amount possible to fix the unit, spends as little money as possible, never to call a vendor ever, ever, ever, unless you have to," Smith says.
She says that policy even applied to mold remediation - if a professional company was called in, Lincoln had to pay for whatever the remediators recommended.
"But if it was just Lincoln, it was a band-aid over it, residents still lived in it."
"There was mold in all these houses."
According to both Smith and Perkins, when residents had questions about that mold:
"Lincoln's response to me, 'don't ever say the four letter word'," Smith says.
"When they say to not say the word 'mold' to someone who is going to live there, that is totally wrong."
"I'm serious, it was really bad, never worked for a company like that, so vile, and all the standard codes of conduct were broken."
NewsChannel 3 received this statement:
"Lincoln Military Housing is actively putting in place our eight point action plan to continue to upgrade the maintenance services our residents receive. This will include a historical review of all of our systems, as well as these allegations."
That eight point plan also includes free mold tests for all residents who ask, as well as maintenance inspections on all the homes they own in Hampton Roads.
A forum was held Monday evening on the Peninsula at the Lincoln Community Center in Yorktown.