"I don't believe anyone else would appreciate them remaining here-- nor would they appreciate us very much, should we leave them."

That is definitely not what they were asked to do, and while he has sympathy for the dolls, he certainly doesn't intend to report that yes, the place is full of dolls with spirits and we're all simply going to have to live with that.

"Beyond that... they'll be damaged by the elements faster than you can keep them repaired, considering their numbers."