An outside agency could be the deciding factor in what takes shape in some empty lots in Downtown Jacksonville.