Can I close up outdoor fireplace vent?

I have a fireplace (wood burning) that has a little vent outside that looks like a dryer vent. Apparently it sucks in fresh air from there. Due to remodeling, I need to close this up as it won't be outside anymore and I don't want to move it. Is it safe to do so (I don't care about efficiency -- we live in FL and rarely use the fireplace).