Question

Should the ash plug be in when I run the stove?

Yes, you should have the ash plug in when running the stove so that hot embers do not get into the ash pan.