OK, well when you distill wine you are in fact distilling something flammable - the ethanol. Xylene is actually 'less flammable' than ethanol in that it has a higher flash point, although this is largely irrelevant since it will be heated for this distillation of course.
It would be better if you did use stirring as this helps to prevent bumping (where the liquid suddenly boils violently). You will get a purer product that way - bumping projects droplets from the boiling flask into the distillation train and thus impurities can make their way through to the receiver. And only fill your flask up to half full.
I would suggest sealing the glass joints with PTFE tape to minimise vapor leakage. It would also be useful to know what kind of receiver adapter you have that fits between the end of the condenser and the receiver flask.
Use of a thermometer at the still head would be prudent. You might notice an azeotrope of water and xylene distilling over at below 100°C if the xylene is a bit damp.
The other thing is to stop distilling before the boiling flask is dry. Distilling to dryness can be dangerous if, for example, the impurities can form peroxides on contact with air.
It seems you've made sure you have good ventilation and a suitable fire extinguisher. Just as long as you have sufficient time to complete the distillation you should be good to go. It would be good to rehearse setting up the equipment beforehand as well.
As an exercise, describe what you'll be doing step-by-step in your own words.
“There is a way of manipulating matter and energy so as to produce what modern scientists call 'a field of force'. The field acts on the observer and puts him in a privileged position vis-à-vis the universe. From this position he has access to the realities which are ordinarily hidden from us by time and space, matter and energy. This is what we call the Great Work."
― Jacques Bergier, quoting Fulcanelli