Wasp Nests

I have a wasps nest in the house cavity, they are entering via a vent that once connected to a pantry cupboard which was removed many years ago, but the vent was left to allow air into the cavity, I tried shifting them the other day by holding a smoking rolled up piece of corrigated cardboard which I held close to the vent, the only thing that got shifted was me, OH had the kitchen window open and the smoke was going in there where she was, for some reason she wasn't to pleased, how was I supposed to know the window was open, just because I was standing next to it, doesn't mean I noticed it2