Block the signal
It’s fairly simple to block most RFID tags. There are RFID-blocking wallets and passport holders available online, or you can build a little portable Faraday cage for your pocket.
To build your own RFID-blocking gear, you will need to build a pocketbook out of copper or aluminum sheets. This has the added benefit of making you look awesome—especially if you use copper and fashion it into a steampunk lunchbox.
There is some evidence that your body itself is a reasonably effective shielding device. Keep your passport and other RFID-enabled items inside your clothes against your skin. That might, at least, limit the number of directions from which you can be tracked.