Not necessarily - if something isn’t found, it can’t be cached. That’s a common cache strategy to get from cache if you have something, but do the full leg work when you don’t.
So when it doesn’t find it, it will check next time. But when it does, and nothing triggers a cache invalidation, then the “obvious” optimization is to just use the result it already has.
In your scenario, you’re working with what is effectively a static object window, from UI perspective at least - there’s no change in state of the window itself, so the cache is considered valid. The RDP/Citrix windows cannot communicate that something changed on the screen, because they would need to do that every frame (as each streamed frame is effectively a diff of the screen capture).
In essence, with this caching behaviour, unless you can “force” a cache invalidation/miss (i.e. setting it as a separate window, making it lose context etc.), it will use the cached value.