Orchestrator using old credential value

We changed a password in one of our Orchestrator credential assets. The following scheduled executions faulted and we could not figure out why. We then discovered that the old password was still being used. Is this some sort of cache issue?