You are very close to an accurate description. What you stated about Cost is correct, but port priority can affect neighbors that aren’t directly connect to the root bridge.
Cost vs Port Priority come into play at different levels of STP’s decision process regarding whether a port needs to be root, designated, or blocking/discarding.
After a root bridge has been elected, other switches must determine which port they have that represents the lowest COST path to the root bridge. When you set a Cost, this change will affect how all other switches see this path that are downstream from the switch where you made the setting. So in this sense, the change doesn’t necessarily affect the entire network topology as you stated, but rather only those switches downstream (from the perspective of the root bridge).
Port Priority is used much further down the decision tree for STP. This really only comes into play when a switch #1 has multiple connections to switch #2 where switch #2 is closer to the root bridge (or is the root bridge itself). In this case, Switch #1 has to decide which of its ports it wants to use as its connection to switch #2. The lower the port priority, the higher preference is given to it. So it really doesn’t matter whether Switch #2 is the root bridge, or happens to be an non-root bridge that switch #1 wants to use to get to the root bridge.
One important point with Port Priority, is that setting a value on switch #1 has NO effect on switch #1. Instead, you have to set the port priority on switch #2, so that switch #1 will learn this from switch #2. So just remember, that port priority set on an upstream port will affect the immediately connected downstream neighbor, but not the switch where it is actually set.