Draft pick trades

I was thinking about this today. There has been instances where the NFL has taken away draft picks from a team due to illegal activities.
So my question is, does the NFL look at the draft pick before they take it away? As a for instance, say the Vikings do something that makes the NFL take away their second round draft pick. But this draft pick was traded to the Browns. Would the Browns just be SOL, or does the NFL take into consideration who currently owns the pick? I know it hasn't happened yet, I just wonder if it could.

 

I would say they make sure the pick is owned by the team being penalized. The other NFL owners would be pretty upset if they were punished for another team's activities.