It works like this.
Every Pokemon has a number that stores which ability it has. Let's call it an ability identification number, or AID for short. This number has a certain range, so let's assume it's one byte long (I don't know its maximum range). If the number is between 0 and 127 (for instance), the Pokemon takes its first ability; if it is between 128 and 255, it takes the second one. The AID is generated when the Pokemon is hatched or is encountered, and it never changes whatever happens to the Pokemon.
In your example, Yanma had either Speed Boost or Compoundeyes, and Yanmega has either Speed Boost or Tinted Lens. Let's say that Speed Boost is in the 0-127 range and Compoundeyes and Tinted Lens are in the 128-255 range. That means that, if a particular Yanma's AID is between 0 and 127 (thus having the Speed Boost ability), it will retain its Speed Boost ability whenever it evolves to Yanmega, as the AID never changes, not even if the Pokemon evolves. Also, if Yanma had Compoundeyes, the ability would change to Tinted Lens whenever Yanma evolves to Yanmega.
Now suppose a Pokemon has only one possible ability. It turns out that, internally, that Pokemon has STILL two possible abilities - but the two possibile abilities are the same. Let's take Linoone as an example. In Advance, Linoone had only one ability: Pickup. Internally, Linoone still has an AID representing its ability. If it is between 0 and 127, Linoone gets Pickup. If it is between 128 and 255, Linoone also gets Pickup.
However, in DP, Linoone has two possible abilities: Pickup and Gluttony. This means that if a Linoone from Ruby, say, is transferred to Diamond, Linoone might have its ability changed to Gluttony. Assuming that Pickup is in the 0-127 range and Gluttony is in the 128-255 range in DP, if Linoone's AID was between 0 to 127 in Ruby, then Linoone's ability would stay as Pickup in Diamond, whereas if it was in the 128-255 range, it would change to Gluttony in Diamond.
I hope I made this matter a bit more clear.