1. Welcome to Smogon Forums! Please take a minute to read the rules.
  2. Welcome to the Orange Islands! Please remember to read the rules. Come chat with us on our IRC channel and enjoy your stay!
  3. New to the forums? Check out our Mentorship Program!
    Our mentors will answer your questions and help you become a part of the community!

Data on the friend safari shiny rate

Discussion in 'Orange Islands' started by The Scientist, Mar 15, 2014.

  1. The Scientist

    The Scientist Heheh, not a Silph worker...
    is a Pokemon Researcher

    Joined:
    Jan 1, 2014
    Messages:
    19
    Hello all,

    I wanted to figure out the shiny rate of the friend safari, so I used an automated encounter bot to run around in the friend safari, tally the number of total encounters and the number of shinies encountered. In addition, dekuNukem helped by providing data for over 40 shiny Pokémon. So far, we have tallied 157 shinies over 74,540 encounters. The average number of encounters per shiny is 475 with a standard deviation of 445. The 95% confidence interval is 405-545. The smallest number of encounters until a shiny was found was 1, and the largest number was 2691.

    A few strongly influential observations (i.e. when it took more than 2500 encounters to find a shiny) may be skewing the data. I'm not confident that it's statistically justifiable to remove these outliers, but if they are removed, the average number of encounters per shiny is 411.31 and the standard deviation is 359.41. The confidence interval is 351-471.

    I attached a few graphs of the data. The first one shows how the data are distributed. For example, there were 31 shinies found in 1-100 encounters, 25 shinies in 101-200 encounters, etc. So if it took 340 encounters until a shiny was found, I tallied it for the 301-400 group.

    histogram.jpg


    The second graph shows the probability of finding shiny within 100 encounters, 200 encounters, 300, etc.

    shinyprobabilities.jpg


    In conclusion, if I were being conservative and including the outliers, I would guess the shiny rate is either 1/512 or 1/455. I will provide an update after another 100 shinies. If anyone can provide any statistical advice or criticism, I'd like to open some discussion about it.

    As a disclaimer, I have the shiny charm, although currently the data from other people testing the effect of the shiny charm on the shiny rate in the friend safari suggest that the shiny charm has no effect.

    I'm thinking of testing a couple variables that are rumored to affect the shiny rate. For example, some people say tipping improves the shiny rate; it's been suggested that the shiny rate is different in different safaris; and perhaps your friend being online/offline affects the shiny rate. Incidentally, one of the shinies I caught and dekuNukem caught had Pokérus. It would be difficult to test, but perhaps the Pokérus rate is also increased in the friend safari? Anyone have ideas about the best way to test that?

    Best,
    The Scientist
    Last edited: Apr 2, 2014
  2. Akhara_Vect

    Akhara_Vect

    Joined:
    Mar 17, 2014
    Messages:
    1
    Did anyone of you have a Shiny charm?Did you switch Safaris in between? Or were all encounters done in the same Safari? On one system? Or on multiple? That's what I am interested in. Does the rate vary between different safaris? And if yes, does it also vary between different systems? Meaning, if the shiny rate varies between safaris and five different people visit my Safari on 5 different systems, do they all have the same rate or does it also vary between systems? I would have done some research on that topic but there is no possible way for me to do it.
    What are your thougths? And great work btw, you're doing us all a great favor!
    Last edited: Mar 17, 2014
  3. The Scientist

    The Scientist Heheh, not a Silph worker...
    is a Pokemon Researcher

    Joined:
    Jan 1, 2014
    Messages:
    19
    I'm so glad someone replied to the thread! Lol, I was getting worried no one read it.

    1. I have the shiny charm, and I'm not sure if dekuNukem did. My guess would be that he does not have the shiny charm since he recently contributed to the "Masuda Method without shiny charm" thread here on Smogon. I was using the same game on the same 3DS the whole time, and I'm not sure if dekuNukem had different games and 3DS's. In any case, I compared both my data set and his data set, and found there was no significant difference between them. Our average number of encounters per shiny was similar.

    2. I switched safaris throughout the study. Sometimes a safari would take over 2500 encounters to find a shiny, and then the same safari would find another in 21 encounters. So, from my interpretation of the data, there is nothing to suggest that different safaris have different shiny rates. However, that still needs to be rigorously tested.

    I attached a PDF of the raw data if you'd like to look through it. The numbers are the number of encounters before a shiny was found. The changing colors represent when I switched to a different safari, and I was never in the same safari twice. The large pink group at the end is dekuNukem's data.

    Attached Files:

    Last edited: Mar 17, 2014
    Animelee likes this.
  4. Hootie

    Hootie

    Joined:
    Apr 29, 2012
    Messages:
    138
    It's funny how I actually found 2 shinies within a time span of a half an hour lol. Nice job with all these statistics since they seem about right with my odds haha.
  5. Zowayix

    Zowayix

    Joined:
    Oct 26, 2008
    Messages:
    337
    It might be a good idea to indicate that you used the Shiny Charm either at the top of the OP or the thread title, since that is likely to drastically change the odds.
  6. The Scientist

    The Scientist Heheh, not a Silph worker...
    is a Pokemon Researcher

    Joined:
    Jan 1, 2014
    Messages:
    19
    I can add that information, although another user here tested the shiny rate in the friend safari, and found there is no difference between a game with the shiny charm and a game without. They did 15,000 encounters per game and concluded the shiny charm had no effect. While the data set is small, it currently seems more likely that the shiny charm has no effect on the shiny rate in the friend safari than that the shiny charm does affect the friend safari shiny rate. Read it from the source- http://www.diyode.com/2014/02/shiny-charm-and-friend-safari-preliminary-data/
    Mewdraco and Animelee like this.
  7. The Scientist

    The Scientist Heheh, not a Silph worker...
    is a Pokemon Researcher

    Joined:
    Jan 1, 2014
    Messages:
    19
    Good news everyone. I'm now very confident that the friend safari shiny rate is 8/4096, or 1/512. To determine the shiny rate, I started off by assuming the data would have a binomial distribution because as a discrete distribution, it approximates the probability of a yes/no event occurring in n samples with replacement. So we can ask the following question: Given p probability, how likely is it that X will happen in n number of events? In other words: given some theoretical friend safari shiny rate (which I had narrowed down to 10/4096, 9/4096, or 8/4096), how likely is it that I will find a shiny after 100 encounters, 200 encounters, 300 encounters, etc.?

    Using the R statistical software, I approximated binomial distributions based on 1,000,000 trials for probabilities of 10/4096, 9/4096, and 8/4096, or 1/409.6, 1/455.11, and 1/512, respectively, at ten levels between 100-1000 encounters, at increments of 100. I then obtained the percent of trials that had a "yes" event (or found a shiny) for each theoretical probability at each level of number of encounters (100 encounters, 200 encounters, etc.). So each of the three probabilities had ten percentages based on how likely you are to find a shiny after 100 encounters, 200 encounters... 1000 encounters, giving 30 total percentages. Finally, using chi-square tests, I compared the percentages of the three probabilities to my own data (expected vs observed), and found that the 1/512 probability was a near perfect match!

    Here are the chi-squared test statistics:

    1/409.6- 0.82037358
    1/455.11- 0.99916429
    1/512- 0.99997423

    1/455.11 is close, but almost two orders of magnitude from 1/512. I attached a graph that shows the percentage of finding a shiny in N encounters at the three probabilities, plotted alongside the actual dataset, which now contains data on 187 shinies. You can see the actual dataset diverges a little from the 1/512, but this isn't unexpected given the sample size. If anyone has any feedback on the above statistical methods, I'd love to hear it.

    [​IMG]

    Attached Files:

    Last edited: Mar 20, 2014
  8. NixHex

    NixHex A new vice president erryonce in a while to keep me busy
    is a member of the Site Staffis a Forum Moderatoris a Pokemon Researcheris a Contributor to Smogonis a Battle Server Moderator Alumnus
    Moderator

    Joined:
    Oct 6, 2009
    Messages:
    2,586
    I've been looking around furiously online for some SOLID information regarding shiny rate in FS and if the Shiny Charm affects it. I just saw this thread today and was very pleased at how comprehensive this is. This is such good work. Keep it up, The Scientist !

    btw this is more for the story than for actual research help but I went shiny hunting last night and found a shiny Spritzee in less than 100 encounters. And I got to watch a full episode of Supernatural! Life is good.
    Lord Ghetsis and The Scientist like this.
  9. Fletchling

    Fletchling

    Joined:
    Nov 17, 2013
    Messages:
    2
    Awesome collection of data. I've been wondering about where, exactly, the shiny charm affects encounter rates, and this is great to know for the safaris.

    I have also been curious if there are any variables that affect encounter rates in the safaris. I was frequenting one safari because it had three 'mon I wanted to shinyhunt (Shuppet, Pumpkaboo, Dusclops), and absolutely nothing over the course of months. I have no idea what the actual encounter number was, but this was someone I visited off and on since I reached Kiloude in October. Eventually I got so frustrated by my lack of shiny ghosts that I went hunting Route 16 and got my shiny Pumpkaboo there, with shiny charm, about a month ago. After that I went to a different ghost safari and encountered a shiny Dusclops in 10 encounters. It made me wonder if there is some sort of calculation done in certain safaris wrt your TSV and their TSV that may change the base encounter rate, or if I was just on the bad luck end of the encounter rate.
    The Scientist likes this.
  10. The Scientist

    The Scientist Heheh, not a Silph worker...
    is a Pokemon Researcher

    Joined:
    Jan 1, 2014
    Messages:
    19
    Big thanks, NixHex and Fletchling . It's more than enough thanks to know that someone has found the data useful.

    Fletchling, so if what you saying is true, would you predict that different safaris have different shiny rates? I'm certainly open to hearing ideas about what could affect the shiny rate, but keep in mind that I regularly go 1000 or 2000+ encounters and then I might find a few shinies in less than 500 encounters. This good and bad luck happens in the same safari without anything having changed, as predicted can happen by a binomial distribution.

    By the way, I've been tipping $5000 to the guy in Kiloude between the last 25 shinies to test the shiny theory. I hope to post results on whether tipping affects the shiny rate in the next week or two.
    Last edited: Mar 21, 2014
    Mewdraco, Level 51 and Animelee like this.
  11. Fletchling

    Fletchling

    Joined:
    Nov 17, 2013
    Messages:
    2
    If there is a variable that calculates encounter rates based on some interaction of the player TSV and the owner TSV, then yes, I would predict that different safaris would have different rates. But it really was just a far-flung theory, rather than anything firmly tested: having no shiny encounter in a certain safari over many different visits could just be simple bad luck that falls well within what you found with your tested rates--like you said, 2000+ encounters without a shiny happens regularly for you when you're testing. I came up with it when people were talking about rate increase possibility theories, like if the safari owner was online. It was the midst of InstaCheck days so I was wondering if TSVs had any other function other than the obvious.

    It's frustrating that Nintendo/GameFreak aren't as open about these pieces of data and that there's no way to confirm it for certain, just test results that are believed accurate.
    The Scientist likes this.
  12. kyria

    kyria

    Joined:
    Mar 28, 2014
    Messages:
    1
    Hello guys,

    I'm new to this forum and to be honest this post was the reason i registered ^^.
    So first i have to say that i'm very thankful for your testings, though i already have searched all over the web to find some really useful/profound data concerning the shiny rate in the friend safari.

    Because you mentioned to be open on ideas about effects on the shiny rate i thought i could share my ideas/experience for the friend safari as well.

    Experience:
    All in all I've catched 5 shiny in the safari, i don't know if it was luck or any higher rater, but 2 of them came very close after i've encountered and catched the first ones. In some other forums i also read that people had similar experiences (as well as Hootie seemed to have that kind of look too).
    So maybe I'm seeing ghosts but when i look into your data i see that except for 3 of your 20 days of testing all second encouters on a day were faster than the the first ones.

    For the shiny charm, i really hoped that it would have an affect (though already the pokeradar does not work) but maybe it would have made catching shinies too easy. :(

    Ideas:
    Some other ideas that came up to my mind which might affect the shiny rate:
    - on/off status -like Fletchling also mentioned the on/offline statuses of either yourself or your friend might affect the rate (maybe also the on/off status of the street pass function) ?
    - catching/ defeating/ escape - in other shiny methods (i.e. shiny chaining or shiny fishing) the way you handle all encounters before the shiny appears has an affect on the shiny rate, so maybe in the friend safari there is one too?
    - total amount of friends - maybe the chance for shinies is increased the more friends you have (i.e. someon with 100 friends has a higher chance for shinies than someone with 10 friends) ?
    - Encouter Power - though this does not seem very likely it just came to my mind that maybe encouter power might also have an effect on the shiny rate?

    Hmm i think thats all by now and again i'm really happy that someone finally got some solid data for the friend safari shinies! =D

    PS.: sorry for my bad english and for writing this much text (came out longer that i expected >.<)
    Last edited: Mar 28, 2014
    The Scientist likes this.
  13. The Scientist

    The Scientist Heheh, not a Silph worker...
    is a Pokemon Researcher

    Joined:
    Jan 1, 2014
    Messages:
    19
    Hi Kyria,

    Thanks for the thoughts. I'm glad you found the data useful. Let me address some of your questions.

    1) I agree that often shinies are caught soon after another. It's happened several times that I went 1000+ encounters without seeing a shiny, then after finally finding a shiny, I get another shiny within 100 encounters. However, I'd like you to keep this in mind. If the shiny rate in the friend safari truly is 1/512, then by the type of distribution the data should possess (binomial), you always have nearly a 20% chance of finding a shiny within the next 100 encounters. That's a pretty decent chance. So if two of your five shinies came very close to each other, that was a statistically like event. Currently, 18.4% of my shinies occurred in less than 100 encounters.

    2) You noticed that many of the day's first shinies had more encounters than the second. The trials for all of those first of the day shinies started before midnight, and then ended when a shiny was found. So I recorded the day when the shiny was encountered, and not when the encounters were started. This would make it more likely for midnight to occur during a long run of encounters. For example, if I start the bot at 9pm and it takes 1000 encounters to find a shiny, it would end at 2am (it does 200 encounters per hour), making that 1000+ encounter shiny be the first shiny of the day. In addition, sometimes I wasn't really awake when I reset the bot, so in the morning when I entered the data, I usually guessed that midnight occurred during the longer runs.

    3) I like your ideas about what could affect the shiny rate. I had thought of the online/offline presence of your friend, but I hadn't thought of the other two. The online/offline presence would be easy to test, and I'm working with someone to hopefully test this. I think your idea of catching/defeating/escaping has some potential. We know that catching the Pokemon resets the fishing chain, and we know that defeating Pokemon via Pokeradar increases the shiny rate until 40. I'm not sure how this would apply to the friend safari since there is no apparent chain other than running into Pokemon. What are your further thoughts on this? How do you envision this applying to friend safari?

    More coming soon on an update of the data and the results of my tipping trials, so stay tuned.
    Animelee likes this.
  14. Animelee

    Animelee

    Joined:
    Oct 4, 2013
    Messages:
    11
    I'm extremely excited to hear any scraps of data you could give to us on your tipping trials! Even if your data isn't "polished" enough to formally present right now, I'd be deeply grateful to hear your preliminary feelings and thoughts on if tipping does have an effect!
  15. The Scientist

    The Scientist Heheh, not a Silph worker...
    is a Pokemon Researcher

    Joined:
    Jan 1, 2014
    Messages:
    19
    OK, I won't keep you waiting. I transcribed my data to a Google Spreadsheet, and I'll update it now and then. I had to organize it a little oddly due to the way Google Spreadsheet makes the charts.

    https://docs.google.com/spreadsheet...htaGsxRlI1S1BDVHNySUJMMlE&usp=drive_web#gid=0

    The data currently contains over 100,000 encounters and 207 shinies. 128 of those come from no tipping, 79 of them are with tipping. A major caveat to the data right now is the much smaller sample size for the tipped shinies. This is going to make the tipping data appear more erratic. However, as of right now, I don't see any reason to suspect that tipping affects the shiny rate. Without tipping, the average is ~1/501, and with tipping the average is ~1/542. Both of those have standard deviations over 500. If tipping does affect the shiny rate, it would be a minor effect, like going from 8/4094 to 1/455 or 1/410. At this point I can't be very confident given the small data set. Overall, the data continues to settle toward a shiny rate of 1/512. Look at the Chi-square test statistics on the far right of the document.

    Again, it looks like IF tipping affects the shiny rate, it would be a minor effect. I just can't rule out that possibility right now. Does anyone think Gamefreak would put such a minor effect into the game? I suppose it's possible.
    chronomancy and Animelee like this.
  16. Animelee

    Animelee

    Joined:
    Oct 4, 2013
    Messages:
    11
    Thank you so much! This means quite a lot to me and my brother! So we will refrain from purposefully tipping (unless in Le Wow) from now on, as it seems to make no difference. Thank you once again, and I will continue to follow you, your posts, and this thread especially!
  17. OutCold

    OutCold

    Joined:
    Mar 31, 2014
    Messages:
    1
    I was looking around for some numbers for FS shiny rate and this was definitely the best I found. I also signed up so I could post my FS Shiny experience from this afternoon. I MM shinies and radar chain shinies alot, so I wanted to try FS RE's or the Horde route. I decided to give the FS a shot. I started by farming Kirlia (I was just looking for a female with HA to later MM a shiny) I did not think about keeping track of encounters at first since I thought it would be a quick find with Cute Charm. I would have to say after about 20-30 encounters, before I could find my Female HA, I ran into a shiny Kirlia.

    I decided to do some more shiny hunting for awhile after that quick luck and this is what I found. I also have the Shiny Charm, and I tipped for the hell of it before going into a new FS for the next 3.

    Shiny
    Dwebbie: After 66 RE's tipped ~ 8k
    Hariyama: After 6 RE's tipped ~ 11k
    Evee: After 517 REs tipped ~ 10k

    I don't think tipping really has any affects on shiny rate. I think it would be silly, but I also think that having a random money sink in a game where money doesn't play any role is silly too lol but i guess its not really that big of a "sink" compared to other useless money sinks in games.

    The four shinies where found in FS where all 3 pokemon where unlocked for me to find, IDK if that might play a role in it too.

    I know four shinies wont help your numbers out but I will try to remember to post my numbers whenever I do some more FS hunting.
    Animelee likes this.
  18. nocturne24

    nocturne24

    Joined:
    Oct 30, 2013
    Messages:
    14
    Hey all. First of all - Scientist, this is an excellent thread. I have been running around in safaris for much of the past 4 months and really appreciated your input. I do not have a clear number, but estimate at least 100 different shiny pokemon I have caught this way.

    Now, I wanted to share a few observations with you lot:
    1. Based on my experience so far I find a shiny for every 3-4h of safari time (400-600 encounters). This does rougly correlate with what is presented above.
    2. I have seen no difference from being in a safari with friends online vs. offline. As mentioned, if the difference is there it may be subtle enough not to be observed just by going in and out of battles
    3. I have also had no significant difference between 3 pokemon safari and 2 pokemon safari. In fact, I had a number of suprisingly fast shiny finds (within 20-30 encounters) in safaris where I only knew two of the three pokemon.
    4. I am uncertain about the different shiny rates in different safaris. I am aware of the high likelihood of recall bias, but on few occassions I would find multiple shiny in a single safari relatively easy, only to waste days in a different one. Again, if the rate was indeed 1/512 -- this should explain such a distribution...
  19. Diegolas

    Diegolas

    Joined:
    Apr 4, 2014
    Messages:
    2
    Hey, I'm not a shiny hunter in the FS yet, but I just wanted to say thank you for all the information that you gave us! Keep going!
  20. StarKO

    StarKO

    Joined:
    Mar 12, 2014
    Messages:
    27
    Thank you for doing this! I was looking for some solid proof of the increased shiny odds in FS, and after reading your analysis I was encouraged to go shiny hunting myself. I've found 2 shinies now, both taking approx 350-400 encounters.

    I should mention that my unlucky friend has now gone over 2000 encounters in FS without finding a shiny. If the odds are indeed 1/512, this has around a 2% chance of happening so it's far from impossible, but still. I really doubt there would be different odds for different safaris but for what it's worth he's been trying to find a shiny Heracross in a bug safari, specifically Heracross/Volbeat/Butterfree.
  21. HFS

    HFS

    Joined:
    Apr 6, 2014
    Messages:
    3
    I just got my shiny charm yesterday so I tried the friend safari and got a Gabite after 71 encounters, so without the shiny charm would be about 213 encounters so I was above average which is a nice change of pace.
  22. Animelee

    Animelee

    Joined:
    Oct 4, 2013
    Messages:
    11
    Hey The Scientist, I was wondering if you had any new data to share with us? Thank you very much, once again!
  23. The Scientist

    The Scientist Heheh, not a Silph worker...
    is a Pokemon Researcher

    Joined:
    Jan 1, 2014
    Messages:
    19
    Sorry, I've been busy lately. However, the bot is still going and I've updated the sheet. It's at ~167,000 encounters and 311 shinies. The average #encounters per shiny is currently at 536 with a standard deviation of 554. I only went to 100 shinies during the tipping trials. More would have been better, but to me it doesn't look like tipping had any effect. The data from both tipping and non-tipping trials had distributions that matched a 1/512 distribution far better than a 1/455 or 1/585 distribution. If anyone wants to do a power analysis or other types of statistics on the data (see Google Doc in signature), be my guest.

    In addition, I'm leaving the country later this week, so data collection will be halted for the next four or so months. It's been fun, and I learned a lot about probability. I'll still be able to check on the thread here and answer questions. Thanks for the conversation!
    chronomancy and Animelee like this.
  24. Animelee

    Animelee

    Joined:
    Oct 4, 2013
    Messages:
    11
    Thank you very much!
  25. Dr. Eyepatch

    Dr. Eyepatch

    Joined:
    Aug 17, 2007
    Messages:
    608
    I have always considered myself The Scientist, but the name suits you much more nicely.

    Firstly, thank you very much for doing this project. It must have been a huge undertaking to venture into unknown territory such as this, and finally having some data behind the likeliness of shiny encounters in the Friend Safari is highly appreciated.

    In terms of personal experience with the Friend Safari, I have captured three or four shinies while visiting, in a relatively fast amount of time. While I do believe all but one encounter occurred while the owner of each respective safari was online, even though it lead me to believe the shiny rate increases in such conditions, I do realize my experiences are much too sparse to contribute with any degree of usefulness. During one safari venture, I had spent 2-3 hours searching for a Supersize Pumpkaboo; I had decided to give up when I ended up finding a shiny Pumpkaboo before ever even seeing a Supersize one.

    I also wanted to mention that, while tipping does not seem to increase shiny encounter rates in the Friend Safari much at all (if at all), it should be important to note that such data only applies to Friend Safaris and should not be considered evidence for its potential increase (or lack thereof) in shiny encounter rates elsewhere.

    As I am sure it will be always, keep up the great work~! ^ ^
    The Scientist likes this.

Users Viewing Thread (Users: 0, Guests: 7)