3 or 4, hm...
Well, quoting a few posts I've seem in this thread and drawing my own conclusions, it seems the standard BP team can be divided on that structure:
Scolipede: speed passer + backup defense booster
Espeon: anti-taunt + anti-phaser + sweeper + backup Sp. def booster
Sylveon: Sp. Def booster + special wall + backup sweeper
Vaporeon: defense booster + physical wall (backup if Zapdos is already in the team)
Smeargle: panic button (with spore and encore) + backup anti-phaser (with ingrain)
Mr. Mime: anti-perish song + backup Sp. Def booster + backup anti-phaser
Zapdos: physical wall (dunno what else)
So what matters for the bp team is whether all these functions are fulfilled, with extra pokemon being backup. But the essential functions are: speed booster, def booster, Sp. Def booster, anti-taunt, anti-phaser, physical wall, special wall, and sweeper. As you can notice, Smeargle and Mr. Mime are just bonuses to the team, acting as either backup, or situational panic buttons against unpredicted counters. Zapdos does the job of being a premier physical wall against pokemon who would threat the team otherwise, mostly Talonflame and Pinsir (which is also the reason why special attackers have been recommended so much in this thread, there isn't a special wall in the standard bp team as effective as Zapdos is a physical one). So the other 4 mons make the essential build of a bp team, fulfilling all the functions described above. Sweeper is an essential function, but it does not require bp on the same set to work, so I've not bolded it.
So a team with limited pokemon would have 2 groups: the baton pass group, consisting of 3 or 4 pokemon who must be able to fulfill all of the jobs in bold while using baton pass, and the cleaning group, a group of pokemon who would be dedicated to removing counters so the baton pass team could function, or do the inverse, sweeping the opposing team after the bp team is used for wall breaking, or even receiving boosts to help in the sweeping. Those would include Zapdos, and possibily a special wall (chansey comes to mind), maybe a third wall, or a sweeper without baton pass. In this way, a limited baton pass team could focus on the removal of counters, but it would have to wait until these counters were properly removed before starting the chain, in which case the chain would be sort of a win condition.
But for that to be possible, all the functions described above MUST be present in pokemon with baton pass on the same set. So the question it: can it be done with 3 pokemon? Maybe a core of Scolipede + Espeon + Sylveon could work, but that would place all the pressure of raising defense on Scolipede, which isn't a bulky pokemon by itself. A bp team with 3 pokemon might be viable, but there is doubt if it would be competitive. A team of 4 pokemon, on the other hand, would probably still be competitive, and require more thought on team building and less auto pilot than a 6 man bp team, which is kind of a positive thing. But would it be competitive to the point of being broken? That's what needs to be discussed.
My suggestion: either do a suspect test or, if that is not desirable, limit bp teams to 4, and wait a few months. If after that they are still being a trouble to the metagame, limit them to 3 and call it a day.