Touhou is a series of games. All of the integer-numbered Touhou games are shmups--a genre of game that involves flying around an area as you should and dodge a barrage of enemies. (For other examples of shmups, see Gradius and Ikaruga.) One thing that makes Touhou different from most shmups is that the focus is more on dodging than shooting: very often, you're faced with intricately-woven bullet patterns which require fancy flying to avoid being hit. This is often known as "bullet hell," or "danmaku" (fire curtain).
A download page with the games can be found here.
Games 6-11 are for Windows and I'd recommend that you start somewhere in that area. Embodiment of Scarlet Devil (6), Perfect Cherry Blossom (7), or Imperishable Night (8) would all be good games to start with.
There are official games that are numbered with non-integers, such as 7.5 (Immaterial and Missing Power), 10.5 (Scarlet Weather Rhapsody), and 9.5 (Shoot the Bullet) that are different from the "main games." The first two are 2-D fighters, and Shoot the Bullet requires you to take pictures of bullet patterns rather than shoot enemies.
One of the reasons that Touhou has become so popular is that its fanbase has spawned a massive quantity of derivative work. There are literally hundreds of albums which contain covers of the melodies found in the official games, and there are lots of unofficial (doujin) games using characters from the main games which span multiple genres. There's also abundant quantities of artwork and fiction (in manga form) for the series' characters. The cast is almost entirely female, which is probably one of the reasons for this.
