It is perhaps not so outlandish to think there may be no life visible to us. Owing to the light speed limit, we can only have contact with a certain region of spacetime. The further out we look, the younger the Universe becomes.
For a planetary system to form (and certainly for terrestrial planets to form), there need to be sufficient heavy elements. These heavy elements are formed in previous generations of stars, and spread when they go supernova. The Sun is believed to be a third generation star - it's likely first and second generation stars could not have had planets. This thus may impose a minimum age after the start of the Universe for life to reasonably emerge, and consequently a somewhat smaller region of space than the entire observable Universe.
That said, there are many known extrasolar planets. Few are particularly earthlike, but that is simply because we cannot so easily detect small rocky planets compared to big gas giants. Life is known to have arose on Earth very early in its existence, which suggests it readily arises in a suitable environment. So for life to be rare, suitable environments would have to be rare. This is the factor we are really unsure of.
By contrast, intelligence took billions of years to arise (as far as we know - I'd have thought we'd have found traces of a previous intelligent species on Earth, but maybe we wouldn't). Thus, we could be the only intelligent species within our visible horizon. Supporting the plausibility of that is the consideration that someone has to be first. Even if intelligent species turn out to be a dime-a-dozen in the Universe in the next few billion years, there has to be a first one, and it will be alone. It could be us.
For a planetary system to form (and certainly for terrestrial planets to form), there need to be sufficient heavy elements. These heavy elements are formed in previous generations of stars, and spread when they go supernova. The Sun is believed to be a third generation star - it's likely first and second generation stars could not have had planets. This thus may impose a minimum age after the start of the Universe for life to reasonably emerge, and consequently a somewhat smaller region of space than the entire observable Universe.
That said, there are many known extrasolar planets. Few are particularly earthlike, but that is simply because we cannot so easily detect small rocky planets compared to big gas giants. Life is known to have arose on Earth very early in its existence, which suggests it readily arises in a suitable environment. So for life to be rare, suitable environments would have to be rare. This is the factor we are really unsure of.
By contrast, intelligence took billions of years to arise (as far as we know - I'd have thought we'd have found traces of a previous intelligent species on Earth, but maybe we wouldn't). Thus, we could be the only intelligent species within our visible horizon. Supporting the plausibility of that is the consideration that someone has to be first. Even if intelligent species turn out to be a dime-a-dozen in the Universe in the next few billion years, there has to be a first one, and it will be alone. It could be us.

