No way, it isn't a silly question at all!
It seems like your asking is - "isn't the brain so important that it should be protected in the torso? Wouldn't this protection be a selective advantage?" This question poses the problem of which is more selectively advantageous- protecting the brain or quickly processing sensory information. The answer is the later, as you say, it does take time to transfer signals to the brain. It is extremely benificial to have your central processing center (brain) close to your sensory organs as this reduces the time it takes to receive and process information. Perhaps the sensory organs could be placed about the torso, thus solving this problem... however, this would offset the benefits of having the sensory organs high on the body where they can gather the most information (see over tall grass etc...)
But this is all moot. Evolution doesn't really -create- perfect systems, it jury rigs workable solutions out of what's present already. Remember, natural selection evolves the simplest solution in the shortest number of generations possible.
ETA: I also wanted to say that even a possible trait may be extremely advantageous, it has to be present in the population to be selected on- the trait must be genetic, the mutation must exist in the population. Also, the mutation causing the advantageous trait must not be coupled with other very negative traits. By the time we reach complex organisms like mice and humans, the body plan is involved in so many other functions that global alterations in its structure are generally deleterious. Thus such a mutation would likely never arise in a mammalian population to be selected for in the first place.
