I may be wrong about this, but it seems to me that the responses elicited by your question have completely missed the mark. That is to say, they didn't really answer your question.
* that "knowing what to believe" is NOT "having actual belief in what we know to believe
In the two instances given, you indicate two different things, "knowledge and belief." The first instance is KNOWLEDGE of WHAT to believe, while the second is BELIEF itself. The difficulty that lies in this concept is, "Once I know what to believe, don't I already believe it?" To this I would answer, no. Consider it in the light of knowing what NOT to believe. For example, I know I shouldn't believe in Santa Claus (the modern figure), yet I may still believe in him. Likewise, I know that I should believe in the existence of the world around me because it is the world I am familiar with, and in all likelyhood it's real, yet I may not believe in it. I may not believe it because I may think that because there is a POSSIBILITY that it isn't real, I have cause to doubt it. While such reasoning may be faulty, if not crazy, it illustrates the difference between knowledge and belief.
I am certain of the truth of mathematics, ergo I know the truth of mathematics. Because I know the truth of mathematics, I MUST believe, also, in the truth of mathematics, for to do otherwise would be silly.
It's can be a difficult concept to grasp, but just look at it this way. Just because I believe it, doesn't mean I know it, and just because I know it, doesn't mean it believe it.
I don't feel satisfied with this post so far, and I hope I'm not confusing, but I'd like to take a different approach to the question now.
The approach I'd like to take concerns two different forms of knowledge. One is sequential and one is holistic. These two forms, utilized by either hemisphere of the brain, and recognized as legitimate sources of knowledge are known as logic and intuition. Logic is the sequential examination of evidence performed to come to a precise conclusion (though logic can be wrong despite its attempt for precision). Intuition is a holistic examination of evidence performed to come to a general conclusion (as logic, it can also be wrong). In logical knowledge, one is able to state each and every reason for a given conclusion, since each evidence is examined individually, and understood by the examiner. However, that is not the case with intuitive knowledge. An intuiter may know something to be true, while not knowing the exact reasons for that knowledge. Please bear in mind that both are considered legitimate sources of knowledge in psychology because both are natural functions of the brain.
Science and philosophy are utilizers of the logical process of the brain, while religion and art are utilizers of the untuitive process of the brain. This is why many, or most religious people (who are not also philosophers or scientists) that you will run across will simply assert that they know what they believe to be true without a logical explanation of it. Religious belief is an intuitive process, while scientific belief is a logical process, both being natural functions of the brain and legitimate sources of knowledge.
This being said, we now have a way of distinguishing between knowledge that is inherently believed and knowledge that must be given belief. All logical knowledge is inherently believed, for if it weren't how could it be said to be knowledge? If I don't believe in that australia exists, how can I be said to know australia exists? Anyone who claims to know, BELIEVES in what they know, otherwise they wouldn't really know it. On the other hand, intuitive knowledge must be given belief, since it takes a leap to accept something as true without knowing why it's true. An example of this would be to believe that it is going to rain tomorrow because you just seem to know that it's going to happen. The actual evidence for it is found all around you, in weather pattern, physical changes within your own body, etc... yet the knower of this knowledge doesn't clearly know why or how he knows it, but just doesn't. What is happening is the evidence is being received by the unconscious mind, interpreted, and a 'sense of knowledge' is had by the conscious mind as the conclusion of the unconscious is being conveyed to the conscious.
Thus, to KNOW what to believe is to consider any given belief an see if it does have correlation to, but no clear supporting evidence to reality, and to believe it is to trust that it is true.
I hope this is clear, because I'm not really sure that it is.