When I was 30, I had 6.7 millimoles/litre cholesterol in my body.
Was it HDL "good" cholesterol, which is on its way to the liver for excretion? Or LDL "bad" cholesterol, which is on its way to be stored in the rest of your body? Nowadays, at least in the USA, they distinguish between the two and only worry about the LDL. If they only gave you one figure for "generic" cholesterol, then I'm worried that you may have been misled.
Doctors asked me to cut down on diet and do exrcise. Which I did and lost 10 kg of weight. (Stopped eating red meat, milk products, egg etc have been eating lot of fruits and vegetables.)
For purposes of managing cholesterol, the top priority is to stop gaining weight, because as your fat cells increase, they will store whatever cholesterol is available within their cell walls.
Your second priority is to reduce or eliminate trans-fatty acids from your diet. As long as your weight is stable and healthy, it's okay to eat saturated fat, which is found in meat, eggs, milk and a few fruits, nuts and seeds, and it's okay to eat unsaturated fat, which is more common in plant-derived foods. Generally your body is able to digest those forms of fat, metabolize them and burn them for energy. But partially hydrogenated fats, which contain trans-fatty acids and are rampant in manufactured food because they retard spoilage, are somewhat more difficult to digest and may end up being stored in your fat cells as cholesterol.
You don't need to give up meat to be healthy, as long as you don't eat charred meat in which the natural fat has been turned into some other and less healthy chemical. Just avoid buying packaged food, especially sweets, because they are usually full of partially hydrogenated fats to increase their shelf life, and so they won't spoil if you leave an open bag of potato chips or cookies out on your coffee table for two weeks. Read the label on every package of food you buy and scan for the words "partially hydrogenated." If you see them, put it back. If you really like sweets you may end up having to learn to cook, or else buy more expensive sweets at a real bakery. (Check their labels too!!!) Fortunately my wife is the best cook in seven counties AND a chocolatiere.
Sugar? Well if you're gaining weight of course you should cut back on the calories. But sugar is not your only source of calories. If you love sweets and can't make it through a day without them (like me), telling you to give up sugar is like telling you to stop breathing. You have to make the decision about what you're willing to give up and still maintain a balanced diet without gaining weight. I eat a normal 2500 calorie diet, and probably half of that is sugar. I eat plenty of fat in meat, eggs, cheese and pastry. But I eat zero trans-fatty acids and I exercise enough that my weight is stable. I've been eating like that for 66 years and I'm fine. Well okay we didn't know about trans-fatty acids when I was a kid but I stopped eating them when they were discovered. Fortunately I hate potato chips and I have never bought cheap candy or pastry.
So I suppose not taking white sugar does help cholesterol level to be maintained at a healthy level!
But what were you eating with that sugar? Was it doughnuts and crackers from the convenience store with a six-month shelf life, that even the bugs won't eat? That crap is full of trans-fatty acids.
Sugar is much maligned. In the past only the rich could afford it so it became a symbol of decadence. Now that everyone in the West can afford it, the communists and the people in the Third World still consider it a symbol of decadence. That is bullshit. Europeans eat plenty of sugar and they're generally healthier than us Americans.
But your first priority is to make sure your doctor is measuring cholesterol correctly. Only the LDL is a problem, so long as you're not gaining weight and your health is otherwise good. Then just manage your diet intelligently so you don't gain weight. Don't get crazy about it.