With a good tailor, Gary looks like a normal human being. However, closer observation reveals a number of distinct differences from the standard model of the genus homo homo sapiens.
Four of his five senses are significantly different (or defective, depending on your viewpoint) from normal. Here, he was mostly born normal, but it wore off.
Taste: He has no sense of taste, literally. (In a figurative sense, his lack of taste is more a matter of opinion.) His tongue lacks taste-buds so he cannot taste sweet, salty, sour, or bitter in the normal sense, though he can feel some chemical differences in these states, as burning or biting sensations. Though he enjoys the "taste" of food, his perception of it is very different from normal people's. He was born with taste-buds and remembers the "normal" taste of food. He also lacks saliva glands so he is required to drink (and eliminate) about a gallon and a half more water a day than a normal person. Both of these conditions are the result of cancer surgery and subsequent radiation.
Vision: He cannot see normal colors but does see color shades that normal people cannot see. This may account for the distinctive colors in his cover art. This condition is congenital. Today, the vision of each of his eyes is specialized. His left eye sees at a distance while his right eye sees close objects and is used for reading. This is known as "mono-vision." This condition is the result of unsuccessful Lasik eye surgery to correct vision so bad that he not only couldn't find his glass if he dropped them, but he couldn't find the ground.
Touch: His sense of touch is similarly specialized. While most people are either right or left-handed, Gary has an unusual from of ambidextrousness. His left hand is used for detailed tasks such as writing or drawing while his right hand is used for strength tasks such as throwing a ball or golfing. There is a difference in size of his hands. His inability to perceive right and left (see below) may be related. This condition is congenital. He is without the sense of feeling in certain parts of his body, unable to feel either pain or pressure. This later condition is the result of nerve damage from cancer surgery. If you touch him on the shoulder and he doesn't recognize you he isn't being rude: one of his shoulders is without feeling.
Hearing: His hearing is very sensitive. In his fifties, hearing tests show he has the audio acuity of a young child. Because of this, he has a low tolerance for noisy environments. He has a fine sense of pitch, in the sense of whether a note is flat or sharp, but he is unable to easily tell if a given note is "higher" or "lower" than another.
Smell: As far as it can be compared to that of others, his sense of smell seems normal.
Remember Mel Brook's Young Frankenstein? Some of Gary's sensory problems, such as the lack of taste buds or pain sensors are physical, but most are differences in the brain. There are a number of other brain abnormalities as well.
He does not perceive directions normally, neither right nor left nor up or down. When told to go to the right, his chances of doing so are no more than 50/50 unless he takes time to think about it. Despite knowing about this disability, he has occasionally been convinced for several hours that other people are mixing up their rights and lefts.
His condition mimics or is a form of dyslexia. He confuses letters such as "b" and "d," and "p."
Gary's memory is strange. He knows a vast about of science, history, news detail, and reads several modern languages and a few ancient ones. However, he has only a vague memory of his personal life beyond a few years in the past. Like many people, he has a weak grasp of names and faces, but for much of his life he could remember most of what he read verbatim decades later, especially facts and numbers. However, he notices that after his forties, he seems to have hit a wall in terms of the ability to quickly access large amounts of raw information that he has not worked currently. For example, while he is an expert in ancient Chinese, if he is away from its study for a few months, he loses almost all of his recognition of characters and has to spend a few days refreshing himself to get his skill back. Right now, he is working in ancient Greek and can bore you for hours on "interesting" aspects of the language and vocabulary but if he moves on Arabic, his memory of the vocabulary and grammar will be gone in a matter of weeks.
Gary has never had any meaningful skills of physical coordination. He has no physical sense of rhythm. This sounds like another joke, like having no sense of taste, but it is also physical difference in the brain. For example, he has to watch (as oppose to listen to) others to mimic clapping to a beat. This may or may nor be related to abnormalities in heart rhythm.
These is some medical literature on some of these conditions being related to a different in the cerebellum. Color-blindness, dyslexia, and problems with rhythm are all related to that part of the brain in someway.
Other Physical Abnormalities
He is missing most muscles in the right side of his neck due to cancer surgery. The remaining muscles on that side are over-developed from long periods of rigidity and cramping, due to related nerve damage. This condition is rather painful but can be treated with alcohol, ideally beer or several bizarre liquor concoctions that only someone without taste buds can appreciate.
There is a some problem with his heart that remains undiagnosed where it lapses into periods of irregular rhythm. This never happens during strenuous physical exercise (Gary enjoys ridiculously long periods of exercise when he has the time), than during periods of change from resting to moving.
Physically, his torso is long in proportion to his arms and legs. He appears to be a tall man sitting down, but he is an average-sized person when standing.
His body temperature is a degree or two cooler than human average.
Grateful for the Appearance of Normality
Despite these defects, Gary functions fairly normally and most of his abnormalities go unnoticed by the casual observer. Gary is grateful for this appearance of normality.
If his disabilities were obvious, he would be tempted to blame people's prejudice whenever they shunned or rejected him. Since he cannot change his physical or mental defects, this constant rejection would make him very sad and perhaps angry at the world. As it is though, he realizes that when people reject or shun him, it is probably because of something stupid that he said or did rather than for what he is. This gives him the continuous hope that he can change and improve himself in people's eyes over time.