The white stuff in your ice cubes is actually very, very tiny air bubbles. Virtually all natural water you deal with is oxygenated to some extent; it’s why fish can breathe in it. Scientists measure dissolved oxygen in streams to determine how healthy the environment is. And when the water flows from your tap, it also tends to be pretty well oxygenated.
As the water freezes, it wants to form a regular crystalline structure (ice). That means impurities like oxygen and other dissolved gasses are pushed away from the crystallization front into the remaining liquid. This means that the last parts of the ice cube to crystallize end up with most of the impurities. If you take a look at an ice cube, you’ll note that the parts of the ice cube that were at the bottom of the tray are the milky colored bits. That’s because those were the last parts to freeze. The ice cubes in your tray froze from the top downward, which pushed the contaminant air downward until it was trapped at the bottom and had nowhere else to go.
You can, however, get bubble-free ice by first deoxygenating it. Oxygen solubility in water decreases with increasing temperature. If you boil the water first, you remove the oxygen from it. Next, cool it and then pour it into your ice cube trays to freeze. It should be much more clear.