This would seem to call for some sort of legal protection. At first glance, it seems like we could just declare privacy-sensitive information to fall under copyright laws and be done with it. If this worked, it would have the virtue of simplicity. Unfortunately, on closer examination it doesn't work at all.
First, original copyright law deals only with expressions. You cannot copyright a fact, for several reasons, not least of which is failure to meet the creativity criterion. This makes it extremely difficult to use copyright machinery to protect such information. Ignoring the creativity problem, you could try to justify other's recordings of your address as a derivative work of your original recording of your address, but that does nothing to prevent people from independently recording your address, getting their own "copyright", and leaving you unprotected.
Second, as we discussed above, copyright in its current form really only deals with the concrete part aspect of communication. Our whole desire for privacy centers around the desire to control the flow of the information, which is to say, the human-experienced part of the communication. This highlights one way in which that division is sensible. Copyright is not a very good mechanism right now because the expression model can't handle this sort of information, and hopefully once copyright is simplified to cover only the concrete part aspect, it will be even less appropriate. We need some other form of protection, one that does not exist right now.
In its current form, copyright is primarily concerned with the recovery of loss. The penalties for copyright violation increase as the damage done to the copyright owner increases. We want a legal mechanism concerned with the prevention of injury, which is completely different.
The closest currently-existing legal mechanism that meets that criterion is trade secret law. There are some similarities: Trade secrets protect information of economic value as long as it is maintained secret. It is concerned with preventing the secret from getting out and being used by somebody else for gain, which sounds like how we'd like to protect privacy.
But there are some serious problems, too: Once the trade secret is independently found, it is no longer protected, so one accidental release of your address without proper trade secret protection and it's no longer a secret. Since by now we've all released all kinds of personal information without trade secret protection, we can't even claim trade secret status on our information in theory or in an ambitious lawsuit. It also (as far as I can tell) deals strictly with monetary value, where our privacy concerns go beyond that, as we wish to be able to consider some information priceless, as is our right to set the value of our information.
Current legislation dealing directly with privacy suffers from the same symptoms as the rest of intellectual property law. It is haphazard and chaotic as it tries to deal piecemeal with each isolated situation as it arises, instead of being based on a cohesive theory of privacy. It is a list of special cases, which is obsolete before it is even put into effect. It is clearly inadequate for the larger task of protecting people's privacy as a whole.
So to answer the question posed in the title question: No, there is no existing legal machinery that we can use or extend to protect our privacy. Even the current privacy laws are too focused to be made generally useful.