It may be illegal if others can see you. With your neighbor being able to see you qualifies as this but if you know he won't complain then your okay but I would question it if he has the idea and brought it up to you. It must mean that he is a little disturbed by it.
It probably is illegal if you share your garden because you can only be naked in private places. If you are naked and answer the door to someone that is not illegal as long as you do not step off of your property and sunbathing nude in your own private garden is also not illegal even if people can see you. If I were you I would sunbathe nude anymore as that man sounds like a bit of a pervert bringing you drinks and stuff. Probably just wants an eyeful.
If you live in England, Wales & Northern Ireland, in theory, it is not illegal to be nude anywhere, although you could be charged with a Public order offence e.g. Conduct likely to cause a Breach of the Peace. It is an offence for someone, male or female, to expose their genitals if they intended another person to see them and to be caused alarm or distress. There is also the common law offence of outraging public decency. If you deliberately annoy your neighbours you could be served with an Antisocial Behaviour Order. However, as your male neighbour appears to have no problem, I can see no reason why you should not continue sunbathing nude.
It is more than likely illegal but I believed the neighbor would ask you to stop first. If you continue my guess is the police would come by and warn you not to do it again. If you continue after you are warned by the police you would more than likely go to jail.
