I want to experience public bath in Japan..is there any in Tokyo ? how much does it costs to go in ? for how many hours? and the most important thing..do they accept foreigner?
what you're looking for are 温泉 (onsen), and can cost anywhere from very reasonable to very expensive. A quick search brought these links up, or you could try asking your hotel's staff if there are any in the area.
watashi to tokyo Odaiba
one name that kept coming up was Oedo Onsen Monogatari. More info and pricing information are in that second link (it's on the expensive side), and here is a short review of the place.
Also, I recommend reading a guide to bathing etiquette, to prevent any misunderstandings, such as these: Onsen Etiquette