So far as everything I read and heard while I was there, the baths were free to all. I believe some were charged to keep out the riff-raff. Rich Romans might set up a private bath in their homes, but would still frequent public baths for the communal aspect of Roman bathing.
I can't remember if the massages were free. The massages were considered an important part of the baths.
In the standard Roman view of Public Works and Services, ensuring the baths were free and accessible was an important part of government administration. If any Consuls (republic) or Emperor (empire) allowed them to lapse, they were viewed unfavorably.
I visited the Diocletian Baths, one of the largest and most opulent in their day. To give you an idea how big, what amounted to the foyer, the entrance to the Baths, is now a large cathedral, the Santa Maria degli Angeli. It's amazing. It's on the Piazza della Repubblica in Rome.