I have gone swimming at the beach in Churchill, Manitoba.
I am not sure how reliable the swimming is. I was there at the end of August, which was so off-season that the main coffee shop in town had closed for the week. The proprietors of my bed and breakfast thought I was crazy, and accompanied me for safety. I remember the water being cold, but not as cold as some places I've swum in the Adirondacks.
Regarding safety, the beach is directly behind the hospital, but it is also covered in signs warning of the dangers of polar bears. A day or two later, a few miles away, a man was mauled by a polar bear while he was on a ladder against a building.
It's funny you say "preferably by public transit", because you can't get there by car. There are roads in Churchill, but they don't go far enough to get to any other towns. The road for which you do not need to take special precautions ends in Thompson, and routes for cars of any kind end in Gillam, but I was able to take a train from Churchill to Winnipeg. That train took 2-3 days, but does connect you to more normal transit options.
Churchill is definitely a tourist destination, but for people wanting to see polar bears, beluga whales, the northern lights, and birds. I spent some of my time there in a t-shirt, but I think most people would not consider it "warm enough" to be a beach resort. I remember meeting an Italian lady there, who gave the impression that she had not yet thawed from the preceding winter. Similarly, I went swimming at Deer Lake, which has a beach without the polar bear warning signs. I remember others using it as a beach, but I don't remember anyone else going in the water.