Well, it all depends on where you're in public.

On a train, drinking a soda is not cool. It's not like there's a sign that says "No drinking or eating on trains." It's just that it's not socially acceptable because you'd be a nuisance to other riders. How so? Because most people who consume a drink/food tend to leave behind their finished cans or wrappers on the floor, and that's what people don't want. And even if you take your trash away, they may still give you dirty looks because they worry you might spill your juice on the seat, or worse, on them, and leave crumbs all over the place.

If, however, you've brought your own water bottle and you want to take a sip from it because it's a hot summer day, you can and I don't think people would mind, provided that you do this discreetly and not make a lot of noise. On a train, where there tends to be a lot of people in very close quarters, you must give a lot of considerations to others. (Okay, you don't need to follow this rule if you're on a Shinkansen bullet train or on some other long distance trains for obvious reasons.)

I'd also not eat or drink at a museum or a city hall unless at a designated area like a kiosk or a cafe, and most public places do have such corners.

Now, on the other hand, if you're at a park and sitting on a bench, you can definitely eat and drink all you want. Also, at a shotengai (shopping street lined with food stalls and other shops) or at an omatsuri (temple fair), then you could, and should, eat and drink as much as you like. And mind you, after you've finished a drink or a sandwich, make sure you take it home with you or ditch it at a public trash can. If you can't find one, which happens because public trash cans are not so readily available in Tokyo sometimes, then try deposit it in a garbage can at a nearby convenience store.

In general, if there's a vending machine selling soda pops, or food stalls nearby, then this is a cue that you can consume food and drinks. But then there are so many convenience stores in Japan, where they usually set up a corner for people to eat and drink, most of the time you don't really have a reason to eat and drink on the go.

I hope this helps.