They do look a bit like I-Cybie batteries in how they're put together (you can even see a circuit board of some sort sticking out from under the plastic wrap), but they're a lot smaller than an I-Cybie or Aibo battery. The battery seems to be a little over half the size of a Pleo battery. I don't think Dongbu put much effort into making the battery look nice since it's not that easily replaceable and the majority of owners will probably never see what their Genibo's battery looks like.
Here's a picture of the battery in relation to some other robot batteries:

(From left to right: Aibo 210, Genibo, Pleo Ucube, I-Cybie)
I tried replacing Baxter's old battery yesterday, but I couldn't get the first panel off after removing the screws, so I'm waiting for a reply from the distributor about how to remove the panels.