You can solder it but its not easy, they are actually spot welded in originally, to do it you gotta scratch some of the shiny surface off first but even then its difficult to get solder to take, and you have to be real quick if you heat the guts of the battery too much it stuffs it, hold the battery with a pair of pliers as close to where you want to solder as possible to conduct the heat away.
Ive done it before and found the easiest way for me was to solder a couple of little wires to the battery top and bottom first then coat the whole thing in silicon to insulate it, then carefully trim most of the original tabs that were welded to the battery away from the circuit board just leaving enough to solder the new little battery wires to, you can then solder these without the battery being in the way. Then stick the battery to the board with a little blob of silicon.