I've seen photos of trousers with one back pocket, on the left side, that seemed to date from the 1880s. There's a photo of Apache scouts gambling in the Time/Life Old West series that I believe also dates from the late 1880s. I'm not sure I'd bless off on back pockets in the early 1860s, though.
Pine River Boys, Co I, 7th Wisconsin
"We're... Christians, what read the Bible and foller what it says about lovin' your enemies and carin' for them what despitefully use you -- that is, after you've downed 'em good and hard."
-Si Klegg and His Pard Shorty