Mystery Monday: The Maine Milburns (or Millburns)

Freshly inspired after a week at the Salt Lake Institute of Genealogy and a few hours of online streaming from RootsTech (and snowed in by the blizzard in Colorado) I dived into one of my boxes of genealogy papers and started scanning and typing.

I have neglected my Milburn tree because, well, "everybody" else has worked on it and written books about it and seemed to have it all sewn up.  But AH HA!  A mystery has surfaced.

I was looking at the records for my grandfather's nephew (or as states; 'my great grand uncle').
John Oliver Milburn.  John's parents, Joseph Milburn & Martha McNutt had married in Nova Scotia, Canada, lived in New Brunswick, and as they made their way to Iowa, Martha gave birth to John Oliver Milburn in Calais, Washington County, Maine.
The profile page for John Oliver Milburn and his wives, Henrietta Stapleton and 

John apparently marries Henrietta Stapleton, probably around 1858 - 1859 in Maine.  They have a daughter Mary who is born in Long Island, Hancock County, Maine in 1859.  A family story said that John was a first mate on a merchant ship that sailed out of Boston, around the world and that his wife and child had died while he was at sea!  He supposedly left the sea life at that time and went on to build the first steel bridges in Texas, one of which was over the Brazos River north of Houston.  He worked for the King Bridge Company and also built bridges in Richardson County, Nebraska.   He eventually moved to Pilger, Nebraska.

John shows up in Iowa in 1880 married to Mary Hower with whom he has 4 children (they eventually move to Nebraska).

The note I found states that Henrietta Stapleton shows up in the 1880 census in Washington County, Maine with daughter Mary, living with Cynthia Winchester as a boarder, and next door to Etta Milburn.  (though I can't find this record just now).

So ... the mystery...  Did John just abandon his wife and daughter?  Were they divorced?  What happened to them?  Where is that 1880 census record the note mentions?

Further info can be found on John's brother, Joseph S Milburn at this google site.
I had also posted a query on Ancestry's Message Board for Stanton County, Nebraska.