The bone, thought to be from a dinosaur called Iguanodon, was found by a member of the public when they were digging among tree roots in their garden.
It was handed in by the anonymous discoverer to Sunderland Museum and Winter Gardens to see what their history experts made of it.
Museum manager Jo Cunningham said it was an "amazing find".
The museum's keeper of geology, Sylvia Humphrey, spotted the possibility of the bone belonging to the Iguanodon species.
She said: "It's really quite a puzzle as to how the bone got there. Dinosaur bones are younger than the rocks of this area, as this region is on the Permian strata, which is 250 million years old.
"The rocks of this region are far too old for it to have lain here, so it has been lost or dropped by someone in the past.
"We think, although we can never be sure, that it is a piece of vertebrae from an Iguanodon, and may originate from the Wealden area."
The Wealden is a rock formation in the South East of England and it has historically yielded many dinosaur fossils, mostly that of Iguanodon-like creatures.
Ms Humphrey contacted specialists at the Natural History Museum in London, who verified the find is from the spine or tail of an Iguanodon-like dinosaur.
Dr Angela Milner, from the Natural History Museum, said: "It is not complete enough to identify it more precisely.
"The rocks around Sunderland are much too old to contain dinosaur bones so there are only two explanations as to how it got there - either by glacial transport or a one-time souvenir from the south coast of England where Iguanodon bones are not infrequently found by fossil hunters."
The Iguanodon, meaning Iguana tooth, grew up to 10m long and walked the earth 130 to 115 million years ago.
It was the first dinosaur to be recognised by 19th Century doctor Gideon Mantell, who collected Iguanodon teeth and bones.
The museum said the bone found in Sunderland has similarities to material from a collection on display at the Great North Museum: Hancock in Newcastle.
Jo Cunningham added: "We're very grateful to our museum visitor for bringing this amazing find in to us.
"It will always remain a mystery as to how it found it's way there, and if they hadn't been digging up their garden it could have lain undiscovered."