Where you stand right now
You paid, you waited, and a certificate came back. That certificate is genuine. The difficulty is that an apostille for Sri Lanka is not the proof its offices work with, because the country sits outside the Hague Apostille Convention and papers going there require embassy legalisation instead. So nothing has been destroyed and nothing has been faked. You are simply standing at the wrong door with a real key.
What the stamp still is
An apostille issued in Newcastle upon Tyne for use in Sri Lanka remains a British act by a British office, correctly performed. It has no power to force acceptance abroad, and it never did — that was never in its gift. Keep the sheet. It shows the date you started, the document it was attached to, and the office that handled it, and that history is worth having if questions arise later.
Restarting the file cleanly
The quickest recovery is usually a clean sheet rather than an argument. Newcastle Register Office at Civic Centre, Barras Bridge, Newcastle upon Tyne NE1 8QH issues certified copies of the births, marriages and deaths registered in the city, and a fresh copy costs far less than the weeks a contested bundle can absorb. Readers searching for what you need for an apostille for Sri Lanka often arrive here at exactly this point.
The mistake behind the mistake
An apostille from United Kingdom to Sri Lanka sounds like a service you can buy, and that phrasing is where people go astray. The single-stamp route is an arrangement between countries that have both signed up, not a product on a British price list. The Legalisation Office (Foreign, Commonwealth & Development Office) will stamp a paper for any destination you write down, because policing your route is not its job.
The road ahead
Document legalisation for Sri Lanka runs through more desks than the stamping route does, so build the calendar with that in mind and start now rather than after one more attempt at a shortcut. Write the route at the top of your list. Order the papers you need. Keep your own originals at home, and let replaceable copies do all the travelling from here on.