I would suggest this is a Victorian folly/playhouse. Is there, or has there been a large house in the immediate area? Where the rock strata formed suitable outcrops near to planned building, properties were often built using stone quarried from these outcrops. When the quarry sites were located within the boundaries of the new house/estate, they were sometimes eventually 'landscaped' and added to the gardens as a rock feature.
Another reason could be that the area was a popular walking venue for the local population, the 'folly' being provided as a shelter by the local land owner. Perhaps the older maps of the area will indicate past use/ownership of the land in question, especially if there is now no evidence of a large house/estate in the immediate local.
I live in an area where you can still find the numerous remains of these outcrop quarries, for example much of the original property in Town Street Horsforth was built from stone quarried no more than 20 yards from the existing HBSC Bank. Here in Rawdon my house was built from stone quarried at the top of the hill, and in Cragg Wood there is clear evidence of a small quarry face being 'beautified' after the house opposite was built.