The City of London has lots of little gardens amid its forbidding edifices, some better known than others. Christchurch Greyfriars can hardly be described as secret, sitting as it does in the shadow of St Paul's and on the main thoroughfare from Holborn to Bank. This bombed out Wren church (on the site of a former Franciscan monastery) is still a very pleasant place to while away a spare half hour, and if you face the right way you can almost forget you're mere metres away from buses and taxis.
These two other gardens, however, are a bit more off the beaten track. The churchyard of St John Zachary is on the corner of Gresham Street and Noble Street, with raised walls and seemingly attached to a corporate headquarters so at first sight can seem off limits to the public. But if the splendidly feline-faced gates are open you can wander in and spend some time on the site of the old church, damaged in the fire of 1666 but not fully cleared away until after World War Two. The site is owned by the Worshipful Company of Goldsmiths who, a plaque near the entrance declares, maintain it "for the enjoyment of the citizens of London". And is much more pleasant than the neighbouring garden of St Ann and St Agnes, which these days is smokers' corner.
St Swithin's garden is hidden away off Cannon Street, in Salter's Hall Court. It's the site of another Wren church that was unsalvagable after being bombed in WW2, though there's thought to have been a church on the site since the 11th century. The church's south wall housed the London Stone from the early 19th century — it's still there round the corner from the garden, though now set low in the wall of an office block.