Have you ever been to a concert in a nuclear bunker? What about a potato barn? Or an aircraft hanger? If your answer is no, then chances are you’ve never attended the East Neuk Festival, where they do classical music just a little bit differently.

Tucked away in the glorious ‘nook’ of Fife, bordered by the dramatic coastline of the Firth of Forth, East Neuk is a 30-minute drive and a world away from the polite Victorian townhouses of St Andrews. Fishing villages cling to the coast, their low, severe architecture built to withstand centuries of wild weather. It’s enticing country for adventure tourists, but not an obvious location for a music festival.

‘The first year was a bit of an experiment,’ admits artistic director Svend Brown. ‘We were an unknown festival in a corner of Scotland that has no concert venues. Would anyone – artists or audience – even come?’ But come they did, enticed by serious international talent, including pianist and conductor Christian Zacharias and the Scottish Chamber Orchestra – artists who have since continued to return, year on year.

2014 marks East Neuk’s 10th anniversary, and the festival will celebrate by inviting back old friends – the Belcea and Elias Quartets, Zacharias, Welsh pianist Llŷr Williams, and of course the Scottish Chamber Orchestra – and by expanding its characteristic mixture of core chamber music and more experimental contemporary works into 10 days rather than the usual five.

It was thanks to one of these experimental works that I found myself wandering through the exquisite walled garden of Cambo House one hot afternoon last July (wearing sunglasses, rather than the wellingtons that previous years have required). Tucked behind trees, spread across lawns, or discovered among the larger shrubs in a herbaceous border were some 30 percussionists, each equipped with a GPS unit as well as their instrument.

John Luther Adams’s Inuksuit is more sound-installation than conventional composition, a ‘single breath’ that grows from the most delicate of musical stirrings, all but lost among birdsong and breeze, to a primal shout of deafening ferocity. Free to move among the musicians we all chose our own path through the soundscape – chasing sounds or simply standing still and letting them shift and coalesce around us in both echo and imitation, exaggeration and intensification of nature.

While the Cambo gardens won’t conceal any musicians this year, the festival will return to the estate’s potato barn – a magnificent (and occasionally muddy) venue for orchestral concerts, and unofficial home to the Scottish Chamber Orchestra during the festival.

Lit darkly green with projections, the early-evening light playing in the grooves of the corrugated iron, it transformed into an unexpectedly magical fairy garden for their performance of Ravel’s Mother Goose Suite that closed the 2013 festival. Anyone attending Britten’s Les Illuminations and Luther Adams’s …and bells remembered… this year (performed by the SCO and Allan Clayton) can expect something equally transformative.

Brown describes East Neuk as ‘very much a site-specific festival’, one whose programming adapts and responds to its range of very different venues. The small local churches of the region are a natural home to chamber music of all kinds, each bringing its own character to the music. The drawing-room elegance of Mozart or Schubert surely belongs in Craill’s graceful Parish Church, whose interior owes more to 18th-century renovations than its 13th-century origins. The churches of Kilrenny and Kilconquhar are more remote, more raw, with a certain contemporary cleanliness to both appearance and acoustic. But for rugged beauty it’s hard to beat St Monan’s.

Walking down a steeply curving path you think you’re approaching the water, but emerge suddenly instead in front of a church whose windows are dashed with spray on stormy days. Dipping its foundations into the water as it gazes out over the Firth of Forth, this tiny church has hosted the Elias Quartet’s ongoing Beethoven Project over the past few years, as well as my personal highlight of the 2013 festival – the Elias’s superbly direct performance of Webern’s Langsamer satz, balancing rhetorical restraint with striking generosity of tone.

The Elias Quartet return in 2014, fulfilling Brown’s vision of a festival that 'brings performers together in new configurations' when they join the Belcea Quartet for performancs of the Brahms Sextet and Stauss’s mighty Metamorphosen. They will also take plart in East Neuk’s first ever Schubertiad, a two-day event which will include the piano Impromptus, Lieder and orchestral music.

But many of this year’s innovations are rather more subtle, as Brown explains. 'We’ve co-commissioned James MacMillan’s Piano Trio, which will have its Scottish premiere here. We’re programming it in a session on its own, so that people can hear it performed twice in quick succession, as well as hearing James himself talk about it. That way the audience have a real chance to get to know it properly, rather than just hearing it once and moving on.’

But with so many other summer festivals clamouring for concert-goers’ attention in Scotland alone, what sets East Neuk apart? ‘Where else in the world,’ asks Brown, ‘can you hear world-class musicians in such intimate venues and then stroll outside and find yourself by the sea with a really great bag of chips?’

It’s a good question, and an even better answer.

Tickets to the 2014 East Neuk Festival are available from eastneukfestival.com