The fate of Hearts' hopes of reaching the group stage of the Europa League is still up in the air.
In the first leg of the play-off, a penalty by Lawrence Shankland put Hearts in the lead, but they were pegged back by a pair of goals by Adrian and Blerim.
Craig Gordon saved a goal in the last minute to keep the deficit at one goal.
They will join the likes of Manchester United and Arsenal in the second tier competition if they overturn the deficit in Edinburgh next Thursday.
Despite taking an early lead against a side from the bottom of the Swiss top flight, they were sloppy in possession and loose at the back in the first half.
The loss of centre-back Craig Halkett to injury after six minutes was felt as a long ball forward from the goalkeeper was headed up in the air by Atkinson, and Guerrero was the first to react as he lashed a volley past the helpless Gordon.
Then, two minutes later, Peter Haring was robbed in the middle by a counter- attack and, after two balls into the box weren't dealt with, Dzemaili smashed a fantastic shot into the top corner.
With better decision-making in the final third, Hearts might have been able to draw level.
Donis Avdijaj, a former Hearts player, was close to scoring twice before that, but he missed.
In the waning moments of the game, substitute Fabian Rohner tried to score on goal, but Gordon denied him with his leg.
Even though they were winless in their first five games in the Swiss league, they were always going to pose a greater threat than their form suggested.
After playing solidly in the first quarter and getting themselves in front, Hearts will be kicking themselves for letting their opponents back into the game.
The loss of Halkett and the rain did not help, but Atkinson was exposed before he was taken off. The visitors only completed half of their passes in the first half.
The loss of Halkett, who broke his foot in the first half, is a concern in the defence and they will need to offer more of a threat from open play.
Two years ago, we played Queen of the South in the Championship. We lost the game but put on a great performance.
The place will be rocking next Thursday and we have given ourselves a chance.
The second leg of the play-off will take place on Thursday, 2 April, after which the winner of the game will play Celtic.