Poom started his football career in the clubs Tallinna Lõvid ("Lions of Tallinn") and Flora Tallinn in his home country, before a brief stint at Finnish club KuPS and Swiss club Wil. In 1994, he moved to English club Portsmouth, but only played seven games and was loaned back to his former club Flora.
In 1997, Poom moved to Derby County, making his Premier League debut against Manchester United at Old Trafford. Poom became a fan favourite at Derby, with supporters often confusing unfamiliar commentators with a low chant of 'Pooooom' before games, which sounded like booing. Poom won the Jack Stamps Trophy (player of the year) at the end of the 99/00 season.
After Derby County's relegation to the First Division in the summer of 2002, Poom moved to Sunderland in November of that year. On 20 September 2003, he headed a 90th minute equaliser for Sunderland against his former club at Pride Park. This was described as "the best goal ever scored by a goalkeeper in the 90th minute on his first match against his former club" by the commentator. The goal was cheered by both sets of fans.
Poom's career at Sunderland was beset by several injuries, and he spent much of the 2003–04 season on the sidelines.
Poom was signed by Arsenal in August 2005, as temporary cover for Jens Lehmann and Manuel Almunia. The deal was made permanent in January 2006. He was Arsenal's third-choice keeper, and did not play a single match in the 2005–06 season. However, he became the first Estonian to receive a Champions League runners-up medal after Arsenal lost to Barcelona. Poom did not play nor did he make the substitutes' bench for the final, but was a member of Arsenal's 25-man European squad, each of whom won a silver medal.
He made his debut for Arsenal in November 2006 in a League Cup match against Everton, replacing the injured Manuel Almunia at half-time; Arsenal won 1–0 with an Emmanuel Adebayor goal. His first and only appearance for Arsenal in the Premier League, came on the last day of the 2006–07 season, against his former club, Portsmouth, in a goalless draw.
On 26 May 2007, Poom signed for Watford, who had recently been relegated from the Premier League, for an undisclosed fee. He had become surplus to requirements with Arsenal, after they signed Polish international goalkeeper Łukasz Fabiański. He started the new season as Watford's first choice, but went on to lose his place to Richard Lee.
Poom returned to the side for the start of the 2008–09 season. However, he sustained a dislocated shoulder in a game against Reading in September 2008 that ruled him out for the majority of the season. Although he returned to training before the end of the campaign, he was not considered for the first-team and he was released from his contract, which due to end in June, on 30 April 2009.