The Music began playing in 1999 and in 2001 a song Take the Long Road and Walk It (Fierce Panda) circulated as a demo, before being released by Fierce Panda. Around this time the NME and Steve Lamacq were describing them as the best unsigned band in Britain, making the 1000-issue single a rarity from its day of release. The band were quickly signed by Hut, who released their first EP You Might as Well Try to Fuck Me.
In 2002, following another EP The People, they released their eponymous debut album entitled The Music which reached #4 in the UK. Their debut single was re-issued to promote the album, but only reached #14. Two further singles from the album, Getaway and The Truth is No Words reached #26 and #18. June 2003 saw them filling in for Zwan on the Pyramid Stage at Glastonbury Festival.
In autumn 2004, they released their second album Welcome to the North (#8) and the accompanying single Freedom Fighters (#15), before touring with the band Incubus. In January 2005, a second single Breakin’ reached #20. August 2005 saw the band play at the dual-venue V Festival. In late 2006 they played a few UK gigs and subsequently posted on their site the video of a live performance of a new song entitled, Fire, which revealed both their latest sound as well as Harvey’s new shaved head look.