Inspired by post-punk legacy of her hometown, Manchester, England, as well classic college rock of the '80s and early '90s, LoneLady began her forays into lo-fi art punk by making four-track recordings with her guitar and drum machine in 2004. Her first single, Hi Ho Bastard/Fear No More, was released by Filthy Home Records in summer 2005; its raw yet atmospheric sound drew comparisons to Scout Niblett as well as rave reviews. That fall, LoneLady issued the Have No Past EP and began playing solo gigs around Manchester. Her first show outside of the area was a showcase at 2006's South by Southwest festival. In 2007, LoneLady connected with two musicians, drummer Oid and keyboardist China Tom, and the project became a trio. The Army/Intuition single arrived early that year, just before another appearance at SXSW. That fall, LoneLady inaugurated Too Pure's singles club with Early Comes the Haste/Joy; released in a limited edition of 500 copies, it marked the band's first sessions in an official studio.