Nussey was the twelfth child of John Nussey (1760-1826), a cloth merchant of Birstall Smithies in Yorkshire, and his wife Ellen, née Wade (c.1771-1857). Nussey first attended a small local school before progressing to the Gomersal Moravian Ladies Academy. Nussey and Brontë first met in January 1831, when they were both pupils at Roe Head School, near Dewsbury in Yorkshire. They corresponded with each other regularly over the next 24 years, each writing hundreds of letters to the other. In 1839, Ellen Nussey's brother, Henry, proposed marriage to Brontë, but she found him dull and refused him.
Through her frequent visits to the Parsonage at Haworth Nussey also became a friend of Anne and Emily Brontë, and was accepted as a suitable friend for his daughters by the Reverend Patrick Brontë. Indeed, when in May 1849, Anne decided to make a visit to Scarborough in the hope that the change of location and fresh sea air might be good for her failing health, and give her a chance to live, she went with Charlotte and Nussey. En route, the three spent a day and a night in York, where, escorting Anne around in a wheelchair, they did some shopping, and at Anne's request, visited the colossal York Minster. However, it was clear that Anne had little strength left.