Ok - just wanted to make sure it wasn't a case of 'pilot error'.
One way that (to me at least) would be intuitive without cluttering up the user interface would be to add a second set of 'drag bars' to the right side of set list markers, indented from the regular ones that you use to drag songs and other items. These 'super drag bars' would drag a set and all items within that set. Maybe they could be red to show the potential danger. :-)