Small tribes.
Ozai did not know about the Sun Warriors, Foggy Swamp, etc.
But if he had known about them, do you think he would have considered them worth conquering? Long term he would if he controlled the rest of the world obviously. But what about in the short term?
Edit: Nobody has given their opinion on the sand tribe. I did not mention them as I thought it was covered by ETC along with any other group I did not know about.