Choose your color comparator:

Compares the Red, Green, and Blue values of a color in order. Equivalent to sorting by hex string.
Compares the Hue, Saturation, and Lightness of a color in order. Produces a "rainbow", but darker colors are scattered throughout.
Compares the Saturation, Lightness, and Hue, in that order. Groups pastels, brights, and mid-tones together for a multi-rainbow effect.
Compares the Saturation, Hue, and Lightness, in that order. Groups pastels, but produces dark bands.
Computes the distance from RGB cube origin (#000), and returns the simple comparison. Can return wildly different results depending on your data set.
Much like any other hue sort, this produces a single rainbow. It uses the linear step function to compare within its hue bands.
Groups colors by hue, then sorts each group by brightness. This avoids the issue of colors looking as if they're in the wrong place due to saturation or lightness issues, but the hue breakpoints have banding built-in. Additionally, whether or not the breakpoints actually break up different "colors" can be dependent on your specific eyesight and color palette.