Hey Max,
Correct me if I'm misunderstanding your issue, but I believe Speedgrades layers apply effects linearly based on the information passed to them by the layer directly below. Could you achieve your solution by using two (or more) primary layers, with the bottom layer being used for manipulating colours, and with the top layer to desaturate?
If switching back and forth is important, you could add a single effects layer over the entire projectt on a track above, and then just apply a primary that desaturates to that effects layer. Then you could just switch it on and off by toggling the visibility of the track.