The only flags I might consider would be a define to disable building in video support or not if it was done cleanly - but this wouldn't be used for RetroPie, but it could be useful for people building from source who wanted to exclude them (someone requested this on the issue tracker I believe).
I have no issue with current compile time, nor do I think flags/defines for enabling/disabling features would be much use. I wouldn't accept any PR for such things without being convinced why we would need it.