No, I'm not suggesting turning features on or off as in haskell. This requires no actual programmatic changes to the code. It's just a conceptual labeling of features, indicating their level of stability. It means that when introducing new features, certain parts of the language must be seen as static (or close to static) while others are more malleable or even very malleable.
The ocaml devs can introduce features more liberally, without worrying about future implications as much. If an alpha feature doesn't survive alpha, it'll be cut/modified eventually. They can rely on user feedback to gauge whether a feature is really worth solidifying into the core language. Perhaps there could even be another level: experimental features, which aren't even guaranteed to play nice with alpha or other experimental features, but allow experimenting with ideas in the wild.
It also sets the expectations of the users: if you want to make sure your code will compile in 5 years' time, use only stable features. If you're ok with features that have been around for a while but may change a little, use beta features. If you're not afraid to mess with the latest stuff, you can use alpha level features.