Something missing from this discussion (that weighs heavily on the difference between beach cats, monohulls, and sail-rigged kayaks) is that blow-over tendencies have more to do with the vessel's heeling / righting moment characteristics than sail area. I think taking this relationship into account could simplify the reefing rules and allow them to be common between boat classes. I apologize in advance because this post turned into a bit of an essay. Whether or not the rules can be changed to heeling moment reduction vs sail area reduction doesn't really matter to me, but the prudent sailor should recognize the difference between them.
Yes, heeling moment does depend total sail area (currently governed by the rules) but also sail aspect-ratio (height / length) and vertical offset of the sail CE from the waterline. Given two sails of the same area, the taller skinnier sail will produce more heeling moment. A sail that is displaced vertically (to allow the boom to clear the crew, for instance) will produce more heeling moment than one that starts lower. Unstayed schooner and cat-ketch rigs typically have lower heeling moments than single-masted rigs of the same sail area. Other sources of lift and drag (even bare masts and stays) create heeling moments that can be significant if the wind is high enough. Symmetric spinnakers flown downwind create pitching moments (not heeling moments) so aren't usually of concern. Asymmetric spinnakers (like those flown by beach cats) do create heeling moments. Finally, heeling moments generally scale with wind speed squared.
Righting moment is increased by adding ama buoyancy (for multis and outriggers), beam width, vertical distribution of hull weight (e.g., keel weight), and crew weight. Watertribe entries are typically shallow bottomed and unballasted, with Matt Layden's Enigma as a notable exception. Ama buoyancy and beam width can be assumed fixed (although I think a trimaran design which could shift the amas to vary the leeward / windward distribution of buoyancy would be pretty cool). Crew weight shift (hiking) might not be an option depending on the boat design, weather conditions, etc. Consider that (especially in high winds when damage might occur) the crew needs to be able to safely move around the boat to make repairs, access gear, etc. Hydrodynamic devices like foils on the outriggers could increase righting moment, but for safety purposes we should assume the boat is not moving so the foils aren't contributing. A conservative estimate of righting moment would be taken with the boat fully loaded and the crew sheltered in cockpits (if appropriate). The final point here is that in general, righting moments aren't affected by wind speed.
OK so my point is that we have two terms, one that scales with wind speed squared, and one that doesn't. Reefing measures should be provided to maintain an appropriate balance between them. For boat stability, the available righting moment must always exceed the possible heeling moment (based on the current wind conditions). The details depend on the boat configuration, but I think it would be possible to define a generic righting moment to heeling moment factor that must be able to be maintained at all times. This would allow more flexibility in boat / rig designs to meet safety requirements.
For example let's consider three typical Watertribe race wind speeds: 10kts (not enough wind), 20kts (too much wind), and 30kts (way too much wind - blammo). Taking the square of each we get 100, 400, and 900 or in ratio terms, 1,4, and 9. Let's say we have a beach cat with a high aspect ratio rig which is designed to fly a hull in 10kts of wind. We could say it is designed to operate at the '1' point. At this wind speed, the cat has no more available righting moment (the crew is already out on trapeze). So the righting moment and heeling moment are matched at 10kts. If the wind increased to 20kts, the heeling moment would go up by four. To keep the boat balanced, the heeling moment would need to be reduced to 25%. Depending on how the sail is reefed, this might be by reducing sail area, height, or both. If the wind increases to 30kts, the cat will need 9X reduction in heeling moment over the baseline 10kt configuration - the sail will probably need to come down. This raises questions about how the team will maintain course and safety with no sails up, but that's another discussion. Now let's compare to a more conservative, underpowered boat that was purpose-designed for an expedition event. It is designed to sail at 20kts (the '4' point) with the sail rig reefed to 50% effective heeling moment. At 10kts, the available righting moment will greatly exceed the max heeling moment - with the sail fully unfurled, the boat will still have twice the necessary righting moment (just 2x more sail with 4x less heeling moment). From the 20kt / 50% reef point, to 30kts, the sail will just need to be taken in about 2.25x (900/400), so we are down to about 23% sail unfurled. However, with this configuration the boat should still be able to make progress on course. Of course, at lower wind speeds, the undersized sail may limit performance, so this design could fly an additional sail (mizzen, headsail, spinnaker, etc) equivalent to 100% of the effective heeling moment of the original rig when the winds are below 10kts.
I know these are fictional examples but I think they show the difference in safety margin between a typical beach cat and a design suitable for a Watertribe race. At a minimum, the reefing rules should require a percent reduction in effective heeling moment rather than just sail area so that the change in vertical CE is accomodated. Maximum righting moment should also be considered - the entrant should be encouraged to choose a hull design (esp. ama buoyancy and outrigger crossbeam length) that maximizes stability. The burden of computation should be left to the entrant rather than the WaterTribe. As a starting point for discussion, the rule could be written something like this: "The maximum righting moment of the vessel (not counting hydrodynamic devices such as foils or unsheltered crew weight displacement) must exceed the heeling moment from the sailing rig at a wind speed of 30kts. When reefing or sail changes (including fully dropping a single sail) are required to satisfy this requirement, the entrant must show the rig can be configured to meet this requirement at wind speeds of 10, 20, and 30kts. If discrete reef points are used to meet this requirement, at least two reef points must be installed and they should be placed such that sequential reef points provide a reduction in effective heeling moment of at least 50% the previous setting. The wind speed at which all sails must be completely furled for safe operation of the boat must not be less than 20kts. When unusual or marginal reefing or righting measures are employed (as determined by the WaterTribe), video demonstration of the technique may be required to avoid disqualification."
Sail heeling moment (usually measured in foot-pounds) can be estimated from plans or by using an online calculator like this one: http://www.wb-sails.fi/news/SailPowerCalc/SailPowerCalc.htm. The maximum righting moment of a boat can be more easily determined (here I'm listing the conservative estimate, you might also want to know the effect of crew hiking out, but I don't think that should be included in any safety assessment):
For outriggers and kayaks (assumed boat rotates around main hull, no hiking) - If known, multiply buoyancy of ama times distance from centerline to ama. Or just test it - measure weight on ama required to cause capsize (or uncomfortable heel) and multiply by distance to centerline. If crew weight can be shifted without leaving cockpit, include weight times distance from centerline.
For catamarans (assumed boat rotates around leeward hull): weigh one hull with the other hull on the ground. Multiply by distance to other hull. Add crew weight times distance from hull to centerline or for cats with port / starboard cockpits, add each crew separately.
For monohulls: by analysing hull shape / weight distribution (including ballast) or measured by same capsize test above - just use a board hooked across the gunwales and add weight at the end of the board until capsize imminent and multiply by distance to centerline. Include crew weight times distance from centerline if appropriate.
RedBeard aka Matt Hazard