#Preying on lonely and vulnerable people is kind of the cult M.O so that tracks

[drunk-on-starlight]

So Shadow Fall directly says that "religious types" (cults) have been springing up after the Empire fell (which...actually explains a lot. There are a lot of cults in star wars canon right now.)

So my best guess for Children of the Watch is that they were a minor, irrelevant cult right up until the Empire came down hard on Mandalore. Presumably that left a lot of people lonely and directionless, which caused their ranks to swell from other not irrelevant clans (hence a Vizla, Eldarr, etc).

This doesn't explain where every other house went, so presumably everyone died. Weirdly grim direction to take things from Rebels, but that was the choice apparently made.