Per the title, a fallout shelter was an interior space in the house or in the basement where sleeping accommodations and food supplies were stored in the event of a nuclear war. The inhabitants would stay there for about two weeks until the radiation was down to a "safe" level.
Archived newspapers indicate that Martha Raye was first offered the role of Clara Appleby on this program. Raye refused to portray a nagging wife, claiming she had never done so in her career. Raye was also a former wife of Skelton's long time orchestra leader, David Rose. However, it is unlikely that was a reason for the refusal, since she appeared numerous other times with Skelton on his series.