Agoraphobia is defined as an "exaggerated fear of being in situations from which escape might be difficult". Agoraphobia is different from other phobias because agoraphobiacs are afraid of loosing control in public. A phobia is defined as an irrational, persistent fear associated with a specific object or situation. It is so strong that the person suffering from the phobia will go out of their way to avoid it. Fear is defined as an "emotion experienced in the face of real immediate danger".
Agoraphobiacs are controlled by the fear that if they panic in public they either will embarrass themselves, or become paralyzed with fear. As a result, they may slowly start to avoid all public places and become prisoners in their own homes. They might feel that if a panic attack could happen anywhere, so nowhere is 100% safe. At least at home, no one will see it happening but them.
To conclude, the "fear of fear" is indeed a central feature in agoraphobia. I could not imagine the feeling of being afraid of being afraid of all the things that could happen to me in public, whether it might be an accident or fear that others could see my anxiety. I would also surmise that the isolation this fear leads to would almost be as horrible as the phobia itself.