In this context, it is the condition that is counter to the factual condition, the "counterfactual." Exposed people have been exposed. That is the fact. The experience of exposed people were they not exposed is the counterfactual. Although we can't observe the counterfactual scenario, what we always want to know is the difference between the fact and the counterfactual. That is the difference in the disease experience of the exposed given that they are exposed (the fact) and the disease experience of the exposed had they not been exposed (the counterfactual).
(Aschengrau & Seage, pp. 212,288)