Nonminimal matter couplings have recently been considered in the context of massive gravity and multigravity. These couplings are free of the Boulware-Deser ghost in the decoupling limit and can thus be considered within an effective field theory setup. Beyond the decoupling limit the ghost was shown to reemerge in the metric formulation of the theory. Recently it was argued that this pathology is absent when formulated in terms of unconstrained vielbeins. We investigate this possibility and show that the Boulware-Deser ghost is always present beyond the decoupling limit in any dimension larger than 2. We also show that the metric and vielbein formulations have an identical ghost-free decoupling limit. Finally we extend these arguments to more generic multigravity theories and argue that for any dimension larger than 2 a ghost is also present in the vielbein formulation whenever the symmetric vielbein condition is spoiled and the equivalence with the metric formulation is lost.