This happens when FaceTime HD Camera is set as a camera by other apps or processes in your system. To fix this, we need to make sure the camera is not being used by other apps before starting VCam.
Restart your system and do not run VCam yet.
Check all the apps in your system that uses the FaceTime HD Camera, and make sure to select XSplit VCam as the camera in their settings instead. Close the apps once done.