You can manually set a resolution by selecting from one of the preset resolutions - 360p, 480p, 540p, 720p, 1080p, and Original Resolution.
You cannot manually set the bitrate, but you can specify a target quality from one of the preset qualities - Standard, High, Very High, and Ultra High.
You may also select any encoder available in the CODEC box in the Custom Record Settings. The quality will only be determined by your quality setting but CPU usage and files size will vary if you use a hardware encoder instead of the x264 software encoder.
To manually set the resolution and bitrate, click on the Settings Tab then select a Custom resolution for Record Settings:
This will open the Record Settings Window. Change the resolution and quality to what you would like to try and click Close:
Scroll down the Settings Tab and, under Status Bar, check both the boxes of Game FPS and CPU Load:
Finally, go to your game and try to record it with your selected resolution and quality. If it seems like it’s consuming too much CPU, go back to the Settings Tab and make the necessary adjustments to your recording settings.