Yes, you can use multiple videos in a source via the Video Playlist plugin.
To do this, simply click Add > Other > Video Playlist…
Then click Add and choose the videos you want to use:
You can also choose the action you want to happen when the videos have finished playing:
If you want it to start immediately when a scene loads:
And if you want it to remember the playback position or show the playback position in the videos:
When you are done, click Ok to save it.
You can now use the plugin to display multiple videos in one source.