There is no perfect size, but we have had success with around 900px for the width, and 400 - 500 px for the height. 

The images in the Photography template gallery are about 300 X 200 pixels. inCMS scales the gallery images to the browser 's size, so if you have a big screen, the images appear in that size, while on a smaller screen, they will appear a little smaller. InCMS maintains the best image quality in the gallery.