You're hitting the 32-bit application memory limit. The limit for any 32-bit application is approximately 4 GB if programmed appropriately.
This is the reason for the anticipation for the Connect Edition of OpenRoads...64 bit application which allows much more memory to be used.