I am having a similar issue, only mine is with a photo set of only 10 or so photos. When I use the clone/heal tool, it lags tremendously! Additionally, when I make edits in the exposure, brightness, lights, etc., there is a short lag before it actually updates the photo.
Before, when I was using LR3, no problems. Only issue I had was with the brush size, that everyone was experiencing.
I am currently using:
Macbook Air 11.6"
Core i5 1.6ghz
4GB ram
256GB SSD
When I look at system info through LR4, it is only using about 1.5GB of ram. I don't have any other programs open either. This is quite weird to me. Now I'm a little frustrated because I need to finish this set of photos, but I don't want to jump back and forth between LR3 and LR4. The point of purchasing LR4 was to be able to make a smooth transition. There goes that ship. lol. Hopefully Adobe can find out what the issue is though, I really like using LR.