I have the same CPU as you with a 256 MB VRAM graphics card.
It maybe a combination of things.
4 GBs of RAM is really the true minimum to run almost any version of OS X from OS X 10.6 Snow Leopard to 10.8 Mountain Lion.
Every iteration of OS X has always need more CPU, GPU and RAM.
I bet everything has been slower since you updated to OS X 10.7 Lion. It will only get worse when and if you decide to update to OS X 10.8 Mountain Lion.
I have 16 GBs of RAM in my iMac and Hulu, YouTube and other streaming video sites run fine.
I suggest that you, at least, double your RAM to 8 GBs.
This may help with your video issues.
Also, make sure that you no other CPU intense type of applications running in the background.
Even if these apps are running idle in the background, they can still eat up CPU cycles.
Thanks for your comments. Unfortunately, my particular machine is limited to 4GB officially, 6GB unofficially so I can't do much to improve the RAM situation. I do understand that when memory runs out, virtual memory kicks in and there's always a performance hit for that but usually when these problems happen I have plenty of free memory available. It seems that the problem is that when streaming video, most of the streams load at about the same pace as playback. This inevitably causes issues, especially with Flash. What's puzzling is that I know I have a fast connection (although that doesn't mean that all the traffic coming over that connection is fast) and my machine appears able to handle it. I can watch full HD movies trailers, Netflix, Apple keynotes, etc. without any issues. It seems like it's the popular sites that cause most of the issues. It could be a cable company, routing issue...it certainly wouldn't be the first time but it just shouldn't be so difficult to watch internet video.
I assumed you had the newer 24 inch iMac model the also had a 3.06 Ghz CPU.
That model could take 8 GBs of RAM.
Install the 6 GBs of RAM
Any increase in RAM from 4 GBs will help.
Use Activity Monitor and report the specs and the bottom of Activity Monitor screen.
That wil tell us just how much CPU and RAM you are using doing normal activity.
Your browser uses up a lot of CPU cycles and RAM when running Flash to stream video.
And yes, your Internet connection speeds can vary.
If you're, at least, streaming a min. data rate of 4 Mbps or greater, any internet video should stream fine.
Make sure you are running the latest Flash plugin.
Also, wouldn't hurt to try another web browser.
If using Safari maybe try Firefox or Camino.