Hey yitzchok,
I just figured that one out as you posted.
I moved the images and images/icons out of the App_Themes folder and changed the site.skin file to reference all images by doing a /images/imagename.ext instead of images/imagename.ext.
My opinion has always been to leave images + css files out of the App_Themes folder, as it's caused me more headache than bliss over the years.
I've done just that with dashCommerce, moved images and css files out of App_Themes and just left the .skin.
I've also changed things a bit so the admin can use a seperate theme than the front end, which in my case was easier...
/matt