What kind of USB you looking for, it has bluetooth for keyboards and such and I think probably supports usb otg but I don't ever use anything like that.
The audio issue isn't just the headphone circuit, you can hear the same thing coming out of the speakers on it if you listen closely. I think most people just don't notice it or aren't in a quiet enough environment.
I'm a bit sensitive to noise and such and like my quiet. I get bothered by electrical noises and things like fan bearings and such too, one of my many eccentricities. My fiancee says she notices is it but it wouldn't really bother her. I also have "nicer" set of headphones so that might be part of it too. Something tells me you won't notice it with a set of Beats or something.
I don't have the problem on the iPad mini, N7 or N4 so it's definitely something with this device.
Overall the device is very nice which is why I'm probably going to get it for a bit. I did swap out Touchwiz with Nova Launcher and that helps make the interface a bit more familiar, TouchWiz seemed to be chewing up battery too so we'll see if that goes away. They're supposed to be rolling out Android 5 too but I don't see the update yet. The screen is gorgeous, and side speakers aren't bad for a tablet, it also has some interesting power saving modes (you can make the screen grayscale which I love).