Yes you can.
just have all the necessary files on your usb, and it'll work. It loads things slower than the SD, but still works pretty well.
Oh, and make sure your SD is NOT in the wii when you boot riivo. it usually helps to have the Riivolution channel on the Wii menu.