Scenario: I have an EeePC (obviously), a wireless router, a Windows XP computer (called Study) and a printer (HP 932c, called Printer) connected to the Windows box via USB.
First, ensure that the network is set up correctly. Follow the directions for setting up a Windows network.
In easy mode, click the Settings tab and click Printers.
In advanced mode, click the Launch button, choose Control Center, then click Printers.
"//", the Windows computer name, "/", and the printer share name. In my example, it's //Study/Printer.
This will only allow you to print when the Windows machine is on. You may also want to plug your printer directly into the EeePC and add it as a local printer, since it's USB.
DeadDuck's personal experience. (Thanks, DeadDuck :)