Scenario: I have an EeePC (obviously), a wireless router, a Windows XP computer (called Study) and a printer (HP 932c, called Printer) connected to the Windows box via USB.

First, ensure that the network is set up correctly. Follow the directions for setting up a Windows network.

• Make sure both computers have the same workgroup name
• Turn on the Windows computer and printer
• Make sure the EeePC is connected to the wireless network

In easy mode, click the Settings tab and click Printers.

In advanced mode, click the Launch button, choose Control Center, then click Printers.

• Click the button Add…
• Choose Network Printer and click Next>
• Network Type is Windows, choose a printer name (no spaces)
• In Path, enter (without quotes or commas)

"//", the Windows computer name, "/", and the printer share name. 
In my example, it's //Study/Printer. 

• You can find these settings by going to your Windows computer and right-clicking on My Computer and looking in the tab Computer Name (for the computer name), then clicking Start, Printers and Faxes and right-clicking on the printer to share and looking for the section Shared As (for the printer name).
• Choose the printer brand (e.g. HP), and model (e.g. DeskJet 932c), then accept the suggested driver
• Press Next>, then print a test page

This will only allow you to print when the Windows machine is on. You may also want to plug your printer directly into the EeePC and add it as a local printer, since it's USB.

DeadDuck's personal experience. (Thanks, DeadDuck :)