No, not with a washcloth!
Often this pre-installed software is set to start up unnecessary process when the computer boots up - eating into memory and slowing performance. I've seen cases where a PC was running both the built-in Windows firewall software and a commercial one the PC maker had installed; two pieces of software competing with each other to do the same thing resulted in a very sluggish internet connection.
By all means, play around with whatever graphics and sound programs, office applications, etc. have come with your new PC, and keep them around if they meet your needs. (In a future series of articles I'll be reviewing free alternatives to commercial software that are far superior to anything that comes pre-installed on consumer PCs.) But at the very least I recommend getting rid of any add-on commercial firewall or virus checker programs. (Note, please, that I am not saying you don't need this protection. More on security below.)
Don't Delete - Uninstall!
Once you've decided which software you won't be needing, it's time to uninstall it. Uninstalling is not the same as deleting the icon from your desktop; the icon is just a convenient pointer to where the startup file for the software is located. Just as you can't demolish a building by taking its picture off Google Maps, you can't clean unwanted software off your machine without going after the underlying file structure. Fortunately Windows makes this relatively easy through the Control Panel's "Programs and Features" section (in Windows 7).
Just find the applications you don't need, select them from the list, and click "Uninstall". Occasionally a program being removed will ask if you really want to do it, or present a survey about why you're getting rid of it, but beyond that the removal is generally pretty easy.
Securing Your PC
As mentioned, most PC packages come with commercial firewall and anti-virus products installed. Make no mistake, these are crucial security functions for PCs that will connect to the internet - so why delete them?
Well, in the case of the firewall - a piece of software that protects your PC from intrusion across the internet - there's already one built in that does everything you're likely to need a firewall to do. You can make sure Windows Firewall is turned on by following these instructions provided by Microsoft.
Commercial virus checker programs that come pre-installed on PCs are almost always only free for a short period of time - 90 days to 1 year, usually. After that, you'll need to pay to get updates (unless you're not interested in being protected against the latest viruses...) - and, not surprisingly, the PC manufacturer gets a small kickback when you purchase the extension.
The good news is that there are free alternatives that are every bit as powerful as the paid ones. I've been using AVG Free for years and found it to be excellent protection; it's updated frequently and it even marks sites as potentially dangerous in the results of a Google search. Recent versions have started showing ads promoting the company's paid "pro" products, so lately I've switched to the Microsoft Security Essentials anti-virus package - free download from MS. It's unobtrusive, doesn't hog a lot of system resources, and so far has caught and prevented two attempts at viral infection for me. Either of these options will work at least as well as their pre-installed for-pay counterparts, and remain free of charge.
Enjoy your new system!
These simple steps can make your investment faster and safer - now go have fun with it!
Addendum: "PC Decrapifier"
I've recently discovered a piece of software called "PC Decrapifier", which takes you step by step through pre-installed and possibly unnecessary software on your PC and makes recommendations as to whether you should keep or remove it.