Sadly, I have to say I'm just a little late to the party on this post. Sickness pushed this back for me and LifeHacker beat me to the punch. I've had years of getting my computers working just like I like them. Years of having the various nuances setup and particular applications that work just the way I like them, have made be a bit of a stickler.
That's why I've become a huge fan of Portable Applications. Portable Applications are typically software that doesn't leave any files or settings on the host computer, but more importantly packaged to run completely in it's own self-contained structure. This is most commonly used for a suite of applications run from a USB drive, therefore portable.
But why be stuck to just using them on a portable drive?
IT Lock-down
Like I said before, I normally setup my computer just how I like them with this software doing this, etc. But sometimes you aren't allowed to install new applications on your computer...let's hypothesize at work, maybe? This is where using these portable applications can come in very handy.
For instance, I just recently posted about text editors and my use of Sublime Text lately as my primary editor. Luckily there is a Portable version to satiate that need. I handle many PDFs and use PDF-Xchange Viewer so I can markup the documents and again a Portable version is available. Anything from a full Office suite to a fully loaded programming environment (Python). Not allowed to install Visio? Try Dia.
Just load these programs somewhere in your user directory and you'll be rolling in no time. Some places might allow you admin access when requested, but I get tired of those tools. On top of that some of these might want further access for simple tweaks, so you won't want to keep calling/requesting for admin every time the program needs to tweak itself or update.
Syncability
Is that a word? Sure. Another advantage is that since these programs run self-contained is that you can use them to keep multiple computers sync'd together. Instead of installing these simply on a work computer or a USB drive, install them into a Dropbox folder and all your Dropbox enabled computers will not only run the same programs, but also sync the settings.
How does this help you? Think about a browser whose extensions, bookmarks, history, and cache are all the same on every computer you use. Email client already setup and ready to use. Toolbars always the same in any number of applications is another advantage.
Honestly, I have a feeling that this is how applications are tending to trend toward. Many applications these days, although built with an installer, simply run without registry edits and settings outside of a central location (most of them these days live in the User's directory under AppData). But many have a simple switch to move the settings directory into somewhere manageable. Only programs that need to utilize or replace core Windows functions or utilize specialized hardware/drivers are going to need special privileges.
Have you used a portable application in a specialized manner? Let me know in the comments.