UPDATE: AS OF NOVEMBER 2010, THE NEW AZURE SDK 1.3 IS AVAILABLE THAT SUPPORTS REMOTE DESKTOP - A MUCH MORE SECURE AND PREFERABLE SOLUTION!
Debugging Windows Azure applications can be time consuming, particularly if you make a mistake and have to redeploy. If you've done any serious work you'll know that it's easy to write an app that runs in the developer fabric but won't work in production.
Sometimes you just want to log on to the box and have a poke around. We have a rudimentary way of doing that using Telnet...

First install the Windows Telnet client - Control Panel > Programs and Features > Turn Windows features on or off > Telnet Client.
Now you need to install a telnet server in Azure. We've written a rudimentary telnet daemon that's available on GitHub . You can download and compile it with Visual Studio and test it out on your development box.
To get it running on Azure, grab a copy of AzureRunMe
AzureRunMe is a kind of bootstrap program that makes it easy to run EXEs, BATch files and many Windows apps that are copy deployable in Azure.
Upload AzureRunMe.cspkg upto your blob storage account (I recommend CloudStorageStudio for this). I use a container name called "packages".
AzureRunMe sets up some nice environment variables for you, including %ipaddress% and %telnet% which are the IP Address and telnet port respectively. Using that information, create a runme.bat file as follows:
Telnetd.exe %ipaddress% %telnet%
Zip up the two files - runme.bat and telnetd.exe together as telnetd.zip - upload that to blob store too.
Now you need an AzureRunMe configuration file, something like this:
You need to fill in your various Azure details of course, and then save as telnetd.cscfg, upload this file to Blob storage too.
Now log onto the Windows Azure portal and provision a new service. Browse for the CSPKG and CSCFG files from blob store. Run the new role.

Optionally you can use the TraceConsole from AzureRunMe to watch the boot-up process.

When the instance is up (give it 10 minutes or so), you should be ready to telnet in
> telnet claptrap.cloudapp.net
And away you go. Try typing SET, DIR etc.
Note that the terminal interaction is rudimentary. In particular the delete key doesn't work. It's not really secure enough for production servers like this, but if you need a more robust solution, feel free to hire us.
*The opinions expressed on this site are my own and do not necessarily represent those of Two10degrees or Active Web Solutions Ltd.