The MIME type for a Mathematica notebook is application/mathematica, the corresponding application is, of course, Mathematica. These are combined to form a line in the .mailcap file that reads:
application/mathematica; mathematica %s
The %s represents the name of the file where the downloaded information
is stored. After inserting this line in the .mailcap file in your home
directory, restart the browser. Now, when a Mathematica
notebook is downloaded from a properly configured server,
Mathematica will automatically be started with the downloaded
file as its current notebook.