Applied Mathematics Computer Users Guide


This section contains general information on commonly asked questions; how do I...?, where do I....?, etc. There is also an extensive set of CICS documentation, including Windows help and basic command-line Unix help - some of these documents are PDFs, requiring the Acrobat Reader.

Rules


Software Available


Application Software on Unix
Compilers on Unix
Windows software

 

Using the Scanner/Backup PC in H10a

 

How do I log on?

How do I use the scanner?

How do I back up data?

Can I add some software to this machine?

Handy Shortcuts

How do I find files by name?
How do I find files by size?
How do I do an operation on files I've found with find?
How do I know if a file is being used by another process?
How do I keep my computer safe from viruses, etc?
What's the best way of destroying my computer?

Email

The recommended email approach for roaming users
How do I read/send email on the AM Unix system?
What's the difference between ap1ips@ and I.Staniforth@?
How do I get my email sent to the AM Unix machines?
I've received a uuencoded message - how can I decode it?
Why should I save my mail in my home directory?
I want to send/receive some large (> 1Mb) files.
How do I create folders/mailboxes in CDE Mailtool?
How do I reduce the amount of junk/spam email I get?

Moving data around

How do I copy files from one machine to another?
Do I really have to keep typing in my password every time I want to connect?
How do I transfer data using FTP?
How do I make my data accessible to an external person with FTP?
How do I let an external person transfer data to/from Sheffield?
What SSH (secure shell) facilities are there here?
How can I move files between machines without ftp?
The British Library document supply service

Shells

What shells are available?
How do I know what shell I'm in?
How do I change shells temporarily on a particular machine?
How do I change shells permanently on a particular machine?
How do I set up shortcut commands, or alias commands?
How do I execute a command automatically every time I log in/start a terminal session?
How do I set up shortcut commands, or alias commands?
What's this PATH thing? a.k.a. where do I find all these applications?

Software

This system's rubbish - you haven't got...
I can't find any of the commands you're talking about.
What libraries are available?
What webbrowers are available and supported on the AM Unix system?
Where can I get a VRML plugin?
What editors are available and supported on the AM Unix system?
What compilers are available and supported on the AM Unix system?
Are there any debuggers?
What is make and how do I use it?
What Windows software is available?
How do I view a file?
Is there a basic drawing package?
Is there a basic image viewer?
How do I install and use TeX/LaTeX/MiKTeX etc?
How do I get bold theta-hat in LaTeX?
Where can I get the SoMaS TeX style files?http://www.shef.ac.uk/appliedmaths/manual/general.html#q71
How do I check for NaN in FORTRAN?
How do I convert between data types in FORTRAN?
How do I include a NAG library routine in a FORTRAN programs?
How do I set up OpenOffice?
What's this Starlink software?
Where can I get WinEdt?
How can I see an array of mugshots of my students?
Can you upgrade Netscape on Solaris?
I need quad precision!

File Formats

What are these image formats anyway?
How do I convert a .doc file or a .rtf file into a PDF?
How do I convert from MS-Word (or anything that loads into MS-Word) to Postscript?
How do I convert a file from Postscript to PDF or vice versa?
How do I convert a file from Postscript or PDF to ASCII?
How do I convert a file from Postscript to Encapsulated Postscript?
How do I convert a file from Encapsulated Postscript to anything?
How do I convert a file from JPEG to Encapsulated Postscript?
How do I rename all .x files to .y?
How do I strip the leading 1,2,3,4,etc characters from filenames?
How do I compress and uncompress files?
How do I prevent Windows from hiding file extensions and automatically renaming over them?

Backing up your data

What is backed up for me?
What do I need to back up myself?
How do I burn CDs on Windows XP?
How do I use the Maxtor disk?
How do I mount a memory stick (flash stick, USB stick) on a Linux machine?

The SPARG big disk

What's the SPARG big disk?

Optimising your programs

Optimisation
Parallelisation
Performance monitoring


Rules

1. You must abide by the University regulations about use of computing facilities
2. You must not attempt to access the root or administrator account on any machine to which you have not explicitly been given the password to.
3. You must not connect or disconnect any peripherals whilst the computer is powered up.  The sole exception to this is USB devices.   You must satisfy yourself that any USB file system is unmounted before disconnecting it.
4. You must not power down Unix computers  under any circumstances.
5. You must only power down Linux computers if you are satisfied that a) you are the owner of the machine, b) nobody else is running jobs on the machine, c) nobody else is accessing the disk.

Root Passwords

Sometimes a user may feel confident enough to want the root password of their Linux or Sun workstation, in order that they may make system configuration changes and install software.

Dave Robson and Richard Balthazor have no objection to this, provided that  the following conditions are met.   These are to ensure the security and integrity of the Applied Maths network as a whole.

1. The machine will have no NFS write access to any other machine in the department. The only exceptions will be for machines also run by the same person (eg JJG's cluster)

2. The root password, and any subseqent changes to the root password, are recorded and stored in the departmental safe (which is only accessible by HoD, dept. secretaries, RLB or DJR)

3. DJR and RLB have full access to the machine, including the root account and password, at any time, if necessary to ensure the security and integrity of the departmental network.

4. The user assumes FULL responsibility for the system security, integrity, and use, including keeping the system secure from unauthorised exploitations by third parties, and integration with the network.

5. In the event of any problem originating in the system as a result of software misconfiguration, Dave and Richard can offer general advice, but this will be at lowest priority; all other departmental computing matters will take priority over this.  In extremis, Dave and Richard will, at the user's request, reinstall the current departmental Linux operating system from scratch (currently Scientific Linux 3.0.4).





Software Available

The following major software is available on machines which mount acms1:/5LOCAL onto /usr/local (ie, all Sun Solaris machines) and machines which mount applied2:/software onto /software (most Linux machines). If you want a piece of software installed globally, contact me. Please note that whilst I try to keep this software up to date, it may not be the latest cutting edge release.  If not, and if you want the latest release, let me know.   However, I will not generally install beta versions/bleeding edge/test releases.   Note that much software is available on iceberg (although paths and versions may differ).

Application software available to the Applied Maths Dept Unix Networks.
Software Version Description Solaris path Linux path Available on iceberg?
Comments
IDL 6.0.1 Interactive Data Language /usr/local/rsi/idl_6.0/bin/idl
/software/bin
No
Licensed only for SPARG users + 2xLRW users - see Robertus or Lucy Wyatt if you aren't
ENVI
4.0
Geographic Image Viewer /usr/local/bin/envi

/software/bin
No
No longer licensed - you can use it, but there will be no more updates and no technical help from RSI.
GAMMA
n/a
Mapping software from Urs Wegmuller
/software/packages/GAMMA_SOFTWARE-20051021
/software/packages/GAMMA_SOFTWARE-20050921
No
The software will only work on acms24 and acms25.

The following environmental variables must be set in your .cshrc5 (Solaris) or .cshrc (Linux):
setenv GAMMA_HOME "/software/packages/GAMMA_SOFTWARE-xxxxxxxx"
 
setenv ISP_HOME "$GAMMA_HOME/ISP_v9.8"
setenv DIFF_HOME "$GAMMA_HOME/DIFF_v6.6"
setenv DISP_HOME "$GAMMA_HOME/DISP_v1.4"
 
setenv PATH ${PATH}:$ISP_HOME
setenv PATH ${PATH}:$DIFF_HOME
setenv PATH ${PATH}:$DISP_HOME
 
setenv PATH ${PATH}:$ISP_HOME/bin
setenv PATH ${PATH}:$DIFF_HOME/bin
setenv PATH ${PATH}:$DISP_HOME/bin
 
setenv PATH ${PATH}:$ISP_HOME/scripts
setenv PATH ${PATH}:$DIFF_HOME/scripts
setenv PATH ${PATH}:$DISP_HOME/scripts

LD_LIBRARY_PATH must include /usr/lib and this must come before any library directories in /software/packages (eg /software/packages/usr/local/lib)

Netscape 4.75 Browser /usr/local/bin/netscape n/a Yes

Netscape/Mozilla
1.2.1
Browser n/a n/a

No

Mozilla
1.x
Browser On Steroids
/software/packages{,7}/usr/local/bin

/software/bin
No
1.1 available for Solaris 2.6 users.
1.4 available for Solaris 2.7+ users.
Maple
9.5
Maths Stuff
/usr/local/bin (ONLY ON ICEBERG/iceberg)

/software/bin
Yes
Watch that PATH.
Matlab
n/a
n/a
ONLY ON ICEBERG/iceberg
n/a

Costs UKP 80 + UKP 20/year per installation on Windows machines only.   Bring me a purchase order if you have money and I'll install it.
Open Office 1.0.2 and 1.0.3 Office tools /usr/local/OpenOffice.org1.0.3 /software/bin (but mostly installed locally)
No

xfig 3.2 Drawing program n/a
/software/bin
No
Version 3.2.5 (alpha release) installed in /software/bin.   If you receive a warning message concerning app-defaults, it is safe to ignore it.   However, if you want a more stable version (3.2.4), ask for it to be installed locally (if it's not already there).
Acrobat Reader 5.0.10 PDF reader /software/packages/usr/local/bin/acroread

/software/bin
Yes
See Troubleshooting section if this fails to start.
Dejavu
3.5
Dejavu document reader
/usr/local/bin

/software/bin
No

Starlink Autumn 2002 Astronomy Data Analysis n/a /star (pointing to /software/packages/starlink)
No
Only for STP/PPARC research
SSW/Solarsoft Feb 11th 2004
Solar Physics data analysis and programming environment n/a
/software/packages
No
Put the following three lines into your .cshrc or .bashrc:

setenv SSW /software/packages/ssw

setenv SSW_INSTR "trace eit mdi cds sxt sumer chianti"

source $SSW/gen/setup/setup.ssw

ERDAS Imagine 8.7
Geographic mapping
/software/packages/imagine/870/bin
n/a
No
Requires Solaris 5.8.  See Pierre Drezet for use - restricted licence.
teTeX
1.0.7-66
Document preparation system
n/a
Local installs.
No
Mike Piff's/Jim Green's style files also available.
gnuplot
4.0.0
Plotting program
n/a

/software/bin
No

Fluent

Fluid Analysis tools
n/a
n/a
Yes



Compilers available to the Applied Maths Dept Unix Network.
Software Version
Solaris path Linux path Available on iceberg?
Comments
gcc
3.4.0
/software/lang/usr/local/bin
/software/bin
Yes
NB There may be an older install of gcc in /usr/bin.   Set your paths accordingly.
Sun Fortran compiler
Workshop 5.0, FORTRAN 90 2.0
/home13/packages/SUNWspro/bin/f90

n/a
Yes

Sun C compiler
5.0
/home13/packages/SUNWspro/bin/cc

n/a
Yes

Sun C++ compiler
5.0
/home13/packages/SUNWspro/bin/CC

n/a
Yes

F
20031017
n/a
/software/bin
No
F is a subset of FORTRAN 90.  Caveat user.
Intel Fortran
8.0
n/a
/software/intel_fc_80/bin
No
Requires /software/intel_fc_80/lib in LD_LIBRARY_PATH

Only licensed for NAR.

Windows software
The following software available for unmanaged Windows XP machines


General Topics


The Scanner/Backup PC in H10a


How do I log on?

 

Turn the machine on.  Log on with your Novell username and password.   When you are prompted for a local username and password, select username 'User' (without the quotes) and a null (or empty) password.  DO NOT CHANGE THIS USERNAME OR PASSWORD.  Be aware that this is a "public" (within the department) machine, and therefore should not be used for storing anything (but particularly confidential stuff, exam papers, etc, etc).

 

How do I use the scanner?

 
Double click on the Irfanview icon on the desktop.   Insert your document face-down onto the glass platen and close the lid.   Select File -> Select Twain Source and make sure that "WIA Hewlett Packard ScanJet 5300C 1.0" is selected.  This should be, unless someone's been messing with it.   Select File -> Acquire and let the scanner warm up and scan a Preview scan. 


 You will see it on screen (of very poor quality - the idea is that you get an idea of where you're scanning).  Make sure you are happy with the amount to be scanned - at this stage you can move the elastic perimeter to exclude unwanted regions of the scan area.  You may also change the resolution (higher resolution = crisper picture = larger file) and whether it is grayscale or color.  When you are happy all is ready, click Scan.   Click on the Save button, and save it to a (local) file.  Save this elsewhere and delete the local file.


When you save, decide what format to save your file in.   BMP is an uncompressed bitmap.  JP(E)G compression is good for photographs.   PNG is good for small compact pictures.   File compression is beyond the scope of this text, but be aware that most compression is lossy, all compressions are optimised for something different, and you will need an image reader that is capable of reading your compressed format.  Please do not leave your pictures on this machine - I will periodically clean it up.  This is not a free file repository! :-)

 

How do I back up data?

 

The following procedure is the very basic way of writing once to a single-write CD-R. Insert a blank CD-R into the  CD drive. A dialogue will appear: select "Open a writeable CD folder using Windows Explorer". Click OK. The folder will open in a new window. Drag and drop the files you want to write into this folder (this may take some time). Click "Write these files to CD".

 

Can I add some software to this machine?

 

No.  Ask Dave Robson or Richard Balthazor to add it.   Any software found on here will be removed without notice.

 


Handy Shortcuts

How do I find files by name?

To find all files whose names contain the phrase 'XYZ' in your current, and all daughter, directories:

% find . -name '*XYZ*' -print

How do I find files by size?

To find all files bigger than sizeinbytes in your current, and all daughter, directories:

% find . -size +sizeinbytesc -print
For example,
% find . -size +1000000c -print

How do I do an operation on all files found with find?

Use the -exec argument with find (man find).For example, to compress all files bigger than 10Mb;

% find . -size +10000000c -exec compress {} \;

How do I know if a file is being used by another process?

For example, has a file finished downloading with ftp? You can use the fuser or lsof commands.

How do I keep my computer safe from viruses, etc?

Keep on top of virus protection and system updates.

For Virii protection in Windows machines, seehttp://www.shef.ac.uk/cics/services/internet/security/virus

To keep your Windows box updated with recent patches; Control Panel -> Windows Update (This can and should be automated so you don't have to rememberregularly; mine is set to check for updates every hour and prompt me toinstall them)

For useful freebies that clean up spyware (programs that download secretlyinto your PC and report on your actions)http://www.lavasoft.de/software/adaware/andhttp://www.safer-networking.org/

What's the best way of destroying my computer?

Running it with the case open. It picks up gunk on the fans, which slow and eventually stop, which means that the processor, power supply, etc, overheat and die by degrees. Please do not run the computer with the case open. :-)


Email


The recommended email approach for roaming users

This is the new recommended email approach for users who aren't in front of one particular computer all day long, particularly those who have a need to access their email on a regular basis from outside the University.

First, contact CICS (helpdesk@shef.ac.uk) and tell them that you want your incoming email (a.user@shef.ac.uk) sent to Webmail, and have them set you up a Webmail account.  This may take a day or so.

Then go to the CICS webpages and click on the Computer User Account Management or Synchronise Password links (they go to the same place).   Log in with your Novell username/password and synchronise your passwords.   This ensures that you will use that (Novell) password to access your mailstore on Impala (check that the account status is active with the tools provided).

Then you can access your email/Webmail from all over the world by logging into www.shef.ac.uk/cics and following the links.  The advantage of this is that you only need a browser.

Unix-experienced users only: If you want to read email with pine, instead of using a web interface, start up Pine. Select Setup -> Config.  Change the smtp-server to mailhost.shef.ac.uk , change the inbox-path to {imap.shef.ac.uk/user=username}INBOX (note the squiggly brackets - and replace username by ap1rlb or whatever), and change the user-domain to be shef.ac.uk Exit, committing your changes, and now select Setup -> Collection Lists, select Mail, set the server to be imap.shef.ac.uk and the path to be empty (NOT the default).  Save this - you will be prompted for your password (Novell password).  Save/commit all changes and exit.  You should now be able to start up Pine and read your email, responding with your username and password when interrogated.


How do I read/send email on the AM Unix system?

I thoroughly recommend the PINE text-based email front end; type 'pine' and follow the very intuitive instructions. Some people use the CDE mailtool. I do not recommend you use the Unix built-in mailer 'mail'.


What's the difference between ap1ips@ and I.Staniforth@?

When you are given a username on the Unix network it will look something like ap1xxx or st1xxx. Mine is ap1ips. You can send e-mail to other users on the AM network by simply using their name (if you know it), and this is the most efficient way to do it. You may also know that someone has a username on the PC network, but not on the Unix system: in this case you could, for example, send a message to ap1ips@Butterley and this would work. However, this is not the best way. Local usernames and system names (such as Butterley) don't make much sense to people outside the University (and not much sense inside, sometimes). To get round this, ACS have introduced an alias system: each user in the University has a mail alias more in keeping with their real name - mine is I.Staniforth@sheffield.ac.uk. This alias points to the username/machine combination which the user has nominated as their preferred mail location. So even if their username or machine changes, you can still contact them with the long form alias. Note that the .ac.uk can be missed off for Sheffield names, and Sheffield can be shortened to Shef. None of this stuff is case sensitive. A final point - if you don't use the alias form, you may be sending someone mail to a machine that they very rarely use - not a good idea.


How do I get my email sent to the AM Unix machines?

If you want any email sent to your email alias (e.g. joe.bloggs@shef.ac.uk) to be routed to your Unix account (e.g. ap1jb@acms1.shef.ac.uk), telephone/email/call at the CICS Helpdesk (tel 21111) and ask them to set this up (this usually goes into effect the following morning when the campus networks update). The default is for incoming email to your alias to be routed to your Novell PC account (e.g. Wintersett/ap1jb). Acms1 is the AM mail server.


I've received a uuencoded message - how can I decode it?

uuencoding is a method of sending non-text files by mail. A uuencoded mail message consists of the usual mail header, followed by a 'BEGIN' line containing the name of the file and its required access permissions. It ends with an 'END' line. To uudecode such a message first save the mail message in a file. Then, in a command window, type 'uudecode filename'. This will create a decoded file in the current directory with the name, which appeared on the 'BEGIN' line of the message.If you have to do this a lot and use CDE mailtool, here's a nifty short cut: under the 'Edit' menu, bring up the 'Properties...' window; under the 'Category' menu, choose 'Alias'; then type in 'uudecode' on the 'Alias:' line, and '|/usr/bin/uudecode' on the 'Adresses:' line; finally click on 'Apply' and dismiss the window. You've now created an alias called uudecode. To decode a message, all you need to do in future is forward it to 'uudecode' and this will produce the decoded file automatically in your home directory.


Why should I save my mail in my home directory?

When mail arrives for you on the AM Unix system, it goes into your system mailbox (in a file in /var/spool/mail). This is a system area shared between all users, and consequently will get full if everyone leaves their old mail in there. When this happens the mail system will stop processing incoming mail. It follows that it is antisocial to leave your mail festering in the system mailbox, and if you want to keep it, you should move it to your home directory either using pine's 'Save' ('S') or the move button in 'mailtool'. To encourage people to do this, the system mailbox files are NOT BACKED UP, so if you blow it, or the disk crashes, you're in big trouble.


I want to send/receive some large (> 1Mb) files.

Use ftp or sftp, not email.



How do I create folders/mailboxes in CDE Mailtool?

Under the mailbox menu is 'New Mailbox'. In this, enter the new folder/mailbox name in the bottom panel labelled 'Enter file name', not the upper panel labelled 'Enter path or folder name'. Click on 'New Mailbox' (thanks to Chris Wilford for this)


How do I reduce the amount of junk/spam email I get?

Prevention

Spammers, those who send 'spam' or junk email (the name coming from the Monty Python 'Spam' sketch), get your email address from trawling through archived mail lists, from newsgroup postings, from web pages, and from seeing you reply to a 'remove me' link on junk email. Therefore:

Cure



Moving data around


How do I copy files from one machine to another?

1. Use scp if both systems permit.

% scp user1@machine1:/path/to/sourcefile user2@machine2:/path/to/destfile

eg

% scp richard@acml1:/home/richard/file.ps ap1rb@acms21:/home11/ap1rb/docs/file2.ps

2. Use ftp or rsh. If you want to copy a whole directory structure from THERE to HERE, try

cd heredirrsh THERE '( cd theredir; tar cf - . )' | tar xf -
Alternatively do it in steps, especially if the two machines can't see each other.
cd parentdirtar -cvf tarfile.tar directory
Then ftp the tarfile.tar to the destination machine and park yourself in the new parent directory:
tar -xvf tarfile.tar


Do I really have to keep typing in my password every time I want to connect?

No.   However, if you follow the below instructions and your account on machine1 is compromised, it will also (naturally) be compromised on machine 2.  Use only with that in mind.


To set up private/public key authentication to allow you to remotely connect to remotehost from localhost (and to scp and sftp) without password authentication, follow these instructions:

1. Generate a private/public key pair for both RSA and RSA1.   You should choose the default location for the files (~/.ssh/identity and ~/.ssh/id_rsa).   You will need to choose a PASSPHRASE.   Your passphrase should be STRONG.  This is NOT your login password; I recommend that it should be at least ten characters long, and preferably more than that; the longer and stronger (ie, non-dictionary, more random) it is, the stronger the encryption.  Using your own password, or using a short phrase, is leaving you open to all sorts of unpleasantness. 


localhost% ssh-keygen -t rsa1
Generating public/private rsa1 key pair.
Enter file in which to save the key (/home1/ap/ap1rlb/.ssh/identity):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home1/ap/ap1rlb/.ssh/identity.
Your public key has been saved in /home1/ap/ap1rlb/.ssh/identity.pub.
The key fingerprint is:aa:bf:ce:cb:19:5b:82:b5:00:88:ef:12:cd:84:e3:1f ap1rlb@iceberg

NB I have doctored the above key fingerprint for this example.    Although it is "public", you should NOT let anyone see it...and indeed ssh won't work if the permissions on the ~/.ssh directory or its files are too lenient.

Do the same generation for the rsa key (ssh-keygen -t rsa)


2. Copy your PUBLIC keys to remotehost and thence, on remotehost, copy them into the file ~/.ssh/authorized_keys (note American spelling), one after the other (each is exactly one line long, although when you view it it may have wrapped.  To check it's really one line long, open it in the vi editor (you know how to use vi, right?  right?).   The details of logging in to remotehost have been snipped from the below examples.


localhost% scp ~/.ssh/*.pub user@remotehost:


and

remotehost% cat id_rsa.pub >> ~/.ssh/authorized_keys

remotehost% cat identity.pub >> ~/.ssh/authorized_keys
At this point, you can check you've done everything OK by ssh-ing to remotehost.  You should be prompted for your PASSPHRASE (not your password) and, when this is entered, you should login successfully.   If you are still prompted for your normal password, make sure that the ~/.ssh directory and the files in it are permissions 700.     Now log out again.

3. Run ssh-agent on localhost.  Note the backward quotes.  This starts the agent program that will store your keys.

localhost% eval `ssh-agent`
Agent pid 25525

4. Run ssh-add to add the keys to the agent.

localhost% ssh-add
Enter passphrase for /home1/ap/ap1rlb/.ssh/id_rsa:
Identity added: /home1/ap/ap1rlb/.ssh/id_rsa (/home1/ap/ap1rlb/.ssh/id_rsa)
Identity added: /home1/ap/ap1rlb/.ssh/identity (ap1rlb@iceberg)

5. Now you should be able to connect to remotehost using ssh without being prompted for either password or passphrase:

localhost% ssh user@remotehost
remotehost%

Now, this will allow you to continue doing this for this session only - ssh, scp, and sftp will work from localhost to remotehost without password prompting.   When you log out from localhost, the ssh-agent will die, and you will have to go to item 3 again next time you log into localhost.    If you don't want to keep doing this, then you need to do a couple more steps.

6. Automate ssh-agent's startup when you start a terminal session.

Add the following lines to your .login:

if ( ! $?SSH_AUTH_SOCK ) then
    eval `ssh-agent`
 endif


...and the following lines to your .logout:

if ( $?SSH_AGENT_PID && "$SHLVL" == "1" ) then
   eval `ssh-agent -k` || kill -HUP $SSH_AGENT_PID
 endif


It is left as an exercise to the reader to figure out exactly what happens here.

7.  Yes, but what about running automatic jobs where I can't type in the passphrase?

The insecure answer is here: http://www.linuxjournal.com/article/8257
I haven't tried this yet.   
When I find a secure answer that works, I'll add it here.




How do I transfer data using FTP?

ftp is a file transfer protocol program, enabling you to transfer files between networked computers. To learn more about ftp commands, type 'man ftp' at the prompt.. The following examples use Unix command syntax. To run ftp or telnet from a Windows95/98/2000 box, click on Start, click on Run, and type 'ftp' or 'telnet' into the box.

The basic ftp commands are 'put' and 'get', to transfer files 'away' or 'towards' you. Multiple files can be transferred with 'mput' and 'mget', and it will prompt you before transferring each file. The prompt can be turned off by typing the 'prompt' command.

You must tell the ftp program whether you are transferring a BINary or ASCII file. An ASCII file is plain text such as a LaTeX .tex file; a BINary file is encoded (such as a postscript file or an MS-Word document). You specify the file type by typing 'bin' or 'ascii' before the transfer.



How do I make my data accessible to an external person with FTP?

telnet acms1.shef.ac.ukUsername: your-usernamePassword: your-passwordcd /ftp/pubmkdir yournamecd yourname

Copy your file, eg yourfile into this directory. The external person can then obtain the data via anonymous ftp as follows:

ftp acms1.shef.ac.uk
Username: ftp
Password: their-email-address
cd /pub/yourname (e.g. cd /pub/richard)
bin [only if your file is binary]
ascii [only if your file is ascii]
get yourfile
bye

NB: This data will be publicly available to anyone who chances to look at this directory. Thus, sensitive material should not be left here. In the interests of disk space, please remove any files once your colleagues have downloaded them.


How do I let an external person (possibly yourself when away from Sheffield) to transfer data onto acms1?


They can upload data to acms1 anonymously, or you can use your own username. NEVER DISCLOSE YOUR USERNAME AND PASSWORD TO ANYONE, EVER.To upload data anonymously:

ftp acms1.shef.ac.uk
Username: ftp
Password: their-email-address
cd /pub/incoming
bin or ascii
put theirfile
bye

...and they may upload their data into this directory only using the 'put' and 'mput' commands (type 'help put' within ftp). When you are logged onto acms1, you may then move to that directory and move the data elsewhere:

telnet acms1.shef.ac.uk
Username: your-username
Password your-password
cd /ftp/pub/incoming
cp theirfile your-destination-directory

To upload data using your own username:

ftp acms1.shef.ac.uk
Username: your-username
Password your-password
cd anywhere you like

...and you may upload your data.

CTCD users

This section is specifically for CTCD users. First you'll need an ftp client.If you use Windows, I recommend WS_FTP LE downloadable from e.g. Tucows. If you use Unix, I recommend the default 'ftp' at the command line.

Then you need to connect with your client as below:

WS-FTP

Host name: ctcd-data.shef.ac.uk
Host type: automatic detect
User ID: Your CTCD username: ask Richard for an account
Password: The password, appropriately
(leave everything else as default)
Unix command line ftp
ftp> open ctcd-data.shef.ac.uk
Username: as above
Password: as above

If you are unfamiliar with the ftp clients, use the Help provided (WS-FTP) or type 'man ftp' at the Unix command line.

A handy tip with MS-Windows ftp

Often you want to start up ftp in MS-Windows (Start -> Run -> type in 'ftp') and upload to somewhere else some file on your machine buried deep in a directory heirarchy, and it is tedious to navigate down with lcd.  (You don't know about lcd?  Read the ftp help).   Here is the easy way of doing it: start up ftp, establish the connection, choose the file type, and type 'put ' (ie, put followed by space, without hitting return).  Then open up Windows <STRIKE>Exploder</STRIKE> Explorer and navigate to the file.  Then, drag and drop the file onto the ftp window. It will insert the full pathname.  



What SSH (Secure SHell) facilities are there here?

acms21.shef.ac.uk is an ssh server with a firewall exemption, so you can connect directly to it from outside the university without going through the insecure telnet/gateway method. Other machines have ssh servers running; if you want your Solaris/Linux box to have one, please let me know.

You can download ssh clients for a number of operating systems from Paul Kerry's ftp server. Note that we already have ssh clients on most Solaris and Linux boxes here - you'll only need to go here if you want one on your Windows box, or are connecting from outside the university.


How can I easily move files between machines which don't have an ftp server?

Use scp (Secure CoPy).

To transfer a file from there to here on a Linux box:

here% scp username@there:/path/to/filename .

e.g.

acml27% scp ap1rlb@acms21:~/docs/paper.pdf .

If you are on a Windows box, download and install WinSCP.


The British Library document supply service

You may need to obtain documents from the British Library document supply service.   They can post, fax, or email the documents.  I strongly suggest you do not request the documents via email.   This is because it arrives as an ebook wrapper to a digital-right-management pdf that can only be downloaded once and printed once (ever), cannot be saved into different formats, and requires a Windows PC with a full version of Acrobat to open.   And all the document is is a scanned photocopy. 

Shells and Shell Commands


What shells are available?

Bourne shell (sh), C shell (csh), Korn shell (ksh), Tcshell (tcsh). There is also a shell mode in emacs.

The Bourne shell (named after Steve Bourne, who wrote it) is the standard Unix shell. It is available on every Unix system, and is good for shell programming. It is still the preferred shell for writing scripts. The interactive facilities are poor though, so...

The C shell was designed to be a basic shell with a C-like syntax, and reasonable interactive facilities like command and filename completion, and a history mechanism for recalling and editing previous commands. Unfortunately these are all rather obscure and hard to use. Also things like redirection and shell programming are broken or brain-damaged. A lot of software assumes you use it.

The Korn ksh was meant to be the best of both worlds, and fancied itself as the new 'standard' shell. However, it fell by the wayside.

The TC shell is extremely user-friendly, with useful features like up-arrow to recall previous commands, and command-line editing (other shells force you to delete back along the command line.) Recommended for novices.

The emacs shell mode is even better, both preserving your underlying shell, and adding excellent editing facilities on top.

You might want to consider perl for scripts.


How do I know what shell I'm in?

% echo $SHELL

How do I change shells temporarily on a particular machine?

Execute the shell binary, eg

% /path/to/shell

How do I change shells permanently on a particular machine?

Ask Richard or Dave.


How do I set up shortcut commands, or alias commands?

Typing long commands over and over again can be tedious.   If you want to set up 'll' as an alias for 'ls -l', for example, you would put the following commands into your resource file (see below)
alias alias 'command'

eg

alias ll 'ls -l'

How do I execute a command automatically every time I log in/start a terminal session?

All of the shells read a 'resource file' located in your home directory.  The name of the file is:

On Solaris: .cshrc5
On Linux: .cshrc (for the TCshell and C-shell) or .bashrc (for the Bourne/Bash shell)

Note that any file prefixed with a period cannot be seen with 'ls' unless you type 'ls -a').   It is also possible that the file doesn't exist - it is not set up by default on Linux machines, so you will need to create it..   Any commands in these shells are run each time you open a terminal session.


What's this PATH thing? a.k.a. where's all the software?

Every shell has an environmental variable called PATH. This is a colon-separated list of directories to be searched sequentially when a command is entered. When a command is found in that list of directories, it stops the search and executes it, and doesn't search for any more commands. In the TC-shell (tcsh), you can see this PATH by typing

% echo $PATH

If a piece of software is located in your PATH, then you just need to type its name to execute it (provided it's executable). Eg,

% ls

If a piece of software isn't in your PATH, then you can't invoke it by typing its name; you have to type the full path to the name, eg

% /software/bin/maple9.5/bin/xmaple

This can soon get tedious.  So you can adjust your PATH.    You can add a directory to the PATH by resetting the environmental variable, eg to add the directory /usr/test to the PATH in the TC-shell,

% setenv PATH ${PATH}":/usr/test"
This will add /usr/test to the end of your path for that terminal session only. If you want it to be permanent, then there's a cunning file that is executed every time you start a new terminal session (ie, log in, or open a terminal window).  THat file is called .cshrc (if you're using the t-shell in Linux, which most people are) or .cshrc5 ( if you're on a Solaris boxes).  It's located in your home directory.   Your .cshrc file does not exist by default; you will need to create it the first time.  Open it in a text editor, like this:

% cd ~

% nedit .cshrc

Add the line:

setenv PATH ${PATH}:/software/bin

This will add the very useful directory /software/bin (full of application software) to your PATH.

Save the file and exit the text editor.  Open up a new terminal window to check, and type in that,

% echo $PATH


Some caveats: it is not a good idea to have different commands named the same as each other and differentiated only by location. Nor is it a good idea to bloat your PATH; it'll be hard to tell if you're picking up the right command, or right library; however make sure that at least /bin, /usr/bin, /usr/local/bin, /software/bin (and, on a Solaris box, /usr/openwin/bin, /usr/ucb, and /usr/ccs/bin) are in the path for minimum functionality. Casual users will rarely need to change their paths. It is advisable to add all your homegrown paths ($HOME/bin, $YOURSOFTWARE/bin, etc) to the end of the PATH and not the beginning, particularly if $YOURSOFTWARE/bin has executables namable by other users...(why this is a risky thing is left as an exercise to the reader...) Finally, it is inadvisable, although not really a hanging offence anymore unless you have root access ;-), to add . ("dot", the symbolic link to your current directory) to your path.

You may also need to set your LD_LIBRARY_PATH, which is a similar idea but used when accessing libraries.  The syntax is similar to the above; eg, in your .cshrc,

setenv LD_LIBRARY_PATH /software/lib


This system's rubbish - you haven't got...

Tell me about it, and I'll try and get it. But please note that duplicating functionality across different platforms is generally of lower priority than installing it in the first place.

Please don't ask unless you really want it.  I spent the better part of a week installing a compiler with some fairly awkward dependencies.  A few months later I asked the user how he was getting on with it.   I was told, "Oh, I didn't use it in the end."   Grrrr.


I can't find any of the commands you're talking about.

Make sure /software/bin and/or /usr/local/bin, and any software-specific path, are on your PATH environmental variable.


What libraries are available?

On acms1 (Solaris server)

Other libraries can be found in /usr/openwin/lib, /home13/packages/SUNWspro/lib, and /usr/local/lib.

On applied (Linux server)


What webbrowers are available and supported on the AM Unix system?

Netscape (although be aware that the Netscape port onto Solaris isn't very stable and falls over at the slightest provocation, like daring to read a webpage that has Javascript).


Where can I get a VRML plugin?

Cortona. Click the button to install, and accept all the default suggestions. This seems to work fine for Win XP; I haven't tried it for other systems.


What editors are available and supported on the AM Unix system?

All these editors (except dtpad) can be accessed by typing editor filename at the command prompt, e.g. % vi test.f. Editor choice is very much a personal affair.


What compilers are available and supported on the AM Unix system?

The current compilers are the Sun Workshop compilers and gcc:



Are there any debuggers?

dbx is line oriented, debugger is the OpenWindows version. You might like to try out GNU gdb (/usr/local/bin).


What is make and how do I use it?

If you're developing programs and you're not using make, you are wasting a lot of your own time. Fed up of typing 'cc -o thing -I/whatsit -I. -O -Lhere -lthis -lthat -lother' every time you want to compile something? Keep having to retype it because you forgot about that OTHER library you needed to link in? Can't remember which programs need recompiling because you've changed a source file? THEN YOU NEED MAKE.

How to start? The Sun manual in answerbook is quite good (it's also available in /usr/local/docs). Read the first bit, then have a go. The best way to write a makefile is to pinch somebody else's and diddle with it. To that end, you'll find one or more sample makefiles in /usr/local/docs/Makefile.templates.

If you really want to burn rubber, there's an excellent O'Reilly book on the subject: "Managing Projects with Make" (Oram and Talbott).

WARNING: the only problem with make is that different versions can differ in extremely subtle ways (and some don't work at all). So porting makefiles between Sun, DEC, SGI etc is rather fraught. GNU provide a make utility, and this is probably the answer.


What software is available for Windows machines?

CICS have a range of software listed here.

The departmental office has the following CDs that you can borrow:

Linda Wilkinson has the Full Acrobat Product on CD.


How do I view a type file?

File type Typical extension Recommended viewer
Postscript .ps gv, ghostview
Any image .gif, .jpg, .jpeg, .tiff, etc xv or Netscape
Adobe .pdf Adobe Acrobat Reader (/usr/bin/acroread for Solaris machines, or get the full version for Windows from Linda Wilkinson)
Microsoft files .doc, .xls Star Office
MS-Windows machine [recommended]


Is there a basic drawing package?

xfig [Unix]


Is there a basic image viewer?

xv [Unix] or Irfanview [Windows]. The latter is extremely versatile and allows you to capture screenshots, do basic manipulation, and convert between a multitude of formats.


How do I convert any standard image format to any other standard image format?

Unix: Use the convert or xv commands.   Windows: download Irfanview.

How do I install and use TeX/LaTeX?

LaTeX, LaTeX2e, MiKTeX, TeX, et al are variants of a (mathematically-biased) very powerful typesetting system, which many people have found to be vastly superior to Microsoft Word's Equation Editor. It is NOT a WYSIWYG editor; rather, the user writes the typesetting codes into the text using any standard text editor and then processes the file to obtain an encoded device-independent (.dvi) file. This can then be processed again to produce a printable file (for example, a Postscript file). A manual of how to use *TeX* is beyond the scope of this document; however I thoroughly recommend the blue spiral-bound book by Knuth, which is in the secretary's office. Another good reference is the LaTeX project homepage.   And before you ask any other questions, have you read the UK List of UK List of TeX Frequently Asked Questions ?

TeX on the Unix machines

 Type 'latex filename.tex' and then 'dvips -o filename.ps filename'.

TeX on a Windows PC

The recommended variant is MiKTeX, available from http://www.MikTeX.org. Select 'Download MiKTeX', download the install wizard, and follow the instructions. You should install a private installation. When you have downloaded and installed MiKTeX, it can be run from an MS-DOS command line (Start - Programs - MS-DOS Prompt).


How do I get bold theta-hat in LaTeX?

In Maths formula, sometimes LaTeX will not interpret the obvious

{\bf \hat{\theta} }
and returns the theta-hat without it being bold (although it does it quite happily with alphanumeric characters being hatted and bolded) So what you require is:
\mbox{\boldmath{$\hat{\theta}$}}

Where can I get the SoMaS TeX style files?

Jim Green has packaged them:

I've repackaged the Butterley SoMaS TeX files (styles for examinations, letters, memos etc) in a form suited to the Unix TeX package teTeX. The package can be found in

/acms22_p3/pub/somastex.tar.gz

or contact me by email: j.j.green@shef.ac.uk


How do I check for NaN in FORTRAN?

A numeric variable takes the value NaN (Not a Number) when, for example, a number is divided by zero. This can be tested by the following code fragment:

.NOT.(x.GE.0.OR.x.LE.0)
(Thanks to Stewart James)

How do I convert between data types in FORTRAN?

You READ from a variable instead of a logical unit. This sample noddy program shows how.

	PROGRAM char2int		
IMPLICIT NONE

CHARACTER*3 chvar
INTEGER ivar
chvar='345'

WRITE (6,*) chvar, ivar
READ (UNIT=chvar,FMT='(I3)') ivar
WRITE (6,*) chvar, ivar

END

How do I include a NAG library function in a FORTRAN program?

The following only applies to the departmental systems.   Iceberg, iceberg, etc, have their own NAG libraries; see the CICS website.

NAG libraries are available on the Linux and Solaris systems for PPARC astronomy (includes STP, Solar) research only.  Your machine must be set up with the /star directory; if it doesn't have a /star directory, ask Richard to install it.

NAG documentation is available on the NAG website under Support/Documentation; there are good FORTRAN examples for most of the function calls.  The documentation is also local under /star/nagdocs.

First you must set up the appropriate variables.

% source /star/etc/login
% source /star/etc/cshrc

Then when linking the NAG libraries, use the following syntax:
                                                                               
 -L/star/lib `nag_link`
                                                                               
eg,
                                                                               
f77 -o a.out yourfile.f -L/star/lib `nag_link`
                                                                               
There's a test suite available.  Copy all the files from /star/nag/fllux20d/test into one of your own
directories, and run the following commands:
                                                                               
% make clean
% make build
% make run_test
                                                                               
This should run OK.


                                                                             

How do I set up OpenOffice?

OpenOffice is the successor to Star Office, and is a suite of word processing, spreadsheet, and similar documents with much the same functionality as Microsoft Office. However it's open source and free, and runs on a variety of operating systems including Windows, Linux and Solaris. We have it installed on most of the AM Linux boxes and the AM Sun system.

To use it on a Solaris box, you must be running at least Solaris 5.7. You need to do the following procedure once only:

% cd /usr/local/OpenOffice.org1.0/program% ./setup
If you get a message that certain patches need to be installed, pleaseemail me with the message and which machine you're on. If you don't getthat message...
...a GUI window will (eventually) appear and you should accept all thedefaults through to the end. (Extra detail should you want: Fill in yourname, details, etc if you wish. Select WORKSTATION installation. Itshould default to putting itself into a subdirectory of your homedirectory, which is fine. If your workstation has Java, it should haveselected itself, which is fine)

After once-only installation, you can invoke it all by dropping into yourlocal OpenOffice.org directory (under your home directory) and typing

% ./soffice
To set up printers, go to that same directory and type
% ./spadmin
Then select the "Printer" and "Unix Name" as appropriate from the manual. (eg - if you want to print on the HP Laserjet 5P/5MP printer in H10a,select New Printer -> Add a Printer -> Next -> Choose "HP LaserJet 5P/5MP"-> Choose "lp -d ps" -> Next)

If you want to install it into the menubar such that you can drag and drop files onto it, expand any desired menu by clicking on the up arrow (eg the old Star Office menu if you used that). Then open up file manager, and drag the ~/OpenOffice.../soffice icon onto the Install Icon button.


What's this Starlink software?

A set of interactive data reduction and analysis tools for use in research. Mostly (as the initiative stemmed from astronomy) these are time-series and 2-D image tools. For more information, see the Starlink website. I have installed the core Starlink software set, and all the libraries, on the applied Linux server. If you want any other packages added, let me know.


Where can I get WinEdt?

Go to the WinEdt website, read all the information about installing and upgrading. When you're happy, select download, and download the latest version from a convenient mirror site. The download is a self-extracting executable. Once it is downloaded, you can register it; email Richard or Dave for the licence code, then select Help -> Register and enter the fields as given in the licence code, click OK, and you're set.


How can I see an array of mugshots of my students in CIES?

Download the form and apply to CICS.



Can you upgrade Netscape on Solaris?

No.   It won't work.  Use Mozilla instead. 


I need quad precision!


Do you, by jove?   You can get quad precision (about 31 decimal places of precision) in certain FORTRAN 90 compilers with a variable declaration such as

integer, parameter :: prec=selected_real_kind(31)

However, not all F90 compilers support this - the Portland Group compilers don't, for example.    You might want to look at the GNU Multiple Precision Library http://www.swox.com/gmp/.  

Andrew Beresford in CICS says:

"I have spoken with a friend of mine who works on the GNU compilers for a living and he has suggested that using a "double double" will produce results good enough for 30 digit integer precision.
                                                                               
There is more information at http://crd.lbl.gov/~dhbailey/mpdist/ - he pointed me specifically towards the DDFUN90 package.
                                                                               
The benefit of doing this is that it should significantly quicker than using an arbitrary precision maths library (like gmp). He also mentioned that x86_64 has no quad precision hardware on-chip and so it is not going to be particularly quick. He believes that only some Crays and some DSP chips have quad precision hardware built in. Other architectures doing quad-prescision are most likely to be doing the "double double" trick in the compiler, which results in slightly poorer (a couple less bits) precision than true quad precision."



File Formats

What are these image formats anyway?

Which should I use?  Why are some files bigger than others?    It's the same image, isn't it? Eww, why does that version look so terrible?

When you scan an image, it is scanned as a bitmap.   That is, every pixel of the image is given a colour (or shade of gray).   These are stored in a huge file - so a 6" by 4" photograph scanned in at a moderate 1200 dpi (dots per inch, or pixels per inch) with 24-bit colour (ie, 2^24 levels of colour) will need to store 6  x 4 x 1200 x 1200 ~ 34 million pixels.   If each pixel takes up 24 bits, that is around 104 Mb.  If you try to download this on a dialup connection, you won't be very happy.

So image compression routines are used. Inevitably, some information will be lost as part of the compression algorithm.   Each of the many compression algorithms available are optimised for different things - some to produce very small files, some to retain good quality line drawings, some to retain good quality photographs.   It is swings and roundabouts which you choose.

jpeg or jpg:  the JPEG image algorithm is optimised for shrinking the size of photographs or paintings with broad changes of colour and  shade.  Thus, it shrinks these very well with minimum loss of data, but performs badly when asked to compress bitmaps that are line drawings, containing high-contrast narrow features.  I suggest you use this with photographs and the like.

png: the PNG algorithm is good generally with excellent compression, and performs better than JPEG when dealing with line drawings.  I suggest you use this with line drawings.

gif: the GIF algorithm is based on proprietry code, and the owners became more aggressive in trying to secure money for its use.   Thus, many software vendors dropped GIF support so they do not have to pay large licence fees.   Although the patents on the algorithms in question expired in 2004, it is uncertain at this time whether major software vendors will pick up the gif support once more, or not.

bmp: The bitmap format doesn't compress at all, so there is no data loss.  But, as the example above suggests, it can result in some huge files.  Useful for a master copy of an image.

The convert command on Unix (part of ImageMagick) is a wonderful tool for converting between image formats. So too is the xv (Unix) and Irfanview (Windows) image viewers.

How do I convert a .doc file or a .rft file into a PDF?

On Windows: install Adobe Acrobat (CD from Linda Wilkinson).   You will acquire a PDF button in MS-Word.

On Unix/Linux: OpenOffice already has a document-to-PDF button.



How do I convert from MS-Word (or anything that loads into MS-Word) to Postscript?


How do I convert a file from Postscript to PDF or vice versa?

ps2pdf oldfilename.ps newfilename.pdfpdf2ps oldfilename.pdf newfilename.ps

(This works with postscript files produced with the above method, and is thus a good method of getting a platform-independent file out of Microsoft J


How do I convert a file from Postscript or PDF to ASCII?

ps2ascii oldfilename.ps newfilename

How do I convert a file from Postscript to Encapsulated Postscript?

ps2epsi oldfilename.ps newfilename

How do I convert a file from Encapsulated Postscript to anything?

There are a number of dodges which may or may not work.Try loading the file into a postscript viewer or drawing package that allows you to Save As. Possibilities include gv, gs, ghostview or xfig (on the Unix system), Ghostgum, Corel Draw or Irfanview (on PCs)


How do I convert a file from jpeg to postscript?

There is a useful package called jpeg2ps. This is very useful for including images into LaTeX documents.


How do I rename all .x files to .y?

The obvious 'mv *.x *.y' doesn't work (why?). Use the following shell script instead:

#!/bin/sh
for i in *.x
do j=`basename $i '.x'`
mv $j.x $j.y
done

'basename' forms the basis for many tricks with filenames containing a suffix.


How do I strip the leading 1,2,3,4,etc characters from filenames?

#!/bin/sh
for i in *.jpg
do j=`echo $i | sed 's/^..//'`
mv $i $j
done

On the third line above, put in as many periods as you have characters needing deletion.


How do I compress and uncompress archive files?

No, you can't do multiple compressions with the same algorithm to iteratively reduce the filesize! Also note that some file formats (particularly image formats) are already effectively compressed, so it may not be possible to compress them further.

An oft-encountered file is a .tar.gz file, which is a group of files first archived into a tarball with the tar command, then gzipped to compress it.   To uncompress it, gunzip it, then tar it.


How do I prevent Windows from hiding file extensions and automatically renaming over them?

Start -> Settings -> Folder Options -> View Tag -> unselect 'Hide file extensions for known file types'
(Thanks to Youra and Stuart)


Backing up your data


What is backed up for me?

If your data resides on a disk not listed below, it is NOT BACKED UP!, so you will need to do it yourself. And if it is critical, you should BACK IT UP YOURSELF INDEPENDENTLY!

On the Novell Network:

U: drive is backed up - see CICS. 
Your C: drive is NOT backed up by us (except for Robertus's, which is backed up weekly).

On Iceberg:

Your homespace is backed up - see CICS.

On the Applied Maths Solaris network:

The following file systems containing user data are backed up and kept for 8 weeks before being recycled.  If you have lost data and think that it should be backed up as below, please let myself and Dave Robson know, and we will try to retrieve it. Please note that retrieval will take several hours, so if you can recreate the data quicker through other means, it will be more efficient to do so! 

The following contain the user home directories and are backed up thrice weekly.

The following are all backed up weekly (on Tuesdays)

The following are backed up with a guaranteed weekly backup (although it can be more frequent) and a tape retention of 3 weeks 2 days.  Thus, the most that can be lost is a week's data; the tapes are retained for 3 weeks 2 days before being recycled.

On the Applied Linux network:

The following systems are backed to tape with a guaranteed weekly backup and a tape retention of 3 weeks 2 days.  Thus, the most that can be lost is a week's data; the tapes are retained for 3 weeks 2 days before being recycled.

If you find you have lost anything, please email us with the filenames and paths you want recovering, eg "/home/user/docs/really_important_presentation.sxw".  The more precise you are, the easier it is for us.  

applied2: /, /var, /boot, /userdata, /software (nb: contains some Solaris data disks)
ctcd-data: /boot, /, /SOFTWARE

acml4: /home
acml5: /home
acml16: /home
acml21: /home
acml23: /home
acml24: /home
acml26: /home
acml27: /home
acml28: /home
acml31: /home
acml32: /home
acml33: /home
acml34: /home
acml35: /home
acml36: /home
acml39: /home

The following /home directories are backed up to an internal disk mounted under /yesterday.   The disks are rsynced each night in the early hours.  Thus, /home/user/somefile.tex is copied to /yesterday/home/user/somefile.tex.  You can thus copy it back without needing us.  If the original file is deleted, the copy will be deleted that night; thus, you have a maximum of 24 hours to self-recover.

acml4: /home
acml5: /home
acml9: /home
acml16: /home
acml20: /home
acml21: /home
acml22: /home
acml23: /home
acml24: /home
acml26: /home
acml27: /home
acml31: /home
acml32: /home
acml33: /home
acml34: /home
acml35: /home
acml36: /home
acml39: /home

The following disks are backed up to tape weekly.  If you find you have lost anything, please email us with the filenames and paths you want recovering, eg "/home/user/docs/really_important_presentation.sxw".  The more precise you are, the easier it is for us.  

/SPARG_DATA{1,2,3,4}
/MJT_DATA{1,2}
/CTCD_DATAB




What do I need to back up myself, and what do I need to know about backups?

Thoughts before you backup:


How do I use the CD-writer in H10a to back up from Windows XP?

Go to the XP machine in H10a (the one with the scanner attached) and log in to your Novell account, and the local "User" account. Do not change the local account's password.   Copy all the files you want to backup to a local directory on the machine (but delete them afterwards, please).    Insert a blank CD (from Lynda or Maureen) into the drive.  A window will open.  Drag the files you want to backup into this window.   When you are ready, click "Write these files".  At the end, click "Finish".  




How do I back up onto the Maxtor disk?

This disk is only for postgraduate members of SPARG. 

Make sure you have got
- the Maxtor disk from Robertus.
- the installation software CD from Robertus.
- the power supply and mains lead
- the USB lead (red-ended at both ends)

You also need to be sitting in front of
- a PC with a USB bus (and know where it is, and how to connect a USB device).

Windows XP instructions:

Setup:

You need to do this once only ever per PC.   Insert the installation CD.  Select English Language.  Select Install Software.  Select Custom Installation.  Agree with all the prompts, particularly making sure that you install for all users.   At the end of the installation, reboot.

Usage:

With your PC up and running, set up the Maxtor disk on the desktop, somewhere where you won't knock it.   Plug it in to the mains (via the supplied power supply) and turn it on at the switch at the back of the device.  Then plug the USB lead first into the back of the Maxtor (into the U-shaped socket), then plug the other end of the USB lead into your PC.   The PC will detect the new drive (and may warn you that a fast USB drive has been plugged into a slow USB connection - ignore this).   It may take up to a minute to detect it fully, so be patient, young grasshopper.  DO NOT TOUCH THE BLUE BUTTON ON THE FRONT, EVER.

The drive is now visible to "My Computer" as a hard disk drive.   On my PC, it is lettered G:, but yours may vary.  It will show a capacity of ~152 Gb.

Now copy your files into your folder in the normal drag/drop way.  Everyone (Stuart Thom, Ben Carter, James Hargreaves, Maria Dymova, Andrea Kerekes, Indrani Roy, Moniba Shams, David Simpson) has a folder.     You can monitor how much is in there by right-clicking on the folder and selecting Properties.

When you are done, look for the Safely Remove Hardware icon on the right-hand side of the XP toolbar (hover your mouse over it - it'll say Safely Remove Hardware).   Click it: a small balloon will pop up saying "Safely remove Maxtor OneTouch drive".   Click on that.   (If you get an error message: "The device 'Generic volume' cannot be stopped right now.  Try stopping the device again later'" appears, click OK, then close any Windows Explorer windows that are open viewing it).    When it says you can safely remove the hardware, unplug the USB connection, then turn off the Maxtor drive at the switch at the back, then bring it all (leads, CD, etc) safely back to me.

Linux RedHat instructions

Setup:

Ask Richard to get your machine set up to use USB from Linux.

Usage:

Physically connect the USB device as above.   Then type 'mount /mnt/usb'.   The device should mount onto that mount point (ie, Stuart Thom's directory is /mnt/usb/Stuart Thom, etc).  Files are copied in the usual ways.

When you are done, leave the /mnt/usb tree (ie, you cannot have any graphical explorers looking at it, you cannot be 'parked' in any /mnt/usb subdirectory at a terminal prompt, and you cannot have any files open in editors, etc), then type 'umount /mnt/usb'.   If it refuses to umount, you haven't cleanly exited the tree - look more closely for processes with files open in the /mnt/usb tree.  Then you can physically disconnect the drive.

Dire Warnings:

DO NOT ATTEMPT TO USE THE BLUE BUTTON ONE-TOUCH BACKUP/RESTORE.

DO NOT TOUCH ANYONE ELSE'S FILES.


DO NOT SAVE MORE THAN 20 Gb OF DATA.

DO NOT DISCONNECT IT WITHOUT FIRST USING THE "SAFELY REMOVE SOFTWARE" PROGRAM (Windows) OR UMOUNT COMMAND (Linux)

DO NOT KEEP IT OVERNIGHT, TAKE IT HOME, ETC. 

If you use more than 20 Gb of space, I'll simply delete your files until it is reduced to < 20 Gb - play fair, there are 8 of you sharing 152 Gb (spot the maths wizard here...)



How do I mount a memory stick (flash stick, USB stick) on a Linux machine?

Insert the memory stick into a free usb slot.  Then type

% mnt /mnt/flash

(If this doesn't work - ie, you get the message

mount: can't find /mnt/flash in /etc/fstab or /etc/mtab

then try

% mount /mnt/usb 

instead.  If this doesn't work, see Richard.

When you've finished copying data, make sure all applications have quite the mounted directory, and type

% umount /mnt/flash

(or /mnt/usb, as appropriate).


The SPARG big disk


What's the SPARG big disk?

A 4 Tb disk usable by the SPARG group, partitioned into 400 Gb chunks (because that's what will fit on a tape, and we don't anticipate anyone having a single file greater than 400 Gb - no Oracle databases here, no sir.)

How do I get at it?

SPARG linux boxes are being upgraded to mount this disk.     It is on an automount, so will not necessarily be visible if you type 'df -k'.    However, the partitions are visible at the root level of your disk, so type 'ls /'; you'll see /sparg_data1, /sparg_data2, etc.   You can cd to those, and then drop down one level into your directory, eg /sparg_data1/ap1rb.

What if I can't see them on / ?

I haven't got to your machine yet, probably because it spends most of its life in Windows mode.   Arrange an appointment for me to fix it up in Linux mode; it'll take half an hour or so.

When I cd to /sparg_dataX, it's really slow to respond.

That's the automounter waking up.   It'll be faster immediately afterwards.

What's an automounter?  Why can't I have a hard mount?

An automounter mounts remote file systems on demand, and unmounts them after some period of inactivity (10 minutes, here).    It saves on network traffic and causes less problems if there are network glitches, or the big disk server goes down.

I can see /sparg_data1, etc, but I don't have an account under it.

See me.

Can I access the sparg big disk from a Windows box?   a Solaris box?   via ftp?

No, no, and no - but we are working on a secure way of doing all of the above.

Can you mount the sparg big disk onto iceberg?

No.  However, you might want to look at this link

I've got a self-administered Linux box (ie, I know the root password).  Can I have write access to the SPARG disk?

No.

What about quotas?

There are none, as yet, and we don't plan on implementing them unless anyone gets silly amounts of data.

Is it backed up?

It's running on two RAIDsets of 8 and 7 disks respectively, both configured as RAID 5, with one hot spare.  What that means is if a disk fails, the hot spare takes over with no dataloss (although as it rebuilds itself, it'll be a bit slower than normal).   We are planning on also making weekly backups to tape, although we are awaiting arrival of some hardware before we can do that.

You said it was 4 Tb, but I can only see 3 x 380 Gb and 1 x 280 Gb disk, which doesn't add up to 4 Tb.

There's (lots of) unallocated space that we're holding in reserve until we see how the first 1.4 Tbs are used; plus there's one 400 Gb hot spare disk, and then the RAID 5 structure consumes disk space because of the parity data.



Optimising your programs


Optimisation

The following is a very basic checklist of how to optimise your program.    It is a summary of the key points of the Optimisation and Parallelisation talk given by RLB each September, which in turn is a summary of an intensive three day Sun workshop, so it cannot replace in-depth reading.

0. Make sure that your code is performing as expected and the answers are correct.

1. Read the manual pages for f90 (or cc) and understand the meaning of the different compile flags.

2. Inspect your code visually for, eg, uneccessary looping.

3. Check your loops for memory access.  Remember that in FORTRAN, arrays are column-major (in C, arrays are row-major).   This means that the elements of an (i,j) array (of i-columns and j-rows) is stored in memory in the sequence (1,1) (2,1) (3,1)...(j-1,1) (j,1) (1,2) (2,2) (3,2)...(j,2)...etc in FORTRAN.   Memory is pulled into the L1 cache in sequential slices of 8kb or so; thus, if you address your array in a column-major manner, this is very efficient; one memory operation will call up many array elements.     Below, the first nested loop is very inefficient as it addresses the array in a row-major manner.    The second will perform much faster.

c    This is a very inefficient way of addressing FORTRAN arrays.
        DO i=1,arraysize
          DO j=1,arraysize
            a(i,j)=100.0
          END DO
        END DO
         
c    This is far better.   The array is accessed in a column-major manner,
c which matches how it is stored in memory.
        DO j=1,arraysize
          DO i=1,arraysize
            b(i,j)=200.0
          END DO
        END DO

4. Use the -fast compile option.   This is a macro of several options - use the -v option to check which they are.   Make sure you understand what each option is doing.     For good code that does not rely on IEEE exception handling, -fast is often the quick and easy way to improve your program runtime.   But if -fast breaks the code, then break down the individual options.

5. Use the Sun performance analyser to check how each (additional) compile flag performs.



Parallelisation Checklist

This assumes compiling a program with the Sun compiler suite and running it on iceberg.  

1. Parallelisation is another optimisation technique.

2. Wallclock time will (usually) decrease, but CPU time will increase, scaling with the number of processors due to housekeeping, cache coherency, etc.

3. Shared Memory Parallelisation (SMP) is loop-dependent.   Only loops will be parallelised.    Non-loop code fragments will be run in serial mode.   For every parallelised loop, the compiler will generate a parallel AND a serial version.   The latter will be generated if there is not enough work to be done - this decision is made at runtime by the program and is not controllable by the user.  

4. Loops are parallelisable if there is independence in the order of operations - there is no fixed ordering and no order of time.

5. A race condition/data dependency is one where shared variables are not protected - the results depend which processor gets there first, and are not reproducible.  

6. The compiler will produce parallelised code for a loop if a) it can prove there are no race conditions, b) it can prove it can break a race condition by rewriting the code, or c) you tell it explicitly there are no race conditions.   If (c), you must be sure that is the case.

7. The easiest way to generate parallised code is to use the -xautopar and -xloopinfo flags at compile time.  This will allow (safe) parallelisation.

% f90 -xautopar -xloopinfo parallelprog.for

8. Set the OMP_NUM_THREADS environmental variable to be the number of processors you want to employ.  Clearly this cannot be greater than the number available, but you may find optimal results with a smaller number.

9. Set the environmental variable SUN_MP_WARN to be 'true' to receive useful diagnostics.

10. Analyse your program with the Sun performance analyser suite!



Performance Analysis

This is a short checklist of how to use the Sun performance analyser, which is available on iceberg.

1. Compile your program with the -g option:
% f90 -g <-all your other flags> program.for

2. Collect the data:
% collect a.out

3. Analyse the data:
% analyzer test.1.er
See where the most expensive lines of code are by examining the Source.    Can you rewrite to improve performance?   Experiment with different compiler options and repeat from step 1.   Each time you run the analyzer a new (incremented) set of test results will be obtained, allowing you to compare with previous runs.

4. When you are happy you have achieved the optimal performance, compile again without the -g flag (which will slow things down!) and run your program as normal.
% f90 program.for
% ./a.out


Applied Maths will take you back to the Applied Mathematics main page. Copyright © 2002 The Department of Applied Mathematics, University of Sheffield, UK. Every effort is made to keep the information on these pages accurate, but this is not a guarantee that it is correct. Please contact the page maintainer for further information. This document validates as HTML 4.0 Transitional. These pages created and maintained by Richard Balthazor.

Valid HTML 4.0! Valid CSS!