Using Globus 4.0.1 at NIEeS
From GridInfo
NIEeS is running Globus 4.0.1 and connecting to the backend Condor Pool, which is part of the CamGrid. NIEeS has configured both Pre Web Services GRAM and WS GRAM for people to use. It also supports GridFTP but not MDS at this moment. The following examples show simple usage of GT4.
In order to use the NIEeS Globus service, you need a NIEeS certificate. At this moment, the NIEeS Condor Pool accepts certificates issued by NIEeS itself or those issued by the UK e-Science Certificate Authority (CA) Please look at the NIEeS Cluster Users Guide or GridInfo wiki for more information about how to obtain your NIEeS certificate.
Contents |
Pre-WS-GRAM
This is the same as GT 2.4. if you already know GT 2.4, you can skip this section.
globus-job-run
globus-job-run is an online interface to job submission, featuring staging of data and executables using a Global Access to Secondary Storage (GASS) server. Staging is the process in which files are copied from the local machine to the remote machine on which the processing takes place and automatically deleted from it when processing is complete.
You can use the following command to submit jobs:
$ globus-job-run cete.niees.group.cam.ac.uk/jobmanager-condor /bin/hostname
pip--escience
The Globus jobmanager, in the NIEeS environment just now, can fork jobs to run on the same machine, or it can submit jobs to a condor pool. In the above example the Globus executable called is jobmanager-condor and it runs the command /bin/hostname on the remote machine. The result returned, "pip--escience", is the name of the machine in the CamGrid Condor pool which ran the command.
If you are going to stage files, you can use the following command. Test.sh is the shell script executing /bin/hostname in this case.
$ globus-job-run cete.niees.group.cam.ac.uk/jobmanager-condor -s Test.sh
pip--escience
One can also stage in stdin and stage out stdout using the same method.
$ globus-job-run cete.niees.group.cam.ac.uk –stdin /home/gtniees/Pre-WS-GRAM/file1.txt \
-stdout /home/gtniees/Pre-WS-GRAM/catout.txt -s cat.sh
gtniees@cete:~/Pre-WS-GRAM$ less catout.txt
Hello!
globus-job-submit
globus-job-submit is a batch interface to the Globus Resource Allocation Manager (GRAM) server. It will return immediately, leaving you with a contact string that you can use to query the status of your job.
This is very similar to globus-job-run but globus-job-submit does not currently support staging of files automatically.
The contact strings are functionally opaque, but look like the following:
$ globus-job-submit cete.niees.group.cam.ac.uk/jobmanager-condor /bin/hostname
https://cete.niees.group.cam.ac.uk:32863/3669/1144673079/
then one can use the contact string to access the following commands:
globus-job-status
globus-job-status can monitor the status of the job. The status could be PENDING, ACTIVE, DONE, FAILED.
$ globus-job-status https://cete.niees.group.cam.ac.uk:32863/3669/1144673079/
PENDING
$ globus-job-status https://cete.niees.group.cam.ac.uk:32863/3669/1144673079/
DONE
globus-job-get-output
once the job is done, collect the output with this command:
$ globus-job-get-output https://cete.niees.group.cam.ac.uk:32863/3669/1144673079/
herschelnode1—escience
globus-job-clean
stops the job if it is running, and cleans up the cached copy of the output:
$ globus-job-clean https://cete.niees.group.cam.ac.uk:32863/3669/1144673079/
WARNING: Cleaning a job means:
- Kill the job if it still running, and
- Remove the cached output on the remote resource
Are you sure you want to cleanup the job now (Y/N) ?
Y
Cleanup successful.
globusjobrun
globusrun is the executable which gives you access to the full power of Resource Specification Language (RSL). globus-job-run and globus-job-submit are both shell script wrappers around globusrun. More detail about RSL can be found here: http://www-unix.globus.org/toolkit/docs/2.4/gram/rsl_spec1.html
Since GGF is trying to develop a standard job description language (JSDL), we are not going to put too much effort at RSL. We can move to WS-GRAM directly.
WS_GRAM
The concept of using WS_GRAM is similar to before. The following are some examples of using WS-GRAM to submit jobs:
gtniees@cete:~/WS-GRAM-job$ grid-proxy-init Your identity: /O=NERC/OU=NIEeS/OU=niees.group.cam.ac.uk/CN=Gen-Tao Chiang Enter GRID pass phrase for this identity: Creating proxy .................................. Done Your proxy is valid until: Tue Apr 4 01:46:14 2006
A trivial fork job
gtniees@cete:~/WS-GRAM-job$ globusrun-ws -submit -c /bin/hostname Submitting job...Done. Job ID: uuid:fb45978c-c30f-11da-ae3c-0013204b41d4 Termination time: 04/04/2006 12:47 GMT Current job state: Active Current job state: CleanUp Current job state: Done Destroying job...Done.
If you want output to go to a file, use the –so switch
gtniees@cete:~/WS-GRAM-job$ globusrun-ws -submit -s -so output1.txt -c /bin/hostname Delegating user credentials...Done. Submitting job...Done. Job ID: uuid:2c2ea4f6-c310-11da-8b67-0013204b41d4 Termination time: 04/04/2006 12:48 GMT Current job state: Active Current job state: CleanUp-Hold Current job state: CleanUp Current job state: Done Destroying job...Done. Cleaning up any delegated credentials...Done. gtniees@cete:~/WS-GRAM-job$ less output1.txt cete
Submitting job with an XML description
gtniees@cete:~/WS-GRAM-job$ less c.rsl
<job>
<executable>/bin/hostname</executable>
</job>
gtniees@cete:~/WS-GRAM-job$ globusrun-ws -submit -s -so xmltestout.txt -job-description-file c.rsl Delegating user credentials...Done. Submitting job...Done. Job ID: uuid:7a87022c-c89e-11da-878d-0013204b41d4 Termination time: 04/11/2006 14:29 GMT Current job state: Active Current job state: CleanUp-Hold Current job state: CleanUp Current job state: Done Destroying job...Done. Cleaning up any delegated credentials...Done.
Submitting job to a batch system (Condor)
gtniees@cete:~/WS-GRAM-job$ globusrun-ws -submit -Ft Condor -s -so output2.txt -c /bin/hostname Delegating user credentials...Done. Submitting job...Done. Job ID: uuid:ce1cc8d8-c310-11da-90c0-0013204b41d4 Termination time: 04/04/2006 12:53 GMT Current job state: Pending Current job state: Active Current job state: CleanUp-Hold Current job state: CleanUp Current job state: Done Destroying job...Done. Cleaning up any delegated credentials...Done.
In the above case, the output file should contain the hostname of the execution node.
gtniees@cete:~/WS-GRAM-job$ less output2.txt badger1--niees--group.grid.private.cam.ac.uk
Running many jobs simultaneously
The following is an example of a job description for running many jobs. In globus terminology this is called a "multijob".
<multiJob>
<executable>/bin/echo</executable>
<stdout>/home/gtniees/WS-GRAM-job/stdout.txt</stdout>
<stderr>/home/gtniees/WS-GRAM-job/stderr.txt</stderr>
<job>
<factoryEndpoint
xmlns:gram="http://www.globus.org/namespaces/2004/10/gram/job"
xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/03/addressing">
<wsa:Address>
https://cete.niees.group.cam.ac.uk:8443/wsrf/services/ManagedJobFactoryService
</wsa:Address>
<wsa:ReferenceProperties>
<gram:ResourceID>Condor</gram:ResourceID>
</wsa:ReferenceProperties>
</factoryEndpoint>
<argument>Hello!!</argument>
<argument>Welcome to NIEeS!</argument>
</job>
<job>
<factoryEndpoint
xmlns:gram="http://www.globus.org/namespaces/2004/10/gram/job"
xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/03/addressing">
<wsa:Address>
https://cete.niees.group.cam.ac.uk:8443/wsrf/services/ManagedJobFactoryService
</wsa:Address>
<wsa:ReferenceProperties>
<gram:ResourceID>Condor</gram:ResourceID>
</wsa:ReferenceProperties>
</factoryEndpoint>
<argument>This is</argument>
<argument>GT4 WS GRAM Testing Job!!</argument>
</job>
</multiJob>
gtniees@cete:~/WS-GRAM-job$ globusrun-ws -submit -s -so multijobout.txt -job-description-file multijob.rsl Delegating user credentials...Done. Submitting job...Done. Job ID: uuid:3655038c-c8a4-11da-aa6a-0013204b41d4 Termination time: 04/11/2006 15:10 GMT Current job state: Pending Current job state: Active Current job state: CleanUp-Hold Current job state: CleanUp Current job state: Done Destroying job...Done. Cleaning up any delegated credentials...Done.
Look at the output file:
gtniees@cete:~/WS-GRAM-job$ less multijobout.txt Hello!! Welcome to NIEeS! This is GT4 WS GRAM Testing Job!!
For more information about the GT4 command line interface, you can go to the following link:
http://www-unix.globus.org/toolkit/docs/4.0/execution/wsgram/WS_GRAM_Command-line_Frag.html
For more information about how to write a Job Description
http://www-unix.globus.org/toolkit/docs/4.0/execution/wsgram/schemas/gram_job_description.html
