using fluent for hpc
DESCRIPTION
USING FLUENT FOR HPC. IT MANAGEMENT OF FME, 3 RD MARCH 2011. important notes. This is not a FLUENT introductory course but an advance course of existing Fluent users to run their models on HPC. Table of contents. THE HPC FACILITY USING PUTTY AND WINSCP TO ACCESS THE SERVER - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/1.jpg)
USING FLUENT FOR HPC
IT MANAGEMENT OF FME, 3RD MARCH 2011
![Page 2: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/2.jpg)
important notes
• This is not a FLUENT introductory course but an advance course of existing Fluent users to run their models on HPC.
![Page 3: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/3.jpg)
Table of contents
• THE HPC FACILITY• USING PUTTY AND WINSCP TO ACCESS THE
SERVER• SENDING FILES TO THE SERVER• RUNNING JOBS• MONITORING JOBS• COPY RESULTS BACK FROM THE SERVER• Using CYGWIN to ACCESS Fluent
![Page 4: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/4.jpg)
HPC @ cict
• Sunfire• 8 CPUs x 6 Nodes - Quad-Core AMD
Opteron(tm) Processor 2376 HE (2.3 GHz)• Interconnected using Infiniband and Ethernet• Each Node has 8 GB of memory• Storage capacity - 100 GB at the moment
![Page 5: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/5.jpg)
QUEUE SYSTEM
• Using Torque - torque-server-2.3.6-1cri.slc4 and torque-mom-2.3.6-1cri.slc4
• Scheduler - maui-server-3.2.6p21-snap.1224706197.2.slc4
• Current MPI does not utilise Infiniband. This soon will be fixed.
![Page 6: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/6.jpg)
Table of contents
• THE HPC FACILITY• USING PUTTY AND WINSCP TO ACCESS THE
SERVER• SENDING FILES TO THE SERVER• RUNNING JOBS• MONITORING JOBS• COPY RESULTS BACK FROM THE SERVER
![Page 7: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/7.jpg)
installing Cygwin
• Cygwin is an opensource software that can be installed on your personal Windows XP or 7 computer.
• It can be downloaded from www.cygwin.com.• To use its X-server functionality, please visit
http://x.cygwin.com/. • The userguide is at
http://x.cygwin.com/docs/ug/cygwin-x-ug.html
![Page 8: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/8.jpg)
running Cygwin
• After installation, a program called Cygwin will be installed.
• A program called Cygwin/X can be executed.• The X-server willbe running.Leave it there.
![Page 9: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/9.jpg)
accessing the system
• Using putty.exe– http://fkm.utm.my/ftp/pub/Windows/putty/putty.exe
• Using winscp.exe– http://fkm.utm.my/ftp/pub/Windows/winscp/winscp.exe
![Page 10: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/10.jpg)
Login into the system via putty and winsCP
Server : fkm.utm.myPort : 2323
or
Server : ce.utmgrid.utm.myPort : 22
![Page 11: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/11.jpg)
Via putty
![Page 12: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/12.jpg)
via putty, enable tunnelling
Enable X tunnellingif you want to use CYGWINX-server to run Fluent GUIfrom the HPC server.
![Page 13: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/13.jpg)
via putty
press YES
![Page 14: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/14.jpg)
via putty
![Page 15: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/15.jpg)
via winscp
Click NEW
![Page 16: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/16.jpg)
via winscp
![Page 17: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/17.jpg)
via winscp
1. click YES2. click Continue3. enter Your Password
![Page 18: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/18.jpg)
VIA winSCP
![Page 19: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/19.jpg)
SENDING FILES TO THE SERVER
1. Search files to be transferred at the left panel.
2. Create a new directory on the right panel.
3. Select files on the left panel.4. Copy from left to right.
![Page 20: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/20.jpg)
FLUENT
• On HPC , you can run Fluent in the Fluent windowing environment or submit your application which needs more than 30 minutes of CPU time to run to the PBS queue. Interactive sessions are recommended only for pre- and postprocessing and for the solution of small problems. Larger problems should be run as batch jobs for either serial runs or parallel runs of Fluent. The user may check the possible command line options by entering the command.
![Page 21: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/21.jpg)
FLUENT
• Fluent interface can be access via CYGWIN.• Cygwin is an opensource software that could
emulate X-server. • The X-server is used to run ANSYS Workbench.• Run the following command in your putty.
/opt/exp_soft/share/istas/ansys_inc/v121/Framework/bin/Linux64/runwb2
![Page 22: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/22.jpg)
FLUENT : Example Workbench
![Page 23: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/23.jpg)
Fluent• Type ‘pico model_journal’• Enter file/read-case your_input_file.cas solve/init/initialize-flow solve/iterate 400 file/binary-files n file/confirm-overwrite n file/write-data your_output_file.dat exit y Press Control–O to save Press Enter Press Control-X to exit
![Page 24: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/24.jpg)
fluent• prepare the pbs-script, type ‘pico pbs-script’• Enter the following
#!/bin/sh#PBS -q utm#PBS -N istas.model3d#PBS -l nodes=1:ppn=8#PBS -M [email protected]#PBS -m abenCPU=8version=3djournal=model_journalcd $PBS_O_WORKDIR/opt/exp_soft/share/istas/ansys_inc/v121/fluent/bin/fluent $version -t$nCPU -g -i $journal -mpi=openmpi -
cnf=$PBS_NODEFILE -ssh
Press Ctrl-O to save Press Enter Press Ctrl-X to exit
![Page 25: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/25.jpg)
fluent
• Finally we want to submit the job,• just type ‘qsub pbs-script’• type ‘qstat’ to see the status of your job.• an email will be sent to you to let you know the
job has started.• another email will be sent to let you know the job
has ended.• You could open WINSCP again to copy the output
back to your PC.
![Page 26: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/26.jpg)
USEFUL PBS commands
• qsub: Once a PBS job script is created, it is submitted to PBS via the qsub command. In its simplest form, qsub takes a single parameter, the name of the script file that you wish to submit.
• qstat: The qstat command will allow you to view the contents of the PBS queue.
• node1:~/test> qstat Job id Name User Time Use S Queue ---------------- ---------------- ---------------- -------- - ----- 147.node1 testjob psmith 0 R default
![Page 27: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/27.jpg)
cont...• qdel: The qdel command takes a single argument, a job
number. You can use qdel to abort execution of your job: qdel 147 would cancel execution of the job shown in the qstat example above.
• qalter: The qalter command is helpful for altering the parameters of a job after it's submitted. qalter takes two arguments: the PBS directive that you wish to change (like -l), and the job number that you want to change. For example, if you forgot to set the walltime that your job requires, you can change it after it's been submitted:
• node1:~> qalter -l walltime=4:00:00 147
![Page 28: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/28.jpg)
cont...
• pbsnodes: The pbsnodes command, while a useful PBS administration command, can also be informative to the PBS user. pbsnodes -a will list all PBS nodes, their attributes, and job status. This is a useful way to get a list of valid machine properties for use in a #PBS -l directive.
node1:~> pbsnodes -a node2 state = free np = 2 properties =
gigabit,pcn,m2048,dual,p1800,athlon ntype = cluster
![Page 29: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/29.jpg)
Exercise
• Download this file
http://hpc.fkm.utm.my/fluent.tar.gz
and unzip it in a folder.There are four files in the zip file,Makefile, model.cas, model_journal, pbs-script
![Page 30: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/30.jpg)
Exercise
• Open WinSCP• Click New• Enter Hostname : ce.utmgrid.utm.my• Enter your username• Enter your password• Transfer the content of Fluent folder to the
server.
![Page 31: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/31.jpg)
Exercise
Copy the whole directory from left to right.
![Page 32: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/32.jpg)
Exercise
• Open Putty• Enter hostname : ce.utmgrid.utm.my• Click Open• Enter your username• Enter your password• If all are OK, you will be getting a Unix shell.
istaz@ce ~/ $ _
![Page 33: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/33.jpg)
Exercise
Login to the servervia putty.
![Page 34: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/34.jpg)
ExerciseCommon Unix Commands
Command Description
ls -l list directory
mkdir xxx create directory xxx
rm xxx delete file xxx
cd .. back to previous level directory
cd - shifted to previous directory
exit log off from server
pwd current director
who who is currently logging
cp aaa bbb copy file aaa to a new file bbb
mv aaa bbb move/rename file aaa to file bbb
clear clear screen
![Page 35: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/35.jpg)
Exercise
• Type ‘ls’ to list the content of the directory.You will see the name of the directory that contains the Fluent file that was uploaded a moment ago.
• Type ‘cd <dirname>’. Move the <dirname> directory.
• Type ‘ls’ again.
![Page 36: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/36.jpg)
Exercise
• Now we can to see the content of our model_journal file.
• Type ‘pico model_journal’.• You will see a series of Fluent command to be
executed when your model is ran.• Type ‘control-x’ to exit.
![Page 37: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/37.jpg)
Exercise
To edit the content of pbs-script, type ‘pico pbs-script’
You could edit the content of PBS-SCRIPT and to exit without save – type ‘control-x’ and type ‘n’ for no to save. To save the content, type ‘control-x’ and type ‘y’ for yes to save.
Now you are satisfied with your setting.
![Page 38: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/38.jpg)
Exercise#!/bin/sh
#PBS -q utm#PBS -N istas.model3d.16nodes#PBS -l nodes=1:ppn=4#PBS -M [email protected]#PBS -m abenCPU=4version=3djournal=model_journalcd $PBS_O_WORKDIR# in one line.../opt/exp_soft/share/istas/ansys_inc/v121/fluent/bin/fluent
$version -t$nCPU -g -i $journal -cnf=$PBS_NODEFILE -ssh
Queue name : utm
Job name
node x ppn = no cpu requiredemail address
email when start, error and finish
fluent version usedthe name of the journal file
![Page 39: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/39.jpg)
Exercise
• File “model_journal” file/read-case model.cas
solve/init/initialize-flow solve/iterate 400 file/binary-files n file/confirm-overwrite n file/write-data model.dat exit y
input
output
This file automates the running of Fluent commands during the execution.The content of this file depends on the type of problem you are solving.
![Page 40: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/40.jpg)
Exercise
• Now it is time to run the job, all you need to do is to run the following command,
qsub pbs-script
If nothing is wrong, your job will be placed in a queue and if there is available CPUs your job will be executed immediately.
![Page 41: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/41.jpg)
Exercise
You could monitor the progress of your job by type command ‘qstat’.
If the list is too long, type ‘qstat | less’. Type ‘q’ to exit.
Example of qstat is
![Page 42: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/42.jpg)
ce.utmgrid.utm.my: Req'd Req'd ElapJob ID Username Queue Jobname SessID NDS TSK Memory Time S Time-------------------- -------- -------- ---------------- ------ ----- --- ------ ----- - -----149111.ce.utmgri pgjsaeed utm pgjsaeed.model3d 27086 1 -- -- 5500: R 392:4149627.ce.utmgri aliff utm Drop 16602 1 -- -- 5500: R 604:3149802.ce.utmgri smahmad3 utm FJSSPHM111.sh 5690 -- -- -- 5500: R 136:0149830.ce.utmgri smahmad3 utm FJSSPHM225.sh 15284 -- -- -- 5500: R 65:20149831.ce.utmgri smahmad3 utm FJSSPHM226.sh 24896 -- -- -- 5500: R 57:29149832.ce.utmgri smahmad3 utm FJSSPHM231.sh 16951 -- -- -- 5500: R 57:21149833.ce.utmgri smahmad3 utm FJSSPHM232.sh -- -- -- -- 5500: Q --149834.ce.utmgri smahmad3 utm FJSSPHM233.sh -- -- -- -- 5500: Q --149835.ce.utmgri smahmad3 utm FJSSPHM234.sh -- -- -- -- 5500: Q --
Username
Exercise
Job number Job name Max timeallowed
Q – still waitingR - running Elapse
Time
![Page 43: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/43.jpg)
Exercise
Tips and tricks1. Use a small number of CPUs to get your job
executes earlier in the queue.2. Start from small problem.3. Always delete old unused files. Keep the
storage space free.
![Page 44: USING FLUENT FOR HPC](https://reader036.vdocuments.us/reader036/viewer/2022081503/56816384550346895dd469ec/html5/thumbnails/44.jpg)
Thank you