Description: Newer HPCs are much more than higher GHz, have more cores, and more memory. The HPCMP now deploys a diverse set of HPCs. Selecting the best one for a given application requires knowing what is available and how they compare. This training collects this information and guides users in selecting an HPC that fits their applications based on HPC hardware and software.

Presenter(s): Dr. Robert Donnelly, GDIT / PET
Location: Webcast
Date & Time: March 15, 2022, 2:00p - 3:30p ET

Distribution Statement D. Distribution authorized to the Department of Defense and U.S. DoD contractors only, Administrative or Operational Use, 15 March 2022.  Other requests for this document shall be referred to the High Performance Computing Modernization Office, 3909 Halls Ferry Road, Vicksburg, MS 39180.

Description: To run jobs on an HPC, you use the batch or queue system. PBS is used on most HPCMP HPCs. To use an HPC effectively, you need to use PBS effectively. Doing so involves knowing the basics but also how to specify the hardware you need. There are also different types of queues, including special ones for interactive use and copying files. PBS features like job dependencies and job arrays help you implement a multi-step workflow and execute a large set of tasks. After submitting jobs, there are various tools to track and further control jobs. You can craft job parameters to maximize throughput and reduce queue wait time. If you have urgent compute needs, you should know the process to request time in priority queues and/or use the reservation system. This training provides the skills to not just use HPCs but to optimize your HPC use.

Presenter(s): Dr. Robert Donnelly, GDIT/PET
Location: Webcast
Date & Time: August 31, 2021, 2:00p - 3:00p ET

Distribution Statement D. Distribution authorized to the Department of Defense and U.S. DoD contractors only, Administrative or Operational Use, 31 August 2021.  Other requests for this document shall be referred to the High Performance Computing Modernization Office, 3909 Halls Ferry Road, Vicksburg, MS 39180.

Description: Building on the HPCMP Storage and File Use (Mar 2021) seminar, this training discusses more advanced topics such as automating file transfers and improving performance of file copies and access.

File transfer topics include: using data transfer jobs before/after calculations; using rsync to keep files updated across various storage locations; and validating transfers via checksums.

Optimization methods include: speeding file copies via scampi and other tools, improving performance by using RAM disks, and optimizing the Lustre file system for a given application by adjusting striping.

UNIX file permissions to enable sharing is revisited, including setting defaults via umask and setting permissions used in a PBS job. Additional topics include using flock to lock files to allow updates from multiple processes or nodes and using split when necessary to ease file transfer.

Presenter(s): Dr. Robert Donnelly, GDIT / PET
Location: Webcast
Date & Time: May 18, 2021, 2:00p - 3:30p ET

Distribution Statement D. Distribution authorized to the Department of Defense and U.S. DoD contractors only, Administrative or Operational Use, 20 May 2021.  Other requests for this document shall be referred to the High Performance Computing Modernization Office, 3909 Halls Ferry Road, Vicksburg, MS 39180.

Description: Convey an understanding of HPC storage areas including proper application and retention policies, and also Inform users of UNIX file methods in general and HPC use in particular.

Knowledge gained will provide the ability to improve performance, reduce load on HPC file systems, and prevent loss of data from retention policies. Home, temporary, archive, and center-wide storage have distinct purposes and policies, and users should understand these and how to effectively use them. This includes methods of uploading, downloading, and copying files. Further, knowledge of UNIX user and group permissions, archive methods, and soft links enable sharing access to files and efficient extraction and use.

Presenter(s): Dr. Robert Donnelly, GDIT / PET
Location: Webcast
Date & Time: March 25, 2021, 2:00p - 3:30p ET

Distribution Statement D. Distribution authorized to the Department of Defense and U.S. DoD contractors only, Administrative or Operational Use, 26 March 2021.  Other requests for this document shall be referred to the High Performance Computing Modernization Office, 3909 Halls Ferry Road, Vicksburg, MS 39180.

Distribution Statement D. Distribution authorized to the Department of Defense and U.S. DoD contractors only, Administrative or Operational Use, 10 November 2020.  Other requests for this document shall be referred to the High Performance Computing Modernization Office, 3909 Halls Ferry Road, Vicksburg, MS 39180.