...
To do so you first need to configure the O2 cluster profile in the MATLAB version being used which is done running the command configCluster
NOTE:
It is strongly recommended to use MATLAB version 2019a or later when submitting multi-node jobs (mpi partition) with MATLAB O2 cluster profile. Earlier versions of MATLAB are using a mechanism to start the MATLAB workers that is not fully compatible with our existing SLURM epilog and could cause jobs to be killed.
Setting up the O2 MATLAB Cluster Profile
Code Block |
---|
>> configCluster
Must set |
...
QueueName and |
...
WallTime before submitting jobs to O2. E.g. >> c = parcluster; >> c.AdditionalProperties.QueueName = 'queue-name'; >> % 5 |
...
hours >> c.AdditionalProperties.WallTime = '05:00:00'; >> c. |
...
saveProfile
Complete. Default cluster profile set to "o2 R2023a". |
now your default cluster profile is set to o2 local R2019a R2023a and you should be able to verify it by running the command parclustercommand parcluster
Code Block |
---|
>> parcluster ans = Generic Cluster Properties: Profile: o2 R2019aR2023a Modified: false Host: compute-a-16-22161 NumWorkers: 100000 NumThreads: 1 JobStorageLocation: /home/abc123/MdcsDataLocationabc/.matlab/3p_cluster_jobs/o2/R2019aR2023a/shared ClusterMatlabRoot: /n/app/matlab/2019a2023a-v2 OperatingSystem: unix RequiresOnlineLicensing: false PreferredPoolNumWorkers: 32 IntegrationScriptsLocationPluginScriptsLocation: /n/app/matlab/2019a/toolbox/localsupport-packages/matlab-parallel-server/scripts/IntegrationScripts/o2 AdditionalProperties: List properties Associated Jobs: Number Pending: 0 Number Queued: 0 Number Running: 0 Number Finished: 0 >> |
Note 1: The configCluster command needs to be executed only on time
Note 2: After running the configCluster command, the default cluster profile is set to the O2 cluster, ; if you want to go back and use the "local" cluster profile, you can change the default profile using the command parallelcommand parallel.defaultClusterProfile('local')
Note 3: Running the configCluster command sets the cluster profile only for the currently used MATLAB version. If later on you use a different version of MATLAB you will need to run configCluster again Note 4: O2 MATLAB cluster profile is not compatible with Orchestra profile. If you plan to run on both clusters it is recommended to use a different version of MATLAB in each cluster (for example 2016b in Orchestra and 2017a in O2)again.
Setting the submission parameter for the O2 MATLAB cluster profile
...
Expand | ||||
---|---|---|---|---|
| ||||
In order to use the O2 MATLAB cluster profile it is required to define at least two submission parameters: the partition to be used and the desired wall-time. In MATLAB 2016b this can be done with the command ClusterInfo.set+Property for example:
Note: In the above example the partition "mpi" is used to set the parameter ClusterInfo.setQueueName, however the MATLAB O2 Cluster Profile can be used with any of the partitions available on the O2 cluster. Several other parameter can be defined in a similar way, this below is the complete list available:
The command ClusterInfo.setUserDefinedOptions can be used to pass additional flag to the scheduler. For example ClusterInfo.setUserDefinedOptions('-o output.log') will pass the flag -o output.log to the scheduler when submitting a job from within MATLAB. Similarly Similarly the command ClusterInfo.get+Property can be used to check the assigned Property Note that, once assigned, each property will be saved in the user ~/.matlab profile folder and will not need to be re-defined unless a change is desired (i.e. different wall-time, partition, amount of memory, etc.) |
Define job submission flags for Version ≥ R2017a
In order to use the O2 MATLAB cluster profile it is required to define at least two submission parameters: the partition to be used and the desired wall-time. This can be done assigning done assigning the properties directly to a parcluster object c as shown in the example below:
Code Block |
---|
>> c=parcluster; % Specify the walltime (e.g. 48 hours) >> c.AdditionalProperties.WallTime = '48:00:00'; % Specify a partition to use for MATLAB jobs >> c.AdditionalProperties.QueueName = 'partition-name'; % Optional flags % Specify memory to use for MATLAB jobs, per node (MB) >> c.AdditionalProperties.Mem = '4000'; % Specify memory to use for MATLAB jobs, per CPU core (MB) >> c.AdditionalProperties.MemUsageMemPerCPU = '40002000'; % Specify the GPU card to run on >> c.AdditionalProperties.GpuCard = 'gpu-card-to-use'; % Request 2 GPUs per node >> c.AdditionalProperties.GpusPerNode = 2; % addAdd directly any sbatch supported flag manually (for example, mem per node and Num tasks per node):. % The "AdditionalSubmitArgs" field can be used for any Slurm flag except the walltime and partition. % This is the method we recommend. >> c.AdditionalProperties.AdditionalSubmitArgs = '--mem=4000 --tasks-per-node=2' % Save changes after modifying AdditionalProperties for the above changes to persist between MATLAB sessions >> c.saveProfile |
Note that set parameters by default will not be retained by default and will need to must be re-entered if the c object is deleted. To save permanently the submission parameter you must execute the command c.saveProfile
Important: Use --mem-per-cpu (or the flag c.AdditionalProperties.
MemUsageMemPerCPU
) instead of --mem to request a custom amount of memory when using the mpi partitionthe mpi partition. The slurm flag --mem is used to request a given amount of memory per node, so , unless you are enforcing a balanced distribution of tasks (i.e. MATLAB workers) per node, you might end up with too much or not enough memory on a given node, depending on how the tasks are allocated.
...