Package ComboCode :: Package cc :: Package managers :: Module Vic :: Class Vic
[hide private]
[frames] | no frames]

Class Vic

source code

Creating a vic manager which communicates with the Vic3 supercomputer and updates modeling results on the home disk.

Instance Methods [hide private]
 
__init__(self, account, path='code23-01-2010', credits_acc=None, time_per_sphinx=30, recover_sphinxfiles=0)
Initializing a Vic instance.
source code
 
setSphinxDb(self, sph_db)
Set the Sphinx db for this Vic instance.
source code
 
updateLineSpec(self)
Update telescope.spec files on VIC3.
source code
 
addModel(self, model_id, command_list)
Add model to the list of to be processed models.
source code
 
addTransInProgress(self, trans)
Add a transition to the list of transitions in progress.
source code
 
addTrans(self, trans)
Add a transition to the vic transition list.
source code
 
queueModel(self)
Queue the current model on VIC3, which will cp mline and cooling output file to VIC, run the shell script there, and create the necessary input files.
source code
 
reset(self)
If a model has been added, and no transitions were required to be calculated, remove that model entry here.
source code
list[string]
makeJobFile(self)
Make the job file that will run the loop on VIC3 and copy the cooling and mline output to VIC3.
source code
 
makeInputFiles(self)
Make the input files with just one line request in each.
source code
 
finalizeVic(self)
Finalize a modeling procedure on VIC: successful and failed results are printed to a file, including the transitions.
source code
bool
checkProgress(self, wait_qstat=0)
Checks progress on all queued model_ids by checking if the output has been copied to the local disk from VIC3.
source code
list[(int,string)]
getQueue(self)
Get a list of unique queue number + cooling model_id for those models that are still in progress on VIC.
source code
Method Details [hide private]

__init__(self, account, path='code23-01-2010', credits_acc=None, time_per_sphinx=30, recover_sphinxfiles=0)
(Constructor)

source code 

Initializing a Vic instance.

Parameters:
  • account (string) - the name of the user account at VIC

    (default: 'vsc30226')

  • path (string) - The output folder in the GASTRoNOoM home folder

    (default: 'codeSep2010')

  • credits_acc (string) - the name of the credits account to be charged. If one, the standard user account is charged.

    (default: None)

  • time_per_sphinx (int) - the expected calculation time for one sphinx model in minutes

    (default: 30)

  • recover_sphinxfiles (bool) - Try to recover sphinx files from the Vic disk in case they were correctly calculated, but not saved to the database for one reason or another.

    (default: 0)

setSphinxDb(self, sph_db)

source code 

Set the Sphinx db for this Vic instance.

Parameters:
  • sph_db (Database()) - The sphinx database

addModel(self, model_id, command_list)

source code 

Add model to the list of to be processed models.

Here, dictionaries are initiated to contain information about the modeling session.

Every entry in the dictionaries have an index number associated with them to uniquely identify a modeling session across the vic manager.

Dictionaries are kept for model ids, parameter sets, transitions, failed calculations and finished calculations, and vic inputfiles for every transition.

The current_model index is the same between a call to the addModel method and the queueModel() OR the reset methods. Anything between uses the same index. queueModel() will move to the next index value, while reset() will reset the current index number.

Parameters:
  • model_id (string) - The cooling model_id
  • command_list (dict()) - The parameters for this GASTRoNOoM model

addTransInProgress(self, trans)

source code 

Add a transition to the list of transitions in progress. They will be checked at the end of the VIC run to see if they have been correctly calculated.

This concerns transitions that are requested, but are already present in the sphinx database with an "IN_PROGRESS" keyword included in the transition dictionary. These will not be calculated again, instead they are remembered and checked at the end of the full modeling run.

Parameters:
  • trans (Transition()) - The transition calculating on Vic3

addTrans(self, trans)

source code 

Add a transition to the vic transition list.

These will be calculated on Vic3 and inputfiles will be prepared for them.

The entries in the transitions dictionary are deleted when the entry has been completely finished.

Parameters:
  • trans (Transition()) - The transition to be calculated on Vic3

queueModel(self)

source code 

Queue the current model on VIC3, which will cp mline and cooling output file to VIC, run the shell script there, and create the necessary input files.

Once everything has been started up, the current model index number is increased by one to allow for a new model to be added. You cannot add a model and then add another model unless you reset or queue the previous model.

reset(self)

source code 

If a model has been added, and no transitions were required to be calculated, remove that model entry here.

This only removes the self.models and self.transitions entries. The other dictionaries are re-initiated anyway when adding a model.

makeJobFile(self)

source code 

Make the job file that will run the loop on VIC3 and copy the cooling and mline output to VIC3.

Returns: list[string]
to be printed strings once all the copying is done, which shows how many transitions are being calculated for which sphinx model id

makeInputFiles(self)

source code 

Make the input files with just one line request in each.

These inputfiles are then converted to format appropriate for Vic3 and subsequently copied to Vic3.

finalizeVic(self)

source code 

Finalize a modeling procedure on VIC: successful and failed results are printed to a file, including the transitions.

This log file can be used as input for ComboCode again by putting LINE_LISTS=2.

checkProgress(self, wait_qstat=0)

source code 

Checks progress on all queued model_ids by checking if the output has been copied to the local disk from VIC3.

In this method, the self.failed and self.finished keywords are updated, and the self.inputfiles and self.transitions is cleaned up on the go.

Parameters:
  • wait_qstat (bool) - wait 10 seconds before checking the qstat query on vic, in order to make sure that the qeueu command on vic is finished queueing up the new models, as this may be slower than the python script. Use this for progress check after just queueing a new model

    (default: 0)

Returns: bool
Have all models been finished on Vic3?

getQueue(self)

source code 

Get a list of unique queue number + cooling model_id for those models that are still in progress on VIC.

Returns: list[(int,string)]
The queue numbers and model_ids still in progress.