services Package

This module contains the ‘services’ layer in TVB application.

Most of the business logic happens here; which is agnostic of the UI.

algorithm_service

Service layer, for executing computational steps in the application. Code related to launching/duplicating operations is placed here.

class tvb.core.services.algorithm_service.AlgorithmService

Bases: builtins.object

Service Layer for Algorithms manipulation (e.g. find all Uploaders, Filter algo by category, etc)

static create_link(data_ids, project_id)

For a list of dataType IDs and a project id create all the required links.

fill_adapter_form(adapter_instance, post_data, project_id)

fill_selectfield_with_datatypes(field, project_id, extra_conditions=None)

static get_algorithm_by_identifier(ident)

Retrieve Algorithm entity by ID. Return None, if ID is not found in DB.

static get_algorithm_by_module_and_class(module, classname)

Get the db entry from the algorithm table for the given module and class.

static get_analyze_groups()

Returns:list of AlgorithmTransientGroup entities

static get_category_by_id(identifier)

Pass to DAO the retrieve of category by ID operation.

static get_generic_entity(entity_type, filter_value, select_field)

get_launchable_algorithms(datatype_gid)

Parameters:datatype_gid – Filter only algorithms compatible with this GUID Returns:dict(category_name: List AlgorithmTransientGroup)

get_launchable_algorithms_for_datatype(datatype, categories)

static get_operation_numbers(proj_id)

Count total number of operations started for current project.

static get_raw_categories()

Returns:AlgorithmCategory list of entities that have results in RAW state (Creators/Uploaders)

static get_selections_for_project(project_id, datatype_gid)

Retrieved from DB saved selections for current project. If a certain selection doesn’t have all the labels between the labels of the given connectivity than this selection will not be returned. :returns: List of ConnectivitySelection entities.

static get_stored_pse_filters(datatype_group_gid)

static get_upload_algorithms()

Returns:List of StoredAdapter entities

static get_visualisers_category()

Retrieve all Algorithm categories, with display capability

get_visualizers_for_group(dt_group_gid)

prepare_adapter(stored_adapter)

prepare_adapter_form(adapter_instance=None, form_instance=None, project_id=None, extra_conditions=None)

static remove_link(dt_id, project_id)

Remove the link from the datatype given by dt_id to project given by project_id.

static save_measure_points_selection(ui_name, selected_nodes, datatype_gid, project_id)

Store in DB a ConnectivitySelection.

static save_pse_filter(ui_name, datatype_group_gid, threshold_value, applied_on)

Store in DB a PSE filter.

authorization

class tvb.core.services.authorization.AuthorizationManager(config_file='')

Bases: builtins.object

Keycloak configuration class

static get_keycloak_instance()

class tvb.core.services.authorization.AuthorizationManagerMeta

Bases: builtins.type

Metaclass used to generate the singleton instance

backend_client_factory

class tvb.core.services.backend_client_factory.BackendClientFactory

Bases: builtins.object

static execute(operation_id, user_name_label, adapter_instance)

static stop_operation(operation_id)

burst_config_serialization

Service for serializing a Burst (Simulator) configuration.

class tvb.core.services.burst_config_serialization.SerializationManager(conf)

Bases: builtins.object

Constructs data types based on a burst configuration. Updates the burst configuration.

static group_parameter_values_by_name(model_parameters_list)

@:param model_parameters_list: Given a list of model parameters like this:

[{“a”: 2.0, ‘b’: 1.0},

{“a”: 3.0, ‘b’: 7.0}])

@:return: This method will group them by param name to get:

{‘a’: [2.0, 3.0], ‘b’: [1.0, 7.0]}

write_model_parameters(model_name, model_parameters_list)

Update model parameters in burst config.

Parameters:

• model_name – This model will be selected in burst
• model_parameters_list – A list of model parameter configurations. One for each connectivity node. Ex. [{‘a’: 1, ‘b’: 2}, ...]

write_noise_parameters(noise_dispersions)

Set noise dispersions in burst config. It will set all nsig fields it can find in the config (at least 1 per stochastic integrator). :param noise_dispersions: A list of noise dispersions. One for each connectivity node. Ex [{‘V’: 1, ‘W’:2}, ...]

burst_service

class tvb.core.services.burst_service.BurstService

Bases: builtins.object

LAUNCH_BRANCH = 'branch'

LAUNCH_NEW = 'new'

static get_available_bursts(project_id)

Return all the burst for the current project.

static get_burst_for_operation_id(operation_id, is_group=False)

static handle_range_params_at_loading(burst_config, all_range_parameters)

static load_burst_configuration(burst_config_id)

static load_burst_configuration_from_folder(simulator_folder, project)

load_simulation_from_zip(zip_file, project)

mark_burst_finished(burst_entity, burst_status=None, error_message=None, store_h5_file=True)

Mark Burst status field. Also compute ‘weight’ for current burst: no of operations inside, estimate time on disk...

Parameters:

• burst_entity – BurstConfiguration to be updated, at finish time.
• burst_status – BurstConfiguration status. By default BURST_FINISHED
• error_message – If given, set the status to error and perpetuate the message.

persist_operation_state(operation, operation_status, message=None)

Update Operation instance state. Store it in DB and on HDD/ :param operation: Operation instance :param operation_status: new status :param message: message in case of error :return: operation instance changed

static populate_burst_disk_usage(bursts)

Adds a disk_usage field to each burst object. The disk usage is computed as the sum of the datatypes generated by a burst

static prepare_burst_for_pse(burst_config)

prepare_data_for_burst_copy(burst_config_id, burst_name_format, project)

prepare_index_for_metric_result(operation, result_filename, burst)

prepare_indexes_for_simulation_results(operation, result_filenames, burst)

static prepare_metrics_operation(operation)

static prepare_simulation_name(burst, project_id)

static remove_burst_configuration(burst_config_id)

rename_burst(burst_id, new_name)

Rename the burst given by burst_id, setting it’s new name to burst_name.

static store_burst(burst_config)

static store_burst_configuration(burst_config, storage_path)

update_burst_configuration_h5(burst_configuration)

update_burst_status(burst_config)

update_history_status(id_list)

For each burst_id received in the id_list read new status from DB and return a list [id, new_status, is_group, message, running_time] tuple.

static update_simulation_fields(burst, op_simulation_id, simulation_gid)

email_sender

tvb.core.services.email_sender.send(address_from, address_to, email_subject, email_content)

Sends an Email Message

exceptions

Exceptions for services layer of the application.

exception tvb.core.services.exceptions.BurstServiceException(message, parent_exception=None)

Bases: tvb.core.services.exceptions.ServicesBaseException

Exception to be thrown in case of a problem at project import.

exception tvb.core.services.exceptions.ImportException(message, parent_exception=None)

Bases: tvb.core.services.exceptions.ServicesBaseException

Exception to be thrown in case of a problem at project import.

exception tvb.core.services.exceptions.InvalidSettingsException(message, parent_exception=None)

Bases: tvb.core.services.exceptions.ServicesBaseException

Exception to be thrown in case of a problem at project import.

exception tvb.core.services.exceptions.MissingReferenceException(message, parent_exception=None)

Bases: tvb.core.services.exceptions.ImportException

Exception to be thrown in case there are missing references when importing a H5 file.

exception tvb.core.services.exceptions.OperationException(message, parent_exception=None)

Bases: tvb.core.services.exceptions.ServicesBaseException

Exception to be thrown in case of a problem related to Launching and Executing TVB specific Operations.

exception tvb.core.services.exceptions.ProjectServiceException(message, parent_exception=None)

Bases: tvb.core.services.exceptions.ServicesBaseException

Exception to be thrown in case of a problem in the projectservice module.

exception tvb.core.services.exceptions.RemoveDataTypeException(message, parent_exception=None)

Bases: tvb.core.services.exceptions.ServicesBaseException

Exception to be thrown in case some one tries to remove an entity that is used by other entities.

exception tvb.core.services.exceptions.ServicesBaseException(message, parent_exception=None)

Bases: tvb.basic.exceptions.TVBException

Base Exception class for Services layer in the application.

exception tvb.core.services.exceptions.StructureException(message, parent_exception=None)

Bases: tvb.core.services.exceptions.ServicesBaseException

Exception to be thrown in case of a problem related to Structure Storage.

exception tvb.core.services.exceptions.UsernameException(message, parent_exception=None)

Bases: tvb.core.services.exceptions.ServicesBaseException

Exception to be thrown in case of a problem related to creating or managing a user.

figure_service

Service layer, for storing/retrieving Resulting Figures in TVB.

class tvb.core.services.figure_service.FigureService

Bases: builtins.object

Service layer for Figure entities.

edit_result_figure(figure_id, **data)

Retrieve and edit a previously stored figure.

static load_figure(figure_id)

Loads a stored figure by its id.

remove_result_figure(figure_id)

Remove figure from DB and file storage.

retrieve_result_figures(project, user, selected_session_name='all_sessions')

Retrieve from DB all the stored Displayer previews that belongs to the specified session. The previews are for current user and project; grouped by session.

store_result_figure(project, user, img_type, export_data, image_name=None)

Store into a file, Result Image and reference in DB.

hpc_operation_service

class tvb.core.services.hpc_operation_service.HPCOperationService

Bases: builtins.object

LOGGER = <Logger tvb.core.services.hpc_operation_service (INFO)>

static check_operations_job()

static handle_hpc_status_changed(operation, simulator_gid, new_status)

import_service

class tvb.core.services.import_service.ImportService

Bases: builtins.object

Service for importing TVB entities into system. It supports TVB exported H5 files as input, but it should also handle H5 files generated outside of TVB, as long as they respect the same structure.

build_operation_from_file(project, operation_file)

Create Operation entity from metadata file.

static check_import_references(file_path, datatype)

create_view_model(operation_entity, operation_data, new_op_folder, generic_attributes=None, add_params=None)

static import_operation(operation_entity, migration=False)

Store a Operation entity.

import_project_operations(project, import_path, is_group=False, importer_operation_id=None)

This method scans provided folder and identify all operations that needs to be imported

import_project_structure(uploaded, user_id)

Execute import operations:

1. check if ZIP or folder

2. find all project nodes

3. for each project node:

   • create project
   • create all operations and groups
   • import all images
   • create all dataTypes

import_simulator_configuration_zip(zip_file)

load_datatype_from_file(current_file, op_id, datatype_group=None, current_project_id=None)

Creates an instance of datatype from storage / H5 file :returns: DatatypeIndex

store_datatype(datatype, current_file=None)

This method stores data type into DB

store_or_link_datatype(datatype, dt_path, project_id)

class tvb.core.services.import_service.Operation2ImportData(operation, operation_folder, main_view_model=None, dt_paths=None, all_view_model_files=None, is_fake=False, info_from_xml=None)

Bases: builtins.object

is_old_form

order_field

kube_service

Service layer used for kubernetes calls.

class tvb.core.services.kube_service.KubeService

Bases: builtins.object

static fetch_endpoints(auth_header, target_application='thevirtualbrain')

static get_pods(application)

static notify_pods(url, target_application='thevirtualbrain')

operation_service

Module in charge with Launching an operation (creating the Operation entity as well, based on gathered parameters).

class tvb.core.services.operation_service.OperationService

Bases: builtins.object

Class responsible for preparing an operation launch. It will prepare parameters, and decide if the operation is to be executed immediately, or to be sent on the cluster.

ATT_UID = 'uid'

fire_operation(adapter_instance, current_user, project_id, visible=True, view_model=None)

Launch an operation, specified by AdapterInstance, for current_user and project with project_id.

group_operation_launch(user_id, project, algorithm_id, category_id)

Create and prepare the launch of a group of operations.

initiate_operation(current_user, project, adapter_instance, visible=True, model_view=None)

Gets the parameters of the computation from the previous inputs form, and launches a computation (on the cluster or locally).

Invoke custom method on an Adapter Instance. Make sure when the operation has finished that the correct results are stored into DB.

initiate_prelaunch(operation, adapter_instance)

Public method. This should be the common point in calling an adapter- method.

launch_operation(operation_id, send_to_cluster=False, adapter_instance=None)

Method exposed for Burst-Workflow related calls. It is used for cascading operation in the same workflow.

static load_operation(operation_id)

Retrieve previously stored Operation from DB, and load operation.burst attribute

static prepare_metadata(algo_category, burst=None, current_ga=<tvb.core.entities.generic_attributes.GenericAttributes object at 0x7f008312c750>)

Gather generic_metadata from submitted fields and current to be execute algorithm. Will populate STATE, GROUP, etc in generic_metadata

prepare_operation(user_id, project, algorithm, visible=True, view_model=None, ranges=None, burst_gid=None, op_group_id=None)

Do all the necessary preparations for storing an operation. If it’s the case of a range of values create an operation group and multiple operations for each possible instance from the range.

static stop_operation(operation_id, is_group=False, remove_after_stop=False)

Stop (also named Cancel) the operation given by operation_id, and potentially also remove it after (with all linked data). In case the Operation has a linked Burst, remove that too. :param operation_id: ID for Operation (or OperationGroup) to be canceled/removed :param is_group: When true stop all the operations from that group. :param remove_after_stop: if True, also remove the operation(s) after stopping :returns True if the stop step was successfully

static store_view_model(operation, project, view_model)

project_service

Service Layer for the Project entity.

class tvb.core.services.project_service.ProjectService

Bases: builtins.object

Services layer for Project entities.

static count_filtered_operations(project_id, filters=None)

Pass to DAO counters for filtered operations

find_project(project_id)

Simply retrieve Project entity from Database.

find_project_lazy_by_gid(project_gid)

Simply retrieve Project entity from Database by gid.

static get_all_operations_for_uploaders(project_id)

Returns all finished upload operations.

static get_datatype_by_id(datatype_id)

Retrieve a DataType DB reference by its id.

static get_datatype_details(datatype_gid)

Returns:an array. First entry in array is an instance of DataTypeOverlayDetails The second one contains all the possible states for the specified dataType.

static get_datatype_in_group(group)

Return all dataTypes that are the result of the same DTgroup.

static get_datatypegroup_by_gid(datatypegroup_gid)

Returns the DataTypeGroup with the specified gid.

static get_datatypegroup_by_op_group_id(operation_group_id)

Returns the DataTypeGroup with the specified id.

static get_datatypes_from_datatype_group(datatype_group_id)

Retrieve all dataType which are part from the given dataType group.

static get_datatypes_in_project(project_id, only_visible=False)

static get_filterable_meta()

Contains all the attributes by which the user can structure the tree of DataTypes

static get_linkable_projects_for_user(user_id, data_id)

Find projects with are visible for current user, and in which current datatype hasn’t been linked yet.

get_linked_datatypes_storage_path(project)

Returns:the file paths to the datatypes that are linked in project

get_operation_details(operation_gid, is_group)

Returns:an entity OperationOverlayDetails filled with all information for current operation details.

static get_operation_group_by_gid(operation_group_gid)

Loads OperationGroup from DB

static get_operation_group_by_id(operation_group_id)

Loads OperationGroup from DB

static get_operations_in_group(operation_group)

Return all the operations from an operation group.

get_project_structure(project, visibility_filter, first_level, second_level, filter_value)

Find all DataTypes (including the linked ones and the groups) relevant for the current project. In case of a problem, will return an empty list.

static get_results_for_operation(operation_id, selected_filter=None)

Retrieve the DataTypes entities resulted after the execution of the given operation.

static is_datatype_group(datatype_gid)

Used to check if the dataType with the specified GID is a DataTypeGroup.

static is_upload_operation(operation_gid)

Returns True only if the operation with the given GID is an upload operation.

static load_operation_by_gid(operation_gid)

Retrieve loaded Operation from DB

static load_operation_lazy_by_gid(operation_gid)

Retrieve lazy Operation from DB

remove_datatype(project_id, datatype_gid, skip_validation=False)

Method used for removing a dataType. If the given dataType is a DatatypeGroup or a dataType from a DataTypeGroup than this method will remove the entire group. The operation(s) used for creating the dataType(s) will also be removed.

remove_operation(operation_id)

Remove a given operation

remove_project(project_id)

Remove Project from DB and File Storage.

static retrieve_all_user_projects(user_id, page_start=0, page_size=20)

Return a list with all projects visible for current user, without pagination.

retrieve_project_full(project_id, applied_filters=None, current_page=1)

Return a Tuple with Project entity and Operations for current Project. :param project_id: Current Project Identifier :param applied_filters: Filters to apply on Operations :param current_page: Number for current page in operations

retrieve_projects_for_user(user_id, current_page=1)

Return a list with all Projects visible for current user.

static set_datatype_visibility(datatype_gid, is_visible)

Sets the dataType visibility. If the given dataType is a dataType group or it is part of a dataType group than this method will set the visibility for each dataType from this group.

set_operation_and_group_visibility(entity_gid, is_visible, is_operation_group=False)

Sets the operation visibility.

If ‘is_operation_group’ is True than this method will change the visibility for all the operation from the OperationGroup with the GID field equal to ‘entity_gid’.

store_project(current_user, is_create, selected_id, **data)

We want to create/update a project entity.

update_metadata(submit_data)

Update DataType/ DataTypeGroup metadata THROW StructureException when input data is invalid.

tvb.core.services.project_service.initialize_storage()

Create Projects storage root folder in case it does not exist.

settings_service

Service layer for saving/editing TVB settings.

class tvb.core.services.settings_service.SettingsService

Bases: builtins.object

Handle all TVB Setting related problems, at the service level.

KEYS_DISPLAY_ORDER = ['ADMINISTRATOR_DISPLAY_NAME', 'ADMINISTRATOR_NAME', 'ADMINISTRATOR_PASSWORD', 'ADMINISTRATOR_EMAIL', None, 'KEYCLOAK_CONFIGURATION', 'ENABLE_KEYCLOAK_LOGIN', 'KEYCLOAK_WEB_CONFIGURATION', None, 'TVB_STORAGE', 'USR_DISK_SPACE', 'MATLAB_EXECUTABLE', 'SELECTED_DB', 'URL_VALUE', None, 'WEB_SERVER_PORT', None, 'DEPLOY_CLUSTER', 'CLUSTER_SCHEDULER', 'MAXIMUM_NR_OF_THREADS', 'MAXIMUM_NR_OF_OPS_IN_RANGE', 'MAXIMUM_NR_OF_VERTICES_ON_SURFACE']

KEY_ADMIN_DISPLAY_NAME = 'ADMINISTRATOR_DISPLAY_NAME'

KEY_ADMIN_EMAIL = 'ADMINISTRATOR_EMAIL'

KEY_ADMIN_NAME = 'ADMINISTRATOR_NAME'

KEY_ADMIN_PWD = 'ADMINISTRATOR_PASSWORD'

KEY_CLUSTER = 'DEPLOY_CLUSTER'

KEY_CLUSTER_SCHEDULER = 'CLUSTER_SCHEDULER'

KEY_DB_URL = 'URL_VALUE'

KEY_ENABLE_KC_LOGIN = 'ENABLE_KEYCLOAK_LOGIN'

KEY_KC_CONFIG = 'KEYCLOAK_CONFIGURATION'

KEY_KC_WEB_CONFIG = 'KEYCLOAK_WEB_CONFIGURATION'

KEY_MATLAB_EXECUTABLE = 'MATLAB_EXECUTABLE'

KEY_MAX_DISK_SPACE_USR = 'USR_DISK_SPACE'

KEY_MAX_NR_SURFACE_VERTEX = 'MAXIMUM_NR_OF_VERTICES_ON_SURFACE'

KEY_MAX_NR_THREADS = 'MAXIMUM_NR_OF_THREADS'

KEY_MAX_RANGE = 'MAXIMUM_NR_OF_OPS_IN_RANGE'

KEY_PORT = 'WEB_SERVER_PORT'

KEY_SELECTED_DB = 'SELECTED_DB'

KEY_STORAGE = 'TVB_STORAGE'

check_db_url(url)

Validate DB URL, that a connection can be done.

static get_disk_free_space(storage_path)

Returns:the available HDD space in KB in TVB_STORAGE folder.

save_settings(**data)

Check if new settings are correct. Make necessary changes, then save new data in configuration file.

Returns:two boolean values -there were any changes to the configuration; -a reset should be performed on the TVB relaunch.

simulator_service

class tvb.core.services.simulator_service.SimulatorService

Bases: builtins.object

async_launch_and_prepare_pse(burst_config, user, project, simulator_algo, range_param1, range_param2, session_stored_simulator)

async_launch_and_prepare_simulation(burst_config, user, project, simulator_algo, simulator)

async_launch_simulation_on_server(operation, zip_folder_path)

static compute_conn_branch_conditions(is_branch, simulator)

get_simulation_state_index(burst_config, simulation_history_class)

prepare_simulation_on_server(user_id, project, algorithm, zip_folder_path, simulator_file)

reset_at_connectivity_change(is_simulator_copy, form, session_stored_simulator)

In case the user copies a simulation and changes the Connectivity, we want to reset the Model and Noise parameters because they might not fit to the new Connectivity’s nr of regions.

reset_at_surface_change(is_simulator_copy, form, session_stored_simulator)

In case the user copies a surface-simulation and changes the Surface, we want to reset the Model parameters because they might not fit to the new Surface’s nr of vertices.

static validate_first_fragment(form, project_id, conn_idx)

texture_to_json

Converts a color scheme texture image to json arrays

tvb.core.services.texture_to_json.color_texture_to_list(img_pth, img_width, band_height)

Parameters:

• img_pth – Path to the texure
• img_width – Texture width
• band_height – Height of a color scheme band

Returns:

A list of img_width/band_height color schemes. A scheme is a list of img_width colors

user_service

Service layer for USER entities.

class tvb.core.services.user_service.UserService

Bases: builtins.object

CRUD methods for USER entities are here.

USER_ROLES = ['ADMINISTRATOR', 'CLINICIAN', 'RESEARCHER']

static check_login(username, password)

Service layer to check if given UserName and Password are according to DB.

static compute_user_generated_disk_size(user_id)

create_user(username=None, display_name=None, password=None, password2=None, role=None, email=None, comment=None, email_msg=None, validated=False, skip_import=False, gid=None, skip_sending_email=False)

Service Layer for creating a new user.

delete_user(user_id)

Delete a user with a given ID. Return True when successfully, or False.

edit_user(edited_user, old_password=None)

Retrieve a user by and id, then modify it’s role and validate status.

static get_administrators()

Retrieve system administrators. Will be used for sending emails, for example.

get_external_db_user(user_data)

static get_user_by_gid(gid)

Retrieves a user by its gid.

static get_user_by_id(user_id)

Retrieves a user by its id.

static get_user_by_name(username)

Retrieves a user by its username.

get_users_for_project(user_name, project_id, page=1)

Return tuple: (All Users except the project administrator, Project Members). Parameter “user_name” is the current user. Parameter “user_name” is used for new projects (project_id is None). When “project_id” not None, parameter “user_name” is ignored.

static is_username_valid(name)

Service layer for checking if a given UserName is unique or not.

reset_password(**data)

Service Layer for resetting a password.

static retrieve_users_except(usernames, current_page, page_size)

Return all users from the database except the given users

static save_project_to_user(user_id, project_id)

Mark for current user that the given project is the last one selected.

validate_user(name='', user_id=None)

Service layer for editing a user and validating the account.

Subpackages

• backend_clients Package
  • backend_client
  • cluster_scheduler_client
  • hpc_scheduler_client
  • standalone_client